Much of the AI ecosystem runs on open-source software. Even teams using commercial APIs rely on open libraries for data processing, evaluation and serving. These ten projects are widely used foundations — worth knowing whether you build models, applications or infrastructure.

1. PyTorch

The deep learning framework behind most modern AI research and many production systems. Originally developed by Meta and now governed by the PyTorch Foundation under the Linux Foundation, it provides tensors, automatic differentiation and GPU acceleration. Use it for: training and experimenting with neural networks.

2. Hugging Face Transformers

A Python library offering a common interface to thousands of pre-trained models for text, vision and audio, together with the Hugging Face Hub where models and datasets are shared. Use it for: loading, fine-tuning and running open models with a few lines of code.

3. llama.cpp

A C/C++ implementation for running language models efficiently on ordinary hardware, including CPUs and Apple Silicon, created by Georgi Gerganov. It introduced the GGUF format and powers many local AI tools. Use it for: fast local inference with quantized models.

4. Ollama

A user-friendly layer for downloading and running open models locally with simple commands and a local API. Use it for: local development and private assistants. See our Ollama guide.

5. vLLM

A high-throughput serving engine for language models, introduced with the PagedAttention technique by researchers at UC Berkeley. Use it for: serving models to many users efficiently on GPUs. See our inference serving guide.

6. LangChain and LangGraph

Frameworks for composing LLM applications — prompts, retrieval, tools and agents — and, with LangGraph, building stateful agent workflows as graphs. Use them for: prototyping and orchestrating multi-step applications.

7. LlamaIndex

A framework focused on connecting LLMs to data: ingestion, indexing and retrieval for RAG applications, plus agent tooling. Use it for: building assistants over documents and databases.

8. ComfyUI

A node-based interface and engine for image, video and audio generation workflows with open models such as Stable Diffusion and FLUX. Use it for: reproducible, highly controllable generative media pipelines.

9. whisper.cpp

A C/C++ port of OpenAI’s Whisper speech recognition model by the creator of llama.cpp, optimized for CPUs and edge devices. Use it for: fast, offline transcription in applications.

10. Open WebUI

A self-hosted web interface for chatting with local and remote models, with users, document chat and tool integrations. Use it for: giving a team a private chat interface on top of Ollama or other model servers.

How to pick

You rarely need all ten. A typical local stack is Ollama plus Open WebUI; a typical serving stack is vLLM behind an API gateway; a typical research stack is PyTorch plus Transformers. Evaluate projects before depending on them — see how to evaluate an open-source AI project.

Common mistakes to avoid

  • Adopting a framework for a simple task. A direct API call can be clearer than a heavy orchestration framework for small applications.
  • Pinning nothing. Fast-moving projects change APIs often; pin versions and upgrade deliberately.
  • Ignoring licenses. Most of these projects use permissive licenses, but models and plugins may not.
  • Skipping security updates. Self-hosted AI tools are software like any other and need patching.

How to start exploring

Pick one project from each layer — a framework, an inference engine and an application toolkit — and build a small end-to-end prototype. Seeing how the pieces connect teaches more than reading documentation for each in isolation.

Related: how to contribute.

Key takeaways

  • PyTorch and Hugging Face Transformers underpin most model development.
  • llama.cpp and Ollama make local inference easy; vLLM serves many users efficiently.
  • LangChain, LangGraph and LlamaIndex orchestrate LLM applications and retrieval.
  • ComfyUI, whisper.cpp and Open WebUI cover media generation, speech and chat interfaces.
  • Choose the smallest stack that solves your problem and pin versions.

Frequently asked questions

Are these projects free for commercial use?

Most use permissive licenses such as MIT or Apache 2.0, but always check the specific license and the licenses of models you run with them.

Which should a beginner learn first?

For application developers, start with Ollama and a provider API; for machine learning, start with PyTorch and Hugging Face Transformers.

How do I keep up with new projects?

Follow GitHub trending, project release notes and our Open-source Projects of the Week coverage.