Exploring of running AI model locally
@amitmund
July 28, 2026
This is a Draft version.
What is Ollama?
Ollama is an open-source platform that let us to download and run large language models (LLMs) locally on our own machines.
It acts as a unified runtime and API that simplifies the deployment of models (LLMs) like:
Llama 3,
Mistral, and
Gemma.
And its designed to remove the need for complex setup.
example:
- GPU driver configuration, or cloud-based subscriptions.
Few alternatives
| Tool | Interface | Best Use Case | Open Source? |
|---|---|---|---|
| Ollama | CLI / API | Developers, Automation | Yes |
| LM Studio | Desktop GUI | Beginners, Model Discovery | No |
| Jan | Desktop GUI | Privacy, Open Source Desktop | Yes |
| llama.cpp | CLI | Max Performance, Control | Yes |
| vLLM | API / Server | High-Throughput Production | Yes |
| LocalAI | API / Docker | Multi-modal (Image/Audio) | Yes |
What is LLMs
LLMs (Large Language Models) are pre-trained AI models designed to understand and generate human-like text based on vast datasets. Ollama is an open-source tool that allows these models to run locally on your own hardware.
A similar example:
git(think of LLM) vs
github(Think about Ollama but running for LLM, so that you can access.)
Some related youtube video:
https://www.youtube.com/watch?v=WEeoiPavAzM
https://www.youtube.com/watch?v=Jq3jcvPe15o