Ollama

Get up and running with large language models locally.

macOS

Download

Windows

Coming soon!

Linux & WSL2

curl https://ollama.ai/install.sh | sh

Manual install instructions

Docker

The official Ollama Docker image ollama/ollama is available on Docker Hub.

Quickstart

To run and chat with Llama 2:

ollama run llama2

Model library

Ollama supports a list of open-source models available on ollama.ai/library

Here are some example open-source models that can be downloaded:

Model

Parameters

Size

Download

Mistral

7B

4.1GB

ollama run mistral

Llama 2

7B

3.8GB

ollama run llama2

Code Llama

7B

3.8GB

ollama run codellama

Llama 2 Uncensored

7B

3.8GB

ollama run llama2-uncensored

Llama 2 13B

13B

7.3GB

ollama run llama2:13b

Llama 2 70B

70B

39GB

ollama run llama2:70b

Orca Mini

3B

1.9GB

ollama run orca-mini

Vicuna

7B

3.8GB

ollama run vicuna

Note: You should have at least 8 GB of RAM to run the 3B models, 16 GB to run the 7B models, and 32 GB to run the 13B models.

Customize your own model

Import from GGUF

Ollama supports importing GGUF models in the Modelfile:

Create a file named Modelfile , with a FROM instruction with the local filepath to the model you want to import.

FROM ./vicuna-33b.Q4_0.gguf

Create the model in Ollama

ollama create example -f Modelfile

Run the model

ollama run example