10 Ways To Run LLMs Locally And Which One Works Best For You

10 Ways To Run LLMs Locally And Which One Works Best For You

Moyi matilabs.ai

RelatedHighlights

What We Learned From a Year of Building With LLMs

Bryan Bischof oreilly.com

Thumbnail of What We Learned From a Year of Building With LLMs

Ollama

Thumbnail of Ollama

Deep Dive into LLMs like ChatGPT

eneral-purpose models

1.1B: TinyDolphin 2.8 1.1B. Takes about ~700MB RAM and tested on my Pi 4 with 2 gigs of RAM. Hallucinates a lot, but works for basic conversation.

2.7B: Dolphin 2.6 Phi-2. Takes over ~2GB RAM and tested on my 3GB 32-bit phone via llama.cpp on Termux.

7B: Nous Hermes Mistral 7B DPO. Takes about ~4-5GB RAM depending on contex

... See more

r/LocalLLaMA - Reddit

You Can Run a Generative AI Locally on Your Computer

Lifehacker lifehacker.com