Part8 - Home Assistant + llama.cpp + Home-3B-v3 : Fully Local LLM Voice

 

Step 1: Download a quantized model

# 1. Home Llama 3.2 3B (Latest Stable - Best Quality & Multilingual)
wget -O Home-Llama-3.2-3B-Q4_K_M.gguf \
  "https://huggingface.co/acon96/Home-Llama-3.2-3B/resolve/main/Home-Llama-3.2-3B.q4_k_m.gguf"

# 2. Home FunctionGemma 270M (Ultra-fast / Low RAM - ideal for sub-second responses)
wget -O Home-FunctionGemma-270m-Q4_0.gguf \
  "https://huggingface.co/acon96/Home-FunctionGemma-270m/resolve/main/Home-FunctionGemma-270m.q4_0.gguf"

# 3. Home 3B v3 (StableLM-Zephyr base)
wget -O Home-3B-v3-Q4_K_M.gguf \
  "https://huggingface.co/acon96/Home-3B-v3-GGUF/resolve/main/Home-3B-v3.q4_k_m.gguf"

 

Step 2: Setup with llama.cpp

docker run -it --rm \
  --name homellm \
  -v /datadocker/home-llm/models:/models \
  -p 8092:8080 \
  ghcr.io/ggml-org/llama.cpp:server-b10499  \
  -m /models/Home-3B-v3-Q4_K_M.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  --threads 4 \
  --jinja

vist test:

http://192.168.2.125:8092/

 

Step 3: Connect  Llama.cpp server to Home Assistant

Connect Home Assistant to llama.cpp server  using a llama.cpp integration.

add llama.cpp integration

URL:http://192.168.2.125:8092/v1