blog.matterxiaomi.com

how to run locally llama.cpp for home assistant on rpi5

2026-04-25T14:22:45Z

To run llama.cpp locally for Home Assistant, you must host a llama.cpp server that provides an API， that Home Assistant can communicate with via an API.

Home Assistant does not have a "llama.cpp" brand integration by default.

connect Home Assistant to it using a compatible integration. such as https://github.com/skye-harris/hass_local_openai_llm.

Run llama-server
Connect to Home Assistant
- Integration
- Voice assistant

Device: Raspberry Pi 5 (8GB)

OS: Debian 12

Runtime: Docker

Engine: llama-server

Model: Gemma 4 E2B (GGUF, quantized)

In the ghcr.io/ggml-org/llama.cpp repository, the images are split by purpose:

Tag	Primary Contents	Best Use Case
`:light`	`llama-cli`, `llama-completion`	Testing/CLI: Best for running models in the terminal or one-off completions without overhead.
`:server`	`llama-server`	Production/API: Ideal for your Home Assistant setup. It provides the OpenAI-compatible endpoint.
`:full`	CLI, Server, and Python conversion/quantization tools.	Development: Use this if you need to convert `.safetensors` to `.gguf` or quantize a model yourself.

:light: Contains only llama-cli and llama-completion.It does not contain the API server.

:server: Contains only llama-server.It contain the API server not contain llama-cli.A lightweight, OpenAI API compatible, HTTP server for serving LLMs.

:full: Contains everything.

You should use the :server tag (or better yet, the :server-arm64 tag since you are on a Raspberry Pi 5).

Run llama-server

The llama-server executable acts as an OpenAI-compatible API that Home Assistant can use.

docker run -it --rm \
  --name llama \
  -v /datadocker/llama-cpp/models:/models \
  -p 8091:8080 \
  ghcr.io/ggml-org/llama.cpp:server \
  -m /models/google_gemma-4-E2B-it-Q4_0.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  --threads 4 \
  --jinja

Note

--entrypoint /app/llama-cli: processes your input (or waits for one), and then exits. It does not listen for network requests on a port.

--entrypoint /app/llama-server: You were using llama-cli, which is for one-off prompts in the terminal. The llama-server is required to handle API calls like curl.

--host 0.0.0.0: Inside a Docker container, the server must listen on 0.0.0.0 to accept connections from your Raspberry Pi's IP or localhost.

--port 8080: This tells the software inside the container to listen on port 8080 (which you mapped to 8091 on your host).

--jinja: support for OpenAI-style function calling.Tool calling must be enabled in inference engine.Detail

output

...
srv          init: init: chat template, thinking = 1
main: model loaded
main: server is listening on http://0.0.0.0:8080
main: starting the main loop...
srv  update_slots: all slots are idle

Now llama.cpp = local LLM → HTTP API server

Test

Once the server logs show "HTTP server listening", run your curl command. Make sure to include a JSON body, otherwise the server might reject the request:

curl http://localhost:8091/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Hello Gemma!"}]
  }'

Connect to Home Assistant

Integration - Add Integration

Custom Integration - Local OpenAI LLM Integration

https://github.com/skye-harris/hass_local_openai_llm

Custom Integration - Configure Integration

Added server URL to the initial server configuration

http://192.168.2.125:8091

Voice assistant - Create conversation agent

Add assistant

How to Run AI Models Locally with llama.cpp on rpi5

2026-04-23T17:42:13Z

Most people access generative AI tools like ChatGPT or Gemini through a web interface or API — but what if you could run them locally?

In this article, you’ll learn how to set up your own local generative AI using existing models such as llama.cpp.

The final result will look like the GIF shown below (note, it’s hosted localhost)

Prerequisites

Prerequisites

Hardware: Raspberry Pi 5 (8GB RAM highly recommended).

OS: Raspberry Pi OS (64-bit) or Ubuntu (64-bit).

Storage: At least 5GB free space (preferably on an SSD/NVMe for speed).

Device: Raspberry Pi 5 (8GB)

OS: Debian 12

Runtime: Docker

Engine: llama.cpp

Model: Gemma 4 E2B (GGUF, quantized)

Install Docker (Debian 12)

Here are the commands I used to Got Gemma 4 E2B running on a Raspberry Pi 5 8GB:

step 1. Pull llama.cpp (light)

First of all, we need an LLM Serving Engine, such as llama.cpp.

# This is the correct lightweight image for Pi (ARM)
docker pull ghcr.io/ggml-org/llama.cpp:light

step 2. Pick a model - Download Gemma (GGUF, quantized)

# llama.cpp only works with GGUF

# Create model directory
mkdir -p /datadocker/llama-cpp
cd /datadocker/llama-cpp/models


https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF/tree/main

https://huggingface.co/bartowski/google_gemma-4-E2B-it-GGUF/tree/main

google_gemma-4-E2B-it-Q4_0.gguf
https://huggingface.co/bartowski/google_gemma-4-E2B-it-GGUF/resolve/main/google_gemma-4-E2B-it-Q4_0.gguf?download=true

Note

There is no official “Gemma 4 E2B GGUF direct URL” from Google.

GGUF files are community-converted and hosted on Hugging Face.

step 3. Docker run and load model

Run llama.cpp server (Docker)

docker run -v /path/to/models:/models --entrypoint /app/llama-cli ghcr.io/ggml-org/llama.cpp:light -m /models/7B/ggml-model-q4_0.gguf

run ok

docker run -it --rm \
  -v /datadocker/llama-cpp/models:/models \
  --entrypoint /app/llama-cli \
  -p 8091:8080 \
  ghcr.io/ggml-org/llama.cpp:light \
  -m /models/gemma-4-e2b-it-Q4_K_M.gguf \

Note

1.pick the model you downloaded earlier.

output

load_backend: loaded CPU backend from /app/libggml-cpu-armv8.2_2.so

Loading model...

▄▄ ▄▄
██ ██
██ ██  ▀▀█▄ ███▄███▄  ▀▀█▄    ▄████ ████▄ ████▄
██ ██ ▄█▀██ ██ ██ ██ ▄█▀██    ██    ██ ██ ██ ██
██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀
                                    ██    ██
                                    ▀▀    ▀▀

build      : b8833-45cac7ca7
model      : google_gemma-4-E2B-it-Q4_0.gguf
modalities : text

available commands:
  /exit or Ctrl+C     stop or exit
  /regen              regenerate the last response
  /clear              clear the chat history
  /read <file>        add a text file
  /glob <pattern>     add text files using globbing pattern


>

step 4. test

> how

[Start thinking]
Thinking Process:

1.  **Analyze the Request:** The user provided a single word: "how". This is an extremely open-ended prompt. It implies a question, a request for a process, or a search for a method.
2.  **Determine the Context:** Since there is no context, I must provide a response that addresses the ambiguity while offering ways to get a useful answer.
3.  **Identify the Goal:** The goal is to prompt the user to specify what they want to know "how" to do.
4.  **Brainstorm Potential Interpretations of "how":**
    *   *How does X work?* (Mechanism, process)
    *   *How do I do Y?* (Instructions, tutorial)
    *   *How to achieve Z?* (Strategy, steps)
    *   *How are we connected?* (Relationship, system)
5.  **Formulate the Response Strategy:**
    *   Acknowledge the brevity.
    *   State that more information is needed.
    *   Provide examples of the types of questions I can answer.
    *   Invite the user to ask the specific question.
6.  **Draft the Response (Self-Correction/Refinement):** Start with a friendly, open invitation. Ensure the tone is helpful and encouraging. (The resulting response should be a clear call to action.)
[End thinking]

Please tell me what you would like to know **how** to do! 😊

I can help you with instructions, explanations, processes, recipes, coding, concepts, and much more.

**For example, you could ask:**

* "How do I bake a cake?"
* "How does photosynthesis work?"
* "How do I change the font in Microsoft Word?"
* "How do I start learning Spanish?"

**Just tell me your question!**

[ Prompt: 8.6 t/s | Generation: 5.6 t/s ]

>

access generative AI tools like llamacp through a web interface or API

Useful links

llama.cpp on GitHub with Docker image

https://github.com/ggml-org/llama.cpp/blob/master/docs/docker.md

Models download via url

https://huggingface.co/bartowski/google_gemma-4-E2B-it-GGUF/tree/main

https://huggingface.co/bartowski/google_gemma-4-E2B-it-GGUF/blob/main/google_gemma-4-E2B-it-Q4_0.gguf

https://huggingface.co/ggml-org/gemma-4-E2B-it-GGUF

How to Run AI Models Locally with Ollama

2026-04-21T20:42:51Z

In this article, you’ll learn how to set up your own local generative AI using existing models such as Gemma 4 and Meta’s LLaMA 3.

The official [Ollama Docker image](https://hub.docker.com/r/ollama/ollama) ollama/ollama is available on Docker Hub.

To run the Gemma 4 model locally using Ollama:

First of all, we need an LLM Serving Engine, such as Ollama: A framework for running large language models locally.

Pull the LLM models via Ollama

Load the LLM models via Ollama

Test the LLM models via Ollama CLI

Install Ollama with Docker
Pull and Run a Model via ollama
Test the LLM models via Ollama CLI
Configuration Checklist

Install Ollama with Docker

There are several ways to install it on your machine,we will Running ollama via Docker.

Download

docker pull ollama/ollama:0.21.0

Version control:

Google officially released the Gemma 4 family on April 2, 2026, and ollama latest version has stabilized support for its unique architectures.

## Run the Container

You need to decide if you want to use CPU-only or GPU acceleration and specified a specific version.

CPU Only

~~~

docker run -d \

  -v ollama:/root/.ollama \

  -p 11434:11434 \

  --name ollama \

  ollama/ollama:0.21.0

~~~

The container starts an API server, but it doesn't come with any LLMs pre-installed.

Pull and Run a Model via ollama

You need to "exec" into the container to pull a model (like Llama 4 ).

# into the container
docker exec -it ollama sh

# Download a model without running it
ollama pull [model-name]

# Run a Model
# A single command (ollama run gemma4:e4b) handles downloading, memory management, and API serving.
ollama run gemma4:e4b

Ollama will automatically download the model.llm server loading model.

Once the model is loaded, you can see generating output:

output

# ollama run gemma4:e2b
>>>

docker logs -f ollama

output

time=2026-04-18T09:18:54.786Z level=INFO source=server.go:1398 msg="waiting for server to become available" status="llm server loading model"
...
time=2026-04-18T09:20:29.899Z level=INFO source=server.go:1402 msg="llama runner started in 97.04 seconds"
[GIN] 2026/04/18 - 09:20:32 | 200 |         1m42s |       127.0.0.1 | POST     "/api/generate"
[GIN] 2026/04/18 - 09:22:32 | 200 | 39.184533484s |       127.0.0.1 | POST     "/api/chat"

Test the LLM models via Ollama CLI

start a chat -Test the LLM models

Verify - cli(Test in CLI)

# ollama run gemma4:e2b
>>> how
Thinking...
Thinking Process:

1.  **Analyze the Input:** The input is "how".
...

Verify - api test

curl http://localhost:11434/api/chat -d '{
  "model": "gemma3",
  "messages": [{
    "role": "user",
    "content": "Why is the sky blue?"
  }],
  "stream": false
}'

output

root@debian:~# curl http://localhost:11434/api/chat -d '{
  "model": "gemma4:e2b",
  "messages": [{
    "role": "user",
    "content": "Why is the sky blue?"
  }],
  "stream": false
}'
{"model":"gemma4:e2b","created_at":"2026-04-18T09:24:45.242321396Z","message":{"role":"assistant","content":"The reason the sky appears blue is due to a phenomenon called **Rayleigh Scattering**. It is a result of how sunlight interacts with the small molecules of the Earth's atmosphere.\n\nHere is a detailed breakdown of the process:\n\n---\n\n### 1. The Ingredients: Sunlight and Atmosphere\n\n**A. Sunlight is White Light:**\nSunlight, which appears white to us, is actu
...

Verify - Web UI(Browser test)

open-webui

https://github.com/open-webui/open-webui

# ollama list
NAME             ID              SIZE      MODIFIED          
gemma4:e2b       7fbdbf8f5e45    7.2 GB    15 minutes ago       
gemma4:e4b       c6eb396dbd59    9.6 GB    About an hour ago    
gemma3:4b        a2af6cc3eb7f    3.3 GB    2 hours ago          
llama3:latest    365c0bd3c000    4.7 GB    3 hours ago

Quick Diagnostic Steps

docker logs -f ollama

Configuration Checklist

OS: 64-bit Debian12 OS

RAM:16G

CPU:Intel Cpu 10400

Larger Models:Gemma 4 E2B (4-bit)

Useful links

Recommended Models of ollama

https://ollama.com/library

gemma4 Model information

https://ollama.com/library/gemma4:e4b

Gemma 4 Inference Memory Requirements

https://ai.google.dev/gemma/docs/core#gemma-4-inference-memory-requirements

Parameters	BF16 (16-bit)	SFP8 (8-bit)	Q4_0 (4-bit)
Gemma 4 E2B	9.6 GB	4.6 GB	3.2 GB
Gemma 4 E4B	15 GB	7.5 GB	5 GB
Gemma 4 31B	58.3 GB	30.4 GB	17.4 GB
Gemma 4 26B A4B	48 GB	25 GB	15.6 GB

model requires more system memory (9.8 GiB) than is available (4.7 GiB)

blog

https://medium.com/tech-ai-chat/running-llm-on-a-local-mac-machine-0dae23d8320b

Ollama vs LiteLLM vs llmama.cpp vs vvllm vs lm studio

2026-04-19T18:27:39Z

How to run a local LLM server step by step

Ollama vs LiteLLM vs llmama.cpp vs vvllm vs lm studio

These tools represent different layers of the AI stack. While they overlap, they generally serve distinct purposes:

Serving (Llama.cpp, vLLM),

Managing (Ollama, LM Studio),

Routing (LiteLLM).

Managing (Ollama, LM Studio)
Serving (Llama.cpp, vLLM)
Routing (LiteLLM)

Managing (Ollama, LM Studio)

Ollama

A local LLM inference/runtime platform.It handles model downloads, storage, and execution with a simple CLI/API. Think of it as a “local LLM server”.

Run AI Models locally integrate via API

LM Studio

A desktop application.

Run AI Models locally with a Chat UI

Serving (Llama.cpp, vLLM)

llama.cpp - run ai model on edge devices.

Run a model on a Raspberry Pi.

vLLM

Build a high-traffic AI startup or production API.

Routing (LiteLLM)

LiteLLM

LiteLLM is not an inference engine; it is a Proxy/Router.A proxy/gateway layer that provides a unified, OpenAI-compatible API for calling many LLM providers (cloud and local).

Matter Bridge in Home Assistant Part2 - Install MatterBridge Connect to Home assistant

2026-02-16T23:14:37Z

Matter Bridge in Home Assistant Part2 - Install MatterBridge Connect to Home assistant

Quick start
Install and configure
How to Use

Quick start

I set up the matterbridge as follows:

Install the Matterbridge docker

Create long-lived access tokens to allow home-assistant-matter-hub docker to interact with your Home Assistant instance.

Communication configure between Matterhub and Home Assistant,Matterbridge to connect to home assistant with url and token

expose homeassistant device as a matter bridge

open http://192.168.2.125:8482/ via chrome browser

Create a new bridge,

Add device "pattern: switch.air_con" in new bridge

start it to generate a pairing QR code

Connect accessory to Apple Home

Install and configure

docker-compose.yml

You need to create an access token in home assistant instance and export it like this:

services:
  matter-hub:
    image: ghcr.io/t0bst4r/home-assistant-matter-hub:3.0.1
    restart: unless-stopped
    network_mode: host
    environment: # more options can be found in the configuration section
      - HAMH_HOME_ASSISTANT_URL=http://192.168.2.125:8123/
      - HAMH_HOME_ASSISTANT_ACCESS_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJhMzcwZDExYjM4MjE0YzFmYThmZTk3NDZjMDQyODU2NSIsImlhdCI6MTc3MTA4MTk1NSwiZXhwIjoyMDg2NDQxOTU1fQ.vhZD-KhJe4XIXd6_XvBE92y4T5W1aICSfBCbTTCvFL4
      - HAMH_LOG_LEVEL=info
      - HAMH_HTTP_PORT=8482
    volumes:
      - /datadocker/home-assistant-matter-hub:/data

Now you can visit it via web ui

http://192.168.2.125:8482/

How to Use

expose homeassistant device as a matter bridge

open http://192.168.2.125:8482/

Create a new bridge for device,home-assistant-matter-hub docker get it from home assistant via api.

type:pattern

value:light.yeelink_cn_ceiling21_s_2_light

{
  "name": "matterbridgeceilling21v2",
  "port": 5543,
  "filter": {
    "include": [
      {
        "type": "pattern",
        "value": "light.yeelink_cn_476690814_ceiling21_s_2_light"
      }
    ],
    "exclude": []
  },
  "featureFlags": {
    "coverDoNotInvertPercentage": false,
    "includeHiddenEntities": false
  }
}

Ecovacs in Home Assistant Part6 - Robot Vacuum Control MCP Server

2026-04-25T14:10:22Z

Official Ecovacs Deebot MCP Server

MCP protocol

https://github.com/ecovacs-ai/ecovacs-mcp/blob/main/ecovacs_mcp/robot_mcp_stdio.py

Created:2025.04.24

Official Doc：

https://open.ecovacs.com/#/serviceOverview

way 1.custom integration

https://github.com/hoangminh1109/ecovacs_cn

way 2. mcp client integraton in ha

mcp client integration

https://www.home-assistant.io/integrations/mcp

mcp client integraton configure

The remote MCP server URL for the SSE endpoint, for example http://example/mcp

Ecovacs SSE Server URL:

https://mcp-open.ecovacs.cn/sse?ak=your ak

useful links

https://open.ecovacs.com/#/serviceOverview

ModelScope vs Hugging Face vs k2-fsa.github.io vs Kaldi vs Sherpa

2026-02-19T16:28:27Z

Hugging Face, ModelScope, and k2-fsa.github.io (specifically the k2-fsa/sherpa-onnx project) represent different approaches to the machine learning ecosystem。

Hugging Face and ModelScope host everything. k2-fsa and Sherpa only do Speech.

k2-fsa and Sherpa are highly specialized tools focused on speech recognition (ASR) and synthesis (TTS)

Sherpa (often referred to as sherpa-onnx or sherpa-ncnn) is a lightweight speech-to-text (ASR) and text-to-speech (TTS) engine. Best for: Deploying speech models on edge devices (Android, iOS, WebAssembly, ARM boards) or high-performance servers, prioritizing low latency and CPU efficiency.

Hugging Face（Global AI model platform）
ModelScope(The Alibaba/Chinese AI Industrial model platform)
specialized tools focused on speech recognition (ASR) and synthesis (TTS)
- k2-fsa (Next-Gen Kaldi)
- Sherpa(The Real-Time Speech Deployment Tool)

These four entities represent two different categories: Model Ecosystems (Hugging Face & ModelScope) and Speech Recognition Frameworks (Kaldi & k2-fsa).

Hugging Face（Global AI model platform）

The Global Industry Standard。It supports NLP, computer vision, audio, and multimodal models via transformers and diffusers libraries.

download models

https://huggingface.co/models

https://huggingface.co/FunAudioLLM/SenseVoiceSmall

https://huggingface.co/funasr/paraformer-zh/blame/7904416f6cb6290ee7dc0b2ddb2993a9fe4f421a/README.md

ModelScope(The Alibaba/Chinese AI Industrial model platform)

ModelScope is an AI model hub led by Alibaba DAMO Academy. It provides pre-trained models, pipelines, and deployment tools, especially strong in Chinese language and speech technologies.ModelScope is often described as the "Chinese Hugging Face."

https://modelscope.cn/search?search=sherpa%20onnx

Inference Framework

1.funasr

2.funasr-onnx

see:https://github.com/modelscope/FunASR?tab=readme-ov-file#sensevoice

specialized tools focused on speech recognition (ASR) and synthesis (TTS)

Kaldi: Speech Toolkit

The "grandfather" of modern speech recognition. It is a C++ based toolkit developed primarily by Dan Povey.

demo

KaldiRecognizer

https://github.com/rhasspy/wyoming-faster-whisper/blob/main/wyoming_faster_whisper/__main__.py

k2-fsa (Next-Gen Kaldi)

Speech Toolkit.The Modern Successor

What it is: Often called "Next-gen Kaldi." It is a complete rewrite of Kaldi’s core concepts to make them natively compatible with PyTorch.

Key Repositories:

Icefall: Where the actual training recipes for speech models (like Zipformer) live.

k2: The core library for differentiable FSTs.the old version of the tool you used before k2-fsa existed.

download sherpa-onnx

https://github.com/k2-fsa/sherpa-onnx/releases

download sherpa-onnx asr models

https://k2-fsa.github.io/sherpa/onnx/pretrained_models/index.html

https://github.com/k2-fsa/sherpa-onnx/releases/tag/asr-models

download Silero VAD ONNX model

# https://k2-fsa.github.io/sherpa/onnx/sense-voice/pretrained.html#sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17
wget https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/silero_vad.onnx

download url:https://k2-fsa.github.io/sherpa/onnx/vad/silero-vad.html#download-models-files

Sherpa(The Real-Time Speech Deployment Tool)

The deployment engine (CPU/GPU, Android, iOS, WebAssembly).It uses models trained in the k2-fsa ecosystem.

How they all fit together

1.k2-fsa is the tool you use to build a high-performance speech model.

silero-vad model download

https://k2-fsa.github.io/sherpa/onnx/vad/silero-vad.html#download-models-files

Once you've trained that model using k2-fsa, you might upload it to Hugging Face or ModelScope so others can download it easily.

Hugging Face hosts models from both ModelScope and k2-fsa/Sherpa, serving as a distribution point for them.

Layer Relationship

Model Hosting & Distribution
   ├── ModelScope
   └── Hugging Face

Inference / Runtime Framework
   └── k2-fsa / sherpa

Create Wyoming server for Home assistant Part2 - stt - wyoming-funasr arm64

2026-02-24T21:59:19Z

Wyoming protocol server for the funasr speech to text system.stt - wyoming-funasr arm64

FunASR: A Fundamental End-to-End Speech Recognition Toolkit.

To make an STT server work with Home Assistant, the industry standard is using the Wyoming Protocol.

Step 1.Development Environment Setup
- Create Python virtual environment
Step 2. Install
Step3.Download and test a model (example: paraformer-zh)
- SenseVoice - Speech Recognition (Non-streaming)
Step 4.FunASR + Wyoming STT full server
- Install wyoming
Strategies to reduce latency

Step 1.Development Environment Setup

Create Python virtual environment

mkdir -p /funasr-wyoming

cd /funasr-wyoming

python3 -m venv venv

source venv/bin/activate

python --version

Python 3.11.2

pip3 install wyoming==1.8.0

pip3 install funasr==1.3.0

pip3 install torch

pip3 install torchaudio

apt list --installed

(venv) root@raspberrypi:/funasr-wyoming# pip3 show funasr
Name: funasr
Version: 1.3.0
Summary: FunASR: A Fundamental End-to-End Speech Recognition Toolkit
Home-page: https://github.com/alibaba-damo-academy/FunASR.git
Author: Speech Lab of Alibaba Group
Author-email: funasr@list.alibaba-inc.com
License: The MIT License
Location: /funasr-wyoming/venv/lib/python3.11/site-packages
Requires: editdistance, hydra-core, jaconv, jamo, jieba, kaldiio, librosa, modelscope, oss2, pytorch_wpe, PyYAML, requests, scipy, sentencepiece, soundfile, tensorboardX, torch_complex, tqdm, umap_learn

Requirements

python>=3.8
torch>=1.13
torchaudio

Step 2. Install

(venv) root@raspberrypi:/funasr-wyoming# pip3 --version

pip 23.0.1 from /funasr-wyoming/venv/lib/python3.11/site-packages/pip (python 3.11)

Install torch via PyPI

pip3 install torch==2.1.0 (CPU-only)

output

Installing collected packages: mpmath, sympy, networkx, MarkupSafe, fsspec, jinja2, torch
Successfully installed MarkupSafe-3.0.3 fsspec-2026.1.0 jinja2-3.1.6 mpmath-1.3.0 networkx-3.6.1 sympy-1.14.0 torch-2.1.0

if ffmpeg is not installed. torchaudio is used to load audio

pip3 install torchaudio==2.1.0 (CPU-only)

output

Successfully installed torchaudio-2.1.0

You will need the wyoming and funasr libraries.

Install FunASR 1.3.0 via PyPI

pip3 install -U funasr==1.3.0

This will pull:

Downloading https://www.piwheels.org/simple/threadpoolctl/threadpoolctl-3.6.0-py3-none-any.whl (18 kB)

Installing collected packages: jieba, jamo, jaconv, crcmod, antlr4-python3-runtime, urllib3, typing_extensions, tqdm, threadpoolctl, six, sentencepiece, PyYAML, pycryptodome, pycparser, protobuf, platformdirs, packaging, numpy, msgpack, llvmlite, joblib, jmespath, idna, filelock, editdistance, decorator, charset_normalizer, certifi, audioread, torch_complex, tensorboardX, soxr, scipy, requests, pytorch_wpe, omegaconf, numba, lazy_loader, kaldiio, cffi, soundfile, scikit-learn, pooch, modelscope, hydra-core, cryptography, pynndescent, librosa, aliyun-python-sdk-core, umap_learn, aliyun-python-sdk-kms, oss2, funasr

output

Successfully installed PyYAML-6.0.3 aliyun-python-sdk-core-2.16.0 aliyun-python-sdk-kms-2.16.5 antlr4-python3-runtime-4.9.3 audioread-3.1.0 certifi-2026.1.4 cffi-2.0.0 charset_normalizer-3.4.4 crcmod-1.7 cryptography-46.0.3 decorator-5.2.1 editdistance-0.8.1 filelock-3.20.3 funasr-1.3.0 hydra-core-1.3.2 idna-3.11 jaconv-0.4.1 jamo-0.4.1 jieba-0.42.1 jmespath-0.10.0 joblib-1.5.3 kaldiio-2.18.1 lazy_loader-0.4 librosa-0.11.0 llvmlite-0.46.0 modelscope-1.34.0 msgpack-1.1.2 numba-0.63.1 numpy-2.3.5 omegaconf-2.3.0 oss2-2.19.1 packaging-26.0 platformdirs-4.5.1 pooch-1.8.2 protobuf-6.33.4 pycparser-3.0 pycryptodome-3.23.0 pynndescent-0.6.0 pytorch_wpe-0.0.1 requests-2.32.5 scikit-learn-1.8.0 scipy-1.17.0 sentencepiece-0.2.1 six-1.17.0 soundfile-0.13.1 soxr-1.0.0 tensorboardX-2.6.4 threadpoolctl-3.6.0 torch_complex-0.4.4 tqdm-4.67.1 typing_extensions-4.15.0 umap_learn-0.5.11 urllib3-2.6.3

detail:https://pypi.org/project/funasr

sudo apt install ffmpeg

output

ffmpeg is already the newest version (8:5.1.8-0+deb12u1+rpt1).

if ffmpeg is not installed. torchaudio is used to load audio

Verify installation

python - << 'EOF'
from funasr import AutoModel
print("FunASR imported OK")
EOF

output

FunASR imported OK

Step3.Download and test a model (example: paraformer-zh)

test.py

from funasr import AutoModel

model = AutoModel(
    model="paraformer-zh",
    model_revision="v2.0.4",
    vad_model="fsmn-vad",
    vad_model_revision="v2.0.4",
    punc_model="ct-punc",
    punc_model_revision="v2.0.4",
)

res = model.generate(input="test.wav")
print(res)

res = model.generate(input="https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/vad_example.wav")
print(res)

SenseVoice - Speech Recognition (Non-streaming)

from funasr import AutoModel
from funasr.utils.postprocess_utils import rich_transcription_postprocess

model_dir = "iic/SenseVoiceSmall"

model = AutoModel(
    model=model_dir,
    vad_model="fsmn-vad",
    vad_kwargs={"max_single_segment_time": 30000},
    device="cuda:0",
)

# en
res = model.generate(
    input=f"{model.model_path}/example/en.mp3",
    cache={},
    language="auto",  # "zn", "en", "yue", "ja", "ko", "nospeech"
    use_itn=True,
    batch_size_s=60,
    merge_vad=True,  #
    merge_length_s=15,
)
text = rich_transcription_postprocess(res[0]["text"])
print(text)

see:https://github.com/modelscope/FunASR?tab=readme-ov-file#sensevoice

python3 test.py

Models are cached in:

/root/.cache/modelscope/hub/models/iic/speech_seaco_paraformer_large_asr_nat-zh-cn-16k-common-vocab8404-pytorch

output

Downloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/iic/speech_seaco_paraformer_large_asr_nat-zh-cn-16k-common-vocab8404-pytorch


2026-01-25 08:58:28,492 - modelscope - INFO - Use user-specified model revision: v2.0.4
2026-01-25 08:58:28,595 - modelscope - INFO - Got 11 files, start to download ...
Downloading [fig/res.png]: 100%|███████████████████████████████████████████████████| 192k/192k [00:00<00:00, 386kB/s]
Downloading [am.mvn]: 100%|█████████████████████████████████████████████████████| 10.9k/10.9k [00:00<00:00, 21.7kB/s]
Downloading [example/hotword.txt]: 100%|███████████████████████████████████████████| 7.00/7.00 [00:00<00:00, 11.9B/s]
Downloading [config.yaml]: 100%|████████████████████████████████████████████████| 3.34k/3.34k [00:00<00:00, 5.66kB/s]
Downloading [configuration.json]: 100%|███████████████████████████████████████████████| 478/478 [00:00<00:00, 766B/s]
Downloading [README.md]: 100%|██████████████████████████████████████████████████| 11.3k/11.3k [00:00<00:00, 18.2kB/s]
Downloading [example/asr_example.wav]: 100%|███████████████████████████████████████| 141k/141k [00:00<00:00, 208kB/s]
Downloading [fig/seaco.png]: 100%|█████████████████████████████████████████████████| 167k/167k [00:00<00:00, 296kB/s]
Downloading [tokens.json]: 100%|█████████████████████████████████████████████████| 91.5k/91.5k [00:00<00:00, 165kB/s]
Downloading [seg_dict]: 100%|███████████████████████████████████████████████████| 7.90M/7.90M [00:03<00:00, 2.76MB/s]
Downloading [model.pt]: 100%|█████████████████████████████████████████████████████| 944M/944M [01:31<00:00, 10.8MB/s]
Processing 11 items: 100%|████████████████████████████████████████████████████████| 11.0/11.0 [01:31<00:00, 8.34s/it]
2026-01-25 09:00:00,347 - modelscope - INFO - Download model 'iic/speech_seaco_paraformer_large_asr_nat-zh-cn-16k-common-vocab8404-pytorch' successfully.█████████████████████████████████████████████| 91.5k/91.5k [00:00<00:00, 165kB/s]
WARNING:root:trust_remote_code: False                                            | 5.00M/944M [00:01<02:50, 5.77MB/s]
Downloading [model.pt]:   2%|█▎                                                  | 23.0M/944M [00:03<02:15, 7.12MB/s]
Downloading [model.pt]: 100%|████████████████████████████████████████████████████▉| 942M/944M [01:31<00:00, 6.28MB/s]
Downloading [seg_dict]: 100%|███████████████████████████████████████████████████| 7.90M/7.90M [00:03<00:00, 4.16MB/s

python3 -c "from funasr import AutoModel; AutoModel(model='paraformer-zh', device='cpu')"

Step 4.FunASR + Wyoming STT full server

Install wyoming

pip3 install wyoming==1.8.0

output

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting wyoming==1.5.0
  Downloading wyoming-1.5.0-py3-none-any.whl (23 kB)
Installing collected packages: wyoming
Successfully installed wyoming-1.5.0

A Wyoming server consists of an AsyncServer and an AsyncEventHandler. The handler processes events like Describe

python3 server.py

You will need the wyoming and funasr libraries.

Describe event

Listen for a Describe event (to tell HA it's an STT service)

AudioStart event

Ha client->

{
  "type": "audio.start",
  "rate": 16000,
  "width": 2,
  "channels": 1
}

AudioStart.is_type(event.type)

AudioChunk event

The AudioChunk event is where you collect the raw PCM data.Receive AudioChunk events and buffer them.

AudioStop event

Trigger the SenseVoice model when AudioStop event is received.AudioStop event is where you trigger the inference.

funasr-wyoming# python3 server.py
funasr version: 1.3.0.
Check update of funasr, and it would cost few times. You may disable it by set `disable_update=True` in AutoModel
New version is available: 1.3.1.
Please use the command "pip install -U funasr" to upgrade.
2026-02-19 14:15:51.321 - INFO - root - download models from model hub: ms
Downloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/iic/SenseVoiceSmall
2026-02-19 14:15:52.591 - WARNING - root - trust_remote_code: False
2026-02-19 14:15:54.679 - INFO - root - Loading pretrained params from /root/.cache/modelscope/hub/models/iic/SenseVoiceSmall/model.pt
2026-02-19 14:15:54.688 - INFO - root - ckpt: /root/.cache/modelscope/hub/models/iic/SenseVoiceSmall/model.pt
2026-02-19 14:16:10.752 - INFO - root - scope_map: ['module.', 'None']
2026-02-19 14:16:10.871 - INFO - root - excludes: None
2026-02-19 14:16:11.383 - INFO - root - Loading ckpt: /root/.cache/modelscope/hub/models/iic/SenseVoiceSmall/model.pt, status: <All keys matched successfully>
2026-02-19 14:16:11.461 - INFO - root - Building VAD model.
2026-02-19 14:16:11.461 - INFO - root - download models from model hub: ms
Downloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/iic/speech_fsmn_vad_zh-cn-16k-common-pytorch
2026-02-19 14:16:13.308 - WARNING - root - trust_remote_code: False
2026-02-19 14:16:13.652 - INFO - root - Loading pretrained params from /root/.cache/modelscope/hub/models/iic/speech_fsmn_vad_zh-cn-16k-common-pytorch/model.pt
2026-02-19 14:16:13.653 - INFO - root - ckpt: /root/.cache/modelscope/hub/models/iic/speech_fsmn_vad_zh-cn-16k-common-pytorch/model.pt
2026-02-19 14:16:13.959 - INFO - root - scope_map: ['module.', 'None']
2026-02-19 14:16:13.959 - INFO - root - excludes: None
2026-02-19 14:16:13.962 - INFO - root - Loading ckpt: /root/.cache/modelscope/hub/models/iic/speech_fsmn_vad_zh-cn-16k-common-pytorch/model.pt, status: <All keys matched successfully>
2026-02-19 14:16:14.020 - INFO - wyoming-funasr-stt-server - Wyoming STT Server started on port 10850
2026-02-19 14:16:19.596 - INFO - wyoming-funasr-stt-server - Transcribe received
2026-02-19 14:16:19.597 - INFO - wyoming-funasr-stt-server - AudioStart received
2026-02-19 14:16:22.530 - INFO - wyoming-funasr-stt-server - AudioStop received. Processing...  95040 bytes of audio...
2026-02-19 14:16:22.557 - 音频长度: 47520 samples, 2.97 秒
2026-02-19 14:16:22.559 - INFO - wyoming-funasr-stt-server - Audio length: 47520 samples, 2.97 s
rtf_avg: 0.578: 100%|█████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:01<00:00,  1.71s/it]
rtf_avg: 1.793: 100%|█████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:01<00:00,  1.72s/it]
rtf_avg: 0.581, time_speech:  2.970, time_escape: 1.727: 100%|████████████████████████████████████████████| 1/1 [00:01<00:00,  1.73s/it]
2026-02-19 14:16:26.056 - 过滤掉 SenseVoice 可能输出的情感/事件标签
2026-02-19 14:16:26.056 - INFO - wyoming-funasr-stt-server - 识别结果原文：可能输出的情感/事件标签- Result: <|zh|><|NEUTRAL|><|Speech|><|withitn|>换货。
2026-02-19 14:16:26.057 - INFO - wyoming-funasr-stt-server - Result: 换货。
2026-02-19 14:16:26.057 - 识别结果: 换货。
2026-02-19 14:16:35.711 - INFO - wyoming-funasr-stt-server - Transcribe received
2026-02-19 14:16:35.711 - INFO - wyoming-funasr-stt-server - AudioStart received
2026-02-19 14:16:38.141 - INFO - wyoming-funasr-stt-server - AudioStop received. Processing...  78400 bytes of audio...
2026-02-19 14:16:38.143 - 音频长度: 39200 samples, 2.45 秒
2026-02-19 14:16:38.143 - INFO - wyoming-funasr-stt-server - Audio length: 39200 samples, 2.45 s
rtf_avg: 0.028: 100%|█████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 14.81it/s]
rtf_avg: 1.693: 100%|█████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:01<00:00,  1.53s/it]
rtf_avg: 0.624, time_speech:  2.450, time_escape: 1.530: 100%|████████████████████████████████████████████| 1/1 [00:01<00:00,  1.53s/it]
2026-02-19 14:16:39.742 - 过滤掉 SenseVoice 可能输出的情感/事件标签
2026-02-19 14:16:39.742 - INFO - wyoming-funasr-stt-server - 识别结果原文：可能输出的情感/事件标签- Result: <|zh|><|NEUTRAL|><|Speech|><|withitn|>关火。
2026-02-19 14:16:39.742 - INFO - wyoming-funasr-stt-server - Result: 关火。
2026-02-19 14:16:39.742 - 识别结果: 关火。

pip3 list

Package                Version
---------------------- --------
aliyun-python-sdk-core 2.16.0
aliyun-python-sdk-kms  2.16.5
antlr4-python3-runtime 4.9.3
audioread              3.1.0
certifi                2026.1.4
cffi                   2.0.0
charset-normalizer     3.4.4
crcmod                 1.7
cryptography           46.0.3
decorator              5.2.1
editdistance           0.8.1
filelock               3.20.3
fsspec                 2026.1.0
funasr                 1.3.0
hydra-core             1.3.2
idna                   3.11
ifaddr                 0.2.0
jaconv                 0.4.1
jamo                   0.4.1
jieba                  0.42.1
Jinja2                 3.1.6
jmespath               0.10.0
joblib                 1.5.3
kaldiio                2.18.1
lazy_loader            0.4
librosa                0.11.0
llvmlite               0.46.0
MarkupSafe             3.0.3
modelscope             1.34.0
mpmath                 1.3.0
msgpack                1.1.2
networkx               3.6.1
numba                  0.63.1
numpy                  1.26.4
omegaconf              2.3.0
oss2                   2.19.1
packaging              26.0
pip                    23.0.1
platformdirs           4.5.1
pooch                  1.8.2
protobuf               6.33.4
pycparser              3.0
pycryptodome           3.23.0
pynndescent            0.6.0
pytorch-wpe            0.0.1
PyYAML                 6.0.3
requests               2.32.5
scikit-learn           1.8.0
scipy                  1.17.0
sentencepiece          0.2.1
setuptools             66.1.1
six                    1.17.0
soundfile              0.13.1
soxr                   1.0.0
sympy                  1.14.0
tensorboardX           2.6.4
threadpoolctl          3.6.0
torch                  2.1.0
torch_complex          0.4.4
torchaudio             2.1.0
tqdm                   4.67.1
typing_extensions      4.15.0
umap-learn             0.5.11
urllib3                2.6.3
wyoming                1.8.0
zeroconf               0.148.0

Strategies to reduce latency

1.Use smaller models

FunASR has paraformer-zh-small or paraformer-zh-medium

2.VAD pre-filtering

Skip silence chunks → speech → Skip silence chunks

Create Wyoming server for Home assistant Part3 - stt - wyoming-funasr arm64 sherpa-onnx

2026-02-19T19:01:24Z

stt - wyoming-funasr arm64 onnx

Wyoming asr server for Home assistant: Step-by-Step Guide for Developers

Building FunASR with sherpa-onnx on an ARM64 (aarch64) system.

To make an STT server work with Home Assistant, the industry standard is using the Wyoming Protocol.

sherpa-onnx: The "engine" that runs the stt. It manages the model files and utilizes the RPi5’s CPU.

Local Control: You use the Wyoming Protocol Integration bridge to link the stt server to Home Assistant's "Assist" pipeline.

Quick start
Create Python virtual environment
Install sherpa-onnx
- Install
Download Model
- download pre-trained models(SenseVoice)
server.py
- step 1. install numpy

Quick start

# Install sherpa-onnx

pip3 install sherpa-onnx sherpa-onnx-bin

pip3 install wyoming==1.8.0

pip3 install numpy

Create Python virtual environment

mkdir -p /funasr-wyoming-sherpa-onnx

cd /funasr-wyoming-sherpa-onnx

python3 -m venv venv

source venv/bin/activate

Install sherpa-onnx

Install

pip3 install sherpa-onnx sherpa-onnx-bin

Installing collected packages: sherpa-onnx-core, sherpa-onnx-bin, sherpa-onnx
Successfully installed sherpa-onnx-1.12.23 sherpa-onnx-bin-1.12.23 sherpa-onnx-core-1.12.23

https://k2-fsa.github.io/sherpa/onnx/python/install.html#method-1-from-pre-compiled-wheels-cpu-only

# pip3 show sherpa-onnx

output

Name: sherpa_onnx
Version: 1.12.23
Summary: 
Home-page: https://github.com/k2-fsa/sherpa-onnx
Author: The sherpa-onnx development team
Author-email: dpovey@gmail.com
License: Apache licensed, as found in the LICENSE file
Location: /funasr-wyoming-sherpa-onnx/venv/lib/python3.11/site-packages

Download Model

This section describes how to download pre-trained SenseVoice models.

download pre-trained models(SenseVoice)

cd /funasr-wyoming-sherpa-onnx



wget https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2025-09-09.tar.bz2

tar xvf sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2025-09-09.tar.bz2

rm sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2025-09-09.tar.bz2

# 安装 ModelScope
pip install modelscope

# SDK 模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('xiaowangge/sherpa-onnx-sense-voice-small')

output

root@raspberrypi:/funasr-wyoming-sherpa-onnx# tree -L 2
.
├── sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2025-09-09
│   ├── model.int8.onnx
│   ├── README.md
│   ├── test_wavs
│   └── tokens.txt

server.py

step 1. install numpy

pip3 install numpy

output

Installing collected packages: numpy
Successfully installed numpy-2.4.2

pip3 install wyoming==1.8.0

output

  Downloading https://www.piwheels.org/simple/wyoming/wyoming-1.8.0-py3-none-any.whl (39 kB)
Installing collected packages: wyoming
Successfully installed wyoming-1.8.0

cd /funasr-wyoming-sherpa-onnx

source venv/bin/activate

python3 server.py

useful links

how to download pre-trained SenseVoice models.

https://k2-fsa.github.io/sherpa/onnx/sense-voice/index.html

https://k2-fsa.github.io/sherpa/onnx/sense-voice/pretrained.html#sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17

https://k2-fsa.github.io/sherpa/onnx/sense-voice/pretrained.html#sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2025-09-09-chinese-english-japanese-korean-cantonese

How to repeat an action until another (specific) device trigger is received in home assistant

2026-02-03T22:23:10Z

How to repeat an action until another (specific) device trigger is received in home assistant

To repeat an action in Home Assistant until a specific device trigger is received, use a repeat-until loop combined with a wait_for_trigger action.

until

alias: New automation find my mobile phone - 找手机
description: >-
  - http://localhost:4999/boards/topic/16760/notify - ## 
  http://localhost:4999/boards/topic/16760 - ##
  http://192.168.2.125:8123/config/automation/edit/1747045045725
triggers:
  - trigger: conversation
    command: "[找手机|手机|手机在哪儿|我手机在哪儿]"
    id: id_find_my_phone
  - trigger: state
    entity_id:
      - binary_sensor.sm_g9910_private_interactive
    from:
      - "off"
    to:
      - "on"
    id: id_phone_interactive_turn_on
    enabled: true
conditions: []
actions:


  - repeat:
      until:
        - condition: trigger
          id:
            - id_phone_interactive_turn_on
      sequence:
        - action: script.turn_on
          metadata: {}
          data: {}
          target:
            entity_id: script.find_my_phone
        - delay:
            hours: 0
            minutes: 0
            seconds: 5
            milliseconds: 0
    enabled: false
 

mode: single

way 2.

sequence:
  - variables:
      stopVar: false
  - parallel:
      - repeat:
          until:
            - condition: template
              value_template: "{{ stopVar == true }}"
          sequence:
            - delay:
                hours: 0
                minutes: 0
                seconds: 1
                milliseconds: 0
            - action: input_boolean.toggle
              metadata: {}
              data: {}
              target:
                entity_id: input_boolean.quick_toggle
      - sequence:
          - wait_for_trigger:
              - trigger: state
                entity_id:
                  - button.push
          - variables:
              stopVar: true
alias: Parallel Test
description: ""

blog.matterxiaomi.com

how to run locally llama.cpp for home assistant on rpi5

Table of Contents

Run llama-server

Connect to Home Assistant

Integration - Add Integration

Voice assistant - Create conversation agent

How to Run AI Models Locally with llama.cpp on rpi5

Table of Contents

Prerequisites

step 1. Pull llama.cpp (light)

step 2. Pick a model - Download Gemma (GGUF, quantized)

step 3. Docker run and load model

step 4. test

How to Run AI Models Locally with Ollama

Table of Contents

Install Ollama with Docker

Pull and Run a Model via ollama

Test the LLM models via Ollama CLI

Configuration Checklist

Ollama vs LiteLLM vs llmama.cpp vs vvllm vs lm studio

Table of Contents

Managing (Ollama, LM Studio)

Serving (Llama.cpp, vLLM)

Routing (LiteLLM)

Matter Bridge in Home Assistant Part2 - Install MatterBridge Connect to Home assistant

Table of Contents

Quick start

Install and configure

How to Use

Ecovacs in Home Assistant Part6 - Robot Vacuum Control MCP Server

ModelScope vs Hugging Face vs k2-fsa.github.io vs Kaldi vs Sherpa

Table of Contents

Hugging Face（Global AI model platform）

ModelScope(The Alibaba/Chinese AI Industrial model platform)

specialized tools focused on speech recognition (ASR) and synthesis (TTS)

k2-fsa (Next-Gen Kaldi)

download Silero VAD ONNX model

Sherpa(The Real-Time Speech Deployment Tool)

Create Wyoming server for Home assistant Part2 - stt - wyoming-funasr arm64

Table of Contents

Step 1.Development Environment Setup

Create Python virtual environment

Step 2. Install

Install torch via PyPI

Install FunASR 1.3.0 via PyPI

Verify installation

Step3.Download and test a model (example: paraformer-zh)

SenseVoice - Speech Recognition (Non-streaming)

Step 4.FunASR + Wyoming STT full server

Install wyoming

Strategies to reduce latency

Create Wyoming server for Home assistant Part3 - stt - wyoming-funasr arm64 sherpa-onnx

Table of Contents

Quick start

Create Python virtual environment

Install sherpa-onnx

Install

Download Model

download pre-trained models(SenseVoice)

server.py

step 1. install numpy

How to repeat an action until another (specific) device trigger is received in home assistant