Part 1 - Local Voice in Home Assistant core
Voice in Home Assistant
I’m using the ESP32-S3-BOX-3B microphone .Rhasspy’s wyoming-piper and wyoming-faster-whisper can be installed in several different ways. The easiest way is with Docker, which will pull a 0.3-0.8GB image.
This is the entire solution if you are interested.
Table of Contents
Conversation agent:Home Assistant
Speech-to-text:wyoming-faster-whisper
Text-to-speech:wyoming-piper
WakeWord: On device
Hardware
A voice assistant needs hardware.such as Android phone or ESP32-S3-BOX or ESP32-S3-BOX-Lite
Software
docker image of the Whisper, Piper, and OpenWakeWord。https://hub.docker.com/r/rhasspy/wyoming-whisper
Quick start
Here are the steps I took in order to get this to work:
Step 1. we need to install Whisper and Piper.
Open xshell and run the following command:
Install wyoming-piper and wyoming-faster-whisper
Step 2.Run Whisper, Piper, and OpenWakeWord on external server -> external voice services
Run a single-container speech-to-text service on Docker
Run a single-container text-to-speech service on Docker
Step 3.Connect it to home assistant core via wyoming integration -> connects external voice services to Home Assistant.
Open Home Assistant go to Settings → Integrations → Add Integration → Wyoming Protocol
It should ask you for a host and a port now. Enter the IP address of your device into the host-field and enter port in the port-field.
Step 4.Exposing your devices to Assist.
Control HA with your voice without any cloud integration
I am running Home Assistant Core using the docker container, Is there any way that we HA-core peasants to run the add-ons for Whisper and Piper？
I have been using Raspberry Pi Zero, 3A and 3B models with Rhasspy as voice assistant satellites, and decided now is the time to swap to HA Voice Assist.
https://github.com/rhasspy/piper
or
https://github.com/rhasspy/wyoming-faster-whisper
https://github.com/rhasspy/wyoming-piper
The easier way is to use official docker image and you can run it on another host if you want, just specify the right IP address and ports when you will configure the integrations.
Im running Home Assistant in a Docker container behind a reverse proxy. I’ve installed external Piper and Whisper containers (rhasspy/wyoming-whisper, rhasspy/wyoming-piper) and connected them through up the wyoming protocol integration.
Outline:
Verify your OS settings
Install Docker on guest system
Install Whisper and Piper on guest system
Install Whisper and Piper integration on host system
Pass the Whisper and Piper from the guest system
Part 1 - Local Voice in Home Assistant core
https://blog.matterxiaomi.com/blog/voice-homeassistant/
Part 2 - Run whisper on external server
https://blog.matterxiaomi.com/blog/run-whisper-on-external-server/
Part 3 - Run piper on external server
https://blog.matterxiaomi.com/blog/run-piper-on-external-server/
Part 4 - Run wakeword on external server
https://blog.matterxiaomi.com/blog/run-wakeword-on-external-server/
How it work
1.wakeword detection
a ESP32 with I2S Mic using ESPHome and successfully detect wakeword.
Assist Hardware Passing(streaming) the wav audio data to stt.
2.
5.intent recognition and intent execution
8.text to speech
Speech Processing
https://www.home-assistant.io/images/blog/2024-12-voice-chapter-8/voice-pipeline.png
source:https://www.home-assistant.io/blog/2024/12/19/voice-chapter-8-assist-in-the-home/
conversation(会话组件)
会话组件允许您与Home Assistant交谈。
step by step
step 1.Run Whisper, Piper, and OpenWakeWord on external server
Run Whisper, Piper, and OpenWakeWord with Docker.Pull the container,change path to your path and run it.
step 2.Connect it to home assistant core via wyoming integration
Just use the ha core and setup the wyoming integration. For host I selected localhost and for port pointed at port10200 and 10300
Whisper and piper arre recognized in home assistant.
The Wyoming integration connects external voice services to Home Assistant using a small protocol.
http://localhost:4999/boards/topic/14594/voice-in-home-assistant#22634
Note
Note that these images are only supported on 64bit architectures
I’ve tries with Raspberry 3B+ and Raspberry OS (32bit) and seems images are only for ARM64.I can not pulling on a rpi3 b+ because it cannot find the arm version.
The issue is that the standard rpi OS is in 32 bit and the images for piper and whisper are 64 bit.
https://www.matterxiaomi.com/boards/topic/15742/local-voice-functionality-in-home-assistant-corehow-to-manually-install-piper-and-whisper-on-home-as#22675
Additional info -
mkdir /root/whisper-data# Piper
docker run -d -p 10200:10200 \
-v /root/piper-data:/data \
--restart unless-stopped \
rhasspy/wyoming-piper \
--voice en_US-lessac-medium
# Whisper
docker run -d -p 10300:10300 \
-v /root/whisper-data:/data \
--restart unless-stopped \
rhasspy/wyoming-whisper \
--model tiny-int8 --language en
# openWakeWord
docker run -d -p 10400:10400 \
--restart unless-stopped \
rhasspy/wyoming-openwakeword \
--preload-model 'ok_nabu'
Additional info -Install Option
Rhasspy’s wyoming-piper and wyoming-faster-whisper can be installed in several different ways. The easiest way is with Docker, which will pull a 0.3-0.8GB image
When install piper in your network you have three options
Option 1.Install wyoming-piper and wyoming-faster-whisper locally with Python virtual environment detail
pip install git+https://github.com/openai/whisper.git Option 2.Install piper locally (and also whisper), with docker detail
Install piper locally (and also whisper), with add-on detail
Option 3.build from source detail
How to improve voice recognition
1.You have to change the model itself
The bigger the model, the better the recognition, however more resources are required and it may take longer to respond.
After I changed from Tiny to Medium, for example Chinese is sooo much better recognized
Useful links
source:https://developers.home-assistant.io/docs/voice/overview/
https://community.home-assistant.io/t/how-to-manually-install-piper/568040
https://www.home-assistant.io/blog/2024/06/26/voice-chapter-7/
https://github.com/rhasspy/wyoming-addons
Comments
Comments are closed