Hugging Face, ModelScope, and k2-fsa.github.io (specifically the k2-fsa/sherpa-onnx project) represent different approaches to the machine learning ecosystem。

Hugging Face and ModelScope host everything. k2-fsa and Sherpa only do Speech.

k2-fsa and Sherpa are highly specialized tools focused on speech recognition (ASR) and synthesis (TTS)

 

Sherpa (often referred to as sherpa-onnx or sherpa-ncnn) is a lightweight speech-to-text (ASR) and text-to-speech (TTS) engine. Best for: Deploying speech models on edge devices (Android, iOS, WebAssembly, ARM boards) or high-performance servers, prioritizing low latency and CPU efficiency.

Table of Contents

These four entities represent two different categories: Model Ecosystems (Hugging Face & ModelScope) and Speech Recognition Frameworks (Kaldi & k2-fsa).

 

Hugging Face（Global AI model platform）

The Global Industry Standard。It supports NLP, computer vision, audio, and multimodal models via transformers and diffusers libraries.

download models

https://huggingface.co/models

 

https://huggingface.co/FunAudioLLM/SenseVoiceSmall

 

https://huggingface.co/funasr/paraformer-zh/blame/7904416f6cb6290ee7dc0b2ddb2993a9fe4f421a/README.md

 

ModelScope(The Alibaba/Chinese AI Industrial model platform) 

ModelScope is an AI model hub led by Alibaba DAMO Academy. It provides pre-trained models, pipelines, and deployment tools, especially strong in Chinese language and speech technologies.ModelScope is often described as the "Chinese Hugging Face."

 

https://modelscope.cn/search?search=sherpa%20onnx

Inference  Framework

1.funasr

2.funasr-onnx

 

see:https://github.com/modelscope/FunASR?tab=readme-ov-file#sensevoice

 

Alibaba Speech-to-Text models tree

Alibaba Speech-to-Text models tree

FunAudioLLM
│
├── FunASR（ASR 推理框架）
│
│   ├── FunASR 1.3
│   │   ├── Paraformer series
│   │   │   ├── paraformer-zh
│   │   │   ├── paraformer-large
│   │   │   ├── paraformer-streaming
│   │   │   └── paraformer-hotword
│   │
│   ├── FunASR 1.5
│   │   ├── Paraformer（升级版）
│   │   ├── FunASR-Nano series
│   │   │   ├── Nano Small
│   │   │   ├── Nano Streaming
│   │   │   └── Nano Int8（边缘部署）
│   │   │
│   │   ├── SenseVoice series
│   │   │   ├── SenseVoice Small
│   │   │   ├── SenseVoice Large
│   │   │   └── SenseVoice multilingual
│   │
│   └── Inference Framework
│       ├── PyTorch
│       ├── funasr-onnx
│       ├── Sherpa-ONNX
│       └── TensorRT
│
└──

https://github.com/FunAudioLLM/SenseVoice

https://github.com/modelscope/FunASR

 

specialized tools focused on speech recognition (ASR) and synthesis (TTS)

 

Kaldi: Speech Toolkit

The "grandfather" of modern speech recognition. It is a C++ based toolkit developed primarily by Dan Povey.

demo

KaldiRecognizer

https://github.com/rhasspy/wyoming-faster-whisper/blob/main/wyoming_faster_whisper/__main__.py

k2-fsa (Next-Gen Kaldi)

 Speech Toolkit.The Modern Successor

What it is: Often called "Next-gen Kaldi." It is a complete rewrite of Kaldi’s core concepts to make them natively compatible with PyTorch.

Key Repositories:

Icefall: Where the actual training recipes for speech models (like Zipformer) live.

k2: The core library for differentiable FSTs.the old version of the tool you used before k2-fsa existed.

 

download sherpa-onnx

https://github.com/k2-fsa/sherpa-onnx/releases

 

download sherpa-onnx asr models

https://k2-fsa.github.io/sherpa/onnx/pretrained_models/index.html

https://github.com/k2-fsa/sherpa-onnx/releases/tag/asr-models

 

 download Silero VAD ONNX model

# https://k2-fsa.github.io/sherpa/onnx/sense-voice/pretrained.html#sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17
wget https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/silero_vad.onnx

 

download url:https://k2-fsa.github.io/sherpa/onnx/vad/silero-vad.html#download-models-files

 

sherpa has the following sub-projects:

k2-fsa/sherpa

k2-fsa/sherpa-onnx

k2-fsa/sherpa-ncnn

The differences are compared below:

 

 

k2-fsa/sherpa

k2-fsa/sherpa-onnx

k2-fsa/sherpa-ncnn

Installation difficulty

hard

easy

super easy

NN lib

PyTorch

onnxruntime

ncnn

CPU Support

x86, x86_64
x86, x86_64,
arm32, arm64
**RISC-V**
x86, x86_64,
arm32, arm64,
**RISC-V**

GPU Support
Yes
(with CUDA
for NVIDIA GPUs)

Yes
Yes
(with Vulkan
for ARM GPUs)

OS Support
Linux, Windows,
macOS
Linux, Windows,
macOS, iOS,
Android
Linux, Windows,
macOS, iOS,
Android
Support
batch_size > 1

Yes

Yes

No

Support RKNN

No

Yes

Yes

Provided APIs

C++, Python
C, C++, Python,
C#, Java, Kotlin,
Swift, Go,
JavaScript, Dart
Pascal, Rust
C, C++, Python,
C#, Kotlin,
Swift, Go

Supported functions
streaming ASR
non-streaming ASR
streaming ASR,
non-streaming ASR,
text-to-speech,
speaker diarization,
speaker identification,
speaker verification,
spoken language
identification,
speech denoising,
speech enhancement,
audio tagging,
VAD,
keyword spotting,
streaming ASR,
non-streaming ASR,
text-to-speech,
VAD,

 

Sherpa(The Real-Time Speech Deployment Tool)

The deployment engine (CPU/GPU, Android, iOS, WebAssembly).It uses models trained in the k2-fsa ecosystem.

 

How they all fit together

1.k2-fsa is the tool you use to build a high-performance speech model.

silero-vad model download

https://k2-fsa.github.io/sherpa/onnx/vad/silero-vad.html#download-models-files

 

Once you've trained that model using k2-fsa, you might upload it to Hugging Face or ModelScope so others can download it easily.

Hugging Face hosts models from both ModelScope and k2-fsa/Sherpa, serving as a distribution point for them. 

 

 

Layer Relationship

Model Hosting & Distribution
   ├── ModelScope
   └── Hugging Face

Inference / Runtime Framework
   └── k2-fsa / sherpa

 

 

 

 

 

Comments


Comments are closed