Wow, it seems like this might beat out flite for very-low-memory TTS? I ended up abandoning a project of mine because I couldn't get high enough quality or low enough memory usage out of flite, so I'm very excited to try this out.
That's the error rate for STT, not TTS. TTS is generally easier than STT because you only need to produce one valid pronunciation and don't need to handle variation within and between individuals.
This looks like an extreme point for AI-based TTS, as formant/tract modeling synths tend to be more accurate if you want TTS in a tiny amount of compute, but sound distinctly robotic.
Presumably it's not, but the TTS voice in the video sounds to me more like formant synthesis than diphone - it reminds me of my DECtalk.
The project credits does mention espeak (which is formant based) as well as various other TTS projects, although it sounds like they are only using the pronunciation part of espeak, not the voice synthesis.
I suppose, but for home automation, esps are best for getting the audio to something more powerful. If this lets a raspberry pi do voice recognition really fast, that alone is worth it.
It looks great, thank you! I'll see if I can use it for my in browser AI assistant project's ( https://aidekin.com ) voice part. It's currently using Nemotron-3.5-ASR and supertonic-3 but overall it requires 1.2gb download.
this is good to see. i also trained a stt under 500kb for sub dollar chips. it had about 20 words that it could understand(like start, stop, left, right, go, up etc) and then the spell mode where you could say the word spell and then say the individual english alphabets and close with spell. it was super fun to work on. these tend to be extremely unstable though, like confusion between p and t (at least for my accent). will have to try this one now.
The voice activity detection alone here is compelling - very useful for doing things like highlighting a speaker who's transmitting in realtime. At that rate the impact on perf will be so minimal that you could easily run it in the browser across devices.
Flite for comparison: https://github.com/festvox/flite
I’ve worked in this space. TTS in a small footprint isn’t the hard part —- it’s doing it accurately that’s hard.
Although for the use cases OP is targeting, lower accuracy may be good enough!
This actually holds for everything in AI.
https://github.com/moonshine-ai/moonshine#when-should-you-ch...
[1] https://github.com/gmn/nanotts
Couldn’t find a link, is that hard to do?
good job on a clear readme.md tbh
TTS (neural diphone synth @ 16 kHz) ~1.8 MiB voice pack
This is in the realm of Microsoft Sam.
The project credits does mention espeak (which is formant based) as well as various other TTS projects, although it sounds like they are only using the pronunciation part of espeak, not the voice synthesis.
https://github.com/moonshine-ai/moonshine#acknowledgements
https://esphome.io/
Played by the great Richard Kind, who my wife swears she saw on the Highline in NYC.