Sensory Speech Technologies on Arm IP: Cortex-M & Ethos-U microNPU

Sensory has gone all-in on supporting the new Cortex-M55/85 and Ethos-U55/65  microNPU from Arm. Sensory’s array of new offerings for speech recognition and Sound ID takes advantage of the Arm U55/65 and or CM55/85 series to include new ways to build models, run deep-learned inference, plus enhanced compatibility with TensorFlow Lite …Let’s explore some of these offerings.

The U55/U65 offers the ability to act as a standalone inference engine for deep-learned speech and Sound ID models allowing low-power solutions with very powerful results. Sensory has deployed a few architecture approaches that take advantage of various combinations of U55/65 and M55/85:

  1. Running a Tensorflow-Lite/Micro model as part of an end-to-end speech recognizer on the U55/65. End-to-end deep-learned systems tend to produce the most accurate general-purpose results but typically require a lot of memory. Fortunately, the U55/65 has the power to do this. If several tens of megabytes or more are available in memory this approach could present an optimal solution for general purpose speech-to-text.
  2. Running an acoustical model on the U55/65 and a language model on the M55/85 allows a more specialized, more accurate, and more compact solution. This would require some domain data or domain understanding to create the language model. The total memory for this approach can be well under 10 MB, and would produce the optimal accuracy for domain-specific use cases. The M85 is powerful enough to run both the acoustic and language models without need for the U55/65.
  3. Sensory-built models for TrulyHandsfree wake words and commands, TrulyNatural large-vocabulary natural language speech, and Sound ID can now run on Tensorflow-Lite/Micro compatible inference engines. This brings Sensory’s accuracy, language coverage, and Sound ID technologies into a new suite of Arm-based inference engine platforms that support TensorFlow.

Sensory is happy and excited to be working closely with Arm to introduce the first wave of speech recognition and Sound ID for the new U55/65, and M55/85 cores!

For a deep dive into this topic, look out for CEO Todd Mozer’s talk at the upcoming Arm Dev Summit, and register today!