<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Sensory Perspectives on AI - Medium]]></title>
        <description><![CDATA[The official publication of Sensory, Inc., where innovation meets insight. Led by our CEO, Todd Mozer, we share thought leadership articles and insightful blogs that explore the forefront of voice and vision AI. - Medium]]></description>
        <link>https://medium.com/sensory-perspectives?source=rss----0d6bb4fd552d---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Sensory Perspectives on AI - Medium</title>
            <link>https://medium.com/sensory-perspectives?source=rss----0d6bb4fd552d---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 01:29:35 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/sensory-perspectives" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Future Architecture of LLMs]]></title>
            <link>https://medium.com/sensory-perspectives/the-future-architecture-of-llms-6da298641724?source=rss----0d6bb4fd552d---4</link>
            <guid isPermaLink="false">https://medium.com/p/6da298641724</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[voice-technology]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[voice-tech]]></category>
            <dc:creator><![CDATA[Todd Mozer's Desk]]></dc:creator>
            <pubDate>Wed, 23 Oct 2024 16:49:26 GMT</pubDate>
            <atom:updated>2024-10-23T16:48:00.785Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pvcN8ICtRGNOeabtNFiSNw.png" /></figure><p>Large Language Models (LLMs) are rapidly improving, with fewer hallucinations, more thoughtfully researched answers, and they are getting easier to work with! However, they are also growing exponentially in size, computational needs, energy consumption, and training costs.</p><p>In parallel with the rise in capabilities, consumption is also rapidly growing. The latest multimodal models with voice input are driving growth in interactive voice assistants across markets like finance, medical, enterprise, consumer, automobiles, hospitality, and retail/food/service industries.</p><p>Most of the growth today in LLMs is in cloud services, and much of this growth is not yet fully monetized with SaaS models, monthly fees, and advertisements. Keeping everything in the cloud is an expensive venture that users may not appreciate due to cost concerns or the intrusion of advertising. However, moving everything on-device is also impractical, as running LLMs can be quite expensive in terms of hardware.</p><p>There are three major platforms for LLMs: Cloud Training, Cloud Inference, and On-Device Inference. Training requires larger and larger systems and will remain a cloud-based offering. The inference market will grow to be much bigger than training and will likely rely on hybrid approaches combining client and cloud.</p><p>If hotwords are deployed, they will need to be on-device. This is essential for both bandwidth reasons and privacy. It’s likely that these voice-activated low-power wakewords will move into the realm of intelligent on-device speech controllers that do not always require a hotword to be spoken, but intelligently modify parametric control to better suit the user experience and optimize performance.</p><p>After the <a href="https://www.sensory.com/wake-word/">hotword</a>, I expect a growing number of use cases will opt for speech-to-text translation on-device. This is also useful for privacy and bandwidth reduction, as sending text will not only be less expensive but will be faster as well, since text data can be 1/1000 the size of speech!</p><p>Small LLMs, utilizing under 7 billion parameters, have emerged. Quantization and other approaches can shrink these down to run efficiently on-device. However, Sensory’s experimentation has found that this shrinking works well for domain-specific applications. For general intelligence and the ability to engage in conversational dialogue, bigger is way better, and these LLMs will likely run in the cloud for some time or possibly on high-end products costing many thousands or tens of thousands of dollars.</p><p>Sensory is seeing increasing demand for hybrid approaches with a few distinct architectures:</p><ol><li>Wakeword on device, LLM in the cloud</li><li>Wakeword and speech-to-text (STT) on-device, LLM in the cloud</li><li>Wakeword and STT on-device with a domain-specific tiny LLM on-device</li><li>Wakeword and STT on-device with a domain-specific small LLM on-device AND cognitive arbitration to decide when to send to a cloud LLM</li></ol><p>We are also seeing interest in hot-word and STT on-device with an on-device SLM, but the SLM quality does not seem to be there yet…it will probably get there as LLM’s and processing capabilities continue to improve.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6da298641724" width="1" height="1" alt=""><hr><p><a href="https://medium.com/sensory-perspectives/the-future-architecture-of-llms-6da298641724">The Future Architecture of LLMs</a> was originally published in <a href="https://medium.com/sensory-perspectives">Sensory Perspectives on AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Small Language Models (SLM), Large Language Models(LLM), or Micro LLM (MLM)?]]></title>
            <link>https://medium.com/sensory-perspectives/small-language-models-slm-large-language-models-llm-or-micro-llm-mlm-27145a88e725?source=rss----0d6bb4fd552d---4</link>
            <guid isPermaLink="false">https://medium.com/p/27145a88e725</guid>
            <category><![CDATA[llm-applications]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Todd Mozer's Desk]]></dc:creator>
            <pubDate>Fri, 23 Aug 2024 17:11:32 GMT</pubDate>
            <atom:updated>2024-08-23T17:11:17.389Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*njVnGGJyBvQS5tajfhv_GA.png" /></figure><p><strong>LLMs &amp; Small LLMs.</strong></p><p>Since the arrival of generative AI and Large Language Models the world has learned a whole new vocabulary. It’s impressive to me that “Large Language Models” and LLMs have moved into tech societies’ mainstream dialog and conversations.</p><p>Language Models are nothing new. Since the early days of Hidden Markov Modeling, language models have been an important component of a voice-based system. In fact, SLM’s used to mean Statistical Language Model as opposed to a simple grammar which was also a Language Model.</p><p>Now SLMs are Small Language Models. But they aren’t small. An LLM can be up to a trillion or more parameters. Today SLM’s are under 5 billion parameters (plus or minus a few billion parameters). They really should be called <strong><em>Small LLMs</em></strong>.</p><p>Typically, Small LLMs are reductions and quantization that focus in on certain domains. So there may be extreme expertise and conversation-ability about a particular topic, but only broad knowledge beyond that.</p><p><strong>Chips for LLMs.</strong></p><p>Building a LLM is very different than running an LLM. The building side is done typically in large data centers with big Nvidia chips. Running an LLM is often on a similar type of cloud or data center platform, but takes significantly less time and or processing power.</p><p>The hope for Small LLMs is to run on device in a product rather than going out to a Cloud, Data Center, or a large On Premise solution. <a href="https://www.sensory.com/embedded-speech-to-text/">On-device solutions</a> offer several advantages in cost (i.e. no service fees), privacy, and availability (no connection required).</p><p>The platforms that run Small LLM’s are nevertheless quite large and power hungry. Think of cell phone chips from Qualcomm or Mediatek or even PC based Intel systems. This works for higher end products that can handle the cost and heat, but not for low-cost consumer electronics, like the dream of wearable assistants that we take with us.</p><p><strong>Creating Micro LLMs.</strong></p><p>There’s a need to go even smaller and break into more mainstream chips and products. We’d want this for reasons such as cost and lower heat dissipation (e.g. we can’t have glasses or earbuds that heat us up!). For this, a Micro LLM is needed that can run on 100 million parameters or less. Much like with Small LLM’s, sacrifices need to be made to reduce computational and memory requirements. A Micro LLM can have the same domain expertise but lose some of the conversational ability with more directed questions and responses.</p><p>Micro LLM solutions can more easily run on the lower end of the Qualcomm and Mediatek type products but are also a really nice fit for new generations of processors with special purpose acceleration for TensorFlow Lite and Tensorflow Lite Micro. ARM for example has an IP line called <a href="https://www.youtube.com/live/LNFiqhIPbfc?si=7eBP0qL5durBvBxu">Ethos U series</a> (55, 85, etc.) that is bringing the inference costs down, Cadence has similar efforts to provide IP for inference.</p><p><strong>Micro LLM’s with Cognitive Arbitration to LLMs</strong></p><p>One of the techniques Sensory has deployed over the years to lower cost or power consumption is to have multi-staged models. For example, for a wake word or sound identification, we deploy a tiny model that can be “always on”, then we have backup bigger models that improve on the accuracy but are only periodically called on.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FfVYJzDkR6m0%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DfVYJzDkR6m0&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FfVYJzDkR6m0%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/8752b33093bffe10af970a6f23854239/href">https://medium.com/media/8752b33093bffe10af970a6f23854239/href</a></iframe><p>This same sort of layered approach can be applied to LLMs, with a MicroLLM running on device and Cloud LLM or SLM deployed as a backup. For example, Sensory has an automotive Voice Assistant solution that can run on device with as little as 34 MB. It accurately captures and responds to automotive commands, but when an out of domain query is given, we can pass on to a Cloud (or bigger on device) solution. Multiple domains can run simultaneously, and model built for cognitive arbitration can determine which Micro, Small, or Large Language Model should respond.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=27145a88e725" width="1" height="1" alt=""><hr><p><a href="https://medium.com/sensory-perspectives/small-language-models-slm-large-language-models-llm-or-micro-llm-mlm-27145a88e725">Small Language Models (SLM), Large Language Models(LLM), or Micro LLM (MLM)?</a> was originally published in <a href="https://medium.com/sensory-perspectives">Sensory Perspectives on AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LLMs and Voice Assistants]]></title>
            <link>https://medium.com/sensory-perspectives/llms-and-voice-assistants-d7acc25524f0?source=rss----0d6bb4fd552d---4</link>
            <guid isPermaLink="false">https://medium.com/p/d7acc25524f0</guid>
            <category><![CDATA[subscription]]></category>
            <category><![CDATA[voice-assistant]]></category>
            <category><![CDATA[ethical-ai]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Todd Mozer's Desk]]></dc:creator>
            <pubDate>Wed, 31 Jul 2024 15:39:42 GMT</pubDate>
            <atom:updated>2024-07-31T15:39:42.168Z</atom:updated>
            <content:encoded><![CDATA[<p>Voice Assistants &amp; Dependency…it’s Gonna get Bad!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WddlSiL5J6Uzwr9J1ue4SQ.png" /><figcaption>Generated using DALL.E</figcaption></figure><p>I’m not particularly concerned about the dystopian futurist predictions of AI taking over the world and destroying humans, certainly not in the near term. But there are some very concerning things ranging from the spreading of bipartisanship (that we’ve seen from news and targeted social media) to the loss of privacy and private data. But I want to bring up something new…The increasing dependency we will have on our voice assistants and the concern that we as consumers will be taken advantage of for this dependency.</p><p>David Cahn from Sequoia Capital has eloquently <a href="https://www.sequoiacap.com/article/ais-600b-question/">written about</a> the lack of ROI in current AI investments. I generally agree with his sentiments (e.g. a few big winners and many losers). But I have a perspective on the winners, why they will win big, and how consumers’ pocketbooks will get hurt, and I want to raise the question about legality and ethics.</p><p>Often the best predictor of the future is the past, and we can look at the Voice Assistants of the past and get a pretty good idea of what’s in store for us, and it’s not pretty. As an active and early user of Alexa/Echo devices and the Google Assistant with their smart speakers here’s the path I have seen:</p><p>1) They start with subsidized or low-margin hardware with the best UX they can create</p><p>2) Advertising starts creeping in and becoming disruptive to the UX</p><p>3) Music playing became crippled (I stopped using Google devices at this point because the experience was so horrible), as they pitched subscription services</p><p>4) Now, Alexa randomly engages with me when I don’t call her name. Where did this crazy idea come from???</p><p>5) Next will be subscription fees since they haven’t made money from purchasing through Alexa</p><p>I rarely use these devices for information access, I mostly use the Echo to play music and now I often link to my phone and play on Spotify so the “smart” is gone from the speaker. So this transition hasn’t been too painful, because I never built a dependence on these <a href="https://www.sensory.com/domain-specific-voice-assistants/">voice assistants</a>. They don’t know much about me beyond what music I listen to and they are fairly generic beyond knowing my name, location, and Prime information.</p><p>In contrast, I have built a dependence on a guitar-playing app that I bought a few years ago for $1. I have downloaded hundreds of my favorite songs, fixed wrong chords, and changed key signatures. It’s all stored in the app. Recently this app has stopped working for me and I need to upgrade to a subscription. They are offering me the “discounted rate” of $99/yr. I have invested a lot of time and energy into this app that has all my data so I’m very tempted to subscribe, despite their ability to change pricing at any time and for any reason, and the very steep increase in pricing from what I was originally sold.</p><p>Now imagine the new generation of <strong>voice assistants</strong>. They are smarter than ever before, they will be extremely helpful, and they will know all about us and we will become DEPENDENT on them. It’s that dependency that will drive the corporate revenues and hurt our pocketbooks!</p><p>Apple’s Siri now has “Apple Intelligence”. It can learn through our emails, texts, posts, what we look at and who we talk to. Google is doing similar things. I expect Siri will evolve from an assistant that everyone complains about to an assistant that everyone loves, and it will become a true personal assistant, really helping us in our daily lives. And once that dependence exists, then what is to stop Apple from following the same path that Amazon, Google and my guitar app tried to do…sell us a product, increase advertising, grow a dependency then charge more for usage!</p><p>As a final thought, I want to talk about the ethics and legalities of selling a product, learning about us, and then charging more for what we already thought we bought. I get it that companies are making investments, and they need revenues and profits to keep investing and growing. However, I find it deceptive that we buy a product, and we think the software is included then we later find out we need to pay more for the full value that we thought we already bought…especially since the value change is from our personal data!</p><p>Electronics have emerged from a world where the software came with the product and was “dumb” and didn’t change (like in our TVs and microwaves of yesteryear), to a world of smart devices collecting data, communicating with a cloud, downloading new features (or bugs, advertisements, solicitations, interruptions, etc.). Now the product we bought can ask us for money for what we thought we already paid for…and we will become dependent on subscribing…Is that fair or legal? What if you bought a <strong>microwave</strong> and the buttons stopped working unless you paid a subscription fee?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d7acc25524f0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/sensory-perspectives/llms-and-voice-assistants-d7acc25524f0">LLMs and Voice Assistants</a> was originally published in <a href="https://medium.com/sensory-perspectives">Sensory Perspectives on AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Your Voice Assistant is MMETYT]]></title>
            <link>https://medium.com/sensory-perspectives/your-voice-assistant-is-mmetyt-4c0a544cb231?source=rss----0d6bb4fd552d---4</link>
            <guid isPermaLink="false">https://medium.com/p/4c0a544cb231</guid>
            <category><![CDATA[voice-assistant]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[gell-mann-amnesia]]></category>
            <dc:creator><![CDATA[Todd Mozer's Desk]]></dc:creator>
            <pubDate>Mon, 15 Jul 2024 15:26:31 GMT</pubDate>
            <atom:updated>2024-07-15T15:26:31.344Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*235ozepW1Aa4igEjqcbNdA.png" /><figcaption>Created with DALL·E by A. Adeboje</figcaption></figure><p>I love <a href="https://pi.ai/">Pi</a>, the voice Assistant from Inflection. It’s extremely helpful to me and has aided in everything from being a knowledgeable tour guide to helping me fix broken products.</p><p>I think it’s really accurate and only makes occasional errors. But I’m wrong. Our Voice Assistants using LLMs are Making More Errors Than You Think (MMETYT).</p><p>This is a well-known phenomenon, but I couldn’t remember the name, so I asked Pi:</p><blockquote>“There’s a syndrome that’s known because when a person looks at answers from devices and sees errors in the things that they are an expert in, they think those are the only errors, and they don’t extrapolate those errors to things they don’t know about. What is this syndrome called?”</blockquote><p>Pi did a great job responding to my poorly worded question and said:</p><blockquote>“The phenomenon you’re referring to is called the Gell-Man Amnesia effect, named after physicist Murray Gell-Man. This effect describes the tendency of people to notice errors in media or information sources regarding subjects they are knowledgeable about, but then fail to apply the same level of skepticism or critical thinking when consuming information about topics they are less familiar with. In other words people tend to compartmentalize their skepticism and forget to apply it universally, leading to a potential bias in their evaluation of information sources.”</blockquote><p>Occasionally I notice Pi making big errors. In talking to her about the Sagrada Familia in Barcelona, she told me Michelangelo built part of it. I think this was an error occurring from a probabilistic word sequence. She started talking about tiles which led to Italian tiles, and then the Italian Renaissance artist, Michelangelo.</p><p>Sometimes asking the LLM to fact check and confirm what they said works to catch these mistakes. Other times it doesn’t. LLMs were generally poor at math, but now PI seems flawless. I noticed where she’s really bad is palindromes, where the letters going backward are the same as forwards. Pi can tell me famous palindromes that exist but when I ask her to make one up it invariably fails. I asked her to make up a palindrome about Pi, and she said:</p><blockquote>“A man, a plan, a Pi, an AMA, Panama! This palindrome is a play on the famous “A man a plan a canal Panama”. “</blockquote><p>I asked: “Are you sure that’s right?” and Pi insisted: “Yes I’m confident that the palindrome I provided is correct. A palindrome is a word or phrase or sequence that reads the same backwards as forwards and “A man, a plan, a Pi, an AMA, Panama!” fits that description.”</p><p>When I told her she was wrong she apologized and offered a simple well known palindrome “Taco cat”. I asked her to make up another one from scratch about Pi and to double check to make sure it fit the definitions, and she failed with:</p><blockquote>“Pie, fill a pie-lover’s crave. A revel. Cars evolver-pile? A fill, I Pi.”</blockquote><p>As LLM’s increasingly are used for information I wonder what percentage of information is incorrect? I guess it’s a low hurdle to be better than today’s popular press, but they are Making More Errors Than You Think!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c0a544cb231" width="1" height="1" alt=""><hr><p><a href="https://medium.com/sensory-perspectives/your-voice-assistant-is-mmetyt-4c0a544cb231">Your Voice Assistant is MMETYT</a> was originally published in <a href="https://medium.com/sensory-perspectives">Sensory Perspectives on AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Is “the bigger the better” true for LLMs? How small can they get?]]></title>
            <link>https://medium.com/sensory-perspectives/is-the-bigger-the-better-true-for-llms-how-small-can-they-get-3d01caa01ce6?source=rss----0d6bb4fd552d---4</link>
            <guid isPermaLink="false">https://medium.com/p/3d01caa01ce6</guid>
            <category><![CDATA[ai-on-device]]></category>
            <category><![CDATA[small-language-model]]></category>
            <category><![CDATA[speech-recognition-ai]]></category>
            <category><![CDATA[wake-word]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Todd Mozer's Desk]]></dc:creator>
            <pubDate>Tue, 02 Jul 2024 11:52:26 GMT</pubDate>
            <atom:updated>2024-07-02T11:52:26.678Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Udz6WuUws6NuPxH_tFJVuQ.png" /></figure><p>With the introduction of Apple AI and on-device Chat GPT there are growing conversations about small language models that run on the edge. I’ve seen them referred to as SLM’s which I find somewhat humorous, as SLM used to be a Statistical Language Model.</p><p>That’s right, language models have been around for a long time. Sensory has been building them since the early 2000’s! What changed wasn’t as much the formula but more the expansion going deeper and wider with larger data sets, more memory, and increased processing power.</p><p>And size does make a BIG difference. Today’s LLMs exceed trillions of parameters with tens or hundreds of terabytes of data. And they are amazing at conversational capabilities. But they don’t run on device, making them less private, more costly, and impractical without a strong data connection.</p><p>The general-purpose use, conversational memory and dialog, and broad expertise are not possible to replicate on device, where practically speaking you would need orders of magnitude decreases in size. However, it is possible to have a very sophisticated domain-dependent system that runs in as little as 35MB, including wake word, speech-to-text, and NLU.</p><p>Many people don’t realize this is possible and are amazed when they see Sensory’s demonstrations of what can be done with minimal memory on an old Android phone.</p><p>Let’s take a look at what works well in this small platform and where sacrifices need to be made (creating our challenges for tomorrow!).</p><h3>The Great Small Stuff</h3><p>- Wake word. Sensory’s <a href="https://rain.agency/raindrops/wake-word-resurgence-upon-us">wake word</a> technology is the best in the world, is available in any language, is supported by all the most popular platforms (OS’s and Chips/DSPs) and can range in size from 20 KB to 1 MB.</p><p>- Speech-to-Text. Sensory’s end-to-end acoustic models can be done with as little as 15 MB of memory. Domain accuracy can be added with small language models that add a small amount of memory.</p><p>- NLU and Actions. Intents and entities can be pulled out using a tiny domain-specific LLM that’s as small as 15MB. We can even accept multiple intents in a single phrase. A simple lookup table to assign results to actions is added requiring very little additional memory</p><p>- No Hallucinations! One of the nice things about going small with an edge voice assistant is that responses tend to be more premeditated so hallucinations can be completely avoided</p><p>As an example in <a href="https://youtu.be/YWI9drN41iU">this video</a>, Sensory uses a 16MB acoustic model, a 2 MB language model, and a 15 MB NLU model with a less than 1 MB wake word! The total memory is about 34 MB, yet it can recognize an unlimited number of words and sentences with intents and entities only restricted by the domain of use.</p><h3>The Challenges of Being Small</h3><p>- Broad expertise. The above approach to a domain-specific voice assistant works great for a specific domain of expertise, but if someone speaks out of domain, it will require a different model to respond intelligently and informatively. This could be a cloud based ChatGPT/Gemini/Llama type solution or a smaller on-device model that still requires many hundreds of MB of memory.</p><p>- Dialog and memory. The smaller domain-specific models can handle simple responses for confirmations but cannot carry on a dialog with memories going back to much earlier sentences.</p><p>Giant models and tiny models each have their own unique advantages. If you are thinking of building an edge-based voice assistant or just need a great technology like a wake word or speech-to-text that’s tiny and accurate, then think <a href="http://sensory.com/">Sensory</a>!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3d01caa01ce6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/sensory-perspectives/is-the-bigger-the-better-true-for-llms-how-small-can-they-get-3d01caa01ce6">Is “the bigger the better” true for LLMs? How small can they get?</a> was originally published in <a href="https://medium.com/sensory-perspectives">Sensory Perspectives on AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Many Voice Assistants Do We Really Need?]]></title>
            <link>https://medium.com/sensory-perspectives/how-many-voice-assistants-do-we-really-need-38f619e6765b?source=rss----0d6bb4fd552d---4</link>
            <guid isPermaLink="false">https://medium.com/p/38f619e6765b</guid>
            <category><![CDATA[voice-ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[voice-assistant]]></category>
            <category><![CDATA[edge-computing]]></category>
            <category><![CDATA[wake-word]]></category>
            <dc:creator><![CDATA[Todd Mozer's Desk]]></dc:creator>
            <pubDate>Tue, 18 Jun 2024 11:06:06 GMT</pubDate>
            <atom:updated>2024-06-18T11:06:06.613Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x-X59s5MzxA07ruvtidrAA.png" /><figcaption>Virtual assistant concept image generated by DALL·E (OpenAI) edited by A.Adeboje</figcaption></figure><p>That’s easy…Just one. One that knows all the current knowledge of the internet, doesn’t hallucinate, knows everything about me, can carry out actions on my behalf, answers to my voice with my wakeword, and runs on device so there’s no risk in my private data leaking out. I think it’s kind of what Apple Siri is striving for with on device LLM’s and Apple Intelligence, but they aren’t there yet.</p><p>Nobody is there yet. In the next 5 or so years we will likely need multiple assistants. The assistants can have different domains of expertise and by staying focused each one can be smaller and more intelligent within its domain. By being smaller it can run on device, so the most private voice assistants will have these more targeted capabilities. For example, a medical assistant can know all my personal health information and history and is a health expert, so it can intelligently keep up on all the latest studies and provide great targeted advice. I could have a separate car assistant that runs without internet and can route to “Grandpa’s house” and knows that when I look for “nearby restaurants” I’m looking for good-tasting vegan food. It knows me!</p><p>Each assistant can have its own name, in fact, Sensory can run multiple wake words in parallel. We can listen for Google, Alexa, Cortana, Siri or others all at the same time. We can personalize these by making them only respond to the right user through the power of speaker verification. Sensory calls this an Enrolled Fixed Trigger (wake word).</p><p>Most importantly, the technology to name your own assistant is here today! Sensory calls it “<a href="https://www.sensory.com/wake-word/">User-Defined Triggers</a>”. You say your wakeup phrase a few times and the Sensory tech stack learns how you say it and enables a tiny low-power recognition of your voice on many dozens of different chips and platforms and in any language, all in an ultra-tiny low heat, low MIPs package!</p><p>In the short run, we will probably be calling out to a few different Voice Assistants, maybe we’ll soon have a master assistant that you get to name and that performs cognitive arbitration to figure out which domain specific models it needs engage to answer questions as accurately as possible with the privacy levels and connectivity you want!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=38f619e6765b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/sensory-perspectives/how-many-voice-assistants-do-we-really-need-38f619e6765b">How Many Voice Assistants Do We Really Need?</a> was originally published in <a href="https://medium.com/sensory-perspectives">Sensory Perspectives on AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Talking to Your LLM Voice Assistant]]></title>
            <link>https://medium.com/sensory-perspectives/talking-to-your-llm-voice-assistant-5dc8e17b837d?source=rss----0d6bb4fd552d---4</link>
            <guid isPermaLink="false">https://medium.com/p/5dc8e17b837d</guid>
            <category><![CDATA[speech-recognition]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[voice-assistant]]></category>
            <dc:creator><![CDATA[Todd Mozer's Desk]]></dc:creator>
            <pubDate>Fri, 31 May 2024 14:05:02 GMT</pubDate>
            <atom:updated>2024-05-31T14:05:02.172Z</atom:updated>
            <content:encoded><![CDATA[<p>Large language models (LLMs) have advanced remarkably, making it inevitable that we will soon interact with our personal voice assistants that are knowledgeable about everything and able to carry out actions for us as well. Basically capable of performing almost any task a super smart helper can handle — from vacation planning, tour guide, education, entertainment, shopping and more!</p><p>Many of the current challenges with LLMs are rapidly being resolved. Newer models like GPT4o can combine vision with speech for enhanced intelligence. The models are being updated more frequently, reducing outdated information, and the updates are happening faster by breaking large models into submodels that can be independently controlled. Issues like hallucinations can be cross-checked for accuracy and minimized through source citing.</p><p>However, a significant problem remains unaddressed. They can’t be always on and always listening and watching and sending data to the Cloud based LLM. It’s too much bandwidth, destroys privacy, and would lead to a lot of unintended interruptions. Today, it can be thought of more generally as how we get an assistant’s attention without being interrupted when we don’t want it and in a way that preserves privacy and conserves energy.</p><p><strong>Full Disclosure</strong>. I started an on device neural net speech recognition company called Sensory many years ago. We specialize in high-accuracy on-device speech recognition including wake words. Many credit Sensory with creating the first usable wake words. Sensory is likely the only company that has licensed wake word technology to major players like Google, Amazon, Microsoft, Cupertino, Samsung, Huawei, and Baidu, and for sure, we are the only company that has licensed all of these companies and hundreds of others. Sensory technology has shipped in over 3 billion products and many of these used our wake words. If you’ve used speech recognition, you’ve probably used Sensory technology — most likely in a wake word. So, I have a bias for wake words!</p><h3>Current Approaches to talking to Voice Assistants</h3><p>Many people want wake words to disappear, envisioning a smart assistant that “knows” when you’re talking to it. This might be ideal but it’s unlikely. Instead, voice assistants will feature a hybrid approach that combines wakewords with on-device vision, touch and various other low power sensors to detect people, noise and environments.</p><p>Let’s examine some of the current approaches:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/752/1*Cfvak7PCOxbm5cooP8qIFQ.jpeg" /></figure><ol><li><strong>Traditional wake words</strong> (e.g., Alexa, Google, Siri): These assistants typically have a low-power listening device to conserve energy, with revalidation on the device or in the cloud to improve accuracy. For instance, an ultra-low-power wake word for Alexa can always be on and running below the OS level on chips to conserve power. Secondary and final checks ensure the word was spoken correctly and understand the intent. This approach works okay but not great. Users can experience false accepts (unwanted activations) and missed activations in noisy environments.</li><li><strong>Push-to-Talk Assistants</strong>: Devices like the Rabbit R1 and Humane pin use a push-to-talk method reminiscent of walkie-talkies. This approach eliminates the technical challenge of speech detection and keeps power consumption low. However, it is impractical when hands are busy or when the device is not easily reachable.</li><li><strong>LLM Smart Assistants of Today</strong>. Some of the nicer new voice assistants use a button to turn “always on”. By this I don’t mean the low power always on listening for a wakeword. I also don’t mean the button press to quickly capture a command. I mean always on and connected to the cloud LLM analyzing everything it hears (and sees in some cases).</li></ol><p>My favorite voice assistant in this category is Pi by Inflection. It’s an amazing product that always listens. It’s ability to understand intent is so good that it kind of works to be always listening without a wakeword even if it consumes way too much power and bandwidth. But I do mean kind of works…I find it’s great when I’m alone, but when it’s noisy or if I have a few friends with me, it becomes too interrupting and seems to want to join the conversation (which it is amazingly good at even if I don’t want it to).I also lose a sense of privacy, unless I use the push to turn on and off with each query.</p><p>The recent Open AI demo of <a href="https://openai.com/index/hello-gpt-4o/">OpenAI’s GPT 4o</a> was really amazing, but I did notice a few interesting issues related to waking up to point out:</p><p>a) The barge-in ability seemed really impressive, and it works great if you want to barge in, but it seemed to be affected by background noise and chatter even though it was in a controlled environment.</p><p>b) They used button presses to turn it on and off so it wasn’t overly interrupting. You had to watch carefully to see them do this.</p><p>c) Sometimes they said “Hey ChatGPT” or “So ChatGPT”…my guess is this was just a natural way to talk to it, and it wasn’t really being used as a wakeword. If it was a wake word, then it was probably built into their cloud based NLU and not a low power on device implementation. Interestingly, I often start my conversation with Pi by saying Hi Pi, even though I know I don’t need to say it!</p><p>d) They didn’t combine the camera with the voice intent to know who was speaking to who…this will come in a future generation!</p><p>e) At one point they tried to turn it off to talk amongst themselves and they must have missed the off button because Chat GPT started talking about the clothes they were wearing and this quickly got manually shut down.</p><h3>Future Approaches in talking to Voice Assistants</h3><p>The art of creating a device that you can talk to whenever you want — without interruptions or the need for button presses, while also ensuring privacy and low power consumption — is extremely complex and has not yet been mastered. Such a device requires several key components:</p><p>1) Better NLU…to identify who is the intended recipient of the conversation.</p><p>2) Better knowledge of the room, people and noise environment. In a quiet room with one person a wake word or button press isn’t even needed.</p><p>3) Cameras, beamforming and various approaches to estimate the intended recipient of speech through directionality. Of course something like this needs to be probabilistically weighted as I might want to talk to a device without facing it!</p><p>4) Wake words. These aren’t going away, and for sure they are the simplest and most useful proven approach of today. They will be deployed AT TIMES in every conversational assistant product of tomorrow. For power and privacy reasons, the wake words should run low power and on device.</p><p>The future Voice Assistants will not have a single approach to waking up. Wakeup will be deployed through a probabilistic deep learned and trained scheme that intelligently shifts between different wakeup modes. They all have a place and the most intelligent of products will deploy them probabilistically based on the scenarios of use! Just like we humans do! Sometimes we touch, or wave, or we direct our gaze, or we call out a name!</p><p>Sensory has technologies to enable end user defined wake words (you choose the name), and an amazing tool called <a href="https://www.sensory.com/voicehub/">VoiceHub</a> that creates high accuracy low power wake words on the fly in any size, any platform and almost any language! If you have read this article and want to deploy a wake word into your conversational LLM, I will offer the first 10 responses a free wake word development on <a href="https://share.hsforms.com/1jzUUlZHHTxq7PdHhsGMiPw3uxhk">VoiceHub</a> with a free year of usage!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5dc8e17b837d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/sensory-perspectives/talking-to-your-llm-voice-assistant-5dc8e17b837d">Talking to Your LLM Voice Assistant</a> was originally published in <a href="https://medium.com/sensory-perspectives">Sensory Perspectives on AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>