<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[ML Brew - Medium]]></title>
        <description><![CDATA[This publication is devoted to ML and DL research community to gather the knowledge at one platform for cutting edge innovation. - Medium]]></description>
        <link>https://medium.com/ml-brew?source=rss----88eb191c508c---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>ML Brew - Medium</title>
            <link>https://medium.com/ml-brew?source=rss----88eb191c508c---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 17:52:23 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/ml-brew" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Weak Speech Supervision (WSS): Make Your Small Data Speaks]]></title>
            <link>https://medium.com/ml-brew/weak-speech-supervision-wss-make-your-small-data-speaks-d696b278cdf6?source=rss----88eb191c508c---4</link>
            <guid isPermaLink="false">https://medium.com/p/d696b278cdf6</guid>
            <category><![CDATA[speech-pathology]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[weak-supervision]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Mirali Purohit]]></dc:creator>
            <pubDate>Sun, 15 Aug 2021 05:22:48 GMT</pubDate>
            <atom:updated>2021-08-15T05:22:48.094Z</atom:updated>
            <content:encoded><![CDATA[<h4>Build Your Deep Learning Model More Powerful</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Rc2auuZbNWDgMaGDgwpKog.png" /></figure><p>In the era of Artificial Intelligence (AI), machines have become smarter. In the past few years, Deep Learning (DL) techniques are providing state-of-the-art results in many fields such as Computer Vision (CV), Natural Language Processing (NLP), Speech, Space Technologies, and many more. Over the period of advancement in the DL field, large data is always a bottleneck in getting the expected performance of models. Craig Mundie writes:</p><blockquote><strong>“Data are becoming the new raw material of business.”</strong></blockquote><p>Data speaks always a great story and gives tremendous information. This blog discusses the way to handle the large data requirement problem and improve the performance of the models in the domain of speech. It talks about an amazing methodology introduced to tackle the data shortage problem: <em>Weak Speech Supervision (WSS). </em>In this blog, we introduced one case study, “Dysarthria Severity Classification” to show the efficiency of the proposed approach.</p><h3>Key Terminologies</h3><p>Some terminologies that you need to go through before reading the blog. This helps you to understand the rest of the blog in a good way. If you already know these terms, you can skip this part of the blog.</p><ul><li><strong>Dysarthria</strong></li></ul><p>One of the greatest blessings we have is speaking ability; however, it is ignored by most of us. However, some people cannot speak fluently or produce speech in normal mode. <em>Dysarthria </em>is a speech disorder in which a person loses the natural way of speaking. Dysarthria caused by a disease, which results in that person is not able to move some articulatory parts (e.g., tongue, lips, jaw, etc.) which are used in speech production. Symptoms of dysarthria are less intelligible speech, uneven rate of speech, slurred speech, speech in an abnormal rhythm, speak slowly, talk too fast, speak softly, have a change in voice, sound hoarse or breath, etc. You can check few examples of dysarthria at the following link:</p><p><a href="https://sites.google.com/view/dysarthria-audios/home">Dysarthria</a></p><ul><li><strong>Severity</strong></li></ul><p>The <em>severity</em> of dysarthria was rated on a coarse scale ranging from none, mild, moderate to severe based on various factors such as degradation in the speech, condition of the patient, etc. This categorization is done by speech pathologists based on the severity of dysarthria.</p><ul><li><strong>Data Scarcity</strong></li></ul><p>In the recent era, DL techniques are providing promising results in many areas. Also, these methods outperform any existing method or human in some fields. With this exceptionally significant achievement, a large amount of data is the hurdle in building any DL-based systems, and this problem is known as <em>Data Scarcity.</em> In short, any DL model needs a huge amount of data to get high performance.</p><ul><li><strong>Weak Supervision</strong></li></ul><p><em>Weak Supervision</em> is the method to generate data with the help of an existing small dataset, and this generated data along with existing original data can be used to train the DL architectures and to improve the performance of the system with more data. Weak supervision gets too much attention in the last decade due to its promising results. To explore more about weak supervision, read this:</p><p><a href="https://dawn.cs.stanford.edu/2017/07/16/weak-supervision/">Weak Supervision: The New Programming Paradigm for Machine Learning</a></p><h3>Problem</h3><p>As dysarthria is a speech disorder, people struggle when they speak and face difficulties when they speak for a longer duration. Because of this, we cannot collect more data and hence there is data scarcity. These people cannot use the Intelligent Personal Assistants (IPAs), i.e., Google Assistant, Amazon Alexa, etc. because currently, these systems work efficiently when speech is in natural mode. Also, knowing the severity of the speech in advance can help in improving the speech, and it can improve the overall performance IPAs. Additionally, severity level can help the speech pathologists in the patient’s treatment. Hence, classification and improvement of dysarthric speech is a very important task.</p><h3>Approach</h3><p>In this research work, the <strong><em>Weak Speech supervision (WSS) </em></strong>technique is introduced to solve the data scarcity problem for dysarthria severity classification. Now, the rest of the blog discusses the DL-based approach in depth.</p><p>Let’s assume we have labeled (i.e., trusted/original)(of size N1) and unlabeled (i.e., untrusted)(of size N2) data of dysarthric speech, where N2 &gt;&gt; N1. We want to build a classifier that classifies dysarthric speech based on severity-level. In this research work, dysarthric speech is classified into two classes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gA0BpPVflSg2-M48UIa1zQ.png" /></figure><p>where x is the feature of the dysarthric speech and y is the corresponding label.</p><p>The WSS technique involves two steps:</p><blockquote>1) generation of the weak data,</blockquote><blockquote>2) utilize weak data along with trusted data to train classifiers.</blockquote><p>You can see the whole process in the following schematic diagram:</p><figure><img alt="Flowchart of WSS for Dysarthria Severity Classification" src="https://cdn-images-1.medium.com/max/734/1*suw2N5wZAoav5i4-RJWiOg.png" /></figure><ol><li><em>Generation of weak data</em> — To generate the weak data, a weak rule needs to be defined. The weak rule can be extracted by observing the patterns, some common behaviour, external knowledge, etc. on the trusted data. In this case, the unlabeled dataset is labeled using weak rules. Based on the severity, several speech parameters also change for dysarthric speech, e.g., speech rate, F0, pitch, voiced-unvoiced segments, energy, power, etc. It is observed that energy changes with the severity-level of dysarthric speech. Hence, we have used the <em>energy</em>-based parameter to define the weak rule. After extracting the energy parameter, we fix the threshold to define the weak rule and label the unlabeled data.</li><li><em>Utilize weak data along with trusted data to train classifiers — </em>As shown in the figure, classifiers are trained, one on trusted data (C1) and another on weak data (C2). For both the classifiers, a different or the same model can be selected. The objective/loss functions are selected accordingly for both the classifiers.</li></ol><h4><strong>Results</strong></h4><p>We have done 3 different experiments by changing the ratio of trusted and weak data. Along with this, we have trained our classifier on trusted data only as a baseline. We got on an average 35.68% and 43.83% relative improvement in terms of accuracy and F1-score w.r.t. baselines, respectively.</p><blockquote>You can check details about experiments, results and our research in the following link:</blockquote><p><a href="https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9287502&amp;casa_token=NWw77oxggCUAAAAA:Xn0r22q30G8RqjpeEjLVF6stOCXlfz3zxmRLBnGdQnvjGMPiOp4nreGJxycffJX3pfczNavPm7PD&amp;tag=1">Weak Speech Supervision: A case study of Dysarthria Severity Classification</a></p><blockquote><em>You can see the implementation of our work on below GitHub link:</em></blockquote><p><a href="https://github.com/Mihir3009/Weak-Speech-Supervision">GitHub - Mihir3009/Weak-Speech-Supervision: A first-of-its-kind system in speech domain that helps users to train state-of-the-art classification models without hand-labeling training data.</a></p><p><strong>Conclusions</strong></p><ul><li>A huge amount of data is the largest bottleneck in building any DL-based system to get the desired result.</li><li>For supervised learning problems, getting enough labeled data for Neural Network (NN)-based architectures is not a feasible task because data hand-labeling is laborious and it requires a human annotation which is time-consuming.</li><li>WSS is the technique that provides an approach to generate more speech data to train NN-based models along with trusted data.</li><li>WSS significantly improves the performance via adding weakly labeled data and introduced a new training paradigm by utilizing it for the first time in the speech domain.</li><li>This study provides a scope of improvements in terms of devising different training methods to utilize weak data and <em>extend WSS in different domains of speech classification.</em></li></ul><blockquote><strong>“Data is the sword of the twenty-first century, those who wield it well, the samurai.”</strong></blockquote><blockquote><strong>~Jonathan Rosenberg, former Googler</strong></blockquote><p>So, we have discussed how <em>WSS </em>can help in the data scarcity problem through a case study of dysarthria severity classification. If you are interested more in Dysarthria check this:</p><p>In the recent era, people use IPAs to a great extent because the use of these systems is easy, and it makes life simpler. However, as we have seen people with dysarthria cannot use these systems efficiently. And for that industries like Google, Amazon, Microsoft, etc. are working to improve these kinds of systems toward dysarthric speech. In 2019, Google has started a project called <em>Euphonia</em> to help people with dysarthria so that they can use the latest speech technologies efficiently.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FOAdegPmkK-o%3Fstart%3D2%26feature%3Doembed%26start%3D2&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DOAdegPmkK-o&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FOAdegPmkK-o%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/22f2d762125585800bcd0a94e147e1fe/href">https://medium.com/media/22f2d762125585800bcd0a94e147e1fe/href</a></iframe><p>If you want to get more updates on ongoing cutting-edge research in the field of applied ML and DL, please follow our publication:</p><p><a href="https://medium.com/ml-brew">ML Brew</a></p><p>Contact us if you want to write something in our publication.</p><h4>Editors of the ML Brew:</h4><p><a href="https://medium.com/@freethinker.technophile">Mihir Parmar</a> — <a href="https://twitter.com/Mihir3009">Twitter</a>, <a href="https://www.linkedin.com/in/mihir-parmar-b44003157/">Linkedin</a>, <a href="https://scholar.google.com/citations?user=2UPwJC4AAAAJ&amp;hl=en">Google Scholar</a>, <a href="https://github.com/mihir3009">Github</a></p><p><a href="https://medium.com/@purohitmirali33">Mirali Purohit</a> — <a href="https://www.linkedin.com/in/mirali-purohit/">Linkedin</a>, <a href="https://scholar.google.com/citations?user=hUF42PEAAAAJ&amp;hl=en">Google Scholar</a>, <a href="https://github.com/Mirzyaaliii">Github</a></p><p><em>Special note</em>— Thanks to Mihir for his equal contribution in carrying out this profound research along with me.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d696b278cdf6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ml-brew/weak-speech-supervision-wss-make-your-small-data-speaks-d696b278cdf6">Weak Speech Supervision (WSS): Make Your Small Data Speaks</a> was originally published in <a href="https://medium.com/ml-brew">ML Brew</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Genetic Algorithms: Biologically-Inspired Deep Learning Optimization]]></title>
            <link>https://medium.com/ml-brew/genetic-algorithms-biologically-inspired-deep-learning-optimization-e4125e04053?source=rss----88eb191c508c---4</link>
            <guid isPermaLink="false">https://medium.com/p/e4125e04053</guid>
            <category><![CDATA[biocomputing]]></category>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[evolutionary-algorithms]]></category>
            <category><![CDATA[genetic-algorithm]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Mihir Parmar]]></dc:creator>
            <pubDate>Tue, 16 Jun 2020 23:02:00 GMT</pubDate>
            <atom:updated>2020-06-16T23:02:00.043Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*qQDNsu1Y1yPeykCCCiCPxg.png" /></figure><p>Recently, there have been significant research advancements in the field of neuroscience, biocomputation, and psychology related to how human-mind process information. In recent years, there has been significant progress in the research to apply bio-inspired techniques in various areas of computer science. The biological explanation of different phenomena has become the centre attraction in building various computational model. Nobel Laureate and Caltech President David Baltimore commented:</p><blockquote>“Biology is today an information science. The output of the system, the mechanics of life, are encoded in a digital medium and read out by a series of reading heads. Biology is no longer solely the province of the small laboratory. Contributions come from many directions.”</blockquote><p>With the use of bio-inspired computing in the field of Artificial Intelligence, the definition of computer science is changed. Computation is everywhere. Peter J. Denning commented this in one of his papers:</p><blockquote>The old definition of computer science — the study of phenomena surrounding computers — is now obsolete. Computing is the study of natural and artificial information processes.</blockquote><p>In the domain of Artificial Intelligence, evolutionary algorithms are playing a vital role in optimization problems. Particularly, genetic algorithms are making optimization easy. You can read more about <a href="https://medium.com/cracking-the-data-science-interview/an-introduction-to-optimization-in-intelligent-systems-c1aa408d3ac2">optimization and local search here</a>.</p><p>This blog will give you the basic intuitive ideas about genetic algorithms as an optimization technique in the context of deep learning. Here, we presumed that reader has a good background in Machine or Deep Learning domain.</p><h3>Optimization Problem</h3><p>A general idea of optimization comes from mathematics that selecting the best element from a set of available alternatives. Given a function <em>f(x)</em>, optimization algorithms try to minimize or maximize this function. In the context of deep learning, we can use cost function as <em>f(x)</em> and try to optimize this cost function with well-known algorithms like gradient descent or adam optimization. The basic idea of these algorithms is to find a set of parameters (i.e., weights and bias) and hyperparameters for minimum or maximum of the cost function. However, optimization is an iterative, time-consuming, and laborious task. Machine Learning or Deep Learning mechanisms are a disaster without better optimization.</p><h3>Genetic Algorithms</h3><p>The core idea behind the genetic algorithm is “Evolution”. Evolution is a biological phenomena observed in nature. Based on the evolution of biological species, this theory leads us to one fascinating optimization algorithm called evolutionary algorithms, and genetic algorithms (GAs) are part of this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/792/1*mbL6SPfQkKmBME9UjXCaiw.png" /></figure><p>Before understanding GAs, we need to clear some basic terminology used in explaining the process of GAs.</p><h4>Basic Terms</h4><ul><li><strong>Individual</strong> — Any possible solution.</li><li><strong>Population</strong> — Group of all individuals.</li><li><strong>Generation</strong> — different populations over iterations</li><li><strong>Chromosome</strong> — Blueprint for an individual.</li><li><strong>Search Space</strong> — All possible solutions to the problem.</li><li><strong>Offspring</strong> — generated child which inherit the characteristics of the parents</li><li><strong>Fitness</strong> — how fit an individual is (the ability of an individual to compete with other individuals)</li></ul><p>The basic idea of GA is to produce the best offspring over different generations via minimizing or maximizing fitness function. GA automatically drives stable solution for equations through evolution, which takes place over a series of discrete generations. Each generation consists of multiple individuals. The GA consists of three major steps to select the best offspring: 1) Variation, 2) Evaluation, and 3) Selection.</p><h4>Variation</h4><p>This process is similar to sexual reproduction. Given an individual in the population, we utilize the variation operators to produce its offspring. Specifically, several copies of each individual or parent are created, each of these is modified by different crossover or mutations. In a crossover, more than one parent is selected, and one or more off-springs are produced using the genetic material of the parents. In simple terms, mutation may be defined as a small random tweak in the chromosome, to get a new solution. You can read in detail about <a href="https://www.tutorialspoint.com/genetic_algorithms/genetic_algorithms_crossover.htm">crossover</a> and <a href="https://www.tutorialspoint.com/genetic_algorithms/genetic_algorithms_mutation.htm">mutation</a> here. Then, each modified copy is regarded as one child.</p><h4>Evaluation</h4><p>In a GA, evaluation is the operation of measuring the quality (performance) of individuals. For each child, its performance is evaluated by a fitness function. The definition of fitness function varies from problem to problem.</p><h4>Selection</h4><p>This is a very crucial part of the process of GA. The key idea behind the selection is to allow only the fittest offspring for the next production (variation). All children will be selected according to their fitness value, and the worst part is removed means they are killed. The rest remain alive offspring and they evolve to the next iteration (generation).</p><p>The overall idea of GA is to evolve the population of individual solutions towards better solutions to an optimization problem. This idea is helpful in solving the traditional deep learning optimization problems. Here, I will provide an intuitive approach to this.</p><h3>GA for Deep Learning Optimization</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*B7Tiw4OKKWvXOnwywoC9VA.png" /></figure><p>The idea behind deep learning optimization is to find the maximum or minimum of the cost function for the best selection of parameters and hyperparameters. To solve this optimization problem, we can devise GA. In this GA, randomly generated weights and bias are considered as an individual. Via using randomization, we will generate different individuals and create a population. For variation, we will use the mutation operator, which consists of different cost functions such as Binary Cross-Entropy (BCE), Mean Square Error (MSE), etc. We will now produce different offspring using these mutation operators. After this, we calculate fitness corresponding to each child. In the deep learning problems, we always minimize the cost function; hence, we will select the offspring with maximum fitness that helps to reduce cost function. We iterate this GA for a number of generation and select the best offspring that can give the optimized output for deep learning architecture.</p><p>Remember that the effectiveness of your GA depends on the selection of fitness function.</p><blockquote>My next blog will give you some ideas about how to select fitness function, and also provide the implementation of GA for one optimization problem.</blockquote><p>Please check out my previous blog on <a href="https://medium.com/@freethinker.technophile/machine-learning-voice-assistants-can-start-responding-to-whisper-b443ba5c4cc4">“Machine Learning: Voice assistants can start responding to Whisper”</a>.</p><p><em>If you would like to follow my work on Computer Science and Intelligent Systems, you can check out my </em><a href="https://medium.com/@freethinker.technophile"><em>Medium</em></a><em> and </em><a href="https://github.com/mihir3009"><em>GitHub</em></a><em>, as well as other research contributions at</em> <a href="https://sites.google.com/view/mihir3009">my website</a><em>. You can also tweet at me on </em><a href="https://twitter.com/Mihir3009"><em>Twitter</em></a><em>, </em><a href="/service/mailto:mihir.daiict@gmail.com"><em>email me directly</em></a><em>, or </em><a href="https://www.linkedin.com/in/mihir-parmar-b44003157/"><em>find me on LinkedIn</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e4125e04053" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ml-brew/genetic-algorithms-biologically-inspired-deep-learning-optimization-e4125e04053">Genetic Algorithms: Biologically-Inspired Deep Learning Optimization</a> was originally published in <a href="https://medium.com/ml-brew">ML Brew</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machine Learning: Voice assistants can start responding to Whisper]]></title>
            <link>https://medium.com/ml-brew/machine-learning-voice-assistants-can-start-responding-to-whisper-b443ba5c4cc4?source=rss----88eb191c508c---4</link>
            <guid isPermaLink="false">https://medium.com/p/b443ba5c4cc4</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[speech-pathology]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[speech-technology]]></category>
            <dc:creator><![CDATA[Mihir Parmar]]></dc:creator>
            <pubDate>Tue, 16 Jun 2020 22:58:41 GMT</pubDate>
            <atom:updated>2019-08-08T08:45:23.696Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2qiZ8nC0sJGxkyzMVE6Tuw.jpeg" /></figure><p>Speech technology is a type of communication technology that enables electronic devices to recognize, analyze and understand spoken word or audio. Subfields of speech technology include speech processing, and its applications like speech recognition, speech verification, Voice Conversion (VC), real-time speech to text conversion, interactive voice response (IVR), speech synthesis and speech analytics.</p><blockquote><strong>Speech is more than just a signal.</strong></blockquote><p>Speech signal contains linguistic information with many other prosodic elements like pitch, intonation, emotion, accent, etc. Due to the fact that speech is a primary form of communication, the growth of speech technology is an important step towards harnessing unstructured voice data. To harness the power of speech, and ameliorate lives with speech technology, Machine Learning (ML) plays a vital role in developing the many speech processing algorithms. With the emergence of <strong>Artificial Intelligence (AI) </strong>solutions for enterprises, speech technology has many applications in all sectors, including law, healthcare, security, finance, enterprise, and personal use. Personal use voice assistants such as Siri, Google Home, and Amazon Alexa are devices that offer individualized speech technology experiences. These Intelligent Personal Assistants (IPAs) made our day-to-day lives easy and more comfortable. Speech processing and Natural Language Processing (NLP) using ML allow intelligent devices, such as smartphones, to interact with users via verbal language. Big companies like Apple, Google, Amazon, Microsoft are doing a remarkable job to improve the voice assistants. However, these IPAs currently are facing the problem in recognizing whispered speech. We tried to solve this problem to some extent using ML-based algorithms. Our research work:</p><blockquote>“Novel MMSE DiscoGAN for Cross-Domain Whisper-to-Speech Conversion”, published in Machine Learning in Speech and Language Processing<strong> (MLSLP), Google</strong>, Hyderabad, India, September 7, 2018. (<a href="https://drive.google.com/file/d/1UVbXRzpaM1_ayaTfvq92RVijn_M8FLnn/view"><strong><em>See our research work here</em></strong></a>).</blockquote><p>In my point of view, Machine Learning is all about the manipulating data in such a way that the machine can learn what we want. To apply ML-based algorithms, understanding regarding the nature of data is most important. If you understand the data, you can do miracles. In enterprises, applications of products increase the excitement level towards solving the problems using ML. First, I talked about the applications of WHiSPer-to-SPeeCH (WHSP2SPCH) conversion in this blog. Second, I focused on the nature of the data we have. That is about understanding the properties of whispered and normal speeches. Then, I talked about why and how we used the ML-based algorithm to solve this conversion problem.</p><h3>Applications</h3><p>Key reasons for whispering could be to make the conversation private, conversation in quiet environments like a library, a hospital, a meeting room, and for various forensic applications. In addition, people suffering from vocal fold-related diseases can sometimes produce whispered speech. In such cases, making the whispered speech more intelligible in order to improve the quality of communication (in person or over the telephone), WHSP2SPCH conversion technique is required. To make this happen using ML, we have to understand the properties of both the speeches.</p><h3>Whispered vs. Normal Speech</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/645/1*QZUvM4OlzZAaf2ZR0q8YXA.jpeg" /></figure><p>Though whispered and normal speech are the modes of communication, they are different w.r.t speech production and perception perspective. The differences between whispered and normal speech include the following:</p><ul><li>A complete absence of periodic excitation or harmonic structure in whispered speech;</li><li>A shift for the lower formant locations;</li><li>A change in the overall spectral slope (becomes flatter, with a reduction from the low-frequency region);</li><li>A shifting of the boundaries of vowel regions in the F1-F2 frequency space;</li><li>A change in both energy and duration characteristics.</li></ul><p>Noise excitation in the whispered speech is normally distributed across the lower portion of the vocal tract, that results in 20 dB reduction of the power than its equivalent normal speech. Moreover, it has also been observed that cortical hemodynamic response was more profound due to the weaker stimulus for the whispered speech than its normal speech counterpart. Technically, whispered speech is unvoiced or aperiodic. However, it has been observed that the sensation of the pitch still exists, which is encapsulated in an intricate way in the whispered speech. Hence, we predicted fundamental frequency (F0) from the cepstral features (i.e., <a href="https://ieeexplore.ieee.org/iel7/7537873/7550716/07550889.pdf"><strong><em>Mel Cepstral Coefficients</em></strong></a>) of the converted normal speech instead of predicting directly from the cepstral features of the whispered speech.</p><p>It is very natural for humans to acknowledge cross-domain relationships so easily due to their efficient perception mechanism. However, it is difficult for machines to achieve the same ability. In other words, finding a mapping function from one domain to the other can be thought as generating an image in one domain given another image in the other domain. Recently, Generative Adversarial Network (GAN)-based architectures have become more popular for discovering such cross-domain relationships. However, traditional GANs fail to perform efficiently when it comes to cross-domain relationships. This problem also brings an interesting challenge from a learning point of view in both domains, Computer Vision and Speech technology.</p><h3>Key Limitations and Solutions</h3><blockquote>The key limitation of the vanilla GAN-based system is in generating the samples that may not correspond to the given input. To address this issue, we recently proposed to use Mean Squared Error (MSE) as a regularizer to the vanilla GAN (i.e., MMSE-GAN) in <a href="https://www.isca-speech.org/archive/Interspeech_2018/pdfs/1565.pdf"><strong><em>this paper</em></strong></a>. Moreover, traditional GANs perform better when we use explicitaly paired data. Traditional GANs fail to maintain cross-domain relationships between data. To solve this problem, we need to change training method of traditional GANs. So, we propose to apply MMSE-GAN and its extension in the form of Discover GAN (i.e., <strong>MMSE DiscoGAN</strong>), to learn the cross-domain relations (w.r.t. attributes of the speech production mechanism) between the whispered and the normal speech.</blockquote><h3>Overview of the Algorithm</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/949/1*TAUkTdLsWiPV0BhS7fIXdw.jpeg" /><figcaption>WHSP2SPCH conversion algorithm</figcaption></figure><p>We have used in total 40 speakers’ whispered and its corresponding normal speech data from the two databases, namely, the <a href="http://chains.ucd.ie/ftpaccess.php"><strong><em>CHAracterizing INdividual Speakers (CHAINS) Speech Corpus</em></strong></a> and the <a href="https://www.researchgate.net/publication/271824674_The_EMG-UKA_Corpus_for_Electromyographic_Speech_Processing"><strong><em>Electromyographic (EMG)-UKA Trail corpus</em></strong></a>. From both the databases, total 1302 and 108 utterances have been taken for the training and the testing, respectively. <em>25</em>-D Mel Cepstral Coefficients (MCC) (including the <em>0</em>th coefficient) and <em>1</em>-D F0 per frame (with <em>25 </em>ms frame duration, and <em>5 </em>ms frameshift) have been extracted using <a href="https://aholab.ehu.es/ahocoder/info.html"><strong><em>AHOCODER</em></strong></a>. However, one of the main issue before learning the mapping function for the WHSP2SPCH conversion system is the time-alignment between whispered and its corresponding normal speech. To that effect, we have used the <a href="http://web.science.mq.edu.au/~cassidy/comp449/html/ch11s02.html"><strong><em>Dynamic Time Warping (DTW)</em></strong></a> algorithm. For more about architecture details, please refer to <a href="https://drive.google.com/file/d/1UVbXRzpaM1_ayaTfvq92RVijn_M8FLnn/view"><strong><em>our paper</em></strong></a>.</p><h3>MMSE DiscoGAN</h3><p>This architecture is used to learn 1) mapping between the cepstral features corresponding to the whispered (Xw) and the normal speech (Xs), and 2) the mapping between the converted cepstral features and the corresponding F0 of the normal speech.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/954/1*29UZsnMKX5f5JuMt5z1AAQ.png" /><figcaption>Discover Generative Adversarial Network (DiscoGAN). W: whispered speech, S: normal speech.</figcaption></figure><p>In particular, we extended MMSE-GAN via MMSE DiscoGAN by including two generators Gws and Gsw. Gws mainly converts the parameters correspond- ing to the whispered speech (Xw) into Xws = Gws(Xw), such that Xws is indistinguishable from the parameters of the normal speech (Xs). Our model also contains two discriminators, Dw and Ds. The discriminator Dw attempts to distinguish between Xw from distribution of whispered speech parameters (i.e., pw) and converted whispered speech parameters (i.e., Xsw = Gsw(Xs)) obtained by converting Xs from distribution of normal speech parameters (i.e., ps) via generator Gsw. Ds performs an analogous operation for the normal speech parameters (Xs). To map the whispered speech parameters to normal speech parameters, we rely on the regularized adversarial objective function using MSE loss and defined as:</p><h3>Adversarial Loss:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/843/1*NnFeoGrEGpE2zSdUO2lOVw.png" /><figcaption>L_G: Generator Loss, L_D: Discriminator Loss</figcaption></figure><p>Here, Gws, Gsw, Dw, and Ds must be jointly trained, with one significant modification of including two reconstruction losses Lw and Ls. This can be mathematically represented as:</p><h3>Reconstruction Loss:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/642/1*fFKlKEcoaDDP0IezqBznow.png" /><figcaption>L_W: loss for reconstructing whispered speech, L_S: loss for reconstructing normal speech</figcaption></figure><p>The minimization of the reconstruction loss, on the conversion of the whispered speech parameters into normal speech parameters, enforces the generated whispered speech parameters to be close to the original whispered speech parameters. The minimization of adversarial loss, on the conversion of whispered speech parameters into normal speech parameters, enforces the generated normal speech parameters to be close to the original normal speech parameters. These two properties are explored to encourage the one-to-one mapping between two domains. (<a href="https://drive.google.com/file/d/1UVbXRzpaM1_ayaTfvq92RVijn_M8FLnn/view"><strong><em>Detailed training process</em></strong></a>)</p><blockquote>You can see the implementation of our work on below GitHub link:</blockquote><p><a href="https://github.com/Mihir3011/whisper-to-speech">Mihir3011/whisper-to-speech</a></p><h3>Results</h3><p>We analyze MMSE DiscoGAN architecture in terms of objective and subjective results. Root Mean Square Error (RMSE) of log(F0) is used as an objective measure since the key goal is to accurately predict F0 from the whispered speech. The more detailed analysis presented in <a href="https://drive.google.com/file/d/1UVbXRzpaM1_ayaTfvq92RVijn_M8FLnn/view"><strong><em>our paper</em></strong></a>.</p><blockquote><strong>Subjective results are presented on the below site:</strong></blockquote><p><a href="https://sites.google.com/view/audiosamples">Audio Samples</a></p><h3>Final Comments</h3><ol><li>WHSP2SPCH conversion can make IPAs or voice assistants more efficient in terms of recognizing the whisper.</li><li>This idea ameliorates the lives of patients who suffer from whispered speech pathological problems.</li><li>This idea also provides a brief overview of using ML in the domain of speech technology.</li><li>In the future, the use of high-quality vocoders, such as WORLD or Wavenet can further improve the voice quality of converted speech signal.</li></ol><h3>Acknowledgments</h3><p>Special thanks to <strong>Kim et. al. </strong>for<strong> </strong>bringing the <a href="https://arxiv.org/abs/1703.05192"><strong><em>idea of DiscoGAN</em></strong></a> in the Computer Vision (CV) Domain. Thanks to <a href="https://medium.com/u/991272e72e68">Google</a> for giving us a platform in terms of <a href="https://sites.google.com/view/mlslp/home"><strong><em>Machine Learning in Speech and Language Processing (MLSLP)</em></strong></a> to publish our research work. Also, the author would like to thanks to the <a href="https://sites.google.com/site/hemantpatildaiict/speech-lab"><strong><em>Speech Research Lab, DA-IICT</em></strong></a>, Gandhinagar for giving the resources and venue to conduct the experiments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b443ba5c4cc4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ml-brew/machine-learning-voice-assistants-can-start-responding-to-whisper-b443ba5c4cc4">Machine Learning: Voice assistants can start responding to Whisper</a> was originally published in <a href="https://medium.com/ml-brew">ML Brew</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>