<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[cogito-engineering - Medium]]></title>
        <description><![CDATA[This “publication” space contains short, technical articles written by Cogito Engineers on developing software and systems. The space focuses on certain themes including real-time processing, cloud-native architectures, machine learning, among others. - Medium]]></description>
        <link>https://medium.com/cogito-engineering?source=rss----7d6ba2b042a9---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>cogito-engineering - Medium</title>
            <link>https://medium.com/cogito-engineering?source=rss----7d6ba2b042a9---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 06:33:16 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/cogito-engineering" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[ACL 2022 — the future of natural language processing]]></title>
            <link>https://medium.com/cogito-engineering/acl-2022-the-future-of-natural-language-processing-9303aa61cfec?source=rss----7d6ba2b042a9---4</link>
            <guid isPermaLink="false">https://medium.com/p/9303aa61cfec</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[computational-linguistics]]></category>
            <category><![CDATA[speech-technology]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[CogitoEngineering]]></dc:creator>
            <pubDate>Fri, 27 May 2022 16:17:39 GMT</pubDate>
            <atom:updated>2022-05-31T12:57:20.504Z</atom:updated>
            <content:encoded><![CDATA[<h3>ACL 2022 — the future of natural language processing</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QV1qJh0fsSL3FCmALgLuPA.jpeg" /><figcaption>ACL 2022 in Dublin (taken from <a href="https://www.2022.aclweb.org/">https://www.2022.aclweb.org/</a>)</figcaption></figure><p>With the Convention Centre only a moderate stone’s throw away from the Cogito Dublin office, it really would have been rude for us not to attend the 60th annual meeting of the Association for Computational Linguistics (ACL) 2022 conference. The field of computational linguistics and natural language processing continues to be the test bed for latest developments in deep neural network-based machine learning, and research in the area is thriving. We picked out some papers that stuck out for us at the conference, which should give you a flavor of where this scientific domain is currently at.</p><h3>Paper highlights</h3><h4>Cross-modal discrete representation learning</h4><p>The first paper we wanted to call out is from Jim Glass’s group at MIT. <a href="https://arxiv.org/abs/2106.05438">The paper</a> focuses on the important research focal point of multi-modal self-supervised learning. Recent work has shown the possibility to encode representations to do with multiple modalities (e.g., text and image) into a single vector which an be used for downstream classification tasks. Their method involves an embedding space used for combining modalities where the initial representation is discretized using vector quantization and recorded as entries in a discrete codebook. Besides demonstrating improved accuracy on some video + audio and video + text tasks, the authors highlight some interpretability benefits of using this codebook approach with semantically similar samples using the same parts of the codebook.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UGHj2_9s5ukQtC2267ZYuQ.jpeg" /><figcaption>Opening slide from the “Cross-modal discrete representation learning” talk</figcaption></figure><h4>Socially Aware NLP for Social Impact</h4><p>During the conference there was a session dubbed <a href="https://www.2022.aclweb.org/invited-talks">“Spotlight Talks for Young Rising Stars”</a> which was comprised of a set of invited talks by five important young researchers. The talk titled “Socially Aware NLP for Social Impact” by Diyi Yang was particularly interesting, and provided an overview of how to adopt a holistic, contextualized approach to developing natural language processing applications.</p><h4>A taxonomy of empathetic questions in social dialogs</h4><p>As transactional question-answering tasks for conversational chatbots become something of a solved problem, research is beginning to focus on the emotional perception of these interactions. <a href="https://aclanthology.org/2022.acl-long.211/">This paper</a>, authored by researchers from Lausanne in Switzerland, focuses on the development of a taxonomy for empathetic questions. Using the <a href="https://github.com/facebookresearch/EmpatheticDialogues">EmpatheticDialogues dataset</a>, the authors use human annotation, including crowdsourcing, to help construct and validate this set of question classes. A key finding from the research is that employing certain question classes in response to a certain emotional state of the user can be an effective means of regulating emotion in the interaction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Tfyp_eFXcjZMD0Y_ABk3Xg.jpeg" /><figcaption>Opening slide from the “A taxonomy of empathetic questions in social dialogs” talk</figcaption></figure><h4>∞ -former: Infinite Memory Transformer</h4><p>Transformers continue to grab the popular science headlines as they power new futuristic technology like the <a href="https://openai.com/blog/gpt-3-apps/">GPT-3</a> massive language model as well as text-to-image synthesis, like <a href="https://openai.com/blog/dall-e/">DALL-E</a> and Google’s new <a href="https://imagen.research.google/">Imagen</a>.</p><p>However, Transformers are still impractical for many applications, in particular owing to the self-attention mechanism and its computational complexity being quadratic with respect to the size of the input data. So basically if you have a real-time streaming use-case, or if your input data is very large, it can be very difficult to exploit the benefit of Transformers. A lot of recent research has focused on addressing this problem, see for instance Transformer XL, however despite this attention, there has not yet been convergence to an acceptable approach.</p><p><a href="https://arxiv.org/abs/2109.00301">The Infinite Memory Transformer paper</a>, by researchers from Google DeepMind and Unbabel, presents the latest technique to addressing this. With self-attention, key and value vectors are combined to regulate the amount of information passed through to the next layer of the network for each input sample. Rather than storing a buffer of these vectors to support incremental processing, the authors here store parameters of a continuous function for each element in the vectors. As newer samples are processed, the parameters for these functions are updated. Additionally, for samples where the attention weights are higher (i.e. potentially indicating more salient regions), an approach is used to ensure higher resolution in the stored functions. They refer to this weighting as “sticky memories”. The experiments presented highlight the potential of this approach for “remembering” important moments in very long sequences.</p><h4>On the robustness of offensive language classifiers</h4><p>Natural language processing plays an important role in modern social media platforms, in particular as a means to identify and regulate offensive language posts which may be racist or hateful. However, there is a challenging adversarial relationship between such classification models and creators of offensive content. As these users see what gets classified as offensive, they will react by modifying certain characters or words to get around the classification algorithm.</p><p>In the paper <a href="https://arxiv.org/pdf/2203.11331.pdf">“On the robustness of offensive language classifiers”</a> researchers from University of Iowa demonstrate just how significantly reduced the accuracy of such offensive language classifiers can be in the context of these adversarial attacks. The research also highlights some potential routes forward for mitigating this reduction in accuracy, including the use of adversarial training and data augmentation simulations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*92kIDdqn6atKJTUQU2RVlA.jpeg" /><figcaption>Opening slide from the “On the robustness of offensive language classifiers” talk</figcaption></figure><h4>Is Attention Explainable? An introduction to the debate</h4><p>Attention is of course the vital ingredient within Transformer layers, and is responsible for much of the advances in NLP, and other domains, in recent years. However, a secondary benefit that people often highlight with Attention is the potential for the attention weights to provide explanations or at least provide some additional information on what parts of the input data were important for making the model inference.</p><p>During this talk, researchers from the University of Louvain provided a survey of different perspectives on the debate around whether Attention can be used for explainability purposes or not. Previous papers, for instance <a href="https://arxiv.org/abs/1902.10186">Jain and Wallace in 2019</a>, provide strong claims that attention weights do not provide meaningful explanations and are often completely uncorrelated with feature importance. Other papers, like <a href="https://arxiv.org/abs/1908.04626">Wiegreffe and Pinter in 2019</a>, argue that although attention weights may not be considered <em>the</em> explanation, they can be considered to be <em>an</em> explanation.</p><p>In <a href="https://arxiv.org/abs/1908.04626">this ACL long paper</a>, the authors emphasize the importance of adopting well-defined concepts like “plausibility” (does a user consider the explanation of the model plausible) and “faithfulness” (how close the explanation is to the inner workings of the model) when assessing explainability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vc9o3P039QdwWvo0T8lSOw.jpeg" /><figcaption>Opening slide from the “Is Attention Explainable? An introduction to the debate” talk</figcaption></figure><h4>Learned Incremental Representations for Parsing</h4><p>Syntactic parsing is the task of assigning syntactic structure to a sentence. For example, we would like parsers to be able to resolve the ambiguity in the following sentence: “We saw the Eiffel Tower flying to Paris”. It is a fundamental NLP task that has been researched for decades and has many applications as a pre-processing step. Most methods that have been proposed are non-incremental. Previous methods for incremental parsing, so-called speculative methods, were inefficient. When ambiguity was encountered these incremental parsers would spin up multiple competing analyses and process them in parallel. In <a href="https://aclanthology.org/2022.acl-long.220">this paper</a> the authors tackle the parsing ambiguity issue by learning discrete tag representations (so-called supertags) that are learned end-to-end along with a neural tree converter that converts the tags to a tree structure. The authors show that they are able to achieve the same state-of-the-art results as non-incremental systems on the Penn Treebank dataset.</p><h4>Online Semantic Parsing for Latency Reduction in Task-Oriented Dialogue</h4><p>One of the main issues faced by task-oriented dialogue systems is that there is typically a lag between when the user finishes their utterance and when the system responds. For example, if the user says “Add pool party with Barack Obama and Joe for tomorrow at 9:00 AM”, under the hood, the system waits until the user is finished speaking, then sequentially executes searches <br>for “Barack Obama” and “Joe” within the user’s contact list, and finally creates an event “pool party” and adds them to it. This sequential execution is slow and leads to long latency times. In <a href="https://aclanthology.org/2022.acl-long.110">this paper</a> the authors propose a graph-based semantic parser that allows nodes of the semantic parse to be executed in parallel before the end of the utterance. So in the example, the searches for “Barack Obama” and “Joe” can be executed in parallel before the user has finished the utterance. The authors show that this can lead to much faster execution times, with latency reductions of between 30 % to 63 %.</p><h3>To more language technology conferences in Dublin!</h3><p>We were thrilled to have such an important language technology conference held in Dublin, in particular given the vibrancy of both academic and commercial work happening in this space here. Coming up next year<br>is the <a href="https://www.interspeech2023.org/">Interspeech 2023</a> conference which will help further highlight Dublin and Ireland as a core hub for speech and language technology research.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9303aa61cfec" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cogito-engineering/acl-2022-the-future-of-natural-language-processing-9303aa61cfec">ACL 2022 — the future of natural language processing</a> was originally published in <a href="https://medium.com/cogito-engineering">cogito-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Interspeech 2021 — The return of engaging, interactive speech conferences]]></title>
            <link>https://medium.com/cogito-engineering/interspeech-2021-the-return-of-engaging-interactive-speech-conferences-982cad437fba?source=rss----7d6ba2b042a9---4</link>
            <guid isPermaLink="false">https://medium.com/p/982cad437fba</guid>
            <category><![CDATA[speech-recognition]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[emotion-recognition]]></category>
            <category><![CDATA[signal-processing]]></category>
            <category><![CDATA[speech-technology]]></category>
            <dc:creator><![CDATA[CogitoEngineering]]></dc:creator>
            <pubDate>Tue, 07 Sep 2021 13:28:46 GMT</pubDate>
            <atom:updated>2021-09-07T13:28:45.250Z</atom:updated>
            <content:encoded><![CDATA[<h3>Interspeech 2021 — The return of engaging, interactive speech conferences</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ut-NXqbjknMMNtQ8YwW7VQ.png" /><figcaption>In-person and remote discussions during the first Survey Talk presentation</figcaption></figure><p>Relief! That sensation as we emerge from the COVID-19 pandemic when we rediscover experiences we previously held dear. That is what Interspeech 2021 felt like. It was a relief to see a large group of in-person attendees and a relief also to see such engagement in the live sessions, with consistently more questions than there was time for. Interspeech 2021 was a far cry from the purely virtual speech and language conferences we have attended over this past year and a half. Finally having people on the ground at the event, combined with an expertly executed hybrid platform and maybe just the fact that it is Interspeech, conspired together to produce this overall feeling of pure (deep exhale) relief.</p><p>The quality of keynote presentations was also exceptionally high this year. Professor Pascale Fung gave a broad overview of the state of the art in end-to-end conversational AI systems and talked through new approaches to handling six core challenges: (1) lack of diversity in automated responses, (2) lack of consistency between successive responses, (3) incorporating external knowledge bases, (4) lack of controllability, (5) lack of versatility and (6) how to train models using less data-intensive methods. She also discussed key ethical challenges including toxic, inappropriate and privacy-abusing responses by automated systems. Mounya Elhilali gave a fascinating talk on how neurons in the brain can “retune” spectrally in order to facilitate human performance in perceptual tasks, in particular in degraded acoustic conditions — and then went on to describe machine learning research which takes advantage of this new scientific knowledge. However, it was the final day keynote by Tomáš Mikolov which really stuck out for us. Mikolov, who is probably best known for his <a href="https://proceedings.neurips.cc/paper/2013/file/9aa42b31882ec039965f3c4923ce901b-Paper.pdf">2013 NeurIPS paper on word2vec</a>, gave an overview of the history of neural language models as well as some inspiring thoughts on his views for future directions in AI. He was clearly sceptical about the claim that a general AI can be achieved using large language models and highlighted his concerns about startups in Silicon Valley making what he considers to be false and misleading claims.</p><p>As is customary now, the Cogito Signal Processing and Machine Learning has compiled and curated a set of ten papers which caught our eyes during the conference. As usual, the papers are from a cross-section of speech technology subdomains like automatic speech recognition (ASR), speaker diarisation and emotion recognition, while also including fundamental machine learning areas like representational learning and new approaches for improving the model development process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ADjPv1tEzyzO_8md_YVsrQ.png" /><figcaption>Interspeech 2021 Computational Paralinguistics Challenge (ComParE)</figcaption></figure><h3>Paper highlights</h3><h4>Paper #1 — <a href="https://www.isca-speech.org/archive/pdfs/interspeech_2021/green21_interspeech.pdf">Automatic Speech Recognition of Disordered Speech: Personalized models outperforming human listeners on short phrases</a></h4><p>Our first paper highlight comes from Google Research, and it involves an update on Google’s concerted effort to make voice interfaces more accessible to folks with a wide diversity of speech conditions. Their findings demonstrate that personalised, speaker-adapted ASR performs much better than state-of-the-art speaker independent systems, and can even perform better than humans especially for speech with very severe impairment.</p><h4>Paper #2 — <a href="https://www.isca-speech.org/archive/pdfs/interspeech_2021/mansfield21_interspeech.pdf">Revisiting Parity of Human vs. Machine Conversational Speech Transcription</a></h4><p>This paper by Mari Ostendorf’s research group looks in more detail at recent findings of the Microsoft ASR system which produces accuracy (word error rate) at the level of human parity on the Switchboard and CallHome datasets. Their findings highlight substantial fine-grained differences in the nature of human versus machine recognition errors. Humans displayed far more deletions compared to ASR and in particular humans tended to miss more conversational words (e.g., backchannels or fillers) compared to the ASR system, even though humans produce a lot of these words. As a result of this difference, the authors claim that there needs to be superhuman word-error rate levels by ASR systems in order to deliver human level conversations.</p><h4>Paper #3 — <a href="https://www.isca-speech.org/archive/pdfs/interspeech_2021/eskimez21_interspeech.pdf">One-Shot Voice Conversion with Speaker-Agnostic StarGAN</a></h4><p>The next paper we selected is in the area of voice conversation. Here researchers from Microsoft use the Generative Adversarial-based STARGAN approach, with several modifications over the original design, for many-to-many voice conversion (i.e. trained using multiple different speakers and applicable to arbitrary new speakers). One key area of novelty of the research includes conditioning the voice conversion model with d-vector speaker embeddings rather than simple one-hot encoding. The results indicate comparable naturalness of the converted speech to the state-of-the-art, but with a higher perceptual similarity to the target speaker.</p><h4>Paper #4 — <a href="https://www.isca-speech.org/archive/pdfs/interspeech_2021/luu21_interspeech.pdf">Leveraging speaker attribute information using multi task learning for speaker verification and diarization</a></h4><p>This paper by researchers from CSTR in Edinburgh looks to improve speaker verification and diarisation tasks by explicitly training models to capture speaker attributes known in other scientific domains to be speaker discriminative — i.e. gender, age, accent, linguistic content and speech-related impairments. The researchers use multi-task learning with separate tasks for each of the speaker attributes, which they hypothesise will encourage the embedding space to learn truly speaker-discriminative information. Their findings indicate improvements in speaker classification and diarisation when using age and nationality as auxiliary recognition tasks.</p><h4>Paper #5 — <a href="https://www.isca-speech.org/archive/pdfs/interspeech_2021/moine21_interspeech.pdf">Speaker Attentive Speech Emotion Recognition</a></h4><p>Researchers from IRCAM here look to improve speech emotion recognition by appropriately handling the speaker-dependent nature of affective expressions. The key idea is to condition the emotion recognition model on the identity of the particular speaker. Their model uses two parallel branches, one for speaker identification and the other for emotion recognition, with an attention mechanism used to link information between the two. Substantial accuracy gains are presented for the speaker dependent system, however the current approach does not generalise to unseen speakers. If the authors can effectively deal with this issue in future work, it may be a promising path forward for the speech emotion recognition field.</p><h4>Paper #6 — <a href="https://www.isca-speech.org/archive/pdfs/interspeech_2021/ge21_interspeech.pdf">Speed up Training with Variable Length Inputs by Efficient Batching Strategies</a></h4><p>A major motivation for attending scientific conferences is to learn about new methods which you can use to improve your own research. This paper by Sony research did exactly that. Training models with variable length sequences as input can be challenging, both in terms of optimising accuracy but also in terms of efficient experimentation. Some approaches, such as using sets of mini-batch sizes or sorted batching has been shown to speed up training but can often result in deteriorated accuracy. The authors describe a novel parameterised batching approach, called semi-sorted batching which can both speed up training and preserve accuracy levels.</p><h4>Paper #7 — <a href="https://www.isca-speech.org/archive/pdfs/interspeech_2021/makishima21_interspeech.pdf">Enrollment-Less Training for Personalized Voice Activity Detection</a></h4><p>In this paper, researchers from NTT focus on the problem of detecting speech from a target speaker in recordings which also contain noise or speech from other speakers. Rather than using a large body of enrollment data to produce a speaker-specific embedding used to condition the voice activity detection model, here the authors instead use the same single utterance both to produce the speaker embedding and also to segment the speech. This approach can be brittle at inference time because of a large mismatch with the data used during training. The authors mitigate this approach by distorting the speech input using SpecAugment and dropout in the speaker embedding part of the model.</p><h4>Paper #8 — <a href="https://www.isca-speech.org/archive/pdfs/interspeech_2021/saebi21_interspeech.pdf">A Discriminative Entity-Aware Language Model for Virtual Assistants</a></h4><p>ASR can often perform poorly when recognising named entities. This is particularly problematic for voice assistants, say when serving a request to have music played by a specific artist. In this paper, researchers from Apple and the University of Notre Dame introduce a lattice rescoring approach based on representations of an input knowledge graph. The authors show how this approach can maintain expected accuracy levels on general voice assistant tasks and improve them for tasks involving named entities.</p><h4>Paper #9 — <a href="https://www.isca-speech.org/archive/pdfs/interspeech_2021/stoidis21_interspeech.pdf">Protecting Gender and Identity with Disentangled Speech Representations</a></h4><p>The issue of privacy and protecting personal information is a major topic in speech technology right now. Ideally only information related specifically to the downstream task at hand (e.g., automatic speech recognition) should be used as input. This paper, authored by researchers from Queen Mary University, involves evaluation of a model architecture based on the variational autoencoder framework which effectively factorises representations related to the speaker’s identity, their gender and the speech content. The authors demonstrate that the system can support omission of gender and identity information from the produced speech embedding, such that privacy-sensitive downstream tasks of gender and speaker recognition produce accuracy at chance levels, while the core tasks of ASR performs well.</p><h4>Paper #10 — <a href="https://www.isca-speech.org/archive/pdfs/interspeech_2021/tjandra21_interspeech.pdf">Unsupervised Learning of Disentangled Speech Content and Style Representation</a></h4><p>Our final paper choice is also related to representational learning and disentangling certain factors, but here applied to reconstructing speech with a target speaking style. The proposed model has separate branches for encoding linguistic content and speaking style, which are then combined by a decoder tasked with reconstructing the speech. A global style embedding computed from a small amount of target speech can be used to clearly modify the speaking style of the reconstructed speech — see <a href="https://unsup-content-style-demo-v1.netlify.app/">this link</a> for audio samples related to this paper.</p><h3>Continuing the path back</h3><p>So, there is definitely quite a bit to digest from this year’s Interspeech. We very much hope that the speech technology community looks to build on the kickstart and resuscitation that Interspeech 2021 has provided. We also hope that he days of unengaging, fully virtual conferences are behind us!</p><h3>Opportunities at Cogito</h3><p>If you are based in the US, EU (Ireland) or India and are interested in opportunities at Cogito, please check out our <a href="https://cogitocorp.com/careers/">careers page</a>! We have an office optional policy that encourages remote work and collaboration!</p><h3>Acknowledgements</h3><p>This article was jointly written by members of the Cogito Signal Processing and Machine Learning team following attendance at the Interspeech 2021 conference.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=982cad437fba" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cogito-engineering/interspeech-2021-the-return-of-engaging-interactive-speech-conferences-982cad437fba">Interspeech 2021 — The return of engaging, interactive speech conferences</a> was originally published in <a href="https://medium.com/cogito-engineering">cogito-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ICASSP 2021 — Broadening the application of audio, speech and language technology through modern…]]></title>
            <link>https://medium.com/cogito-engineering/icassp-2021-broadening-the-application-of-audio-speech-and-language-technology-through-modern-9b00ee2f892f?source=rss----7d6ba2b042a9---4</link>
            <guid isPermaLink="false">https://medium.com/p/9b00ee2f892f</guid>
            <category><![CDATA[speech-technology]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[emotion-recognition]]></category>
            <category><![CDATA[speech-recognition]]></category>
            <category><![CDATA[signal-processing]]></category>
            <dc:creator><![CDATA[CogitoEngineering]]></dc:creator>
            <pubDate>Mon, 14 Jun 2021 14:11:47 GMT</pubDate>
            <atom:updated>2021-06-22T17:02:10.778Z</atom:updated>
            <content:encoded><![CDATA[<h3>ICASSP 2021 — Broadening the application of audio, speech and language technology through modern signal processing</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5GQdv_M2sq-QB4Iceynw4g.png" /><figcaption>ICASSP 2021 — Image taken from the <a href="https://2021.ieeeicassp.org/">ICASSP 2021 website</a></figcaption></figure><p>Another speech conference, another virtual event. Although the scientific content continues to advance and develop, the apparent lack of interaction, idea-sharing and collaboration is a stark contrast to the in-person conferences of the past. Despite the organiser’s valiant efforts to foster communication, for instance with the introduction of the <a href="https://gather.town/">gather.town</a> application, conversation tabs on presentations lay empty and even questions during keynotes were minimal. One of our co-workers who had an in-person meeting with his team the other day said to us, “what we got done in one hour tonight would have taken weeks via video calls”. Although there have been many benefits of virtual conferences in terms of making them more accessible for folks around the world and making the scientific content more searchable, we think this comment effectively sums up the gaping hole that the lack of in-person scientific meet-ups has left.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SYjHyjzAUcvz5h18tMYkgA.png" /><figcaption>ICASSP 2021 — Opening ceremony</figcaption></figure><p>Nevertheless, there were some great papers at this year’s ICASSP with innovative new ideas to do with semi-supervised representational learning, reducing the latency of streaming, speech recognition-dependent classification algorithms, making voice interfaces more accessible for folks with atypical speech, among many others. Here are some of our top paper picks:</p><h3>Paper highlights</h3><h4>Paper #1 — <a href="https://ieeexplore.ieee.org/document/9414644">Extending Parrotron: An End-to-End, Speech Conversion and Speech Recognition Model for Atypical Speech</a></h4><p>The automatic speech recognition (ASR) wars continue, with many companies and academic groups jockeying for the lowest word error rate (WER). Although ASR has been the application which is driving a lot of fundamental advances in machine learning, admittedly these sessions can be a little tiresome. This paper from Google, however, really stuck out for us in particular because of its potential social benefit. People with speech disorders have been largely locked out from effectively using voice interfaces to date and this paper describes some research to address this. The approach involves application of the Parrotron voice conversion system, which has been previously applied to producing fluent speech from atypical speech. The system is extended to do voice conversion and ASR simultaneously, and results indicate that with an hour of atypical speech the adaptation can produce substantial improvements in WER. The paper further provides an analysis of the beneficial adaptation effect across a range of speech impairments.</p><h4>Paper #2 — <a href="https://ieeexplore.ieee.org/document/9413734">Joint ASR and Language Identification Using RNN-T: An Efficient Approach to Dynamic Language Switching</a></h4><p>Automatic conversion of speech to text in a scenario where speakers are dynamically changing from one language to another (aka “code-switching”) is perhaps the hardest task for modern speech processing. This paper takes on the challenge, and superimposes two additional tricky requirements of streaming processing and on-device compute hardware restrictions. The study presented here evaluates various RNN-t based model architectures with multi-task learning over language identification and word error rate minimisation tasks. Experiments carried out on audio data involving both English and Hindi again highlight how challenging this multi-language recognition task is, however the approach used here does take some important steps forward in making this technology realisable in practical commercial applications.</p><h4>Paper #3 — <a href="https://ieeexplore.ieee.org/document/9413680">A Comparison Of Discrete Latent Variable Models For Speech Representation Learning</a></h4><p>Finding salient, reusable features through representational learning has been challenging for acoustic speech processing. Other disciplines like computer vision and natural language processing (NLP) found effective ways of doing semi-supervised feature learning, but finally in the last few years speech processing is catching up. This paper from Facebook AI Research is a fairly straightforward comparison of different representational learning approaches. Sometimes a small, focused, but well-executed study can be the most compelling, particularly when restricted to the conference paper format. This paper is a great example of this. The basic take-home message from this paper is that contrastive learning approaches are far superior to autoencoding, for speech feature learning (at least for phoneme discrimination tasks).</p><h4>Paper #4 — <a href="https://ieeexplore.ieee.org/document/9414546">Unsupervised and Semi-Supervised Few-Shot Acoustic Event Classification</a></h4><p>This paper by researchers at University of California and Amazon, builds on the recent Facebook AI paper on VQ-wav2vec (a self-supervised representational learning approach for audio, which produces discrete outputs that can be used as input to NLP models). Here the VQ-wav2vec approach is applied to the problem of few-shot acoustic event classification. <br>Contrastive training is used to develop the VQ-wav2vec model, which produces a discrete output. The discrete output can be used as input to the BERT model architecture developed for NLP tasks.<br>The BERT training is adapted for audio and the output is then used as the representation for classifying acoustic events where just a few reference samples are available. The results indicate that although the standard wav2vec approach is more effective than VQ-wav2vec (when comparing directly). However, if you additionally include BERT training with the VQ-wav2vec discrete output, you can get considerable gains in accuracy.</p><h4>Paper #5 — <a href="https://ieeexplore.ieee.org/document/9414193">Towards immediate backchannel generation using attention-based early prediction model</a></h4><p>For real-time applications, latency is typically a big concern. When you are using streaming ASR as an upstream preprocessing step, the latency introduced can cause poor user experience for certain use-cases. This paper authored by researchers from Hitachi proposes a novel approach to mitigating this latency problem. The study focuses on the application of generating backchannels in spoken dialogue systems. The proposed solution involves using an attention-based, long-short term memory (LSTM) network architecture, but interestingly with an early loss error function that allows for early predictions before the utterance has completed.</p><h4>Paper #6 —<a href="https://ieeexplore.ieee.org/document/9413596"> Generating empathetic responses by injecting anticipated emotion</a></h4><p>There has been increasing interest recently in incorporating emotion sensing in speech-based conversational systems. This paper, by researchers at the Harbin Institute of Technology, focuses on the generation of empathetic automated voice responses. The approach involves using an auxiliary recognition task of estimating the anticipated emotion, based on analysis of the previous utterances of the speaker. The authors present promising results on the EmpDialogue dataset, and this line of research is paving the way for voice assistants which respond both with the information requested but also conveyed in a speaking style desired by the user.</p><h4>Paper #7 — <a href="https://ieeexplore.ieee.org/document/9413851/">Wave-Tacotron: Spectrogram-Free End-to-End Text-to-Speech Synthesis</a></h4><p>The Tacotron text-to-speech (TTS) synthesis system, equipped with its WaveRNN audio sample generation module, revolutionised the TTS domain several years ago in producing truly natural sounding synthetic speech. One big outstanding problem with the Tacotron system is that it is so computationally expensive, requiring considerable and costly compute hardware even at inference time, and typically with a high degree of latency. The original Tactotron architecture involves two steps: first generate a spectrogram and second generate waveform samples. The approach described in the present paper avoids this two-step process and uses a complete end-to-end training approach, where the model generates blocks of audio samples, rather than an intermediate spectrogram. The blocks of samples can be simply concatenated together to produce the output synthetic speech. Although the naturalness results are not quite as good as the original Tacotron system, there are substantial gains reported in terms of reducing computational load and the duration of inference steps.</p><h4>Paper #8 — <a href="https://ieeexplore.ieee.org/document/9413910">Contrastive Unsupervised Learning for Speech Emotion Recognition</a></h4><p>As mentioned above, speech emotion recognition is getting increased research attention, in particular because of the potential in many commercial applications like Amazon Alexa. This paper by a team from Amazon, in collaboration with researchers from the University of Chicago, involves yet another example of contrastive pre-training — here applied to the problem of speech emotion recognition. The authors use contrastive predictive coding (CPC) for unsupervised feature learning on a large, unlabeled speech corpus (in this case LibriSpeech), before using the learnt representations as input to an emotion classification network, whose weights are trained on the IEMOCAP and MSP-PodCast datasets.</p><h4>Paper #9 — <a href="https://ieeexplore.ieee.org/document/9414698">Speech Emotion Recognition Based on Listener Adaptive Models</a></h4><p>Due to the subjective nature of emotion perception from speech, various listeners can have different impressions of the conveyed emotion and can often display a reasonable degree of disagreement. In this study, researchers from NTT propose a listener adaptive (LA) model which incorporates listener information to the model architecture via three adaptation layers. Initial results of this research indicate that encoding listener information in training via adaptation layers can produce improvements in classification accuracy. The authors found that the adaptation layers adjacent to classification decision layers were particularly effective.</p><h4>Paper #10 — <a href="https://ieeexplore.ieee.org/document/9414542/">LSSED: A Large-Scale Dataset and Benchmark for Speech Emotion Recognition</a></h4><p>The last emotion-related paper we want to highlight looks to tackle the problem of data volume which has plagued the speech emotion recognition discipline, since data-hungry deep learning models became popular again. The LSSED dataset, which was collected by the authors, is over 200 hours in total duration, making it several times larger than almost all other available datasets, perhaps with the exception of the MSP-PodCast dataset. Although the data elicitation approach means the speech is not completely natural (it is admittedly much more natural than feigned emotional speech), the sheer size of the dataset makes it potentially very valuable to the speech research community when it is released on <a href="https://github.com/tobefans/LSSED">this GitHub page</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/806/1*gqvGiW5iQW0y2lToacX2lw.png" /><figcaption>Wordcloud we created based on ICASSP 2021 paper content(thanks Omkar!)</figcaption></figure><h3>The continued impact of audio and speech processing</h3><p>So that concludes our ICASSP 2021 paper roundup. The impact of audio and speech processing on modern commercial applications shows no signs of slowing down, and from the papers at this year’s conference, the associated scientific research also continues at a very fast pace. Challenging problems like insufficient datasets, models being too computationally heavy to support real-life use-cases, latency issues, the lack of reusable representations, accuracy bias across different kinds of voices — are all being quickly resolved. See you (hopefully in-person) next year!</p><h3>Opportunities at Cogito</h3><p>If you are based in the US, EU (Ireland) or India and are interested in opportunities at Cogito, please check out our <a href="https://cogitocorp.com/careers/">careers page</a>! We have an office optional policy that encourages remote work and collaboration!</p><h3>Acknowledgements</h3><p>This article was jointly written by members of the Cogito Signal Processing and Machine Learning team, along with Omkar Prabhu, following attendance at the ICASSP 2021 virtual conference. Thanks also to Ian Kelly for proofreading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9b00ee2f892f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cogito-engineering/icassp-2021-broadening-the-application-of-audio-speech-and-language-technology-through-modern-9b00ee2f892f">ICASSP 2021 — Broadening the application of audio, speech and language technology through modern…</a> was originally published in <a href="https://medium.com/cogito-engineering">cogito-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Keyboard shortcuts in React, and how they help the annotation team]]></title>
            <link>https://medium.com/cogito-engineering/keyboard-shortcuts-in-react-and-how-they-help-the-annotation-team-14d22fd36c4a?source=rss----7d6ba2b042a9---4</link>
            <guid isPermaLink="false">https://medium.com/p/14d22fd36c4a</guid>
            <category><![CDATA[react]]></category>
            <category><![CDATA[apps]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[annotation-tools]]></category>
            <dc:creator><![CDATA[CogitoEngineering]]></dc:creator>
            <pubDate>Mon, 29 Mar 2021 20:48:03 GMT</pubDate>
            <atom:updated>2021-03-31T08:05:01.011Z</atom:updated>
            <content:encoded><![CDATA[<p>Cogito’s Annotation Team has been requesting keyboard shortcuts for some time to help improve the efficiency of their workflows, and we have just added them to our Annotation Tool. This article walks you through the process of adding keyboard shortcuts to a React application, and explains how they will help annotators in the new version of the tool. We use the <a href="https://github.com/greena13/react-hotkeys">react-hotkeys</a> library, which we refer to in the post below.</p><h3>Shortcuts in the Annotation Tool</h3><p>First, let’s take a look at how shortcuts work in the Annotation Tool.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgfycat.com%2Fifr%2Frareflashycattle&amp;display_name=Gfycat&amp;url=https%3A%2F%2Fgfycat.com%2Frareflashycattle&amp;image=https%3A%2F%2Fthumbs.gfycat.com%2FRareFlashyCattle-size_restricted.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=gfycat" width="1910" height="844" frameborder="0" scrolling="no"><a href="https://medium.com/media/e556e09c5a63b92867d5930ce2e5ef03/href">https://medium.com/media/e556e09c5a63b92867d5930ce2e5ef03/href</a></iframe><p>Before the shortcuts were added, all annotation had to be done using the mouse. Now, the user interface (UI) displays a list of shortcuts and their key bindings, and users can change them dynamically. Each annotator can customize their own shortcuts, and they are saved in the database so they persist after reloading the page.</p><h3>Meeting with the annotation team</h3><p>Before adding shortcuts to the Annotation Tool, we had to find out what features the Annotation Team wanted. We set up a meeting with the whole team, giving annotators the opportunity to voice any issues they had with the current workflows in the Annotation Tool.</p><p>Some of the most commonly requested features were:</p><ul><li>Playing / pausing the audio</li><li>Skipping back and forth</li><li>An easier way to create and modify intervals on the audio player</li></ul><p>After adding these features, we experimented with different methods of modifying intervals, until we came up with our final product. Here’s an example of how someone might use these shortcuts to annotate a series of intervals:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgfycat.com%2Fifr%2Fclassicunevenibex&amp;display_name=Gfycat&amp;url=https%3A%2F%2Fgfycat.com%2Fclassicunevenibex&amp;image=https%3A%2F%2Fthumbs.gfycat.com%2FClassicUnevenIbex-size_restricted.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=gfycat" width="1910" height="844" frameborder="0" scrolling="no"><a href="https://medium.com/media/0c00c8212ddacd4ac28b18c76a560136/href">https://medium.com/media/0c00c8212ddacd4ac28b18c76a560136/href</a></iframe><h3>Adding shortcuts to your own app</h3><p>Now that we’ve seen how keyboard shortcuts are used within the Annotation Tool, let’s take a look at how they can be added to your own React application.</p><h4>Installation</h4><p>Assuming you already have a React application, all you need to install is <a href="https://github.com/greena13/react-hotkeys">react-hotkeys</a>.</p><pre>npm install react-hotkeys --save</pre><h4>Using react-hotkeys</h4><p><a href="https://github.com/greena13/react-hotkeys">react-hotkeys</a> provides several components that are useful for implementing keyboard shortcuts into your React app.</p><ul><li>The <a href="https://github.com/greena13/react-hotkeys#globalhotkeys-component">GlobalHotKeys component</a> allows you to trigger actions with key events that happen anywhere in the document.</li><li>The <a href="https://github.com/greena13/react-hotkeys#hotkeys-components">HotKeys component</a> only triggers actions with key events that happen in one of their descendants.</li></ul><p>In our Annotation Tool, we use GlobalHotKeys, since we want our shortcuts to be usable from anywhere on the page.</p><h4>keyMaps and handlers</h4><p>In order to use either of these components, you must pass them both a keyMap and a handlers prop.</p><p>Within a keyMap, actions are linked to key bindings, and within handlers, actions are linked to functions that are called when the corresponding key combination is detected.</p><p>The keyMap can be defined with just a key mapping or with additional data that is used to display key bindings in the UI. Shortcuts could be a single key, a combination of keys pressed at the same time (written as ”a+b+c”), or a sequence of keys pressed and released one after the other (written as ”a b c”).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/be1ab6870858d879cb5a117f3d34b5b0/href">https://medium.com/media/be1ab6870858d879cb5a117f3d34b5b0/href</a></iframe><p>Handlers are simply defined as actions linked to the functions they will call.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/047b72b8c09b74207032febc0d0f15b8/href">https://medium.com/media/047b72b8c09b74207032febc0d0f15b8/href</a></iframe><p>If you don’t need dynamic shortcuts, you can stop here and simply add a line like this anywhere in your code, passing it the appropriate keyMap and handlers.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5273f176571dfead7de7ca80bef15888/href">https://medium.com/media/5273f176571dfead7de7ca80bef15888/href</a></iframe><p>Note that it is perfectly acceptable to have multiple GlobalHotKeys components with separate keyMaps and handlers within your code. They are still all able to resolve their individual actions, and if there are any duplicate keyMaps or handlers, whichever GlobalHotKeys was mounted first takes precedence.</p><h4>Preventing default behavior</h4><p>Some key combinations already have default behavior assigned to them by the browser. For example, pressing ”space” scrolls down a page, and ”Control+p” prints the page. If we want to prevent this default behavior from occurring, we can use this function to intercept the event before it gets handled, and prevent the default handler from being activated.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/904d4d597b2c88060e8ff41bdb0cb00d/href">https://medium.com/media/904d4d597b2c88060e8ff41bdb0cb00d/href</a></iframe><p>We can use this by wrapping our handlers in it wherever they are defined:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f5445b8696d6ab2520534f3744c23335/href">https://medium.com/media/f5445b8696d6ab2520534f3744c23335/href</a></iframe><h4>Dynamic shortcuts</h4><p>In order to change shortcuts dynamically, we need several things:</p><ul><li>A UI element that allows us to view and edit shortcuts</li><li>A way to store the shortcuts so they persist between sessions</li></ul><h4>UI for changing shortcuts</h4><p>For the UI, we use Material-UI to style the table, but you can use any other framework as well, or do your own custom styling. If you want to use Material-UI, it can be installed like this:</p><pre>npm install <a href="http://twitter.com/material">@material</a>-ui/core</pre><p>First, create a dialog to edit shortcuts.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2b2f997075e65ce7926b4b91dbe51458/href">https://medium.com/media/2b2f997075e65ce7926b4b91dbe51458/href</a></iframe><p>It should look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/605/1*ZaHo8Uez0yuUTD2aGcHUkQ.png" /><figcaption>Change shortcuts dialog</figcaption></figure><p>Next, create a dialog to show while a shortcut is being changed:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4d1add9337187f1da1e634c41b3b1090/href">https://medium.com/media/4d1add9337187f1da1e634c41b3b1090/href</a></iframe><p>It should look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/593/1*7kx8Y7pr9LzsMnnEx_zSBg.png" /><figcaption>Changing shortcut dialog</figcaption></figure><p>Create a button to press to edit shortcuts, and a Popover that will display the current key bindings when hovering over it:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/15189e683e70e174b2439253ef41c041/href">https://medium.com/media/15189e683e70e174b2439253ef41c041/href</a></iframe><p>They should look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/277/1*5vtjA9DLcQwPXGhYwi9W9Q.png" /><figcaption>Edit shortcuts button</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*fbvRZmUQzpLDlGj4nZKq1g.png" /><figcaption>Shortcuts popover</figcaption></figure><p>Now, wherever you want these components to show up, you need to define the following in order to keep track of the state of the shortcut UI while changing shortcuts:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/367e51bf82e50fc6876bc99dd04ab737/href">https://medium.com/media/367e51bf82e50fc6876bc99dd04ab737/href</a></iframe><p>And finally, render the UI components like so:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ede181b80cac5bfda0fa4366d0206703/href">https://medium.com/media/ede181b80cac5bfda0fa4366d0206703/href</a></iframe><h4>Storing the shortcuts</h4><p>How the shortcuts are stored and retrieved largely depends on your database implementation, but we can go over some generic details here.</p><p>First, create a default keyMap somewhere. This allows shortcuts to operate without users having to define them first.</p><p>It typically makes sense to store the current keyMap in some kind of global store, so any HotKeys component that needs it is able to access it. When retrieving shortcuts from the database, we can do something like this to make sure that a keyMap is defined for our HotKeys components.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3e3a45db21fce2214302cb173f104c1e/href">https://medium.com/media/3e3a45db21fce2214302cb173f104c1e/href</a></iframe><p>Additionally, we need to retrieve the shortcuts from the database whenever the page is opened, so we can add something like this to our code, where retrieveShortcuts makes the appropriate API calls to the backend.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1ef66466b13ed8aad72ddf998de8f428/href">https://medium.com/media/1ef66466b13ed8aad72ddf998de8f428/href</a></iframe><p>Whenever we change the shortcuts locally, we also want them to be uploaded to the database, so we add a line like this within the cancelListening function in showChangeShortcutDialog (it has already been added in the code snippet above).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b0fe4fb1b1b68d73885961b7701d7fbd/href">https://medium.com/media/b0fe4fb1b1b68d73885961b7701d7fbd/href</a></iframe><h4>Putting it all together</h4><p>Once we have our keyMap stored in the database, all that’s left to do is add GlobalHotKeys components wherever handlers are defined. The allowChanges prop allows the keyMap and handlers to be changed dynamically.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/33adfd4e2e14a6ce2ff6d5f8c093091d/href">https://medium.com/media/33adfd4e2e14a6ce2ff6d5f8c093091d/href</a></iframe><h3>Opportunities at Cogito</h3><p>If you are based in the US, Ireland or India and are interested in opportunities at Cogito, please check out our <a href="https://cogitocorp.com/careers/">careers page</a>! We have an office optional policy that encourages remote work and collaboration!</p><h3>Acknowledgements</h3><p>This article was written by Ben Knower during his internship at Cogito — while being mentored by Jacob Duyon. Thanks also to Richard Brutti and Matthew Roddy for the detailed proofreading and feedback.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=14d22fd36c4a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cogito-engineering/keyboard-shortcuts-in-react-and-how-they-help-the-annotation-team-14d22fd36c4a">Keyboard shortcuts in React, and how they help the annotation team</a> was originally published in <a href="https://medium.com/cogito-engineering">cogito-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Enabling SSO for your React and Django app with SAML 2.0]]></title>
            <link>https://medium.com/cogito-engineering/enabling-sso-for-your-react-and-django-app-with-saml-2-0-754ef752acc1?source=rss----7d6ba2b042a9---4</link>
            <guid isPermaLink="false">https://medium.com/p/754ef752acc1</guid>
            <category><![CDATA[apps]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[django]]></category>
            <category><![CDATA[speech-technology]]></category>
            <dc:creator><![CDATA[CogitoEngineering]]></dc:creator>
            <pubDate>Tue, 24 Nov 2020 20:27:35 GMT</pubDate>
            <atom:updated>2021-03-31T08:06:41.835Z</atom:updated>
            <content:encoded><![CDATA[<p>This guide walks you through integrating SAML 2.0 authentication with a Django API server and React Single-Page Application (SPA). Authentication is set up so that the front-end React application obtains a JSON Web Token (JWT) from the Django API server once the API server has successfully authenticated with the Single Sign-on Provider — in this article we use Okta.</p><h3>Our annotation tool at Cogito</h3><p>At Cogito, we use a similar setup to authenticate users for our annotation tool (see screenshot). With this, we are able to deliver securely labeled data to power our machine learning models.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t0j5xFZ4ZsEHuifoTCh4vg.png" /><figcaption>Screenshot of the Cogito Annotation Tool</figcaption></figure><p>Here is the step by step workflow of the implementation:</p><p>1. Log in and visit your identity provider app home (i.e. Okta)<br>2. Click your app<br>3. Identity provider redirects to the Django backend server<br>4. Backend server completes authentication and generates a JWT token<br>5. Redirect to the frontend server with the JWT token as a URL parameter<br>6. Frontend stores JWT token and can now make authenticated API requests<br>7. Frontend can refresh token until configured refresh period expires</p><p>Start here to configure SAML2.0 authentication with a backend Django API server and a React application:</p><ul><li>Install requirements</li><li>Setup JWT token Authentication</li><li>Download your SAML2.0 metadata XML file</li><li>Store JWT token in React</li></ul><h3>Installing requirements</h3><p>For this blog post we use the following technologies:</p><ul><li><a href="https://www.okta.com/products/single-sign-on/">Okta SSO provider</a></li><li><a href="https://reactjs.org/">React</a></li><li>Django and the following additional packages: <a href="https://github.com/fangli/django-saml2-auth">Django SAML2 Auth</a> and <a href="https://www.django-rest-framework.org/">Django Rest Framework</a></li></ul><p>We assume you are familiar with Django and have an application ready. Let’s start by adding the <a href="https://github.com/fangli/django-saml2-auth">django-saml2-auth</a> package.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fec4d43ee9e6499c3e01cefe11b50f81/href">https://medium.com/media/fec4d43ee9e6499c3e01cefe11b50f81/href</a></iframe><p>Now that you’ve installed django-saml2-auth, let’s configure the authentication views and their settings.</p><p>Add the following to your root “urls.py” file. Note: This rewrites the user login + admin login URL patterns:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/71daed9d7a6f8ca74096129921e2aa0f/href">https://medium.com/media/71daed9d7a6f8ca74096129921e2aa0f/href</a></iframe><p>Next, add the “django-saml2-auth” app to your “INSTALLED_APPS”:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2789703c69ab5d3d16371023fe400b83/href">https://medium.com/media/2789703c69ab5d3d16371023fe400b83/href</a></iframe><p>Then, you need to add the “SAML2_AUTH” variable to your “settings.py” file. This is a map containing all the required and optional configurations for “django_saml2_auth”. Below is a bare minimum example for a React SPA and Django API server with separate host names:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5807c52d0f75422ca4aeb02a15782edc/href">https://medium.com/media/5807c52d0f75422ca4aeb02a15782edc/href</a></iframe><p>Make sure the path in the “ENTITY_ID” URL lines up with your views’s URLs. For example, if you changed the “/sso” URL to “/saml2_auth” in your “urls.py” the “ENTITY_ID” should be “<a href="https://backend.example.com.com/saml2_auth/acs/%60">https://BACKEND.example.com.com/saml2_auth/acs/</a>”.</p><p>There are several other optional settings that “django_saml2_auth” provides. For example, you can define attributes in your SSO provider and sync them with Django or trigger hooks that run before/after login. <a href="https://github.com/fangli/django-saml2-auth">Checkout their docs</a> to learn more about these options. That’s all you need. Next we configure your SSO provider.</p><h4>Install Django Rest Framework</h4><p><a href="https://www.django-rest-framework.org/">Django Rest Framework</a> is a useful, well documented API development framework. With DRF you are able to rapidly develop APIs by abstracting away the boilerplate glue that Django doesn’t provide.</p><p>DRF’s documentation is excellent. To install DRF to your Django application <a href="https://www.django-rest-framework.org/#installation">follow the instructions here</a>.</p><h3>Set up JWT Auth with Django</h3><p>Next, we set up your Django app to allow JWT based authentication. You may want to write your own JWT authentication class, just be sure to add this to “DEFAULT_AUTHENTICATION_CLASSES” in your “settings.py” file. For this guide we use the <a href="https://github.com/styria-digital/django-rest-framework-jwt">drf-jwt plugin</a>. <a href="https://medium.com/styria-digital.github.io/django-rest-framework-jwt/">Read the instructions here</a> or continue following below:</p><p>Install drf-jwt:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b6553a3e991f3f3e391756820d92bdfa/href">https://medium.com/media/b6553a3e991f3f3e391756820d92bdfa/href</a></iframe><p>Add this to your urls.py:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c9178838517c0df255872733369b529e/href">https://medium.com/media/c9178838517c0df255872733369b529e/href</a></iframe><p>Note: You do not need to use the “obtain_jwt_token” view, because a JWT token is automatically generated by the “django_saml2_auth” application each time a user is authenticated.</p><p>Finally, add this to your “settings.py”:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8f2e8740f4203ae046b03b0ec6edb9c5/href">https://medium.com/media/8f2e8740f4203ae046b03b0ec6edb9c5/href</a></iframe><p>You can checkout additional configurations on the <a href="https://styria-digital.github.io/django-rest-framework-jwt/#additional-settings">drf-jwt docs page</a>.</p><h3>Download your SAML2.0 metadata XML file</h3><p>Assuming you have an Okta application set up already, you’ll need to export the IDP metadata XML file and keep it in your app repo. <a href="https://support.okta.com/help/s/article/How-do-we-download-the-IDP-XML-metadata-file-from-a-SAML-Template-App">Here are instructions from Okta on how to do this.</a></p><p>Once you have that file, store it inside your Django app settings directory. Then update the “METADATA_LOCAL_FILE_PATH” with the relative path. This setting is inside the “SAML2_AUTH” dictionary of your “settings.py” file.</p><h3>Use JWT token for API requests in React</h3><p>Once you are here, you can now login to your backend server with SSO. Simply login with the default Django login URL: “/accounts/login”. You will be redirected to your identity provider. Once authenticated, you’ll then be redirected to your React application (see “FRONTEND_URL” setting) with a JWT token in the URL.</p><p>The final step is to create a React component that retrieves the token from the URL and stores it somewhere. In this guide, we keep the JWT token in local storage. Depending on your application requirements, you might want to store the token in context or session storage.</p><p>Note: This example uses the “query-string” and “react-router” libraries to get and parse URL parameters. <a href="https://reactrouter.com/web/guides/quick-start">Check out react-router here</a>.</p><p>Add this component to the landing page where “FRONTEND_URL” sends you:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9cc1b0ba59fdd4728daa458ee2d63b24/href">https://medium.com/media/9cc1b0ba59fdd4728daa458ee2d63b24/href</a></iframe><p>That’s it! You can now authenticate your React application’s API requests with the JWT token.</p><h3>Opportunities at Cogito</h3><p>If you are based in the US, Ireland or India and are interested in opportunities at Cogito, please check out our <a href="https://cogitocorp.com/careers/">careers page</a>! We have an office optional policy that encourages remote work and collaboration!</p><h3>Acknowledgements</h3><p>This article was written by Jacob Duyon and was based on collaboration between Jacob and Mauri Niininen. Thanks to Richard Brutti for providing detailed proofreading and feedback.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=754ef752acc1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cogito-engineering/enabling-sso-for-your-react-and-django-app-with-saml-2-0-754ef752acc1">Enabling SSO for your React and Django app with SAML 2.0</a> was originally published in <a href="https://medium.com/cogito-engineering">cogito-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Interspeech 2020 — continued advances in speech and language processing]]></title>
            <link>https://medium.com/cogito-engineering/interspeech-2020-continued-advances-in-speech-and-language-processing-80445e381d01?source=rss----7d6ba2b042a9---4</link>
            <guid isPermaLink="false">https://medium.com/p/80445e381d01</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[signal-processing]]></category>
            <category><![CDATA[emotion-recognition]]></category>
            <category><![CDATA[speech-synthesis]]></category>
            <category><![CDATA[speech-recognition]]></category>
            <dc:creator><![CDATA[CogitoEngineering]]></dc:creator>
            <pubDate>Wed, 04 Nov 2020 09:33:52 GMT</pubDate>
            <atom:updated>2021-03-31T08:08:04.517Z</atom:updated>
            <content:encoded><![CDATA[<h3>Interspeech 2020 — continued advances in speech and language processing</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w4nQA7OEujNMCRCWsPvNMA.png" /><figcaption>Taken from the <a href="http://www.interspeech2020.org/">Interspeech 2020 website</a></figcaption></figure><p>It has been a long and challenging year since the <a href="https://interspeech2019.org/">last Interspeech</a> in Graz. The organisers this time made valiant attempts to maintain a normal, in-person meeting of the speech processing community — knowing the value of having this body of people meet and interact face-to-face. However, it was inevitable that postponing the conference by just a few weeks would not be enough to avoid a fully virtual affair. Although there were some issues along the way, including problems accessing the virtual portal and a last-minute keynote cancellation — the organisers did a good job in creating an engaging conference experience.</p><h3>Themes</h3><p>One significant theme I noticed at the conference was that semi-supervised learning approaches are finally starting to gain real traction. Of course computer vision and natural language processing domains have effectively exploited unsupervised and semi-supervised learning approaches for some time. Although there has been a lot of research in applying these techniques to audio and speech processing, the effect on results often seems to be marginal. The student-teacher learning framework in particular has shown real promise in providing a way to leverage very large volumes of unlabeled data for improving not just accuracy and generalization but also for training more compact and computationally light models.</p><p>The topic of COVID-19 of course seeped its way into the conference proceedings. In fact a part of this year’s Computational Paralinguistics Challenge (ComParE) included a sub-challenge on classifying speech as being produced by a person using a mask. The ComParE session also contained papers focused on continuous recognition of breathing signals. In addition to being COVID-related, there is a clear trend in the speech processing community towards better understanding the situational context in which speech is produced, as well as understanding it more deeply than just the spoken words.</p><p>The keynote presentation from Shehzad Mevawalla involved an overview of conversational AI, in particular as related to virtual assistants like Amazon Alexa. However, despite this the vast majority of papers still focus on processing of speech produced by individuals. It is surprising to me that the areas like speech synthesis and speech emotion recognition that have made great strides in recent years have not begun to shift focus to conversational and interactive settings. I hope that efforts, like the new MSP-Conversation Corpus (see more in paper highlights below), help to advance the research more in this direction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lc1BcQekjdA7hePnONqsqw.png" /><figcaption>Keynote presentation on Amazon Alexa from Shehzad Mevawalla</figcaption></figure><h3>Paper highlights</h3><p>Here are some papers which caught my eye during the week. I tried to select papers across the speech processing spectrum, including presentations to do with speech synthesis, automatic speech recognition, speech emotion recognition and audio event detection.</p><h4>Paper #1 — Controllable Neural Text-to-Speech Synthesis using Intuitive Prosodic Features</h4><p>One of the really attractive properties of early formant and parametric speech synthesis was the fact that you could have fine-grained control over specific aspects of the voice source for modifying prosody and voice quality. Unfortunately the naturalness of the synthesis was nowhere near what it is today and there was never a really straightforward method for controlling the suprasegmentals. There was also a paper from Amazon on controlling prosody in text-to-speech (TTS) synthesis, but I found this paper from Apple researchers to be particularly striking. The proposed architecture is based on the Tacotron 2 system for generating a Mel spectrogram using phone embeddings as input.</p><p>The proposed architecture includes a prosody encoder using intuitive features based on fundamental frequency, energy and spectral tilt. The prosody feature vector is concatenated with the Tacotron decoder input to enable prosody control. Then at synthesis time, the user can select values between -1 and 1, for each of the prosody features, to modify the synthesised prosody at the sentence level. Mean-opinion score naturalness scores closely match the baseline system for all but the low extremes of duration and energy settings, and produce perceptually clear modifications in prosody. It will be interesting to see how future research deals with providing finer-grained prosodic control.</p><h4>Paper #2 — The MSP-Conversation Corpus</h4><p>The research team in the Multimodal Signal Processing (MSP) lab in the University of Texas at Dallas have been doing really important work in the speech emotion recognition domain for some time, developing an ever-increasing dataset with crowd-sourced labeling of emotion using audio data from podcasts and radio shows. The lack of large, authentic and well-labeled datasets has hampered improvement in this field for quite some time. This paper announces the availability of the MSP-Conversation corpus, which uses the same source data as the MSP-Podcast dataset, but rather than using audio from individual speakers it selects segments with audio from both sides of the two-party conversation. The dataset leverages previously labeled recordings for finding areas with a diversity of emotion, and uses a joystick-based continuous annotation approach which is then corrected for annotator reaction times. I hope that this set helps enable improvements in conversational speech processing, which has at times received less research attention than I have expected.</p><h4>Paper #3 — Adaptive Domain-Aware Representation Learning for Speech Emotion Recognition</h4><p>This paper focuses on the problem of domain or context invariance in speech emotion recognition. The authors consider “domain” to cover gender, age and other factors, and such categories can be hard to generalise over for many emotion recognition approaches. The model architecture here is based on multi-task learning with one branch for classifying domain and the other for classifying the emotion. There is a pathway between the two model branches, where representations from the domain branch are used as input to a “domain-aware attention layer” which is used to create a separate attention embedding for the emotion classification branch. Although the results show only marginal improvements over the best performing comparison model, I suspect that given a larger dataset that this difference in accuracy may be extended. I still struggle to understand why researchers continue to use such small datasets as IEMOCAP for audio-specific speech emotion recognition studies, given the availability of large, more appropriate datasets like MSP-Podcast.</p><h4>Paper #4 — Improved Noisy Student Training for Automatic Speech Recognition</h4><p>Google continues to relentlessly break new ground in automatic speech recognition (ASR). The so-called Noisy Student Training (NST) framework has gained significant attention for semi-supervised image recognition tasks. NST is an iterative training strategy whereby a teacher model is used to label unlabeled data. Low confidence labels are discarded. The student is then trained on high confidence labels produced by the teacher. At the following “generation” the student becomes the teacher and the process is repeated iteratively. At each generation the confidence threshold used to filter out labels is lowered. The authors apply this technique here to the problem of ASR and use the SpecAugment the data augmentation technique. The results demonstrate improvements in word error rate for both clean and noisy recordings of the LibriSpeech corpus.</p><h4>Paper #5 — Conformer: Convolution-augmented Transformer for Speech Recognition</h4><p>When I saw the title of this paper, the 1992 track “Informer” by Snow started playing in my head and I had to add it! It turns out that this is another extremely compelling ASR paper from Google. The paper uses a model architecture which merges transformer and convolutional neural network (CNN) modules to combine the benefits of capturing long term interactions and localised feature representations, respectively. In addition to again achieving impressive results on the LibriSpeech corpus, these scores can be achieved with a model involving far fewer parameters than state-of-the-art model architectures.</p><h4>Paper #6 — ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context</h4><p>One last ASR paper from Google for good measure! The paper focuses on the problem of computational complexity of using transformer models for ASR with their multiple self-attention layers which get more expensive the longer the input audio duration. Like the “Conformer” paper, the authors here look to exploit the more computationally light nature of convolutional models. In this paper, the authors deal with the limitation of only local context being used by convolutional networks by incorporating the so-called “squeeze-and-excitation” approach used in computer vision. A global context is determined by average pooling of representations over time, which is then incorporated as a type of residual connection near the top of the overall network. Other tricks are used to reduce computational complexity including the use of depthwise separable convolution, as well as use of the “swish” activation function. The experimental results confirm the hypothesis that larger context reduces word error rate, but also that the proposed approach is an effective way of efficiently using large context with a far less computationally heavy model compared to the baseline.</p><h4>Paper #7 — Should we hard-code the recurrence concept or learn it instead? Exploring the Transformer architecture for Audio-Visual Speech Recognition</h4><p>You see relatively few audio-visual speech recognition papers, but this paper from the Trinity College-based Sigmedia group was a definite highlight. The paper deals with the thorny problem of effectively synchronising the asynchronous modalities of audio and in video inputs, in a way which effectively exploits the synergies between the two modes. To deal with this issue the researchers augment LSTM and transform architectures to include an “align” layer which involves concatenation of context vectors derived from separate attention layers applied to audio and video representations.</p><p>A harder problem still is combining these asynchronous signals in a streaming-based scenario which I hope to see in future work from this group.</p><h4>Paper #8 — Relational Teacher Student Learning with Neural Label Embedding for Device Adaptation in Acoustic Scene Classification</h4><p>Generalising over channel effects and different audio capture devices continues to be a challenge for audio and speech processing.<br>This paper leverages the student-teacher learning framework for acoustic scene classification. However, the vanilla student-teacher approach is not optimally suited when there is a device mismatch between source and target data<br>Here the authors propose a three step framework using neural label embeddings. In the first step a neural network model is trained using one-hot acoustic scene labels. The goal of the second step is to train the “neural label embeddings” which is essentially a dictionary of vectors, one for each acoustic scene. These weights are trained by minimising a symmetric Kullback Leibler (KL) divergence loss between the output of the model trained in step one and the projection output of the neural label embeddings applied to the source labels. The third step is intended to adapt the trained model to generalize better to the target device. The model here uses the weights learned in the first step, and these weights are fine-tuned through a training phase involving target input data and labels. The inclusion of the relational teacher student learning combined with KL divergence helps to better utilize small volumes of target data.</p><h4>Paper #9 — Abstractive Spoken Document Summarization using Hierarchical Model with Multi-stage Attention Diversity Optimization</h4><p>Although both extractive and abstractive summarisation techniques have proven to be effective at producing more concise versions of text documents, it is still very difficult to effectively summarise conversational speech. One of the main contributions of this paper is an attention mechanism, applied to a hierarchical encoder-decoder model architecture, which is used to mitigate the so-called “diversity problem” where summary outputs contain repeated words and phrases. The presented results show impressive improvements over conventional methods using the AMI conversational meeting corpus, where models are pre-trained using the CNN/Daily Mail dataset.</p><h4>Paper #10 — Towards Learning a Universal Non-Semantic Representation of Speech</h4><p>The final selected paper from Interspeech 2020 focuses on the problem of representational learning for non-ASR speech processing problems like speaker identification, speech emotion recognition and voice-based medical diagnosis. The paper here builds on some previous, recent semi-supervised learning work by Google Research which exploits the assumption that representations of snippets of audio that are temporally close should have a higher degree of similarity compared to snippets further away or from different contexts. The design here also leverages the reduced temporal granularity required for model outputs for non-ASR problems. The authors train a triplet loss network (which they refer to as TRILL) and use fine-tuning with small amounts of target data. Another major contribution is the proposal of a non-ASR speech processing benchmark dataset (which they refer to as NOSS), which combines commonly used datasets to do with speaker identification, language identification, emotion recognition and health condition detection. By fine-tuning the TRILL representation the substantial performance gains are observed in most of the detection problems included in NOSS.</p><h3>Onwards!</h3><p>I do cross my fingers that the <a href="https://www.interspeech2021.org/">next Interspeech</a> in Brno will again be with the speech community physically converging on the one setting. There is so much of the experience that is missing from virtual conferences. At the same time, I do wonder that once the pandemic lifts if the nature of scientific conferences will be changed irreparably. The virtual conference is indeed much more accessible to researchers around the world, so it could be that larger conferences like Interspeech go forward as at least a semi-virtual meeting.</p><p>With so much uncertainty and suffering in the world today, I cannot help feeling extremely grateful to be working in a field that is thriving as much as speech and language processing is. I think that Shehzad Mevawalla, from Amazon Alexa, captured this sentiment in the final line from his keynote presentation — “what a wonderful time to be a speech researcher”.</p><h3>Related articles</h3><ul><li><a href="https://medium.com/@CogitoCorp/icassp-2020-a-fully-virtual-meeting-of-the-signal-processing-community-20e76daebae2">ICASSP 2020 — A fully virtual meeting of the signal processing community — ICASSP 2020 conference summary</a></li><li><a href="https://medium.com/@CogitoCorp/interspeech-2019-machine-learning-enabled-creativity-and-innovation-in-speech-tech-30d46df482e8">Machine Learning-enabled Creativity and Innovation In Speech Tech — Interspeech 2019 conference summary</a></li><li><a href="https://medium.com/@CogitoCorp/a-simple-technique-for-fairer-speech-emotion-recognition-6952b3d8beea">A simple technique for fairer speech emotion recognition — Interspeech 2019 pre-conference blog post</a></li><li><a href="https://medium.com/@CogitoCorp/icassp-2019-towards-trustworthy-signal-processing-and-machine-learning-449e9dc6130f">Towards trustworthy signal processing and machine learning — ICASSP 2019 conference summary</a></li><li><a href="https://medium.com/@CogitoCorp/icassp-2018-redefining-signal-processing-for-audio-and-speech-technologies-5031d0f69d34">Redefining signal processing for audio and speech technologies — ICASSP 2018 conference summary</a></li><li><a href="https://medium.com/@CogitoCorp/interspeech-2017-robots-deep-neural-networks-and-the-future-of-speech-8a147106bf42">Robots, Deep Neural Networks and the Future of Speech — Interspeech 2017 conference summary</a></li><li><a href="https://www.isca-speech.org/archive/Interspeech_2019/pdfs/1708.pdf">Gender de-biasing in speech emotion recognition — Interspeech 2019 Cogito paper</a></li><li><a href="https://www.isca-speech.org/archive/Interspeech_2018/abstracts/1610.html">Attention-based Sequence Classification for Affect Detection — Interspeech 2018 Cogito paper</a></li></ul><h3>Opportunities at Cogito</h3><p>If you are based in the US, Ireland or India and are interested in opportunities at Cogito, please check out our <a href="https://cogitocorp.com/careers/">careers page</a>! We have an office optional policy that encourages remote work and collaboration!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=80445e381d01" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cogito-engineering/interspeech-2020-continued-advances-in-speech-and-language-processing-80445e381d01">Interspeech 2020 — continued advances in speech and language processing</a> was originally published in <a href="https://medium.com/cogito-engineering">cogito-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A simple technique for fairer speech emotion recognition]]></title>
            <link>https://medium.com/cogito-engineering/a-simple-technique-for-fairer-speech-emotion-recognition-903273734048?source=rss----7d6ba2b042a9---4</link>
            <guid isPermaLink="false">https://medium.com/p/903273734048</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[bias-in-ai]]></category>
            <category><![CDATA[speech-technology]]></category>
            <category><![CDATA[emotion-recognition]]></category>
            <dc:creator><![CDATA[CogitoEngineering]]></dc:creator>
            <pubDate>Thu, 15 Oct 2020 15:02:29 GMT</pubDate>
            <atom:updated>2021-03-31T08:08:39.593Z</atom:updated>
            <content:encoded><![CDATA[<p>Cogito’s algorithms are used to generate real-time feedback and guidance for call center agents, helping them become more self-aware and empathetic in how they approach and handle customer conversations. This promotes positive behavior change, leading to improvements in key performance indicators for companies across various industries. In order to deliver such a system at scale, it is essential to ensure that the machine learning models powering the product are fair across a large set of speakers from a wide variety of backgrounds. As a result, the topic of mitigating unfairness, or bias, is of central concern to our machine learning research and development efforts.</p><p>During the Interspeech 2019 conference in Graz, Austria — Cogito researchers presented a paper which looks at the phenomenon of gender bias in speech emotion recognition. We propose a simple model training procedure which is both effective at mitigating bias and is more stable during training than a highly cited baseline method.</p><p>By using a very standard neural network model, based on 2D convolutional layers applied to Mel frequency coefficients, trained to recognize emotional activation on a dataset of 33,000+ naturally occurring utterances from radio shows — we demonstrate that model performance is more favorable for male speakers compared to females. A popular de-biasing approach, previously proposed by researchers at Google and Stanford (see <a href="https://arxiv.org/pdf/1801.07593.pdf">paper</a>), is found to be effective at improving fairness across gender but at the cost of introducing highly unstable model training and reduced accuracy. In the figure below this “adversarial” approach produces a wide variance in accuracy metrics across multiple training rounds. Our method (referred to as “non-adversarial” in the below figure), on the other hand, produces consistently high accuracy while at the same time mitigating the effects of gender bias.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*aYpxN0pNeFAiu5XmQoQ8Dg.png" /><figcaption>Distributions of validation set concordance correlation coefficient (CCC) for recognizing emotional activation plotted as a function of de-biasing method. Distributions are produced by simply varying the random seed used for weight initialization and stochastic training operations.</figcaption></figure><p>A promising aspect of our proposed training procedure is that it can be applied to arbitrary variables, therefore serving as an effective strategy for mitigating bias against a host of sensitive demographic categories beyond gender.</p><p>Of course, machine learning techniques are only one part of the solution. Consequently, our research continues to focus on developing more holistic protocols which encompass data collection, human annotation and production monitoring of model accuracy, in order to deliver fairer machine learning models at scale.</p><h3>Opportunities at Cogito</h3><p>If you are based in the US, Ireland or India and are interested in opportunities at Cogito, please check out our <a href="https://cogitocorp.com/careers/">careers page</a>! We have an office optional policy that encourages remote work and collaboration!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=903273734048" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cogito-engineering/a-simple-technique-for-fairer-speech-emotion-recognition-903273734048">A simple technique for fairer speech emotion recognition</a> was originally published in <a href="https://medium.com/cogito-engineering">cogito-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[13 best practices to improve your logging]]></title>
            <link>https://medium.com/cogito-engineering/13-best-practices-to-improve-your-logging-75665183f05b?source=rss----7d6ba2b042a9---4</link>
            <guid isPermaLink="false">https://medium.com/p/75665183f05b</guid>
            <category><![CDATA[operations]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[sumo-logic]]></category>
            <category><![CDATA[logging]]></category>
            <dc:creator><![CDATA[CogitoEngineering]]></dc:creator>
            <pubDate>Thu, 15 Oct 2020 14:54:52 GMT</pubDate>
            <atom:updated>2021-03-31T08:07:19.733Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XV1dMjhRiMSIZMpSesXf_Q.png" /></figure><h3>Introduction</h3><p>Troubleshooting distributed applications can be quite difficult. When you add real-time streaming, message queues, asynchronous processing of events and a complex mesh of microservices into the mix, the level of difficulty goes up. Combine this with several hundred of gigabytes (or terabytes) of daily logging data from which you need to find the issue, and deciphering those cryptic log messages written by various developers over the years. This is the daily challenge that many engineers are facing, especially if your role includes operational responsibilities.</p><p>This article provides some insights on how to improve your logging practices in order to make troubleshooting sessions more productive.</p><h3>Overview</h3><p>We have implemented a centralized log monitoring and analytics system using <a href="https://www.sumologic.com/">Sumo Logic</a>. We collect logs from EC2 instances using agents that are deployed and configured using Puppet. On our Kubernetes clusters we collect logs from pods and Sumo agents (FluentBit and FluentD) is configured and deployed by Helm charts. We also leverage hosted collectors to ingest logs and custom metrics from various sources to monitor performance of our machine learning models in production, like Kubernetes control planes, AWS CloudTrail, CloudWatch, Aurora RDS services, and even S3 buckets containing datafeeds.</p><p>This type of centralized system allows scalability of logging. We create new AWS accounts and spin up new environments frequently using automated tools. The infrastructure code provides the metadata related to a particular customer, location, environment level, service, component, application, hostname, source name, etc. that is used to configure the collectors accordingly. As a new customer environment with hundreds of instances comes up, engineers can start observing logs and metrics immediately.</p><p>The centralized logging system provides a “single-pane-of-glass” to perform log monitoring of production systems. It provides the ability to run complex search queries that enable on-call engineers to quickly identify issues. We have shared commonly used search queries that give insights to engineers on usage patterns, but also help to find anomalies. The ability to compare data between multiple running systems is used to narrow down system integration issues on the customer side, such as Single-Sign-On, Computer Telephony Integration (CTI) feeds, or real-time audio feeds that the Cogito platform depends on.</p><p>The following recommendations are based on hands-on experience troubleshooting issues in Cogito production environments.</p><h4>1. Semantic logging</h4><p>Putting more semantic meaning in the logged events allows to get more value out of our log data. We log audit trails, what users are doing, transactions, timing information, data flows, and so on. We log anything that can add value when aggregated, charted, or further analyzed. In other words, we log anything that is interesting!</p><p>Think about the five W’s:</p><ol><li>When did it happen (timestamps, etc.)</li><li>What happened (actions, state or status changes, error codes, impact level, etc.)</li><li>Where did it happen (hostnames, gateways, etc.)</li><li>Who was involved (user names, client IDs, process names, session IDs, etc.)</li><li>Where it came from (source IPs, telephony extensions, etc.)</li></ol><h4>2. Use Developer-Friendly Log Formats</h4><p>Using log formats like JSON is beneficial as these are readable by humans and machines. This lets us to aggregate and create views and graphs of the log data to answer business questions, such as “how many users were logged in when the incident happened?”, “how many calls were in progress when we lost call guidance?”, or “how many extensions on customer sites were impacted by this error?” With JSON formatted log messages it is easy to parse the fields and aggregate or count events. Sumo Logic provides a rich set of operators that can be combined to create complex queries to answer the questions we get from our Customer Support team members.</p><p>For example the following Sumo query creates a report on annotation durations by user by date sorted by users and time, accessing and parsing JSON fields in the log messages:</p><pre>_sourceCategory=&quot;/cogito/cloud/prod/annotation&quot; <br>| replace(_raw,&quot;&#39;&quot;, &quot;\&quot;&quot;) as message<br>| parse field=message &quot;* Saving labeldata by user:* for call:* labels:*&quot; as date,  user, call, dict<br>| json auto field=dict maxdepth 1 nodrop<br>| fields  timesubmitted, timeclipaccessed,  user <br>| parseDate(timesubmitted,&quot;yyyy-MM-dd&#39;T&#39;HH:mm:ss&quot;) as TS<br>| parseDate(timeclipaccessed,&quot;yyyy-MM-dd&#39;T&#39;HH:mm:ss&quot;) as CA<br>| TS - CA as duration<br>| duration/(3600*1000) as hours<br>| timeslice 1d<br>| sum(hours) as mytotal by user, _timeslice<br>| sort by user, _timeslice asc</pre><h4>3. Use a consistent format for all timestamps</h4><p>Logs are easier to correlate cleanly when using a consistent timestamp format and time zone. We have settled on a UTC standard format based on ISO 8601. This allows to build log search queries to correlate events across multiple systems within a given time period. Typically these queries are used to find where the observed problem originated and the sequence of events that led to this particular problem. Having inconsistent formatting in timestamps, or relying on the timestamp, created at the log ingestion time makes these types of queries more difficult and less valuable for troubleshooting.</p><h4>4. Configure Logging and Metadata Tagging Correctly</h4><p>To facilitate efficient search queries, we have configured our log collectors with specific patterns that tag the log messages and export a standard set of log files. The metadata provides the ability to search and compare logs across:</p><ul><li>customers</li><li>locations</li><li>environments</li><li>services</li><li>components</li><li>applications</li><li>host names</li><li>log source names</li><li>different types of log files</li></ul><p>This capability is very valuable as it allows us to compare a set of standard log search queries and find patterns across different dimensions. For example we can compare the behavior of a particular component across multiple customers to find differences in usage patterns, or run a query across all hostnames in a cluster for a particular service so we can see if one of the hosts is having a bad day.</p><h4>5. Enable Tracing of Dataflows</h4><p>There are many different kind of frameworks that provide generic tracing capabilities across distributed systems, like <a href="https://aws.amazon.com/xray/">AWS X-ray</a> or <a href="https://github.com/jaegertracing/jaeger">Jaeger</a>. These are certainly valuable tools for anomaly detection, diagnosing steady state problems, distributed profiling, resource attribution, and workload modeling of microservices.</p><p>However, a simple UUID as part of the payload in log messages enables tracing of dataflows between components. These UUIDs can be used to correlate a particular dataflow (call, session, transaction, etc.) across all the components involved with this dataflow and find problems in the system. Combining the UUID with other metadata allows to narrow down the component that has the problem. We have a set of shared common log queries that use this method using callIDs to quickly identify issues in production.</p><h4>6. Identify the Source Location Info</h4><p>Sometimes in troubleshooting you have to read the source code when trying to understand how the system is supposed to work. Including the reference to the source code of the log event (such as class, function, filename) can help to find the corresponding section in the source faster. Many logging frameworks have the capability to show even the line number of the code where a particular log message was created. This is a useful capability when troubleshooting a system that the engineer is not very familiar with. However, this practice may cause performance penalties as looking up the current thread, file, or method is a costly operation in some languages.</p><h4>7. Personally Identifiable Information (PII)</h4><p>To be in compliance with General Data Protection Regulation (<a href="https://gdpr-info.eu/">GDPR</a>) or California Consumer Privacy Act (<a href="https://oag.ca.gov/privacy/ccpa">CCPA</a>), it is important to be careful about what kind of PII data goes into logs, and what kind of retention policies your company has established for storing the log data. You may need to create a separate log partition for PII data that has a retention policy that meets the GDPR or CCPA requirements. It is a good idea to check with your security and/or legal teams to fully understand the requirements and constraints of these laws when dealing with PII. Logging PII may be required for your application, but you need to understand the implications and consequences.</p><h4>8. Avoid Stack Traces by Default</h4><p>Dumping stack traces into logs is oftentimes significantly more information than is needed, and does little to help operations personnel quickly take action. In the event that the log is exposed publicly, stack traces provide attackers with additional information that could be used to compromise the system. Stack traces should only be available when running logging at the TRACE level, and hopefully you never have to troubleshoot a high volume production system with TRACE level turned on.</p><h4>9. Never Log Secrets</h4><p>This should be obvious, but human mistakes can happen. Production logs may have a retention policy for over one year due to various regulatory reasons. If some secrets are logged by accident this may require a lot of extra work to replace these secrets in the respective production system(s).</p><h4>10. Cost of Logging</h4><p>Some commercial log management and analytics services charge by the ingested log volume. You may need to consider the default logging levels in production. A single production system with DEBUG level logging turned on can produce hundreds of gigabytes or terabytes per day, and you can end up with a large bill for logs that did not produce any business value. Having awareness of the state of logging in your production systems and being mindful of not leaving DEBUG level turned on unnecessarily will help to reduce the cost of logging.</p><h4>11. Avoid Logging Unnecessary Health Checks</h4><p>Applications that expose a health check should provide the ability to prevent that health check from getting logged, either via configuration or request parameter. You may see a lot of needless log entries making their way to your logging system that do not offer any intrinsic value at the scale at which they are received.</p><h4>12. Provide Runtime Ability to Change the Logging Level</h4><p>Sometimes we need to get more detailed logging (DEBUG or TRACE) to debug a production issue. Having the capability to change the logging level on demand while the system is running without a need for a reboot of the service is very beneficial to help in troubleshooting.</p><h4>13. Use Severity Categories Correctly</h4><p>The severity values may have different meanings for a developer compared to an operations person. Think severity from the operations perspective — what action do you want the person reading the logs to take? If your software component recovers from ERROR without human actions, should the log event be WARN or INFO instead? If your component emits millions of ERROR log messages daily when the system is operating nominally, are you sending the correct message to your operations team? If you refactor your software to add some error recovery, did you change your log messages as well?</p><h3>Conclusion</h3><p>Logging produces a high volume stream of unstructured machine data on your application and infrastructure stack. This data can be transformed into rich, actionable insights and with proper queries can be used to answer questions that have direct impact to your customer service level and satisfaction.</p><h3>Opportunities at Cogito</h3><p>If you are based in the US, Ireland or India and are interested in opportunities at Cogito, please check out our <a href="https://cogitocorp.com/careers/">careers page</a>! We have an office optional policy that encourages remote work and collaboration!</p><h3>Acknowledgements</h3><p>This article was written Mauri Niininen as part of an effort to improve logging standards. A special thanks to Khalid Hasanov and Ian Kelly for their feedback and to Richard Brutti for proofreading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=75665183f05b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cogito-engineering/13-best-practices-to-improve-your-logging-75665183f05b">13 best practices to improve your logging</a> was originally published in <a href="https://medium.com/cogito-engineering">cogito-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using NATS as a messaging technology for real-time compute systems]]></title>
            <link>https://medium.com/cogito-engineering/using-nats-as-a-messaging-technology-for-real-time-compute-systems-b3b90594e81e?source=rss----7d6ba2b042a9---4</link>
            <guid isPermaLink="false">https://medium.com/p/b3b90594e81e</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[nats-streaming]]></category>
            <category><![CDATA[cloud-native]]></category>
            <dc:creator><![CDATA[CogitoEngineering]]></dc:creator>
            <pubDate>Thu, 15 Oct 2020 14:22:57 GMT</pubDate>
            <atom:updated>2021-03-31T08:06:02.479Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/460/1*ZmBWab8bx1zexvLdyuV6Mw.png" /></figure><h3>Introduction</h3><p>A major part of the Cogito platform is to do with orchestrating and executing real-time compute jobs. To help enable this we needed to select a messaging technology which meets the following design constraints:</p><ul><li>Scale to hundreds of thousands of concurrent processing jobs</li><li>Have extremely low latency, so as not to negatively impact on user experience</li><li>Ensure enterprise-grade security</li><li>Enable robust processing — messages cannot get missed and failing jobs need to be retried</li><li>Be resilient and highly available</li><li>Be compatible with microservices written in Go</li><li>Be suited to the Kubernetes and Cloud Native Computing Foundation ecosystem</li><li>Enable sending different types of compute jobs to different types of supporting processing systems</li><li>Be suited to creating dynamic channels for real-time systems</li><li>(Preferably) be lightweight with no external dependencies</li></ul><p>After much experimentation with various technologies we opted to go with NATS. This article provides an overview of NATS and discusses the various challenges we encountered with scaling it and having it meet the design constraints listed above.</p><h3>NATS overview</h3><p><a href="https://nats.io/">NATS</a> is an open-source messaging technology and is part of the Cloud Native Computing Foundation project. Its main selling point is its simplicity to deploy within modern container-based, microservices architectures.</p><p>NATS has clients in a variety of programming languages. Given our extensive use of Go for server layers in our compute microservices, we used <a href="https://github.com/nats-io/go-nats">go-nats</a>.</p><p>NATS streaming (aka STAN) is a service layered on top of plain NATS. Its design goals were to ensure messages are not lost with message persistence and replaying functionality. It enables durable subscription so that inactive subscribers receive the message when they once again become active.</p><h3>Challenges</h3><p>This section describes some of the challenges we had to overcome when implementing this messaging system to functionally enable our use-case and to scale appropriately.</p><h4>NATS vs NATS streaming</h4><p>The first decision we had to make was whether to go with NATS or NATS streaming. After experimenting with both, NATS streaming appeared to be the obvious choice, in particular because of guarantees of message delivery (i.e. “at least once delivery”, compared to “at most once delivery” with plain NATS).</p><p>NATS Streaming can be deployed in four modes, and we had to decide which one to use.</p><ol><li><strong>Standalone mode</strong> — backed by memory, file, or SQL database</li></ol><p><em>Pros</em></p><ul><li>An easy way to do a quick proof-of-concept with one instance of NATS Streaming</li></ul><p>Cons</p><ul><li>Not suited to any serious production deployment, as it does not run in high availability (HA)</li></ul><p>2. <strong>Cluster mode</strong> — HA mode with local file storage. This relies on the <a href="https://raft.github.io/">Raft protocol</a> for distributed consensus</p><p><em>Pros</em></p><ul><li>Provides HA deployment and can stand for a failure as long as there is a quorum. A quorum is a majority of members from a peer set: for a set of size <em>N</em>, quorum requires at least <em>(N/2)+1</em> members. For example, if there are 5 members in the peer set, we would need 3 nodes to form a quorum. If a quorum of nodes is unavailable for any reason, the cluster becomes unavailable and no new logs can be committed.</li></ul><p><em>Cons</em></p><ul><li>Does not support channel partitioning and is not suitable for large message sizes. From our experiments we observed that Raft starts frequent leader elections when the message size reaches 1 Mb and the rate is 50 per second. It is still possible to tune the Raft configurations, by changing <em>raft_hearbeat_timeout</em>, <em>raft_lease_timeout</em>, and <em>raft_election_timeout</em> cluster configurations to aim for large message sizes. By increasing the mentioned timeouts to 2 minutes the cluster mode performed several times better than fault tolerance mode for message sizes of 1 Mb with a rate of 50–200 per second.</li></ul><p>3. <strong>Fault tolerance mode</strong> — relies on a shared file system or a shared SQL database</p><p><em>Pros</em></p><ul><li>Provides HA deployment, does not need <em>(N/2)+1</em> members available as it is not based on consensus.</li><li>It can also be combined with the partitioning mode to provide horizontal scalability</li></ul><p><em>Cons</em></p><ul><li>It requires an external dependency such as a shared file system or an SQL store and relies on the availability of the external store. It may be slower than the clustering mode for small message sizes.</li></ul><p>4. <strong>Partitioning mode</strong> — provides capabilities to partition channels among different servers</p><p><em>Pros</em></p><ul><li>Shares the load between several servers running with the same cluster ID</li></ul><p><em>Cons</em></p><ul><li>Is better suited to clients produce and consume from separate channels</li></ul><p>Considering the pros and cons of each deployment mode and the results of our benchmarking experiments, we initially decided to use NATS Streaming in cluster mode. This mode makes the deployment simpler and does not introduce another external dependency. However, after further consideration and testing we found that for use-cases where we had reasonably large messages “fault tolerance” mode was found to be more performant, corroborating findings described in <a href="https://carlosbecker.com/posts/nats-streaming-server-ft">this article</a>.</p><p>For our particular use case, we have clusters of different types of processing workers. So we use channels to indicate which worker type should be used to start a processing job. We do not really care what replica takes the job so long as it is the appropriate worker type. Things become a little more complicated, however, when we want to terminate the processing job. Here it is not sufficient to refer to just the worker type, so we have to point to the exact replica which is executing the processing. So for termination, we need to create a new channel for every new processing job.</p><p>Due to the persistence capability of NATS streaming, a file structure is created for each new channel using a shared file system. Amazon Web Services (AWS) has limits (see <a href="https://docs.aws.amazon.com/efs/latest/ug/limits.html#limits-client-specific">this documentation</a>) on the number of separate file locks for EFS file systems. As a result of this, and due to the scale requirements we need to meet running our platform on AWS, NATS streaming was deemed not to be suitable for terminating jobs. To deal with this, we decided to use plain NATS for orchestrating termination of processing jobs.</p><h4>Request/reply functionality with NATS</h4><p>As mentioned previously, plain NATS enables “at most once delivery” of messages. From our experimentation, we found that messages were missed with a relatively high frequency which clearly does not meet our design requirements.</p><p>To deal with this, when using plain NATS we use the request/reply functionality (see the schematic of this below).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/449/1*Tgm11Tx7dCbHzcl49ew-zA.png" /><figcaption>NATS request/reply functionality. Taken from <a href="https://docs.nats.io/nats-concepts/reqreply">NATS documentation</a></figcaption></figure><p>The Golang code snippet below demonstrates how to implement request/reply by the publisher. This uses the <a href="https://github.com/nats-io/stan.go">Go client for NATS Streaming</a>, and specifies the `channel`, the message data in the `payload` and a `timeout` variable to indicate how long to wait for a reply from the subscriber. In this example code, we get the embedded NATS connection from the existing STAN connection.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/807004ff812edb774f989a1397825715/href">https://medium.com/media/807004ff812edb774f989a1397825715/href</a></iframe><p>On the subscriber side, the following Golang code can be used to acknowledge receipt of the message.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/51b603140cf6f892e28e023e6eff88a9/href">https://medium.com/media/51b603140cf6f892e28e023e6eff88a9/href</a></iframe><p>Request/reply with plain NATS helps us guarantee that the job termination messages are received by the specific replica subscriber.</p><h4>Secure communication using mTLS</h4><p>A further design requirement was to ensure that we have secure message communication. NATS and NATS streaming offer multi-tenant functionality as well as authentication. However, we do not use NATS authentication mainly because we do not expose NATS as a service outside our cluster. We use the upstream <a href="https://github.com/nats-io/k8s/tree/master/helm/charts/stan">Helm chart</a> provided by NATS, however we needed to customize this considerably as it did not appear to be well maintained. We extended it to make it work properly with Kubernetes StatefulSet and made NATS Streaming storage options configurable from the Helm chart. We use ISTIO proxy containers running in all our Kubernetes pods which ensure that inter-service communication is done via mutual TLS (mTLS) which ensures secure message communication.</p><h4>Metrics scraping with Prometheus</h4><p>We use Prometheus to scrape metrics from running containers. We then use this data to power dashboards for system monitoring and as input to alert generation to enable effective production support. The <a href="https://github.com/nats-io/prometheus-nats-exporter">Prometheus NATS exporter</a> helps to aggregate metrics from NATS servers and send to Prometheus.</p><h3>Summing up and moving forward</h3><p>After overcoming the various challenges to do with scaling and performance, we found that NATS and NATS streaming to be a lightweight, secure and efficient messaging system. They are very well suited to orchestrating real-time processing jobs in Cloud-native, microservices architectures, but we needed to use both in combination in order to meet our use-cases.</p><p>We are looking forward to the release of <a href="https://github.com/nats-io/jetstream">Jetstream</a> later on this year, which will involve incorporating the persistence functionality of NATS streaming into NATS. Besides this, Jetstream offers the potential to more effectively implement horizontal autoscaling which is either complicated or impossible depending on the deployment mode used with the current version of NATS streaming. As a result, we will need to revisit the design of NATS streaming / NATS in combination, which will likely lead to a more straightforward messaging architecture.</p><h3>Opportunities at Cogito</h3><p>If you are based in the US, Ireland or India and are interested in opportunities at Cogito, please check out our <a href="https://cogitocorp.com/careers/">careers page</a>! We have an office optional policy that encourages remote work and collaboration!</p><h3>Acknowledgements</h3><p>This article was written by Khalid Hasanov and was based on important research and development work in 2020 by Khalid, Ian, Luis and Mihai. Thanks also to Richard Brutti and Ian Kelly for the detailed proofreading and feedback.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b3b90594e81e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cogito-engineering/using-nats-as-a-messaging-technology-for-real-time-compute-systems-b3b90594e81e">Using NATS as a messaging technology for real-time compute systems</a> was originally published in <a href="https://medium.com/cogito-engineering">cogito-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>