<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Gennaro S. Rodrigues on Medium]]></title>
        <description><![CDATA[Stories by Gennaro S. Rodrigues on Medium]]></description>
        <link>https://medium.com/@gennsev?source=rss-603bda3d0d35------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*IQyMsdwhXEE9zgxcR9s60A.png</url>
            <title>Stories by Gennaro S. Rodrigues on Medium</title>
            <link>https://medium.com/@gennsev?source=rss-603bda3d0d35------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 09:05:41 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@gennsev/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Future of Work With AI: Beyond Hype and Fear]]></title>
            <link>https://ai.plainenglish.io/the-future-of-work-with-ai-beyond-hype-and-fear-4662b8bc3e2a?source=rss-603bda3d0d35------2</link>
            <guid isPermaLink="false">https://medium.com/p/4662b8bc3e2a</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[business]]></category>
            <category><![CDATA[management]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[workplace]]></category>
            <dc:creator><![CDATA[Gennaro S. Rodrigues]]></dc:creator>
            <pubDate>Thu, 12 Oct 2023 06:17:27 GMT</pubDate>
            <atom:updated>2023-10-12T19:49:00.337Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*swNfWemiP476UU7cTF2-Rw.png" /><figcaption>(Image generated by DALL·E)</figcaption></figure><h4>The true impact and role of AI in shaping our work future</h4><p>AI has evolved to be more than a tool for solving engineering problems and developing tech solutions. It is now a universal tool. Whether most people feel comfortable using it (or know how to) is a great question. How fast they will get comfortable with it is even more critical. It is one of the most significant performance improvement tools to arise in the last decades, so it is a matter of adapting or going extinct. Further, will business leaders use it to improve the workplace or merely reduce headcount?</p><p>This article will discuss those questions. I will also try to convince you that AI is an indispensable tool for the workspace of the future — or, really, the present times.</p><h3>A Brief Overview of Data Science Evolution</h3><p>Only a few years back, we began to notice something intriguing. Engineers and scientists, when equipped with the right mindset and profile, morphed into exceptional consultants. Over time, these professionals began to incorporate their exact science skills into consultancy work. This integration ranged from data analysis to statistics and slowly evolved into more intricate applications, such as market predictions and macro and microeconomic forecasts. This was the birth of <em>data science</em> and the journey that led us to embrace <em>machine learning</em>. In essence, <em>machine learning</em> involves using data to train <strong>AI models</strong> to solve problems, a process that has become more efficient and powerful over time.</p><p>Like computers in the late 1970s, <strong>AI has evolved into a universal tool</strong>. In the 70s, people other than computer engineers had to learn how to use and adapt to computers. Now, the same is happening with AI. But are they ready for it? It is clearly a game changer regarding work performance, but how fast can people adapt and embrace this evolution?</p><p>Well, apparently, it is more than a revolution and a performance-improving tool: it is a must for the future of work if we intend to keep improving our lives.</p><h3>Market Challenges and Solutions</h3><p>Recent research from Indeed, Glassdoor, and data sourced from the World Bank [1] highlighted a concerning trend: a significant decline in available labor. The primary reason is demographics, with the world’s population aging. This demographic challenge results in hiring becoming more challenging, labor costs increasing, and heightened demands on the workforce.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/627/1*mQ4WN-Yd70ABBDtByiBrTw.png" /><figcaption>Population estimate growth from 2026 to 2036, % of people age 15–65 (Source: [1])</figcaption></figure><blockquote>Fewer people of working age mean the supply of workers will dwindle. Combine this aging population with other trends, such as reduced immigration, and the stage is set for chronic recruiting challenges. Source: [1]</blockquote><p>Furthermore, the active labor force percentage is in decline in most significant economies, even China. This trend propels the idea of machines replacing human labor, especially as the aging population requires support.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/662/1*7wQhWUSJ-iRnP7DHrT4BWA.png" /><figcaption>Percentage of the total projected population age 25–54 by country (Source: [1])</figcaption></figure><p>However, instead of simply replacing humans, technology, particularly AI, can amplify productivity without causing unemployment. Given the current labor shortage, enhancing productivity using technology may be our best solution.</p><blockquote>An aging world needs a workforce large enough, or performatic enough, to maintain it. AI can be the solution.</blockquote><h4>The AI potential and a new skill set</h4><p>A recent research by LinkedIn[2] emphasized the potential of AI, especially generative AI, to create content such as images, audio, and text. They found that nearly half of American executives see AI as a means to boost productivity significantly, and 44% plan to increase their AI usage in the coming year. The research also shows that people are developing their AI use skills steadily, with people adding AI-related skills to their LinkedIn profiles.</p><blockquote>Job postings referencing new AI technologies also are climbing rapidly: The share of global English-language job postings mentioning GPT or ChatGPT increased 21x since November 2022. Professionals and businesses are starting to embrace AI-assisted technologies, such as ChatGPT, in their daily work and adapt their processes. Source: [2]</blockquote><p>The following image shows how AI skills have been proliferating across the world. As expected, some countries show a more steady advance than others, but there is a clear worldwide trend. LinkedIn users from all around the globe have been adding AI-related skills to their profiles at a fast pace.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*t9vUgcdZ5_ab3ErDfBgq6A.png" /><figcaption>Share of LinkedIn members who have added AI-related skills to their profile across countries (Source: [2])</figcaption></figure><p>In the US, the Technology, Information, and Media sector unsurprisingly holds the predominant portion of AI-skilled individuals, at 2.2%. This is substantially higher compared to sectors like Education at 1.2%, Professional Services at 0.9%, Financial Services at 0.9%, and Manufacturing trailing at 0.8%. However, a paradigm shift is evident as other industries show accelerated adaptation rates. Analyzing the trajectory at which AI skills are being integrated into professional profiles, it’s surprising to see that individuals within Financial Services (30x), Retail (29x), and Wholesale (24x) are incorporating AI competencies at a rate surpassing even Technology, Information, and Media (11x).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sCXrrp6MOi3EA0MAxG16qA.png" /><figcaption>Share of LinkedIn US members who have added AI-related skills to their profile across industries (Source: [2])</figcaption></figure><p>Upon examining ten prominent US industries (see image above), Financial Services emerges distinctively, being the sole sector where the proportion of AI-skilled professionals and the rate of AI skill acquisition surpass industry averages. This trend highlights the potential of non-traditional tech sectors not only being early AI adopters but also important contributors to AI-driven innovation in the workplace.</p><h4>AI as a tool</h4><p>Another survey by Microsoft [3] showed three main insights:</p><ol><li><strong>Technical debt is costing innovation</strong>:<strong> </strong>The research identified that the technical debt in the tech sector is stunting innovation. Companies must invest in upskilling their workforce and upgrading their tech stacks to capitalize on AI’s potential fully.</li><li><strong>There is an alliance between AI and employees</strong>: Contrary to the fear of AI taking over jobs, the research showcased a beneficial relationship between AI and workers. There’s a clear indication that workers are receptive to AI and actively looking for ways AI can complement their roles.</li><li><strong>It is imperative to learn AI</strong>: As AI becomes more integrated into various sectors, there’s a growing need for everyone, from tech professionals to non-tech roles, to understand and use AI as a tool. This doesn’t mean everyone should become AI developers, but having a foundational understanding of AI’s capabilities and applications can significantly augment job roles.</li></ol><p>The research offers a counter-narrative to the fear of AI replacing jobs. It revealed that while 49% of individuals fear job loss due to AI, about 70% are comfortable delegating tasks to AI to reduce their workload and enhance productivity. These numbers show that people are more motivated by improving productivity than they fear AI-induced job loss.</p><blockquote>Not only did 3 in 4 people share that they would be comfortable using AI for administrative tasks (76%), but most people also said they would be comfortable using it for analytical (79%) and even creative work (73%). Source: [3]</blockquote><p>Business leaders feel the same: the research indicates that, from all of the capabilities and opportunities brought by AI, the idea of using it to reduce headcount is far from the most common one on the heads of employers. Indeed, executives prefer to use it as a tool to improve performance, as the following image depicts. Further, business leaders are focused on finding ways of using AI to improve their employee&#39;s well-being, eliminating their time spent on low-value and boring activities, acquiring more knowledge, and increasing inclusivity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0l0sFRM8svnpROKTTJFXqw.png" /><figcaption>What business leaders most value about AI in the workplace (Source: [3])</figcaption></figure><blockquote>Amid fears of AI job loss, business leaders are 2x more likely to choose ‘increasing employee productivity’ than ‘reducing headcount’ when asked what they would most value about AI in the workplace. Source: [3]</blockquote><h3>Hype?</h3><p>The Gartner Hype Cycle, an annual analysis, visually represents the maturity, adoption, and application of specific technologies, helping industry leaders discern between what’s mere hype and what holds genuine promise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZEyzfVUEX0iTTIYJgnED9A.png" /><figcaption>The Hype Curve for 2023 (Source: Gartner)</figcaption></figure><p>This year’s curve places technologies like generative AI and cloud computing at its peak of inflated expectations. This positioning suggests that these technologies are currently receiving a lot of attention and thus might be surrounded by over-enthusiasm. There’s a risk of potential disillusionment in the short term. However, as the curve historically indicates, after this potential desilusion, a period of enlightenment and productivity typically follows. For industry leaders and innovators, understanding this cycle can be invaluable in strategizing investments and anticipating market shifts.</p><h3>Concluding thoughts</h3><p>The interplay between AI and the labor market, accentuated by demographic shifts and technological advancements, paints a complex but promising picture. There’s a balance between AI’s genuine potential and its inflated expectations. The future of work is undoubtedly evolving, with AI playing an important role. However, while the technology offers great possibilities for improving productivity and filling labor gaps, it’s essential for industry leaders to approach it with a well-informed perspective, tempering enthusiasm with practicality. As we navigate this AI-augmented landscape, continuous learning, adaptability, and strategic foresight will be paramount for individuals and organizations. This technology, as any, has its limits. Let us hope we’ll reach it wisely.</p><h3>References</h3><p>[1] <strong>Indeed &amp; Glassdoor’s Hiring and Workplace Trends Report, </strong>2023. Link: <a href="https://www.indeed.com/lead/indeed-glassdoor-hiring-and-workplace-trends-report-2023">https://www.indeed.com/lead/indeed-glassdoor-hiring-and-workplace-trends-report-2023</a></p><p>[2] <strong>Future of Work Report AI at Work, </strong>August 2023. Link: <a href="https://economicgraph.linkedin.com/research/future-of-work-report-ai">https://economicgraph.linkedin.com/research/future-of-work-report-ai</a></p><p>[3]<strong> 2023 Work Trend Index: Annual Report: Will AI Fix Work? </strong>Link: <a href="https://www.microsoft.com/en-us/worklab/work-trend-index/will-ai-fix-work">https://www.microsoft.com/en-us/worklab/work-trend-index/will-ai-fix-work</a></p><h3>In Plain English</h3><p><em>Thank you for being a part of our community! Before you go:</em></p><ul><li><em>Be sure to </em><strong><em>clap</em></strong><em> and </em><strong><em>follow</em></strong><em> the writer! 👏</em></li><li><em>You can find even more content at </em><a href="https://plainenglish.io/"><strong><em>PlainEnglish.io</em></strong></a><strong><em> 🚀</em></strong></li><li><em>Sign up for our </em><a href="http://newsletter.plainenglish.io/"><strong><em>free weekly newsletter</em></strong></a><em>. 🗞️</em></li><li><em>Follow us on </em><a href="https://twitter.com/inPlainEngHQ"><strong><em>Twitter</em></strong></a><strong><em>(X</em></strong>), <a href="https://www.linkedin.com/company/inplainenglish/"><strong><em>LinkedIn</em></strong></a>, <a href="https://www.youtube.com/channel/UCtipWUghju290NWcn8jhyAw"><strong><em>YouTube</em></strong></a>, and <a href="https://discord.gg/in-plain-english-709094664682340443"><strong><em>Discord</em></strong></a><strong><em>.</em></strong></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4662b8bc3e2a" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/the-future-of-work-with-ai-beyond-hype-and-fear-4662b8bc3e2a">The Future of Work With AI: Beyond Hype and Fear</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Attention to LLM Architectures: An Intelligent ML Engineering Guide]]></title>
            <link>https://ai.plainenglish.io/attention-to-llm-architectures-an-intelligent-ml-engineering-guide-2298c5d70a58?source=rss-603bda3d0d35------2</link>
            <guid isPermaLink="false">https://medium.com/p/2298c5d70a58</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Gennaro S. Rodrigues]]></dc:creator>
            <pubDate>Mon, 09 Oct 2023 02:51:22 GMT</pubDate>
            <atom:updated>2023-10-11T01:51:19.201Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*udA-5l7VFhtjxpiww0fC_Q.png" /><figcaption>(Image generated by DALL·E)</figcaption></figure><h4>Attention might be all you need, but do you need all of it?</h4><p>In the age of prompt engineering, is there still room for model training and development, or shall we just bend our kneels to GPT and its lookalikes?</p><p>Large language models (LLMs) have become incredibly popular in recent years, with models like GPT generating human-like text. Under the hood, these models use different architectures optimized for different purposes.</p><p>Although the traditional Encoder-Decoder model (exemplified by Google&#39;s BERT) is more complex than their Decoder-only counterparts, OpenAI&#39;s GPT achieves exceptional performance following the later configuration. How come a less complex model is better than a more complex one? Further, given the incredible capacities of GPT, one might be tempted to use it even for the most basic NLP tasks, such as text classification and sentiment analysis. But should you do it?</p><p>In this article, we’ll see how the model architecture that introduced the basics of attention and transformers is used and how we can tear it apart to achieve better and more viable results for each particular use case.</p><h3>Transformers and Encoder-Decoder Models</h3><p>The encoder-decoder architecture contains two main components — an encoder and a decoder (duh). Encoder-decoder models are commonly used for sequence-to-sequence tasks like translation and summarization. The Transformer model, proposed in the famous paper “Attention Is All You Need” [1], is a specific instance of the Encoder-Decoder architecture that uses self-attention mechanisms to improve on the shortcomings of predecessor models like LSTMs and GRUs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/531/1*3pt4caWEST8QYrl75L5dlA.png" /><figcaption>The Transformer Encoder-Decoder Architecture (Source: [1])</figcaption></figure><p>In the figure above, the <em>Encoder</em> is the block on the left side, and the <em>Decoder</em> is the block on the right. The <em>Encoder</em> takes in the input text and encodes it into a dense vector representation. We can think of this encoded representation as the <strong>semantic meaning </strong>and the<strong> context </strong>of the input. The <em>Decoder</em> then takes this representation and generates the output text token-by-token. That is why, in the figure, you can see <em>Outputs</em> being inputted into the <em>Decoder</em>: this represents the output tokens being inserted back at it, one by one. Naturally, the <em>Decoder</em> needs to know when to start and when to stop: those are represented by special tokens that define the start and the end of a text.</p><h3>Encoder-Only Models</h3><p>Encoder-only models contain only the encoder component. These models encode the input text into a semantic vector representation but do not generate any output text. Instead of connecting the encoder output to a decoder, its output is then connected to a head trained for one particular use case (normally not text generation).</p><p>Those models are useful for feature extraction from text. They are trained on tasks that do not require sequence-to-sequence mapping. The final output from the encoder can be used in various ways depending on the task — such as token classification, sequence classification, or even as features for other machine learning models.</p><p>One example of an encoder-only model is BERT [2]. It uses the encoder part of the original Transformer model to generate a fixed-size representation of an input text sequence, which can then be fine-tuned for various tasks such as text classification and named entity recognition. Because BERT only utilizes the encoder component, it is not inherently suited for sequence-to-sequence tasks out of the box, unlike the complete encoder-decoder Transformer architecture. Instead, BERT is generally used for tasks that require understanding the context of individual tokens in a text sequence.</p><h3>Decoder-Only Models</h3><p>Decoder-only models like GPT [3, 4] contain only the decoder part. The input to decoder-only models is a sequence of tokens that serve as the context for generating subsequent tokens.</p><p>While traditional Transformers (encoder-decoder) are trained for sequence-to-sequence tasks, decoder-only models are trained to predict the next token in a sequence given the preceding tokens. These models are typically auto-regressive, which means they generate output text token-by-token by conditioning the next token on previously generated tokens. This differs from the traditional encoder-decoder, where the encoder processes the entire input sequence at once, and the decoder generates the output sequence token-by-token.</p><p>Decoder-only models are generally used for tasks that require generating text or other types of data from some initial context. This includes text completion, text generation, and even tasks like image generation in the case of models like DALL-E [5].</p><p>Because they only include the decoder, these models may have fewer parameters than a full Transformer that includes both an encoder and a decoder. However, models like GPT-3 and GPT-4 have shown that decoder-only architectures can scale to a huge number of parameters.</p><h3>Why not just use GPT for everything?</h3><p>Well, you could. In most cases, GPT-like models can generate text with success and understand context without an encoder by its side. Further, we have been empirically convinced that those models can also perform tasks such as text classification and sentiment analysis with high accuracy: by providing GPT with a suitable prompt, its capabilities are almost endless. But as with all great things, it is not free. Far from it, actually.</p><p>The table below shows how BERT can achieve better results than GPT, particularly for those semantic and classification tasks (for more information on the dataset and benchmark used, refer to [6]). Keep in mind that this is a reference to the first version of GPT, not the latest one.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/796/1*VYpCYSrpk0S4EEcV7r_4dQ.png" /><figcaption>Comparing BERT to GPT (Source: [2])</figcaption></figure><p>As Data Scientists, then, we shall make a choice. And the choice depends on the end application. Choosing an encoder-decoder solution only because it provides good accuracy might unnecessarily increase a project’s development time and cost. The same goes for selecting a decoder-only model like GPT: although it can classify texts or detect sentiments in sentences, it comes with the price of a gigantic model. For basic solutions such as text classification and sentiment analysis, encoder-only models can be a suitable in-house low-cost solution.</p><h3>Summary</h3><p>Regarding their inner workings:</p><ul><li>Encoders extract semantic meaning and context from text, outputting a representation of those.</li><li>Decoders generate text from text by predicting which is the most probable next word, given a previous one.</li></ul><p>When joining forces, an Encoder-Decoder model generates text by first extracting a semantic meaning and context from an input (using the <em>Encoder</em>) and then predicting words one by one using a combination of those meanings and the previous word just predicted (using the <em>Decoder</em>).</p><ul><li>Encoder-decoder models provide greater flexibility but are slower to train. Those are used for translation and summarization.</li><li>Encoder-only models are efficient for classification and feature extraction but cannot generate text.</li><li>Decoder-only models are fluent at text generation, like conversations.</li></ul><h3>References</h3><p>[1] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … &amp; Polosukhin, I. (2017). <em>Attention is all you need</em>. In Advances in neural information processing systems. <a href="https://arxiv.org/abs/1706.03762">arXiv:1706.03762</a></p><p>[2] Devlin, J., Chang, M. W., Lee, K., &amp; Toutanova, K. (2018). <em>BERT: Pre-training of deep bidirectional transformers for language understanding</em>. <a href="https://arxiv.org/abs/1810.04805">arXiv:1810.04805</a></p><p>[3] Radford, A., Narasimhan, K., Salimans, T., &amp; Sutskever, I. (2018). <em>Improving language understanding by generative pre-training</em>. <a href="https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf">OpenAI Technical Report</a></p><p>[4] Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., &amp; Sutskever, I. (2019). <em>Language models are unsupervised multitask learners</em>. <a href="https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf">OpenAI Technical Report</a></p><p>[5] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., &amp; Sutskever, I. (2021).<em> DALL·E: Creating images from text</em>. <a href="https://openai.com/blog/dall-e">OpenAI Blog</a></p><p>[6] Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., &amp; Bowman, S. R. (2018). <em>GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding</em>. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP. <a href="https://arxiv.org/abs/1804.07461">arXiv:1804.07461</a></p><h3>In Plain English</h3><p><em>Thank you for being a part of our community! Before you go:</em></p><ul><li><em>Be sure to </em><strong><em>clap</em></strong><em> and </em><strong><em>follow</em></strong><em> the writer! 👏</em></li><li><em>You can find even more content at </em><a href="https://plainenglish.io/"><strong><em>PlainEnglish.io</em></strong></a><strong><em> 🚀</em></strong></li><li><em>Sign up for our </em><a href="http://newsletter.plainenglish.io/"><strong><em>free weekly newsletter</em></strong></a><em>. 🗞️</em></li><li><em>Follow us on </em><a href="https://twitter.com/inPlainEngHQ"><strong><em>Twitter</em></strong></a><strong><em>(X</em></strong>), <a href="https://www.linkedin.com/company/inplainenglish/"><strong><em>LinkedIn</em></strong></a>, <a href="https://www.youtube.com/channel/UCtipWUghju290NWcn8jhyAw"><strong><em>YouTube</em></strong></a>, and <a href="https://discord.gg/in-plain-english-709094664682340443"><strong><em>Discord</em></strong></a><strong><em>.</em></strong></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2298c5d70a58" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/attention-to-llm-architectures-an-intelligent-ml-engineering-guide-2298c5d70a58">Attention to LLM Architectures: An Intelligent ML Engineering Guide</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Not-So-Large Language Models: Good Data Overthrows the Goliath]]></title>
            <link>https://medium.com/data-science/not-so-large-language-models-good-data-overthrows-the-goliath-a8226bd1ae61?source=rss-603bda3d0d35------2</link>
            <guid isPermaLink="false">https://medium.com/p/a8226bd1ae61</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[gpt-4]]></category>
            <dc:creator><![CDATA[Gennaro S. Rodrigues]]></dc:creator>
            <pubDate>Wed, 23 Aug 2023 00:44:31 GMT</pubDate>
            <atom:updated>2023-08-23T00:44:31.126Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cxCw-sTXc3-rJb8yIFtV4g.png" /><figcaption>(Image generated by DALL·E)</figcaption></figure><h4>How to make a million-sized language model that tops a billion-size one</h4><p>In this article, we will see how Language Models (LM) can focus on better data and training strategies rather than just brute size to achieve LLM-like results (sometimes even better) and how people are already doing it successfully and democratically.</p><p>Large Language Models (LLMs) have evolved significantly. They bring remarkable features, from generating human-like text to understanding intricate contexts. While much of the initial excitement revolved around models with a massive number of parameters, recent developments suggest that size isn’t the only thing that matters. Lately, a new concept called Small Language Models (SLM) has risen with justice as a motivation to develop language models more intelligently.</p><h3>The Rise of Large Models</h3><p>As LLMs entered the stage, the narrative was straightforward — bigger is better. Models with more parameters are expected to understand the context better, make fewer mistakes, and provide better answers. But as the models grew, so did their hunger for computational resources. Training these behemoths became an expensive task, one that not everyone is willing (nor able) to pay for.</p><h3>An Emphasis on Quality and Efficiency</h3><p>Recognizing the unsustainability and diminishing returns of just adding more parameters, researchers began to rethink strategies. Instead of merely throwing dollars into the cloud fire (adding another billion more parameters), some researchers shifted to utilizing better data and more efficient training strategies. The idea is elegant: a well-trained smaller model might outperform a poorly trained larger model. But can it?</p><h4>Chinchilla and the Optimal Point for LLMs Training</h4><p>The “Chinchilla paper” [1], a significant contribution to the field, offers intriguing insights into LLMs’ training. Experiments seem to indicate that there is an “optimal point” when training LLMs. Beyond this point, pouring more resources into training in the form of more parameters does not necessarily result in a proportional increase in performance. The paper emphasizes that it’s not only the size of a model that defines its performance. Instead, it’s about the quality of that data and how much data you use. The authors found that for compute-optimal training, the model size and the number of training tokens should be scaled equally: for every doubling of the model size, the number of training tokens should also be doubled.</p><p>They test this by training Chinchilla, a 70 billion parameters model trained on 1.4 trillion tokens. Despite being much smaller, Chinchilla outperforms Gopher on almost all evaluations, including language modeling, question answering, common sense tasks, etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3ZbeGcVywV1x3hoqNEtIdA.png" /><figcaption>Chinchilla Size and Training Tokens vs. SOTA LLMs. (Source: [1])</figcaption></figure><p>Even with its reduced size, Chinchilla performs better than its SOTA counterparts on a variety of tasks:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TI06ZIjoKkyoXOBakkJpug.png" /><figcaption>Massive Multitask Language Understanding (MMLU).<strong> </strong>Reporting the average 5-shot accuracy over 57 tasks with model and human accuracy comparisons taken from [2], and the average prediction for SOTA accuracy in June 2022/2023 made by 73 competitive human forecasters in [3]. (Source: [1])</figcaption></figure><p>Reading comprehension and automated reasoning are standard tasks a language model is typically tested on. It tests the model’s ability to understand the broader context of the text. In our case, it could be exemplified as predicting words that could only be expected if the model could understand the relation between this word and the context that came before it (sometimes far from this word’s position). It is usually evaluated using benchmarks and datasets such as RACE-h, RACE-m [4], and LAMBADA [5]. Chinchilla outperforms much bigger models even on this type of hard-to-define and test tasks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FUBOdkB6sOS-_oL0UKjHPg.png" /><figcaption>On Reading Comprehension, Chinchilla considerably improves performance over <em>Gopher. </em>(Source: [1])</figcaption></figure><p>And Chinchilla is one of many LMs showing promising results despite not focusing on augmenting size.</p><h4>LLaMA</h4><p>LLaMA[6] goes even further. The authors introduce smaller foundation language models ranging from 7B to 65B parameters. They are trained on over 1 trillion tokens using only publicly available data, making them compatible with open sourcing.</p><p>LLaMA-13B outperforms the much larger 175B parameter GPT-3 on most benchmarks while being over 10x smaller. The authors argue that given a target performance level, smaller models trained longer are preferable to larger models for a given compute budget due to better inference efficiency.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ePltp9Uy_Rw9J_mw5R226g.png" /><figcaption>LLaMA zero-shot performance on Common Sense Reasoning tasks. (Source: [6])</figcaption></figure><p>Some projects have even managed to run LLaMA (or rather a version of it) on budget Android smartphones, further proving that we are on the right path to democratizing access to performative LMs using low computing resources (LLaMA.c [7]).</p><p>LLaMA-65B (I know, not that small anymore, but still…) is competitive with the current state-of-the-art models like PaLM-540B, which use proprietary datasets. This clearly indicates how good data not only improves a model’s performance but can also make it democratic. A machine learning engineer would not need enormous budgets to get good model training on a good dataset.</p><h4>Good data trumps the Goliath</h4><p>Further reinforcing the thesis that LMs don’t need to be gigantic to perform well, TinyStories [8] presents a synthetic dataset of stories containing only words that small children (up to four years old) can understand. It can be used to train small language models (SLMs) with under 10 million parameters that can generate multi-paragraph stories with good grammar, reasoning, and coherence. This contrasts previous works where 125M+ parameter models — such as GPT-Neo (small) and GPT-2 (small) — struggled to produce a coherent text.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J4TzDfrUVbWiiss9dRrMrw.png" /><figcaption>A model trained with TinyStories can produce output comparable to one from a model that is two orders of magnitude bigger. (Source: [8])</figcaption></figure><p>One of the exciting aspects of TinyStories is that the dataset itself was created by GPT-3.5 and GPT-4. The authors also introduce a new SLM evaluation paradigm using GPT-4 to “grade” generated stories on dimensions like grammar, plot, and creativity. This overcomes the limitations of standard benchmarks requiring constrained outputs.</p><h3>Conclusion</h3><p>The journey of LMs showcases a pivotal lesson in AI: Bigger is not always better. As the community continues to evolve and innovate, there’s a realization that efficiency, quality of data, and optimized training strategies hold the key to the future of machine learning.</p><h4>Key Takeaways</h4><ul><li>Chinchilla proves that there is an optimal point when training LMs regarding the number of tokens and the quality of training data used. It is as important as (or more) defining the number of parameters of the model;</li><li>LLaMa shows Chinchilla-like results are achievable using only publicly available data, proving this strategy to be democratically available;</li><li>Datasets like TinyStories can be used to train small language models (less than 100 million) that outperform billion-sized models on specific tasks.</li></ul><h4>References</h4><p>[1] Hoffmann, Jordan, et al. “Training compute-optimal large language models.” <em>arXiv preprint arXiv:2203.15556</em> (2022).</p><p>[2] D. Hendrycks, et al. “Measuring massive multitask language understanding.” <em>arXiv preprint arXiv:2009.03300 (</em>2020).</p><p>[3] J. Steinhardt. Updates and lessons from AI forecasting, 2021. URL https://bounded-regret.ghost.io/ai-forecasting/.</p><p>[4] Lai, Guokun, et al. “RACE: Large-scale ReAding Comprehension Dataset From Examinations.” In <em>Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing</em>, pages 785–794, Copenhagen, Denmark. Association for Computational Linguistics.</p><p>[5] Paperno et al., 2016 “The LAMBADA dataset: Word prediction requiring a broad discourse context.”<em> arXiv:1606.06031 </em>(2016).</p><p>[6] Touvron, Hugo et al. “LLaMA: Open and Efficient Foundation Language Models.” <em>ArXiv</em> abs/2302.13971 (2023)</p><p>[7] <a href="https://github.com/karpathy/llama2.c">https://github.com/karpathy/llama2.c</a></p><p>[8] Eldan, Ronen and Yuan-Fang Li. “TinyStories: How Small Can Language Models Be and Still Speak Coherent English?” <em>ArXiv</em> abs/2305.07759 (2023)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a8226bd1ae61" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/not-so-large-language-models-good-data-overthrows-the-goliath-a8226bd1ae61">Not-So-Large Language Models: Good Data Overthrows the Goliath</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Data Scientist's Life in Portugal: The new Silicon Valley in Europe.]]></title>
            <link>https://medium.com/@gennsev/a-data-scientists-life-in-portugal-the-new-silicon-valley-in-europe-3bf75fa8b37?source=rss-603bda3d0d35------2</link>
            <guid isPermaLink="false">https://medium.com/p/3bf75fa8b37</guid>
            <category><![CDATA[careers]]></category>
            <category><![CDATA[personal-development]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Gennaro S. Rodrigues]]></dc:creator>
            <pubDate>Wed, 18 May 2022 21:58:22 GMT</pubDate>
            <atom:updated>2022-05-18T21:58:55.634Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SS5JTV2yVf7LGlwEAFRGjA.jpeg" /></figure><h3>A Data Scientist&#39;s Life in Portugal: The New Silicon Valley in Europe</h3><h4>Once upon a time, the Portuguese decided to try their luck and sail to the unknown. It paid off. And they are on it again.</h4><p>Low cost of living, pleasant climate, and tax break incentives. Personally, I love saving (and investing) money. And the beach. Also, I&#39;m not too fond of taxes, so yeah, it seems like a good deal for me. And apparently for a lot of people.</p><p>Portugal has been one of the top tourist destinations for Europeans for many decades. The rich culture, spectacular beaches, idyllic climate, and high standard of living with affordable costs make Portugal not only a tourist spot but also a place for settling in permanently. Not only for other Europeans seeking a nice place to retire but also for foreigners from all over the world. The amount of Americans living in Portugal has increased by more than a third fold since 2010 [1].</p><p>But the country has been attracting the attention of not only students and tourists: business people and <em>startupers </em>are investing quite a bit of time and money in Portugal. The results are nothing less than interesting. The country has a booming ecosystem with an ever-growing number of innovative startups, angel investors, and career opportunities.</p><h3>Portugal Fintechs and Startup Environment</h3><p>In an effort to boost the country’s economy and make it an actual tech lead example, the Portuguese government and the European Union are putting money and effort into several entrepreneurship incentive programs.</p><h4>Vila Nova de Gaia: The European Silicon Valley</h4><p>Vila Nova de Gaia, an otherwise quiet town in the north of the country, close to the city of Porto, has received <a href="https://www.jornaldenegocios.pt/empresas/detalhe/paulistas-apostam-mil-milhoes-num-silicon-valley-em-gaia">an investment of more than 700 million euros</a> intended to found the effort of a new tech hub. The total investment from the Brazilian investors is expected to exceed 1 billion euros in five years.</p><blockquote><strong>“Gaia Innovation City will be a tech innovation hub, with a complete structure focused on technology and education, bringing together the academic and professional worlds.” (the city’s official website)</strong></blockquote><p>The project consists of a 21-hectare space to attract tech giants like Google, Facebook, IBM, Amazon, and Netflix. It will also be the home of a tech campus to strengthen the relations between academics and the industry. Constructions shall start in 2023 and be finished in around 2030, and the project shall create about 15000 jobs.</p><h4>Portugal Web Summit</h4><p>The Web Summit is considered the most important technology event in Europe and, according to some, worldwide. Its first editions were held in Dublin, but since 2016 it has been held in Lisbon, Portugal. In 2018, the Portuguese government and Web Summit announced a partnership to keep it in Lisbon every year until 2028.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QIUMoWDrbggVYeJInNr6Bw.png" /><figcaption>Some of the Web Summit speakers from 2021</figcaption></figure><p>It doesn&#39;t take a market analyst to convince anyone of the importance of holding an event of this magnitude and significance in a city making efforts to thrive in the tech business. The Web Summit is attended by companies like Apple, Airbus, Vogue, Siemens, and more. It attracts the brightest of the minds and is the perfect place for networking, learning, and being part of the changes our industry needs.</p><blockquote>“Web Summit brings together the people and companies redefining the global tech industry.” (CGTN)</blockquote><h4>A Personal Report from a Data Scientist in Lisbon</h4><p>I have personally been working for a Brazilian company that has an office in Lisbon for the last three months, and it has been just great. Walking on the streets, we hear nothing but foreign languages. Don’t take me bad here, I love the Portuguese language, but there is something romantic in seeing so many different languages mix. It is also proof that the country is successfully becoming the international hub it aims to be (it is clear that most of those people are not tourists).</p><p>Our office steps away from Avenida da Liberdade (known as the Portuguese version of the Champs Elysée, but far from being an awkward replica, it is its own thing) is ideally located in the heart of the city. It is located at the <a href="https://www.thefintechhouse.com/">Fintech House</a>, a co-working space that embodies all the aspirations and efforts we have discussed so far. A co-working space is a perfect place to foster new ideas, discuss new solutions and form new partnerships. Many of us have missions that complete each other.</p><p>Portugal is nothing but a goldmine for a growing company like the one I work for. And we are definitely enjoying it. I can see my colleagues grow as professionals and persons. A positive work environment changes everything. A data scientist with the right mindset can do great things.</p><p>Whether you are an entrepreneur looking for a place to settle, a student looking for cutting-edge learning, or an IT professional looking for the opportunity to change your career forever, I am sure you can find it (or develop it) in Portugal. The country is changing and innovating, and in this industry, success comes with innovation and growth. Destroying the status quo.</p><p>I am planning to go back to Brazil soon, but I’ll definitely be back again for more. Portugal has a lot to offer. I’ve been learning and taking a lot from here, and I’ll be back for more.</p><h4>Resources</h4><p>[1] <a href="https://www.wsj.com/articles/portugal-retirement-what-to-know-11649450537">https://www.wsj.com/articles/portugal-retirement-what-to-know-11649450537</a></p><p>[2] <a href="https://viveurope.com/portugal-startup-hub/">https://viveurope.com/portugal-startup-hub/</a></p><p>[3] <a href="https://websummit.com/">https://websummit.com</a></p><p>[4] <a href="https://www.thefintechhouse.com/">https://www.thefintechhouse.com</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3bf75fa8b37" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Multithreading vs. Multiprocessing in Python]]></title>
            <link>https://medium.com/data-science/multithreading-vs-multiprocessing-in-python-3afeb73e105f?source=rss-603bda3d0d35------2</link>
            <guid isPermaLink="false">https://medium.com/p/3afeb73e105f</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[multithreading]]></category>
            <category><![CDATA[getting-started]]></category>
            <category><![CDATA[parallel-computing]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Gennaro S. Rodrigues]]></dc:creator>
            <pubDate>Sun, 27 Sep 2020 19:45:40 GMT</pubDate>
            <atom:updated>2020-09-30T15:37:26.747Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XbMhaKrrzEPmMZg8" /><figcaption>Photo by <a href="https://unsplash.com/@raulcachophoto?utm_source=medium&amp;utm_medium=referral">Raul Cacho Oses</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4><a href="https://towardsdatascience.com/tagged/getting-started">Getting Started</a></h4><h4>The differences between the two, and how and when to use them</h4><p>Python is often regarded as an almost obscene programming language. A libertarian place where everything is permitted and our wildest dreams come true. Forget about labels and definitions, and, above all, brackets. It is a language that welcomes everyone, from the most experienced programmer (that weird guy that still uses Fortran for some reason you don’t really want to know about) to the egotistic millennial newbie. Well, that is, except if you are working with multiprocessing or multithreading.</p><p>The challenges of working with parallelism and concurrency in Python begins with the fact that even the internet does not understand very well how it works. Indeed, I’ve found multiple wrong statements on stackoverflow responses (some with a high count of up votes) and even very serious-looking blogs. Therefore, differently from other cases, where <em>programming</em> may also be called <em>stackoverflowing</em>, you better know at least the basics of what you are doing before starting to work with multiprocessing or multithreading in Python.</p><p>Or else, you might have a bad time.</p><h3>The Basics</h3><p>When looking for the difference between python multiprocessing and multithreading, one might have the impression that they work pretty much the same. That could not be more wrong. The key differences are:</p><ul><li>A process is an independent instance executed in a processor core. Threads are components of a process and run concurrently (inside that process).</li><li>Processes do not share the same memory space, while threads do (their mother’s memory, poetic, right?).</li><li>Threads are lighter and cause less overhead. Also, because they share the same memory inside a process, it is easier, faster, and safer to share data.</li><li>True parallelism can ONLY be achieved using multiprocessing. That is because only one thread can be executed at a given time inside a process time-space. This is assured by Python’s global interpreter lock (GIL) (see <a href="https://realpython.com/python-gil/">Python GIL at RealPython</a>).</li><li>Processes execution is scheduled by the operating system, while threads are scheduled by the GIL.</li></ul><p>We should also understand the difference between concurrency and parallelism.</p><ul><li><strong>Concurrent</strong> execution means that two or more tasks are progressing at the same time.</li><li><strong>Parallel</strong> execution implies that two or more jobs are being executed simultaneously.</li></ul><p>Now remember: multithreading implements concurrency, multiprocessing implements parallelism. Processes run on separate processing nodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/519/1*_nSpk2ZTATQpps0XOHJHGA.png" /><figcaption><em>Processes run in parallel. Threads execute concurrently. </em>Image by Author</figcaption></figure><h3>The Use Cases</h3><p>So how do we decide between multiprocessing and multithreading? The short answer is:</p><ul><li>Multithreading for I/O intensive tasks and;</li><li>Multiprocessing for CPU intensive tasks (if you have multiple cores available)</li></ul><p>But why?</p><h3>Multithreading: I/O bound tasks</h3><p>Imagine you have a task that relies on the input from another part of the system, or from the user.</p><p>There is no reason to dedicate a process for that job because it would spend most of its time waiting for the stimuli to begin its work. It would be a waste of resources to dedicate a computing node for that, and it would bring no speedup whatsoever, given that the bottleneck is not on execution but on a third-party call. In this case, multithreading is the way to go.</p><p>Indeed, multithreading is usually implemented to handle GUIs. E.g., in a text editor program, one thread would be responsible for getting input from the keyboard, while another would check the spelling, and a third one could handle the formatting configurations.</p><p>I/O intensive programs can also directly benefit from multithreading because their bottlenecks are usually reading or writing operations (or downloads in the case of web scrappers, for example). Check the following example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6bd4768711deb17496fec9afbfc876fb/href">https://medium.com/media/6bd4768711deb17496fec9afbfc876fb/href</a></iframe><p>This function loads some URL addresses into memory and writes them in a local text file (50 times, always overwriting the same file). That is pretty dumb, but its purpose is just to stress an I/O bound execution. You can check this code and execute it yourself in <a href="https://github.com/gennsev/Python_MultiProc_Examples/blob/master/Example.ipynb">this Notebook</a>.</p><p>Let us run that method over a list of 16 addresses and check their execution times for some different thread counts. That would get us the following graph (executing on a MacBook Air 2019).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*qn6Gg7AOsEJNZJa9.png" /><figcaption>Image by Author</figcaption></figure><p>Where the result for one thread stands for the serial execution of the code. We can see by the graphs that multithreading is capable of considerably reducing the algorithm’s execution time. We could keep increasing the number of threads, but the gains would stop once we reach the point where each address is handled by a thread.</p><h3>Multiprocessing: CPU bound tasks</h3><p>Now imagine you got a massive amount of data loaded in memory, and it all needs to be processed. Programs that are computing-bound will benefit from multiprocessing because their bottleneck is time and resources. Image and graphics processing are an excellent example of that. They consist of vast amounts of math operations, which can often be divided into separate tasks because their data is independent. GPUs are the state-of-the-art hardware in this regard, being designed to process large chunks of data in parallel.</p><p>To simulate computing-intensive algorithm, we will use the following code:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/efdce417735e4e4118cb2afe9535e895/href">https://medium.com/media/efdce417735e4e4118cb2afe9535e895/href</a></iframe><p>Again, you can check this code in <a href="https://github.com/gennsev/Python_MultiProc_Examples/blob/master/Example.ipynb">this notebook</a> and execute it yourself (I <strong><em>strongly </em></strong>suggest you to take a look on that, as we have to make some workarounds to execute it properly on windows).</p><p>My MacBook features a dual-core processor with hyper-threading, so there is no use to force more than 4 processes running on it. Those are the results for the performance tests on this benchmark:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*RMTFnj3ghxCuIa71.png" /><figcaption>Image by Author</figcaption></figure><p>The multithreading provided no speedup (the execution times are virtually the same). However, we had a significant speedup when using multiprocessing. It was not enough to drop the execution time to one fourth of the original because of the time spent with process management. Remember that processes are much heavier than threads.</p><h4>Multiprocessing on I/O intensive tasks</h4><p>Even though this is not their best use case, there is no reason to expect multiprocessing to perform poorly on I/O intensive tasks. It is a waste of resources: imagine dedicating a processor core to a function that will, for a significant part of its execution, just wait for an input. A processor, even a cluster, only has so many processing nodes. We are better off dedicating them to real parallelism of applications that can only be accelerated by those means.</p><p>But for the sake of science, what would happen if we executed the first I/O-bound code using multiprocessing? The following graph shows it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*Cy0tF2o1sqsYjVH_.png" /><figcaption>Image by Author</figcaption></figure><p>As expected, multiprocessing is also able to provide speedup on I/O bound computations. It is also noticeable that we kept the speedup even with 8 processes. The reason for that is processes start executing like threads in this case. The speedup is not coming from the parallelism, but from concurrency.</p><h3>A Data Science Example</h3><p>Now, let’s take a deeper look on how we can implement multiprocessing and multithreading in Python and how a Data Scientist can profit from them. There are many different ways to do it, and I encourage you to get some ideas from the documentation (<a href="https://docs.python.org/3/library/concurrent.futures.html">here</a>, <a href="https://docs.python.org/3.8/library/threading.html">here</a>, <a href="https://docs.python.org/3.8/library/multiprocessing.html">and here</a>) and try them out on the examples model I provided at <a href="https://github.com/gennsev/Python_MultiProc_Examples/blob/master/Example.ipynb">this notebook</a> as an exercise.</p><p>In a VERY broad aspect, a data science project follows four steps:</p><ol><li>Get the data;</li><li>Process the data;</li><li>Do something with the data;</li><li>Store other data</li></ol><p>Now, you may say: “nah, my project is way different”. Well, I beg to differ, but I’m sure you can follow this blog post to get some ideas on how to accelerate your own so-different-from-all-the-others project.</p><blockquote>Remember the golden rule: multithreading for I/O bound tasks and multiprocessing for CPU bound task.</blockquote><p>Now, it should not be hard to follow that the “get the data” and “store other data” steps are probably the ones where we should use multithreading. All the others (“process the data” and “do something with the data”) are most probably better off using multiprocessing. You may argue that processing a lot of data might present an I/O bottleneck, and in some cases you might be right. If that is your case, you should try to break up this processing into “getting data” and “processing data” parts, so that the best strategy can be applied to each of them.</p><h3>Text Classification</h3><p>Let’s see how we can apply what we’ve learned in a classic Natural Language Processing (NLP) problem: text classification. The idea is to find out the category of a news article just by its text (e.g.: if it should be categorised as “sports”, “finances”, “economy”, etc).</p><blockquote>If you are here only to see how multiprocessing and multithreading can be used in a real project, just follow along, you don’t really need to know anything about NLP to understand this example. However, if you are interested by the subject, you should start by studying what <a href="https://en.wikipedia.org/wiki/Word_embedding">word vectors</a> are. I like <a href="https://medium.com/@jayeshbahire/introduction-to-word-vectors-ea1d4e4b84bf">this article</a>. You can also follow <a href="https://github.com/gennsev/tclass">this GitHub repository</a> where I and a colleague have been working on some text classification models (there are some cool notebooks there).</blockquote><p>Our project is quite simple, we will:</p><ol><li>Get some data from Reuters News Agency containing news and their respective labels (defining what type of news they are) (<a href="https://www.cs.umb.edu/~smimarog/textmining/datasets/">Data source</a>);</li><li>Vectorize them using GloVe pre-trained word vectors (trained from Wikipedia) (<a href="https://nlp.stanford.edu/projects/glove/">GloVe project page</a>);</li><li>Train a model using Random Forests with scikit-learn to classify texts under the given labels.</li></ol><p>By now it shall be straightforward to see that step 1 can possibly be accelerated in Python using multithreading, while step 3 should use multiprocessing.</p><h4>Loading Data</h4><p>Let’s start by the pre-trained GloVe word vectors loading. You can check the full code and execute it yourself in <a href="https://github.com/gennsev/Python_MultiProc_Examples/blob/master/Example-2.ipynb">this notebook</a>. Loading this pre-trained word vector file can take a lot of time. The file is considerably long, and we have to process it line per line. Each line contains a word and then a list of values for each dimension of its word vector.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a2feb69b83ec6839a7ab748844894a7d/href">https://medium.com/media/a2feb69b83ec6839a7ab748844894a7d/href</a></iframe><p>The GloveVectorizer() loads the pre-trained vectors in its __init__ function, and can do it both asynchronously or serially. This is how it performs the basic line-per-line file reading:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/62f9f3729cdaf80564a72865bda93e15/href">https://medium.com/media/62f9f3729cdaf80564a72865bda93e15/href</a></iframe><p>And here is the multithread implementation:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6f3b5f3397a701defb86497a06a80c13/href">https://medium.com/media/6f3b5f3397a701defb86497a06a80c13/href</a></iframe><p>Again, I strongly recommend you to check how it was implemented in the <a href="https://github.com/gennsev/Python_MultiProc_Examples/blob/master/Example-2.ipynb">full code</a>.</p><p>The ThreadPoolExecutor runs its threads asynchronously. The last for loop is used to guarantee the execution will only keep going after all the threads submitted to the executor are finished. Check the <a href="https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ThreadPoolExecutor">Python Documentation</a> for more details on how ThreadPoolExecutor works.</p><p>But how faster does the vectors loading gets with multithreading? In my MacBook Air the first serial version loads the 400000 word vectors in around 269.19898986816406s. The asynchronous approach loads the 400000 word vectors in 27.559515953063965s, using 10 workers (it could probably reach the same execution time with even less workers, as the bottleneck is reading the lines, not processing).</p><h4>Training the model</h4><p>Now the cool part: training and testing.</p><p>Luckly for us, scikit-learn offers multiprocessing nativelly, just by setting it up on the model’s parameters. The two following code examples train a same model with the same data serially or using a set number of jobs (which are mapped to processes in scikit-learn).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b495f91a2f2c5a5b20defe6f90fa183b/href">https://medium.com/media/b495f91a2f2c5a5b20defe6f90fa183b/href</a></iframe><p>Using multiprocessing in scikit-learn is as easy as setting the n_jobs model parameter. Here, we will set it to two:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a1f82a2894bf9b9de7342e7eef01fc84/href">https://medium.com/media/a1f82a2894bf9b9de7342e7eef01fc84/href</a></iframe><p>It’s so easy you may doubt it works. But it does. The following bar graph shows the training time of this same model for different numbers of jobs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/473/0*ZvBRwxViokO3XQF8.png" /><figcaption>Image by Author</figcaption></figure><p>Oh, and if you are curious, the model accuracy (RandomForestClassifier.scoreTrain) turns out not bad at all:</p><pre>Train score: 0.9992707383773929<br>Test score: 0.9346733668341709</pre><h4>Conclusions</h4><p>I think it covers pretty much everything. This last example shows how Python multiprocessing and multithreading features can be used to accelerate real projects, and sometimes with little-to-none code modifications. All that glitters is not gold, though. You will soon find out, when looking forward to more complex parallel and asynchronous executions in python that things can get quite messy.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3afeb73e105f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/multithreading-vs-multiprocessing-in-python-3afeb73e105f">Multithreading vs. Multiprocessing in Python</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating  a Data Scientist’s Personal Website (with Hugo and Netlify)]]></title>
            <link>https://medium.com/data-science/creating-a-data-scientists-personal-website-with-hugo-and-netlify-c0e041da9fc0?source=rss-603bda3d0d35------2</link>
            <guid isPermaLink="false">https://medium.com/p/c0e041da9fc0</guid>
            <category><![CDATA[personal]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[hugo]]></category>
            <category><![CDATA[website]]></category>
            <category><![CDATA[netlify]]></category>
            <dc:creator><![CDATA[Gennaro S. Rodrigues]]></dc:creator>
            <pubDate>Fri, 28 Aug 2020 16:57:43 GMT</pubDate>
            <atom:updated>2020-10-20T20:26:47.382Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Image credit: https://unsplash.com/@andrewtneel" src="https://cdn-images-1.medium.com/max/750/1*7tqj68lRWgpd5Cd53rWMUg.png" /><figcaption>Image credit: <a href="https://unsplash.com/@andrewtneel">https://unsplash.com/@andrewtneel</a></figcaption></figure><h4><a href="https://towardsdatascience.com/tagged/getting-started">Getting Started</a></h4><h3>Creating a Data Scientist’s Personal Website (with Hugo and Netlify)</h3><h4>Sharing your projects, blog and publications with a personal website makes you a real professional</h4><p>As a Data Scientist, it seems fit to hold a personal page to show off my projects and blog. It makes the whole thing way more professional.</p><p>I’ve been an academic for the most part of my life, and I’ve always held a personal page inside my university network to share a small blog and projects. After finishing my Ph.D. at the end of last year, I’ve since been longing to move it to another place. It made no sense to keep up using the university resources for that, even if they were still available for me.</p><p>So my journey begins. I head off to this new adventure of creating a new website and ask for some help from my friends. One of them uses <a href="https://wordpress.com/">WordPress</a>, and another one tried to convince me of using <a href="http://www.wix.com/">Wix</a>. But for some reason, they didn’t strike me as the best choice for a data scientist and computer engineer. They are just too “easy” and “popular”, and that’s not how we do things, right? Another friend told me about that <a href="https://pages.github.com/">pages system from GitHub</a>. But it did not convince me either.</p><p>That was when, reading a random text in a random blog on a random page, I read the following: <em>Made with </em><a href="https://gohugo.io/"><em>Hugo</em></a><em> • Deployed to </em><a href="https://www.netlify.com/"><em>Netlify</em></a><em>. </em>I got interested immediately. And that random guy (or girl, I don’t even know) who didn’t even try, ended up enlightening me how to develop and deploy my own page.</p><p>Ok, so what is it about?</p><h3>Hugo</h3><p>Hugo is a framework for building websites. It is powerful and has a great community with forums full of questions and answers and a bunch of themes created and shared by the users.</p><p>Creating a Hugo site is really quite easy, and you can just follow <a href="https://gohugo.io/getting-started/quick-start/">this get-started guide</a> to deploy your own in no time. What I like the most about Hugo is that is developer-friendly (you really feel like you are developing something, not just playing the web designer), easy to learn, and highly expansible. Really, you can get as complex as you want with your website, or just keep it simple. Do it as you like.</p><h3>Netlify</h3><p><a href="https://www.netlify.com/">Netlify</a> is a platform for building and deploying sites from Git repositories (such as GitHub, GitLab, or BitBucket). It provides continuous deployment services, global CDN, ultra-fast DNS, atomic deploys, instant cache invalidation, one-click SSL, a browser-based interface, a CLI, and many other features for managing your Hugo website.</p><p>There are many ways to deploy your Hugo site and put it online. You can check a list with some tutorials <a href="https://gohugo.io/hosting-and-deployment/">on this page</a>. However, I’ve found Netlify to be a great option that offers some superior resources, even in the free tier.</p><h3>Deploying a Site</h3><p>That’s way easier than it sounds, and can be done in around 30 minutes (15 if you can type and make decisions fast). Follow those 3 steps:</p><ol><li>Get a site running. <a href="https://gohugo.io/getting-started/quick-start/">Follow the Hugo tutorial</a>.</li><li>Create a git repository in your favorite platform for your site (I’m using GitHub). Then, you can deploy your site online using Netlify. This is how you do it:</li></ol><p>2.1: Create a netlify.toml file at the root of your repository. This file configures how Hugo and Netlify work together. Then, copy-paste the following code:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1fc492fe53e837d130ec6b2f66afd848/href">https://medium.com/media/1fc492fe53e837d130ec6b2f66afd848/href</a></iframe><p>2.2.: Create an account at <a href="https://www.netlify.com/">Netlify</a> and click at <strong>New site from Git</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dQLssMYCZBIcxrr0NxMIeA.png" /><figcaption>Creating new site from Git. Image by Author</figcaption></figure><p>2.3.: Select your Git provider and the repository that holds your site.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KIyt4RvrGcUovVM3KxTVMQ.png" /><figcaption>Selecting a Git repository for continuous deployment. Image by Author</figcaption></figure><p>2.4.: Configure the build options and deployment of your site at Netlify, like the screenshot below. (PS: the standard is to deploy from your master branch, but you can set any branch you like)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uEzD6LtmMpSCt2VxF8JA7g.png" /><figcaption>Netlify build options for Hugo. Image by Author</figcaption></figure><p>Now click on <strong>Deploy site</strong>, and voilà: your site will be built and get online in some instants.</p><p>3. You will notice in your overview page that your deployed site domain contains a Netlify extension. You can just use it the way it is, but you need your own domain (a proper <strong><em>www.&lt;your_name_or_nickname&gt;.com</em></strong>) if you want to be fancy.</p><p>You can get a custom domain by Netlify (you will have to pay for it, ofc), or you can register your domain with <a href="https://www.godaddy.com/">GoDaddy</a>, <a href="https://domains.google.com/r">Google Domains</a>, or any other service of your choosing. I’ve chosen Google Domains. Just because. Configuring your custom domain is easy, and I will not cover it here because it differs depending on the service you use. But <a href="https://docs.netlify.com/domains-https/custom-domains/">this documentation provided by Netlify</a> will teach you all you need. PS: Your custom domain might take some hours to start working for your site, DNS-related issues.</p><h3>Congrats, you have your site :)</h3><p>Now you are done. If you are programmer, data scientist, or anything related, you probably have worked with continuous deployment before. Then I don’t have to tell you that all you have to do to update your site is to commit (or merge) to your repository’s master branch. If you are not, you probably did not follow this tutorial anyway.<br>I hope you have a nice time sharing your work on your new page!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c0e041da9fc0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/creating-a-data-scientists-personal-website-with-hugo-and-netlify-c0e041da9fc0">Creating  a Data Scientist’s Personal Website (with Hugo and Netlify)</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>