<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Assaf Yablon on Medium]]></title>
        <description><![CDATA[Stories by Assaf Yablon on Medium]]></description>
        <link>https://medium.com/@yablonassaf?source=rss-ea8f25ff99bd------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*zOyA7G0QL7-mjjtDtxk8-w.jpeg</url>
            <title>Stories by Assaf Yablon on Medium</title>
            <link>https://medium.com/@yablonassaf?source=rss-ea8f25ff99bd------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 10:27:02 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@yablonassaf/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Unity Unveils New TEXT-to-VIDEO GAMES generative AI service: Unity Muse!]]></title>
            <link>https://medium.com/@yablonassaf/unity-unveils-new-text-to-video-games-generative-ai-service-unity-muse-3e7abc88d9c6?source=rss-ea8f25ff99bd------2</link>
            <guid isPermaLink="false">https://medium.com/p/3e7abc88d9c6</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[unity]]></category>
            <dc:creator><![CDATA[Assaf Yablon]]></dc:creator>
            <pubDate>Fri, 30 Jun 2023 00:13:19 GMT</pubDate>
            <atom:updated>2023-06-30T00:14:36.180Z</atom:updated>
            <content:encoded><![CDATA[<h3>Unleashing Creativity: Unity Muse and Unity Sentis Set to Revolutionize Content Creation</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FdR4IuN2tF78%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DdR4IuN2tF78&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FdR4IuN2tF78%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/bd6c3ba853ad2c49d43a61efa88f2453/href">https://medium.com/media/bd6c3ba853ad2c49d43a61efa88f2453/href</a></iframe><p>Unity Technologies, the trailblazer in game development, recently made a groundbreaking announcement with the introduction of two AI-powered tools, Unity Muse and Unity Sentis. These tools, which are currently in closed beta, have been designed with the intent of accelerating creation time and streamlining workflows, marking a monumental stride in the realms of gaming, entertainment, and industrial applications. Let’s dive into what these tools offer and how they are set to reform the content creation landscape.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TDL3W6IA-g-0WO2DROUdtw.jpeg" /></figure><h3>Unity Muse: AI-powered Creative Assistant</h3><p>Unity Muse is a comprehensive platform that integrates AI-driven assistance to facilitate creators during the development process. The ultimate goal of Muse is to empower creators to effortlessly generate a wide range of content within the Unity Editor using natural inputs like text prompts and sketches.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1003/1*dl9M8GugCOvIHm7ZicoL6g.jpeg" /></figure><h3>Muse Chat: A Quantum Leap in AI-based Search</h3><p>The flagship feature of the Muse platform is Muse Chat. This feature utilizes AI-based search capabilities across Unity’s extensive documentation, training resources, and support content. Muse Chat presents users with well-structured, precise, and up-to-date information from Unity’s knowledge base, inclusive of working code samples, to expedite development and troubleshoot issues efficiently.</p><h3>The Road Ahead for Muse</h3><p>This is just the tip of the iceberg as Unity Muse is expected to unveil additional features in the coming months. These features include the capability to create textures, sprites, and even fully animate characters through natural input. Unity Muse is poised to continually evolve, providing indispensable assistance both within the Unity Editor and on the web.</p><h3>Unity Sentis: Bridging Neural Networks and Real-time Experiences</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/842/1*X0pK1JKErqUidCvb1EX4Ww.png" /></figure><p>Enter Unity Sentis, an avant-garde tool that enables users to integrate neural networks into their creations. By embedding an AI model in the Unity Runtime for games or applications, Sentis augments gameplay and other functionalities directly on end-user platforms.</p><h3>Sentis: The Gamechanger</h3><p>Unity Sentis is a game-changer in the truest sense. It’s the first and only cross-platform solution that allows embedding AI models into a real-time 3D engine. Developers can build their model once and embed it, facilitating it to operate on a myriad of platforms, including mobile, PC, web, and game consoles. This eliminates the complexity, latency, and costs associated with model hosting in the cloud.</p><h3>AI Verified Solutions: Augmenting the Unity Ecosystem</h3><p>In addition to Unity Muse and Unity Sentis, Unity Technologies also announced the availability of AI Verified Solutions. These are third-party packages from developers such as Atlas, Convai, Inworld AI, Layer AI, Leonardo Ai, and others. These solutions encompass AI-powered smart NPCs, AI-produced VFX, textures, 2D sprites, 3D models, generative speech, and in-game testing with AI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*NgUK2x68ChAJ20aWRQmVZA.jpeg" /></figure><h3>A Paradigm Shift in Content Creation</h3><p>The advent of Unity Muse and Unity Sentis signifies a paradigm shift in content creation. Unity believes that every aspect of the creative process will be impacted by AI. AI can help creators be more productive while staying fully in control of their vision. It offers the possibility of in-game features and capabilities that couldn’t be built otherwise, potentially revolutionizing player experiences by embedding AI models in the runtime so content reacts and responds to players and users in new ways.</p><h3>Final Thoughts</h3><p>Unity Muse and Unity Sentis mark a monumental leap in content creation, bringing AI-powered tools to the fingertips of creators across various domains. With features like Muse Chat and the ability to embed AI models into real-time 3D engines with Sentis, Unity is pushing the boundaries of what’s possible in the realm of gaming, entertainment, and industrial applications.</p><p>As AI continues to evolve and integrate more deeply into content creation, the opportunities for innovation are limitless. Whether you are just getting started or are a seasoned pro, these tools are here to open new avenues of creativity and innovation. This is not just an evolution; it’s a revolution in content creation driven by artificial intelligence.</p><p>So, what are you waiting for? Grab this opportunity to become part of a groundbreaking journey that’s set to redefine the way we create and experience content in the digital world.</p><p>— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — -</p><p>*Follow Assaf on Medium for the latest updates and insights on AI, technology, and more.*</p><p>![Unity Technologies](unity-technologies.jpg)</p><p>*Unity Technologies is known for its real-time 3D development platform, empowering developers and creators across the globe.*</p><p>Please share your thoughts and join the conversation by commenting below. Don’t forget to clap if you found this article insightful and follow Assaffor more exciting content.</p><p>*Disclaimer: The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy or position of any agency or company.*</p><p>— -</p><p>## About the Author</p><p>Assaf is a Senior AI Expert and a regular contributor to AI Chronicles, an AI-focused blog on Medium. With extensive knowledge in Artificial Intelligence, Machine Learning, and emerging technologies, ChatGPT is dedicated to providing readers with the latest news, insights, and trends in the AI industry.</p><p>Connect with Assaf on LinkedIn: <a href="https://www.linkedin.com/in/yablonassaf/">(1) Assaf Yablon | LinkedIn</a> for real-time updates and discussions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3e7abc88d9c6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Microsoft will launch GPT-4 with AI videos on Wednesday]]></title>
            <link>https://medium.com/@yablonassaf/microsoft-will-launch-gpt-4-with-ai-videos-on-wednesday-75d882e0260e?source=rss-ea8f25ff99bd------2</link>
            <guid isPermaLink="false">https://medium.com/p/75d882e0260e</guid>
            <category><![CDATA[gpt-3]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[gpt-4]]></category>
            <category><![CDATA[chatgpt]]></category>
            <dc:creator><![CDATA[Assaf Yablon]]></dc:creator>
            <pubDate>Fri, 10 Mar 2023 16:40:22 GMT</pubDate>
            <atom:updated>2023-03-10T21:07:06.318Z</atom:updated>
            <content:encoded><![CDATA[<h4>The new GPT has multimodal models offering different possibilities, including videos.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/789/1*gkQHio3klxEKqipq_fGp_Q.gif" /></figure><p>Over the past few months, ChatGPT has become an increasingly prevalent tool, and it appears that Microsoft is gearing up to enhance it with an update that could once again bring it into the spotlight. This is because the company is reportedly set to launch GPT-4 as early as next week, which could potentially enable users to create AI-generated videos from simple text prompts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-rOGbNxzI5cIZeGl" /></figure><p>The news was announced by Andreas Braun, Chief Technology Officer at Microsoft Germany, during a recent event titled “AI in Focus — Digital Kickoff” (as reported by Heise). According to Braun, the upcoming GPT-4 update will include multimodal models that offer a completely different range of possibilities, such as generating videos from text inputs. With this latest development, it seems that Microsoft is looking to stay ahead of the curve when it comes to AI tools and continue to push the boundaries of what is possible in the field of machine learning.</p><p>This is the original quote by Microsoft Germany CTO Andreas Braun: “We will introduce GPT-4 next week, there we will have multimodal models that will offer completely different possibilities — for example, videos”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YvR9OKXpnSXGu0PX.jpg" /></figure><p>It’s worth noting that ChatGPT won’t be the first AI tool capable of generating videos from text prompts. Back in 2022, Meta (formerly known as Facebook) released Make-A-Video, which uses AI to create realistic videos based on short text inputs. Based on recent reports, it appears that the upcoming version of ChatGPT may also have similar capabilities.</p><p>During the AI event, Microsoft revealed that GPT-4 would be a “multimodal” tool. Holger Kenn, Director of Business Strategy at Microsoft Germany, elaborated that this would enable the AI to translate a user’s text into various formats such as images, music, and videos.</p><p>As an example of the tool’s potential, Microsoft demonstrated how a call center could utilize GPT-4 to automatically transcribe phone conversations between employees and customers into text. This would eliminate the need for staff to summarize the calls manually, saving time and effort.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8raWXYNPdtPuX5P9" /></figure><p>Interestingly, Microsoft did not discuss the integration of ChatGPT into its Bing web browser, which has recently sparked controversy. It’s possible that the company considered this to be a sensitive topic. Nevertheless, with GPT-4 slated to launch next week, we may soon see what the next iteration of ChatGPT is capable of and whether Microsoft has addressed any outstanding issues with its AI assistant.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=75d882e0260e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Nvidia has introduced “Prismer”, a new vision-language model that requires significantly less…]]></title>
            <link>https://medium.com/@yablonassaf/nvidia-has-introduced-prismer-a-new-vision-language-model-that-requires-significantly-less-748882dfb272?source=rss-ea8f25ff99bd------2</link>
            <guid isPermaLink="false">https://medium.com/p/748882dfb272</guid>
            <category><![CDATA[nvidia]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[chatgpt]]></category>
            <dc:creator><![CDATA[Assaf Yablon]]></dc:creator>
            <pubDate>Wed, 08 Mar 2023 06:36:52 GMT</pubDate>
            <atom:updated>2023-03-08T06:36:52.220Z</atom:updated>
            <content:encoded><![CDATA[<h3>Nvidia has introduced “Prismer”, a new vision-language model that requires significantly less training data!</h3><h4>What’s the person in the picture doing? “Playing baseball”</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1018/1*xOAU7EthB1ncWuOnmt7hng.png" /></figure><p>Hey, have you heard about the latest breakthrough from Nvidia? They’ve introduced a new vision-language model called Prismer designed to be highly efficient in data. How? By using an ensemble of diverse, pre-trained domain experts.</p><p>Prismer is truly impressive because it achieves fine-tuned and few-shot learning vision-language reasoning performance that’s on par with the current state-of-the-art models. And get this: it does all of that while requiring up to two orders of magnitude less training data!</p><p>In other words, Prismer is a game-changer when it comes to vision-language models. It’s able to deliver top-notch performance with <strong><em>significantly less data needed for training</em></strong>.</p><h3>Let me give you an overview</h3><p>I will start with telling you about the challenges with large pre-trained models. While they are excellent at generalizing across a wide range of tasks, they come with a significant cost. They require a lot of computational resources for both training and inference, and they also need large amounts of training data.</p><p>The challenges become even greater in the domain of vision-language learning, which requires extra skills unique to visual and multi-modal reasoning. Most solutions involve training one massive model with a massive amount of image-text data. However, this approach has its limitations.</p><p>That’s why they have developed an alternative approach with Prismer. They use distinct and separate sub-networks, or “experts,” that are optimized independently for specific tasks. This approach allows us to use domain-specific data and architectures that would not be feasible with a single large network. It also leads to improved training efficiency because the model can focus on integrating specialized skills and domain knowledge rather than trying to learn everything at once.</p><p>Prismer is a visually conditioned autoregressive text generation model designed for open-ended vision-language reasoning tasks. It utilizes powerful vision-only and language-only models for web-scale knowledge and modality-specific vision experts that encode various types of visual information. These experts include low-level vision signals such as depth and high-level vision signals such as instance and semantic labels, as a form of auxiliary knowledge.</p><p>All expert models are individually pre-trained and frozen, and they are connected through lightweight trainable components that only make up about 20% of the total network parameters. With Prismer, they were able to scale down multi-modal learning while achieving excellent performance.</p><h3>Let me tell you about the design of the Prismer model:</h3><p>I will break this down for you. Prismer is a transformer model that uses both an encoder and a decoder. The encoder takes an RGB image and its corresponding multi-modal labels as input and produces a sequence of RGB and multi-modal features. The decoder then uses these features to generate a sequence of text tokens.</p><p>The key to Prismer’s design is that it leverages pre-trained experts while keeping the number of trainable parameters to a minimum. To do this, the majority of the network weights of the pre-trained experts are frozen, ensuring their learned knowledge is maintained and preventing catastrophic forgetting.</p><p>To link the multi-modal labels and the vision and language parts of Prismer, two types of parameter-efficient trainable components are inserted:</p><ul><li>Experts Resampler: This component learns a pre-defined number of latent input queries and compresses the multi-modal features into a much smaller number of tokens as a form of auxiliary knowledge distillation.</li><li>Adaptor: This component down-projects the input features into a smaller dimension, applies a non-linearity, and then up-projects the features back to the original input dimension. With the residual connection, all adaptors are initialized with near-zero weights to approximate the identity function.</li></ul><p>Prismer is a generative model trained with a single objective to predict the next text token autoregressively. This means all vision-language reasoning tasks are reformulated as a language modeling or prefix language modeling problem. For instance, given the multi-modal tokens and a question as the prefix, the model generates the answer for the visual question-answering task, or given the multi-modal tokens, the model generates its caption for the image captioning task.</p><p>Overall, Prismer is an innovative and effective way to approach vision-language modeling, offering both open-ended and closed-ended settings for generating text tokens.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/627/1*ZdUSzcVXuTx1KeyEZCS9FQ.png" /><figcaption>A bear is sitting on the grass. Source: <a href="https://arxiv.org/pdf/2303.02506v1.pdf">2303.02506v1.pdf (arxiv.org)</a></figcaption></figure><h3>Let me give you some insights into Prismer <strong>performance</strong>.</h3><p><strong>How does Prismer’s fine-tuned performance compare to other models?</strong></p><p>Prismer and PrismerZ have demonstrated superior performance on NoCaps and VQAv2 considering their model sizes. This suggests that the strong backbone experts are responsible for good generalization, while the modality experts provide an additional boost in performance, particularly in image captioning tasks and the LARGE model variant.</p><p><strong>Can you provide examples of Prismer’s impressive performance?</strong></p><p>Despite being trained on significantly less data than BLIP and LEMON, Prismer achieved comparable image captioning performance. It also achieved VQAv2 accuracy comparable to GIT, despite being trained on much less data.</p><p><strong>How does Prismer compare to the current state-of-the-art VLMs?</strong></p><p>While there is a noticeable performance gap between Prismer and current state-of-the-art VLMs (such as CoCa, GIT-2, and PaLI), it’s important to note that these models require significantly higher training costs and access to large-scale private training data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AR4p7qe3YZa5lYIoN3fvwQ.png" /><figcaption>source: Shikun Liu</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dOmqQlxRqOn_f-7AtohfZw.png" /><figcaption>Prismer and PrismerZ outperform other similar-sized VLMs in NoCaps and VQAv2 datasets. Prismer matches performance with VLMs trained on much more data. Bubble size shows trainable parameters in pre-training.</figcaption></figure><p><strong>Could you provide further details?</strong></p><p>To accelerate the training process, the researchers trained the BASE model on a combined dataset of Conceptual Captions and SBU, containing a total of 3 million data points.</p><p>The experiments were evaluated on the VQAv2 test-dev split using a smaller resolution of [224 x 224]. Here are the results the model was able to prduce:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-ZtzTq5qGzVzLYoxemnmLg.png" /><figcaption>predicted “<em>A bottle of ck one next to a computer keyboard.”</em></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hK-a6Tvx6Gts6Ei_vEl3dA.png" /><figcaption>predicted “A couple dancing tango in front of a crowd.”</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UaPGKr4ZfCQ3Tc3MZ1JD_g.png" /><figcaption>predicted: “A man wearing a helmet and goggles with parachutes in the background.”</figcaption></figure><h3>Let me summarize the conclusions</h3><p>In this paper, the researchers introduced Prismer, a vision-language generative model designed for reasoning tasks. By leveraging an ensemble of diverse, pre-trained experts, Prismer achieved competitive performance in image captioning, VQA, and image classification benchmarks, comparable to models trained on up to two orders of magnitude more data.</p><p>However, there are some limitations to Prismer during its implementation, which the researchers discussed for full transparency.</p><p>First, Prismer was built on a small-scale language model, so it lacks the ability to perform few-shot in-context prompting by design.</p><p>Second, when experimenting with inference on a pre-trained Prismer with a different segmentation expert pre-trained on a different dataset, the model showed limited adaptability to a different expert with a different set of semantic information, leading to a notable performance drop.</p><p>Third, Prismer entangles its multi-modal features from all experts included during pre-training. Thus, only having a partial number of experts during inference would result in a significant performance drop. Although the researchers attempted to use a different training objective such as masked auto-encoding to design Prismer to reason on an arbitrary number of experts, it ultimately led to degraded fine-tuned performance.</p><p>Finally, the current design of Prismer converts all expert labels into an image-like 3-dimensional tensor via modality-specific post-processing for simplicity. Still, there are other efficient methods to represent expert knowledge, such as converting object detection into a sequence of text tokens. This may lead to stronger reasoning performance and a more stable training landscape in future works.</p><h3>Citation</h3><p>@article{liu2023prismer, title={Prismer: A Vision-Language Model with An Ensemble of Experts}, author={Liu, Shikun and Fan, Linxi and Johns, Edward and Yu, Zhiding and Xiao, Chaowei and Anandkumar, Anima}, journal={arXiv preprint arXiv:2303.02506}, year={2023} }</p><p><em>The study conducted by:</em></p><p><a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Liu%2C+S">Shikun Liu</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Fan%2C+L">Linxi Fan</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Johns%2C+E">Edward Johns</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Yu%2C+Z">Zhiding Yu</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Xiao%2C+C">Chaowei Xiao</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Anandkumar%2C+A">Anima Anandkumar</a></p><h3>Links:</h3><p>Study: <a href="https://arxiv.org/abs/2303.02506">https://arxiv.org/abs/2303.02506</a><br>Code: <a href="https://github.com/NVlabs/prismer">https://github.com/NVlabs/prismer</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=748882dfb272" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Here is the simplest illustration I could find of how ChatGPT actually works.]]></title>
            <link>https://medium.com/@yablonassaf/here-is-the-simplest-illustration-i-could-find-of-how-chatgpt-actually-works-7fb70c6a8800?source=rss-ea8f25ff99bd------2</link>
            <guid isPermaLink="false">https://medium.com/p/7fb70c6a8800</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Assaf Yablon]]></dc:creator>
            <pubDate>Tue, 07 Mar 2023 03:50:13 GMT</pubDate>
            <atom:updated>2023-03-07T03:50:13.686Z</atom:updated>
            <content:encoded><![CDATA[<h4>Let us predict the [word] in the square brackets.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sIBsEe2g6OTTZRIZmlKdRQ.png" /></figure><p>This is the Transformers library from Hugging Face.</p><p>It illustrates how words can be predicted using Transformer technology.</p><p>In the following code snippet, you can see how the [mask] is predicted, along with the likelihood for each option.</p><p>If you understand this, you are already more than halfway to understanding how ChatGPT works.</p><p><em>Now let’s dive deeper:</em></p><h3><strong>What are transformers?</strong></h3><p>Transformers are a type of neural network architecture that has been widely adopted in Natural Language Processing (NLP) tasks. They are designed to learn the relationships between words in a sentence or sequence of text by processing the entire input at once, as opposed to traditional RNNs or LSTMs which process inputs sequentially.</p><p>The core idea behind transformers is to use self-attention mechanisms to capture dependencies between all pairs of words in a sentence or sequence. This allows the model to focus on the most relevant parts of the input at each step of the processing, leading to better performance on tasks such as language translation, sentiment analysis, and text generation.</p><p>One of the main advantages of transformers over traditional neural network architectures is their ability to handle long-range dependencies in text, which is often difficult for RNNs and LSTMs. Additionally, transformers are highly parallelizable and can be trained more efficiently on large-scale datasets.</p><p>Overall, transformers have revolutionized the field of NLP and are now a standard tool for many NLP tasks. Their ability to learn complex relationships between words in text has made them a key component in many state-of-the-art NLP models, including BERT, GPT-2, and T5.</p><h3>How does it relate to ChatGPT?</h3><blockquote>Hint: “GPT” = Generative Pretrained <strong>Transformer</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fReGbO8bVrp5kRBFzkSLJA.png" /><figcaption>source: atriainnovation</figcaption></figure><p>You’re right. ChatGPT is based on Transformer technology, specifically the Generative Pretrained Transformer (GPT) architecture.</p><p>It is an advanced conversational AI model that uses state-of-the-art Transformer technology to generate natural-sounding responses to user input. GPT is a type of transformer that was first introduced by OpenAI in 2018 and has since become a widely adopted architecture for NLP tasks.</p><p>The GPT architecture is based on a transformer decoder, which is a type of neural network that generates output sequences based on input sequences. The key feature of the GPT architecture is that it is pretrained on large amounts of text data, allowing it to learn general patterns and relationships between words and phrases in language. This pretrained model can then be fine-tuned on specific tasks, such as conversation generation, to produce more accurate and context-specific responses.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wz-c39aov6qM7Gdero_lSQ.jpeg" /><figcaption>ChatGPT by OpenAI</figcaption></figure><p>ChatGPT is one such model that has been fine-tuned on a vast amount of conversational data, making it capable of generating responses that are both coherent and contextually relevant. It works by processing user input and using the learned relationships between words to generate a response that is most likely to be a natural continuation of the conversation.</p><p>The use of GPT technology in ChatGPT allows for a more natural and engaging conversation between the user and the model. ChatGPT can understand complex language structures, recognize context and intent, and generate responses that are not only accurate but also human-like in their tone and style.</p><p>Overall, ChatGPT represents a significant advancement in the field of conversational AI, offering a powerful tool for building chatbots, virtual assistants, and other applications that require natural language processing. With its sophisticated transformer architecture and advanced language modeling capabilities, ChatGPT is poised to revolutionize the way we interact with AI technology in the future.</p><h3>Q&amp;A:</h3><p><em>Amazing. I kind of get it in some way. But I wonder what makes it so much more likely to see a doctor then call a doctor.. ? Is it that this % means in the training set you could find seeing a doctor around 60% in the sentences and 17% you found calling a doctor? And that’s why the probability is much more higher that seeing is the correct answer? Or am I totally wrong?</em></p><p>Great! You’re pretty close to understanding the overall concept. The likelihood of a word being predicted is not directly proportional (it’s not linear) to the frequency of its appearance in the training data. Instead, it depends on a concept called conditional probability.</p><p>To validate your understading, if the phrase “see a doctor” appears more frequently in the data than “be a doctor”, then the model is more likely to predict the word “see” than “be” when given this phrase.</p><p>The important thing to remember here is that the performance of these models is highly dependent on the data used to train them.</p><p>— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —</p><p><strong>I encourage you to play with the code and try it for yourself.</strong></p><p>Code snippet:</p><p><a href="https://colab.research.google.com/drive/11O-4vAgefjuoOg4CusPHs8J9DyJyLHG6?usp=sharing">Link to Colab</a></p><pre># Importing the required libraries<br>from transformers import pipeline<br><br># Specifying the pipeline for the BERT model to fill in masked words<br>bert_unmasker = pipeline(&#39;fill-mask&#39;, model=&quot;bert-base-uncased&quot;)<br><br># Creating the input text with a masked word<br>text = &quot;I have to wake up in the morning and [MASK] a doctor&quot;<br><br># Running the model on the input text to predict the masked word<br>result = bert_unmasker(text)<br><br># Printing the top predictions for the masked word<br>for r in result:<br>  print(&quot;{:&lt;60}{:&lt;5.1f}%&quot;.format(r[&#39;sequence&#39;], r[&#39;score&#39;]*100))</pre><p>This is the output generated:</p><pre>i have to wake up in the morning and see a doctor    64.6 %<br>i have to wake up in the morning and call a doctor   17.8 %<br>i have to wake up in the morning and find a doctor    7.5 %<br>i have to wake up in the morning and get a doctor     5.7 %<br>i have to wake up in the morning and be a doctor      0.7 %</pre><p>If you have any questions, please reach out to me personally!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7fb70c6a8800" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Amazon outperforms GPT-3.5 by 16%!]]></title>
            <link>https://medium.com/@yablonassaf/amazon-outperforms-gpt-3-5-by-16-a0169686caaa?source=rss-ea8f25ff99bd------2</link>
            <guid isPermaLink="false">https://medium.com/p/a0169686caaa</guid>
            <category><![CDATA[amazon]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[large-language-models]]></category>
            <dc:creator><![CDATA[Assaf Yablon]]></dc:creator>
            <pubDate>Tue, 07 Mar 2023 02:58:47 GMT</pubDate>
            <atom:updated>2023-03-07T02:58:47.943Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>Amazon’s breakthrough has the potential to revolutionize the way we interact with AI.</strong></h4><p>Amazon’s recent breakthrough in natural language processing (NLP) is a significant milestone in the field of AI. The company has developed a language model with 1 billion parameters, which surpasses the previous state-of-the-art model (GPT-3.5) by 16% in terms of accuracy, achieving an impressive rate of 91.68%. This improvement can be attributed to the adoption of a new approach called chain-of-thought (CoT) prompting.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QG7o8ZY6ospxdcdTylcBzA.jpeg" /><figcaption>Overview of Amazon’s Multimodal-CoT framework consists of two stages: (i) rationale generation and (ii) answer inference. Both stages share the same model architecture but differ in the input and output. source: <a href="https://arxiv.org/pdf/2302.00923.pdf">2302.00923.pdf (arxiv.org)</a></figcaption></figure><p>CoT prompting involves generating intermediate reasoning steps for prompting demonstrations. The technique is a combination of both language and vision modalities and is accomplished through a two-stage architecture that distinguishes between generating rationales and inferring answers. This approach ensures that the answer inference process can benefit from more robust rationales generated from multimodal information (GPT-3.5).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0TQJAx_h1Zw8cTLi.png" /><figcaption>Amazon scientists are working on large-scale technical challenges in a variety of research areas across the globe. source: amazon science</figcaption></figure><p>The integration of language and vision modalities is an essential aspect of Amazon’s approach. By combining these two modalities, the model can learn to reason about both visual and textual information. This way, the model can generate explanations that are grounded in both modalities, thereby increasing the accuracy and comprehensiveness of the generated explanations.</p><p>Amazon’s approach has significant implications for NLP and AI in general. By improving the <strong>accuracy </strong>of language models, the technology can be used to develop applications that can understand and interact with human language more effectively. This has significant implications for industries such as customer service, chatbots, and content generation, where AI can assist humans in performing tasks more efficiently.</p><p>The chart below compares Amazon’s performance with that of other models.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/854/1*klVUuSDtWutOy0iYpu_kRA.png" /><figcaption>Results in bold are the best performance by Amazon. source: <a href="https://arxiv.org/pdf/2302.00923.pdf">2302.00923.pdf (arxiv.org)</a></figcaption></figure><p>Moreover, the approach has the potential to solve many problems in areas such as medicine, law, and finance, where accurate and comprehensive explanations are crucial. For example, the technology can be used to develop chatbots that can answer medical questions accurately, reducing the burden on medical personnel and enhancing the quality of care provided to patients.</p><p>In conclusion, Amazon’s breakthrough in NLP is an exciting development in the field of AI. The integration of language and vision modalities through CoT prompting has improved the accuracy of language models significantly. The technology has the potential to revolutionize the way we interact with AI and can be used to develop applications that solve many of the world’s most pressing problems.</p><p><strong>Additional examples:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/884/1*Kro8z1cMf91wcpERd8hT7A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/1*Dec5NViOsSj72YOXVdekUA.png" /></figure><p><strong>The study, code, and datasets are publicly available:</strong></p><ol><li><a href="https://arxiv.org/pdf/2302.00923.pdf">Study</a></li><li><a href="https://github.com/lupantech/ScienceQA/tree/main/data">Dataset</a></li><li><a href="https://github.com/amazon-science/mm-cot">Code</a></li><li><a href="https://www.linkedin.com/posts/yablonassaf_gpt-cot-amazonscience-activity-7033476823409692672-5GGN?utm_source=share&amp;utm_medium=member_desktop">LinkedIn</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a0169686caaa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[NVIDIA to Release Megatron530B, LLM 3x Bigger than GPT-3!]]></title>
            <link>https://medium.com/@yablonassaf/nvidia-to-release-megatron530b-llm-3x-bigger-than-gpt-3-7d3d9c7cf470?source=rss-ea8f25ff99bd------2</link>
            <guid isPermaLink="false">https://medium.com/p/7d3d9c7cf470</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[megatron530b]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Assaf Yablon]]></dc:creator>
            <pubDate>Mon, 06 Mar 2023 02:11:16 GMT</pubDate>
            <atom:updated>2023-03-06T02:11:16.868Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is going to happen?</h3><p>NVIDIA is about to release NeMo LLM, a service that provides a means of using large language models trained on various frameworks. Developers can utilize NeMo LLM to deploy enterprise AI applications on the cloud. The most thrilling aspect is that users can experience Megatron 530B, one of the largest language models, via the NeMo LLM service.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*i0LZy1QiSBKpSFOm.jpg" /><figcaption>Source: Nvidia</figcaption></figure><p><strong>What is Megatron 530B?</strong></p><p>In 2021, NVIDIA and Microsoft collaborated to create Megatron-Turing Natural Language Generation 530B, one of the world’s largest models for reading comprehension and natural language inference. This model can simplify tasks such as summarization and content generation. With 530 billion parameters based on the GPT-3 architecture, this model will soon be accessible to developers through the early access program on the NVIDIA NeMo LLM service. Model checkpoints will also be available through HuggingFace and NGC shortly. Users can already apply for early access.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Sr_NycuBIYulOiL2.png" /><figcaption>Source: Nvidia</figcaption></figure><p><strong>More about Megatron 530B:</strong></p><p>Parameters are an essential component of machine learning models as they are learned from historical training data. In the language domain, a model’s sophistication is believed to be directly proportional to the number of parameters it possesses. By having a larger number of parameters, language models can acquire a more nuanced understanding of language, allowing them to perform complex tasks such as summarizing books and generating programming code.</p><p>Nvidia partnered with Microsoft to create a dataset of 270 billion tokens taken from English-language websites to train Megatron 530B. Tokens are units of text in natural language that can be words, characters, or parts of words, and are used to break down text into smaller units. Megatron 530B learned from this dataset by analyzing examples and learning patterns among data points, such as grammatical and syntactical rules.</p><p>The majority of the dataset used to train Megatron 530B was sourced from The Pile, an 835GB collection of 22 smaller datasets created by EleutherAI, an open-source AI research effort. The Pile comprises academic sources like Arxiv and PubMed, communities such as StackExchange and Wikipedia, and code repositories like GitHub. Filtered snapshots of the Common Crawl, a vast collection of web pages including news stories and social media posts, were also combined with The Pile.</p><p>Megatron 530B has demonstrated the ability to perform basic mathematical operations even when the symbols are “badly obfuscated,” according to Nvidia. While not perfectly accurate, the model has shown the capability to understand natural language and answer questions, which is a significant advancement in natural language processing. Understanding complex linguistic structures is critical in NLP, and the ability to analyze and comprehend them is essential for creating sophisticated models.</p><p><strong>Where is Nvidia Positioned in the AI War?</strong></p><p>Nvidia wears two hats in this battle. Firstly, the company is set to play an active role by officially launching its Large Language Model Cloud. Secondly, and more significantly, Nvidia is essentially the producer of the weapons for the AI warriors: chips for generative AI. This is currently Nvidia’s advantage in the race, but the company will face strong competition in both fields. Microsoft and Google have already started developing their own specialized hardware for AI, and Amazon and Meta are also following suit.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3Qds3AEGtPW_8Nb9.PNG" /><figcaption>Source: Nvidia</figcaption></figure><p>“Large language models hold the potential to transform every industry. Millions of developers can now create language services and power scientific discoveries without needing to build a massive model from scratch.”</p><p>Jensen Huang, founder, and CEO of NVIDIA.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*buK7N0qSLXzhjnyv.jpg" /><figcaption>Source: Nvidia</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7d3d9c7cf470" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>