<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Christian Lin on Medium]]></title>
        <description><![CDATA[Stories by Christian Lin on Medium]]></description>
        <link>https://medium.com/@crlc112358?source=rss-f60365b66f2e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*3UMwW1lw-c8djqgx8D43JA.png</url>
            <title>Stories by Christian Lin on Medium</title>
            <link>https://medium.com/@crlc112358?source=rss-f60365b66f2e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 20:37:30 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@crlc112358/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Swin Transformer: Hierarchical Vision Transformer using Shifted Windows]]></title>
            <link>https://medium.com/@crlc112358/swin-transformer-hierarchical-vision-transformer-using-shifted-windows-ca1ccc8760b8?source=rss-f60365b66f2e------2</link>
            <guid isPermaLink="false">https://medium.com/p/ca1ccc8760b8</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Christian Lin]]></dc:creator>
            <pubDate>Fri, 13 Oct 2023 10:44:41 GMT</pubDate>
            <atom:updated>2023-10-13T10:44:41.293Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b1hOZBmt0aLZjPc8Xf7OhA.png" /></figure><ul><li><a href="https://arxiv.org/abs/2103.14030">Swin Transformer: Hierarchical Vision Transformer using Shifted Windows</a></li><li><a href="https://github.com/microsoft/Swin-Transformer">GitHub - microsoft/Swin-Transformer: This is an official implementation for &quot;Swin Transformer: Hierarchical Vision Transformer using Shifted Windows&quot;.</a></li></ul><p>This study introduces a new tool for computer vision called the Swin Transformer. This tool is built on the Transformer model, which was originally made for language tasks. Adapting it for vision is tricky because images and words are very different, especially in terms of size and detail. Our solution is a multi-level Transformer that uses a special windowing technique. This technique only focuses on certain parts of the image at a time but still connects them all together. This makes our tool both efficient and able to handle different image sizes. The Swin Transformer works well for many vision tasks. For instance, it’s really good at classifying images and detecting objects in them. In tests, it even did better than previous top-performing tools. Plus, our special design and windowing approach can also improve other similar tools.</p><blockquote>Preliminary</blockquote><h4>Transformer in NLP:</h4><p>Transformers first made their appearance in the groundbreaking paper “Attention Is All You Need”. Before this, sequence-to-sequence models in NLP relied heavily on recurrent neural networks (RNNs) and their variants. The Transformer architecture discarded recurrence in favor of self-attention mechanisms, providing parallel processing benefits and capturing long-range dependencies in data.</p><p><strong>Key Features:</strong> The self-attention mechanism, the heart of the Transformer, allows the model to focus differently on various parts of the input data. For instance, in a sentence, some words might be more relevant to understanding the meaning of a particular word. Self-attention weighs this relevance, allowing the model to make contextually informed decisions.</p><h4>Challenges in Adapting Transformers to Vision:</h4><ul><li><strong>Scale Variations:</strong> Unlike text where words typically have uniform significance, visual data is replete with entities of varying scales. For example, an image might contain a vast landscape and tiny birds in the distance. A model should be adept at recognizing both the landscape and the birds with equal finesse.</li><li><strong>High Resolution:</strong> Images are grids of pixels. An average image can have thousands to millions of pixels. If each pixel or a small patch is treated as a sequence token (akin to words in NLP), it becomes computationally expensive for the Transformer to process due to its quadratic complexity with sequence length.</li></ul><h4>Initial Attempts:</h4><p>Vision Transformers (ViTs) were among the first to adapt the Transformer architecture for vision. They divided images into fixed-size patches, treated each patch as a token by linearly embedding them, and then processed them as a sequence. While ViTs showcased the potential of Transformers in vision, they typically required vast amounts of data and heavy computation to excel, often relying on large-scale pre-training.</p><p><a href="https://medium.com/@crlc112358/vision-transformer-am-image-is-worth-16x16-words-transformers-for-image-recognition-at-scale-e87a03287afa">Vision Transformer ~ Am Image is Worth 16X16 Words: Transformers for Image Recognition at Scale</a></p><h4>The Need for Hierarchical Structures:</h4><p>Given the multi-scale nature of visual information, a model should ideally process data at multiple resolutions or scales. Hierarchical structures in neural networks facilitate this by having layers dedicated to different scales. This approach is reminiscent of the way convolutional neural networks (CNNs) operate, where initial layers capture fine details and deeper layers capture more abstract, larger-scale features.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*aPLFaDbZVt6wZ9ng.png" /><figcaption>Four common hierarchical structures applied in computer vision</figcaption></figure><h4>Window-Based Mechanisms:</h4><p>CNNs, the de facto standard for vision tasks before Transformers, use a mechanism where each neuron in a layer looks at a small local region (or receptive field) of the input. This allows for efficient processing of high-resolution images. Adapting a similar local processing mechanism in Transformers can ensure they’re more adept at handling visual data. Window-based mechanisms restrict the self-attention to local regions, making computations more manageable and efficient.</p><h4>Shifted Windowing:</h4><p>While local processing is efficient, it’s also crucial that different parts of an image communicate or share information. The shifted windowing technique addresses this. Imagine dividing an image into non-overlapping windows and processing them. In the next layer, these windows are shifted (like a sliding puzzle) so that they overlap with neighboring windows from the previous layer. This ensures that distant regions of an image can eventually “talk” to each other in deeper layers, providing a global context.</p><blockquote>Methodology</blockquote><h4>Hierarchical Representation with Non-overlapping Patches:</h4><p>The Swin Transformer initiates its processing by partitioning an input image into fixed-size, non-overlapping patches, analogous to the way Vision Transformers (ViTs) function. Each of these patches captures a localized segment of the image, ensuring that spatial relationships and visual cues within this segment are maintained. These patches are then linearly embedded, converting two-dimensional visual information into a sequence of tokens suitable for sequential processing.</p><p>The foundational step of translating visual data into token sequences ensures that spatial structures are not lost. Instead, they are reformatted into a representation that allows them to be processed similarly to textual sequences, leveraging the power of Transformer architectures.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xp3clTjIq-HsSBmCyecvOA.png" /><figcaption>(a) The Swin Transformer, as proposed, constructs multi-level feature maps by integrating image patches (illustrated in gray) in its more advanced layers. Its linear computational complexity relative to the input image size is a result of limiting self-attention computations to individual local windows (depicted in red). Consequently, it has the versatility to act as a foundational backbone for both image categorization and detailed recognition tasks. (b) On the other hand, earlier ViT generates feature maps at a singular, reduced resolution and exhibit quadratic computational complexity relative to image size because they compute self-attention across the entire image.</figcaption></figure><h4>Shifted Window-based Self-attention:</h4><p>Within each Transformer layer, self-attention operations are confined to specific non-overlapping windows. Rather than the traditional all-to-all attention mechanism, where every token examines every other token, in this local processing setup, a token’s attention span is restricted to its immediate window. This results in a dramatic reduction in computational overhead, as the number of attention computations is curtailed.</p><p>To ensure that the model doesn’t develop myopia, focusing only on local contexts, the windows undergo a shifting procedure across layers. This strategic shifting enables tokens in subsequent layers to be influenced by and interact with tokens from neighboring windows. Over depth, this creates a network where tokens can accumulate information from increasingly distant parts of the original image.</p><p>Through this dual approach of local windowed attention and strategic shifting, the model efficiently processes information while preserving the capacity to integrate broader context.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vOmeuIK1GAQ9Q5324rsjFw.png" /><figcaption>(a) The architecture of a Swin Transformer (Swin-T); (b) two successive Swin Transformer Blocks. W-MSA and SW-MSA are multi-head self attention modules with regular and shifted windowing configurations, respectively.</figcaption></figure><h4>Efficient batch computation for shifted configuration:</h4><p>In the Swin Transformer, self-attention is not computed globally (as with traditional transformers) but within localized windows. This ensures that tokens within a window only attend to other tokens in the same window, reducing computational complexity.</p><p>However, to ensure that tokens from different windows can interact and “see” each other at different layers of the model, the Swin Transformer uses a “shifted” windowing scheme. After one layer’s self-attention computations are completed using these localized windows, the windows are “shifted” for the next layer. This way, tokens in a window in one layer overlap with tokens from neighboring windows in the subsequent layer.</p><p>The challenge here is how to efficiently compute these shifted window-based self-attentions, especially when processing multiple data items in a batch (common in deep learning training for better GPU utilization and convergence).</p><p>Swin Transformer uses a special algorithmic arrangement to perform these shifted window self-attention computations across the entire batch of data in a unified and efficient manner. By organizing the data properly and leveraging parallel processing capabilities of modern GPUs, the Swin Transformer can rapidly compute self-attentions for these shifted windows across many data items simultaneously, without excessive memory usage or computational overhead.</p><p>This efficient computation ensures that the Swin Transformer maintains its advantages in modeling both local and global contexts without sacrificing speed or efficiency, even when trained with large batches of data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AANVE32gj84cAXbQs6QBRA.png" /><figcaption>Illustration of an efficient batch computation approach for self-attention in shifted window partitioning.</figcaption></figure><p>In this article, we provide the big picture of Swin Transformer model. If you want to know more detail about this, you can follow the links shown as follow to further understand the related foundation and knowledge.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FtFYxJZBAbE8%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DtFYxJZBAbE8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FtFYxJZBAbE8%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/cfe73d0a6d65819d24eefa9f0cc6b69e/href">https://medium.com/media/cfe73d0a6d65819d24eefa9f0cc6b69e/href</a></iframe><p>In this article, I briefly share my viewpoints on the paper. I hope you can learn more about it after reading it. I also offer the video link about the paper, hope you guys like it!!!!</p><p>If you like the article, please give me some 👏 , share the article, and follow me to learn more about the world of multi-agent reinforcement learning. You can also contact me on <a href="https://www.linkedin.com/in/bor-jiun-lin-b99b80191/"><strong>LinkedIn</strong></a>, <a href="https://www.instagram.com/imchristianbutnotchristian/?hl=zh-tw"><strong>Instagram</strong></a>, <a href="https://www.facebook.com/profile.php?id=100010297082645"><strong>Facebook</strong></a><strong> </strong>and<strong> </strong><a href="https://github.com/ChristianLin0420"><strong>Github</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ca1ccc8760b8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Vision Transformer ~ Am Image is Worth 16X16 Words: Transformers for Image Recognition at Scale]]></title>
            <link>https://medium.com/@crlc112358/vision-transformer-am-image-is-worth-16x16-words-transformers-for-image-recognition-at-scale-e87a03287afa?source=rss-f60365b66f2e------2</link>
            <guid isPermaLink="false">https://medium.com/p/e87a03287afa</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Christian Lin]]></dc:creator>
            <pubDate>Sun, 08 Oct 2023 09:11:58 GMT</pubDate>
            <atom:updated>2023-10-08T09:11:58.898Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N_ZPycR85puMXGcO3m18wA.png" /></figure><ul><li><a href="https://arxiv.org/abs/2010.11929">An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale</a></li><li><a href="https://github.com/google-research/vision_transformer">GitHub - google-research/vision_transformer</a></li></ul><p>In the ever-evolving landscape of machine learning, the “Vision Transformer” (ViT) marks a significant departure from conventional wisdom. Traditionally, convolutional neural networks (CNNs) have reigned supreme in visual tasks, tailored to process images by focusing on local spatial hierarchies. The Vision Transformer, as presented in the groundbreaking paper, challenges this norm. Rather than leaning on convolutions, ViT employs a transformer architecture, predominantly used for natural language processing tasks until now. The model divides images into fixed-size patches, linearly embeds them, and then processes them sequentially using the transformer’s self-attention mechanism. Remarkably, when provided with ample data and computational power, ViT not only rivals but often surpasses state-of-the-art CNNs on major vision benchmarks. Its success heralds a potential shift in computer vision, underscoring the versatility and power of the transformer architecture beyond textual data.</p><blockquote>Preliminary</blockquote><h4>Historical context:</h4><p>Computer vision’s evolutionary journey has been predominantly marked by the ascendancy of convolutional neural networks (CNNs). These networks, with their specialized design, focus on local spatial hierarchies — a feature that has proven to be indispensable for interpreting image data. Over the years, from academic benchmarks like ImageNet to myriad real-world applications, CNNs have consistently established and broken performance records, unequivocally emerging as the go-to architecture for visual tasks. <em>Figure Suggestion</em>: A timeline illustrating the evolution of image recognition models leading up to CNNs.</p><h4>The Transformer Revolution:</h4><p>In the realm of natural language processing, a parallel revolution unfolded with the introduction of the Transformer architecture. Originating from the seminal “Attention is All You Need” paper, the Transformer’s self-attention mechanism offered dynamic, contextually rich sequence representations. In a short span, it unseated many existing models, quickly becoming the de facto standard for processing textual data.</p><p>If you want to recap the knowledge background of Transformer, you can reference the following previous article.</p><p><a href="https://medium.com/@crlc112358/transformer-attention-is-all-you-need-8e617307874a">Transformer ~ Attention is All You Need</a></p><h4>Bridging Two Worlds:</h4><p>Merging the worlds of vision and text, the “Vision Transformer” paper dared to explore uncharted waters. It postulated a provocative question: Is it feasible for the transformer, inherently designed for sequential textual data, to challenge, or even surpass, the feats achieved by CNNs in visual domains? The answer is shown in paper. Yes, it is possible!!!</p><p>The Vision Transformer’s methodology embodies elegance and innovation. Instead of the traditional approach of processing images pixel-by-pixel or via local convolutional filters, the Vision Transformer adopts a patch-based mechanism. An image is dissected into a grid of fixed-sized patches. These patches, once transformed into linear embeddings, are sequenced and processed by the transformer. This paradigm shift transforms the perception of images from a 2D spatial entity to a sequence of informational patches.</p><blockquote>Methodology</blockquote><h4>From Image to Sequence:</h4><p>At the heart of the Vision Transformer (ViT) lies a radical departure from traditional image processing. Instead of treating images as 2D arrays of pixels, ViT interprets them as sequences. The first step involves partitioning each image into fixed-sized patches, akin to chopping up a picture into uniform tiles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/930/0*N-r9aV3nU1PF0_ws.png" /><figcaption>An illustration of an image being divided into uniform patches.</figcaption></figure><h4>Linear Embeddings &amp; Positional Information:</h4><p>Each of these patches is then linearly embedded into a flat vector. To retain the spatial configuration and position of each patch within the image, positional embeddings are added. This transformed sequence of vectors, now holding both visual and positional information, is ready to be processed by the transformer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hyMr7cFqnl7AGEVunLtydg.png" /><figcaption>Left: Filters of the initial linear embedding of RGB values of ViT-L/32. Center: Similarity of position embeddings of ViT-L/32. Tiles show the cosine similarity between the position embedding of the patch with the indicated row and column and the position embeddings of all other patches. Right: Size of attended area by head and network depth. Each dot shows the mean attention distance across images for one of 16 heads at one layer. See Appendix D.7 for details.</figcaption></figure><h4>Leveraging the Transformer Architecture:</h4><p>The core processing engine for these sequences is the transformer architecture, renowned for its efficacy in natural language processing tasks. The self-attention mechanism within the transformer processes these sequences, ensuring each patch is contextually aware of every other patch in the image.</p><h4>Classification Head &amp; Output:</h4><p>After traversing through the transformer’s layers, the sequence reaches the classification head. ViT uses the representation corresponding to the “<strong>CLASS</strong>” token (a special token introduced at the beginning of the sequence) to generate the final output, usually through a linear layer followed by a softmax for classification tasks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*TZCTipOEfYgYrm2nG74OPA.png" /><figcaption>An illustration showcasing the ‘class’ token’s representation being processed to produce the final class probabilities.</figcaption></figure><blockquote>Experiments</blockquote><h4>Comparison to SOTAs:</h4><p>They wanted to see how two biggest models, ViT-H/14 and ViT-L/16, perform when compared with the best image-recognition models out there. They first looked at Big Transfer (BiT), which is a model that learns by using other large image-recognition models. The next one is Noisy Student, which is trained on a mix of labeled and unlabeled images. Right now, Noisy Student is the best at recognizing images from the ImageNet dataset.</p><p>In the following table, ViT-L/16 model was better than BiT-L, even though they used the same training data, and the proposed model was quicker to train. The ViT-H/14 model did even better, especially with tougher datasets like ImageNet and CIFAR-100. What’s amazing is that it still trained faster than other top models. But remember, many factors can impact training speed, like how the training is set up, which computer algorithms are used, and more.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1FwPWRcNUddNfT9LuIBSbw.png" /><figcaption>Comparison with state of the art on popular image classification benchmarks.</figcaption></figure><p>In this section we provide the big picture of Vision Transformer model. If you want to know more detail about ViT, you can follow the links shown as follow to further understand the related foundation and knowledge.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FHZ4j_U3FC94%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DHZ4j_U3FC94&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FHZ4j_U3FC94%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/49816a3bb25b04361cff7ca281f9659e/href">https://medium.com/media/49816a3bb25b04361cff7ca281f9659e/href</a></iframe><p>In this article, I briefly share my viewpoints on the paper. I hope you can learn more about it after reading it. I also offer the video link about the paper, hope you guys like it!!!!</p><p>If you like the article, please give me some 👏 , share the article, and follow me to learn more about the world of multi-agent reinforcement learning. You can also contact me on <a href="https://www.linkedin.com/in/bor-jiun-lin-b99b80191/"><strong>LinkedIn</strong></a>, <a href="https://www.instagram.com/imchristianbutnotchristian/?hl=zh-tw"><strong>Instagram</strong></a>, <a href="https://www.facebook.com/profile.php?id=100010297082645"><strong>Facebook</strong></a><strong> </strong>and<strong> </strong><a href="https://github.com/ChristianLin0420"><strong>Github</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e87a03287afa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transformer ~ Attention is All You Need]]></title>
            <link>https://medium.com/@crlc112358/transformer-attention-is-all-you-need-8e617307874a?source=rss-f60365b66f2e------2</link>
            <guid isPermaLink="false">https://medium.com/p/8e617307874a</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <dc:creator><![CDATA[Christian Lin]]></dc:creator>
            <pubDate>Sat, 07 Oct 2023 11:12:39 GMT</pubDate>
            <atom:updated>2023-10-07T11:12:39.768Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EeNnfYb0MuJCxvpdl3nrWw.png" /></figure><ul><li><a href="https://arxiv.org/abs/1706.03762">Attention Is All You Need</a></li><li><a href="https://github.com/jadore801120/attention-is-all-you-need-pytorch">GitHub - jadore801120/attention-is-all-you-need-pytorch: A PyTorch implementation of the Transformer model in &quot;Attention is All You Need&quot;.</a></li></ul><p>In the groundbreaking paper “Attention is All You Need,” researchers from Google introduced the Transformer architecture, a novel approach to handling sequential data in machine learning. Departing from traditional recurrent models like LSTMs and GRUs, the Transformer solely relies on attention mechanisms, effectively allowing it to weigh the significance of different parts of an input sequence without the constraints of their order. This innovative design has paved the way for unparalleled performance in tasks like translation, outperforming the then state-of-the-art models. Furthermore, the Transformer’s architecture has since become foundational, spawning successors such as BERT, GPT, and more that dominate today’s natural language processing landscape. The paper is a testament to the transformative power of attention mechanisms in deep learning.</p><blockquote>Preliminary</blockquote><h4>The Landscape of Sequence-to-Sequence Learning:</h4><p>Before the arrival of the Transformer model, sequence-to-sequence learning was largely dominated by models like RNNs, LSTMs, and GRUs. These recurrent architectures processed input sequences step by step, maintaining an internal state from one step to the next. While effective, they had limitations in handling long-term dependencies and were computationally intensive.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uQl2cnE54bMGsUTO.png" /><figcaption>A sequence-to-sequence model using an LSTM</figcaption></figure><h4>The Rise of Attention Mechanisms:</h4><p>The attention mechanism emerged as a solution to enhance the capabilities of recurrent models. By allowing a model to focus on different parts of the input for different tasks, attention offered a way to capture context more effectively. Early integration of attention into RNNs and LSTMs showcased promising improvements, particularly in tasks like machine translation.</p><h4>The Limitations of Existing Paradigms:</h4><p>Despite the advancements brought by attention, challenges persisted. Recurrent models, even with attention, were limited by their inherent sequential processing nature. The sequentiality posed both computational challenges, as parallelization was constrained, and representational challenges, as capturing very long-term dependencies remained tricky.</p><h4>A Glimpse into the Paper’s Contribution:</h4><p>“Attention is All You Need” is not just a title; it’s a statement of the paper’s primary thesis. Instead of combining attention with recurrent structures, the researchers proposed an architecture where attention stands alone at the center stage. This paper elucidates this novel design, illustrating how it not only overcomes the limitations of its predecessors but also sets new benchmarks in sequence modeling tasks.</p><h4>Significance in the AI Landscape:</h4><p>The Transformer model, introduced in this paper, has had reverberating impacts on the world of AI. As readers will discover, its design principles have become foundational, giving rise to several state-of-the-art models in Natural Language Processing and beyond.</p><blockquote>Methodology</blockquote><h4>Overview:</h4><p>The Transformer model, unveiled in this paper, offers a fresh perspective on handling sequence transduction tasks. Instead of banking on the recurring structures found in previous models, the Transformer harnesses the unbridled power of attention mechanisms. This architectural nuance facilitates increased parallel processing and exhibits robust efficiency when grappling with dependencies over long sequences.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/555/1*13RpcG91sax853jdVYVPtQ.png" /><figcaption>An overview of Transformer Architecture</figcaption></figure><h4>Self-Attention Mechanism:</h4><p>Central to the Transformer’s prowess is the self-attention mechanism. In a marked departure from traditional methods, this mechanism facilitates interactions between any given word and every other word in a sequence. The result is a richer, more context-aware representation that adeptly captures the sequence’s semantic nuances.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*TXIFhtj-3v1P2eB-i2FO8g.png" /><figcaption><em>An infographic illustrating the self-attention process</em></figcaption></figure><h4>Multi-Head Attention:</h4><p>The Transformer’s design philosophy champions the diversity of attention. It doesn’t remain confined to one attention perspective; instead, it employs multiple heads that attend to distinct portions of the input in tandem. This facilitates the capture of varied relationships and nuances within the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/0*aw7gBycWOxrXE9Qv.png" /><figcaption><em>A depiction of multiple attention heads module</em></figcaption></figure><h4>Positional Encoding:</h4><p>In the Transformer’s architecture, there’s no inherent sense of position in a sequence, making positional encodings a necessity. These encodings, when combined with the input embeddings at the foundational layers of the encoder and decoder, equip the model with indispensable positional information.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XCcPCKi46IBAla8n.png" /><figcaption>To give the model a sense of the order of the words, we add positional encoding vectors — the values of which follow a specific pattern.</figcaption></figure><p>In this section we provide the big picture of Transformer model. If you want to know more detail about Transformer, you can follow the links shown as follow to further understand the related foundation and knowledge.</p><p><a href="http://jalammar.github.io/illustrated-transformer/">The Illustrated Transformer</a></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FD0COTRYrVEs%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DD0COTRYrVEs&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FD0COTRYrVEs%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/35b28df19b24db532b3f8fce7f9d4020/href">https://medium.com/media/35b28df19b24db532b3f8fce7f9d4020/href</a></iframe><p>In this article, I briefly share my viewpoints on the paper. I hope you can learn more about it after reading it. I also offer the video link about the paper, hope you guys like it!!!!</p><p>If you like the article, please give me some 👏 , share the article, and follow me to learn more about the world of multi-agent reinforcement learning. You can also contact me on <a href="https://www.linkedin.com/in/bor-jiun-lin-b99b80191/"><strong>LinkedIn</strong></a>, <a href="https://www.instagram.com/imchristianbutnotchristian/?hl=zh-tw"><strong>Instagram</strong></a>, <a href="https://www.facebook.com/profile.php?id=100010297082645"><strong>Facebook</strong></a><strong> </strong>and<strong> </strong><a href="https://github.com/ChristianLin0420"><strong>Github</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8e617307874a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Computer Vision Paper ~ DiffPose: Toward More Reliable 3D Pose Estimation]]></title>
            <link>https://medium.com/@crlc112358/computer-vision-paper-diffpose-toward-more-reliable-3d-pose-estimation-addf42bb7f79?source=rss-f60365b66f2e------2</link>
            <guid isPermaLink="false">https://medium.com/p/addf42bb7f79</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[epose]]></category>
            <category><![CDATA[aritificial-intelligence]]></category>
            <category><![CDATA[diffusion-models]]></category>
            <dc:creator><![CDATA[Christian Lin]]></dc:creator>
            <pubDate>Thu, 22 Jun 2023 14:42:20 GMT</pubDate>
            <atom:updated>2023-06-22T14:42:20.178Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/806/1*EhpAKs6mLd35RseCDidp6Q.png" /></figure><ul><li><a href="https://arxiv.org/abs/2211.16940">DiffPose: Toward More Reliable 3D Pose Estimation</a></li><li><a href="https://github.com/GONGJIA0208/Diffpose">GitHub - GONGJIA0208/Diffpose: [CVPR 2023] DiffPose: Toward More Reliable 3D Pose Estimation</a></li></ul><p>3D human pose estimation from a single image is a challenging task due to the inherent ambiguity and occlusion that can lead to high uncertainty and indeterminacy. Diffusion models, which have recently been shown to be effective for generating high-quality images from noise, offer a promising approach to this problem. In this paper, they propose a novel pose estimation framework, DiffPose, that formulates 3D pose estimation as a reverse diffusion process. Our DiffPose framework incorporates several novel designs to facilitate the diffusion process for 3D pose estimation, including a pose-specific initialization of pose uncertainty distributions, a Gaussian Mixture Model-based forward diffusion process, and a context-conditioned reverse diffusion process. They evaluate our proposed DiffPose on two widely used pose estimation benchmarks, Human3.6M and MPI-INF-3DHP, and show that it significantly outperforms existing methods.</p><blockquote><strong><em>Preliminary</em></strong></blockquote><h4>Diffusion models</h4><p>Diffusion models are a type of generative model that can be used to generate realistic images, text, and other data. They work by starting with a random noise image and then gradually adding noise to the image until it becomes a realistic image. The process of adding noise is called diffusion, and the model is trained to reverse the diffusion process to generate new images.</p><p>Diffusion models are relatively easy to train and can be used for a variety of applications, such as image denoising, image generation, and text-to-image synthesis.</p><p>Here is a simple analogy to help you understand diffusion models. Imagine you have a piece of paper and you start by drawing a random squiggle on it. Then, you gradually add more and more noise to the squiggle until it becomes unrecognizable. Now, imagine you are given the squiggle with all the noise added to it, and you are asked to recreate the original squiggle. This is essentially what diffusion models do. They are trained on a dataset of images, and they learn to reverse the diffusion process to recreate the original images from the noisy versions.</p><p>If you want to know more about Diffusion models, please reference the following links to know more!!!</p><ul><li><a href="https://medium.com/@crlc112358/computer-vision-paper-denoising-diffusion-probabilistic-models-78bc38bc98b8">Computer Vision Paper ~ Denoising Diffusion Probabilistic Models</a></li><li><a href="https://medium.com/@crlc112358/computer-vision-paper-denoising-diffusion-implicit-models-e2b5807e83b8">Computer Vision Paper ~ Denoising Diffusion Implicit Models</a></li><li><a href="https://medium.com/@crlc112358/computer-vision-paper-high-resolution-image-synthesis-with-latent-diffusion-models-65fcf3191539">Computer Vision Paper ~High-Resolution Image Synthesis with Latent Diffusion Models</a></li></ul><blockquote><strong><em>Methodology</em></strong></blockquote><p>The following figure demonstrate the overview of the Diffpose. In the following subsections, I will introduce the methodology of Diffpose.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zMrHUdHz98dV_9cmqdhjNg.png" /></figure><h4>Initializing 3D Pose Distribution H_K</h4><p>In previous diffusion models, the reverse diffusion process starts with random noise and gradually denoises it to generate a high-quality output. However, in 3D pose estimation, our input is an estimated 2D pose that has its own uncertainty in 3D space. To help our diffusion model handle this uncertainty, they initialize a corresponding 3D pose distribution H_K that captures the uncertainty of the 3D pose. This way, the reverse diffusion process can start from the distribution H_K with sample-specific knowledge, which leads to better performance.</p><p>Here is how they construct the x, y, and z uncertainty distribution for each joint of an input pose:</p><ol><li>First estimate the uncertainty of each joint in 2D space.</li><li>Then use these uncertainties to construct a 3D pose distribution HK.</li><li>The x, y, and z uncertainty distributions for each joint are then sampled from the distribution HK.</li></ol><p>This process allows our diffusion model to start from a more realistic and informative initialization, which leads to better performance.</p><p><strong><em>Initializing (x, y, z) distribution.</em></strong> To initialize the x and y uncertainty distributions, they use the heatmaps from an off-the-shelf 2D pose detector. These heatmaps reveal the uncertainty of the 2D pose predictions, so they are a good way to model the uncertainty of the x and y joint locations.</p><p>The z uncertainty distribution is more difficult to obtain, because it is not known by the 2D pose detector. One way to compute the z distribution is to calculate the occurrence frequencies of z values in the training data. Another approach is to use the Context Encoder to initialize the z distribution. They empirically observe that this approach leads to faster convergence.</p><h4>Forward Pose Diffusion</h4><p>Designing a forward diffusion process for 3D pose estimation can be challenging, because the uncertainty distribution H_K, which is based on heatmaps, often has irregular and complex shapes. It is not straightforward to express H_K mathematically.</p><p>To overcome this, they propose to use a Gaussian Mixture Model (GMM) to model the uncertainty distribution H_K. A GMM is a probabilistic model that can characterize intractable and complex distributions. It is also very effective to represent heatmap-based distributions.</p><p>They set the number of Gaussian components in the GMM at M. Then, they use the Expectation-Maximization (EM) algorithm to optimize the GMM parameters φ_GMM to fit the target distribution H_K.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/859/1*XQxfHjVvSsmhgSLz7yxm-w.png" /></figure><h4>Reverse Diffusion for 3D Pose Estimation</h4><p><strong><em>Context Encoder φST. </em></strong>Performing the reverse diffusion process solely using Hˆ_K as input for the diffusion model g presents challenges. This is due to the limited context information available from the input videos/images, making it difficult for g to generate precise poses from the ambiguous distribution H_K. To address this issue, we propose leveraging the context information provided by the input to guide g and improve the accuracy of its predictions. By incorporating the observed inputs, the context information can restrict the model’s denoising process and steer it towards generating more precise predictions.</p><p>To guide the diffusion model, they utilize the spatial-temporal context. This context information is obtained from the 2D pose sequence derived from V_t (or a single 2D pose derived from I_t if V_t is not available). By incorporating this context information, they enhance the reverse diffusion process and provide the diffusion model g with additional information. This aids in reducing uncertainty and improving the accuracy of the generated 3D poses. To facilitate this, they introduce the Context Encoder φ_ST, which extracts spatial-temporal information fST from the 2D pose sequence. The reverse diffusion process is then conditioned on f_ST.</p><p><strong><em>Reverse Diffusion Process.</em></strong> In the reverse diffusion process, the objective is to recover a definite pose distribution H_0 from the indeterminate pose distribution Hˆ_K during training, or H_K during testing. In this description, we will use the notation Hˆ_K. Initially, we utilize the Context Encoder φ_ST to extract f_ST from the 2D pose sequence. Additionally, to enable the diffusion model to learn appropriate denoising techniques at each diffusion step, they generate a unique step embedding f_Dk using a sinusoidal function to represent the k-th diffusion step. Given a noisy pose hˆ_k sampled from Hˆ_k, we use the diffusion model, conditioned on the diffusion step k and the spatial-temporal context feature f_ST, to progressively reconstruct hˆ_{k−1} from hˆ_k. The process is as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/661/1*CtD8_tg-BiwFK2ll-07taQ.png" /></figure><p>This paper introduces DiffPose, a new framework based on diffusion, designed to address the challenges of uncertainty and indeterminacy in monocular 3D pose estimation. DiffPose starts by initializing the indeterminate 3D pose distribution and then iteratively reduces the uncertainty in this distribution to obtain a reliable and high-quality 3D human pose distribution for accurate pose estimation. Through extensive experiments, DiffPose has demonstrated state-of-the-art performance on two commonly used benchmark datasets.</p><p>In this article, I briefly share my viewpoints on the paper. I hope you can learn more about it after reading it. I also offer the video link about the paper, hope you guys like it!!!!</p><p>If you like the article, please give me some 👏 , share the article, and follow me to learn more about the world of multi-agent reinforcement learning. You can also contact me on <a href="https://www.linkedin.com/in/bor-jiun-lin-b99b80191/"><strong>LinkedIn</strong></a>, <a href="https://www.instagram.com/imchristianbutnotchristian/?hl=zh-tw"><strong>Instagram</strong></a>, <a href="https://www.facebook.com/profile.php?id=100010297082645"><strong>Facebook</strong></a><strong> </strong>and<strong> </strong><a href="https://github.com/ChristianLin0420"><strong>Github</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=addf42bb7f79" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Computer Vision Paper ~ PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human…]]></title>
            <link>https://medium.com/@crlc112358/computer-vision-paper-poseformerv2-exploring-frequency-domain-for-efficient-and-robust-3d-human-232ac17a483d?source=rss-f60365b66f2e------2</link>
            <guid isPermaLink="false">https://medium.com/p/232ac17a483d</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[pose-estimation]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Christian Lin]]></dc:creator>
            <pubDate>Sun, 18 Jun 2023 13:58:05 GMT</pubDate>
            <atom:updated>2023-06-18T13:58:05.779Z</atom:updated>
            <content:encoded><![CDATA[<h3>Computer Vision Paper ~ PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimation</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/773/1*BKI0h3_E5r10EFx5uM-5tg.png" /></figure><p><a href="https://openaccess.thecvf.com/content/CVPR2023/html/Zhao_PoseFormerV2_Exploring_Frequency_Domain_for_Efficient_and_Robust_3D_Human_CVPR_2023_paper.html">CVPR 2023 Open Access Repository</a></p><p>In recent times, there have been successful methods using transformers to estimate human poses in 3D from 2D sequential data. One such method called PoseFormer has achieved impressive results by analyzing the spatial relationships between human joints in each video frame and considering the motion across frames using transformer layers. However, in real-world situations, PoseFormer and similar methods face two limitations:</p><ol><li>The length of the input joint sequence.</li><li>The accuracy of detecting the 2D joints. Existing methods often use self-attention on all frames of the input sequence, which becomes computationally expensive when trying to improve the accuracy of the estimation with more frames, and they struggle with the noise introduced by imperfect 2D joint detection.</li></ol><p>To address these challenges, they introduce PoseFormerV2. This new method efficiently handles lengthy sequences of joint data by representing them in a more compact form in the frequency domain. This allows us to expand the analysis of the joint relationships and enhance the robustness to noisy 2D joint detection. PoseFormerV2 builds upon the original PoseFormer with minimal modifications, effectively combining features from both the time and frequency domains. This results in a better balance between speed and accuracy compared to its predecessor. Through extensive experiments on two widely-used datasets (Human3.6M and MPI-INF-3DHP), they demonstrate that theirproposed approach significantly outperforms the original PoseFormer and other transformer-based methods.</p><blockquote><strong><em>Preliminary</em></strong></blockquote><h4>PoseFormer</h4><p>The following figure show the overall PoseFormer architecture. In the following paragraph, I will explain all details about PoseFormer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/1*iZUIw1jdVDQ120iVEYb86g.png" /></figure><p><strong><em>The spatial transformer encoder</em></strong> is responsible for capturing spatial relationships among the joint embeddings of each frame, represented as z^i_0 ∈ R^{1×J×c}. In this process, the self-attention mechanism is used. Each transformer block takes J tokens as input. The output of the spatial transformer encoder after L layers for the i-th frame is represented as z^i_L ∈ R^{1×J×c}.</p><p>Afterwards, the per-frame representations are flattened and concatenated, resulting in an input Z_0 ∈ R^{F ×(J · c)} for the temporal transformer encoder.</p><p><strong><em>The temporal transformer encoder</em></strong> operates similarly. The input Z_0 is combined with a learnable temporal positional embedding E_{T_Pos} ∈ R^{F ×(J · c)} to incorporate index-related information for each frame. The temporal encoder consists of M transformer layers, which effectively model the dependencies between frames throughout the entire sequence. The output of the temporal encoder is represented as ZM ∈ R^{F ×(J · c)}.</p><p>In this stage, the number of tokens for each transformer layer is F, which corresponds to the length of the input sequence.</p><p><strong><em>Regression head</em></strong> is responsible for estimating the 3D pose of the central frame within the sequence. It utilizes a straightforward approach: a 1D convolution is employed to gather temporal information, and a linear projection generates the final pose representation y ∈ R^{1×(J·3)}. This representation consists of J joints, with each joint having a 3D coordinate (x, y, z).</p><h4>Limitation of PoseFormer</h4><p>Modeling the dependencies between joints within each frame and capturing human motions across frames using transformer layers is a straightforward approach. This dense modeling technique leads to improved accuracy in pose estimation. However, it comes with computational challenges, particularly when the input sequence length increases. The computation required for self-attention grows quadratically with the token number, which refers to the joint number in the spatial encoder and the sequence length in the temporal encoder. This becomes especially problematic when the input sequence length becomes larger.</p><p>While the token number for spatial transformer layers (joint number) remains constant regardless of the frame number, it’s important to note that the sequence length indirectly impacts the computational requirements of the spatial encoder. This is because GPUs have limited parallelization capabilities. Therefore, increasing the sequence length affects the computational efficiency of the spatial encoder in real-world scenarios.</p><p>Apart from efficiency concerns, the original PoseFormerV1 method is also sensitive to the quality of the 2D joint detection used as input.</p><blockquote><strong><em>Methodology</em></strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TEEMORF2iGFLAnqgaYf1gQ.png" /></figure><h4>Frequency Representation of Skeleton Sequence</h4><p>The proposed approach involves transforming the input skeleton sequence into the frequency domain using Discrete Cosine Transform (DCT) and utilizing only a subset of low-frequency coefficients. DCT coefficients encode different levels of temporal information for the input time series. The low-frequency coefficients capture the overall shape or contour of the sequence, while the high-frequency coefficients represent the finer details such as jitters or sharp changes.</p><p>To illustrate why they chose this representation, let’s consider an example of an 81-frame joint trajectory of the “Directions” action from the Human3.6M dataset, detected using the CPN method. They show the reconstructions of the trajectory using the first 3, 9, and 27 DCT coefficients in Figure 3. As they increase the number of DCT coefficients used, the reconstructed trajectory becomes closer to the raw input but becomes less smooth. With only 3 DCT coefficients (represented by the orange curve), they capture the overall trend of the original trajectory. With 9 and 27 coefficients (represented by the pink and green curves), they preserve the characteristics of the raw sequence while removing high-frequency noise (zig-zags).</p><p>Based on these observations, they leverage a small number of informative low-frequency DCT components from the input joint sequence as a compact and denoised representation in their work. This significantly reduces the effective length of the input sequence and enhances the robustness of their model against noise present in 2D joint detection.</p><h4>Architecture</h4><p><strong><em>Spatial transformer encoder</em></strong> operates as follows: given a 2D skeleton sequence x ∈ R^{F ×J ×2}, where F represents the sequence length (typically 81 frames) and J represents the number of joints, they first sample a smaller subset of frames, denoted as F’ (usually much smaller than F), centered around the sequence center (frame index 0 in above figure). These sampled frames, denoted as x ∈ R^{F’×J×2}, are then provided as input to the spatial encoder.</p><p>The spatial encoder processes the sampled frames and produces an output denoted as z ∈ R^{F’×(J·c)}, which represents frame-level features in the time domain. The design of the spatial encoder is directly inspired by PoseFormerV1.</p><p>To capture the long-range human dynamics present in the original sequence, they leverage its frequency-domain representation. The output zTime from the spatial encoder is considered “short-sighted” as its receptive field (F’) is limited compared to the entire sequence length (F).</p><p><strong><em>Low-frequency DCT coefficients</em></strong>. To efficiently utilize the long-range information, they convert the full sequence x ∈ R^{F ×J ×2} into DCT coefficients, denoted as C ∈ R^{F ×J ×2}. They then apply a low-pass filter and keep only the first N coefficients C’ ∈ R^{N ×J ×2} for each joint trajectory. This selection retains the temporal information of the original sequence while eliminating high-frequency noise. The low-frequency coefficients C’ are flattened and linearly projected to obtain z^{Freq} ∈ R^{N ×(J ·c)}, which represents the embedding of the frequency coefficients.</p><p>Next, z^{Freq} is combined with a learnable frequency positional embedding E_{FPos}, similar to the ETPos used in PoseFormerV1. The features from both the time domain (z^{Time}) and the frequency domain (z^{Freq}) are concatenated together, resulting in a combined representation. This process is formulated as</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/485/1*oCejc-5_z9iim6kcXqJT9Q.png" /></figure><p><strong><em>Time-Frequency Feature Fusion</em></strong>. In their approach, they utilize transformer layers for modeling temporal dependencies across frames, similar to PoseFormerV1. However, unlike PoseFormerV1, which only extracts features in the time domain, their proposed method combines features from both the time domain and the frequency domain. To bridge the gap between these two domains, they introduce some modifications to the vanilla transformer layers.</p><p>Firstly, the self-attention mechanism is shared between the time-domain and frequency-domain features. However, separate feed-forward networks are used for each domain. Additionally, they incorporate FreqMLP (Frequency Multi-Layer Perceptron) in the feed-forward networks specifically for the time-domain features zTime (refer to Figure 4b and 4c).</p><p>In their FreqMLP, they apply Discrete Cosine Transform (DCT) before the vanilla MLP and Inverse Discrete Cosine Transform (IDCT) after the MLP. The idea behind this is to filter out high-frequency noise from the frequency domain features using a low-pass filter. However, it’s possible that some detailed human motion features, such as fast local movements, may also be lost as noise. To address this, FreqMLP acts as a trainable frequency-domain filter, allowing us to dynamically adjust the weight of each frequency component in the embedding of 2D joint coordinates (i.e., time-domain features). It serves as a complement to the frequency features. These modules are formulated as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/733/1*9G8D3FgQRjiWgp52IajJ0g.png" /></figure><p>In the given formulation, MSA represents Multi-head Self-Attention, and F’ represents the number of sampled central frames. It’s crucial to acknowledge that the concatenation operation increases the number of tokens for the transformer. However, they address this by limiting the spatial encoder to observe only a restricted number of central frames and incorporating a small fraction of low-frequency DCT coefficients to expand its receptive field. This approach allows for flexible reduction of overall computation. Moreover, it not only reduces computational costs but also improves the model’s resilience to noise in comparison to PoseFormerV1.</p><p>In this article, I briefly share my viewpoints on the paper. I hope you can learn more about it after reading it. I also offer the video link about the paper, hope you guys like it!!!!</p><p>If you like the article, please give me some 👏 , share the article, and follow me to learn more about the world of multi-agent reinforcement learning. You can also contact me on <a href="https://www.linkedin.com/in/bor-jiun-lin-b99b80191/"><strong>LinkedIn</strong></a>, <a href="https://www.instagram.com/imchristianbutnotchristian/?hl=zh-tw"><strong>Instagram</strong></a>, <a href="https://www.facebook.com/profile.php?id=100010297082645"><strong>Facebook</strong></a><strong> </strong>and<strong> </strong><a href="https://github.com/ChristianLin0420"><strong>Github</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=232ac17a483d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Large Language Model Paper Reading~LoRA: Low-Rank Adaptation of Large Language Models]]></title>
            <link>https://medium.com/@crlc112358/large-language-model-paper-reading-is-conditional-generative-modeling-all-you-need-for-df343b0ce96?source=rss-f60365b66f2e------2</link>
            <guid isPermaLink="false">https://medium.com/p/df343b0ce96</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[language-model]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Christian Lin]]></dc:creator>
            <pubDate>Sat, 10 Jun 2023 08:35:20 GMT</pubDate>
            <atom:updated>2023-06-10T08:37:08.355Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*e5sUai2vY4mdBuKCh2qkwQ.png" /></figure><p><a href="https://arxiv.org/abs/2106.09685">LoRA: Low-Rank Adaptation of Large Language Models</a></p><p>A significant approach in natural language processing involves training large-scale models on general data and adapting them to specific tasks or areas. However, as we continue to develop larger models, fully retraining all model parameters becomes increasingly impractical. For instance, deploying individual instances of fine-tuned models based on GPT-3 175B, each with 175 billion parameters, is extremely costly. To address this challenge, we propose a method called Low-Rank Adaptation (LoRA). LoRA involves freezing the pretrained model weights and introducing trainable rank decomposition matrices into each layer of the Transformer architecture. This technique significantly reduces the number of trainable parameters needed for downstream tasks. In comparison to fine-tuning GPT-3 175B with Adam, LoRA can decrease the number of trainable parameters by 10,000 times and reduce GPU memory requirements by threefold. Despite having fewer trainable parameters, LoRA achieves comparable or even superior model quality compared to fine-tuning methods like RoBERTa, DeBERTa, GPT-2, and GPT-3. Notably, LoRA offers higher training throughput and does not introduce additional inference latency like adapters do. We also conducted empirical investigations into the rank-deficiency issue in language model adaptation, providing insights into the effectiveness of LoRA.</p><blockquote><strong><em>Aren’t Existing Solutions Good Enough?</em></strong></blockquote><h4>Adapter Layers Introduce Inference Latency</h4><p>There are various types of adapters that have been developed. We will focus on two specific designs: one created by Houlsby et al. in 2019, which incorporates two adapter layers per Transformer block, and a more recent design by Lin et al. in 2020, which features one adapter layer per block along with an additional LayerNorm. While it is possible to reduce latency by pruning layers or utilizing multi-task settings, there are no direct methods to bypass the additional computational overhead incurred by adapter layers. Although adapter layers are designed to have a small number of parameters (sometimes less than 1% of the original model) by using a narrow bottleneck dimension, this limitation affects the FLOPs they can contribute. However, in the case of large neural networks that rely on hardware parallelism to maintain low latency, adapter layers must be processed sequentially. This discrepancy becomes noticeable in online inference scenarios where the batch size is typically small, even with a very small bottleneck dimension (refer to following table). The issue worsens when the model is sharded, as in the works of Shoeybi et al. in 2020 and Lepikhin et al. in 2020, due to the increased depth requiring more synchronous GPU operations like AllReduce and Broadcast, unless the adapter parameters are redundantly stored multiple times.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ydylr21EVgL69eNUqnpoww.png" /></figure><h4>Directly Optimizing the Prompt is Hard</h4><p>There is another approach, known as prefix tuning, which presents a distinct challenge. It has been observed that optimizing prefix tuning is challenging, and its performance varies non-monotonically with trainable parameters, aligning with findings from the original paper. Moreover, allocating a portion of the sequence length for adaptation inherently reduces the sequence length available for processing the downstream task. This limitation leads us to believe that tuning the prompt may be less effective compared to alternative methods.</p><blockquote><strong><em>Methodology</em></strong></blockquote><h4>Low-Rank-Parameterized Update Matrices</h4><p>Neural networks consist of multiple dense layers that carry out matrix multiplication. The weight matrices within these layers usually possess full rank. A study by Aghajanyan et al. in 2020 demonstrates that pre-trained language models have a low “intrinsic dimension” and can effectively learn even when projected onto a smaller subspace. Drawing inspiration from this, we hypothesize that weight updates during adaptation also exhibit a low “intrinsic rank.&quot;here are various types of adapters that have been developed. We will focus on two specific designs: one created by Houlsby et al. in 2019, which incorporates</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B254pxKd-wYezozjxSBrnQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/363/1*GUY5vMJKoUenkBMa9dpEGg.png" /></figure><h4>A Generalization of Gull Fine-tuning</h4><p>A more generalized approach to fine-tuning allows for training only a subset of the pre-trained parameters. LoRA goes a step further by not requiring the accumulated gradient updates to weight matrices to possess full rank during adaptation. By applying LoRA to all weight matrices and training all biases, we can achieve a similar level of expressiveness as full fine-tuning by setting the LoRA rank equal to the rank of the pre-trained weight matrices. In simpler terms, as we increase the number of trainable parameters, training with LoRA approaches the training process of the original model, while adapter-based methods converge towards an MLP (Multi-Layer Perceptron) and prefix-based methods converge towards a model that is unable to handle long input sequences.</p><h4>No Additional Inference Latency</h4><p>During production deployment, it is possible to calculate and retain the value of W as W0 + BA, where W0 and BA belong to the space R^(d×k). Inference can then be conducted as usual. When there is a need to switch to a different downstream task, W0 can be obtained by subtracting BA and subsequently adding a distinct B′A′. This operation is rapid and incurs minimal memory overhead. Importantly, this ensures that no extra latency is introduced during inference compared to a fine-tuned model, as it is inherently designed that way.</p><p>In this article, I briefly share my viewpoints on the paper. I hope you can learn more about it after reading it. I also offer the video link about the paper, hope you guys like it!!!!</p><p>If you like the article, please give me some 👏 , share the article, and follow me to learn more about the world of multi-agent reinforcement learning. You can also contact me on <a href="https://www.linkedin.com/in/bor-jiun-lin-b99b80191/"><strong>LinkedIn</strong></a>, <a href="https://www.instagram.com/imchristianbutnotchristian/?hl=zh-tw"><strong>Instagram</strong></a>, <a href="https://www.facebook.com/profile.php?id=100010297082645"><strong>Facebook</strong></a><strong> </strong>and<strong> </strong><a href="https://github.com/ChristianLin0420"><strong>Github</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=df343b0ce96" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reinforcement Learning Paper Reading~Is Conditional Generative Modeling All You Need for…]]></title>
            <link>https://medium.com/@crlc112358/reinforcement-learning-paper-reading-is-conditional-generative-modeling-all-you-need-for-9a35959505ef?source=rss-f60365b66f2e------2</link>
            <guid isPermaLink="false">https://medium.com/p/9a35959505ef</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[diffusion-models]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Christian Lin]]></dc:creator>
            <pubDate>Sat, 20 May 2023 15:08:19 GMT</pubDate>
            <atom:updated>2023-05-20T15:08:19.626Z</atom:updated>
            <content:encoded><![CDATA[<h3>Reinforcement Learning Paper Reading~Is Conditional Generative Modeling All You Need for Decision-Making?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*V1C9-XVPWT19G3up7poH2A.png" /></figure><p><a href="https://openreview.net/forum?id=sP1fo2K9DFG">Is Conditional Generative Modeling all you need for Decision Making?</a></p><p>Recent advancements in conditional generative modeling have opened up possibilities for generating high-quality images solely based on language descriptions. In this study, we explore the potential of these methods in directly addressing sequential decision-making challenges. Rather than approaching decision-making from the perspective of reinforcement learning (RL), we adopt a lens of conditional generative modeling. Surprisingly, we discover that our approach yields policies that surpass existing offline RL methods on established benchmarks. By formulating a policy as a return-conditional diffusion model, we showcase how we can bypass the requirement for dynamic programming and alleviate many complexities associated with traditional offline RL. Moreover, we showcase the benefits of modeling policies as conditional diffusion models by considering additional conditioning variables, such as constraints and skills. Training our models with a single constraint or skill enables test-time behaviors that satisfy multiple constraints simultaneously or demonstrate a composition of skills. Our findings demonstrate the remarkable potential of conditional generative modeling as a powerful tool for decision-making.</p><blockquote><strong><em>Introduction</em></strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tgAE8cHuN03dQ7flaMvuAg.png" /></figure><p>Adopting the framework of conditional generative modeling expands the scope of offline decision-making beyond the sole conditioning on returns, as depicted in the aforementioned figure. To illustrate this concept, we present an illustrative example (outlined in Appendix A) involving a robot exhibiting linear dynamics within an environment comprising two concentric circles, as depicted in the accompanying figure. Our dataset consists of state-action trajectories, each adhering to one of two constraints: (i) the robot’s final position falls within the larger circle, and (ii) the robot’s final position lies outside the smaller circle. Leveraging conditional diffusion modeling, we employ the dataset to acquire a constraint-conditioned model capable of generating trajectories that fulfill any combination of constraints. During inference, the learned trajectory model adeptly combines constraints from the dataset to generate trajectories satisfying the merged constraint. As illustrated in the subsequent figure, the constraint-conditioned model successfully generates trajectories, with the robot’s final position residing between the concentric circles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7rhc00j2Kni5WhRIYfbguA.png" /></figure><blockquote><strong><em>Methodology</em></strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IMz9QfHq2BKbtyHacJKTaQ.png" /></figure><p>Offline RL offers a valuable approach to solving sequential decision-making tasks without depending on TD-learning or encountering distribution-shift concerns. In light of this, we frame the problem of sequential decision-making as a conventional conditional generative modeling problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/957/1*BZfxczV7fRlJZUKV7Emahw.png" /></figure><p>The objective is to estimate the conditional data distribution using pθ in order to generate specific segments of a trajectory x0(τ) based on corresponding information y(τ). The information y(τ) can encompass various factors such as the trajectory’s return, the constraints it satisfies, or the skill it demonstrates. To accomplish this, we develop our generative model based on the conditional diffusion process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/957/1*Sh5XUhqiJhbv2WGxtDVX-A.png" /></figure><h4>Diffusing over States</h4><p>Applying the diffusion process to all pixel values in an image is a straightforward approach. Consequently, it may appear intuitive to extend this process to model the state and actions of a trajectory. However, when considering the reinforcement learning context, directly employing a diffusion process to model actions introduces several practical challenges. Firstly, while states in RL are typically continuous, actions exhibit greater variability and often possess a discrete nature. Moreover, action sequences, often represented as joint torques, tend to be more high-frequency and less smooth, rendering them considerably more challenging to predict and model accurately. Considering these practical considerations, we make the deliberate decision to limit the diffusion process to states exclusively, as precisely defined below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/957/1*Iup3b6IYWHHZANR43srMfw.png" /></figure><p>Here, k denotes the timestep in the forward process and t denotes the time at which a state was visited in trajectory τ. Moving forward, we will view x_k(τ) as a noisy sequence of states from a trajectory of length H. We represent x_k(τ) as a two-dimensional array with one column for each timestep of the sequence.</p><h4>Planning with Classifier-free Guidance</h4><p>To utilize a diffusion model for planning purposes, conditioning the diffusion process on characteristics y(τ) becomes essential. One possible approach involves training a classifier p_φ(y(τ)|xk(τ)) to predict y(τ) based on noisy trajectories x_k(τ). However, when y(τ) represents the trajectory’s return, estimating a Q-function is required, which entails a separate and intricate dynamic programming procedure.</p><p>To avoid the complexities of dynamic programming, an alternative approach is to directly train a conditional diffusion model conditioned on the returns y(τ) found in the offline dataset. Nonetheless, as our dataset comprises sub-optimal trajectories, the conditional diffusion model might be influenced by such sub-optimal behaviors. To mitigate this issue, we employ classifier-free guidance with low-temperature sampling, enabling us to extract high-likelihood trajectories from the dataset. These trajectories correspond to the most desirable set of behaviors within the dataset. For a comprehensive discussion comparing Q-function guidance and classifier-free guidance, please refer to Appendix K. Formally, the implementation of classifier-free guidance involves sampling x_0(τ) by initializing it with Gaussian noise x_K(τ) and iteratively refining x_k(τ) into x_{k-1}(τ) at each intermediate timestep using perturbed noise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/957/1*NCqXUbhF4oaxQsFHeLS4DQ.png" /></figure><p>where the scalar ω applied to (ε_θ (x_k (τ ), y(τ ), k) − ε_θ (x_k (τ ), Ø, k)) seeks to augment and extract the best portions of trajectories in the dataset that exhibit y(τ).</p><p>Incorporating these components, the process of sampling from the Decision Diffuser parallels RL planning. Initially, we observe a state within the environment. Subsequently, we generate samples of states further along the horizon using the diffusion process, conditioning it on y and the history of the last C observed states. Finally, we determine the appropriate action to reach the nearest predicted state based on our inverse dynamics model.</p><h4>Conditioning beyond Returns</h4><p>So far we have not explicitly defined the conditioning variable y(τ). Though we have mentioned that it can be the return under a trajectory, we may also consider guiding our diffusion process towards sequences of states that satisfy relevant constraints or demonstrate specific behavior.</p><p><strong>Maximizing Returns</strong> To generate trajectories that maximize return, we condition the noise model on the return of a trajectory so ε_θ(x_k(τ), y(τ), k) := ε_θ(x_k(τ), R(τ), k). These returns are normalized to keep R(τ ) ∈ [0, 1]. Sampling a high return trajectory amounts to conditioning on R(τ) = 1. Note that we do not make use of any Q-values, which would then require dynamic programming.</p><p><strong>Satisfying Constraints</strong> Trajectories may satisfy a variety of constraints, each represented by the set C_i, such as reaching a specific goal, visiting states in a particular order, or avoiding parts of the state space. To generate trajectories satisfying a given constraint C_i, we condition the noise model on a one-hot encoding so that ε_θ(x_k(τ), y(τ), k) := ε_θ(x_k(τ), 1 (τ ∈ C_i), k). Although we train with an offline dataset in which trajectories satisfy only one of the available constraints, at inference we can satisfy several constraints together.</p><p><strong>Composing Skills</strong> A skill i can be specified from a set of demonstrations B_i. To generate trajectories that demonstrate a given skill, we condition the noise model on a one-hot encoding so that ε_θ(x_k(τ), y(τ), k) := ε_θ(x_k(τ), 1 (τ ∈ B_i), k). Although we train with individual skills, we may further compose these skills together during inference.</p><h4>Training the Decision Diffuser</h4><p>To train the Decision Diffuser, our conditional generative model for decision-making, we adopt a supervised learning approach. Using a dataset D comprising labeled trajectories, where each trajectory is associated with its achieved return, satisfied constraint, or demonstrated skill, we simultaneously train the reverse diffusion process p_θ parameterized by the noise model ε_θ, along with the inverse dynamics model f_φ. This training is accomplished by optimizing the following loss function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nt3gyT38_JupWS8U85JDNg.png" /></figure><p>The corresponding algorithm is shown as follow:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eWtKp_aFcTtpAbUo4OxUwQ.png" /></figure><p>The experimental results and details can be referred from the original paper. If you really interest in the topic, please take time reading it!!!!</p><p>In this article, I briefly share my viewpoints on the paper. I hope you can learn more about it after reading it. I also offer the video link about the paper, hope you guys like it!!!!</p><p>If you like the article, please give me some 👏 , share the article, and follow me to learn more about the world of multi-agent reinforcement learning. You can also contact me on <a href="https://www.linkedin.com/in/bor-jiun-lin-b99b80191/"><strong>LinkedIn</strong></a>, <a href="https://www.instagram.com/imchristianbutnotchristian/?hl=zh-tw"><strong>Instagram</strong></a>, <a href="https://www.facebook.com/profile.php?id=100010297082645"><strong>Facebook</strong></a><strong> </strong>and<strong> </strong><a href="https://github.com/ChristianLin0420"><strong>Github</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9a35959505ef" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[一個『幸運』的旅途～Microsoft Research Development Intern]]></title>
            <link>https://medium.com/@crlc112358/%E4%B8%80%E5%80%8B-%E5%B9%B8%E9%81%8B-%E7%9A%84%E6%97%85%E9%80%94-microsoft-research-development-intern-775a129a74a9?source=rss-f60365b66f2e------2</link>
            <guid isPermaLink="false">https://medium.com/p/775a129a74a9</guid>
            <category><![CDATA[microsoft]]></category>
            <category><![CDATA[intern]]></category>
            <category><![CDATA[internships]]></category>
            <category><![CDATA[research-and-development]]></category>
            <category><![CDATA[internship-experience]]></category>
            <dc:creator><![CDATA[Christian Lin]]></dc:creator>
            <pubDate>Sat, 29 Apr 2023 16:42:17 GMT</pubDate>
            <atom:updated>2023-04-29T16:42:17.045Z</atom:updated>
            <content:encoded><![CDATA[<p>2022–2023 微軟RDI實習生分享</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ds3TasXQYpBOVEuU9nlO-A.jpeg" /><figcaption>沒錯我就是那個太高需要彎腳的左一😎</figcaption></figure><h3><strong><em>Who AM I</em></strong></h3><p>大家好，我是來自清華大學資訊工程所碩士一年級的林柏均，我的英文名字是Christian。在公司裡大家都習慣叫我Chris，因為Christian真的很難唸，而且有人甚至會叫我Christi『<strong><em>NA』</em></strong>，我的天～～～</p><p>這篇Medium最主要是希望可以跟大家分享我身為Microsoft Research Development Intern的一些心得、經歷及建議！希望大家可以從這篇文章或多或少更多的了解微軟究竟是怎麼樣的公司，在這個如日中天的公司實習又是個怎麼樣的經驗🤩！</p><h3>Why Do I Want to Apply the Program</h3><p><a href="https://www.microsoft.com/taiwan/campus/">台灣微軟-未來生涯體驗計劃</a></p><p>簡單來說，『未來生涯體驗計劃』的目的跟宗旨大致如下：</p><blockquote>來自各種背景的同學們可以在步入社會前，藉由此計畫去了解及體驗在企業實習的氛圍與工作內容。與此同時，藉由一系列的活動企劃、職能訓練、提攜制度等，來充實及培養工作態度及能力</blockquote><p>當然實習計畫當然不是單單幾句話就可以涵蓋的，在這裡的每位實習生都在這將近結束的實習計畫中體驗到的屬於自己的旅途，很多經歷是需要大家親身去感受才可以認知到其價值的😉！</p><p><strong><em>那回到題目本身，為什麼我會想要申請台灣微軟的實習計畫呢？</em></strong></p><p>其實這跟本身的經歷有關！五年前指考結束後飛到加州去玩，當時跟許多矽谷的工程師甚至是名校的學生們聊天，言語間感受到身體力行去完成一個專案是一件很需要勇氣跟耐心的挑戰，但卻可以得到意想不到的收穫。在傳統資工系中，很大部分時間我們會去學習許多面向的基礎到進階『理論』，當然我們也很常coding，但是許多東西一旦到了業界就會發現無可用之處，白白浪費了許多時光，更甚的是在四年大學生涯裡沒有人會告訴你，甚至教你去設計一個專案，並完成它的能力！因此大一的時候我毅然決然以一個幾乎沒有任何程式基礎的菜鳥進入了新創公司擔任iOS Fullend Developer！當然這又是另一個故事惹～～～🥲</p><p>在那段期間，我的工作內容從UI/UX design, Frontend, Backend甚至到Deployment都是由我來操刀😎，其中跌了不少許多學生可能在職場需要花上數年才可以體悟到的痛處，也因此讓我學習到許多很寶貴的經驗！『<strong>但是</strong>』！！！！這段期間我其實沒有太多與他人一起完成一個專案的經驗，對於與其他工程師合作方面的經驗少之又少。</p><p>因此，在去年春天我在網站上耳聞有關微軟的生涯體驗計劃，對此感到興趣，遂而決定申請，期盼自己在學生生涯可以圓滿以上提到的缺失！最後非常有幸地參與到這個大家庭中！</p><h3>How to Apply the Program</h3><p>那究竟台灣微軟的申請過程如何呢🤔？其實大致上面試要準備的素材根絕大部分的公司實習一模一樣，就是你的resume！內容大概如下：</p><ol><li>自我介紹 (Introduction)</li><li>經歷 (Experiences)</li><li>作品集 (Your Works)</li></ol><p>看下來真的跟99%的公司很像吧！所以其實大家不用刻意擔心微軟實習要準備的資料是不是很不一樣的問題，但是一定要切記！！！提供的內容必須為了所申請職位的『<strong>特色</strong>』去設計，那這邊剛好順帶一提微軟實習究竟有哪些職位摟！</p><blockquote><strong><em>面試進程</em></strong></blockquote><p>以我自己申請RDI的角度來分享的話可以大致列為以下幾個步驟：</p><ol><li>電話面試：英文自我介紹</li><li>第一次白板題</li><li>第二次白板題</li><li>第三次白板題(optional)：但我遇到惹～～～</li><li>接到Offer</li></ol><p>這邊我沒有想要太多詳細分享面試的過程，必經每個人的面試竟過可能都有些不同，所以我簡單分享一下大致的過程而已！至於時間的部分每個人都不太一樣，所以我這邊就簡單帶過啦！</p><p>個人給想要申請技術職(TAI &amp; RDI)的各位小小的一些建議，就是在面試時面對白板題又或著是情境題，請多多跟面試官們闡述你的想法，即是想法可能是錯的也要提出自己的想法，而不是靜靜的不發表心中的想法，這樣其實面試官根本無從得知你是如何解決一個問題，以及如何思考問題的方式ㄛ！</p><h3>What is My Job</h3><p>大家都知道我的職稱是Research Development Intern，簡稱RDI。但微軟一年收的RDI名額其實並不少，每個人所分配到的部門都不同，甚至就算在同部門的實習生們也可能因為不同的主管而接觸不同的工作內容，因此接下來我謹代表我個人來分享一下身為RDI所會接觸的各項事務！！！</p><p>我所隸屬的部門稱為Customer Success Unit，簡稱CSU組。在CSU裡面正如剛剛提到的每個主管所接觸的範疇都大不相同，但是我們有個共通點，那就是將微軟既有的服務(i.e Azure, Microsoft365, OpenAI等等)與客戶所需結合，與此同時要兼顧安全性、低成本等等考量。具體來說在CSU內部的許多工程師的頭銜會是 — — — — — Architect，也就是架構師！現在是不是姆薩薩ㄚ！！！</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FJrKPtW6yzrE%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DJrKPtW6yzrE&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FJrKPtW6yzrE%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/371468b92cdfe13489c9830da22240e4/href">https://medium.com/media/371468b92cdfe13489c9830da22240e4/href</a></iframe><p>其實所謂的架構師有點類似設計房屋大樓的設計師，在搭建建築前，其實建築師們都會先畫出所謂的架構圖，房屋的樑架要怎麼搭建，材料要用什麼，逃生路線再怎麼規劃等等都會在搭建前多次討論，最終定案後就開工！！！那其實不論系統架構師、雲端架構師等等就跟這些建築師很像！！！設計一套系統前，都會與客戶多次討論，商量要用到哪些服務(i.e Database, WebApp)，這就好比樑柱；相同的服務也可能因為不同的容量、功能等(對比建築材料)而有價位上的不同；最後，就是安全的設計，諸如防火牆(Firewall)又或著是認證系統(Authentication Design)啊都好似建築中的逃生系統或防火系統！希望這樣的對比大家可以了解到到底什麼事架構師，我也在☝️提供了個短影片去比較說架構師(Architect) &amp; 工程師(Engineer)之間的一些差別唷～～～</p><p>那身為實習生的我究竟都在做什麼呢？簡單來說我就是把房間蓋起來的人啦XDDD 每個大樓都有無數的小房間，身為菜鳥的我就是要來做這個，把每一個用到的服務用Coding的方式呈現出來，很多時候可能需要搭建網站將所用到的服務視覺化呈現給客戶看，讓他們了解到微軟的服務究竟是怎麼運作，有哪些功能！哪些限制！讓客戶在開發他們內部的系統時可以更快上手避免掉太多無謂的嘗試！這些被省下的時間，以客戶的角度來看就是省錢的象徵，那對公司而言省錢省力開發意味著什麼呢？那就是『成功』(Success)啦！！！ 回到剛剛的開頭我是什麼組的？！</p><blockquote>客戶成功事業群(Customer Success Unit)</blockquote><blockquote>That’s How We Make Our Clients Success！</blockquote><p>這樣大家是不是更了解我們這個組到底在幹嘛了吧！！！嘿嘿嘿😎 <br>接著我就列一下我覺得大家可能問的問題，然後盡可能地回答啦！！！</p><blockquote>RDI 究竟要用到哪些程式語言呢？</blockquote><p>以我自己在這將近一年的實習期間，有用過Python, Java, C#, 和Javascript來開發相關的專案，但如果說是不是要再進入實習前就掌握很多程式語言呢？我會說有很棒，『<strong>但是但是但是</strong>』沒有必要！！！會這麼說的原因其實是因為不同的程式語言只是在用的語法上有些異同，重點要掌握的是程式語言背後的邏輯，像我在實習前幾乎沒碰過多少次Java(邪教語言)跟Javascript，更別說C#碰都沒碰過！這些幾乎都是我進入後才開始學的，從這裡大家也可以看出微軟是一個很願意接收學習的地方，當然還是要學得快ㄚ！別想著給我兩個月的時間學Java怎麼print，那真的就……很優秀了喔😆</p><blockquote>RDI 會不會寫的都是雜Code</blockquote><p><strong>Nope!!! Nope!!! Nope!!! </strong>絕對不會有雜Code，當然如果你跟我說寫Log是雜扣的話，那我建議按完小鈴鐺跟追蹤後就可以換個分頁摟！哈哈哈！當然有時候會面對一些客戶本來想要但是後來不想的情境，但對我而言這不是什麼浪費時間的行為，很多時候你反而可以從中學到很多小東西，包括客戶注重的點是什麼，那下次有類似的情況，妳就可以不經意間就把這些小東西『<strong>主動</strong>』先加上去，這就可以讓客戶覺得說你是個非常專業且細心的工程師，不是來打雜的『碼農』摟～～～</p><blockquote>RDI 會不會需要加班</blockquote><p>啊哈哈哈～～～ 我可以很有自信的說『沒有』，甚至我的主管在我第一天onboard就跟我說你不準下班後還給我工作喔，我整個心想『我也不要ㄚ』(喂～～ 開玩笑的啦XDD)！這邊我想提個大家可以嘗試的東西，那就是跟主管多多溝通，去告訴他你的狀況，以我而言有時候要趕paper就會比較沒時間去弄工作，那主管就會去調整我的工作內容，讓我處理一些相較不棘手的tasks，我覺得這就是微軟很特殊且讓我喜歡的點，哪像某些T商啊！哼哼哼！整個母湯～～～</p><p>以上就是我自己身為RDI的一些工作內容描述跟一些大家常見的問題，希望有回答到頭腦滿是問號的妳跟你跟祢～～～</p><h3>What About Activities?</h3><p>相信有追蹤微軟IG粉專的各位，應該有看到許多微軟實習生的活動對吧！！！對吧？😅 為了防止各位沒看到令人心動的活動，這邊直接附上微軟IG的連結啦，還不趕緊追蹤按讚；誒～～～ 還有我的Medium也追蹤分享一下啦！拜託啦🥹</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KksWz3VU9MEyFs9S4Urr8Q.png" /></figure><p><a href="https://www.instagram.com/microsoft_intern/">https://www.instagram.com/microsoft_intern/</a></p><p>結果只有Link好難過QQ，但是還是要記得追蹤按讚ㄛ！！！<br>接下來我會分享一些印象深刻的活動！！！</p><blockquote>校園分享會</blockquote><p>實習階段我第一個參加的活動就是校園分享會啦！！！其實分享會的目的就是讓全台更多的大學碩士生們更加了解『<strong>微軟未來生涯體驗計畫</strong>』，藉由實習生們親自分享自己的工作內容以及各式互動環節讓對計畫感到興趣的同學們了解更多有關微軟實習生的一切～～～</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QZopETdiEKXyKHWifrarAQ.jpeg" /><figcaption>中央場之突然不知道被誰偷拍？！</figcaption></figure><p>我自己本身有在清華大學、中央大學跟陽明交通大學分享，別問我為什麼去了那麼多場，因為我很閒啊(誒誒誒ㄟ 沒有請不要當真～～～)！但純是我自己真的對計畫內容跟工作內容感到非常喜歡，也希望大家可以藉由這些場合對微軟有更多認識，更加喜歡微軟！！！</p><blockquote>尾牙(這算活動嗎🤔 你管我？我就是想講啦！)</blockquote><p>再來就是年初的尾牙！！！這是個難得可以看到來自各個辦公室的微軟員工們齊聚一堂談笑風生的場合，這邊簡單跟大家科普一下，其實微軟在台灣的辦公室不僅僅止於微風信義，其實在板橋、高雄等地方都設有辦公室喔！我自己其實對尾牙感到印象深刻的有兩個點！</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JaVAtWzdNQ3kPLWMTPWLEg.jpeg" /><figcaption>實習生們的全家福👨‍👩‍👧</figcaption></figure><p>第一就是你可以用一個更加方松的心態去跟不同層級的主管去聊天，甚至上到總經理！每個員工都很願意跟實習生們聊天分享各式各的事物！都說到這難道不用上照片嗎哈哈哈😹</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nhrqdzIDSTgq1yW-qvA44g.jpeg" /><figcaption>Top Gun版總經理😎</figcaption></figure><p>而且藉由這個場合真的可以認識很多平常見不到的實習生們！說到這份上，其實很多時候大家的班不一定有overlap，所以會發現就算來微軟半年了，還是有不少不認識的實習生，『<strong>難道</strong>』這就是RDI孤僻的徵兆(好啦 我應該沒有這個問題啦XDDDD)</p><blockquote>領袖營</blockquote><p>最後一個分享的就是個人最印象深刻的『領袖營』，這其實是一個很有趣的活動，藉由兩天一夜的行程，以及一連串活動來增進實習生們間的認識及合作能力！其中我自己超級喜歡闖關階段的各式小關卡設計，根本不是大學營隊可以比的ㄚ！！！整個愛死🤩</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7uQYsZWjYMhcXv8EMKf1uw.jpeg" /><figcaption>四個Williamの迷因🕷️</figcaption></figure><p>在領袖營的期間，其實也接觸到其他部門的RDI，開始可以更加瞭解一些其他部門實習生的工作內容，讓我對微軟的整個開發面有了更深層的了解！</p><h3>How Do I Feel About The Program</h3><p>這邊先跟大家小小分享一些我自己的獨白！！！</p><blockquote>我們被社會的枷鎖箝制了10多個年頭，學習別人餵給我們的一切，用盡一切去追求很多本不是我們所想要的東西，到頭來卻發現自己根本不了解自己真正所想要的東西！！！</blockquote><blockquote>希望閱讀到此的各位花點時間去多多了解你究竟想要在未來1年、5年甚至是10年後過上怎麼的日子？做上什麼樣的工作？如果真的不知道要怎麼做到這些，請勇敢踏出你所熟悉的環境，去實習，去做專案，甚至參加比賽！！！</blockquote><p>其實實習的階段要進入尾聲了，離離開這間世界頂尖的科技公司只剩倒數兩個月，內心其實蠻多惆悵的！認識了很多來自不同背景的人，大家都有自己的特色跟值得學習的地方，尤其是CSU組的各位們，我是裡面晚一個月才onboard的，但是大家都很願意教我有關實習的一切，讓我更快進入軌道！</p><p>這邊要很感謝我的大主管John在當時選擇了我，一直到現在給我不少的建議跟有趣的專案，讓我在CSU學到了很多很多科技人一輩子都不見得可以碰到的體驗跟知識！然後更要感謝我的主管Sam，每週花上不少時間帶我去了解更多微軟雲端相關知識，甚至在OpenAI開火的階段讓我拿到很多人排對排不到的API KEY！！！讓我去碰很多前沿的服務！最後是教會我怎麼去跟客戶溝通，甚至是設計相關服務的一些小細節！而且不斷包容我犯錯的地方，從來都不讓我加班的好主管🥹</p><p>最後的最後，真的得說自己很幸運可以在CSU組，認識到一群很讚的實習生朋友們！一起吃飯，一起體驗許多新奇的東西，一起互相慶生，滿滿的回憶！希望計畫結束後大家都可以踏上屬於自己璀璨的旅途，把這段珍貴的回憶Ctrl+V到自己的人生！！！</p><p>If you like the article, please give me some 👏 , share the article, and follow me to learn more about the world of multi-agent reinforcement learning. You can also contact me on <a href="https://www.linkedin.com/in/bor-jiun-lin-b99b80191/"><strong>LinkedIn</strong></a>, <a href="https://www.instagram.com/imchristianbutnotchristian/?hl=zh-tw"><strong>Instagram</strong></a>, <a href="https://www.facebook.com/profile.php?id=100010297082645"><strong>Facebook</strong></a><strong> </strong>and<strong> </strong><a href="https://github.com/ChristianLin0420"><strong>Github</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=775a129a74a9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reinforcement Learning Paper Reading~Planning with Diffusion for Flexible Behavior Synthesis]]></title>
            <link>https://medium.com/@crlc112358/reinforcement-learning-paper-reading-planning-with-diffusion-for-flexible-behavior-synthesis-bacbfc3b45ee?source=rss-f60365b66f2e------2</link>
            <guid isPermaLink="false">https://medium.com/p/bacbfc3b45ee</guid>
            <category><![CDATA[diffusion-models]]></category>
            <category><![CDATA[denoising]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Christian Lin]]></dc:creator>
            <pubDate>Sun, 23 Apr 2023 23:19:06 GMT</pubDate>
            <atom:updated>2023-04-23T23:19:06.994Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/793/1*eeLvrZr8kqW7L9Aw3-3bjg.png" /></figure><p><a href="https://diffusion-planning.github.io/">Planning with Diffusion for Flexible Behavior Synthesis</a></p><p>The paper introduces a new approach to model-based reinforcement learning, in which as much of the trajectory optimization process as possible is incorporated into the sampling problem. This is in contrast to traditional methods, which rely on classical trajectory optimizers in conjunction with learned dynamics models. The proposed approach utilizes a diffusion probabilistic model that iteratively denoises trajectories, allowing for sampling from the model and planning with it to become nearly identical. The paper explores the properties of diffusion-based planning methods and demonstrates the effectiveness of this approach in control settings that emphasize long-horizon decision-making and test-time flexibility. The authors also show how classifier-guided sampling and image inpainting can be reinterpreted as planning strategies within this framework.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/751/1*73o039iLiZhoJn28weWvdw.png" /><figcaption>Diffuser is a diffusion probabilistic model that plans by iteratively refining trajectories.</figcaption></figure><p>If you want to know more about articles related with diffusion models, please feel free to read the following articles.</p><ul><li><a href="https://medium.com/@crlc112358/computer-vision-paper-denoising-diffusion-probabilistic-models-78bc38bc98b8">Computer Vision Paper ~ Denoising Diffusion Probabilistic Models</a></li><li><a href="https://medium.com/@crlc112358/computer-vision-paper-denoising-diffusion-implicit-models-e2b5807e83b8">Computer Vision Paper ~ Denoising Diffusion Implicit Models</a></li><li><a href="https://medium.com/@crlc112358/computer-vision-paper-high-resolution-image-synthesis-with-latent-diffusion-models-65fcf3191539">Computer Vision Paper ~High-Resolution Image Synthesis with Latent Diffusion Models</a></li></ul><blockquote><strong><em>Methodology</em></strong></blockquote><p>Although learned models can be used to overcome this challenge, they may not be suitable for planning algorithms designed for ground-truth models, leading to suboptimal planners. The authors propose a tighter coupling between modeling and planning by incorporating as much of the planning process as possible into the generative modeling framework using a diffusion model of trajectories. By separating the dynamics information from the perturbation distribution, a single diffusion model can be reused for multiple tasks in the same environment. This approach involves finding trajectories that are physically realistic and high-reward under the perturbed distribution.</p><h4>A Generative Model for Trajectory Planning</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/751/1*gB5H7ALfmzYbvQmLRaiTqg.png" /></figure><p><strong>Temporal ordering.<br></strong>Integrating sampling from a trajectory model with planning engenders a novel constraint whereby the prediction of states in temporal order becomes nonviable. To elaborate, consider the instance of goal-conditioned inference, p(s1 | s0, sT ), where the next state s1 relies on both a past and a future state. This exemplifies a broader principle: while dynamics prediction is causal and governed by past occurrences, decision-making and control can be anti-causal as they rely on the future. As a result, temporal autoregressive ordering is untenable, and Diffuser is formulated to predict all plan timesteps concurrently.</p><p><strong>Temporal locality.<br></strong>Notwithstanding its non-autoregressive or non-Markovian nature, Diffuser evinces a relaxed form of temporal locality. In the illustration above, we present a dependency graph for a diffusion model comprising a sole temporal convolution. The receptive field of a given prediction solely encompasses proximate timesteps from both past and future. Consequently, each denoising step can only make predictions predicated on the local consistency of the trajectory. Nevertheless, the aggregation of numerous denoising steps can prompt global coherence by harnessing local consistency.</p><p><strong>Architecture.<br></strong>This paper composes of recurrent temporal convolutional residual blocks. The model’s structure is akin to the successful U-Nets used in image-based diffusion models, albeit with one-dimensional temporal convolutions instead of two-dimensional spatial convolutions, as shown in Figure A1. Since the model is entirely convolutional, its prediction horizon is not contingent on the model architecture but instead depends on the input dimensionality, which can change dynamically during planning if needed. The overall architecture is shown as follow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/751/1*7zHk-gfjacHy9-N0nI06Cg.png" /></figure><h4>Reinforcement Learning as Guided Sampling</h4><p>To address reinforcement learning challenges through the employment of Diffuser, a reward system must be established. This necessitates resorting to the control-as-inference graphical model. To achieve this objective, let O_t denote a binary random variable that represents the optimality of timestep t within a given trajectory. It is noteworthy that p(O_t = 1) is set to exp(r(s_t, a_t)). Sampling from the collection of optimal trajectories can be performed by establishing h(τ) = p(O_{1:T} | τ) within the system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*aFnhgE_uiVzVHy1c1D7ZQw.png" /></figure><p>And now we have exchanged the reinforcement learning problem for one of <strong><em>conditional sampling</em></strong><em>. </em>The reverse diffusion process transitions can be approximated as Gaussian:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/629/1*n2vSzqHRJf6RWzSImEpveg.png" /></figure><p>where μ, Σ are the parameters of the original reverse process transition pθ(τ_{i−1} | τ_i) and</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/629/1*bJ5dmE9En3puxkx1ktJZIw.png" /></figure><p>The relationship outlined in this paragraph provides a direct mapping between classifier-guided sampling and the reinforcement learning problem context. Initially, we train a diffusion model, pθ(τ), on the states and actions present in all available trajectory data. Next, we train a separate model, J_φ, designed to predict the cumulative rewards of trajectory samples, τ_i. The gradients of J_φ are then utilized to guide the trajectory sampling process by altering the means μ of the reverse process. Following this, the first action of a sampled trajectory, τ ∼ p(τ | O_{1:T} = 1), can be executed within the environment, after which the planning process recommences through a typical receding-horizon control loop.</p><p>Following figure shows the pseudocode of the algorithm</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/730/1*fGcov7ew9WmRBAOQRZ5whg.png" /></figure><h4>Goal-Conditioned RL as Inpainting</h4><p>Certain planning problems can be more effectively formulated as constraint satisfaction exercises rather than the maximization of rewards. In such scenarios, the aim is to generate any possible trajectory that fulfills a given set of constraints, such as reaching a specified goal location. By leveraging the two-dimensional array representation of trajectories, this situation can be translated into an inpainting problem, where state and action constraints function in an analogous manner to observed pixels in an image. It is imperative that all unobserved positions in the array are populated by the diffusion model in a manner that aligns with the observed constraints. The perturbation function required for this undertaking takes on the form of a Dirac delta for observed values and remains constant otherwise. To elaborate, if ct is the state constraint at timestep t, then</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/730/1*zTmtBC_CTOqDouFHh36vIQ.png" /></figure><p>The definition of action constraints mirrors that of state constraints. In practical terms, this process can be realized by sampling from the unperturbed reverse process, τ_{i−1} ∼ pθ (τ_{i−1} | τi), and then substituting the values obtained with conditioning values, c_t, after all diffusion timesteps, i ∈ {0, 1, . . . , N}. It is noteworthy that even reward maximization problems necessitate conditioning-by-inpainting, as all sampled trajectories ought to commence from the present state.</p><blockquote><strong><em>Properties of Diffuser</em></strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z6Cf6zv2wVzyi0y8UcrVPQ.png" /></figure><p><strong>Learned long-horizon planning<br></strong>As the planning method is closely related to sampling and is only differentiated by the incorporation of a perturbation function, the success of Diffuser as a long-term predictor can be directly translated to successful long-term planning. This is exemplified in above figure(a), where we showcase the benefits of learned planning in a goal-reaching environment, demonstrating the ability of Diffuser to generate viable trajectories in sparse reward scenarios where conventional shooting-based approaches often falter.</p><p><strong>Temporal compositionality<br></strong>Single-step models rely on the Markov property creates transitions within a given distribution, enabling them to generalize beyond the distribution. However, Diffuser generates globally consistent trajectories by improving local consistency, allowing it to combine familiar sub-sequences in novel ways. Specifically, in above figure(b), they train Diffuser on straight-line trajectories and demonstrate its ability to generalize to V-shaped trajectories by integrating trajectories at their intersection point.</p><p><strong>Variable-length plans<br></strong>The planning horizon of the model is not determined by architectural choices but by the size of the input noise, as proposed model is fully convolutional in the horizon dimension of its prediction. This characteristic enables it to generate plans of variable length, where the denoising process is initialized by input noise τ N ∼ N (0, I), which is shown on above figure(c).</p><p><strong>Task compositionality<br></strong>Diffuser provides information on environment dynamics and behaviors but is not dependent on the reward function. As the model serves as a prior over possible futures, planning can be guided by lightweight perturbation functions h(τ) that correspond to various rewards, even in combination. We show this in our demonstration by planning for a new reward function that was not present during the training of the diffusion model, highlighting the flexibility of the perturbation function in guiding the planning process, which is shown on above figure(d).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/751/1*N5GEFjxaFvQyaKmRDkiOhQ.png" /></figure><p>In this article, I briefly share my viewpoints on the paper. I hope you can learn more about it after reading it. I also offer the video link about the paper, hope you guys like it!!!!</p><p>If you like the article, please give me some 👏 , share the article, and follow me to learn more about the world of multi-agent reinforcement learning. You can also contact me on <a href="https://www.linkedin.com/in/bor-jiun-lin-b99b80191/"><strong>LinkedIn</strong></a>, <a href="https://www.instagram.com/imchristianbutnotchristian/?hl=zh-tw"><strong>Instagram</strong></a>, <a href="https://www.facebook.com/profile.php?id=100010297082645"><strong>Facebook</strong></a><strong> </strong>and<strong> </strong><a href="https://github.com/ChristianLin0420"><strong>Github</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bacbfc3b45ee" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Computer Vision Paper ~High-Resolution Image Synthesis with Latent Diffusion Models]]></title>
            <link>https://medium.com/@crlc112358/computer-vision-paper-high-resolution-image-synthesis-with-latent-diffusion-models-65fcf3191539?source=rss-f60365b66f2e------2</link>
            <guid isPermaLink="false">https://medium.com/p/65fcf3191539</guid>
            <category><![CDATA[generative-ai]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[diffusion-models]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Christian Lin]]></dc:creator>
            <pubDate>Sun, 23 Apr 2023 06:11:59 GMT</pubDate>
            <atom:updated>2023-04-23T06:11:59.900Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/787/1*vt800aNQsMWBlJBbj9ZEyA.png" /></figure><p><a href="https://arxiv.org/abs/2112.10752">High-Resolution Image Synthesis with Latent Diffusion Models</a></p><p>The article discusses the limitations of diffusion models (DMs) for image generation due to their direct operation in pixel space, which makes optimization and inference expensive. To address these limitations, the authors propose the use of DMs in the latent space of pretrained autoencoders, which allows for a better balance between complexity reduction and detail preservation. This approach, known as <strong>latent diffusion models (LDMs)</strong>, also enables the use of cross-attention layers to generate high-resolution images from general conditioning inputs such as text or bounding boxes. The authors demonstrate that LDMs achieve state-of-the-art results in image inpainting and class-conditional image synthesis, and perform competitively in unconditional image generation, text-to-image synthesis, and super-resolution, while significantly reducing computational requirements compared to pixel-based DMs.</p><p>If you want to know more about the fundemenatal knowledge about diffusion models, please feel free to read the following articles.</p><ul><li><a href="https://medium.com/@crlc112358/computer-vision-paper-denoising-diffusion-probabilistic-models-78bc38bc98b8">Computer Vision Paper ~ Denoising Diffusion Probabilistic Models</a></li><li><a href="https://medium.com/@crlc112358/computer-vision-paper-denoising-diffusion-implicit-models-e2b5807e83b8">Computer Vision Paper ~ Denoising Diffusion Implicit Models</a></li></ul><blockquote><strong><em>Motivation</em></strong></blockquote><h4>Democratizing High-Resolution Image Synthesis</h4><p>Diffusion models (DMs) are likelihood-based models that can spend a lot of computational resources on modeling imperceptible details of the data. Training and evaluating such models on RGB images requires significant computational resources, and their mode-covering behavior often results in excessive capacity usage. As a result, DMs are not accessible to many researchers and have a large carbon footprint. To address this, a method is needed to reduce the computational complexity of DMs without impairing their performance, to make them more accessible to researchers and users.</p><h4>Departure to Latent Space</h4><p>The researchers’ approach involves analyzing already trained diffusion models in pixel space to find a more computationally efficient space for training diffusion models for high-resolution image synthesis. They divide learning into two stages: a <strong>perceptual compression stage</strong> and a <strong>semantic compression stage</strong>.(Following figure demonsrates the rate-distortion trade-off of a trained model) They aim to find a space that is perceptually equivalent to the data space but more computationally suitable for training diffusion models. To do this, they train an autoencoder to provide a lower-dimensional and efficient representational space. The researchers then train DMs in the learned latent space, which exhibits better scaling properties with respect to the spatial dimensionality, resulting in a model class called Latent Diffusion Models (LDMs).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*nGYGianauZyEGC6e80c12g.png" /></figure><blockquote><strong><em>Methodology</em></strong></blockquote><p>The authors propose a method to reduce the computational demands of training diffusion models for high-resolution image synthesis. They observe that although diffusion models can ignore perceptually irrelevant details, they still require costly function evaluations in pixel space. To address this, they introduce an autoencoding model that learns a low-dimensional space that is perceptually equivalent to the image space. This offers several advantages, such as making DMs computationally more efficient and enabling the training of multiple generative models. The resulting model class is called Latent Diffusion Models (LDMs). In the following subsections, I will go through all important approaches applied in LDM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/661/1*XWnMvuqUF_exHNWKcRYlgw.png" /></figure><h4>Perceptual Image Compression</h4><p>The authors use an encoder-decoder architecture to encode an RGB image x into a latent representation z, and then reconstruct the image x from z. The encoder downsamples the image by a factor f and they experiment with different downsampling factors. To avoid high-variance latent spaces, they use two different types of regularizations: <strong>KL-reg.</strong> and <strong>VQ-reg.</strong> The VQ-reg. uses a vector quantization layer within the decoder. By using mild compression rates, the authors achieve better reconstructions than previous works, which relied on an arbitrary 1D ordering of the latent space.</p><h4>Latent Diffusion Models</h4><p>Diffusion Models are probabilistic models that learn a data distribution p(x) by gradually denoising a normally distributed variable. They rely on a reweighted variant of the variational lower bound on p(x), which mirrors denoising score-matching. For image synthesis, successful models rely on an equally weighted sequence of denoising autoencoders, trained to predict a denoised variant of their input with following objective:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/593/1*EREZQk8WhZkpOwfu46GAgg.png" /></figure><p>The authors propose a new way of compressing images to a smaller and more efficient space, while retaining important details. This compressed space allows for faster and more effective generative models. Unlike previous methods that used complex models, this new approach takes advantage of the structure of images to better compress them. The model uses a UNet neural network, which is optimized for compressing images, and can efficiently generate new images with a single pass through the decoder network. Thus, new objective can be conducted as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/593/1*GCStf8GZsc_dTNYOR1xZzw.png" /></figure><h4>Conditioning Mechanisms</h4><p>Similar to other types of generative models, diffusion models usually poccess the potential ability to model conditional distributions and the possibility of controlling the synthesis process through different types of inputs, such as text or semantic maps. However, in the context of image synthesis, the use of other types of conditioning beyond class-labels or blurred input images is not well researched. To address this, the authors propose augmenting diffusion models with a cross-attention mechanism, which is effective in learning attention-based models of various input modalities. The authors also introduce a domain-specific encoder to preprocess input from various modalities and map it to the UNet via the cross-attention layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/669/1*K8yk6GE-DIaQ_vlJypebFw.png" /></figure><p>Here, φ_i denotes the a (flattened) intermediate representation. Based on image-conditioning pairs, we then learn the conditional LDM via</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/584/1*jLb46g1EN4ChkTEPm1sAnA.png" /></figure><p>The paper describes the success of the proposed latent diffusion models in improving the training and sampling efficiency of denoising diffusion models without sacrificing quality. In combination with the cross-attention conditioning mechanism, the experiments showed that the proposed models outperformed state-of-the-art methods in various conditional image synthesis tasks without requiring task-specific architectures.</p><p>In this article, I briefly share my viewpoints on the paper. I hope you can learn more about it after reading it. I also offer the video link about the paper, hope you guys like it!!!!</p><p>If you like the article, please give me some 👏 , share the article, and follow me to learn more about the world of multi-agent reinforcement learning. You can also contact me on <a href="https://www.linkedin.com/in/bor-jiun-lin-b99b80191/"><strong>LinkedIn</strong></a>, <a href="https://www.instagram.com/imchristianbutnotchristian/?hl=zh-tw"><strong>Instagram</strong></a>, <a href="https://www.facebook.com/profile.php?id=100010297082645"><strong>Facebook</strong></a><strong> </strong>and<strong> </strong><a href="https://github.com/ChristianLin0420"><strong>Github</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=65fcf3191539" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>