<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Richa Gupta on Medium]]></title>
        <description><![CDATA[Stories by Richa Gupta on Medium]]></description>
        <link>https://medium.com/@richa.lightg?source=rss-ca0581857059------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*4SfYXYoBJ9yriy00</url>
            <title>Stories by Richa Gupta on Medium</title>
            <link>https://medium.com/@richa.lightg?source=rss-ca0581857059------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 14:41:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@richa.lightg/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Comparing Loss Functions in GANs: LSGAN, WGAN and WGAN-GP]]></title>
            <link>https://medium.com/@richa.lightg/comparing-loss-functions-in-gans-lsgan-wgan-and-wgan-gp-c4a3a8452b5f?source=rss-ca0581857059------2</link>
            <guid isPermaLink="false">https://medium.com/p/c4a3a8452b5f</guid>
            <dc:creator><![CDATA[Richa Gupta]]></dc:creator>
            <pubDate>Fri, 04 Apr 2025 15:54:19 GMT</pubDate>
            <atom:updated>2025-04-04T16:00:13.115Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uuTACCvxD4rMM5vy0X7L7g.png" /></figure><p>Generative Adversarial Networks (GANs) have revolutionized the field of deep learning by enabling the generation of high-quality synthetic images. Their applications extend far beyond just images — they’re shaping everything from creating realistic human faces and enhancing medical imaging to generating music, writing poetry, optimizing network signal processing and even designing new molecules for drug discovery. From pixels to possibilities, GANs are pushing the boundaries of AI-driven creativity!</p><p>However, training GANs can be challenging due to instability and mode collapse. Different loss functions have been proposed to improve training stability and performance. This blog, aims to compare three popular GAN variants:</p><ul><li><strong>LSGAN (Least Squares GAN)</strong></li><li><strong>WGAN (Wasserstein GAN)</strong></li><li><strong>WGAN-GP (Wasserstein GAN with Gradient Penalty)</strong></li></ul><p>We shall analyze their performance based on FID (Fréchet Inception Distance) and Inception Score while leveraging TensorBoard for training visualization.</p><h3>Dataset and Preprocessing</h3><p>The models were trained on the <strong>MedMNIST</strong> dataset, specifically the <strong>PathMNIST</strong> subset. This dataset consists of colon pathology images categorized into 9 different classes. The dataset details are as follows:</p><ul><li><strong>Type:</strong> Multi-Class (9 classes)</li><li><strong>Total Images:</strong> 107,180</li><li><strong>Training Set:</strong> 89,996 images</li><li><strong>Validation Set:</strong> 10,004 images</li><li><strong>Test Set:</strong> 7,180 images</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rFHpaAZc2CS0AAzWtw5IFg.png" /><figcaption>Source: <a href="https://medmnist.com/">https://medmnist.com/</a></figcaption></figure><h3>Preprocessing Steps</h3><p>Before training the GANs, the dataset was preprocessed as follows:</p><ol><li><strong>Transforming to Tensor:</strong></li></ol><ul><li>The dataset was loaded using the MedMNIST PathMNIST class and converted into PyTorch tensors using:</li></ul><pre>dataset = PathMNIST(split=&quot;train&quot;, download=True, transform=transforms.ToTensor())</pre><p><strong>2. Batching and Shuffling:</strong></p><ul><li>The dataset was divided into mini-batches and shuffled to randomize the training order:</li></ul><pre>data_loader = DataLoader(dataset, batch_size=batch_size, shuffle=True)</pre><p><strong>3. Handling Inception Score Evaluation:</strong></p><ul><li>Since the dataset contains grayscale images (single channel), they were converted to RGB for compatibility with the Inception model:</li></ul><pre>if images.shape[1] == 1:     <br>  images = images.repeat(1, 3, 1, 1)</pre><ul><li>Images were resized to <strong>299 × 299</strong> pixels for evaluation.</li></ul><p>These preprocessing steps ensured compatibility with GAN models and evaluation metrics while maintaining the dataset’s original structure.</p><h3>Implementation Details</h3><h3>LSGAN</h3><p>LSGAN modifies the standard GAN loss function by using a least-squares loss instead of binary cross-entropy. This helps in stabilizing training and generating higher-quality images. The generator loss is given by:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/287/1*sAdsUO9_k7TqVa_7o3_w0g.png" /></figure><p>while the discriminator loss is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/449/1*LeHsOT3muw5IlTGfY62Jdw.png" /></figure><h3>WGAN</h3><p>WGAN replaces the standard discriminator with a critic that estimates the Wasserstein distance between real and generated distributions. The loss is formulated as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/315/1*C2R8cMHeOAV4TM29RNlhtw.png" /></figure><p>To enforce the Lipschitz constraint, weight clipping is applied to the critic’s parameters.</p><h3>WGAN-GP</h3><p>WGAN-GP improves upon WGAN by replacing weight clipping with a gradient penalty, ensuring that the critic satisfies the 1-Lipschitz constraint more effectively. The loss function includes an additional term:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*TprvRZpsT9WVvyKJHQMXKg.png" /></figure><p>where is a hyperparameter controlling the penalty strength.</p><h3>Training and TensorBoard Usage</h3><p>Each model was trained using TensorFlow/Keras, and TensorBoard was used for monitoring training dynamics. The key TensorBoard metrics tracked included:</p><ul><li><strong>Generator and Discriminator Losses</strong></li><li><strong>Sample Images at Different Epochs for Visual Inspection</strong></li></ul><p>To enable TensorBoard logging, the following setup was used:</p><pre>import tensorflow as tf<br>log_dir = &quot;logs/LSGAN&quot;</pre><p>Training was conducted for <strong>50 epochs (LSGAN)</strong> and <strong>20 epochs (WGAN &amp; WGAN-GP)</strong> on the PathMNIST dataset.</p><h3>Results and Performance Analysis</h3><p>The FID and Inception Score were calculated separately using the evaluation code in our notebooks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*J-onzGj2EsrV2zDie--nVQ.png" /><figcaption>Result Comparison Table</figcaption></figure><h3>Key Observations</h3><ul><li><strong>LSGAN</strong> outperforms WGAN in terms of FID, indicating better visual quality in generated images.</li><li><strong>WGAN-GP</strong> achieves the best Inception Score, suggesting improved diversity in generated samples.</li><li><strong>WGAN</strong> performs the worst in both metrics, likely due to instability from weight clipping.</li></ul><h3>Visual Comparisons</h3><p>Due to the limited number of training epochs (<strong>20 and 50 epochs</strong>), the generated images are not highly detailed or visually distinguishable. However, the loss graphs provide critical insights into how different loss functions influence GAN training dynamics. These graphs reveal how the models stabilize over time and indicate the effectiveness of each loss function in minimizing divergence between real and generated distributions.</p><p><strong>LSGAN:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rP8LiTqNsy8Z2weC5pxqDQ.png" /><figcaption>TensorBoard Interface for LSGAN</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/325/1*YOl1bjZtD88qPHf7lsVG_g.png" /><figcaption>LSGAN Generated Images</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/679/1*PanXSLxqHz4pYtSyv4phww.png" /><figcaption>LSGAN Discriminator and Generator Loss Graphs</figcaption></figure><p><strong>WGAN:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C85L4uDVRgLSg1aMcvkHKA.png" /><figcaption>TensorBoard Interface for WGAN</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/287/1*7EETU7qj4-bAM21diEaPJg.png" /><figcaption>WGAN Generated Images</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/605/1*RFybGZFllDnRzMAUf_yXAQ.png" /><figcaption>WGAN Discriminator and Generator Loss Graphs</figcaption></figure><p><strong>WGAN-GP:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VZxgbI6TFKh0FKPun81Wnw.png" /><figcaption>TensorBoard Interface for WGAN-GP</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/326/1*JlOzgf0JOtAvotc11UF6cQ.png" /><figcaption>WGAN-GP Generated Images</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/675/1*ypns8D6_eWqQ6xFRHPEDQg.png" /><figcaption>WGAN-GP Discriminator and Generator Loss Graphs</figcaption></figure><p><strong><em>Note:</em></strong></p><p><em>For both WGAN and WGAN-GP, the dataset was converted to grayscale before training. This decision was made because grayscale images reduce the complexity of the learning task, making it easier for the critic to focus on structural features rather than color variations. Unlike Inception Score evaluation, which requires RGB images for feature extraction, the critics in WGAN and WGAN-GP operate effectively on single-channel images. By converting the dataset to grayscale, we streamline training and prevent unnecessary computational overhead while preserving essential image features.</em></p><p><em>These grayscale images were then converted back to the original RGB for inception score calculation.</em></p><h3>Conclusion</h3><p>Each loss function brings distinct advantages and drawbacks:</p><ul><li><strong>LSGAN</strong> stabilizes training and produces high-quality images.</li><li><strong>WGAN</strong> improves convergence but struggles with weight clipping.</li><li><strong>WGAN-GP</strong> balances stability and quality by enforcing a better Lipschitz constraint.</li></ul><p>This comparison helps us understand which GAN loss function to use depending on the application.</p><ul><li><strong>LSGAN</strong> is suitable for generating high-quality medical images where stability is critical.</li><li><strong>WGAN-GP</strong> is ideal for generating diverse datasets with improved realism, such as synthetic faces or fashion images.</li><li><strong>WGAN</strong> can be useful in scenarios where enforcing a strict Lipschitz constraint is less important, such as basic image generation tasks.</li></ul><p>For future work, exploring advanced architectures like StyleGAN or applying adaptive learning techniques could further enhance GAN training efficiency and output quality.</p><p>You can find the complete implementation is available on <a href="https://github.com/RichaGupta1901/GAN-Loss-Comparison">GitHub</a> for reference and further exploration.</p><p>What are your thoughts on these GAN loss functions? Share your experiences and insights in the comments below!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c4a3a8452b5f" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>