<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Nazia Nafis on Medium]]></title>
        <description><![CDATA[Stories by Nazia Nafis on Medium]]></description>
        <link>https://medium.com/@nazianafis?source=rss-428a067ab315------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*D2QBYAkRvC5TmmPySUwPcw.png</url>
            <title>Stories by Nazia Nafis on Medium</title>
            <link>https://medium.com/@nazianafis?source=rss-428a067ab315------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 06:41:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@nazianafis/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[A Stable Diffusion  model fine-tuned on Trypophobia-triggering images]]></title>
            <link>https://medium.com/geekculture/stable-diffusion-model-fine-tuned-on-trypophobia-triggering-images-tryp22-pattern-ce1bb5259b37?source=rss-428a067ab315------2</link>
            <guid isPermaLink="false">https://medium.com/p/ce1bb5259b37</guid>
            <category><![CDATA[trypophobia]]></category>
            <category><![CDATA[stable-diffusion]]></category>
            <category><![CDATA[dreambooth]]></category>
            <dc:creator><![CDATA[Nazia Nafis]]></dc:creator>
            <pubDate>Wed, 18 Jan 2023 01:46:40 GMT</pubDate>
            <atom:updated>2025-08-07T22:23:05.453Z</atom:updated>
            <content:encoded><![CDATA[<h4>THE AI AISLE</h4><h3>A Stable Diffusion model fine-tuned on Trypophobia-Triggering images</h3><h4>My submission for the HuggingFace DreamBooth Hackathon ‘23.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xh4upEN-vZg3S9t-bRyNfA.png" /></figure><p>Studies suggest that as many as 17% of children and adults (about one in six people) experience some degree of trypophobia. My little brother has severe — he begins to gag and will smack the screen out of your hand if you try to show him patterns involving tightly-packed holes. I, on the other hand, am not affected at all. This is why I find the topic interesting and chose it for my DreamBooth Hackathon submission in the ‘science’ category.</p><blockquote>NOTE:</blockquote><blockquote>Images used throughout this article have been generated using the aforementioned model and may trigger Trypophobia.</blockquote><blockquote>VIEWER DISCRETION IS ADVISED.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PAkG8ohNZDXGXg68LSLjgw.png" /><figcaption><em>“a zoomed-out photo of [a single red rose, a blooming pink lotus, a daffodil], with tryp22 pattern in the background, HD” (Image by Author)</em></figcaption></figure><h3>What is Trypophobia?</h3><p>Trypophobia comes from the Greek words ‘<em>trypa’</em> (punching or drilling holes) and ‘<em>phobia’</em> (fear or aversion), and refers to a strong reaction — not limited to fear but also involving disgust, revulsion, or discomfort — to objects that have a particular pattern of closely packed holes or bumps on them.</p><p>Some common objects that trigger trypophobia include lotus seed pods, honeycombs, sponges, and even bread. However, research on trypophobia is limited, and the exact cause of it is unknown. Potential explanations include it being an evolutionary response to rotting, infectious items and an extension of a biological fear of venomous creatures such as frogs and snakes, which also often have such patterns on them.</p><p>The symptoms of someone who is having a strong reaction to trypophobia-triggering objects/images include goosebumps or a crawling sensation under the skin, gagging, rapid heartbeat, visual discomfort, distress, trembling, and panic attacks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iLJsS5aqXka1r8kIGys3JQ.png" /><figcaption><em>“a macro shot of a bug on a flower, tryp22 pattern background” (Image by Author)</em></figcaption></figure><h3>How did I Fine-tune a Stable Diffusion Model?</h3><p><a href="https://github.com/Stability-AI/stablediffusion">Stable Diffusion</a> is a popular text-to-image diffusion model, though it can also be used for other tasks such as inpainting, outpainting, and image-to-image translations guided by text prompts. It works by tokenizing English language words (using <a href="https://huggingface.co/docs/transformers/model_doc/clip">CLIP</a>) and iteratively applying input weights to latent space in a way that gradually brings the output closer to the input prompt.</p><p><a href="https://arxiv.org/abs/2208.12242">DreamBooth</a> is the technique used here to fine-tune a Stable Diffusion model. Using it, you can implant any topic of your choice (like I did here with ‘trypophobia-triggering’ patterns) into the output domain of the model.</p><p>For this task, first, I created a set of 22 high-quality images (more on that in another article) and defined a unique identifier ‘<strong><em>tryp22 pattern</em></strong><em>’</em> to refer to it. This identifier is usually a made-up word that can later be implanted in text prompts to get a seamlessly combined image as output. For ease of use, I made a private dataset on the HuggingFace Datasets hub to host my data.</p><p>Next, I fine-tuned the diffusion model on this dataset using the free version of Google Colab. The pipeline included:</p><ul><li>A text encoder (CLIP) to convert the prompts into text embeddings.</li><li>A variational autoencoder to convert images to their compressed representations (latent) and decompress them at the time of inference.</li><li>A UNet model to apply the denoising operation on the latent of the variational autoencoder.</li></ul><p>Then, I did some tweaking around hyperparameters (learning rate, maximum train steps, training batch size, and seed value) before beginning the actual training. On the 12GB Nvidia Tesla K80 GPU provided in Google Colab, it took about 45 minutes for it to complete.</p><p>Finally, I pushed the model to the HuggingFace Models hub, where it <a href="https://huggingface.co/nazianafis/tryp22-pattern?text=a+zoomed+out+photo+of+a+daffodil%2C+with+tryp22+pattern+in+the+background%2C+HD">currently resides</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PSi-EC2bApNtzPrtUpRwtQ.png" /><figcaption><em>“a high-quality [close-up of a cat, close-up of a dog, photo of a frog], tryp22 pattern” (Image by Author)</em></figcaption></figure><h3>How can you use the model to create images of your own?</h3><p>If you’ve come this far, chances are you are not having a strong adverse reaction to these images! So, here’s how you can use the model to create your own unique digital art pieces:</p><h4><strong><em>Step 1:</em></strong></h4><p>Follow the link: <a href="https://huggingface.co/nazianafis/tryp22-pattern?text=a+zoomed+out+photo+of+a+daffodil%2C+with+tryp22+pattern+in+the+background%2C+HD">https://huggingface.co/nazianafis/tryp22-pattern</a></p><h4><strong><em>Step 2:</em></strong></h4><p>Type in a text prompt of your choice, and <strong>use my unique identifier, <em>“tryp22 pattern”</em></strong>.</p><p><strong>Note:</strong> A good picture is a combination of a subject and a background. Since the model is trained on images containing trypophobia-triggering patterns, the background part will be taken care of when you use my identifier. Make sure you specify a well-known subject too, in your prompts so that the combination makes a unique artistic image.</p><h4><strong><em>Step 3:</em></strong></h4><p>Hit ‘Compute’.</p><p>You will see an artistic image on the topic of your choice in the style of trypophobia-triggering patterns. Be sure to download it if you like it since creating the exact image again would be virtually impossible for the model, even with the same prompt.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uem4JpS2XXmHpYXiG6hsXg.png" /><figcaption>HuggingFace link: <a href="https://huggingface.co/nazianafis/tryp22-pattern?text=a+zoomed+out+photo+of+a+daffodil%2C+with+tryp22+pattern+in+the+background%2C+HD">https://huggingface.co/nazianafis/tryp22-pattern</a></figcaption></figure><p>That’s it! The card for this model (nazianafis/tryp22-pattern) is located <a href="https://huggingface.co/nazianafis/tryp22-pattern?text=a+zoomed+out+photo+of+a+daffodil%2C+with+tryp22+pattern+in+the+background%2C+HD">here</a>. Do check it out, and if you have a HuggingFace account, consider giving it a ‘like’! ❤️</p><p><a href="https://huggingface.co/nazianafis/tryp22-pattern?text=a+zoomed+out+photo+of+a+daffodil%2C+with+tryp22+pattern+in+the+background%2C+HD">nazianafis/tryp22-pattern · Hugging Face</a></p><p><em>My objective behind this work is for it to act as a conversation starter for Trypophobia — and by extension — for so many other psychological quirks and nuances that we have, whose cause we yet do not know.</em></p><p><em>If you’re someone who is interested in human psychology and is looking to collaborate on research topics involving an understanding of human emotions in text or images with machine learning, let’s connect! You can connect with me </em><a href="http://linkedin.com/in/nazianafis"><em>on LinkedIn</em></a><em>, or comment </em><a href="https://nazianafis.medium.com/"><em>here</em></a><em> on Medium.</em></p><p><em>Until next time!</em> (∗ ･‿･)ﾉ゛</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ce1bb5259b37" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/stable-diffusion-model-fine-tuned-on-trypophobia-triggering-images-tryp22-pattern-ce1bb5259b37">A Stable Diffusion  model fine-tuned on Trypophobia-triggering images</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[WORDLE Assistant — Vocabulary no bar, anymore!]]></title>
            <link>https://nazianafis.medium.com/i-made-this-app-to-help-you-get-unstuck-at-wordle-7d8d9f145dd7?source=rss-428a067ab315------2</link>
            <guid isPermaLink="false">https://medium.com/p/7d8d9f145dd7</guid>
            <category><![CDATA[wordle]]></category>
            <dc:creator><![CDATA[Nazia Nafis]]></dc:creator>
            <pubDate>Thu, 14 Jul 2022 13:17:13 GMT</pubDate>
            <atom:updated>2023-01-24T10:50:32.180Z</atom:updated>
            <content:encoded><![CDATA[<h4>AI AISLE</h4><p>A limited vocabulary should be the last thing holding you back from maintaining a Wordle streak.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*kppG3vP4xfPAV_q_O_vSGQ.jpeg" /></figure><p>Wordle is a popular game that got viral in a short amount of time, and you’ve probably already tried it out (if not, check it <a href="https://www.nytimes.com/games/wordle/index.html">here</a>). It might however be disheartening sometimes to not be able to guess a word correctly just because you’re not aware of an English-language word that can be formed with the already-guessed letters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*CYvzlq0bT6joVk2fC3INaw.png" /><figcaption>📷 — A little humor from <a href="https://en.wikipedia.org/wiki/Bill_Watterson">Bill Watterson</a></figcaption></figure><p>But of course, a limited vocab shouldn’t be holding you back from hopping onto the Wordle train that has already taken the world by storm!</p><p><strong>So, I made 🔗 </strong><a href="https://huggingface.co/spaces/nn007/WORDLE-Assistant"><strong>this Streamlit app</strong></a><strong>.</strong></p><h3>How to use it?</h3><p>The big assumption here is that you’ve already unveiled at least a few letters, which you know would/would not be in the final five-letter word.</p><p>The app is up and running on HuggingFace Spaces. After you’ve made a couple of guesses on Wordle, and let’s say are stuck, you just need to enter the inclusion letters (the letters in green/yellow) and exclusion letters (the letters in grey) in their designated input points on the app, and hit Enter.</p><p>You’ll get a list of five letter words that do contain your inclusion letters and do not contain your exclusion letters.</p><p>Now, all you have to do is to match whichever word suits your situation on Wordle the best! There you have it!</p><h3>What goes on under the hood?</h3><p>We make use of <a href="https://www.nltk.org/">NLTK</a>’s English-language corpus.</p><p>NLTK is a leading platform for building Python programs to work with human language data. It provides easy-to-use interfaces to over 50 corpora and lexical resources, along with a suite of text-processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning.</p><p>We obtain all valid five-letter English words from the corpus using the words() method.</p><pre>from nltk.corpus import words<br>five_letters = [word for word in words.words() if len(word)==5]</pre><p>As long as we provide at least one inclusion/exclusion letter, the app will return an output.</p><pre>for word in five_letters:</pre><pre>        if all(c in word for c in <strong>inclusions</strong> and<br>           not any(c in word for c in <strong>exclusions</strong>):</pre><pre>            clue_result.append(word)</pre><pre>    st.write(clue_result)<br></pre><p>Yep, the logic really takes up only these many lines of code!</p><h3>How about a quick Demo?</h3><p>After three guesses, let’s say you’re at this stage:</p><p>You’re not sure what words begin with an ‘R’, end with an ‘S’, and contain an ‘E’ and a ‘U’ in between.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/1*kpe-K9ZLIU0hgEnoOsI6_Q.png" /></figure><p>Just enter this information in <a href="https://huggingface.co/spaces/nn007/WORDLE-Assistant">the assistant</a>.</p><p><strong>Note: </strong>The sequence in which you input the letters does not make a difference in the output. The case does not matter too.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/746/1*b2rH0BLFnjXCqw9zfkMMZA.png" /></figure><p>Finally, from the possible list of words, select the one that fulfills your requirements.</p><p>In this particular case, the word needs to begin with an ‘R’ and end with an ‘S’ (as noted previously). This directs us to exactly one match: ‘rebus’</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/619/1*_stK8KpexyODXru74nXwcQ.png" /></figure><p>That’s it!</p><p>That’s your Wordle word of the day solution!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/591/1*Ce2l-AvpfngoNmoiOBqTZA.png" /></figure><p><strong>PS. </strong>I acknowledge that finding the word that suits your requirements, out of the list of words this app generates, could be a bit of a hassle.</p><p>So why not add some more code that will take into account the position of letters in the required word too?</p><p>Well, I did not include that functionality because sometimes, you might not have even a single letter in green (a letter whose position in the word is confirmed to be correct). This might have restricted the usage of this app.</p><p>So for now, the app seems pretty inclusive of all the use cases! 😊</p><p>Happy Wordling!</p><p><a href="https://www.streamlit.io/">Streamlit</a> is an open-source Python library using which, in just a few minutes, you can build and deploy your own app. Try for yourself!</p><p><a href="https://docs.streamlit.io/library/get-started/create-an-app">Create an app - Streamlit Docs</a></p><p><em>You can find the code for this app in my </em><a href="https://github.com/nazianafis/WORDLE-Assistant"><em>GitHub repo</em></a><em>. Feel free to fork it if you have ideas on how it can be improved upon. Connect with me </em><a href="http://linkedin.com/in/nazianafis"><em>over LinkedIn</em></a><em>, or you can also follow my writings </em><a href="https://nazianafis.medium.com/"><em>here</em></a><em>.</em></p><p><em>Until next time!</em> (∗ ･‿･)ﾉ</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7d8d9f145dd7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Lightweight PyTorch Implementation of Neural Style Transfer]]></title>
            <link>https://medium.com/geekculture/a-lightweight-pytorch-implementation-of-neural-style-transfer-86603e5eb551?source=rss-428a067ab315------2</link>
            <guid isPermaLink="false">https://medium.com/p/86603e5eb551</guid>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[visual-computing]]></category>
            <category><![CDATA[pytorch]]></category>
            <category><![CDATA[neural-style-transfer]]></category>
            <category><![CDATA[vgg19]]></category>
            <dc:creator><![CDATA[Nazia Nafis]]></dc:creator>
            <pubDate>Sat, 18 Dec 2021 13:24:51 GMT</pubDate>
            <atom:updated>2022-07-31T04:48:33.401Z</atom:updated>
            <content:encoded><![CDATA[<h4>THE AI AISLE</h4><h4>Create your own digital art with convolutional neural networks, in five simple steps!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wif1a4KqFHzVI40QOzbQKQ.png" /><figcaption>Image by author</figcaption></figure><p>Art transcends human existence. We see its significance throughout the course of history — from pre-historic times, through some of the greatest river valley civilizations, forts and courts of monarchs, right up to the modern techno age. Art has been a means to express one’s vision of the world. The legendary Pablo Picasso once said,</p><blockquote><em>“It took me four years to paint like Raphael,<br>but a lifetime to paint like a child.”</em></blockquote><blockquote><em>— Pablo Picasso</em></blockquote><p>Most art follows a pattern — a pattern that is pleasing and stimulates our brain. The next time you see an artwork, try to notice its color theme, or the brush strokes in it. You will see a pattern emerging out of it. We humans are skilled in sub-consciously recognizing these patterns. Now, with the help of neural networks, this ability to recognize and artificially recreate patterns has been developed too.</p><h4>The Research Paper by Gatys et al.</h4><p>A<a href="https://arxiv.org/abs/1508.06576"> research paper</a> titled<strong> </strong>‘A Neural Algorithm of Artistic Style’ by <em>Gatys et al</em>, (originally released to ArXiv in 2015 and subsequently accepted by the CVPR conference in 2016), was the first in neural style transfer, and is still considered the most ground-breaking work till date in the domain.</p><p>In this article, we will build a lightweight PyTorch implementation of neural style transfer as discussed in this seminary paper, and learn how to transfer popular art styles onto <em>any</em> image in five simple steps. Let’s go!</p><h3>Table of Contents</h3><ol><li>Overview</li><li>How Does it Work?</li><li>Getting Started<br>– File Description<br>– Dependencies<br>– How to use NST on images of your own?</li><li>Obtained Output</li><li>Applications of NST</li><li>Acknowledgements</li></ol><h3>Overview</h3><p>Neural style transfer (NST) is a technique that takes two images — a content image and a style reference image — and blends them together so that the output looks like the content image, but painted in the style of the style reference image.</p><p>It is an example of image stylization (an image processing and manipulation technique) within the broader field of <strong>non-photorealistic rendering</strong>.</p><h3>How Does it Work?</h3><ul><li>We make use of a pre-trained convolutional neural network (VGG19, in our case) to extract the image details.<br>– Starting from the network’s input layer, the <strong>first few layer activations</strong> represent low-level features like colors and textures (the<strong> ‘style’</strong>).<br>– As we step through the network, the <strong>final few layers</strong> represent higher-level features (the <strong>‘content’</strong>)— for example the cat’s eyes and ears in this case.</li><li>This is why, we begin by taking our content image, feeding it through VGG19, and sampling the network activations at a late convolution layer(conv4_2).</li><li>Then, we take our style image, feed it through the same network, and sample the network activations at the early to middle convolution layers (conv1_1, conv2_1, conv3_1, conv4_1, conv5_1).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tobu4DVYO2MY9ZxVgPacGQ.png" /><figcaption>VGG 19 architecture (image by author)</figcaption></figure><ul><li>These activations are encoded into a <strong>Gram matrix representation</strong>, and serve to denote the ‘style’ of the image.</li><li>Our goal is to synthesize an output image that exhibits the contents of one image, with the style of another. For this, we calculate the following losses:<br>– <strong>content loss</strong>, which is the L2 distance between the content image and the generated image<strong>,<br></strong>– <strong>style loss</strong>, which is the sum of L2 distances between the Gram matrices of the representations of the content image and the style image, extracted from different layers of VGG19,<br>– <strong>total variation loss</strong>, which is used for spatial continuity between the pixels of the generated image, thereby de-noising it and giving it visual coherence, and<br>– <strong>total loss</strong>, which is the sum of all of the above losses multiplied by their respective weights.</li><li>An iterative <strong>optimization technique</strong> (L-BFGS, in our case) is then employed to gradually minimize theses losses to achieve the desired results.</li></ul><h3>Getting Started</h3><h4>➡ <strong>File Description</strong></h4><p>You can have a look at the code for neural style transfer in my <a href="https://github.com/nazianafis/Neural-Style-Transfer">repository</a> on GitHub.</p><ul><li><strong>vgg19.py</strong> contains the VGG19 model definition, specifying which layers are to be used for style representation and which one for content.</li><li><strong>NST.py</strong> is the file that contains steps such as manipulation, generation, optimization, tuning and saving of the final output image.</li></ul><pre>Neural-Style-Transfer<br>    ├── data<br>    |   ├── content-images<br>    |   ├── style-images<br>    ├── models/definitions     <br>    │   ├── <strong>vgg19.py</strong>         # VGG19 model definition<br>    ├── <strong>NST.py</strong>               # The main python file<br>    ├── LICENSE<br>    └── README.md</pre><h4>➡ <strong>Dependencies</strong></h4><ul><li>Python 3.9+</li><li>Framework: PyTorch</li><li>Libraries: os, numpy, cv2, matplotlib, torchvision</li></ul><h4>➡ <em>How to use NST on images of your own?</em></h4><ol><li>Clone the <a href="https://github.com/nazianafis/Neural-Style-Transfer">repository</a> and move to the downloaded folder:</li></ol><pre>$ git clone https://github.com/nazianafis/Neural-Style-Transfer </pre><pre>$ cd Neural-Style-Transfer</pre><p>2. Move your content/style image(s) to their respective folders inside the data folder.</p><p>3. Go to NST.py, and in it, set the PATH variable to your downloaded folder. Also set CONTENT_IMAGE, STYLE_IMAGE variables as your desired images</p><pre>$ PATH = &lt;your_path&gt;<br>   <br>$ CONTENT_IMAGE = &lt;your_content_image_name&gt;<br>$ STYLE_IMAGE = &lt;your_style_image_name&gt;</pre><p>4. Run NST.py:</p><pre>$ python NST.py</pre><p>5. That’s it! Find your generated image in the output-images folder inside data.</p><h3><strong>Obtained Output</strong></h3><p>The following output images were generated using no image manipulation programs other than the code described in this article. I encourage you to try it!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wM6yZ3YRSE0rnIZFpU5_1w.png" /><figcaption>📷 — All output images by author</figcaption></figure><h3>Practical Applications</h3><p>There are many real-world use cases for NST. A non-exhaustive list would contain:</p><ul><li><strong>Photo and video editing: </strong>With recent advances in style transfer, anyone (artist or not) can create their own artistic masterpieces and share with the world. This trend has also been evident from the rise of notable end-user applications such as <a href="https://deepart.io/">DeepArt</a> and Prisma.</li><li><strong>Commercial art: </strong>With humongous popular inquisitiveness in NFTs (non-fungible tokens), the market for art creation and consumption is only growing in size. Artists can now lend/sell their art style to others, allowing new and innovative representations of their styles to live alongside original masterpieces.</li><li><strong>Gaming and Virtual Reality:</strong> Metaverse is the latest buzzword. We are swiftly moving towards digitalization of the real world, through a combination of augmented reality (AR) and virtual reality (VR) services, and visual computing (image and video manipulation, computer graphics, computer vision) skills are touted to be in high demand in near future.</li></ul><p>If you’re thinking of getting into visual computing, the time is now! 😊</p><h3>Acknowledgements</h3><p>These are some of the resources I referred to while working on this project. You might want to check them out.</p><ul><li>PyTorch’s <a href="https://pytorch.org/tutorials/advanced/neural_style_tutorial.html">tutorial on NST</a></li><li>Aleksa Gordic’s <a href="https://github.com/gordicaleksa/pytorch-neural-style-transfer">implementation</a></li><li>The original paper on neural style transfer by <a href="https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Gatys_Image_Style_Transfer_CVPR_2016_paper.pdf"><em>Gatys et al</em></a>.</li><li>The original paper on <a href="https://arxiv.org/abs/1409.1556">VGG19</a></li><li><a href="https://commons.wikimedia.org/wiki/Category:Images">Wikimedia</a>, <a href="https://unsplash.com/">Unsplash</a> for all the content and style images</li></ul><p>In the coming days, I plan on learning more about image generation/manipulation via machine learning. I also aim to create more art with AI (I’ve been fascinated with fractals for some time now!). Feel free to contact me if you find bugs(🕷)/have suggestions.</p><p><a href="https://github.com/nazianafis/Neural-Style-Transfer">GitHub - nazianafis/Neural-Style-Transfer: Transferring the style of one image to the contents of another image, using PyTorch and VGG19.</a></p><p><em>I hope this article was of use to you. You can connect with me </em><a href="http://linkedin.com/in/nazianafis"><em>on LinkedIn</em></a><em>, or follow my writings </em><a href="https://nazianafis.medium.com/"><em>here</em></a><em>.</em></p><p><em>Until next time!</em> (∗ ･‿･)ﾉ゛</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=86603e5eb551" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/a-lightweight-pytorch-implementation-of-neural-style-transfer-86603e5eb551">A Lightweight PyTorch Implementation of Neural Style Transfer</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[11 Tips For A Smooth Udacity Nanodegree Experience]]></title>
            <link>https://nazianafis.medium.com/what-you-can-learn-from-my-udacity-nanodegree-experience-part-i-74aed83890?source=rss-428a067ab315------2</link>
            <guid isPermaLink="false">https://medium.com/p/74aed83890</guid>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[nanodegree]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[life-lessons]]></category>
            <category><![CDATA[mooc]]></category>
            <dc:creator><![CDATA[Nazia Nafis]]></dc:creator>
            <pubDate>Wed, 04 Aug 2021 15:49:49 GMT</pubDate>
            <atom:updated>2023-01-24T10:54:03.638Z</atom:updated>
            <content:encoded><![CDATA[<h4>STUDY TIPS</h4><h3>11 Tips for a smooth Udacity Nanodegree experience</h3><h4>Read this before enrolling!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SchninASo4gr0S4gCHxi_w.jpeg" /><figcaption>📷 — <a href="https://unsplash.com/@josefandiaz?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Josefa nDiaz</a></figcaption></figure><p>Recently, I completed my Udacity <a href="https://graduation.udacity.com/confirm/XGKQ3GDU">Nanodegree in Data Science</a>, and it was nothing short of fantastic! If you too are interested in enrolling in the program but aren’t entirely sure how to go about it, <strong>here are some tips you might find useful:</strong></p><h3>1. Be true to yourself</h3><p>Why do you want to join the program?</p><blockquote>“When you know your why, you’ll know your way.”</blockquote><blockquote>―Michael Hyatt</blockquote><p>I’ve seen students drop out mid-program because of a multitude of reasons, including, but not limited to, time constraints (difficulty managing the nanodegree with a full-time job/family responsibilities/studies), financial constraints, change of plans, or even an altogether shift of interest from the subject.</p><p>One way to save yourself from such situations is to first <strong>decide why you want it</strong>, and then chart out a plan for it. Is it to brush up your concepts in a domain? Does the topic intrigue you? Do you want to build a portfolio of projects for display? Are you looking for a career switch and wish to use the nanodegree to springboard into the field?</p><p>Those are all perfectly valid reasons! The point is, you need to have a reason.</p><p>Once you have that, you can begin charting out a plan for it. Read ahead now for <strong>tips that can help you save time and money.</strong></p><h3>2. Get through the syllabus and the prerequisites</h3><p>Nothing can be more painful than enrolling in a program and realizing its unmet requirements later. Not only would it make you take longer to complete the program (thereby incurring additional costs), but it would also be a cause of unnecessary stress.</p><p>That’s why when you’ve zeroed-in on a program, scan its <strong>syllabus</strong> to get an overview of what will be taught, and the <strong>prerequisites</strong> to upskill yourself, to ensure you don’t fall behind once the program begins.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rBbwymhDt6iGxu2LWCXczg.jpeg" /></figure><h3>3. Keep an eye out for scholarships</h3><p>Udacity provides scholarships from time-to-time. Information about it is regularly updated <a href="https://www.udacity.com/scholarships">here</a>.</p><p>You can avail them to get an opportunity to <strong>complete a nanodegree program for free!</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*XjYwYEvLCo47gjQGZvcW-g.gif" /></figure><h3>4. Make use of the free courses</h3><p>Udacity also provides a plethora of <strong>free courses</strong> in collaboration with tech giants such as Google, Microsoft Azure, Amazon AWS, and Facebook AI. You can find the entire list <a href="https://www.udacity.com/courses/all?type=free%20courses">here</a>.</p><p>So don’t just sit there waiting for scholarships! Pick up a course and begin learning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*irS7o1FGLsJjD9habGnCIA.jpeg" /></figure><h3>5. Keep an eye on discounts</h3><p>Now, you do not necessarily have to shell out ~$1600 for the recommended 4 month nanodegree program. Udacity conducts<strong> flash sales from time-to-time </strong>(usually towards the month-end) in which programs are available at discounted rates.</p><p>Use them to your advantage!</p><h3>6. Pick the right payment plan</h3><p>There are two plans available: an upfront one-time payment, and a monthly pay-as-you-go variant.</p><p>Unless you absolutely struggle with meeting deadlines, you’ll be fine with the monthly plan. Typically a four month nanodegree program should take you <strong>nothing more than two months</strong> — many have even done it <a href="https://www.reddit.com/r/Udacity/comments/fsmfju/can_you_finish_a_nano_degree_in_one_month_while/">in a month</a>.</p><h3>7. Schedule your studies</h3><p>Plan your nanodegree journey out in advance. In particular, <strong>do not underestimate the projects </strong>— they will take long — and might even need multiple resubmissions.</p><p>Plan accordingly!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6ZzEjOBiqu0azUDAIxJR-A.png" /></figure><h3>8. Project-first approach</h3><p>Unlike other MOOCs where first a topic’s foundation is laid brick-by-brick, Udacity nanodegrees are built around projects. It’s a somewhat top-down approach, and you need to get used to it in order to sail smoothly through the courses.</p><p>Depending on the nanodegree program, there will be around 4 to 7 projects for you to complete towards the graduation. <em>Protip</em>: <strong>Submit the projects on a priority basis</strong> and then begin watching the videos.</p><h3>9. Shun shyness</h3><p>There’s a wonderful opportunity to <strong>network with technical mentors</strong>, peers, and alumni. You can forge lifelong meaningful relationships.</p><p>Make good use of it!</p><h3>10. Optional, yet crucial exercises</h3><p>Complete the <strong>optional extracurricular exercises</strong> like GitHub review and LinkedIn review. They provide valuable real-life wisdom, and you stand only to benefit from them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jl_3Fo3rXApoMtJ1QCxbHw.png" /></figure><h3>11. Lifelong lessons</h3><h4>11.1 Go back to the extracurriculars</h4><p>As on the date of writing this review, Udacity provides lifelong static access to the course content once you’ve graduated. This is the time to revisit any missed/overlooked courses to get in-depth clarity in those concepts.</p><h4>11.2 Project portfolio</h4><p>Next, showcase the projects you’ve made on your <strong>personal portfolio website</strong>. You guessed it right, recruiters value demonstrated hands-on experience with technologies more than mere certificates!</p><p>Preferably, try making <strong>blog posts</strong> explaining the problem statement, your methodology, and the achieved outcomes. That’s how it works in the corporate world. You’ve worked on a dataset — cleaned, adjusted, analyzed, and visualized it — but if you can’t explain your findings to others in an easy, jargon-less way, well then it’s not the best.</p><p>Work on those <strong>communication skills</strong>!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*8WcCptWWKY6g-ByJ9n53UQ.gif" /></figure><p>Your time is valuable, and competition around the world is getting sharper by the minute — which means that you cannot afford to lose on either time or skills. Navigate wisely!</p><p>I wish you all the best for your data science/machine learning journey. Let me know if you’re looking for guidance in the field, would be happy to assist.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FXzWR4MGjAY2OQvZMcRZuw.jpeg" /></figure><p><em>I hope this article was of use to you. You can connect with me </em><a href="http://linkedin.com/in/nazianafis"><em>on LinkedIn</em></a><em>, or follow my writings </em><a href="https://nazianafis.medium.com/"><em>here</em></a><em>.</em></p><p><em>Until next time!</em> (∗ ･‿･)ﾉ゛</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=74aed83890" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The story behind ‘random.seed(42)’ in machine learning]]></title>
            <link>https://medium.com/geekculture/the-story-behind-random-seed-42-in-machine-learning-b838c4ac290a?source=rss-428a067ab315------2</link>
            <guid isPermaLink="false">https://medium.com/p/b838c4ac290a</guid>
            <category><![CDATA[random]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[random-seed]]></category>
            <category><![CDATA[hitchhikers-guide]]></category>
            <category><![CDATA[douglas-adams]]></category>
            <dc:creator><![CDATA[Nazia Nafis]]></dc:creator>
            <pubDate>Tue, 03 Aug 2021 11:18:16 GMT</pubDate>
            <atom:updated>2023-01-24T10:52:45.480Z</atom:updated>
            <content:encoded><![CDATA[<h4>PYTHON CHARMERS CLUB</h4><h4>Answer to the great question of “Life, the universe, and everything”!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ulUYE4jV6GFw90odCkPxXA.jpeg" /><figcaption>📷 — <a href="https://unsplash.com/@ihor_malytskyi?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Ihor Malytskyi</a></figcaption></figure><p>You must have come across <em>random.seed(42)</em> some time or the other in data science/machine learning. So what does it mean?</p><p><strong>First, let’s review some basics:</strong></p><h3>What is random() ?</h3><p><em>random()</em> is a function that is used to generate <strong>pseudo-random</strong> numbers in Python.</p><p>Computer algorithms can only produce seemingly random (or pseudo-random numbers), although some naturally occurring phenomena could be incorporated to create randomness artificially.</p><figure><img alt="Sherlock shouting “That’s so RANDOM!”" src="https://cdn-images-1.medium.com/max/500/1*bPO4ujkM5vwNTEcC5EzoZQ.gif" /></figure><p>For the <em>random() </em>function, Python uses <a href="https://documentation.help/Python-2.5/node112.html"><strong>Mersenne Twister</strong></a> as the core generator. It produces 53-bit precision floats and has a period of 2**19937–1. The Mersenne Twister is also one of the most extensively tested random number generators in existence. Being completely deterministic, it is not suitable for all purposes, and is completely unsuitable for cryptographic purposes.</p><h3>What is random.seed() ?</h3><p><em>random.seed()</em> function initializes the random number generator with the given value.</p><p>The <em>random.seed(a=None, version=2) </em>function takes the following two arguments:</p><ul><li>If <em>a</em> is omitted or <em>None</em>, the current system time is used. If randomness sources are provided by the operating system, they are used instead of the system time. And if <em>a</em> is an <em>int</em>, it is used directly.</li><li>With <em>version=2</em> (which has become the default since Python 3.2), a <em>str</em>, <em>bytes</em>, or <em>bytearray</em> object gets converted to an <em>int </em>and all of its bits are used. In version 1, the <em>hash()</em> of <em>a</em> was being used instead.</li></ul><p>The seed is given an integer value to ensure that the results of pseudo-random generation are reproducible. By re-using a seed value, the same sequence should be reproducible from run to run as long as multiple threads are not running. <strong>Reproducibility </strong>is a very important concept that ensures that anyone who re-runs the code gets the exact same outputs.</p><p><strong>Input</strong></p><pre>import random<br>for i in range(5):<br>    random.seed(1)<br>    print(random.randint(1, 1000))</pre><p><strong>Output</strong></p><pre>138<br>138<br>138<br>138<br>138</pre><h3>What is the significance of random.seed(42) ?</h3><p>It’s a pop-culture reference!</p><p>In Douglas Adams’s popular 1979 science-fiction novel <strong><em>The Hitchhiker’s Guide to the Galaxy</em></strong><em>,</em> towards the end of the book, the supercomputer <a href="https://hitchhikers.fandom.com/wiki/Deep_Thought">Deep Thought</a> reveals that the answer to the great question of “life, the universe and everything” is 42.</p><blockquote>[Seven and a half million years later…. Fook and Lunkwill are long gone, but their descendants continue what they started]</blockquote><blockquote>“All right,” said Deep Thought. “The Answer to the Great Question…”<br>“Yes..!”<br>“Of Life, the Universe and Everything…” said Deep Thought.<br>“Yes…!”<br>“Is…” said Deep Thought, and paused.<br>“Yes…!”<br>“Is…”<br>“Yes…!!!…?”<br>“Forty-two,” said Deep Thought, with infinite majesty and calm.”</blockquote><blockquote><strong>―Douglas Adams, The Hitchhiker’s Guide to the Galaxy</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9Szd_JYR5Up3tAAnKkEbHw.jpeg" /><figcaption>📷 — screen grab from the <a href="https://www.imdb.com/title/tt0371724/">movie</a></figcaption></figure><p>Adams’ choice of the number 42 has become a fixture of geek culture. People find humor in it, and some, out of reverence for the classic sci-fi literature use 42 in various places.</p><p>There have been many theories from fans in an attempt to explain why the number 42 was chosen.</p><ul><li>Some propose that it was chosen because 42 is 101010 in binary code, others have pointed out that light refracts through a water surface by 42 degrees to create a rainbow.</li><li>Others have commented that light requires 10⁻⁴² seconds to cross the diameter of a proton.</li><li>Yet others say that 42 is Adams’ tribute to the indefatigable paperback book, and is the average number of lines on an average page of an average paperback.</li><li>Another common theory is that 42 refers to the number of laws in cricket, a recurring theme of the books<a href="https://hitchhikers.fandom.com/wiki/42">¹</a>.</li></ul><p>However, Douglas Adams himself revealed the reason why he chose 42 in <a href="https://groups.google.com/g/alt.fan.douglas-adams/c/595nPukE-Jo/m/koaAJ3tPBtEJ">this message</a>. “It was a joke. It had to be a number, an ordinary, smallish number, and I chose that one. I sat at my desk, stared into the garden and thought ‘42 will do!’ ”</p><p>Check out the <strong><em>The Hitchhiker’s Guide to the Galaxy</em></strong> if you haven’t already, and keep practicing Python programming! 🐍</p><p><a href="https://www.goodreads.com/book/show/386162.The_Hitchhiker_s_Guide_to_the_Galaxy">The Hitchhiker&#39;s Guide to the Galaxy (The Hitchhiker&#39;s Guide to the Galaxy, #1)</a></p><p><em>I hope this article was of use to you. You can connect with me </em><a href="http://linkedin.com/in/nazianafis"><em>on LinkedIn</em></a><em>, or follow my writings </em><a href="https://nazianafis.medium.com/"><em>here</em></a><em>.</em></p><p><em>Until next time!</em> (∗ ･‿･)ﾉ゛</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b838c4ac290a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/the-story-behind-random-seed-42-in-machine-learning-b838c4ac290a">The story behind ‘random.seed(42)’ in machine learning</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Predicting Customer Churn  for Music Streaming Service ‘Sparkify’]]></title>
            <link>https://nazianafis.medium.com/predicting-customer-churn-for-music-streaming-service-sparkify-3e281a93273?source=rss-428a067ab315------2</link>
            <guid isPermaLink="false">https://medium.com/p/3e281a93273</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[sparkify]]></category>
            <category><![CDATA[random-forest]]></category>
            <category><![CDATA[apache-spark]]></category>
            <dc:creator><![CDATA[Nazia Nafis]]></dc:creator>
            <pubDate>Sat, 31 Jul 2021 16:12:57 GMT</pubDate>
            <atom:updated>2022-07-09T11:27:25.161Z</atom:updated>
            <content:encoded><![CDATA[<h4>DATA SCIENCE MINES</h4><h3>Predicting Customer Churn for a Music Streaming Service</h3><h4>Customer churn can be tricky for businesses — machine learning can help!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K0tHXjLSkE3rJoLPQW1oKw.jpeg" /><figcaption>📷 — <a href="https://unsplash.com/@thefakebhogra?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Siddharth Bhogra</a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vZvmmVqveS2XurbxMlLPUg.png" /></figure><h3>Project Overview</h3><p>This project involves a fictitious, subscriber-focused music streaming service ‘Sparkify’ (akin to Spotify). The aim is to utilize machine learning to predict <strong>customer churn</strong> even before it actually happens, so that Sparkify can take steps to check it.</p><blockquote><strong>What is Churn? — </strong>Churn rate (also known as ‘the rate of attrition’) is the rate at which customers stop doing business with an entity.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*9zeJ56Led1bV9C7rJwJ5mA.png" /></figure><p>The approach, methodology, and conclusions are documented here. To have a glimpse of the project code itself, check <a href="https://github.com/nazianafis/Sparkify-Churn-Prediction">here</a>.</p><h3>Problem Statement</h3><p>Business need to retain their customers in order to thrive. Churn (the process of losing customers) is therefore an important business problem.</p><p>We are given a dataset of Sparkify’s subscribers, alongwith some attributes. Our goal is to predict whether a user will churn or not. For this, we will first find correlations amongst attributes to determine what factors lead a user to churn. Next, we will make a prediction on a given user with certain level of confidence.</p><h3>Metrics</h3><p>The <em>Sparkify-Churn-Prediction.ipynb </em>is the main ipython notebook that contains all the code. It imports multiple modules from <em>pyspark </em>and <em>time </em>libraries. Random seed is chosen as 42. For our business problem, churn is defined as a user who has ‘Cancellation Confirmation’ page.</p><p>In feature engineering, <strong>multicollinearity </strong>(strongly correlated features), specifically with <strong>correlation &gt;= 0.8</strong> are removed, which leaves us with 43 features. The features are then transformed to bring them closer to normal distributions. The entire feature engineering code has been compiled to be scaled up later for work on larger datasets.</p><p><strong>F1 score</strong> is used as the evaluation metric. Machine learning pipeline is built, trained, and evaluated on naive predictor, logistic regression, and random forest classifier. Performance of the models on training and testing datasets is noted and hyperparameters are tuned accordingly.</p><h3>Data Exploration and Visualization</h3><p>The dataset for this project has been provided by Udacity. There are 3 datasets available:</p><ul><li><strong>Full</strong> dataset (~12GB) available on Amazon EMR cluster,</li><li><strong>Medium</strong> dataset available on IBM Watson cluster, and</li><li><strong>Mini</strong> dataset (~128MB) available locally.</li></ul><p>This project uses the mini dataset to get used to the data and to carry out all the necessary preparation work. Later we will expand it for the full dataset and deploy it on <strong>Amazon EMR</strong> cluster.</p><p>We use <strong>Apache Spark</strong> analytics engine for processing large datasets. We make some exploratory analysis and use two machine learning models apart from naive predictor: Logistic regression and Random forest.</p><h3>Step 1: Exploratory Data Analysis</h3><h4>1.1 Overview of the Dataset</h4><p>The dataset contains user demographic information (e.g. username, gender, location) and activity (e.g. song listened, event type, device used).</p><p>There are 286000 rows in the dataset with the following attributes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/565/1*7F-i0YUoaLLZbqSu1__aHQ.png" /></figure><h4>1.2 Define Churn</h4><p>For our project, we define churn as users who have a <strong>‘Cancellation Confirmation’</strong> page.</p><pre>flag_churn_event = udf(lambda x: 1 if x == <strong>&quot;Cancellation Confirmation&quot;</strong> else 0, IntegerType())</pre><p>We find that <strong>23.1%</strong> users have churned.</p><h4>1.3 Compare Behavior of Non-Churn vs Churn Users</h4><p><strong>1.3.1 By User Levels: </strong>There are two user levels: <em>free </em>and <em>paid</em>. Exploratory analysis reveals that there are more free users than paid users, and the free users have a slightly higher churn rate than paid users.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/464/1*0jf1R5UHyIGwLZ6_azPGXw.png" /></figure><p>While free users have larger population, they contribute much less page visits compared to paid users, implying that paid users engage more richly with the service.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/515/1*aB-tTQ1axotn5YgjDVhcHw.png" /></figure><p><strong>1.3.2 By Days of the Week:</strong> Most users are logged in during the weekdays as compared to the weekends.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/736/1*6SrxSZakFqsoHrhbelK7oQ.png" /></figure><p><strong>1.3.3 By Page Events:</strong> <em>NextSong</em>, <em>Thumbs Up</em>, and <em>Add to Playlist</em> are the three most recurring page events, which signifies that Sparkify<em> </em>is a popular service with largely positive interactions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1018/1*cmMuaniIZ0gO2nZ-qJGd_g.png" /></figure><p><strong>1.3.4 By Thumbs Up/Thumbs Down:</strong> Users who have given more Thumbs Down have a tendency to churn more than their counterparts, extending the logic that dissatisfied users are more likely to churn.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/502/1*chUqbTuHTqXtHRduNzDmfQ.png" /></figure><h3>Step 2: Feature Engineering</h3><h4>Step 2.1 Create Features to Train the Model on</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1002/1*SQSp62o4Au7eeT5QXCxWLA.png" /></figure><ul><li>User’s most recent level (free/paid)</li></ul><pre>func_levels = udf(lambda x: 1 if x==&quot;paid&quot; else 0, IntegerType())<br>levels = df_sub2.select([&#39;userId&#39;, &#39;level&#39;, &#39;ts&#39;])\<br>    .orderBy(desc(&#39;ts&#39;))\<br>    .dropDuplicates([&#39;userId&#39;])\<br>    .select([&#39;userId&#39;, &#39;level&#39;])\<br>    .withColumn(&#39;level&#39;, func_levels(&#39;level&#39;).cast(IntegerType()))</pre><ul><li>Amount of time, number of artists, number of songs, and number of sessions that user has engaged with</li></ul><pre>engagement = df.groupBy(&#39;userId&#39;)\<br>    .agg(<br>         countDistinct(&#39;artist&#39;).alias(&#39;num_artists_dist&#39;), <br>         countDistinct(&#39;sessionId&#39;).alias(&#39;num_sessions&#39;),<br>         countDistinct(&#39;song&#39;).alias(&#39;num_songs_dist&#39;),<br>         count(&#39;song&#39;).alias(&#39;num_songs&#39;),<br>         count(&#39;page&#39;).alias(&#39;num_events&#39;),<br>         Fsum(&#39;length&#39;).alias(&#39;tot_length&#39;))</pre><ul><li>Mean and Standard Deviation of the number of songs listened to, per artist</li></ul><pre>per_artist = df.filter(~df[&#39;artist&#39;].isNull())\<br>    .groupBy([&#39;userId&#39;, &#39;artist&#39;])\<br>    .agg(count(&#39;song&#39;).alias(&#39;num_songs&#39;))\<br>    .groupBy(&#39;userId&#39;)\<br>    .agg(avg(col(&#39;num_songs&#39;)).alias(&#39;avg_songs_per_artist&#39;),<br>         stddev(col(&#39;num_songs&#39;)).alias(&#39;std_songs_per_artist&#39;)<br>    ).fillna(0)</pre><ul><li>Mean and Standard Deviation of the number of songs listened to, per session, and the time spent per session</li></ul><pre>per_session = df.groupBy([&#39;userId&#39;, &#39;sessionId&#39;])\<br>    .agg(<br>         max(&#39;ts&#39;), <br>         min(&#39;ts&#39;), <br>         count(&#39;song&#39;).alias(&#39;num_songs&#39;)<br>    )\<br>    .withColumn(&#39;time&#39;, (col(&#39;max(ts)&#39;)-col(&#39;min(ts)&#39;))/lit(1000))\<br>    .groupBy(&#39;userId&#39;)\<br>    .agg(<br>         stddev(col(&#39;time&#39;)).alias(&#39;std_time_per_session&#39;), <br>         avg(col(&#39;time&#39;)).alias(&#39;avg_time_per_session&#39;),<br>         stddev(col(&#39;num_songs&#39;)).alias(&#39;std_songs_per_session&#39;),<br>         avg(col(&#39;num_songs&#39;)).alias(&#39;avg_songs_per_session&#39;)<br>    ).fillna(0)</pre><p><strong>Join the Engineered Features together:</strong></p><pre>dataset = churn.join(levels, [&#39;userId&#39;])\<br>    .join(time_gender, [&#39;userId&#39;])\<br>    .join(engagement, [&#39;userId&#39;])\<br>    .join(per_artist, [&#39;userId&#39;])\<br>    .join(per_session, [&#39;userId&#39;])\<br>    .join(agents, [&#39;userId&#39;])\<br>    .join(pages, [&#39;userId&#39;])\<br>    .join(locations, [&#39;userId&#39;])</pre><h4>Step 2.2 Check Multicollinearity</h4><p>We assess correlation between each pair of features and remove any feature that has &gt;0.8 correlation with any other feature.</p><pre>corr = dataset_pd[correlated_cols].corr()<br>cols_to_remove = []<br>counter = 0<br>for coln in corr.columns:<br>    counter += 1<br>    <strong>if corr[coln].iloc[counter:].max() &gt;= 0.8:</strong><br>        cols_to_remove.append(coln)<br>print(f&quot;Highly correlated features that should be removed:\n\n{cols_to_remove}\n\n&quot;)<br>cols_to_keep = dataset_pd.columns.drop(cols_to_remove).tolist()<br>print(f&quot;Features to keep:\n\n{cols_to_keep}&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9_fQQjbQAV3Xb3yBxtrDZQ.png" /></figure><h4>Step 2.3 Feature transformation</h4><p>We apply log transformation on skewed features to bring their distributions closer to normal.</p><pre>for col_name in col_names:<br>    if col_name in columns_to_transform:<br>        dataset = dataset.withColumn(<br>            col_name, log(dataset[col_name]+1)<br>        )</pre><p>And here’s what we get:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*9A7ru0SvilZ9Qlsl_OpY0A.png" /></figure><h3>Step 3: Machine learning</h3><p>The goal of our machine learning model is to predict Churn (<em>label=1</em>) vs Non-Churn (<em>label=0</em>) based on the previously reengineered features.</p><h4>3.1 Train-Test Split</h4><p>We split our dataset in 80–20 ratio for training and testing.</p><pre>train, test = dataset.drop(&#39;userId&#39;).randomSplit([0.8, 0.2], seed=42)</pre><h4>3.2 Evaluation Metric</h4><p>Churned users are a small subset (23%). If we try to build a model that simply predicts ‘no churn’, it will have reasonably good accuracy (77%) but very poor performance.</p><p>So, instead of accuracy, we use F1 scores as the evaluation metric.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/806/1*moXhv-aLwqeVtkkbKsXWkA.png" /></figure><p>When predicting a churn, <em>precision </em>aims to make sure that it is really a churn, whereas <em>recall </em>aims to not miss any real churns. F1 averages between <em>precision </em>and <em>recall</em>, hence gives a balanced outlook.</p><h4>3.3 Spark Pipeline and Functions</h4><p>After train-test split, we create a PySpark machine learning pipeline that consists of:</p><ul><li>VectorAssembler, which vectorizes input features</li><li>MaxAbsScaler, which re-scales each feature to range [-1, 1]</li><li>A classifier of choice, in this case MultiClassificationEvaluator</li></ul><pre>def buildCV(classifier, paramGrid):<br>    # Configure an ML pipeline<br>    assembler = <strong>VectorAssembler</strong>(inputCols=feature_cols, outputCol=&quot;rawFeatures&quot;)<br>    scaler = <strong>MaxAbsScaler</strong>(inputCol=&quot;rawFeatures&quot;, outputCol=&quot;scaledFeatures&quot;)<br><br>    # Cross validation<br>    crossval = CrossValidator(<br>        estimator=pipeline,<br>        estimatorParamMaps=paramGrid,<br>       evaluator=<strong>MulticlassClassificationEvaluator</strong>(metricName=&#39;f1&#39;),<br>        numFolds=3)</pre><pre>    return crossval</pre><h4>3.4 Model Evaluation and Validation</h4><p>We compare the model performance of following classifiers by using their default hyperparameters.</p><ul><li>Naive predictor, which always predicts no-churn</li><li>Logistic regression, and</li><li>Random forest Classifier.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*UgHXwJSTef04AfXzbKVHuw.png" /></figure><p>The naive model sets a baseline of model performance, F1 = 0.67 and accuracy = 0.77. Unsurprisingly, the two machine learning classifiers perform better than the naive model. <strong>Random Forest</strong> achieves the best performance on training set (F1 = 0.92, accuracy = 0.93) and on the testing set (F1 = 0.74, accuracy = 0.76).</p><h4>Hyperparameter Tuning and Refinement</h4><p>We use <strong>Grid Search</strong> on random forest classifier to tune the hyperparameters.</p><pre>classifier = RandomForestClassifier(labelCol=&#39;label&#39;,\<br>             featuresCol=&#39;scaledFeatures&#39;)</pre><pre>paramGrid = ParamGridBuilder()\<br>            .addGrid(classifier.numTrees,[20,75])\<br>            .addGrid(classifier.maxDepth,[10,20])\<br>            .build()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*a60j6mo8igLByO64j2I39w.png" /></figure><p>After Grid Search, we obtain improved F1 scores, from 0.921 to 1.0 for training and from 0.744 to 0.796 for testing datasets. Accuracy has improved as well, from 0.927 to 1.0 for training and from 0.765 to 0.824 for testing datasets.</p><p>Best parameters:</p><pre>maxDepth:10<br>numTrees:75</pre><h4><strong>Identifying the Most Important Features</strong></h4><p>The most important features in churn prediction turn out to be the time since user’s registration, the amount of advertisements that user has encountered, and the number of thumbs up/thumbs down the user has given.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/732/1*ejmnSdkzQS5IcmlgEDsIpA.png" /></figure><h3>Conclusion</h3><ul><li>Our machine learning model is able to predict churn reasonably well (with f1 score of 0.74). The model performance can further be improved by tuning the hyperparameters or incorporating additional features.</li><li>The most important features in churn prediction are the time since user’s registration, the amount of advertisements that user has encountered and the number of thumbs up/thumbs down the user has given.</li><li>This information can be utilized by Sparkify to chart out a plan as to what actions to take. Reducing the amount of advertisements could be one. Another could be to figure out why relatively newer customers are churning more.</li><li>Sparkify could also have a look at its competitors — are newer customers being drawn towards them? What is it that they find appealing there? Is it the intuitiuve interface, or better music recommendations, or something entirely different? The findings could then be integrated with the existing Sparkify service. A/B tests will be needed to statistically assess the cost-benefit analysis of each action.</li></ul><h4>Improvements</h4><ul><li>We have used Logistic Regression and Random Forest Classifier models on the dataset. For further improvement, <strong>XGBoost </strong>and <strong>LightGBM </strong>models could be some good supervised learning approaches in our scenario.</li><li>On aggregating all features onto the user level, we also recognize a critical issue — some users could be impulsive and on having even a slightly unsatisfactory experience during their last operations, they would move on to the cancellation page to churn.</li><li>In this case, we would need to have the temporal order of the operations for each user, in order to understand the user experience with Sparkify better. So, treating the data as a <strong>Time Series</strong> could provide closer-to-reality outcomes. If we decide to treat the data as time series, <strong>Recurrent Neural Networks</strong> could be a good modeling choice.</li></ul><p>That’s all! I encourage you to dig into the full dataset and draw more analyses from it. You can also have a look at the code in my GitHub <a href="https://github.com/nazianafis/Sparkify-Churn-Prediction">here</a>. Feel free to fork it and embark on your own data exploration journey!</p><p><em>I hope this article was of use to you. You can connect with me </em><a href="http://linkedin.com/in/nazianafis"><em>on LinkedIn</em></a><em>, or follow my writings </em><a href="https://nazianafis.medium.com/"><em>here</em></a><em>.</em></p><p><em>Until next time!</em> (∗ ･‿･)ﾉ゛</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3e281a93273" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Applying Data Exploration Techniques on Stack Overflow Survey 2020]]></title>
            <link>https://nazianafis.medium.com/best-country-to-work-in-2020-15a790b00904?source=rss-428a067ab315------2</link>
            <guid isPermaLink="false">https://medium.com/p/15a790b00904</guid>
            <category><![CDATA[data-exploration]]></category>
            <category><![CDATA[stackoverflow-survey]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[udacity]]></category>
            <dc:creator><![CDATA[Nazia Nafis]]></dc:creator>
            <pubDate>Sat, 12 Jun 2021 07:35:43 GMT</pubDate>
            <atom:updated>2022-07-09T11:33:13.901Z</atom:updated>
            <content:encoded><![CDATA[<h4>DATA SCIENCE MINES</h4><h3>Exploratory Data Analysis (EDA) on Stack Overflow Survey 2020</h3><h4>Have a look at the results of my exploratory data analysis, and discover some more correlations yourself!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZD1fvWxCg7W_H_amrzdvlw.jpeg" /><figcaption>📸— <a href="https://unsplash.com/@kellysikkema?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Kelly Sikkema</a></figcaption></figure><p>Every year since 2011, Stack Overflow has been conducting a survey to understand its user base in particular and the global IT scenario in general.</p><p>The <strong>2020 results have recently been released</strong>. You can download the dataset from Stack Overflow website <a href="https://insights.stackoverflow.com/survey">here</a>. CSV file ‘<em>survey_results_public.csv</em>’ contains the main survey results. I suggest getting a glimpse of the dataset before proceeding.</p><p>With some basic data exploration techniques on answers given by the survey respondents, I was able to unveil some very interesting correlations. This article is aimed at explaining the results I obtained from gathering, assessing, cleaning, analyzing, modeling, and visualizing the survey data based on <a href="https://en.wikipedia.org/wiki/Cross-industry_standard_process_for_data_mining">CRISP-DM</a> principles.</p><p>For an interactive experience, check out the code in my <a href="https://colab.research.google.com/drive/1Xqh_Obd0xeg8yzp6U-KrVHLqiAan-Hqr#scrollTo=znpYHV1hVd0F">Google Colab</a>.</p><blockquote><strong><em>Side Note: What is CRISP-DM?</em></strong></blockquote><blockquote><em>Cross Industry Standard Process for Data Mining (CRISP-DM) is a process model that describes the data science life cycle. It consists of the following main steps:</em></blockquote><ul><li>Business understanding — What does the business need?</li><li>Data Understanding — What data do we have / need? Is it clean?</li><li>Data Preparation — How do we organize the data for modeling?</li><li>Modeling — What modeling techniques should we apply?</li><li>Evaluation— Which model best meets the business objectives?</li><li>Deployment— How do stakeholders access the results?</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/494/1*VH9RlYPSjL6YOtBtHxunqQ.png" /><figcaption>📸—<a href="https://www.kdnuggets.com/2017/01/four-problems-crisp-dm-fix.html">KDnuggets</a></figcaption></figure><h3>Outline of the Analysis</h3><p>While the survey itself is exhaustive and one can find multiple correlations amongst its attributes, for this analysis I focused on the following three questions:</p><blockquote>1. Which country provides the highest job satisfaction for developers?</blockquote><blockquote>2. Is there a difference in job satisfaction between developers who had an IT related undergraduate major and the ones who didn’t?</blockquote><blockquote>3. Do developers from non IT backgrounds think differently about the importance of having a formal education, such as a university degree in Computer Science, for their career?</blockquote><h3>Step 1 : Data Acquisition and Preparation</h3><p>The original survey contains data of 64460 respondents for 61 attributes. I filtered the dataset to take into account only those respondents who identify as ‘developer’ (thereby excluding away any student coders and people who code only as a hobby). As a result, the subset now contained data of just 47192 respondents.</p><p>Next I had to manage empty values. Because both <em>JobSat</em> and <em>NEWEdImpt</em> columns contained categorical data, I filled all their empty cells with the respective mode values.</p><p>Then, I filled all empty values in <em>ConvertedComp</em> column with the respective mean salary of the developer’s country (because salaries depend quite a lot on the country the developer lives in).</p><p>After the above operation, there were still some empty values in <em>ConvertedComp</em> column. This was due to the fact that the mean <em>ConvertedComp</em> value for some countries was equal to NaN (that is, all of the respondents from those countries hadn’t answered the salary question). So I dropped those records.</p><h3>Step 2 : Data Exploration</h3><h4><strong>2.1 Countries with the Highest Job Satisfaction Rates for Developers</strong></h4><p>Because the survey contains data of respondents from more than 180 countries, I narrowed it down to the top 20 based on each country’s total number of respondents. This was desirable because now each country in the top 20 had at least 400 respondents (which was a good enough number for my analysis).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qUbysuR6-K-QFujXhbxv9A.png" /></figure><p>I found that Sweden topped the charts, with almost 50% of the developers feeling ‘very satisfied’. United States stood second and Netherlands came in third.</p><h4>2.2 <strong>Undergraduate Major and Job Satisfaction</strong></h4><p>I split the <em>UndergradMajor</em> column into two — ‘IT Related’ and ‘non IT Related’. Majority of the developers on Stack Overflow (about 65%) had an IT related undergraduate major. Then I calculated the percentages of each value in <em>JobSat</em> to measure job satisfaction for each type of major, and the results were intriguing, to say the least.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m02buyaynr9rIGbJAjMhqQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OlZj8sI6pYY8EDklwG_H5A.png" /></figure><p>I found that developers whose undergraduate major was ‘non IT related’ had better job satisfaction as compared those who had an ‘IT related’ background, with 53% and 35% respondents feeling very satisfied respectively. I thought the difference was unimaginably large, and it piqued my interest. (Side note: Are you from an IT related background? Are you satisfied with your job? Why / why not? Think!)</p><h4>2.3 <strong>Importance of Formal Education</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YRyfMoUsjoFRC6Q9tR1qRg.png" /></figure><p>I found that developers with IT related backgrounds agreed more with the importance of having a formal education, such as a university degree in Computer Science, for their career. The opposite was true for non IT background developers. I found this result intuitive, as the group that had higher job satisfaction despite being from a non IT background, was also the one giving less weight to a formal education in IT for their career — it all made sense!</p><h3>Conclusion</h3><p>Here is a quick summary of my analysis of the Stack Overflow Annual Developer Survey 2020 data:</p><ol><li>I measured <strong>job satisfaction levels</strong> for each country and found that Sweden topped the charts. United States stood second and Netherlands came in third.</li><li>I looked at <strong>undergraduate majors</strong> and found that developers from non IT backgrounds had better job satisfaction.</li><li>I found that developers from non IT fields believed that <strong>having a formal education</strong> was not as important for their career. Their belief was backed up by the fact that despite not having a background in IT, these respondents were not only working as developers but also had higher job satisfaction.</li></ol><p>That’s all! I encourage you to dig deeper into the dataset and draw more analyses from it. You can also have a look at the code in my <a href="https://github.com/nazianafis/StackOverflow-Data-Analysis">GitHub here</a>. Feel free to fork it and embark on your own data exploration journey!</p><p><em>I hope this article was of use to you. You can connect with me </em><a href="https://www.linkedin.com/in/nazianafis/"><em>on LinkedIn</em></a><em>, or follow my writings</em><a href="https://medium.com/@nazianafis"><em> here</em></a><em>.</em></p><p><em>Until next time!</em> (∗ ･‿･)ﾉ゛</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=15a790b00904" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>