<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Deepthi Sudharsan on Medium]]></title>
        <description><![CDATA[Stories by Deepthi Sudharsan on Medium]]></description>
        <link>https://medium.com/@deepthi.sudharsan?source=rss-c19a1f7cb39------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Gb6fPOAx_JN0pakDSVxthg.jpeg</url>
            <title>Stories by Deepthi Sudharsan on Medium</title>
            <link>https://medium.com/@deepthi.sudharsan?source=rss-c19a1f7cb39------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 08:42:06 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@deepthi.sudharsan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Inside AI Interviews: Stories, Patterns, and What Actually Matters]]></title>
            <link>https://medium.com/@deepthi.sudharsan/inside-ai-interviews-stories-patterns-and-what-actually-matters-555684c38598?source=rss-c19a1f7cb39------2</link>
            <guid isPermaLink="false">https://medium.com/p/555684c38598</guid>
            <category><![CDATA[interview-questions]]></category>
            <category><![CDATA[interview]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[job-interview]]></category>
            <dc:creator><![CDATA[Deepthi Sudharsan]]></dc:creator>
            <pubDate>Wed, 21 Jan 2026 02:47:04 GMT</pubDate>
            <atom:updated>2026-01-22T05:06:16.921Z</atom:updated>
            <content:encoded><![CDATA[<p>What AI Job Interviews Are Really Like (From Someone Who’s Been Through Many)</p><p>For a long time, I thought my AI interviews were unusually chaotic. After interviewing with over <strong>20+ companies</strong> — often across <strong>multiple roles within the same company</strong> — I started seeing a pattern in the interview process for these different AI roles. I wouldn’t say this made my interview completely devoid of chaos, but rather, it was more organized chaos. My preparation process became more structured and streamlined.</p><p>Across <strong>AI Research, Research Engineer, Applied Scientist, AI Engineer, Data Scientist, and AI PM roles</strong>, I’ve been through what easily adds up to <strong>50+ interview rounds</strong>. Some companies interviewed me for one role. Some for two or three. Some are across different teams with completely different expectations. At the same time, through mentoring on <a href="https://topmate.io/deepthi_sudharsan"><strong>Topmate</strong></a>, I’ve helped <strong>dozens of candidates</strong> prepare for AI interviews — many of whom went on to land roles at companies like <strong>Microsoft, PwC, Palo Alto Networks, Accenture</strong>, and others. Being on <em>both sides</em> of the table gave me a rare vantage point: I could see <strong>how interviewers think</strong>, <strong>where candidates consistently struggle</strong>, and <strong>what to expect in these interviews</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ajtXCGKO_PvYBPLp" /><figcaption>Photo by <a href="https://unsplash.com/@mkvandergriff?utm_source=medium&amp;utm_medium=referral">Maranda Vandergriff</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>While interview preparation is a topic that is a lot to unpack, I wanted to share a free guide, a high-level overview of how the interview process for different AI roles looks, that has helped me and my mentees prepare for our interviews.</p><h3>How AI Interview Processes Usually Differ by Role</h3><p><em>Note: These are some of the patterns I identified from my own personal experiences and from the experiences of my mentees. Each individual’s experience might differ based on the role, company, interviewer, etc., and hence, this might not necessarily be an exhaustive list of patterns.</em></p><blockquote><em>TL;DR:</em></blockquote><blockquote>Depending on the AI role, organization, and team, the interview process comprises a mix of some of the following types of interviews, either one or more rounds of each.</blockquote><blockquote>1. Resume-based round that touches a bit upon the technical implementation and concepts of the techniques that you have used in your previous projects or organizations.</blockquote><blockquote>2. DSA round with one or more LeetCode questions.</blockquote><blockquote>3. ML coding round that tests your ability to implement AI/ML algorithms from scratch or complete an incomplete AI/ML code.</blockquote><blockquote>4. ML fundamentals round, which tests your basic AI/ML concepts, and the follow-up questions dig a bit deeper into the breadth and depth of the core concepts.</blockquote><blockquote>5. ML System Design round tests your ability to design and implement systems that solve a specific problem while ensuring it is scalable and optimized.</blockquote><h4>1. AI Research (Predoctoral roles)</h4><p>I have had the pleasure of interviewing for Predoctoral research positions at FAANGM companies (and even worked with Microsoft as a Research Fellow), and from my experiences, some of the types of interviews you can expect are definitely as follows:</p><ul><li>A basic resume-based round to understand why you would like to pursue research, and for your interviewer to understand if you have what it takes to research and build upon existing works or ideas, as well as constantly learn and keep up with the numerous publications that are put out in the area of interest.</li><li>A round or two where you are asked to “peer-review” some of the recent or trending papers as if you are an actual reviewer for any of the top paper conferences, to test your ability to read, analyze, and evaluate research publications. You are often asked to report the strengths and weaknesses of the paper(s), suggest changes or modifications to the overall work, explain interconnected concepts or ideas (and even propose new ones extending the read papers).</li><li>A DSA round that tests more of your problem-solving capability, and checks whether you consider all possible test cases and scenarios while implementing your solution.</li><li>A fundamental AI/ML round where you are evaluated on your AI/ML basics. It usually starts off a resume-based discussion to set the stage for some of the concepts that you have already applied in your previous work, and accordingly, subsequent follow-up questions would be asked to test the depth and breadth of your basic knowledge.</li><li>Behavioural round to understand your inclination towards research.</li></ul><h4><strong>2. AI Research Engineer</strong></h4><p>While for the most part, the AI Research Engineer roles that I interviewed for had a similar process to that of the predoctoral research fellow or research scientist roles, there were a few minute differences.</p><ul><li>The DSA round had a bit more emphasis — you might have multiple coding questions, or the problem statement provided would be extended further to test more of your approach, etc.</li><li>In a couple of US-based startups, I was asked to not just report and give feedback on the research papers assigned, but I was also tasked with reproducing one of the papers or writing a pseudo-working implementation of the problem solved in the paper.</li></ul><h4><strong>3–5. Data Scientist/AI Engineer roles/Applied Scientist</strong></h4><p>Similar to how the Research and Research Engineer roles shared a lot of similarities, these three common and important roles share a lot of similarities amongst one another and also have a lot of similarities to the interview process of AI Research Engineer roles.</p><ul><li>Unlike Research roles, you aren’t really technically tested on your research skills here (although research experience is an added advantage for Applied Science roles).</li><li>Data Scientist roles, especially in some of the big MNCs and a handful of startups, usually start with/have DBMS-related questions in their OA (Online Assessment) or as a part of their coding round. I also noticed that these roles gave more emphasis to core and fundamental AI questions rather than GenAI (but again, it differs company to company based on the problem that they are trying to solve). I was often given take-home assignments that tested my AI/ML coding proficiency and implementation. Weirdly, there were also a lot of times where I noticed the job role changing in the title of the interviews scheduled, where some companies used Data Scientist and AI engineer so synonymously.</li><li>For AI Engineer and Applied Science roles, especially, I noticed that there were distinct coding rounds that usually followed one another — the DSA round and the AI/ML coding round. While you can expect LeetCode questions in your DSA round, the AI/ML coding round tests more of your knowledge in implementing some of the basic AI/ML algorithms from scratch. A lot of emphasis was also given on the system design round to understand your thought process when it comes to implementing scalable or optimized systems, or how you would handle the specific scenario that the organization is working on, and which you would be solving if you make it through the interview process successfully. (Edit: I also want to add that in some places, the AI/ML coding round was to fill up the proper commands or lines in an incomplete code snippet/template to get it working).</li><li>For all of these roles, apart from the AI/ML Basics interview, these roles tend to have multiple such interviews where the interviewer delves much deeper into the technical questions, including mathematical derivations or the basic math behind some of the core concepts. A commonly asked example to explain what I mean: the interviewer doesn’t just want to know how your LLM behaves if the temperature is shifted to 0 or 1, but rather how the softmax function is affected and what exactly happens to the probability distribution when the temperatures are changed.</li></ul><h4><strong>6. Others (AI Product Manager, AI Consultant)</strong></h4><ul><li>While I haven’t personally interviewed for a lot of these roles, and neither have my mentees, I wanted to shed a bit of light on these roles as well.</li><li>AI PM — In the handful of times I have interviewed for these kinds of roles, I was asked to write PRDs for an AI product (existing or imaginary), come up with some of the features or things that can be improved or added to an AI product, how I would go about choosing or navigating different components and features to prioritize and build a roadmap, etc.</li><li>AI Consultant — Funnily enough, all of my experience with this role was with the Big 4 companies (out of which, I interned in one), and I think a lot of emphasis was put on my problem-solving skills more. A common experience with a lot of these roles was the initial online assessment that, at a stretch, tested my problem-solving skills, Python skills, DBMS/SQL knowledge, and tested if I could complete an incomplete AI/ML code with the appropriate commands and function definitions.</li></ul><p>P.S. Speaking of “AI interviews”, I also had the opportunity to be interviewed by AI — thrice in fact — as a part of the interview process for some of the upcoming, US-based AI companies. And I must say, while the interview process was different and everyone on Reddit had a lot of different views and opinions on that, it isn’t really that hard to crack. If you end up having an AI interview at the start of any interview process, it most likely asks you general questions or scenario-based questions, and only in rare cases, they have a RAG system of sorts in place or upload your resume as context for the AI to ask you resume-specific questions and validate your technical strengths.</p><p>This was quite a long post to write, and it especially feels like a win these days to write posts without AI, haha! While this post covers the different types of interview processes I had, it doesn’t necessarily cover any specific type of questions or FAQ. Apart from just figuring out what to expect in terms of the type of interviews, over the period of time, after interviewing a lot, you can also roughly guess or know the questions that are very frequently asked and that you can most commonly expect. I might or might not create a separate post on this in the future!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=555684c38598" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TopMate: Your Gateway to Expert Guidance and Networking]]></title>
            <link>https://medium.com/@deepthi.sudharsan/topmate-your-gateway-to-expert-guidance-and-networking-ec0af509e1aa?source=rss-c19a1f7cb39------2</link>
            <guid isPermaLink="false">https://medium.com/p/ec0af509e1aa</guid>
            <category><![CDATA[guidance]]></category>
            <category><![CDATA[teaching]]></category>
            <category><![CDATA[learning]]></category>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[platform]]></category>
            <dc:creator><![CDATA[Deepthi Sudharsan]]></dc:creator>
            <pubDate>Tue, 29 Aug 2023 09:51:40 GMT</pubDate>
            <atom:updated>2023-08-29T09:51:40.511Z</atom:updated>
            <content:encoded><![CDATA[<p>I tried Topmate in July, and so far, it has been a hit and I love it!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/627/1*0hz3EJtgaAxi2DC_Ci9S4g.png" /></figure><p>I had seen much hype over <a href="https://topmate.io/deepthi_sudharsan">Topmate </a>services the past few months and I decided to try it out in July. Here is my experience!</p><p>I was one of the students in the first batch of a very new branch that was started at my University. Being the first batch, I was always on the lookout for seniors who could guide me and provide me with valuable support and tips. So once the next batch came along, I decided to take it upon myself to guide and support my juniors and be the senior that I wished I had!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*kfje_5hMh3GWVSqicgaUJQ.png" /><figcaption>Few of the many people I have helped recently and what they had to say!</figcaption></figure><p>I was constantly helping and resolving their queries, providing them with tips, and motivating them when they felt low. With my help, some students were able to land jobs in consulting firms like the Big 4, Oracle, or programs with different ML startups, etc. Some got their LinkedIn profiles reviewed resumes restructured and so on. And this whole journey has made me feel happier than ever seeing how much of success stories I could be a very small part of! The number of talented individuals that I got to meet has been great!</p><p>So I decided to try <a href="https://topmate.io/deepthi_sudharsan">Topmate</a> in July. I charge a very nominal fee so that I know the individuals are not just booking and then not turning up so that I do not end up losing my free time and not being able to help other aspiring individuals. With the 5–6 bookings that I got, I got to learn a lot, meet amazing people, help them, and guide them, and I got a lot of positive feedback!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l3DBKqjU7VOounaH2vlGVQ.png" /><figcaption><a href="https://topmate.io/deepthi_sudharsan">https://topmate.io/deepthi_sudharsan</a></figcaption></figure><p>In August, I went back to helping directly on LinkedIn but I wasn’t able to organize or schedule time for everyone and that is when I felt, “Yes, Topmate made it easy, let me go back there so I can help and reach as many people as possible!”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*acoffMRrP35yUjjYXoJj_A.png" /><figcaption>Few of the Topmate testimonials. Image Courtesy: <a href="https://topmate.io/deepthi_sudharsan">Topmate</a></figcaption></figure><p>So I have decided to continue providing guidance and networking on Topmate. The best part about Topmate is how easy it is for people to schedule and book calls that too for specific services and how they take care of the rest — including providing reminders on WhatsApp, reaching out to each individual regarding their experience, syncing calendars, and taking care of the meeting conflicts, etc. The website is so easy to use and easy to access so it makes the whole process smooth and anyone can easily navigate through the platform themselves very easily.</p><p>I highly recommend Topmate for :</p><ul><li>Those who can provide services and help others grow</li><li>Those who are looking for a platform to schedule and connect with individuals who can guide them and help them grow</li></ul><p>What services do I provide or how I can be of help?</p><p>I not only provide guidance and reviews etc., but I also learn along with each and every individual as everyone has some story to tell and can teach us something. So if you would like to just network and exchange knowledge or have discovery calls, that is also available!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WvlnrI5JOZd7C2tkdyS8aw.jpeg" /><figcaption>Services I provide include — Just networking with one another, resume review, LinkedIn review, and Career Guidance. I also provide a 30-minute 1:1 discovery call! Image Courtesy: <a href="https://topmate.io/deepthi_sudharsan">Topmate</a></figcaption></figure><p>With the knowledge that I have gained over a period of time in terms of preparing for job interviews, preparing for the GRE, job search, reaching a wide audience via professional platforms like LinkedIn, etc., and domain-specific knowledge of AI, I can share these learning with you so that it can help you in your growth to success.</p><p>I am also open to learning more from you and having conversations regarding your topics of interest and we can also have conversations on things that I learned through curiosity like entrepreneurship, content creation, social media marketing, finance, etc. through the networking or discovery call services provided by me.</p><p>So, if you’re looking for interview prep, guidance, or simply want to connect and expand your professional network, feel free to connect with me on <a href="https://topmate.io/deepthi_sudharsan">Topmate</a>. Let’s collaborate, learn, and succeed together! 🤝</p><p>Remember, success is sweeter when shared. I’m always open to networking, learning from your experiences, and offering my insights to help you shine in your journey. Feel free to reach out, and connect, and let’s create more success stories together!</p><p>Connect with me on Topmate: <a href="https://topmate.io/deepthi_sudharsan">https://topmate.io/deepthi_sudharsan</a>! Can’t wait to connect and expand our learnings!</p><p>Note: This post is not sponsored and is not an ad. It is solely my experience with the platform — raw and unfiltered.</p><p><em>Disclaimer: The views and opinions expressed in this article are solely those of the author and do not necessarily reflect the official policy or position of any organization, company, or entity the author may be associated with. The author’s intent is to provide insights and perspectives based on their personal experiences and knowledge in the field.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ec0af509e1aa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Decoding the Relationship: Language Models and Natural Language Processing]]></title>
            <link>https://medium.com/@deepthi.sudharsan/decoding-the-relationship-language-models-and-natural-language-processing-bbc0cd6754e2?source=rss-c19a1f7cb39------2</link>
            <guid isPermaLink="false">https://medium.com/p/bbc0cd6754e2</guid>
            <category><![CDATA[beginner]]></category>
            <category><![CDATA[introduction]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Deepthi Sudharsan]]></dc:creator>
            <pubDate>Sun, 20 Aug 2023 00:12:21 GMT</pubDate>
            <atom:updated>2023-08-20T00:12:21.898Z</atom:updated>
            <content:encoded><![CDATA[<p>The new debate — Is LLM part of NLP or not?</p><p>In the ever-evolving landscape of technology, concepts like Machine Learning (ML), Artificial Intelligence (AI), Deep Learning (DL), and Data Science were once puzzling intersections in the realm of innovation. The talk of the town was the confusion on how these four were related and how they would all fit in a bigger picture. So the internet around this time was filled with Venn Diagrams for it and people explaining how and in what way they are all connected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/262/1*i793qgyTp0Au-KNSI9Cvhw.png" /><figcaption>Image Courtesy: <a href="https://gist.github.com/krajiv26/839ea54316a257c1fef797deb4d13a8b">https://gist.github.com/krajiv26/839ea54316a257c1fef797deb4d13a8b</a></figcaption></figure><p>Now, the scenario is a tad bit different. The post-GPT-3.5 era has introduced a new debate on how LLMs and NLP are related. And now the internet is flooded with Venn Diagrams for the same. One of which I have used below for the purpose of this post.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/1*vZ95XVYnu09phxjpbEFfHQ.jpeg" /><figcaption>Image Courtesy: <a href="https://blog.dataiku.com/large-language-model-chatgpt">https://blog.dataiku.com/large-language-model-chatgpt</a></figcaption></figure><p>Let us answer that question today. So how are they related?</p><p>LLMs are a subset of NLP. They are not the same, but LLMs are a crucial component of NLP as they are an advancement of NLP capabilities.</p><p>Natural Language Processing or NLP, is the study of human language by AI. It enables computers to comprehend, decipher, and produce text.</p><p>Large Language Models are AI models, such as GPT-3, that can produce, manipulate, and comprehend text in a way that is human-like, improving NLP skills.</p><p>So next time someone asks you this question, you know what the answer is!</p><p><em>Disclaimer: The views and opinions expressed in this article are solely those of the author and do not necessarily reflect the official policy or position of any organization, company, or entity the author may be associated with. The author’s intent is to provide insights and perspectives based on their personal experiences and knowledge in the field.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bbc0cd6754e2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[This best coding practice could have saved me 1 hour of debugging time!]]></title>
            <link>https://medium.com/@deepthi.sudharsan/this-best-coding-practice-could-have-saved-me-1-hour-of-debugging-time-19d9f438b74d?source=rss-c19a1f7cb39------2</link>
            <guid isPermaLink="false">https://medium.com/p/19d9f438b74d</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[developer]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[anaconda]]></category>
            <dc:creator><![CDATA[Deepthi Sudharsan]]></dc:creator>
            <pubDate>Sat, 19 Aug 2023 03:48:20 GMT</pubDate>
            <atom:updated>2023-08-19T10:05:39.050Z</atom:updated>
            <content:encoded><![CDATA[<p>Anaconda Environments — Python Developers’ best friend?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*AKlfbpzqtaWlMfqY" /><figcaption>Photo by <a href="https://unsplash.com/@cdr6934?utm_source=medium&amp;utm_medium=referral">Chris Ried</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Ever since I learnt to code in Python and ever since I started working on Anaconda, I had always heard this advice :</p><blockquote>Try to maintain different anaconda environments for different projects.</blockquote><p>A lot of developers, including myself, at times, give in to our laziness and create all our notebooks and scripts on the default base environment and call it a day. It would seem like a convenient and good idea….but not in the long run.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/0*riEVMXAOCvfO0jmU" /><figcaption>Image Courtesy: DEV Community</figcaption></figure><p>I have seen scenarios where when working on different projects — be it for college or personal projects etc., I used to encounter errors due to clashes in the compatibility of different packages. And when you go about trying to fix these issues by say, uninstalling some packages and installing another version of it based on compatibility of a package, it would lead to some other incompatibility in another project or some conflict with another package in the existing project. Then it all just becomes a game of uninstalling and installing. I literally spent about 1 hour on one of my recent personal projects (The post to that is embedded below) when I was trying to find a good HTML2PDF convertor and was having a hard time getting any of them to work and eventually landing up in a mess of incompatible packages. This is what happens when you get too comfortable with just creating notebooks on the fly in the base environment itself 😐.</p><p><a href="https://medium.com/@deepthi.sudharsan/tired-of-print-outs-with-hyperlink-references-aipr-might-be-your-solution-1d6650b434a1">Tired of print-outs with hyperlink references? AIPR might be your solution!</a></p><p>So, let me take you through the steps for creating new conda environments and switching between conda environments in the Anaconda Prompt/Terminal.</p><p><strong>Creating a new Anaconda Environment:</strong></p><pre>conda create --name new_env_name</pre><p><strong>Activating a conda Environment:</strong></p><pre>conda activate new_env_name</pre><p><strong>Switching between Environments:</strong></p><pre>To deactivate the current environment, type: conda deactivate <br>To activate another environment, use: conda activate different_env_name</pre><p>See, wasn’t it pretty easy? You are just one command away from juggling between environments and managing your projects and environments like a pro!</p><p>Additional Tip: Package installations in Python can be done using <em>pip install</em> or using <em>conda install</em>. It’s best to stick with one: either use ‘pip’ or ‘conda’ for installing packages. Mixing both can lead to conflicts and compatibility issues.</p><p>There is a dependency management tool similar to conda and pip for Python dependency management that is picking up popularity. Do you know which one it is? A separate post on it will be up soon!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=19d9f438b74d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[One Month Experience as a Microsoft Research Fellow]]></title>
            <link>https://medium.com/@deepthi.sudharsan/one-month-experience-as-a-microsoft-research-fellow-fa242f26d022?source=rss-c19a1f7cb39------2</link>
            <guid isPermaLink="false">https://medium.com/p/fa242f26d022</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[research]]></category>
            <category><![CDATA[experience]]></category>
            <category><![CDATA[learning]]></category>
            <dc:creator><![CDATA[Deepthi Sudharsan]]></dc:creator>
            <pubDate>Thu, 17 Aug 2023 13:37:14 GMT</pubDate>
            <atom:updated>2023-08-17T13:37:14.934Z</atom:updated>
            <content:encoded><![CDATA[<p>A month full of new learning and adventure</p><p>It has been one whole month since I started my position as a Microsoft Research Fellow. Transitioning from an intern at a Consulting firm to the research subsidiary of a product firm has obviously been a different experience. I have always been passionate about NLP, LLMs, and GenAI, but now with the whole world talking about these topics post-GPT 3.5, I have lots of people to share these interests with.</p><p><a href="https://www.linkedin.com/posts/deepthi-sudharsan_ai-mlsa-researchfellow-activity-7087088003902304256-P40L?utm_source=share&amp;utm_medium=member_desktop">Deepthi Sudharsan on LinkedIn: #ai #mlsa #researchfellow #msr #microsoftresearch #newbeginnings #newrole... | 36 comments</a></p><p>For the past 4 weeks, I have been working with LLMs, attending sessions where I get to learn more about LLMs and how to leverage them, and I have had the best technical interactions with a lot of talented folks on LLMs and AI in general as well. Every day I get the opportunity to learn something new and get to interact with many amazing people, especially the ones focused on my domain.</p><p>One of my key learnings so far is that with the pace at which advancements are being made in GenAI, keeping up to date with <em>some</em> latest technologies (it is impossible for anyone to keep up to date with all of them) is important because you never know how the course of action on any problem you are working on could currently change and could be benefitted by these latest technologies. And one way to be up to date with some of the latest trends is to interact with as many people as possible in your domain, network with them, and become aware of the latest trends. This coupled with other helpful resources like newsletters, LinkedIn content, etc., can come in very handy to become aware of how to guide yourself in terms of building your <strong><em>to-learn</em></strong> tech stack.</p><p>Until the next post,</p><p>Cheers!</p><p>P.S. If this is the first post of mine that you are viewing, do stick around if possible and join me on my journey exploring and doing what I love in the domain I am most passionate about! I hope to share my journey and learnings with everyone on this platform!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fa242f26d022" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tired of print-outs with hyperlink references? AIPR might be your solution!]]></title>
            <link>https://medium.com/@deepthi.sudharsan/tired-of-print-outs-with-hyperlink-references-aipr-might-be-your-solution-1d6650b434a1?source=rss-c19a1f7cb39------2</link>
            <guid isPermaLink="false">https://medium.com/p/1d6650b434a1</guid>
            <category><![CDATA[html]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[beautifulsoup]]></category>
            <category><![CDATA[fun]]></category>
            <category><![CDATA[streamlit]]></category>
            <dc:creator><![CDATA[Deepthi Sudharsan]]></dc:creator>
            <pubDate>Sat, 05 Aug 2023 11:20:49 GMT</pubDate>
            <atom:updated>2023-08-05T11:20:49.046Z</atom:updated>
            <content:encoded><![CDATA[<p>My fun 30-minute project for a very “lazy” problem?</p><p>Yesterday was a very good day in terms of getting random bursts of ideas — be it necessary or not. I wanted to read an article called “<a href="https://eugeneyan.com/writing/llm-patterns/">Patterns for Building LLM-based Systems &amp; Products</a>” and I wanted to take a copy of it on my office printer so that I could read it whenever I felt like going through it (I have found it very interesting so far and might write a post on it soon). But when I was flipping through the pages in a cubicle far away from my gadgets, I realized that in order to fully understand the context I would have to check out a few of the many hyperlinks in that document, and for that I would have to rely on a screen. Then, something happened.</p><p>I thought why not do a fun little project within as less time as possible to create a temporary solution to this problem? Though I did think long term like creating a browser extension etc., that would definitely not help with the time crunch I was looking at. So I decided to create a solution and deploy it on Streamlit.</p><p><a href="https://github.com/DeepthiSudharsan/AIPR">GitHub - DeepthiSudharsan/AIPR: AIPR - Aiding in Printing References</a></p><p>Presenting to you, my 30 long minutes of work — AIPR. A silly, but kind of useful project for when you are lazy. You can try it out by clicking the Streamlit embed below. (P.S. It is as rough of a draft as possible)</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fprintaid.streamlit.app%2F%3Fembed%3Dtrue&amp;display_name=Streamlit&amp;url=https%3A%2F%2Fprintaid.streamlit.app&amp;image=https%3A%2F%2Fstorage.googleapis.com%2Fs4a-prod-share-preview%2Fdefault%2Fst_app_screenshot_image%2F88c649c4-3b27-4c76-9fce-889a37fac227%2FHome_Page.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=streamlit" width="560" height="630" frameborder="0" scrolling="no"><a href="https://medium.com/media/b4757e0a0c499493262cb6caeebdeeb2/href">https://medium.com/media/b4757e0a0c499493262cb6caeebdeeb2/href</a></iframe><p>The workflow is as simple as it could get (but was frustrating due to some compatibility issues I was facing).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E5WFjiNb6fCeSlhksHIOCQ.png" /></figure><p>I was pretty much stuck in the PDFKit part and had tried different HTML2PDF libraries and each of them seemed to have some or the other issue and compatibility conflicts. I ended up figuring out the error with PDFKit and got it to work. Phew!</p><p>Let us now try to understand this simple code logic. First, let’s import all the necessary libraries required for both the overall workflow and the Streamlit deployment and some new Streamlit components. (For the complete code, check the Github Repo).</p><pre># Importing Necessary Libraries<br>import streamlit as st<br>from streamlit_tags import st_tags<br>from bs4 import BeautifulSoup<br>from urllib.request import Request, urlopen<br>from pypdf import PdfMerger<br>import pdfkit<br>import tempfile<br>from streamlit_card import card</pre><p>The next step is to get a URL as input from the user and parse and get all HTTPS URLs.</p><pre>hdr = {&#39;User-Agent&#39;: &#39;Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11&#39;,<br>       &#39;Accept&#39;: &#39;text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8&#39;,<br>       &#39;Accept-Charset&#39;: &#39;ISO-8859-1,utf-8;q=0.7,*;q=0.3&#39;,<br>       &#39;Accept-Encoding&#39;: &#39;none&#39;,<br>       &#39;Accept-Language&#39;: &#39;en-US,en;q=0.8&#39;,<br>       &#39;Connection&#39;: &#39;keep-alive&#39;}<br><br>    req = Request(site, headers=hdr)<br>    html_page = urlopen(req)<br>    soup = BeautifulSoup(html_page, &quot;html.parser&quot;)<br><br>    links = []<br>    for link in soup.findAll(&#39;a&#39;):<br>        url1 = link.get(&#39;href&#39;)<br>        if &quot;https://&quot; in url1:<br>            links.append(link.get(&#39;href&#39;))<br>    links.insert(0, site)</pre><p>We are using Urllib to fetch and open the URL and we are looking for all the hyperlink tags &lt;a&gt; on the webpage using BeautifulSoup and filter and append all the https:// hyperlinks in it to a list.</p><p>As a next step, we use PDFKit on these links to convert the HTML page to PDF and keep saving it in a temporary PDF file and merging it all together using PyPDF.</p><pre>merger = PdfMerger()<br>    for url in links:<br>        try:<br>            with tempfile.NamedTemporaryFile(suffix=&#39;.pdf&#39;, delete=False) as temp_pdf:<br>                st.toast(f&quot;Processing url {url}&quot;)<br>                pdfkit.from_url(https://medium.com/url,%20temp_pdf.name)<br>                merger.append(temp_pdf)<br>                st.toast(&quot;Converted to PDF...&quot;)<br>        except Exception as e:<br>            st.toast(f&quot;Cannot generate PDF for {url}&quot;)<br>    <br>    merged_pdf_path = tempfile.NamedTemporaryFile(suffix=&#39;.pdf&#39;, delete=False).name<br>    merger.write(merged_pdf_path)<br>    merger.close()</pre><p>And Voila! We have our AIPR in place!</p><p>I just wanted to share this fun, silly project of mine so do check out the Streamlit deployment and the code, and feel free to contribute to the repository!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1d6650b434a1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Who will win the Animals LLM race?]]></title>
            <link>https://medium.com/@deepthi.sudharsan/who-will-win-the-animals-llm-race-58948b964f6?source=rss-c19a1f7cb39------2</link>
            <guid isPermaLink="false">https://medium.com/p/58948b964f6</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[animals]]></category>
            <category><![CDATA[trends]]></category>
            <dc:creator><![CDATA[Deepthi Sudharsan]]></dc:creator>
            <pubDate>Thu, 03 Aug 2023 13:07:47 GMT</pubDate>
            <atom:updated>2023-08-03T13:07:47.450Z</atom:updated>
            <content:encoded><![CDATA[<p>There is a new LLM named after an animal in the building? Do we have enough animal names to keep up with the number of LLMs that are being released now?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LnYAyAJqbMgwBgGedJXJHw.png" /></figure><p>I remember it was just a few years back when I first discovered <a href="https://huggingface.co/">HuggingFace 🤗</a> while exploring NLP. Back then, someone had mentioned to me that the inspiration for the name of the transformer BERT came from <a href="https://en.wikipedia.org/wiki/Bert_(Sesame_Street)">Sesame Street</a> and that seemed impressive and fun to me as I had grown up watching it (the Hindi Version — Galli Galli Sim Sim) and now I was doing something technical involving that name.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UFsrBEA-fP5ezTRn-9lgEA.png" /></figure><p>Forward to [maybe] two years down the lane and with the amount of interest and work being done in the areas of LLMs and GenAI, we see a lot of unique and creative names coming up. Not so creative as its application or use case since we seem to be inclined towards naming them after animals. I don’t know if the pressure to keep up with so many advancements in LLMs is more confusing or wondering how people are able to find use cases that when somehow abbreviated, is an animal name.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/966/1*r7bkBG4p9gniuhxDLeetLw.png" /></figure><p>Joking aside, one of my dream colleges, <a href="https://www.berkeley.edu/">UC Berkley</a>, has a <a href="https://gorilla.cs.berkeley.edu/">“Gorilla” LLM</a> (finetuned on LLaMA Models) that is picking up speed with the LLM groups. It teaches LLMs APIs and generates the API code for your task. I am yet to go through the work completely but I found it incredibly interesting to find yet another animal-name-inspired LLM and felt like penning it right away.</p><p>Have you read the paper? Feel free to add your views, thoughts, and constructive suggestions!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=58948b964f6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Rekindling the flame — Hello Again!]]></title>
            <link>https://medium.com/@deepthi.sudharsan/rekindling-the-flame-hello-again-7488950c762a?source=rss-c19a1f7cb39------2</link>
            <guid isPermaLink="false">https://medium.com/p/7488950c762a</guid>
            <category><![CDATA[poetry]]></category>
            <category><![CDATA[new-beginnings]]></category>
            <category><![CDATA[updates]]></category>
            <category><![CDATA[life]]></category>
            <dc:creator><![CDATA[Deepthi Sudharsan]]></dc:creator>
            <pubDate>Sat, 29 Jul 2023 03:24:03 GMT</pubDate>
            <atom:updated>2023-07-29T03:26:32.982Z</atom:updated>
            <content:encoded><![CDATA[<h3>Rekindling the flame — Hello Again!</h3><p>A long-awaited reconnect with blogging</p><p>I got inspired by my fellow research fellows! Shoutout to <a href="https://medium.com/u/7bda5ca67a02">Palak</a> if you are seeing this! I tried writing a poem, but to no avail but with ChatGPT of course, success prevailed!</p><p>Below is written by <a href="https://chat.openai.com/"><strong>ChatGPT</strong></a><strong> </strong>(Read till the end, if you want to read one of my failed original versions that GPT was asked to modify!)</p><p>In the distance, I’ve been away so long,<br>A year and a half, like ages passed by,<br>But fear not, for I’ve journeyed along,<br>Engaging in passions, reaching for the sky.</p><p>Within these weeks, life’s currents flowed,<br>I embraced a new MLSA stride,<br>Consulting’s realm, a Big4’s abode,<br>DL/CV POCs created with pride.</p><p>As GenAI rose, I reached higher ground,<br>B.tech AI, a graduation mark,<br>Microsoft’s call, a research fellow crowned,<br>Delving into NLP, where dreams embark.</p><p>So here I stand, doing what I adore,<br>Amidst the wonders of a world that grows,<br>With every step, I learn and explore,<br>In this dance with knowledge, my spirit glows.</p><h3><strong>Here is the original self-written version:</strong></h3><p>It has been long since I’ve been gone,<br>A year and a half feels like a lifetime.<br>But don’t you worry about me,<br>I was doing just fine.<br>I spent my time doing what I like,<br>While the world just kept spinning by.</p><p>A lot has happened in these long weeks,<br>I became an MLSA when placements mattered.<br>Got into consulting and did ML,<br>Coded PoCs using CV and DL.<br>Worked on GenAI when it took the world by storm,<br>Just like when Barbie came along.</p><p>Bid farewell to my friends at Big4,<br>Switching to a research fellow for my next move.<br>Continuing to learn and doing what I love,<br>While it has only been a few weeks at Microsoft Research.</p><p>So, join me as I delve right back,<br>While I write about my learning, grab a snack.<br>Until my next post, so long,<br>Hopefully not too long.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7488950c762a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring NLP — Kickstarting NLP (Step #4)]]></title>
            <link>https://pub.towardsai.net/exploring-nlp-kickstarting-nlp-step-4-98ee3c418aec?source=rss-c19a1f7cb39------2</link>
            <guid isPermaLink="false">https://medium.com/p/98ee3c418aec</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[pos-tagging]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Deepthi Sudharsan]]></dc:creator>
            <pubDate>Tue, 22 Mar 2022 16:34:10 GMT</pubDate>
            <atom:updated>2023-07-29T03:47:40.326Z</atom:updated>
            <content:encoded><![CDATA[<h3>Exploring NLP — Kickstarting NLP (Step #4)</h3><p>If you are new to my “Exploring NLP” series, please check my introductory article <a href="https://medium.com/@deepthi.sudharsan/exploring-and-mastering-nlp-a-journey-through-the-depths-afca5ca9640d">here</a>.</p><p><a href="https://medium.com/@deepthi.sudharsan/exploring-and-mastering-nlp-a-journey-through-the-depths-afca5ca9640d">Exploring and Mastering NLP — A journey through the depths</a></p><p>Welcome back! Now continuing the series, I will be sharing some of my notes on (mainly) POS tagging. I would especially like to thank Mr. Sachin Kumar S sir from CEN, Amrita Coimbatore, for taking this course for me this semester and some of the information and images gathered here are from the resources and materials provided or created by him.</p><h3><strong>POS (Parts Of Speech) Tagging:</strong></h3><p>It is the lowest level of Syntactical Analysis. It is the task of assigning the Part of Speech tags to words.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*69fTMBl9KwfwKyuaiwT2BQ.png" /></figure><p>This kind of sequence labeling can also be seen in labeling genetic sequences in genome analysis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*RMzQZIRyH5Ghfn7lYzpzsw.png" /></figure><p>Given a sentence, say, “It looks like chocolate”, the POS tags will be as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/880/1*xnk4PORue9eG4BRDsClrNw.png" /></figure><p>PRP is personal pronoun , VB stands for verb, IN stands for preposition or subordinating conjunction and N stands for Noun, etc.</p><p><strong>Open and Closed Classes:</strong></p><p>Open classes are classes for which new words are frequently added to it. Example: nouns, verbs, etc. Whereas in closed classes, they have a pre-defined set of words that come under those classes like, for example, articles.</p><p>In POS tagging, there is a problem of ambiguity as words often have more than one POS tag. For example, “like” can be assigned both IN and VB tags.</p><p>Tag types can be of two types: Coarse-grained and Fine-grained.</p><ol><li>Coarse-grained tag types include Nouns, Verbs, Adjectives, etc.</li></ol><p>2. Fine-grained, include noun-proper-singular, verb-past, adjective-simple, etc.</p><p><strong>Ways to Find POS Tags:</strong></p><ol><li>Pointwise prediction — predict each token tag individually using a classifier</li><li>Generative sequence model – using HMM (Hidden Markov Model)</li><li>Discriminative sequence models - CRF (Conditional Random Field), Structured prediction, etc</li></ol><h3>HMM for POS Tagging:</h3><p>Observable states - Words in the given word sequence/sentence</p><p>Hidden/Unobserved states - POS tags</p><p><em>To calculate the probability associated with a particular sequence of tags, In HMM, transition and emission probabilities are used.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/466/1*XFbcC0YZPBdfu3x2iBf7Rg.png" /></figure><ol><li>A transition happens from one hidden state to another or hidden states have a transition from one state to the next hidden state. A certain probability is associated while moving from one state to another which is known as transition probability.</li><li>Emission corresponds to generating a token from a hidden state. A certain probability is associated with a given hidden state to generate a token</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/387/1*td217UHiofcwsaW0a9y4-w.png" /></figure><p><strong>Assumptions made in HMM :</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*eLEEtMWcczQd6UCuyepg9A.png" /></figure><h3>Brute — Force :</h3><p>Let&#39;s tag the sentence with random POS tags, say, “<em>It/PRP looks/N like/PRP chocolate/VB”. </em>Now let’s calculate the probability of the sequence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*JFCy6PBAs_dG6nsJy6GfTg.png" /></figure><p>Let&#39;s tag the sentence with the correct POS tags, say, “<em>It/PRP looks/VB like/IN chocolate/N”. </em>Now let’s calculate the probability of the sequence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/1*IfGHLWHRPIm70eHucs_bOA.png" /></figure><h3>Viterbi Algorithm :</h3><p>The Viterbi algorithm finds is a dynamic programming method to find the most probable sequence of tags or hidden states. It is kind of optimizing HMM to find the most probable hidden states.</p><p>In the given sentence, “It looks like Chocolate”, the token <em>like</em> has two POS tags, VB, and IN. The path which has the lowest probability is removed and the backward pass gives the path or the sequence.</p><p>In this example, the probability of the sequence &lt;S&gt; PRP VB VB N &lt;E&gt; becomes 0.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/886/1*dbudSMqDmDCJOibcXNq6kA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*0VgrMCl5-nwysEHZG_BOtA.png" /><figcaption>Here is a snippet from the code that I had written for Viterbi. Will post the code here soon.</figcaption></figure><h3>References:</h3><ol><li><a href="https://universaldependencies.org/docsv1/en/pos/all.html#al-en-pos/ADJ">https://universaldependencies.org/docsv1/en/pos/all.html#al-en-pos/ADJ</a></li><li><a href="https://universaldependencies.org/u/pos/">https://universaldependencies.org/u/pos/</a></li><li><a href="https://youtu.be/IqXdjdOgXPM">https://youtu.be/IqXdjdOgXPM</a></li><li><a href="https://www.freecodecamp.org/news/an-introduction-to-part-of-speech-tagging-and-the-hidden-markov-model-953d45338f24/">https://www.freecodecamp.org/news/an-introduction-to-part-of-speech-tagging-and-the-hidden-markov-model-953d45338f24/</a></li><li><a href="https://www.mygreatlearning.com/blog/pos-tagging/">https://www.mygreatlearning.com/blog/pos-tagging/</a></li><li>Speech and Language, Jurafsky</li></ol><h3>Previous Parts from this series:</h3><p>Part #1 : <a href="https://medium.com/@deepthi.sudharsan/exploring-nlp-kickstarting-nlp-step-1-e4ad0029694f">https://medium.com/@deepthi.sudharsan/exploring-nlp-kickstarting-nlp-step-1-e4ad0029694f</a></p><p>Part #2 : <a href="https://medium.com/@deepthi.sudharsan/exploring-nlp-kickstarting-nlp-step-2-157a6c0b308b">https://medium.com/@deepthi.sudharsan/exploring-nlp-kickstarting-nlp-step-2-157a6c0b308b</a></p><p>Part #3 : <a href="https://medium.com/@deepthi.sudharsan/exploring-nlp-kickstarting-nlp-step-3-e6ea6963679d">https://medium.com/@deepthi.sudharsan/exploring-nlp-kickstarting-nlp-step-3-e6ea6963679d</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=98ee3c418aec" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/exploring-nlp-kickstarting-nlp-step-4-98ee3c418aec">Exploring NLP — Kickstarting NLP (Step #4)</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring NLP — Kickstarting NLP (Step #3)]]></title>
            <link>https://pub.towardsai.net/exploring-nlp-kickstarting-nlp-step-3-e6ea6963679d?source=rss-c19a1f7cb39------2</link>
            <guid isPermaLink="false">https://medium.com/p/e6ea6963679d</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[learning]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Deepthi Sudharsan]]></dc:creator>
            <pubDate>Tue, 22 Mar 2022 14:49:06 GMT</pubDate>
            <atom:updated>2023-07-29T03:47:01.083Z</atom:updated>
            <content:encoded><![CDATA[<h3>Exploring NLP — Kickstarting NLP (Step #3)</h3><p>If you are new to my “Exploring NLP” series, please check my introductory article <a href="https://medium.com/@deepthi.sudharsan/exploring-and-mastering-nlp-a-journey-through-the-depths-afca5ca9640d">here</a>.</p><p><a href="https://medium.com/@deepthi.sudharsan/exploring-and-mastering-nlp-a-journey-through-the-depths-afca5ca9640d">Exploring and Mastering NLP — A journey through the depths</a></p><p>Here are a few concepts that I went through over the week, especially on word embeddings. I did do some hands-on, which I shall share as a part of the series soon! I would like to thank Mr. Sachin Kumar S sir from CEN, Amrita Coimbatore, for taking this course for me this semester and some of the information and images gathered here are from the resources and materials provided or created by him.</p><h3><strong>N-grams :</strong></h3><p>N — gram is the simplest Language Model that assigns probabilities to a sentence or sequence of tokens. (Models that assign a probability value to the sequence of tokens are called Language models)</p><p>An n-gram is a sequence of n-tokens. The intuition behind n-gram is that instead of computing the probability of a word given the entire history, approximate the history by using a few words. The assumption that the probability of the next word depends only on the previous word is known as the Markov assumption. N-gram models look into n-1 words in the past to predict the probability of a word.</p><h3><strong>Word Embeddings :</strong></h3><p>The transformation from a word to its n-dimensional vector representation (of words that carry semantic meaning) for further NLP tasks is called Word embedding. Example: Word2Vec</p><p>There are more than 10 million tokens in the English language corpus. Word embeddings encode each word to a vector that will become a point in a word vector space</p><p>The intuition is that such word vector space will encode all the semantics of the language or words that have a similar context will get mapped to proximate points in a vector space.</p><h3><strong>One Hot Encoding :</strong></h3><p>Representing a word in a vector format such that the length of the vector is equal to the vocabulary size and the element at the index of that particular word will alone be 1 and the rest will be 0.</p><p>Suppose we have a sentence “I Love NLP”. The OHE for the sentence looks like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/909/1*VrxqJNxE5Fv0UfAMZRNvVQ.png" /></figure><p>This type of representation does not give an intuition on semantics between words or how related words are (contextual closeness).</p><p>I think the disadvantage of OHE also arises from the fact that for very large data, the OHE vector is going to be extremely large and sparse (with lots of zeros to be stored). Context is also not captured by the OHE representation.</p><h3><strong>SVD-based approach :</strong></h3><p>For a huge corpus, we perform SVD on a co-occurrence matrix, and after decomposition, we get the vectors U (Eigen vector-matrix), S (Eigen Values matrix), and V.T (V transpose - inverse of the Eigen vector-matrix).</p><p>Below is the co-occurrence matrix for the sentences “Roses are red” and “Sky is Blue”. The co-occurrence matrix indicates how many times the row<strong> </strong>word is surrounded by the column<strong> </strong>word in the sentences<strong>.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/430/1*upktWbIJA97X27uPg-v_SQ.png" /><figcaption>Source: <a href="https://stackoverflow.com/questions/24073030/what-are-co-occurence-matrixes-and-how-are-they-used-in-nlp">https://stackoverflow.com/questions/24073030/what-are-co-occurence-matrixes-and-how-are-they-used-in-nlp</a></figcaption></figure><p>The vectors of matrix U which is the Eigen vector-matrix or the left singular vectors, will be considered as the vector representations for the words.</p><p>I personally found this article interesting to read more about this approach: <a href="https://medium.com/analytics-vidhya/co-occurrence-matrix-singular-value-decomposition-svd-31b3d3deb305">https://medium.com/analytics-vidhya/co-occurrence-matrix-singular-value-decomposition-svd-31b3d3deb305</a></p><p>So from the above link, I learned the advantages and disadvantages of the method.</p><ol><li>We know the relationship between words in the co-occurrence matrix.</li></ol><p>2. Can be computed once and referred to multiple times once computed but, this comes with a disadvantage when new words are frequently added to the corpus and the dimensions of the matrix continuously change.</p><p>3. Generally, it takes a lot of memory, especially when there are a lot of words that do not co-occur, and most of the elements are zero.</p><h3><strong>Term Document Matrix :</strong></h3><p>As the name kind of suggests, we create a matrix representation of dimensions Vocabulary size x Number of documents. For a word from the vocabulary, we are representing the number of times it occurs in each document.</p><p>It does not capture, the order/position and semantics of the words.</p><h3><strong>Distributional nature of word vector :</strong></h3><p>It’s based on the hypothesis that the meaning of the word can be inferred on the basis of the context it appears in.</p><p>They aren’t good at representing rare words that occur very few times in the corpus. It suffers from the problem of ambiguity and fails to handle morphology (evaluate and evaluate are considered different).</p><p>Word Embeddings can be broadly classified into two categories — Frequency (Example: Count Vector, TF-IDF Vector, Co-Occurrence Matrix ) and Prediction (Continuous Bag of Words(CBOW), Skip-Gram) based Embedding.</p><h3><strong>Word2Vec :</strong></h3><p>Word2Vec is a shallow, two-layer neural network, trained to reconstruct linguistic contexts of words. It has two architectures: Continuous Bag-of-Words and Skip-gram.</p><h3><strong>Continuous Bag of Words :</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*UmdemcUyB5neyvZSg9-cUA.jpeg" /><figcaption>Sources: <a href="https://www.kdnuggets.com/2016/05/amazing-power-word-vectors.html/2">https://www.kdnuggets.com/2016/05/amazing-power-word-vectors.html/2</a></figcaption></figure><p>Objective Function :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*hTPs7GnzeaaLRx8MJUv0Bw.png" /><figcaption><a href="https://slideplayer.com/slide/13396296/">https://slideplayer.com/slide/13396296/</a></figcaption></figure><p>To create word embeddings, n-words in the past and future are used.</p><p>Increasing the window size does not help, it adds more noise.</p><p>Here, context and order of words do not matter, so we are just taking the bag of words, and hence “BOW” is part of its name. It also gives the semantic relationship of the words.</p><p>CBOW performs better than Skip grams because they are able to make better use of input data to better capture the word vectors efficiently.</p><h3><strong>Skip Grams :</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*gyZVDoLST6pECHbU_BC3hw.jpeg" /><figcaption>Source: <a href="https://deepdatascience.wordpress.com/2017/04/22/word2vec-skip-gram-model/">https://deepdatascience.wordpress.com/2017/04/22/word2vec-skip-gram-model/</a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/806/1*DsN-oe5hnGbh2Qk0vQ5HYg.png" /><figcaption>The “negative samples” are selected using a “unigram distribution”, where more frequent words are more likely to be selected as negative samples. Image Source: <a href="https://cs224d.stanford.edu/lectures/CS224d-Lecture3.pdf">https://cs224d.stanford.edu/lectures/CS224d-Lecture3.pdf</a></figcaption></figure><p>Here, the target word is input while context words are output, unlike CBOW.</p><p>It is an unsupervised learning approach, so it can learn from any raw text provided and it requires less memory. Setting the number of neurons in the hidden layer and getting the context position is difficult and the time required for training is high.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/688/1*NY6Pc0odg2ogMSDT7X3X2g.png" /></figure><h3><strong>GLoVE (Global Vectors for Word Representations):</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*QJnoVwEm44RqNYoa5PZVsA.png" /><figcaption><a href="https://trailhead.salesforce.com/en/content/learn/modules/glove-and-word-vectors-for-sentiment-analysis/use-glove-for-natural-language-processing">https://trailhead.salesforce.com/en/content/learn/modules/glove-and-word-vectors-for-sentiment-analysis/use-glove-for-natural-language-processing</a></figcaption></figure><p>•Glove model combines word-word co-occurrence counts and window-based approaches</p><p>•Very large co-occurrence matrix needs to be computed at the beginning, denoted by ‘<em>P</em>’</p><p>•For each pair of words that can co-occur, minimize the distance between the inner product of two words and the log count of two words</p><p>•Squared distance value is weighted using <em>f(Pij), </em>which will assign lower weights to the pair having a high count</p><p>•Here individual word occurrences are considered</p><h3><strong>FastText :</strong></h3><p>Used for efficient text classification and representation, FastText represents each word as a bag of character n-grams. Since the embeddings are learned using n-grams, it will have information about the suffixes and prefixes.</p><p>FastText library for word embedding and classification allows creating of unsupervised learning or supervised learning algorithms for obtaining word vector representations.</p><p>FastText can obtain vectors, even for out-of-vocabulary (OOV) words, by summing up vectors for its component character n-grams, provided at least one of the character n-grams was present in the training data.</p><h3><strong>References :</strong></h3><ol><li>‘Speech &amp; language processing’ , Daniel Jurafsky, James H Martin, preparation [cited 2020 June 1] (Available from: <a href="https://web./">https://web.</a> Stanford. edu/~ jurafsky/slp3 (2018))</li><li><a href="https://en.wikipedia.org/wiki/Word_embedding">https://en.wikipedia.org/wiki/Word_embedding</a></li><li><a href="https://towardsdatascience.com/https-medium-com-tanaygahlot-moving-beyond-the-distributional-model-for-word-representation-b0823f1769f8">https://towardsdatascience.com/https-medium-com-tanaygahlot-moving-beyond-the-distributional-model-for-word-representation-b0823f1769f8</a></li><li><a href="https://medium.com/analytics-vidhya/co-occurrence-matrix-singular-value-decomposition-svd-31b3d3deb305">https://medium.com/analytics-vidhya/co-occurrence-matrix-singular-value-decomposition-svd-31b3d3deb305</a></li><li><a href="https://deepdatascience.wordpress.com/2017/04/25/word2vec-continous-bag-of-words/">https://deepdatascience.wordpress.com/2017/04/25/word2vec-continous-bag-of-words/</a></li><li><a href="https://towardsdatascience.com/skip-gram-nlp-context-words-prediction-algorithm-5bbf34f84e0c">https://towardsdatascience.com/skip-gram-nlp-context-words-prediction-algorithm-5bbf34f84e0c</a></li><li><a href="https://www.kdnuggets.com/2016/05/amazing-power-word-vectors.html/2">https://www.kdnuggets.com/2016/05/amazing-power-word-vectors.html/2</a></li></ol><h3>Previous Parts from this series:</h3><p>Part #1 : <a href="https://medium.com/@deepthi.sudharsan/exploring-nlp-kickstarting-nlp-step-1-e4ad0029694f">https://medium.com/@deepthi.sudharsan/exploring-nlp-kickstarting-nlp-step-1-e4ad0029694f</a></p><p>Part #2 : <a href="https://medium.com/@deepthi.sudharsan/exploring-nlp-kickstarting-nlp-step-2-157a6c0b308b">https://medium.com/@deepthi.sudharsan/exploring-nlp-kickstarting-nlp-step-2-157a6c0b308b</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e6ea6963679d" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/exploring-nlp-kickstarting-nlp-step-3-e6ea6963679d">Exploring NLP — Kickstarting NLP (Step #3)</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>