<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Aryaman Sriram on Medium]]></title>
        <description><![CDATA[Stories by Aryaman Sriram on Medium]]></description>
        <link>https://medium.com/@aryamansriram?source=rss-79cfba7bd5c9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*C534zNog9dOalfUFVs1seA.jpeg</url>
            <title>Stories by Aryaman Sriram on Medium</title>
            <link>https://medium.com/@aryamansriram?source=rss-79cfba7bd5c9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 04:17:23 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@aryamansriram/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[TextGrad and the Art of Automated Prompt Optimization]]></title>
            <link>https://medium.com/@aryamansriram/textgrad-and-the-art-of-automated-prompt-optimization-8704b10f0c51?source=rss-79cfba7bd5c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/8704b10f0c51</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[prompt-optimization]]></category>
            <dc:creator><![CDATA[Aryaman Sriram]]></dc:creator>
            <pubDate>Wed, 08 Apr 2026 14:56:18 GMT</pubDate>
            <atom:updated>2026-04-08T14:56:18.546Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*efE69RtqSS3SqRQmR2yAog.jpeg" /></figure><p>AI is moving at a blazing pace. Every time you feel comfortably on track with the latest developments, there is a new idea making the rounds that everyone seems to be talking about.</p><p>It is crazy to think that not more than a year or two ago, prompt engineering was the hottest buzzword being thrown around. For those who are unfamiliar, prompt engineering refers to the art of crafting instructions for a large language model — without any need to actively train or fine-tune it — such that it performs better on your downstream task. At a time when models were not particularly capable, this skill was highly sought after as a way to extract maximum value out of what were, frankly, not-so-great models.</p><p>For anyone who has done prompt engineering, they know it is a painful and messy process. You run the prompt on a dataset, analyze results, make small changes, then look at the results again to make sure you have not broken something else in the process — which you almost always have. Naturally, practitioners in the industry started thinking seriously about optimizing and automating this workflow, which gave birth to a number of interesting techniques. This blog series aims to explore those techniques and help you integrate them into your own workflow.</p><h3>TextGrad</h3><p>TextGrad was one of the early entrants into the space of automated prompt optimization. The core idea is elegant: model the prompt optimization process as a PyTorch training loop. The authors drew deliberate parallels between training a deep learning model and optimizing a prompt.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_3NQx6lk7CfF6Z1qsSHLew.png" /></figure><p>The ethos of deep learning remains entirely intact here — it is simply translated from the domain of linear algebra into the domain of natural language.</p><h3>Key Components of TextGrad</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HQrNZyyXIv15dKEzWQW0YQ.png" /></figure><p><strong>Generator Model:</strong> This is the model whose prompt you are trying to optimize — typically the model that will be used in production, taking the optimized prompt as input.</p><p><strong>Critique Model:</strong> Generally a powerful reasoning model. Its job is to critically analyze the generator’s outputs, produce textual gradients (i.e., pointed feedback), and guide the optimization. This model does the heavy lifting.</p><p><strong>Optimizer:</strong> At its core, this is simply a prompt that instructs the system on how to incorporate the critique’s feedback into the existing prompt. Drawing on the deep learning analogy further, two additional concepts are worth noting:</p><ul><li><strong>Constraints:</strong> Special instructions passed to the optimizer to steer it in a particular direction. For example: <em>“The resulting prompt must not exceed 200 characters”</em> or <em>“The resulting prompt should only allow the outputs ‘good’, ‘partially_good’, or ‘bad’.”</em> Constraints are especially useful when you need to preserve the structure or format of the prompt across optimization steps.</li><li><strong>Momentum:</strong> Just as momentum in gradient descent helps the optimizer avoid getting stuck in local minima by carrying a moving average of past gradients, TextGrad supports an analogous concept. Here, the optimizer retains a summary of feedback from previous iterations and folds it into subsequent optimization steps. This prevents the prompt from oscillating and helps the optimization converge more smoothly over multiple epochs.</li></ul><p>These parameters can have a significant impact on how effectively your prompt gets optimized.</p><h3>Results from the Paper</h3><p>The TextGrad paper (Yuksekgonul et al., 2024) evaluated the framework across a diverse set of domains, demonstrating strong performance well beyond simple prompt tuning.</p><p><strong>Code Optimization (LeetCode Hard):</strong> TextGrad improved the completion rate on LeetCode Hard problems from 23% with zero-shot GPT-4 to 36%, surpassing the state-of-the-art Reflexion method which had achieved 31%. <a href="https://www.emergentmind.com/papers/2406.07496">Emergent Mind</a> Notably, Reflexion relied on in-context demonstrations while TextGrad operated zero-shot.</p><p><strong>PhD-Level Question Answering (GPQA):</strong> TextGrad applied three iterations of solution updates followed by majority voting, achieving 55% on GPQA — the best-known result at the time of publication. <a href="https://gonzoml.substack.com/p/textgrad-automatic-differentiation">Substack</a> It also outperformed chain-of-thought prompting on MMLU subsets covering physics and machine learning.</p><p><strong>Prompt Optimization (Big Bench Hard &amp; GSM8k):</strong> The authors optimized prompts for the lighter GPT-3.5-turbo model using feedback from the more powerful GPT-4o, and TextGrad outperformed both zero-shot chain-of-thought and DSPy on these benchmarks. <a href="https://gonzoml.substack.com/p/textgrad-automatic-differentiation">Substack</a></p><p><strong>Molecule Optimization:</strong> TextGrad was applied to 58 protein targets from the DOCKSTRING benchmark. Across 29 targets and three initial fragments, it successfully designed molecules with comparable or better binding affinity (Vina scores) and greater druglikeness (QED scores) than clinically approved molecules targeting the same proteins. <a href="https://arxiv.org/pdf/2406.07496">arXiv</a></p><p><strong>Radiotherapy Treatment Planning:</strong> TextGrad-generated radiotherapy plans showed better dose coverage for the planning target volume (PTV) and lower incidental doses on organs at risk (OARs) compared to clinically optimized plans — indicating a more favorable treatment balance. <a href="https://www.emergentmind.com/papers/2406.07496">Emergent Mind</a></p><p>Across these domains, TextGrad achieved state-of-the-art results in code optimization and PhD-level question answering, demonstrated effective prompt optimization, and provided compelling proof-of-concept results in scientific applications such as drug discovery and treatment planning. <a href="https://arxiv.org/html/2406.07496v1">arXiv</a></p><h3>Personal Learnings from Using TextGrad</h3><p>TextGrad is a simple and effective approach to prompt optimization. Its resemblance to existing frameworks like PyTorch makes it easy to pick up, and the conceptual mapping to deep learning means that many of your existing intuitions transfer directly. A few things I learned from using it in practice:</p><p><strong>Compute costs add up fast.</strong> Since the technique relies on a heavy critique model — using a weaker one does not work well — it is slow and resource-intensive. Running it locally is possible but not advisable; the cloud is your friend here.</p><p><strong>~100 samples is a sweet spot.</strong> Data sizes around 100 examples tend to work well, and you can meaningfully improve your prompt in as few as 3 epochs.</p><p><strong>Batch size matters more than you’d expect.</strong> Similar to mini-batch gradient descent, batch size has a real effect on optimization quality. I found that very small batches (size 5) do not give the optimizer enough signal to update the prompt meaningfully across the full dataset, while very large batches (80–100) can overwhelm the context window. Experiment to find what works for your use case.</p><p><strong>Label quality is everything.</strong> Garbage in, garbage out. If your dataset has noisy or incorrect labels, the optimization process will be steered in the wrong direction, and no amount of tuning will fix that.</p><p><strong>Iterate on your loss function too.</strong> The loss function is not a set-it-and-forget-it component. You want to make sure the criticisms it produces actually align with your objective. A poorly designed loss function will generate misleading feedback and corrupt the entire optimization trajectory. Treat it as a first-class citizen of the system.</p><h3>References</h3><ul><li>Textgrad Paper: <a href="https://arxiv.org/abs/2406.07496">https://arxiv.org/abs/2406.07496</a></li><li>Textgrad Documentation: <a href="https://textgrad.com/">https://textgrad.com/</a></li><li>Textgrad Github Repository: <a href="https://github.com/zou-group/textgrad">https://github.com/zou-group/textgrad</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8704b10f0c51" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Paper Review: Implicit Neural Representations
with Periodic Activation Functions]]></title>
            <link>https://medium.com/@aryamansriram/paper-review-implicit-neural-representations-with-periodic-activation-functions-c1e00179cb8e?source=rss-79cfba7bd5c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/c1e00179cb8e</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[signal-processing]]></category>
            <dc:creator><![CDATA[Aryaman Sriram]]></dc:creator>
            <pubDate>Sun, 04 Feb 2024 08:03:04 GMT</pubDate>
            <atom:updated>2024-02-04T08:03:04.527Z</atom:updated>
            <content:encoded><![CDATA[<p>How do we represent signals? When it comes to audio, its simply a sound wave with time on the x-axis and amplitude on the y-axis. When it comes to an image, its a 2-D grid with the corresponding pixel value at a particular coordinate value. A 3-D object is represented using a point cloud.</p><p>Now what if I told you this could be done differently. Consider representing a grayscale image. What if we had a machine that would take as input, the co-ordinates of the grid, the size of the image and it would output the image pixel value at that location. By feeding in all the co-ordinates of the grid we would be able to get back the entire image.</p><p>There would be a number of advantages to representing an image this way:</p><ul><li><strong>Unrestricted resolution: </strong>The image would now not be restricted to a 2-D grid with fixed co-ordinates. One would be able to get the signal values at much finer resolutions by simply changing the input values.</li><li><strong>Memory: </strong>The memory occupied by the image would only depend on the complexity of the signal which would in turn make the model complex. It would not depend on the resolution.</li></ul><p>This is essentially the idea that the paper suggests. It describes the formulation of a neural network that can take the co-ordinates of a signal and output the corresponding signal value. Let’s take a look at how they do this.</p><h3>Simple Solution: ReLU MLP</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3anEwehJ13yBlbcjRA8n_w.png" /></figure><p>The simplest solution one can think of is a ReLU Multilayer Perceptron(MLP). Unfortunately like most of the simple solutions, this does not work very well because of the following reasons:</p><ul><li><strong>ReLU MLPs are not shift-invariant: </strong>Because of the way a ReLU function works (it is 0 until the input is 0 and linear for every positive input), it is tough to get the same output from a ReLU for different co-ordinates. This can be a problem if the signal has the same value at different locations.</li><li><strong>ReLU MLPs are bad at modeling derivatives: </strong>The derivative of a ReLU is a constant and the derivative of a constant is 0. Hence these MLPs find it hard to model signals which carry information in their derivatives.</li></ul><h3>Enter SIRENs</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x0jt_5yC0bmrOf3JDWXoBg.png" /></figure><p>SIRENs replace the ReLU function in the MLP to a sine function. This solves the above encountered problems:</p><ul><li><strong>SINE functions are periodic: </strong>Sine functions are periodic with the period π. Hence if two different co-ordinates are input to a SIREN, there is a good chance that they may end up with the same output activation.</li><li><strong>The derivative of a SIREN is a SIREN: </strong>The derivative of a sine function is a cos function which is essentially a sine function shifted by 90º. Hence it can model derivatives well.</li></ul><h3>Applications</h3><p>Hey that’s cool and all but where do I use these SIRENs.</p><ol><li><strong>Poisson Image Reconstruction and Editing</strong></li></ol><p>One could train the SIREN just conditioned on the gradient of the image and use it to reconstruct the image only using the gradients. The loss function would look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2vEK6omxZye-unJG-BwmIQ.png" /></figure><p>Where ▽⏀ is the gradient of the output from the siren and ▽f(x) is the gradient of the ground truth of the image. The authors trained a 5 layer SIREN with the loss function above and an ADAM optimizer with a learning rate of 1/10⁴ for 10,000 iterations. Some of their results are shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RmSsQZVu_FHjW8d76kuoOg.png" /></figure><p>Another example of an application is <strong>Poisson Image Editing. </strong>Take the image of the bear and the pyramids above. If we were to simply fuse the two images together (by taking a mean) we may not get a good composite image (There are some waves above the bear that might show up on the pyramid which is well not very tasteful!). What we could do instead is fuse the gradients of the two images. That would remove any such details that could mess with the composition. We could then use this composite gradient and fit a SIREN to it to reconstruct our clean composite image.</p><p>2. <strong>Image Inpainting</strong></p><p>The authors in this application talk about learning priors over a SIREN network. They said what if we could learn the parameters of a SIREN using just a few samples from an image and then use the SIREN to reconstruct the entire image. That’s what happens here:</p><p>In this particular use case a few samples of the image are taken and encoded using a latent encoder. The encoding then passes through a ReLU MLP and is mapped to the weights of a SIREN. The SIREN can then be used to reconstruct the entire image. As shown below the odd numbered rows (top to bottom) represent the samples and the even numbered rows represent the reconstructions. The authors were able to do this just by using a 5 layer SIREN and running it for 5000 iterations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gmwzPaXL9ijpNGEK0G7pdg.png" /></figure><p>3. <strong>3-D Reconstruction</strong></p><p>The paper also describes training a SIREN to reconstruct a 3-D object like a pillar or a room. They do this by fitting the following loss function over a SIREN. The SIREN in this case predicts something called as an <a href="https://en.wikipedia.org/wiki/Signed_distance_function">SDF</a> or a <strong>signed distance function. </strong>In layman’s terms, signed distance function is defined for points on or around a particular surface. Any point on the surface of the object has an SDF of 0. and away from the surface has a negative value.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cJtPBjhb6535MTxnqE_uQA.png" /></figure><p>In the above loss function:</p><ul><li>The first term constrains the gradient to 0 for all points</li><li>The second term constrains output value to 0 for surface points and the dot product of the gradient of the output and the normal vector to the surface to 1</li><li>The last term penalizes any non-surface points who output a value of 0.</li></ul><h3>SIREN Initialization</h3><p>While SIRENs are an interesting idea, they don’t work well unless initialized a certain way. The authors propose the following initializations:</p><p>Weights are sampled from the following distribution:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/170/1*Fv13xOGm3-Zeq24gRoTlvQ.png" /></figure><p>Where n is the number of input neurons in the layer.</p><p>The first layer of the SIREN is initialized with the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/129/1*nQd0ZtBChNeqYBwJz3LIbA.png" /></figure><p>Where w0 = 30.</p><p>The reason they do that, without going into all the complex math is that these initializations make sure the input into every SIREN activation wt . X is always normally distributed with mean 0 and standard deviation 1. This in turn makes sure that the output of a siren is always <a href="https://en.wikipedia.org/wiki/Arcsine_distribution">arcsine</a> distributed in the range -1 and 1. These distributions stay consistent irrespective of the number of layers in the SIREN.</p><h3>Conclusion</h3><p>I think SIRENs are a really cool idea. I never thought images could be represented differently and voila! here we are with SIRENs. They can be used in the future for a number of downstream tasks related to signal processing and I can’t wait to find out what they are.</p><p><em>I hope you enjoyed this read. If you are reading my blog for the first time, hi, I’m Aryaman. I try to write explain deep learning papers in a much simpler language. For more reads like these, please do follow me on Medium and leave some claps and comments! Until next time 😁</em></p><h3>References</h3><ul><li>Paper Link: <a href="https://arxiv.org/abs/2006.09661">https://arxiv.org/abs/2006.09661</a></li><li>Yannic Kilcher’s beautiful video explanation: <a href="https://www.youtube.com/watch?v=Q5g3p9Zwjrk&amp;t=1879s">https://www.youtube.com/watch?v=Q5g3p9Zwjrk&amp;t=1879s</a></li><li>Vincent Katzmann’s (one of the authors) presentation: <a href="https://www.youtube.com/watch?v=Or9J-DCDGko&amp;t=875s">https://www.youtube.com/watch?v=Or9J-DCDGko&amp;t=875s</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c1e00179cb8e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Some Books I read and You should Too : Part 1 — Shoe Dog by Phil Knight]]></title>
            <link>https://medium.com/@aryamansriram/some-books-i-read-and-you-should-too-part-1-shoe-dog-by-phil-knight-32bcbb940f4d?source=rss-79cfba7bd5c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/32bcbb940f4d</guid>
            <category><![CDATA[inspiration]]></category>
            <category><![CDATA[shoe-dog-review]]></category>
            <category><![CDATA[nike]]></category>
            <category><![CDATA[shoes]]></category>
            <category><![CDATA[entrepreneurship]]></category>
            <dc:creator><![CDATA[Aryaman Sriram]]></dc:creator>
            <pubDate>Sat, 27 Nov 2021 07:22:06 GMT</pubDate>
            <atom:updated>2021-11-27T07:22:06.708Z</atom:updated>
            <content:encoded><![CDATA[<h3>Some Books I read and You should Too : Part 1 — Shoe Dog by Phil Knight</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/559/0*lzKrpMMlLhQKtcZo.jpg" /></figure><blockquote>I’d tell men and women in their mid-twenties not to settle for a job or a profession or even a career. Seek a calling. Even if you don’t know what it means, seek it.</blockquote><p>Through this brilliantly gripping masterpiece Phil Knight, the co-founder of Nike takes us through the amazingly rewarding journey of building a billion dollar brand. What’s particularly great about this book is having a chance to peek into his mind when he was having the worst days and how he was able to fight through every obstacle he ever faced. While we mostly see the success, the billions of dollars which makes us want to be a billion dollar business owner, this book offers us an insight into the price that needs to be paid to get there. Here are some of the key highlights for me from this book.</p><h3>Just Get Started</h3><p>The Nike that we know today started originally as Blue Ribbon sports inc. While at Stanford Business school a young Phil Knight had this crazy idea of sourcing Japanese running shoes and selling them over in the United States. Sounds simple right? Funnily enough the biggest players in the business then Adidas and Puma hadn’t figured this out. After graduating, unlike most of his peers, Knight took a loan from his father for funding a trip all around the world, making a stop in Japan to negotiate with shoe manufacturers, coming back and getting started.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gCXFGDUfVutCy-iJ" /><figcaption>The original Blue Ribbon Sports Logo</figcaption></figure><p>Now personally if I had been in Knight’s shoes at the time, I would have had thoughts like ‘This is probably a stupid idea, let me not waste time with this and focus on getting a job’ or ‘If this was to work why hadn’t the giants in the business figured it out’ or even ‘Why should I take the pain of going to Japan for an idea that doesn’t really have potential’. But clearly not paying heed to such thoughts is what made Knight who he is.</p><p>The clear lesson is just get started. Believe in the idea even if no one does. If you aren’t happy with your job, start looking for a new one which you would be happy with. Want to start a business, start working on it. The worst that could happen is that you fail. But if you don’t the rewards are limitless. So don’t think just do!</p><h3>Never Give Up</h3><p>I guess this is where you are probably going, ‘Back with that cliche again’. Well yes but there’s no other way. The book talks about points in Nike’s journey where while their sales were rising exponentially, so was their debt to finance the company. There were cases where their Japanese partners were trying to sabotage them. There was even a time when a bank had threatened to report the company to the FBI on account of huge accumulation of debt when Knight had a family to take care of. Just think about the kind of mental strength that he would have needed to get through all of this. What if Knight had decided it was getting too much for him and decided to hang up the boots. Everything he had worked for, gone. There would be no Nike, no Jordan’s nothing.</p><p>I am pretty sure everyone has had moments in life where they have given up. I have too and the worst feeling is having that bitter feeling, that regret: ‘If Only I had worked harder….’. I guess the key is as Knight says, to seek a calling. To know deep down that this is what will make me happy, and I am willing to die trying. No matter what happens, keep the faith and not give up.</p><h3>Passion Beats Everything</h3><p>We get introduced to several interesting characters who played a vital part in building Nike. Mr. Bill Bowerman, the co-founder for one who was in modern terms the head of product. Mr Bowerman was Phil Knight’s running coach at the University of Oregon who went on to become the coach national runners for the Olympics. Knight describes in detail the mad passion that Bowerman had for running shoes, or running for that matter. Any shoes he got he would constantly tear them apart, examine their soles, try out his own developments. He would constantly keep experimenting months on end with different tools available to him. In fact some of Nike’s bestselling products like the Cortez and the Waffle Trainer came from him.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*dgM7r8yVeK_YqbCp" /><figcaption>Bill Bowerman</figcaption></figure><p>Another man madly passionate about the business was Mr Jeff Johnson, Nike’s first employee. Johnson initially came on board dissatisfied with his previous job and ended up falling in love with the company. He would work hours on end meeting unrealistic sales targets, sending back new ideas for shoe innovations etc. In fact he was the one who set up Nike’s east coast business from scratch in a minimal amount of time.</p><p>Reading about these two and many more taught me that when you love what you do anything is possible. The book clearly shows that work for them was not work it was fun. Every new shoe experiment, every small bit of increase in sales was like a drug to them and they were always hungry for more. If they hadn’t been so passionate, I doubt if the company would have reached the heights it did.</p><h3>It’s Not About the Money</h3><p>While Nike is a huge company today with its founder’s ending up with millions of dollars, Knight clearly mentions it was never about the money. It was about building a great shoe brand and everyone in the company at the time thought the same way. In fact they consciously made efforts to keep the company from going public because it would make the company responsible to investors and they would lose their freedom. They only went public when they had no other choice to sustain the business.</p><p>Even when Knight talks about Nike’s endorsement deals, he talks about how for the athletes like Michael Johnson, Michael Jordan, Lebron James, Nike was like their own family. The book mentions several incidents where the relationship between the athletes and Knight is so much more than just a professional one.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ozk3mXLHkGLkPbnU.jpg" /><figcaption>Phil Knight</figcaption></figure><p>This book is particularly close to my heart because looking back at my life I have given up millions of times. Given up on things because I was too scared to spend time on them, time which I thought I should have spent studying. Chosen certain paths rather than taking a break and asking myself the question ‘What is my calling’, ‘What is it that I want from life exactly’. Long periods of indecision, lack of clarity are what have defined my life until now.</p><p>But reading this book sure has motivated me to just stop thinking, get started and keep the faith and most importantly never to give in. If you are in the same boat, hope reading it does the same or you too. Until then… Happy Reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=32bcbb940f4d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DeepVoice- An Intuitive Guide]]></title>
            <link>https://medium.com/analytics-vidhya/deepvoice-an-intuitive-guide-85d5bcba00c7?source=rss-79cfba7bd5c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/85d5bcba00c7</guid>
            <category><![CDATA[andrew-ng]]></category>
            <category><![CDATA[text-to-speech]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[baidu]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Aryaman Sriram]]></dc:creator>
            <pubDate>Fri, 22 May 2020 11:31:41 GMT</pubDate>
            <atom:updated>2020-05-22T16:57:50.354Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Text to speech conversion is a necessary problem to be solved in the world. It extends the reach of content by providing more than one way to consume it,makes content more accessible to those who cannot read etc. Researchers at Baidu AI came up with a long but intuitively very simple pipeline to tackle this problem and provide an end to end solution to it.The post assumes you have a basic understanding of how RNNs work.It gives me great pleasure to cover this paper since it was contributed to by one of my machine learning heroes: Andrew Ng.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*1wuEnsHwEjJMhyd0.png" /></figure><h3>The DeepVoice Solution Pipeline</h3><p>DeepVoice consists of the following networks:</p><ul><li><strong>The grapheme-to-phoneme model</strong>: As the name suggests it converts text to phonemes.</li><li><strong>The segmentation model: </strong>It locates boundaries between any two phonemes.</li><li><strong>The phoneme duration model:</strong> Predicts the duration of each individual phoneme.</li><li><strong>The fundamental frequency model:</strong> Predicts if the phoneme is voiced and its fundamental frequency throughout its duration</li><li><strong>The audio synthesis model:</strong> Synthesizes the final audio corresponding to the required text.</li></ul><h3>Network Architecture</h3><h4>The grapheme to phoneme model</h4><p>It employs the encoder-decoder architecture, often seen in Neural Machine Translation. It includes 3 bidirectional layers of 1024 <a href="https://towardsdatascience.com/understanding-gru-networks-2ef37df6c9be">GRU(Gated Recurrent Units)</a> each in the encoder and 3 unidirectional layers of the same size in the decoder and a <a href="https://www.youtube.com/watch?v=RLWuzLLSIgw">beam search</a> with a width of 5 candidates. This model predicts phonemes which are the phonetic equivalents of words.</p><h4>The segmentation model</h4><p>This model learns alignment between given utterance and its corresponding sequence of phonemes. Its trained using the <a href="https://towardsdatascience.com/intuitively-understanding-connectionist-temporal-classification-3797e43a86c">CTC loss</a>. In order to detect precise phoneme boundaries, this model is trained to predict pairs of phonemes instead of a single phoneme. Phoneme pairs are predicted at timesteps closer to the boundary between them the probability of finding both phonemes together is maximum at the boundary. The encoding can be seen as follows:</p><ul><li>Hello can be written as HH EH L OW in terms of phonemes</li><li>On padding with a silence token on either side, sil HH EH L OW sil</li><li>No consecutive token pairs are constructed as: “(sil, HH), (HH, EH), (EH, L), (L, OW), (OW, sil)”</li></ul><p>It consists of 2 2-D Convolution layers, followed by 3 bidirectional GRU layers having 512 GRU units each, and finally a softmax output layer. Beam search of width 50 is further used to decode phoneme boundaries.</p><h4>Phoneme Duration and Fundamental Frequency Model</h4><p>The input to the model is phoneme sequences along with stresses(emphasized parts) encoded into a one hot vector. The architecture comprises of 2 fully connected layers with 256 units each followed by 2 unidirectional recurrent layers with 128 GRU units each. It outputs: the phoneme duration, probability that the phoneme is voiced and 20 time dependent frequency values. The loss function used is a joint loss from all the three outputs.</p><h4>Audio Synthesis Model</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/819/1*fi2ErhJy_FRgus8gvWZWNA.png" /></figure><p>This model is a modification of the Wavenet network. As the paper mentions it consists of two networks:</p><blockquote>“A conditioning network, which upsamples linguistic features to the desired frequency, and an autoregressive network, which generates a probability distribution P(y) over discretized audio samples y ∈ {0, 1, . . . , 255}”</blockquote><p>The only difference in the wavenet here is instead of simply using transposed convolutions for upsampling, a method called as quasi-RNN is used to encode the inputs and then perform upsampling using repetitions.</p><h3>Training and Inference</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/646/1*SBq-pFMkIqDxEl5SC2UrfQ.png" /><figcaption>The training and inference cycles for the pipeline</figcaption></figure><h4>Training</h4><ul><li>Text is first fed into the grapheme-to-phoneme model which predicts phonemes for the corresponding text. This model is trained first.</li><li>These phonemes along with the audio form of the same text are fed into the segmentation model to predict the time steps at which pairs of phonemes have a maximum probability to occur. This gives an estimation of phoneme boundaries.</li><li>The duration and frequency prediction model is then trained using phoneme boundaries and phoneme inputs to predict the duration for each and every phoneme. The audio input data is used to provide the base frequency for each and every phoneme.</li><li>Finally using the phonemes,duration and base frequencies, the audio synthesis network is trained. Ground truth audio is provided as the label to be trained against.</li></ul><h4>Inference</h4><ul><li>The segmentation model is not used in the inference pipeline</li><li>Text is fed through the grapheme-to-phoneme model to produce phonemes.</li><li>These phonemes are then passed to the duration and frequency prediction model which predict the duration and base frequencies for the same and these inturn are used to predict the audio.</li></ul><h3>Why Deep Voice?</h3><h4>End to End Solution</h4><p>Deep Voice provided an end to end deep learning solution to the TTS problem unlike its predecessors which used special hand engineered features for the same.</p><h4>Minimal use of hand engineered features</h4><p>Deep Voice used one hot encoded phonemes, duration and frequency as features which were easily available from audio transcripts. Unlike other models like <a href="https://deepmind.com/blog/article/wavenet-generative-model-raw-audio">Wavenet</a> and <a href="https://mila.quebec/en/publication/char2wav-end-to-end-speech-synthesis/">char2wav</a> which used complex features instead.</p><h4>Production Ready System</h4><p>Deep Voice is very fast and provides speedy inference. This is great as compared to Wavenet which requires several minutes of runtime and <a href="https://arxiv.org/abs/1612.07837">SampleRNN</a> which require almost 4–5X more compute during inference.</p><h3>Conclusion</h3><p>Deep Voice was trained on an internal English speech database containing approximately 20 hours of speech data segmented into 13,079 utterances.<br>It was able to generate great results and proved to be superior to its predecessors.It was able to run at great runtime speeds even on CPU’s. I hope the post was able to shed some light on how deep voice works. I strongly urge the you to go through the paper since it does not really need a post like this. The paper is so beautifully written that is becomes very simple to understand it.</p><p><em>Link to the original paper</em>:<em> </em><a href="https://arxiv.org/pdf/1702.07825.pdf"><em>https://arxiv.org/pdf/1702.07825.pdf</em></a></p><p><em>Link to blogpost to get more clear intuition on training and inference pipeline: </em><a href="https://blog.athelas.com/paper-1-baidus-deep-voice-675a323705df"><em>https://blog.athelas.com/paper-1-baidus-deep-voice-675a323705df</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=85d5bcba00c7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/deepvoice-an-intuitive-guide-85d5bcba00c7">DeepVoice- An Intuitive Guide</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Super Resolution Convolutional Neural Network- An Intuitive Guide]]></title>
            <link>https://medium.com/analytics-vidhya/super-resolution-convolutional-neural-network-an-intuitive-guide-2fe9b5277569?source=rss-79cfba7bd5c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/2fe9b5277569</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[image-restoration]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[image-processing]]></category>
            <dc:creator><![CDATA[Aryaman Sriram]]></dc:creator>
            <pubDate>Sun, 17 May 2020 22:03:15 GMT</pubDate>
            <atom:updated>2020-05-20T14:23:20.194Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Extracting high resolution images from low resolution images is a classical problem in computer vision. The SRCNN paper published in 2015 was a major improvement to the pre-existing solutions for this problem since it was simple, efficient and provided an end to end solution. This blogpost is intended to give you an intuitive understanding of the same.<br>This blogpost assumes that you have a basic understanding of how convolutional neural networks work.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/416/0*jDDOJg50YzZKsWdO.jpg" /></figure><h3>Modelling the problem</h3><p>Assuming the input image, is an image Y, the goal of the algorithm was to find a function F(Y) that was as similar to the ground truth high resolution image X.</p><h3>Problem Solving Approach</h3><p>Before SRCNN came about, a pre-existing method called <a href="https://www.di.ens.fr/willow/pdfs/IASC08.pdf">Sparse Coding</a> was used for image restoration. It used a pipeline, which involved extracting overlapping patches from the image, mapping them to a higher resolution space and then aggregating this high resolution vectors and then restore the image by aggregating these vectors. But it involved a complex pipeline and use of complex mathematical techniques. The SRCNN authors were smartly able to model the same pipeline in the form of a CNN.</p><h4>Patch Extraction</h4><p>It involved extraction or cropping out of various patches from an image in an overlapped manner and converting them into a high dimensional vector for further processing. This can be thought of as a window sliding over each pixel of an image and converting each window into a high dimensional vector.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ulfFYH5HbWpLTIfuebj5mQ.gif" /></figure><h4>Non Linear Mapping</h4><p>Mapping the above extracted high dimensional vectors to other high dimensional vector non linearly. Keep in mind in this case the latter high dimensional vectors are the ones which will be a part of the high resolution image.</p><h4>Restoration</h4><p>Aggregate the above vectors into a high resolution image thus restoring the input image.</p><h3>Network Design</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/0*VCfCjSzvmoiqZSoC.png" /></figure><ul><li>Patch Extraction: 64 filters of size 9 x 9 x 3 were used to perform the first phase which is patch extraction of the solution pipeline. The first layer can be expressed as:<br>F1(Y) = W1 ∗ Y + B1<br>where Y is the image,<br>W1 corresponds the the 64 filters being used,<br>B1 corresponds to the biases being used.</li><li>Non Linear Mapping: The previously found feature maps were then put through a <a href="https://medium.com/@danqing/a-practical-guide-to-relu-b83ca804f1f7">relu</a> activation function. 32 filters of size 64 x 1 x 1 were used for mapping the previously activations from the relu to a higher resolution space. A relu activation was applied to the same to make sure that the mapping was not linear. It can be expressed as:<br>F2(Y) = max (0, W2 ∗ F1(Y) + B2)<br>where Y is the image,<br>W2 corresponds the the 32 filters being used,<br>B2 corresponds to the biases being used.</li><li>Restoration: The high resolution maps were then subjected to 3 filters (since the image is composed of 3 channels) of size 5 x 5 in order to aggregate the high resolution mappings in the previous layer. This was a linear mapping and can be expressed as:<br>F(Y) = W3 ∗ F2(Y) + B3<br>where Y is the image,<br>W3 corresponds the the 3 filters being used,<br>B3 corresponds to the biases being used.</li></ul><h3>Training</h3><p>The input to the model was a standard low resolution image. While training its output was set to be its equivalent high resolution image. Mean squared error was used as the loss to be minimised. The error in this case was nothing but the difference between the ground truth high resolution image and the generated high resolution image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/982/1*JYIQVKXu17PrrfWjOe3V7w.png" /><figcaption>Loss function for SRCNN where F(Y) represents the predicted high resolution image and Xi represents the ground truth high resolution image.</figcaption></figure><h3>Why SRCNN?</h3><h4>Simplified Structure</h4><p>As mentioned above it aggregated the solving technique of complex algorithmic pipelines used earlier into a single convolutional network thus providing an end to end solution for the problem.</p><h4>A win for deep learning</h4><p>The authors were able to demonstrate how deep learning was useful in solving classical computer vision problems which at the time was impactful.</p><h4>Efficient</h4><p>With moderate number of filters, the network was able to achieve fast processing speeds even on CPU’s.</p><h4>Improving Performance</h4><p>The authors were also able to demonstrate that diverse datasets and more number of layers,were able to improve the the performance of the dataset.</p><h3>Conclusion</h3><p>On comparing the results from SRCNN with state of the art methods like SC(Sparse Coding), A+(<a href="https://homes.esat.kuleuven.be/~konijn/publications/2014/3926_postprint.pdf">Adjusted Anchored Neighbourhood Regression method</a>) and ANR(<a href="http://people.ee.ethz.ch/~timofter/publications/Timofte-ICCV-2013.pdf">Anchored Neighbourhood Regression method</a>), in terms of the PSNR (<a href="https://www.ni.com/en-us/innovations/white-papers/11/peak-signal-to-noise-ratio-as-an-image-quality-metric.html">peak signal to noise ratio</a>) metric, SRCNN turned out to be a clear winner. It rightly proved to be a simple, robust and accurate solution.I hope this post provided a good insight into this algorithm. I highly encourage you to go through the original paper and further research about other solutions for this classical computer vision problem.</p><p><em>Link to the original paper and code : </em><a href="https://paperswithcode.com/paper/image-super-resolution-using-deep"><em>https://paperswithcode.com/paper/image-super-resolution-using-deep</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2fe9b5277569" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/super-resolution-convolutional-neural-network-an-intuitive-guide-2fe9b5277569">Super Resolution Convolutional Neural Network- An Intuitive Guide</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[YOLO V1- An Intuitive Guide]]></title>
            <link>https://medium.com/analytics-vidhya/yolo-v1-an-intuitive-guide-ad24c707b0eb?source=rss-79cfba7bd5c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/ad24c707b0eb</guid>
            <category><![CDATA[convolution-neural-net]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[object-detection]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Aryaman Sriram]]></dc:creator>
            <pubDate>Sat, 16 May 2020 07:28:20 GMT</pubDate>
            <atom:updated>2020-05-19T14:19:53.800Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Real time object detection finds its use in a number of applications like security, surveillance, video analytics etc. YOLO V1 algorithm which was proposed in 2016 took the world by storm by being one of the fastest, most efficient algorithms for real time object detection. The following blogpost provides an insight into the same and aims to give you an intuitive understanding of the algorithm before you dive into all the math behind it.<br>The post assumes that you have an understanding of how neural networks and convolutional neural networks function.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*HnJ9faDWtx8jMWHfax4VcQ.jpeg" /></figure><h3><strong>The Problem</strong></h3><p>The problem of object detection consists of two parts</p><ul><li>Object Localization: Locating a particular object in the image/ video.</li><li>Object Recognition: What is the label to be assigned to the object which is located in the image/video.</li></ul><h3>Previous Work and its Limitations</h3><h4>Deformable Parts Model (DPM)</h4><p>This model involved a complex pipeline consisting of separate modules for feature extraction,region classification and prediction of bounding boxes. As it can be noted the model had way too many modules and complexities. Also, the image was divided into fixed size windows and the feature extractor was made to slide over all the windows to extract features from the entire image, which as in turn computationally expensive and not adequate for real time object detection.</p><h4>Regional- Convolutional Neural Network (R-CNN)</h4><p>This model again involved a very complex pipeline. It involved a region proposal network to propose regions most likely to contain an object. A CNN was further used for feature extraction and support vector machines were used on top to generate a confidence score. This system was very slow given its complexity.</p><h4>Deep Multibox</h4><p>This algorithm was pretty similar to RCNN. The only difference was instead of selective search a CNN was trained to predict region proposals. The drawback of this model was that it was able to make a single class prediction but was not able to detect general objects i.e different types of objects.</p><h3>YOLO</h3><p>YOLO stands for You Only Look Once. It signifies that unlike sliding a feature extractor on the image multiple times, the algorithm looks at the image only once for detecting objects in it. YOLO involves dividing the image into an S x S grid.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*CYTDLg54ol-NpBOnrhFo2A.jpeg" /></figure><p>If the center of an object falls into a grid cell,that particular cell is considered to be responsible for detecting that particular object. Each grid cell predicts B number of bounding boxes and a confidence score for each of them signifying how strongly the algorithm feels that there is an object in the bounding box. Boxes with confidence scores over a particular threshold (0.25) are considered to be final ones.The confidence score has the formula:</p><p>Pr(obj) * IOUᵗʳᵘᵗʰₚᵣₑₔ</p><p>Here, Pr(obj) stands for probability that an object exists in the predicted bounding box.IOU stands for Intersection Over Union. As the name suggests it is the area of intersection between the actual bounding box and the predicted bounding box, divided by the Union of the areas between the actual and the predicted boxes.</p><p>Predicting a bounding box means predicting 5 values: x,y,w,h and confidence score (described above). (x,y) are the the co-ordinates of the center of the predicted box relative to its corresponding grid cell, w and h are the width and height of the box relative to the image.Finally, the algorithm also predicts conditional class probabilities i.e names of objects predicted in each grid cell. These probabilities are conditional since they are dependent upon an object actually being detected in a particular bounding box.</p><h3>Network Design</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BOQy9y-JxAylPWaF.jpeg" /></figure><p>The YOLO CNN consists of 24 convolutional layers and 2 fully connected layers. The detailed structure can be seen above. It involved the use of 3 1 x 1 convolutional layers to reduce the the amount of features coming in.</p><p>The output is a tensor (a 3-D matrix) of dimensions 7 x 7 x 30. This is because the model is trained on the Pascal VOC dataset which has 20 object classes to predict (C=20). The image is divided into a 7 x 7 grid (S=7). Each cell is responsible for 2 bounding boxes (B=2). Each box predicts 5 values and a C number of probabilities to predict. Hence the output turns out to be <br>S x S x (B*5 +C) which on substitution becomes 7 x 7 x 30.</p><h3>Training</h3><p>The convolutional layers of the yolo network were pre-trained on the imagenet dataset for over a week. Since imagenet dataset has over a 1000 classes of objects, the objective was to extract as many features as possible.<br>The network then was made to perform detection and optimise the following loss function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cp4hsihXQVGsJUV2.png" /></figure><p>Rather than delving into the math of it, simply put the loss function takes into account three losses</p><ul><li>Classification loss: Difference between the predicted class probabilities and the actual class probabilities</li><li>Localization loss: Difference between predicted bounding box values (x,y,w and h) and actual bounding box values.</li><li>Confidence loss: Error in the confidence score</li></ul><p>To delve deeper into the loss function I suggest going through <a href="https://medium.com/@jonathan_hui/real-time-object-detection-with-yolo-yolov2-28b1b93e2088">this amazing post</a> by Jonathan Hui.</p><h3>Why YOLO?</h3><h4>Speed</h4><p>As compared to the previous detection systems YOLO turned out to be very fast, since rather than a sliding window approach, it involves passing the image through the network only once. It was able to run at 45 frames per second on a Titan X GPU without any kind of batch processing.</p><h4>Simplicity</h4><p>Unlike alot of the other models, YOLO involves a single CNN. Hence no complex pipelines are required, which makes the model easy to understand and faster to execute. All tasks like feature extraction, bounding box prediction, object classification are taken care of a single network itself.</p><h4>Fewer Bounding Boxes</h4><p>Yolo predicts fewer bounding boxes than other models. As compared to R-CNN which propose around 2000 boxes per image, YOLO propose only around 98.</p><h3>Conclusion</h3><p>While YOLO v1 was highly successful, it did have its own limitations. It failed to detect small objects which appeared in groups. It was not able to generalize to objects with unusual aspect ratios. Also, in this algorithm, the errors weren’t normalized i.e a small error cannot be neglected in a smaller box as it can in a larger one. That being said, the YOLO algorithm has been through a number of revisions, the latest being YOLO v4. I hope the post was able to give you an understanding of how this algorithm functions. I encourage you to go through the paper and also try it out on your own system.</p><p><em>Link to the original paper:</em> <a href="https://arxiv.org/abs/1506.02640">https://arxiv.org/abs/1506.02640</a></p><p><em>Link to imageai api to try out yolo on your own system: </em><a href="https://imageai.readthedocs.io/en/latest/detection/">https://imageai.readthedocs.io/en/latest/detection/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ad24c707b0eb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/yolo-v1-an-intuitive-guide-ad24c707b0eb">YOLO V1- An Intuitive Guide</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>