<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sharin Khander on Medium]]></title>
        <description><![CDATA[Stories by Sharin Khander on Medium]]></description>
        <link>https://medium.com/@sharinkhander?source=rss-4e0edb9cb39e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*x5_CbpvA8wUH6cJxmuW_mg.png</url>
            <title>Stories by Sharin Khander on Medium</title>
            <link>https://medium.com/@sharinkhander?source=rss-4e0edb9cb39e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 22:55:28 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sharinkhander/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Right AI Model Depends on the Job]]></title>
            <link>https://sharinkhander.medium.com/the-right-ai-model-depends-on-the-job-9046d3bc7ae0?source=rss-4e0edb9cb39e------2</link>
            <guid isPermaLink="false">https://medium.com/p/9046d3bc7ae0</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[startupş]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[product-management]]></category>
            <dc:creator><![CDATA[Sharin Khander]]></dc:creator>
            <pubDate>Fri, 23 Oct 2026 03:41:27 GMT</pubDate>
            <atom:updated>2026-10-23T03:42:42.439Z</atom:updated>
            <content:encoded><![CDATA[<h3>A Guide to Choosing AI Models</h3><h3>The problem</h3><p>Nine new frontier AI models came out in the last month.</p><p>If you’ve lost track, you’re not alone.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_JhPCz6bH98b2wpw4yHA8A.png" /></figure><p>Benchmarks test models on graduate-level math and physics. They can show how far models have come. They don’t tell you which one will write the best cold email.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*az1s4R0mu2xo1EjyBl16hA.png" /></figure><p>Don’t get me wrong. Benchmarks like <em>Humanity’s Last Exam</em> are fascinating. They help us measure progress toward AGI, and they make for great headlines.</p><p>So what we should ask instead is:</p><blockquote><strong><em>What do I actually need AI to do?</em></strong></blockquote><h3>Framework</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fdK4ky4v0GCCeytK-jyXMA.png" /></figure><h4>1. Be a thinking partner</h4><p>A thinking partner is a smart colleague. It challenges your idea, spots a gap, or helps you see the problem from another angle.</p><blockquote><strong><em>The goal isn’t a polished answer. It’s better thinking.</em></strong></blockquote><p>That means speed matters. You’re brainstorming. You’re in the middle of an idea. You don’t want to ask a question, go make lunch, and come back ten minutes later to find out what the model thinks.</p><p>You’re in a flow state. The AI should keep up.</p><h4>2. Be a task executor</h4><p>This is where AI takes the grunt work off your plate: synthesizing hours of user interviews, summarizing millions of rows in a database, or building a new onboarding flow.</p><blockquote><strong><em>The model should deliver useful work on the first pass, without a follow-up turn.</em></strong></blockquote><p>The worst feeling is checking back a few hours later, expecting the task to be 80% done, and finding it stuck at 10%.</p><h4>3. Be a proactive agent</h4><p>A proactive agent sits between a thinking partner and a task executor.</p><p>You ask an executor to do something. An agent keeps watching and tells you when something changes or finds something you didn’t know to look for. For example:</p><blockquote><strong><em>Find relevant startup jobs that match my skills and comp requirements.</em></strong></blockquote><p>The right agent might help me find a great role at the next unicorn, one I’d never have found on Y Combinator job boards.</p><h3>The trade-offs</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5RDM0XJCxEAZbpZgVFTBlw.png" /></figure><p>A model’s answer is only part of the decision. I also look at <strong>how long it took</strong> and <strong>what it cost</strong>.</p><p>Speed matters most when I’m using AI as a thinking partner. A slow answer can break the conversation.</p><p>Cost matters most when I’m running an unsupervised task. If the model gets stuck or needs another try, I want room to experiment without worrying about every token.</p><h3>How I Decided</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MauYJlAiY7nolXpprt-KIw.png" /></figure><h4>Best thinking partner: Claude Opus 5.5 🏆</h4><p>When I’m brainstorming, I want speed more than polish, so I set reasoning to low. The model spends less time thinking before it replies.</p><p>I’d rather get a quick answer that’s a little off than wait for a perfect one. If it misses, I correct it. The point is to keep ideas moving without interruption.</p><h4>Best task executor: GPT-6.1 Sol 🏆</h4><p>I chose this model because it’s cheap, and my ChatGPT subscription already covers it.</p><p>Cost matters when a task runs in the background. I’m not watching every step. Sometimes the job needs another try. Sometimes it wanders somewhere I didn’t expect.</p><p>With a cheap model, those detours cost little, so I can leave it alone. I delegated the task so I could stop supervising it. An expensive model would put me right back to watching.</p><h4>Best proactive agent: GPT-6.1 Astra 🏆</h4><p>I use Astra for background jobs. I prefer it to its cheaper cousin, Luna, and some of that is FOMO.</p><p>When I ask it to search the web, I don’t want it to miss anything that could change my next move: a great deal, a new competitor, or an idea that changes how I see my startup.</p><p>For that kind of work, I’ll pay a little more for a model I trust to notice what matters.</p><h3>A few things to note</h3><h4>Before you blame the model</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QB7P1RkdXih-kED04HqPvQ.png" /></figure><p>It’s only one part of the setup.</p><p>The same model can behave differently in Claude Code, Codex, or Cursor. Change the harness, the instructions, or context and you might get a different result.</p><p>That makes evaluations tricky. You might be testing the model, the harness, the instructions, and the context all at once. Change one, and the model might succeed or fail at the task.</p><h4>How “reasoning” factors into it</h4><p>A model does some work before it answers. You don’t see most of it, and you probably don’t need to. Watching every reasoning token can feel like livestreaming someone’s inner monologue.</p><p>Set reasoning to <strong>high</strong>, and the model spends more time working through the problem before it replies. That can help with harder tasks. It can also mean a longer wait and more tokens.</p><h4>Every new model has different prompting guidelines.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BKvep3wVQz4gakjlZwmFaQ.png" /></figure><p>When a new model comes out, revisit the instructions you carried over from the old one.</p><p>Anthropic said it removed more than 80% of Claude Code’s system prompt for Opus 5 and Fable 5, with no measurable drop on its coding evaluations. <a href="https://claude.dev/blog/the-new-rules-of-context-engineering-for-claude-5-generation-models/">Anthropic’s guide</a> explains how it decided what to cut.</p><p>That made me think about habits like adding “double-check your work” to every prompt. On one model, it might help. On another, it might add tokens and crowd out clearer instructions.</p><p>The lesson isn’t to stop checking the work. It’s to test the instructions you keep. If they improve the result, keep them. If they don’t, they’re taking up space and burning cash.</p><h4>Model Scorecard</h4><p>I wrote a separate eval for each kind of task: code review, strategy review, and ad optimization for my business.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BmYeczSox40pPuCf4bujgQ.png" /></figure><p>Here’s one question I asked:</p><blockquote><strong><em>If you had zero sales across 30 ads, what should you do next?</em></strong></blockquote><p>The right answer is to investigate the tracking first. Maybe an instrumentation bug broke the numbers.</p><blockquote><strong><em>The model told me to kill the ads because they weren’t performing.</em></strong></blockquote><p>That sounds decisive, but it skips the first question: are the numbers even right?</p><p>Models can sound sure and still miss what matters. The useful skill is knowing when to question an answer, what to check, and how to correct it.</p><p>Some models are trained to pause and check their work. That can help, but extra reasoning tokens don’t guarantee a better answer. Test whether they improve the result you care about.</p><h4>Length ≠ Intelligence</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SPlkC4QabacEaDKuSgIkLw.png" /></figure><p>If two answers score the same, but one takes twice as long to read, that extra length has a cost: your attention. I want AI that gets straight to the point.</p><h3>Steal this: build an evaluation suite in one afternoon</h3><h4><strong>Start with five decisions you made this month.</strong></h4><p>Maybe you chose a strategy, created a pitch, or decided what to change in a design. Ask Claude or Codex to find five recent examples in its saved reasoning traces and explain how each one led to a decision.</p><p>Then turn those examples into a test:</p><ol><li><strong>Write the answer key first.</strong> Decide what a good answer looks like and what a bad one looks like before you show the model the examples.</li><li><strong>Give every model the same prompt.</strong> Save each full response.</li><li><strong>Read the results.</strong> Do you agree with the answer? Does the reasoning support the decision?</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nURcthe18Svh8zDdDR4STQ.png" /></figure><p>If you don’t have real examples, you can make up the questions, answers, and scoring criteria. But eventually, you need to read the results and decide whether the model’s answer and its reasoning make sense.</p><h4>Running this efficiently</h4><p>I use <a href="https://www.hermes-agent.org/">Hermes</a> to switch between AI providers. If you’ve used <a href="https://myclaw.ai/openclaw">OpenClaw</a>, it works in a similar way.</p><p>I message all my agents through Signal. Cursor helped me get it all working.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aJ8_4z4uFq8WUN7l6Ar1PA.png" /></figure><h3>Bottom Line</h3><p>Pick the model for the job. Test it on your own work. Weigh quality against speed and cost.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3h2D3uilULbWP4yqEFDv9Q.png" /></figure><p>Test models on your own work, then weigh answer quality against speed and cost. Then do it again next week, because there will be a new model released 😅</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9046d3bc7ae0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My love letter to Vercel]]></title>
            <link>https://sharinkhander.medium.com/my-love-letter-to-vercel-e6bca4478b12?source=rss-4e0edb9cb39e------2</link>
            <guid isPermaLink="false">https://medium.com/p/e6bca4478b12</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[vercel]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[software]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Sharin Khander]]></dc:creator>
            <pubDate>Wed, 17 Jan 2024 23:12:47 GMT</pubDate>
            <atom:updated>2024-01-17T23:25:30.329Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Not a sponsored post — unless Vercel wants to change that and toss some cash my way! 😄</em></p><h3>The TL;DR</h3><ul><li>Vercel simplifies the process for engineers to deploy and run the user-facing parts of their applications, known as the frontend.</li><li>Applications typically consist of two parts: the frontend and the backend. Vercel allows teams to deploy the frontend separately, making the process much easier compared to deploying both parts on a single server.</li><li>Deploying with Vercel offers features like deploy previews, functions as a service, analytics, and more.</li><li>In addition to the frontend, Vercel has expanded to include cloud-like features such as storage and databases.</li><li>While the method of deploying frontends was once a personal choice, tools like Vercel are now becoming the standard for teams getting their front ends online. Even the Medium website is built with Vercel!</li><li>The enthusiasm expressed for Vercel in this post is not sponsored but comes from years of personal use and satisfaction with the product.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6vwz_EacemAtDa3-HfSo7g.png" /></figure><h3>What does deploying mean, and why do I need to do it for my frontend?</h3><p>The frontend of your application is what your users see. But it does more than that. Your typical frontend code does the following:</p><ul><li>Builds the visual part of a webpage using HTML and CSS.</li><li>Handles interactivity and animations using JavaScript.</li><li>Makes calls to internal and external APIs to fill pages with data, like a user profile page.</li></ul><p>Frontends are organized as a set of files representing pages, their structure, and accompanying JavaScript for interactivity, logic, and API calls. Think of it like the front of a house (yes, chef) connecting the customer to the kitchen, ensuring orders reach the kitchen and the right food comes back to the customer. Deploying your frontend means putting all this online for users to see and interact with.</p><h4>🔍 DEEPER LOOK 🔍</h4><p>Some engineering teams keep all their application code, both the frontend and backend, in one big folder known as a monorepo. Others use separate folders for the frontend and backend, and some even have different folders for various parts of the backend. The choice of setup depends on what priorities a team has and how complicated their application is.</p><p>However you grasp the frontend, the key point is that frontend code needs a place to be. Developers create the frontend on their laptops and have to put it on a server so that web users can reach it. How do they do that? Well, there are a few choices:</p><ol><li>Put it on a separate server, not connected to your backend.</li><li>Put it on the same server as your backend.</li><li>Use Infrastructure as a Service (IaaS) like AWS or Platform as a Service (PaaS) like Heroku, Railway, etc. This can be with or without your backend.</li></ol><p>But here’s the catch — all these options need some setup and configuration. You’ll have to deal with things like setting up authentication/networking, caching your frontend globally for quick access, handling DNS for a nice URL, and other interesting tasks. I’m not trying to overwhelm you with unfamiliar terms or concepts; there’s just a bunch of things to take care of.</p><p>Vercel, along with similar products like Netlify (which I’ve also used and enjoyed), is designed specifically for deploying frontends and making the whole process super simple. It is quite expensive tho! But let’s unpack the value it adds…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*-YLTgkDdKWHbznOM3SuZBg.gif" /><figcaption>Deploying to Vercel be like</figcaption></figure><h3>The basic Vercel product:</h3><p>Vercel makes deploying and using your frontend as easy as can be. For example, I have my frontend code in a GitHub repository, just like any software project. To get it online, all I have to do is create a Vercel account, grant it access to my repository, and set up the domain name. And just like that, your site is live!</p><p>Behind the scenes, Vercel takes my code, puts it on servers worldwide, adds a URL to it, and makes sure it runs and loads super fast. And guess what? This is just the beginning!</p><p>If I want to make changes to my website, I simply push those changes to my Github repo, and Vercel does the magic — it automatically updates the site with the new code on the server. They even give you a neat dashboard for each site you deploy, showing you status, recent updates, and the domains it’s hanging out at. Easy peasy!</p><p>Here’s another cool thing: deploy previews. Imagine you’re making changes to your website and thinking of merging them. With Vercel, when you open a pull request with these changes, they automatically try it out on a new server. This way, you can see how your site looks with the changes before actually applying them.</p><h4>🚨 CONFUSION ALERT🚨</h4><p>You might wonder, if your code is on the Github website, why isn’t it automatically “deployed” there? Why can’t users access the code on Github directly? Well, the answer is a bit complex, but the simple idea is that Github isn’t designed for that. It lacks things like domain handling, making the server visible to the outside world, preview features, and all the other cool stuff that platforms like Vercel offer. There’s a rarely mentioned feature called Github Pages that kinda lets you do this, but it’s not fancy enough for a proper application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*8uRdjjJyz1sdr4uUxd6VuQ.gif" /><figcaption>Vercel responding to the last meme</figcaption></figure><h3>Next.js and Vercel: A Winning Combo</h3><p>Here’s the exciting part about Vercel — they’re the creators of Next.js, one of the hottest web frameworks for React right now. Next.js is not only open source and free but also more popular than Vercel’s commercial product.</p><p>Now, what’s a web framework? It’s like a toolbox that makes it way easier to build a web server, especially for frontends, without drowning in custom code. Most modern web applications use some kind of web framework. Here are a few you might know:</p><ul><li>Django or Flask for Python</li><li>Ruby on Rails for Ruby</li><li>Laravel for PHP</li><li>Spring for Java</li></ul><p>As React gained popularity, Next.js emerged as a React-specific web framework (others include Create React App, Gatsby, etc.).</p><p>What makes it cool is that Vercel, being the brains behind Next.js, seamlessly blends it into their commercial product. This integration brings some smart performance boosts. Plus, every function in the “api” folder magically transforms into a serverless function in Vercel, providing an easy way to build a backend without breaking a sweat.</p><h3>Vercel’s Frontend Cloud: More Than Just Deployment</h3><p>Vercel is not just about deploying your frontend — it has grown into a full-fledged frontend cloud offering various features to support your entire application:</p><p>1. <strong>Edge Functions:</strong><br> — Run server-side or backend code without the hassle. Perfect for tasks like authentication and experimentation.<br> <br>2. <strong>Analytics:</strong><br> — Get insights into your site’s performance and accessibility through statistics and dashboards.</p><p>3. <strong>CLI / API:</strong><br> — Interact with Vercel programmatically. Run deployments, perform checks, and more with ease.</p><h3>Edge Functions Highlight</h3><p>The real game-changer here is Edge Functions. Unlike typical serverless functions confined to one geographical region, Vercel’s implementation has them running globally. Their team claims it’s faster and more cost-effective.</p><h3>Evolution to a Frontend Cloud</h3><p>While most significant applications are still manually deployed on servers, Vercel is paving the way. It might be more accurate to call it a frontend cloud — a collection of services centered around the frontend that simplifies app development. Recent additions include:</p><p>- <strong>Storage (Vercel Blob):</strong><br> — Easy object storage for files and videos, strategically placed across the globe close to your users.</p><p>- <strong>Databases:</strong><br> — Managed databases, starting with a key-value store and Postgres.</p><h3>Cloud with a Frontend Focus</h3><p>Yes, Vercel is now a cloud, but it’s not your typical cloud. It focuses on the frontend, abstracting away the complexities of the backend. It provides managed infrastructure, allowing developers to build faster and easier.</p><h3>Continuous Evolution</h3><p>From its days as Zeit to its current form, Vercel has transformed from a simple frontend deployer into a comprehensive platform for building and running entire applications. Exciting times in the world of Vercel!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/706/1*LiXcKPyf1Nggp9pu9f9zIw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e6bca4478b12" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How do Large Language Models (LLMs) work?]]></title>
            <link>https://sharinkhander.medium.com/how-do-large-language-models-llms-work-ce7b26642ae4?source=rss-4e0edb9cb39e------2</link>
            <guid isPermaLink="false">https://medium.com/p/ce7b26642ae4</guid>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Sharin Khander]]></dc:creator>
            <pubDate>Tue, 26 Dec 2023 19:29:21 GMT</pubDate>
            <atom:updated>2024-01-02T16:36:30.627Z</atom:updated>
            <content:encoded><![CDATA[<p>Large language models (LLMs) like ChatGPT, Bing’s new “Sydney” mode, and Google’s Bard are <a href="https://www.theverge.com/2023/12/27/24016212/new-york-times-openai-microsoft-lawsuit-copyright-infringement">dominating the headlines</a>.</p><p>Instead of discussing which jobs they’re going to make obsolete, this post will explore how these models function, including where they get their data and the basic math that enables them to generate convincingly realistic text.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/492/1*tzKxfVfRfrDdBSVHnenfqA.png" /></figure><h3>Machine Learning 101</h3><p>LLMs are a kind of Machine Learning model, just like many others. To grasp how they operate, let’s start by understanding the basics of Machine Learning in general.</p><p>Note: there are excellent visual resources online that explain Machine Learning in more detail and probably better than I can — I recommend checking them out! However, I’ll cover the fundamental concepts here.</p><p>The easiest way to grasp basic ML models is by thinking about prediction: <strong>based on what I already know, what’s likely to happen in a new situation?</strong> It’s similar to how your brain works.</p><p>Imagine you have a friend who’s always late. You’re planning a party, so you expect him to be late again. It’s not certain, but given his track record, you think there’s a good chance. If he arrives on time, you’re surprised, and you remember that for next time; maybe you’ll adjust your expectations about him being late.</p><p>Your brain has many of these models constantly working, but right now, we don’t fully understand how they actually work on the inside. In the real world, we have to make do with algorithms — some simple, some really complicated — that learn from data and make predictions about what might happen in new situations. Typically, models are trained to do specific things (like predicting stock prices or creating an image), but they’re starting to become more versatile.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/476/1*9RLK66MCdKZSMEAA7mH0sg.png" /></figure><p>A Machine Learning model is a bit like an API: it takes in inputs, and you teach it to produce certain outputs. Here’s the process:</p><p>1. <strong>Gather training data: </strong>Collect a bunch of data about what you want to model.<br> <br>2. <strong>Analyze training data:</strong> Look at the data to find patterns and details.</p><p>3. <strong>Pick a model:</strong> Choose an algorithm (or a few) to understand the data and its workings.</p><p>4. <strong>Training: </strong>Run the algorithm, it learns, and stores what it figures out.</p><p>5. <strong>Inference:</strong> Present new data to the model, and it gives you its thoughts.</p><p>You create the model’s interface, deciding what information it receives and what it gives back, based on the specific job it needs to do.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BM4drtQtAlZjOuUOoSJwow.png" /></figure><p>What does the algorithm do, you ask? Well, think of it as a super-smart analyst. It discovers connections in the data you provide, connections that are often too tricky for you to spot on your own. The data usually has some X stuff — like features, settings, details — and some Y stuff — what actually went down. If you’re checking out this data:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YvShI6CUOnR6pjGtC5Eevw.png" /></figure><p>You don’t need ML to tell you that when X is 15, Y is around 150,000. But what if there are 30 different X things? Or the data looks all weird? Or it’s text? ML is all about tackling tricky situations where our human skills fall short. That’s it.</p><p>That’s why ML algorithms can be as basic as linear regression (like in Statistics 1) or as complex as a neural network with millions of nodes. The super-advanced models in the news lately are crazy complicated and took lots of people and years of research. But in many companies, Data Scientists use simple algorithms and still get good results.</p><p><strong><em>🔍 Deeper Look 🔍</em></strong></p><p>Building strong ML models from the ground up is a very specialized field. Some Data Scientists and ML Engineers create models using tools like PyTorch and Tensorflow, while others enhance existing open source models. You can also choose to have the whole model development process outsourced and use a ready-made one created by someone else.</p><p><strong><em>🔍 Deeper Look 🔍</em></strong></p><p>Creating a model is like a trial-and-error process. Unless your data is really straightforward, you’ll probably have to test different methods and make constant adjustments before your model starts making sense. It’s a mix of science, math, art, and a bit of randomness.</p><h3>Language models and generating text</h3><p>When your data has a time element — like predicting future stock prices or understanding an upcoming election — it’s pretty clear what a model is up to. It’s using the past to predict the future. However, many ML models, such as language models, don’t deal with time series data at all.</p><p>Language models are just ML models that handle text data. You train them on something called a “corpus” (or just a body) of text, and then you can use them for various tasks, like:</p><ul><li>Answering questions</li><li>Searching</li><li>Summarizing</li><li>Transcription</li></ul><p>The idea of language models has been around for a while, but the recent rise of deep learning with neural networks has been a big deal; we’ll talk about both.</p><h3>Probabilistic Language Models</h3><p>In simple terms, a probabilistic language model is like a probability map for words or groups of words. In English, it examines a chunk of text and analyzes which words show up, when they appear, how frequently, and the order they come in, and stuff like that. All this info is captured statistically.</p><p>Now, let’s quickly make our own language model.</p><p>Here are two sentences that might or might not express what I really think:</p><p><em>“The best Manhattan cocktail specification uses two ounces of Van Brunt Empire Rye, one ounce of Cocchi Di Torino Sweet Vermouth, one dash of Angostura Bitters, and one dash of Orange Bitters. I stir it in a mixing glass for about 60 turns, pour into a chilled Nick and Nora glass, and serve garnished with a Maraschino cherry.”</em></p><p>To make a simple probabilistic language model, we’ll collect n-grams, a fancy stats term for groups of words. Let’s set n=1, meaning we’ll just count how often words show up:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a4RZXG7q7UrUN-e-kUMe7A.png" /></figure><p>And if n=2:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RM5F0CsAgJWzK7ze43ER6g.png" /></figure><p>What the model does is make a bunch of n-grams, paying attention to which words show up together and in what order.</p><p><em>😰 </em><strong><em>Don’t sweat the details</em></strong><em> 😰</em></p><p>I go through the n-gram exercise just to show that what many models are doing isn’t really that complicated (though for some, it is). So, don’t stress if you didn’t get all of the details above.</p><p><em>😰 </em><strong><em>Don’t sweat the details</em></strong><em> 😰</em></p><p>Once you have that info stored, you predict which words might come next. If we were to create a new sentence from our two cocktail sentences, we’d put words together in a way that resembles the ones before.</p><h3>Neural networks and language models</h3><p>Probabilistic language models have been around for decades. Lately, though, it’s become more popular to use neural networks — a more complex algorithm — for language models. These networks learn what’s happening in a more meaningful way using something called embeddings. It’s tricky for models to learn directly from words, but much easier for them to learn from mathematical representations of those words.</p><p>Embeddings are a method of taking data with tons of dimensions — like giant bodies of text, with lots of discrete words and combinations — and mathematically representing them with less data, without losing much detail. Working with the text from 100 different 1,500-word blog posts about cocktail making (that’s 150K words!) is tough for an ML model. But if we can turn that information into a bunch of numbers, then we’re on the right track.</p><p>Equipped with a more user-friendly representation of words and text, neural networks can learn important things about text, like:</p><ul><li>Semantic relationships between words</li><li>Bringing in more context (sentences before and after a word or sentence)</li><li>Figuring out which words are important and which aren’t</li></ul><p>This stuff gets pretty complex. But the goal is simple: a model powerful enough to consider a lot of context when predicting the next word, sentence, or paragraph, just like our brains do.</p><h3>Large Language Models Now</h3><p>ChatGPT and its cousins are essentially massive language models (that’s why they’re called that). They’re built on layers of advancements from the past decade, including:</p><ul><li>Word2Vec models</li><li>LSTM (long short term memory) models</li><li>RNN (recurrent neural networks)</li><li>Transformers (yes) (also known as “foundation models”)</li></ul><p>You don’t really have to know what each of these is. The key is to realize that these big language models aren’t some sudden scientific breakthrough. Researchers have been steadily progressing towards today’s reality for years, and each new development played a crucial role in reaching this point. There was a big hype cycle when LSTMs gained traction in 2019 (even though the concept was introduced in the 90s), and the same goes for all these. Research is a bit weird!</p><p>The way ChatGPT and LLMs create whole paragraphs of text is by playing a word guessing game, over and over.</p><p>Here’s the rundown:</p><p>1. You give the model a prompt (this is the “predict” phrase).<br>2. It predicts a word based on the prompt.<br>3. It predicts a second word based on the first word.<br>4. It predicts a third word based on the first two words.<br>5. …</p><p>It’s pretty basic when you break it down. But it turns out the word guessing game can be super powerful when your model is trained on all the text on the entire internet. Data Scientists often say about ML models, “garbage in means garbage out” — meaning your models are only as good as the data you use to train them. With OpenAI partnering with Microsoft, they’ve been able to use a ton of computing resources to gather this data and train these models on powerful servers.</p><p>With the entire written internet as context, LLMs can produce sentences that rarely deviate from something “normal,” unlike older models. If the phrase “I Angostura my cocktail with Manhattan ice around glass twist” doesn’t exist anywhere on the web, the model probably won’t generate it. And this simple truth is a big reason why these models are so good.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rOT-wCjWdxxfA-gTZHan7w.png" /><figcaption>ChatGPT gathered information from various cocktail recipes on the internet and used this data to help decide which words to generate.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*QDYGzWLc9UfkSlpvNW09Ug.jpeg" /><figcaption>Do the large language models “understand”?</figcaption></figure><p>This raises a key question: Do these LLMs really grasp the answers they provide? Figuring this out involves a mix of math, philosophy, and semantics (what does “understand” really mean?).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ce7b26642ae4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Come Up With a Viable B2B Startup Idea]]></title>
            <link>https://sharinkhander.medium.com/how-to-come-up-with-your-next-b2b-startup-idea-da8759fd8d7c?source=rss-4e0edb9cb39e------2</link>
            <guid isPermaLink="false">https://medium.com/p/da8759fd8d7c</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[b2b]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[startup-lessons]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Sharin Khander]]></dc:creator>
            <pubDate>Sun, 17 Dec 2023 19:01:49 GMT</pubDate>
            <atom:updated>2023-12-27T16:20:04.994Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Sharin Khander Medium startup idea" src="https://cdn-images-1.medium.com/max/1024/1*0R78eG9s6koldlQG3LaekQ.png" /></figure><p>I’ll delve into the origin stories of B2B powerhouses, and explore the spark that ignited their journeys. We’ll dissect the thought processes of founders, the problems they tackled, and the ingenious solutions they crafted.</p><h3><strong>What defines an excellent B2B startup idea?</strong></h3><p>It all boils down to 3 essential elements:</p><p><strong>Element 1: The problem matters to people.</strong></p><ul><li>Many people should be eager to spend money on solving the problem. Startups often struggle not because their idea is bad, but because the market for the solution is too limited.</li><li>To build a sizable business, aim for a path to $100 million in annual revenue and eventually reach $1 billion.</li><li>For a useful framework, explore Hunter Walk’s <a href="https://hunterwalk.com/2017/04/12/why-i-care-about-problem-size-more-than-market-size/">LUV</a>. Additionally, find insights on <a href="https://medium.com/sequoia-capital/the-market-curve-44097b626f6d">what defines a venture-scale idea and a guide</a> to help you assess your market size.</li></ul><p><strong>Element 2: An underserved market</strong></p><ul><li>Current solutions must be subpar in addressing the problem.</li><li>Kunal Shah, founder of CRED, offers a valuable framework named Delta-4: <em>How would customers rate the existing solution on a scale of 1 to 10? How would they rate your solution?</em></li><li>For your solution to matter, it needs to be +4 better. If the existing solution scores 6 or higher, replacing it becomes challenging (think Excel).</li></ul><p><strong>Element 3: Your passion for solving the problem</strong></p><ul><li>Choose a problem you’re eager to dedicate years of your life to solving.</li><li>In the words of Dylan Field, co-founder of Figma: <em>“If you are, let’s say, three to four years in on an idea that you hate, you’re just going to burn out and you’re going to quit. It won’t feel good, and you’ll be hating life. Don’t just go for an idea because it’s kind of working. Go for an idea that you really care about because even if it doesn’t work, you’ll still learn from it, and you’ll still have one.”</em></li><li>Takeaway: Choose an idea that solves an important, underserved problem that excites you. More guidance on how to do this and what it entails is provided below.</li><li><em>“It’s essential to work on something you’re deeply interested in. Interest will drive you to work harder than mere diligence ever could. When in doubt, optimize for interestingness.”<br> — </em><a href="http://paulgraham.com/greatwork.html#f1n"><em>Paul Graham</em></a></li></ul><figure><img alt="Sharin Khander B2B startup idea" src="https://cdn-images-1.medium.com/max/1024/1*OwYrU3bKXLRQtklZUCWe1Q.png" /></figure><h3>Discovering a standout B2B startup idea</h3><p>There are 3 dependable ways to find a great idea:</p><p>1. <strong>Past pain: </strong>Spot a significant challenge you faced in a previous job — then create a solution for it.<br>2. <strong>Ponder and probe:</strong> Choose an area you’re keen on, brainstorm and experiment while engaging with numerous potential customers — keenly searching for challenges and interest.<br>3. <strong>Present pull:</strong> Recognize something you’ve already built that is gaining traction — and shift your focus entirely to that.</p><p>Let’s take a look at how the largest B2B companies today discovered their ideas:</p><figure><img alt="sharin khander startup ideas" src="https://cdn-images-1.medium.com/max/1024/1*EL5By4cavoTS4XtnVHM7Vw.png" /></figure><h3>Path 1 — Past Pain for B2B Startup Ideas</h3><p>Expected this to be the primary path for great startup ideas, but it only accounts for around 40% of them.</p><h3>Examples of Success:</h3><p>1. <a href="https://www.gong.io/"><strong>Gong:</strong><br></a> — Pain: Understanding sales process issues.<br> — Solution: A system capturing sales insights for organizational visibility.<br> — Founder Quote: “The CRM was showing me stuff, but it wasn’t anything meaningful.”</p><p>2. <a href="https://retool.com/startups?hsa_ver=3&amp;_keyword=retool%20startups&amp;hsa_ad=613307028855&amp;hsa_acc=7420316652&amp;hsa_grp=138156054183&amp;hsa_cam=17502105052&amp;utm_campaign=17502105052&amp;utm_medium=search&amp;hsa_src=g&amp;utm_source=google&amp;gad_source=1&amp;hsa_mt=e&amp;gclid=CjwKCAiA1fqrBhA1EiwAMU5m_0naTLb2GxhEstkOgbTqwgvO76BZhAKI7AufA25JuLW_2Rk43aHK2RoCwLsQAvD_BwE&amp;utm_content=613307028855&amp;hsa_tgt=kwd-2065824871850&amp;utm_term=retool%20startups&amp;hsa_net=adwords&amp;adgroupid=138156054183&amp;hsa_kw=retool%20startups"><strong>Retool:</strong></a><br> — Pain: Building similar internal tools repeatedly.<br> — Solution: Recognizing common building blocks and creating a universal tool.<br> — Founder Quote: “There’s got to be a better way of doing this.”</p><p>3. <a href="https://linear.app/"><strong>Linear:</strong></a><br> — Pain: Disappointment with Jira for project management.<br> — Solution: Creating Linear to address issues with existing tools.<br> — Founder Quote: “This doesn’t feel right, and we should do something about it.”</p><p>4. <a href="https://withpersona.com/"><strong>Persona:</strong></a><br> — Pain: Inadequacy in identity tooling at Square.<br> — Solution: Identifying a unique approach to address identity verification challenges.<br> — Founder Quote: “There was a disconnect between existing identity vendors.”</p><p>5.<strong> </strong><a href="https://hex.tech/"><strong>Hex:</strong></a><br> — Pain: Inability to find a needed solution as a buyer.<br> — Solution: Taking the initiative to build what was lacking in the market.<br> — Founder Quote: “I didn’t set out to be a founder. The idea was very obvious to me.”</p><p><strong>Reflection Questions:</strong><br>1. What valuable tools were built at previous jobs for the company or teammates?<br>2. What tool do you wish existed (and would pay for) from your past work experiences?<br>3. Identify internal products repeatedly built at previous companies.</p><h3>Path 2 — Ponder and Probe for B2B Startup Ideas</h3><p>Key Approach:<br>- <strong>Action: </strong>Whiteboard and tinker, but validate ideas by talking to potential users.<br>- <strong>Focus:</strong> Find a problem space with traits: (1) importance, (2) underserved, and (3) personal excitement.<br>- <strong>Tip: </strong>Root the idea in experience, if possible.</p><h3>Examples of Success</h3><p>1.<strong> </strong><a href="https://www.figma.com/"><strong>Figma:</strong></a><br> — Approach: Focused on design after narrowing down potential areas.<br> — Insight: Recognized the significance of drones and WebGL for new possibilities.<br> — Founder Quote: “It makes so much sense on the whiteboard. And in reality, zero sense.”</p><p>2. <a href="https://www.vanta.com/demo?cq_src=google_ads&amp;cq_cmp=19942392160&amp;cq_con=153569869571&amp;cq_term=vanta&amp;cq_med=&amp;cq_plac=&amp;cq_net=g&amp;cq_pos=&amp;cq_plt=gp&amp;utm_term=&amp;utm_campaign=Branded-ALL-NAMER&amp;utm_source=google&amp;utm_medium=cpc&amp;utm_content=&amp;hsa_acc=4880914058&amp;hsa_cam=19942392160&amp;hsa_grp=153569869571&amp;hsa_ad=654204747745&amp;hsa_src=g&amp;hsa_tgt=kwd-26743636&amp;hsa_kw=vanta&amp;hsa_mt=e&amp;hsa_net=adwords&amp;hsa_ver=3&amp;gad_source=1&amp;gclid=CjwKCAiA1fqrBhA1EiwAMU5m_8T_BB_c9h-1o48-F-AuyHemU5XoUvSJ51qk3H2GQTwBoJXzpWtEvhoCbfEQAvD_BwE"><strong>Vanta:</strong></a><strong><br></strong> — Approach: Explored team collaboration and security, going deep to find the biggest pain.<br> — Insight: Avoided the abstract by choosing specific spaces to explore.<br> — Founder Quote: “Pick one or two spaces and go really deep.”</p><p>3. <a href="https://www.notion.so/product"><strong>Notion:</strong></a><br> — Approach: Began as a no-code website builder, pivoted to focus on docs and wikis.<br> — Insight: Discovered user preference for collaborative features in the editor.<br> — Founder Quote: “Peeling the onion…we should double down on docs and wiki.”</p><p>4.<a href="https://www.ycombinator.com/companies/zip"><strong> Zip:</strong></a><br> — Approach: Explored multiple ideas, eventually focused on procurement.<br> — Insight: Sought large markets with entrenched but subpar companies.<br> — Founder Quote: “Look for a market that was large and had entrenched companies that weren’t great.”</p><p>5. <a href="https://www.databricks.com/resources/ebook/state-of-data-ai?scid=7018Y000001Fi0MQAS&amp;utm_medium=paid+search&amp;utm_source=google&amp;utm_campaign=14636097724&amp;utm_adgroup=127320330459&amp;utm_content=ebook&amp;utm_offer=state-of-data-ai&amp;utm_ad=662844269736&amp;utm_term=databricks&amp;gad_source=1&amp;gclid=CjwKCAiA1fqrBhA1EiwAMU5m_9GXNUYQHZHU3hh3GGcUspek0YZR5NYkN4FrvlleaxvvMSLCLIwLyRoCcMwQAvD_BwE"><strong>Databricks:</strong></a><br> — Approach: Identified underutilized technology (Apache Spark) and aimed for a transformative impact.<br> — Insight: Realized the gap between academic projects and enterprise-ready solutions.<br> — Founder Quote: “Maybe we can really actually ourselves make it happen.”</p><p><strong>Reflection Questions:</strong><br>1. What emerging but underserved trend can you identify (e.g., security, collaboration)?<br>2. What transformative but underutilized technology could you leverage (e.g., data scalability)?<br>3. How many potential customers have you engaged with regarding your idea?<br>4. On a scale of 1 to 10, how important is the problem to potential customers?<br>5. On a scale of 1 to 10, how underserved is the current market, and where does your solution rank? Aim for a delta of at least 4.</p><h3>Path 3 — Present Pull for B2B Startup Ideas</h3><p><strong>Key Approach:</strong><br>- Action: Identify features or functionalities within your current projects that exhibit strong user pull.<br>- Focus: Pivot fully towards the aspects showing clear signs of demand and interest.<br>- Tip: Be attentive to what users are gravitating towards within your existing work.</p><h3>Examples of Success:</h3><p>1. <a href="https://amplitude.com/amplitude-vs-google-analytics?utm_source=google&amp;utm_medium=cpc&amp;utm_campaign=sitelinks&amp;utm_source=adwordsb&amp;utm_medium=ppc&amp;utm_campaign=sitelinks&amp;utm_term=amplitude&amp;gad_source=1&amp;gclid=CjwKCAiA1fqrBhA1EiwAMU5m_2Vj8CnIP7gNLfz-ZZ424YGKwXEw0cpZDjOgWzzKecvVinLPz-jHtBoCwS8QAvD_BwE"><strong>Amplitude:</strong></a><br> — Approach: Recognized a gap in analytics tools while developing a voice recognition app.<br> — Insight: Found market demand by sharing insights with others.<br> — Founder Quote: “They were like, ‘Wow, that’s amazing. I really need to understand that about my business.’”</p><p>2. <a href="https://sprig.com/"><strong>Sprig:</strong></a><br> — Approach: Initially exploring one startup idea, pivoted based on market demand.<br> — Insight: Used outcome-driven innovation to add rigor to product development.<br> — Founder Quote: “I’ll pay you for this. This was the market pull I was looking for.”</p><p>3.<a href="https://www.loom.com/"><strong> Loom:</strong></a><br> — Approach: Evolved from two failed video-related projects, identified unexpected user pull.<br> — Insight: Discovered potential through a unique feature during client interaction.<br> — Founder Quote: “That made it clear to us that we should double down on this new direction.”</p><p>4. <a href="https://segment.com/"><strong>Segment:</strong></a><br> — Approach: Created Analytics.js as a growth hack, noticed positive user response.<br> — Insight: Discovered potential in being an ‘API layer’ for data tools.<br> — Founder Quote: “That day, it goes straight to the top of HN, and the rest is history.”</p><p>5. <a href="https://slack.com/"><strong>Slack:</strong></a><br> — Approach: Transitioned from an unsuccessful video game project, identified effective communication tools.<br> — Insight: Leveraged IRC features that users gravitated toward.<br> — Founder Quote: “We spent a long time working on reference letters…while we were thinking about what we might want to do next.”</p><p><strong>Reflection Questions:</strong><br>1. What feature or functionality in your current projects is exhibiting the most user pull?<br>2. What problems are you currently facing in your startup that you wish someone would solve? Consider solving them yourself.<br>3. Is there a side project you’ve been wanting to build but haven’t yet? Explore its potential and user interest.</p><h3>In a nutshell, if you’re aiming for a standout B2B startup idea:</h3><p>Look for ideas that are (1) important (2) underserved, and (3) you’re excited to solve.</p><h3>Choose one of these paths and consider these questions:</h3><p><strong>Path 1: Past pain</strong><br>- What valuable solutions did you or others create in previous jobs?<br>- Identify a tool you wish existed (and you’d pay for) in your past workplaces.<br>- Reflect on recurring internal products you built in previous companies.</p><p><strong>Path 2: Ponder and probe</strong><br>- Identify emerging, underserved trends (e.g., security, collaboration).<br>- Recognize transformative yet underutilized technologies (e.g., data scalability).<br>- Gauge the importance (1 to 10) and underserviced nature (1 to 10) of the problem you’re exploring.</p><p><strong>Path 3: Present pull</strong><br>- Pinpoint the one feature gaining the most traction from what you’ve built.<br>- Identify a problem in your current startup that needs solving; consider solving it yourself.<br>- Explore the side project you’ve been wanting to build but haven’t.</p><p>You’re onto something promising when you detect signs of pull and pain. Now armed with this framework, assess whether your idea is worth a full-fledged commitment.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da8759fd8d7c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Do We Mean By Serverless Architecture?]]></title>
            <link>https://sharinkhander.medium.com/what-do-we-mean-by-serverless-architecture-caefd5bee64b?source=rss-4e0edb9cb39e------2</link>
            <guid isPermaLink="false">https://medium.com/p/caefd5bee64b</guid>
            <category><![CDATA[serverless-architecture]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[serverless]]></category>
            <dc:creator><![CDATA[Sharin Khander]]></dc:creator>
            <pubDate>Thu, 14 Dec 2023 19:03:20 GMT</pubDate>
            <atom:updated>2023-12-15T17:49:50.776Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is Serverless?</h3><p><em>It’s the future, y’all.</em></p><figure><img alt="Sharin Khander serverless architecture" src="https://cdn-images-1.medium.com/max/786/1*oFK1crKTtaxcJ1nqir8HhA.png" /></figure><h3>The TL;DR</h3><ul><li>Serverless has become a catchall for applications and products that operate without the need for teams to manage their own servers.</li><li>Traditional applications run on servers, requiring complex and resource-intensive management.</li><li>In practical terms, serverless often adopts a function-based architecture instead of a traditional app-based one.</li><li>Leading cloud providers, such as AWS, provide serverless platforms, with AWS Lambda being a prime example.</li></ul><h3><strong>Diverse Interpretations and Growing Popularity</strong></h3><ul><li>The term “serverless” is interpreted in various ways by different individuals.</li><li>Despite these diverse meanings, serverless is gaining popularity and holds the potential to influence the future of app development.</li><li>The FUTURE!</li></ul><h3><strong>Context: Backend Architecture Today</strong></h3><p>As a developer crafting your app, consider how you deploy your backend. Most apps comprise two key elements:</p><p>1. <strong>Frontend: </strong>This is the user interface found in web browsers or phone apps.</p><p>2. <strong>Backend: </strong>This houses the app’s logic and data, typically involving a database and APIs.</p><p>Take Medium as an example.</p><ul><li>Medium’s frontend, built with HTML, CSS, and Javascript, allows me to write posts, view analytics, and send emails.</li><li>The Medium backend stores content, manages recipients and stats, and incorporates APIs for mail delivery and subscription additions.</li><li>The frontend and backend collaborate to present what you see on your favorite app’s page.</li></ul><p>Developing and deploying the backend is often the most intricate task, enduring the most “stress” and requiring scalable solutions. As user numbers rise, backend components, such as databases, need to efficiently handle an increasing number of queries, illustrating the potential complexity.</p><p>When ensuring your database and APIs can handle your app’s traffic, you have a couple of choices:</p><p>- <strong>Scaling Vertically: </strong>Boost the capacity of the server running your database — make it bigger, faster, and stronger.<br> <br>- <strong>Scaling Horizontally:</strong> Duplicate your server and distribute the load among multiple copies.</p><p>Scaling vertically is straightforward (just instruct AWS to consume more of your budget), but it’s not sustainable or cost-efficient. In contrast, scaling horizontally is notoriously challenging.</p><p>Scaling isn’t your only hurdle when running your backend. Guaranteeing server performance involves stack-dependent tasks, such as:</p><p>1. Setting up servers, installing necessary components, and so on.<br>2. Monitoring your app to ensure smooth request processing.<br>3. Keeping an eye on hardware to check CPU, RAM, and Disk utilization.</p><p>These are just a few of the challenges engineers (especially those with a DevOps focus) face when managing their own servers.</p><h4>🚨 Don’t Be Confused</h4><p>Scaling requirements extend to frontends as well. Components that act as a bridge between the frontend and backend — such as when using a web framework like NextJS — must operate on a server. While our focus is primarily on the backend in this discussion, it’s important to note that this is a somewhat simplified perspective.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*himU9fHeUwR5QqUb-d-D7A.png" /></figure><p>To navigate through the complexities, let’s consider three definitions of serverless below. The third and final one is particularly crucial, as it aligns with what developers generally mean when discussing serverless.</p><p>1) <strong>Serverless as SaaS:</strong><br>When developers outsource any part of the software stack to a third party via Software as a Service (SaaS), it qualifies as serverless. Examples include using GitHub for code hosting, Auth0 for app authentication, Datadog for monitoring, and Stripe for payments. While acknowledging that these SaaS solutions eliminate the need for developers to build and maintain their infrastructure, it’s not the typical sense of serverless. This is often referred to as the MBA definition — i.e., it’s considered less relevant.</p><p>2) <strong>Serverless as PaaS:</strong><br>Descending further into the infrastructure layer, outsourcing parts of your infrastructure to managed services from cloud providers is another interpretation of serverless. Examples include Vercel and Netlify for frontends, Confluent for event busses, and S3 for object storage. Terms like “serverless database” or “serverless frontend” apply here.</p><p>3) <strong>Serverless as Functions:</strong><br>The most specific and commonly referenced definition of serverless by developers is deploying apps through functions instead of monolithic apps and endpoints. Each major cloud provider offers its own product for this:<br> — AWS has Lambda<br> — GCP has Cloud Functions<br> — Azure has Functions<br>In essence, with serverless architecture, you don’t just avoid managing servers; you deploy your app as a series of separate functions, a departure from the traditional approach. Serverless, in this context, is both an architectural decision and a product.</p><p>So when encountering the term “serverless,” it’s crucial to consider context. Some might use the term to sound sophisticated, but as explored, not all “serverless” products necessarily replace traditional server functionalities. Approach with skepticism 🙂.</p><h3>AWS Lambda and Its Counterparts: The Serverless Essence</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/628/1*0vrTcNIk88unzR-r53iQ_Q.png" /></figure><p>When developers mention serverless, they’re often referring to platforms like Lambda. So, what exactly does Lambda (and its equivalents from other cloud providers) offer?</p><ul><li>In Lambda, your code is deployed as distinct, logically independent functions.</li><li>These functions can be triggered either manually or through defined events within the AWS ecosystem.</li></ul><p>Consider a scenario where you have a single function in your app responsible for inserting a new user into the database. Instead of running this function, along with all others, on a central server, you can upload them individually to Lambda. Subsequently, you can trigger these functions directly from your frontend. Remarkably, Lambda’s user interface even allows you to paste your code directly as a function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1fWUIPWik_ShAYYgogw4LA.png" /></figure><h3>The Irony of AWS Lambda and Its Not-So-Serverless Reality</h3><p>When you trigger a Lambda function, AWS takes the reins, either by spinning up a new server or allocating it to an existing one — hence, the term serverless.</p><p>Yet, it’s ironic that Lambda, often hailed as a serverless solution, isn’t entirely devoid of servers. Here’s why:</p><ul><li>Lambda isn’t a database; you still need one and must contend with scalability concerns.</li><li>Sizing Lambda functions is necessary; you must pre-allocate a specific amount of RAM to them. Sounds a bit like managing a server, doesn’t it?</li></ul><p>In practice, Lambda and its counterparts are often employed for non-core aspects of an app. Many customer case studies focus on areas like image pipelines, analytics, or general machine learning. It’s rare to find high-growth startups running their core app logic on Lambda, or so it seems for now.</p><h4>🔍 DEEP DIVE</h4><p>AWS crafted a proxy service specifically for their managed database service, RDS, facilitating its connection to functions operated through Lambda. Strangely enough, on the features page of their “serverless” product, they guide you on handling the “server-full” components of your app. It’s a little SUS, IMO.</p><figure><img alt="Sharin Khander software engineer" src="https://cdn-images-1.medium.com/max/734/1*CIEKcdxciaZbAbe9y029VQ.png" /></figure><h3><strong>Conclusion</strong></h3><p>There’s a challenge with function-driven architectures these days — the developer experience isn’t quite up to par, especially when it comes to developing functions locally and then moving them to the cloud. For example, I had a frustrating experience with GCP Cloud Functions where testing specific data types was a headache due to limited tools for mocking or running on my laptop. That’s why frameworks like SST (Serverless Stack) are emerging to enhance the developer experience for platforms like Lambda.</p><p>On the flip side, the database space is evolving with more intriguing developments. Fauna claims to be serverless, charging solely based on usage, not on server size or similar metrics. BigQuery, a widely used data warehouse, is undeniably serverless. Although we may be on the brink of advancements, it’s definitely a trend worth keeping an eye on.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=caefd5bee64b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Low-Code Development: A Comprehensive Guide for Tech Enthusiasts and Beginners]]></title>
            <link>https://sharinkhander.medium.com/low-code-development-a-comprehensive-guide-for-tech-enthusiasts-and-beginners-097e2f8b454d?source=rss-4e0edb9cb39e------2</link>
            <guid isPermaLink="false">https://medium.com/p/097e2f8b454d</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[low-code]]></category>
            <category><![CDATA[low-code-development]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[product-management]]></category>
            <dc:creator><![CDATA[Sharin Khander]]></dc:creator>
            <pubDate>Mon, 11 Dec 2023 20:23:57 GMT</pubDate>
            <atom:updated>2023-12-12T15:34:15.224Z</atom:updated>
            <content:encoded><![CDATA[<p><em>What is “low code” and why do developers cringe at the mention of it?</em></p><h3>TL;DR</h3><ul><li>Traditional software uses skilled &amp; costly engineers, who build from the ground-up using coding languages like JavaScript or Python.</li><li>Low-code is a visual development approach with less code, using drag-and-drop interfaces and templates.</li><li>Low-code spectrum includes ‘no-code’ platforms with no coding, progressing to ‘developer tools’ with more code for enhanced functionality.</li></ul><p>Here are some examples of low-code tools you might have seen before:</p><figure><img alt="Sharin Khander low code" src="https://cdn-images-1.medium.com/max/1024/1*FIztS98EW01JrRE4NVmoOA.png" /></figure><p>To help you understand better, let’s explore what the usual process looks like when building an app with traditional code.</p><h3>What’s it like to build an app from scratch with code?</h3><p>When creating software, there’s a lot to think about in the development process, known as the development lifecycle. If you’re building from scratch or using a low-code platform, the details of this lifecycle can vary.</p><p><em>📌 Note: Since I’m more of an web app girlie, I’m going to focus on that specific development process. But the concepts are reasonably universal for mobile apps, native apps, etc.</em></p><h3>Setting up your backend</h3><p>Software typically consists of a backend and frontend.</p><ul><li>The frontend interacts with the user and calls the backend for data.</li><li>The backend manages data and logistics behind the scenes.</li><li>Setting up the backend involves choosing a database type (SQL or NoSQL) and platform (MySQL, PostgreSQL, MongoDB).</li><li>Decisions depend on various factors, beyond the scope of this post.</li></ul><p>Once you’ve got everything set up, the next step is designing your database. This means creating rules for how data is stored and organized, including:</p><p><strong>- What type of data it is (text, number, yes/no, etc.)<br>- How it’s organized (different tables and their relationships)<br>- Names for everything (columns)</strong></p><p>After choosing your database type, you set it up on a server or cloud instance like AWS or GCP, often with staging and production versions for safe testing.</p><p>Once this is done, your backend is ready to store and manage data. For example, when someone signs up for your app, their data can be stored and easily retrieved when needed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JeMapekBuGhdRSShqn129g.png" /></figure><h3>Now, let’s set up your frontend.</h3><p>This is where you create the graphical user interface (GUI) and link it to your data. You’ll be coding components, actions, and more. Hosting is required, and sometimes it’s on the same server as your backend.</p><p>Frontends and backends can be built using various programming languages. Traditionally, HTML, CSS, and JavaScript were for the frontend, while the backend used languages like Java or C++. However, JavaScript has gained popularity for the backend in recent years. It’s important to note that developers often use prebuilt components and capabilities from software libraries, saving time and effort.</p><p>During the frontend step, you’ll go back and forth between developing frontend and backend elements. This involves making and debugging connections, adjusting your data schema as needed, and integrating new resources. Software engineering is an iterative process, and your frontend and backend work closely together.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UV5oSRRDH3ybuooOj0Rm4g.png" /></figure><h3>App Management</h3><p>Once your main product is good to go, most apps need additional features to manage how they’re used and improved. This includes:</p><p><strong>Permissions:</strong><br> — Decide who can do what in the app, like reading or editing data.</p><p><strong>CI (Continuous Integration):</strong><br> — Test the app regularly to catch and fix issues early on.</p><p><strong>CD (Continuous Deployment):</strong><br> — Quickly release new versions of the app to users.</p><p><strong>Observability:</strong><br> — Keep an eye on the app’s speed and reliability.</p><p><strong>Documentation:</strong><br> — Provide clear instructions on how to use the software.</p><p>While there’s complexity involved, the specific requirements can vary based on the organization. These are some fundamental aspects to consider.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zw7JonNlR6if7YHU-px6mA.png" /></figure><h3>Maintainence and Iterations</h3><p>After finishing the initial work, you transition from building to maintaining your product. Products usually get better over time with new features, bug fixes, and enhancements for users.</p><p>It’s important to mention that these steps don’t cover the internal team discussions during planning, scoping the application, refining, and iterating on versions. In my experience, it often feels like the classic challenge of having too many people involved, all contributing ideas — imagine a lot of cooks in the kitchen, but with a very patient engineer at the center.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S3mxxDCO_X0Tvahvpi4D6Q.png" /></figure><p>In short, creating apps from scratch with code is often a challenging task. The time it takes can vary widely — simpler apps might be ready in a day, while more complex ones can take months or even years to develop.</p><h3>What’s It Like Building a “Low-Code” App?</h3><p>Building an app with low-code tools can be similar or different from the traditional method we talked about earlier. The main point is this: using low-code makes app building faster and more accessible for people who aren’t super technical. However, there are some tradeoffs.</p><p>Firstly, low code allows more people to get into software development without years of computer science study. For example, if your marketing team needs a BI tool and you already have a database, they might use a low-code tool to build it themselves. If you’re a self-employed dog walker and need a website, you can probably create one quickly using a low-code platform like Squarespace or Wix.</p><p>The process of building an app with low code is usually simpler.</p><h3>Setting up your backend:</h3><p>- If you don’t have a backend, you can create one using low-code tools like Airtable or Google Sheets. These tools have user-friendly interfaces to interact with your data without needing a backend engineer.<br>- If you already have a database, you can connect it to your low-code frontend builder using a REST API or integration.<br>- In some cases, like most website builders, the database is integrated seamlessly, and you might not even notice it’s there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6Tyd7hXunML2CGsruAahJw.png" /><figcaption>Setting up a database in AirTable is as simple as an Excel Spreadsheet.</figcaption></figure><p>Of course creating columns and adding data in Airtable is much easier than doing it with SQL in PostgreSQL. Whether Airtable qualifies as a “database” is a complex discussion, but for basic apps, it’s a great low-code choice.</p><p>Most platforms will automatically host your app or automation in the cloud, eliminating the need for the ‘finding a server’ step.</p><h3>Setting up your frontend:</h3><p>Components in apps are typically pre-built and drag-and-drop. This saves a lot of time and effort, sparing you from writing hundreds of lines of code to create basic elements like tables, buttons, and forms. These elements are standard in almost every app and can be a hassle to recreate each time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*x9Av_YvWQ_2dBURtSPIzZA.gif" /><figcaption>Retool allows you to drag-and-drop components onto a canvas and then customize them using their UI or code.</figcaption></figure><h3>The App Management Nitty-Gritty</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bhGKorWYotO-jNh7IvVcXg.png" /><figcaption>Platforms you’re already using often have infrastructure mgmt options like debugging tools built-in.</figcaption></figure><h3>Maintainence &amp; Iterations</h3><p>Because each element is easier to build with low code, making changes and fixes is quicker. Maintenance is simpler too, as the platform itself keeps many core computing elements up to date, and it takes care of things like API connections and servers. The platform usually notifies you if something isn’t working correctly.</p><p>These simplifications free up builders from repetitive or lengthy coding tasks. Instead, they can focus on core products or critical tools, creating unique functionality in their apps without worrying too much about building the basic components.</p><p>While some people in the software industry, especially marketers of low-code tools, see low code as a game-changer, many developers may have a different perspective. But why?</p><h3>What’s the downside?</h3><p>Does it seem too good to be true? Well, to a certain extent, it is. While low-code can simplify development, it comes with limitations, and the level of customization isn’t as flexible as traditional coding.</p><p>Here’s the catch:</p><p>1. <strong>Hitting the wall:</strong> Developers often resist low-code tools because they can hit a roadblock in terms of customization. With traditional code, you have more control, but with low-code, your app is limited to what the tool allows.</p><p>2. <strong>Learning curve:</strong> Users and developers using a low-code tool need to learn its unique features, which can be time-consuming. This is true for traditional code as well, but if you’re a developer, you’ve likely already started that journey.</p><p>3. <strong>Cost considerations:</strong> Using these builders extensively can become expensive, especially if they don’t cover all your needs.</p><p>4. <strong>Proprietary nature:</strong> Low-code tools often use their own language and building methods, making it challenging to transfer to other platforms, creating vendor lock-in.</p><p>Despite these limitations, low-code platforms aren’t useless. Depending on the app you’re building, you can get cool products quickly. However, it requires research to find the right tool for the job. Just like anything else, picking the right tool is crucial.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/726/1*vPKQWx2NaokSNRlOqw4-QQ.png" /><figcaption>What if I told you low code doesn’t make you less of a developer?</figcaption></figure><h3>Considering the downsides, what are low-code tools good for?</h3><p>1. <strong>Creating MVPs and Prototypes:</strong><br> — Many companies use low-code platforms to build a minimum viable product (MVP) or a prototype. This serves as a temporary solution until they’re ready to build a fully customized product from scratch.</p><p>2. <strong>Fast Growth in Startups:</strong><br> — Startups in rapid growth phases find low code useful. It allows for quick experimentation and building without heavy investment in a custom solution, which may change as the startup evolves.</p><p>3. <strong>Pitching Ideas and Budget Planning:</strong><br> — Having an MVP is excellent for pitching product ideas to budget holders and funders. It provides a tangible demonstration of the concept.</p><p>4. <strong>Quick Solutions for Non-Technical Teams:</strong><br> — Low code is beneficial for teams without engineering resources. It empowers semi-technical users to quickly solve certain tasks without extensive coding knowledge.</p><p>5. <strong>Developer-Focused Tools:</strong><br> — Some low-code tools like Retool, Appsmith, and OutSystems target developers to speed up engineering time on internal tools. These tools are gaining popularity as their limitations decrease, allowing developers to build faster.</p><p>6. <strong>Streamlining Development Efforts:</strong><br> — Incorporating low-code engineering tools into your development stack is a way to streamline efforts and save engineers time. While critical tools might not be suitable for proprietary low-code platforms, many mid-level apps could benefit from this approach, reducing the workload on engineers by around 70–80%.</p><h3>Conclusion</h3><p>Ultimately, low-code development is a valuable addition to the software development landscape. The key is understanding project needs and choosing the right tool — whether traditional coding for customization or low-code for rapid development. As technology evolves, striking the right balance between control and speed will be crucial for successful integration into the broader software development ecosystem.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=097e2f8b454d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Databases 101: A Simple Guide for Non-Technical Users]]></title>
            <link>https://sharinkhander.medium.com/databases-101-5e5464430b74?source=rss-4e0edb9cb39e------2</link>
            <guid isPermaLink="false">https://medium.com/p/5e5464430b74</guid>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[software]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Sharin Khander]]></dc:creator>
            <pubDate>Fri, 08 Dec 2023 21:55:16 GMT</pubDate>
            <atom:updated>2023-12-11T18:57:32.713Z</atom:updated>
            <content:encoded><![CDATA[<p><strong><em>With 300+ databases out there, what do they all do?</em></strong></p><p>Just like choosing the right flour for baking, selecting the optimal database for your project requires careful consideration. Different databases excel at different tasks, and using the wrong one can significantly impact your project’s outcome.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T8eGBAZgvJWphI9JT9aYEw.jpeg" /></figure><p>Imagine the frustration of using all-purpose flour for pizza dough, only to end up with a disappointing, limp crust. Similarly, relying on a generic database for complex log analysis might lead to performance issues and hinder your project’s progress. That’s why your engineering lead might be requesting extra time to create a dedicated data store for log analysis — they’re essentially choosing the perfect flour for the task. By taking the time to select the right tool for the job, they are ensuring that your project has the best possible foundation for success.</p><h3>Finding the Right Data Flour: Why Your Choice of Database Matters</h3><p>Just like choosing the wrong flour can ruin your pizza, choosing the wrong database can lead to inefficient and inaccurate data management.</p><h3>Understanding Data and Its Usage</h3><p>To choose the right database, ask yourself these two key questions:</p><p>1. <strong>What does the data look like?</strong></p><p>While technically any data can be stored in any database, many are designed for specific data types or structures. For example, a graph database excels at handling interconnected data like social networks, while Snowflake shines with massive analytical datasets.</p><p>Remember, the data type isn’t just about storage; it significantly impacts the insertion and retrieval of data.</p><p>2. <strong>What do you need to do with the data?</strong></p><p>Different types of data and use cases demand different database functionalities. Consider the distinction between transactional and analytical databases:</p><ul><li><strong>OLTP (Online Transactional Processing):</strong> These are the core databases for applications, handling user data and frequent read/write operations. They prioritize data integrity and quick, small queries.</li><li><strong>OLAP (Online Analytical Processing):</strong> These databases are used for data science and analytics, working with large and complex queries on sporadically accessed data.</li></ul><p>The way users interact with the data necessitates different storage structures. OLTP and OLAP are just two examples, but numerous use cases and specialized databases exist.</p><h3>3 Main Types of Databases and Their Roles</h3><figure><img alt="Sharin Khander data" src="https://cdn-images-1.medium.com/max/1024/1*KGFv6rGB7mKiuLsS1bBmwg.png" /></figure><p>Here’s a simplified breakdown of 3 major categories of databases:</p><ol><li><strong>Databases for User-Facing Apps:</strong> These are the workhorses behind your application, storing all the data it needs to function properly. This includes user information, product details, transactions, and more. Think of them as the backbone of your app.</li></ol><p><strong>Examples: PostgreSQL, MySQL, MongoDB</strong></p><p>2. <strong>Databases for Analytics: </strong>Want to analyze user behavior, predict trends, or train machine learning models? These databases are your allies. They’re designed for fast and efficient querying of massive datasets, empowering data teams to unlock valuable insights.</p><p><strong>Examples: Snowflake, Redshift, BigQuery</strong></p><p>3. <strong>Databases for Operations:</strong> Keeping your app running smoothly requires continuous monitoring, logging, and security measures. These databases handle this critical task, ensuring efficient operation and troubleshooting.</p><p><strong>Examples: Elasticsearch, Prometheus, InfluxDB</strong></p><p>Remember, some databases may overlap categories. For instance, Redis, a NoSQL database, can be used for both app development and in-memory operations.</p><p>Understanding these categories and examples will help you select the right database for your specific needs.</p><h3>Databases that power a user-facing app</h3><p>The databases that drive a user-facing app, commonly known as production databases or production data stores, are the places where developers store and retrieve the data necessary for their app to function. All the information you see on your screen is stored in these databases.</p><ul><li><strong>If you’re Twitter</strong>: tweets, user profiles, trending topics, DMs</li><li><strong>If you’re Gmail</strong>: emails, settings, spam filters</li><li><strong>If you’re Amazon</strong>: orders, users, credit cards</li></ul><p>Production databases are built to support small and quick queries. The most important thing is data integrity and reliability — you don’t want to lose an order or a credit card.</p><h3>User-facing DBs / Relational Databases</h3><p>Relational databases are the OGs, having been in existence since the early days of database technology. They remain the go-to option for storing essential data in modern applications. In these databases, data is organized in a highly structured format with well-defined tables, columns, and relationships between tables. This structured setup ensures that queries can be executed swiftly and dependably.</p><ul><li><strong>Postgres</strong> is an open source relational database initially released back in 1997. It’s one of the standard “boilerplate” choices for teams getting started on a new app. You can run Postgres yourself on a server, or pay someone like Amazon to run it for you.</li><li><strong>MySQL</strong> is a free-to-use relational database that first came out in 1995. Many teams choose it as their go-to database. However, it’s worth noting that since Oracle acquired Sun (and thus MySQL), there are legal considerations if you want to change the MySQL code for commercial purposes. When comparing MySQL to Postgres, they are quite similar, but they do have some key differences in how they handle data types, storage engines, and a few other aspects.</li><li><strong>MSSQL</strong> is Microsoft’s own SQL database. It’s widely used by big companies but not as much by smaller startups.</li><li><strong>Oracle</strong> was the first-ever commercial relational database. Although it’s known for being a slow-moving company today, it still runs the apps for many Fortune 50 companies. Like MSSQL, Oracle is meant for large organizations, not startups.</li><li><strong>MariaDB</strong> is a free relational database created by the original makers of MySQL. They seem a bit unhappy with Oracle’s restrictions on MySQL contributions, so they highlight the open community and the fact that their software will never need a license.</li><li><strong>SQLite</strong> is a library that allows you to have a small database within your application. It’s used in many apps, typically as a middleman database rather than the main storage. According to the website, it’s on every Android, iPhone, and more.</li></ul><h3>User-facing DBs / NoSQL Databases</h3><ul><li><strong>NoSQL</strong> databases simplify things by getting rid of the strict structure of relational databases. You can just put your data in without worrying about the format right away. While there are various types of NoSQL databases (for analytics, operations), the ones mentioned here are specifically for user-facing and production use.</li><li><strong>MongoDB </strong>made NoSQL popular and is the go-to if you need a NoSQL database for your app. Their cloud service, Atlas, is like the AWS RDS but for basic NoSQL needs.</li><li><strong>Cassandra</strong> is a NoSQL database designed for big companies with massive data needs. It’s focused on storing and retrieving data quickly. Unlike MongoDB, which is a document database, Cassandra is columnar. This means it stores data in entire columns, similar to Snowflake. Using Cassandra is more similar to using a relational database.</li><li><strong>DynamoDB</strong> is AWS’s proprietary NoSQL database.</li><li><strong>Firebase</strong> is a set of tools (bought by Google a while ago) for making apps, especially for smaller teams who want simplicity. Their database is called Firestore, and it’s a popular choice when you need a quick and easy database to get started.</li></ul><h3>User-facing DBs / Graph Databases</h3><p>Each company’s data is unique, and for some, it’s best represented as a graph of connected nodes.</p><ul><li><strong>Facebook</strong>, for example, <a href="https://engineering.fb.com/2013/06/25/core-infra/tao-the-power-of-the-graph/">uses an in-house graph database called Tao</a> to manage their social connections. Graph databases are mainly used for user-facing applications, but they can also be handy for analytical purposes at times.</li><li><strong>Neo4J</strong> is the go-to graph database, and it’s open source. If you plan to scale up, you might need to check out their enterprise license. In Neo4J, data is shown as nodes, and these nodes can be linked in various ways. They use a language called Cypher, kind of like SQL, for querying the graph.</li></ul><h3>Databases that power analytics</h3><p>What a data team wants from their database is quite different from what app developers need. In data science and analytics, stored data is often repetitive, there’s a large volume of it, and queries involve combining information from various tables. Typically, someone at a computer doing research or a system creating a pipeline queries the data, and it gets added at regular intervals (like twice a day).</p><h3>Analytical DBs / Data Warehouses</h3><p>Data warehouses are like special databases designed for analyzing data, such as user activities or monthly revenue. They’re good at handling big, complex queries involving multiple tables. While they’re usually relational, the way they’re set up can vary.</p><ul><li><strong>Snowflake</strong> is a cloud data warehouse for analytics. It organizes data column-wise, which makes big queries faster, making it a popular choice for analytical databases.</li><li><strong>BigQuery</strong> is another cloud data warehouse for analytics and is quite similar to Snowflake.</li><li><strong>Redshift</strong> is also a cloud data warehouse for analytics, and it’s very much like Snowflake and BigQuery.</li><li><strong>ClickHouse</strong>, on the other hand, is an open-source data warehouse for analytics that gained popularity after being developed at Yandex., on the other hand, is an open-source data warehouse for analytics that gained popularity after being developed at Yandex.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/72/0*Rr3G8YLBx4faaEiE.png" /></figure><blockquote><strong><em>Why can’t I use Postgres as a data warehouse?</em></strong></blockquote><blockquote>You could use traditional methods, and some people still do. However, modern cloud data warehouses are much faster for handling big queries on substantial amounts of data. When it comes to databases, you can technically use any tool for any task, but there’s a point where you need a specialized tool to get the job done efficiently.</blockquote><h3>Analytical DBs / Data Lakes</h3><p>- <strong>Data Lakes</strong> are like huge safes for data. You can put in all sorts of data — organized, messy, big, small — and worry about organizing it when you want to use it. This is different from a data warehouse, which, like a relational database, has strict rules about tables, columns, and data types.</p><ul><li><strong>S3</strong> is Amazon’s storage solution, usually used for images and videos, but it’s also commonly used as a data lake. You can’t directly “query” S3 like a database, so you need another layer on top (like AWS’s Lake Formation) to help with that.</li><li><strong>Databricks</strong> has something called the “lakehouse platform,” which, underneath the marketing, is an open source data lake that acts like a data warehouse, blurring the line between the two.</li></ul><p>While you probably won’t see it much these days, HDFS used to be a popular way to build a data lake before the era of cloud data warehouses. It’s powered by Hadoop, an original framework for handling big data, but it’s known for being really, really hard to set up and run.</p><h3>Databases that power operations</h3><p>You won’t come across this type much unless you’re an engineer. The third category of databases is for data stores that help developers run internal operations. These include monitoring how well an application is doing, storing logs and security info, making apps faster, and acting as go-betweens for other databases.</p><h3>Operational DBs / Key Value Stores</h3><p>While most of the databases we’ve talked about store data permanently on a hard drive, there’s a special class that only keeps data in memory. These are for temporary data that you need to store and use quickly, even if it’s okay if it disappears later. These are called key-value stores (or KV stores) because they store data like a dictionary, where each entry has a key.</p><ul><li><strong>Redis</strong> is the original key-value store. It’s an in-memory database used for various things like building caches, managing authentication sessions, and real-time data retrieval for chat and messaging. Redis is open source, but you can also pay them to host it for you.</li><li><strong>SingleStore </strong>(formerly known as MemSQL) is another in-memory database like Redis, but it’s more focused on general-purpose workloads rather than the typical use cases of in-memory databases.</li></ul><h3>Operational DBs / Time Series Databases</h3><ul><li><strong>Time series databases</strong> are designed for storing data based on time, like daily financial records, second-by-second sensor readings, or hourly health checks on servers.</li><li><strong>Timescale</strong> is like an upgrade to Postgres, but it’s sold as a separate cloud-hosted database. It takes the familiar relational database and adds features specifically for handling time-series data, such as automatic partitioning and optimized queries.</li><li><strong>Prometheus</strong> is an open-source time series database that comes with built-in alerting and visualization features.</li><li><strong>InfluxDB</strong> is another open-source time series database.</li></ul><h3>Operational DBs / Logs and Search Databases</h3><ul><li><strong>Elastic</strong> is a NoSQL database made for storing and searching logs, which are detailed records of things like server performance, API requests, and internal information. Elasticsearch is often used with Kibana, its companion tool for visualizing data.</li><li><strong>Solr</strong> is another NoSQL database designed for storing and searching logs.</li></ul><p><strong>Conclusion</strong></p><p>From user-facing databases powering applications to analytical databases unlocking insights, and operational databases ensuring smooth operations, each type serves a distinct purpose. Understanding these distinctions helps in making informed choices, ensuring that, just like selecting the perfect flour for baking, the foundation for success is laid in the world of databases.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5e5464430b74" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Decoding Microservices]]></title>
            <link>https://sharinkhander.medium.com/decoding-microservices-9356dc50f59b?source=rss-4e0edb9cb39e------2</link>
            <guid isPermaLink="false">https://medium.com/p/9356dc50f59b</guid>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[engineering-management]]></category>
            <category><![CDATA[elon-musk]]></category>
            <dc:creator><![CDATA[Sharin Khander]]></dc:creator>
            <pubDate>Fri, 01 Dec 2023 16:48:15 GMT</pubDate>
            <atom:updated>2023-12-01T18:04:35.868Z</atom:updated>
            <content:encoded><![CDATA[<p>A little over a year ago Elon Musk sparked a Twitterstorm discussing Twitter’s backend problems, specifically targeting the issue of “microservices bloatware.”</p><figure><img alt="Sharin Khander Elon Musk tweet" src="https://cdn-images-1.medium.com/max/1024/1*uSIctN4LXaEqW1A7kaRGwQ.png" /></figure><p>I’ll dive into the world of microservices, unraveling their significance, and addressing the complexities they introduce.</p><h3><strong>Breaking Down Microservices: A TL;DR</strong></h3><p>Microservices offer a revolutionary approach to application development by breaking down traditional monolithic structures into small, independent services. Unlike the conventional model where all services run on the same infrastructure, microservices empower each unit to operate autonomously. Despite their effectiveness in scalability and downtime prevention, the adoption of microservices comes with its share of intricate challenges.</p><h3><strong>The Anatomy of Services</strong></h3><p>Before delving into the “micro” aspect, it’s crucial to understand what a “service” entails. In the context of application development, a service refers to any independent component handling specific functions, whether on the backend or frontend.</p><figure><img alt="Sharin Khander lyft" src="https://cdn-images-1.medium.com/max/1024/1*KKTmE52PJAxbsn29TNMuCw.png" /></figure><p>Using the example of Lyft, let’s imagine you open up the app on your phone and request a ride. A lot of things are happening behind the scenes:</p><ul><li><strong>Finding the drivers that are closest to you</strong></li><li><strong>Ensuring those drivers are vacant or finishing up a trip</strong></li><li><strong>Matching you with the optimal driver</strong></li><li><strong>Deciding what price to show you</strong></li><li><strong>Deciding what price to show the driver</strong></li><li><strong>Creating an ideal route for the driver to navigate to you</strong></li></ul><p>These could all be described as services. This is of course illustrative — I’m sure I’m missing plenty, Lyft’s backend is hella complex.</p><p>Here’s a simple way to think about it: Imagine services as tasks, and a good rule of thumb (though not always perfectly accurate) is to consider them in relation to API endpoints. Each task, like finding drivers or deciding on a price, might have its own special endpoint: a sort of specific spot it connects to in the system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ro_tpHT0W-kNHmc2_Ar-cg.png" /></figure><p>In reality, some tasks may not have their own special spots (endpoints), or some might, but they mostly use other spots. For example, the price task might use a big machine learning model, while the match task could use a simple method or some “if” statements. The closest drivers task might just read basic data from a database. We don’t need to know the details, and it’s not important. What matters is that each of these tasks is its own service and plays a specific role in the backend.</p><p><em>Point of Clarity: Not only are API endpoints and their rules considered services; other parts of your application, like the part you see (HTML, CSS, JavaScript), can also work on their own. They might even be split into their own microservices. Additionally, databases today are often copied across many servers to make sure they’re always available.</em></p><h3>Monoliths vs. Microservices: The Architectural Shift</h3><p>Historically, applications were developed using monolithic architecture, where all services coexisted within the same infrastructure. However, as companies like Lyft faced challenges in scaling, uptime, and specialized infrastructure, a paradigm shift towards microservices emerged. Microservices offer a more flexible and scalable approach, allowing services to operate independently, thus overcoming the limitations of monolithic development.</p><p>Imagine if Lyft had built its systems in a monolithic way. In that case, all the different tasks like finding drivers, determining prices, and creating routes would happen on the same server or be duplicated across identical servers. It’s like they’re all sitting together, doing their jobs side by side, and they might even share the same server with the database and the part you see on your app.</p><p>Building things this way, in one big group of code, is called monolithic development. It’s easy to do and makes sense for many applications because you don’t have to worry too much about where your code goes. For small applications, it usually doesn’t affect how well it works. But as companies like Lyft grew and more people started using their app, they faced some problems with the monolith approach:</p><p>1. <strong>Scale: </strong>Some tasks are used more than others. If they’re all on the same infrastructure, you can’t make one task work better without making all of them work better.<br> <br>2. <strong>Uptime:</strong> If one task has a problem and stops working, the whole system is affected.</p><p>3. <strong>Specialized Infrastructure: </strong>For things like machine learning or video processing, you need special tools like GPUs. Monolithic architecture doesn’t allow you to use these specialized tools efficiently.</p><p>4. <strong>Developer Experience: </strong>When all tasks are on the same system, making a change to just one task means you have to update and test the entire system. Plus, all tasks need to be written in the same programming language.</p><figure><img alt="sharin khander engineer microservices" src="https://cdn-images-1.medium.com/max/1024/1*GzvpI6h1jeV7JW7Tj5aA0A.png" /></figure><p>To overcome these issues, companies found a smarter way to build their services together.</p><p>In a microservices-based approach, each task is separated into its own independent piece. They can work on their own without depending on the others and communicate through something called an API endpoint. This diagram by Martin Fowler illustrates this concept well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SXwl4NjXocUHQHhSYIYPIQ.png" /></figure><h3><strong>Building Microservices in Practice</strong></h3><p>In a microservices-based architecture, services operate as independent entities, communicating through API endpoints. This separation enables effective scaling, improved uptime, utilization of specialized infrastructure, and a more tailored developer experience. The shift from monoliths to microservices is akin to separating goods into different shipping containers for optimal efficiency, reflecting the diversity and independence of software services.</p><figure><img alt="Sharin Khander microservices" src="https://cdn-images-1.medium.com/max/500/1*C77IvvAA3K4ZRzMKnoYQ7g.jpeg" /></figure><h3><strong>Complexity Costs and Ecosystem Considerations</strong></h3><p>While the benefits of microservices are evident, it’s essential to acknowledge the associated complexity costs. Managing a network of interacting services introduces challenges in authentication, communication, SLA agreements, and language disparities among development teams. No single solution works for both monolithic and microservices architectures. You should choose the one that fits your organization best, considering its specific constraints.</p><h3>Elon Musk’s Microservices Commentary</h3><p>Elon Musk’s Twitter commentary on Twitter’s backend sheds light on the ongoing debate around microservices. His remark about turning off “microservices bloatware” raises questions about the necessity and efficiency of incorporating microservices into a platform. The ensuing discussion emphasizes the need to strike a balance between functionality and streamlined architecture.</p><h3><strong>Conclusion</strong></h3><p>As Elon Musk’s Twitter commentary echoes through the engineering community, the debate on microservices continues to evolve. Whether your organization leans towards monoliths or microservices, understanding the nuances is key to making informed architectural decisions in the ever-evolving landscape of software development.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9356dc50f59b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Science Notebooks: A Must-Have in Your Team’s Arsenal]]></title>
            <link>https://sharinkhander.medium.com/data-science-notebooks-a-must-have-in-your-teams-arsenal-15b451e1cad2?source=rss-4e0edb9cb39e------2</link>
            <guid isPermaLink="false">https://medium.com/p/15b451e1cad2</guid>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Sharin Khander]]></dc:creator>
            <pubDate>Wed, 29 Nov 2023 18:37:04 GMT</pubDate>
            <atom:updated>2023-11-29T18:37:04.818Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*DYiuAhet9CmqLaqn8cyKXw.jpeg" /></figure><p>Have you ever wondered how data scientists transform raw code into actionable insights about users? Enter data science notebooks, or as some call them, “code notebooks.” These tools provide a bridge from a jumble of code to a neatly formatted, visual representation of iterative code blocks. In this post, we’ll unravel the mystery behind code notebooks, exploring why they’ve become a staple for data teams and developers alike.</p><h3>Data Science notebooks — or just “code notebooks” — help you go from this blob of code with churn data:</h3><figure><img alt="Sharin Khander pandas" src="https://cdn-images-1.medium.com/max/1024/1*qGye-56B0H_8nC_KLPCzRA.png" /></figure><h3>To this nicely formatted, iterative, visual set of code blocks:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IylICIMV2Y5mwkSqiB2YZg.png" /></figure><h3>To this beautifully visualized graph that tells a story:</h3><figure><img alt="sharinkhander pandas data" src="https://cdn-images-1.medium.com/max/1024/1*VhyBjFIdvVjt3MQKSVTJAQ.png" /></figure><h3>Understanding Code Files vs. Notebooks</h3><p>Your typical developer writes code organized into files. When executed, the entire code runs sequentially, producing a desired output. However, for data analysts and scientists, the game is different. Their work involves iterative tasks like querying data, cleaning it, modeling, and charting. Unlike developers, they need constant intermediate feedback for each line of code.</p><h3>The Notebook Revolution</h3><p>Before 2011, data teams wrote code in files like everyone else. Then came the iPython Notebook, a game-changer. The notebook approach allows you to write a block of code, run it, observe the output, and proceed to the next block. What makes notebooks invaluable?</p><ol><li><strong>Iterative Nature: </strong>Run code step by step, ensuring each part works before moving on.</li><li><strong>Visual Appeal: </strong>Integrated tables, charts, and text enhance code visualization.</li><li><strong>Organizational Excellence: </strong>Notebooks function like presentations — name each cell, reorder them, and add markdown for storytelling.</li></ol><h3>Peeling Back the Layers of a Notebook</h3><p>Let’s break down the original code block querying churn data. In a notebook, a data scientist can import libraries, upload data, visualize it, filter it, and validate the results — all step by step. While this could be done in a single file, the notebook’s iterative nature allows for gradual progress, ensuring each step achieves the intended outcome.</p><p>A huge key advantage of notebooks is their visual aspect. They allow us to easily view well-formatted tables, as seen above, and even create code to visualize data in charts. Crafting stories is a crucial aspect of the vital role data teams play.</p><h3>Notebooks: More than a Panacea</h3><p>Despite their advantages, notebooks have their critics. Some argue they’re primarily suited for exploration, not production. However, their value in exploratory data work is undeniable. Visual appeal, organization, and storytelling capabilities make them indispensable for data teams.</p><h3>Evolution of Notebooks: A Brief History</h3><p>The iPython Notebook, born in 2011, was not the first notebook interface. Mathematica, dating back to 1988, was the pioneer. The notebook concept acts as a frontend for basic code, with the kernel — the computational engine — executing code and formatting results for visual presentation.</p><h3>Diversity in Kernels and Deployment Models</h3><p>Initially Python-centric, notebooks now support various languages with interchangeable kernels. The <a href="https://towardsdatascience.com/jupyter-notebook-extensions-517fa69d2231">Jupyter notebook</a>, formerly iPython, has become the go-to, with over 100 official kernels available. As notebooks extend beyond data science, languages like Java, JavaScript, Scala, and Go have joined the fray.</p><h3>Running Notebooks: Local or in the Cloud?</h3><p><a href="https://towardsdatascience.com/jupyter-notebook-extensions-517fa69d2231">Jupyter</a>, an open-source project, allows local installation for those technically inclined. However, cloud-hosted solutions like Hex offer enhanced collaboration and powerful kernels. The choice depends on individual needs, with cloud deployment gaining traction for its convenience and capabilities.</p><p>Unravel the potential of data science notebooks, whether you’re cleaning and categorizing data, visualizing trends, prototyping ML models, or crafting compelling data stories.</p><p><strong>More Reading:</strong></p><ul><li>At <a href="https://netflixtechblog.com/notebook-innovation-591ee3221233">Netflix</a>, engineers rely extensively on notebooks in their machine learning systems, possibly more so than anyone else.</li><li>One of the first and my favorite in-depth look at the topic from <a href="https://www.uber.com/blog/no-code-workflow-orchestrator/">Uber</a>.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=15b451e1cad2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Equals CEO Bobby Pinero’s Lessons on Freemium, Friction, and User Activation]]></title>
            <link>https://sharinkhander.medium.com/equals-ceo-bobby-pineros-lessons-on-freemium-friction-and-user-activation-18dfff616043?source=rss-4e0edb9cb39e------2</link>
            <guid isPermaLink="false">https://medium.com/p/18dfff616043</guid>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[startup-lessons]]></category>
            <category><![CDATA[startupş]]></category>
            <category><![CDATA[pricing-strategy]]></category>
            <category><![CDATA[freemium]]></category>
            <dc:creator><![CDATA[Sharin Khander]]></dc:creator>
            <pubDate>Wed, 29 Nov 2023 17:09:13 GMT</pubDate>
            <atom:updated>2023-11-29T17:15:18.314Z</atom:updated>
            <content:encoded><![CDATA[<p>The allure of freemium models can be tantalizing, promising rapid user acquisition and market penetration. However, the story of <a href="https://equals.com/">Equals</a>, a next-gen spreadsheet startup, takes a cautionary turn as <a href="https://twitter.com/bobbypinero?lang=en">CEO Bobby Pinero</a> shares the tumultuous journey of how embracing freemium nearly derailed their business. I’ll share my key takeaways from this story, but you can check out the full article on one of my go-to product blogs, <a href="https://www.lennysnewsletter.com/?utm_source=navbar&amp;utm_medium=web&amp;r=jaf00">Lenny’s Newsletter</a>.</p><figure><img alt="Sharin Khander freemium" src="https://cdn-images-1.medium.com/max/1024/1*q0KHgy2PQg0BaUikx9jceA.png" /></figure><h3>The Temptation and Initial Success</h3><p>Equals launched in April 2022 with a high-friction onboarding strategy — every user had to engage in a one-on-one call and pay upfront. This approach worked exceptionally well, propelling Equals to a successful Series A funding round of $16 million from a16z within five months.</p><p>However, the initial success led to growing demands for less friction. Users clamored for a self-serve option, the ability to skip data source requirements, and, above all, a free plan. Seeing an opportunity to emulate successful SaaS companies like Notion and Figma, Equals decided to take the plunge into freemium.</p><h3>The Freewheeling Descent</h3><p>The introduction of freemium initially led to a fourfold increase in daily and weekly users. However, the honeymoon phase was short-lived. Engagement, retention, and revenue soon took a nosedive, leaving Equals in a precarious position.</p><p>Two key lessons emerged:</p><ol><li><strong>The Customer Can Be Wrong:</strong> Despite user demands and industry trends, what users want isn’t always what’s best for the product. Initial user surge post-freemium didn’t translate into long-term engagement, highlighting the gap between user requests and the product’s sustainable success.</li><li><strong>“Freemium” and “Friction” are Tied at the Hip:</strong> The success or failure of freemium is directly tied to the onboarding friction. Attempts to reduce friction, such as allowing CSV uploads instead of live data sources, led to a decrease in engagement. Friction, it turned out, was crucial for driving both short-term adoption and long-term retention.</li></ol><h3>Undoing the Damage: Adding Back Friction</h3><p>To reverse the downward spiral, Equals had to act decisively. The solution was counterintuitive — add back some friction to the onboarding process. Requiring every sign-up to start a 14-day free trial and input a credit card turned out to be the game-changer.</p><p>Lessons learned:</p><ol><li><strong>Onboarding is More Sales than Product:</strong> Onboarding is not just about getting users into the product; it’s about ensuring they experience the first moments of value. The allure of a new product is a powerful motivator for users to complete essential setup tasks.</li><li><strong>Friction Can Drive Commitment:</strong> Forcing users to invest upfront, through a credit card and a limited-time decision window, increased their commitment to complete challenging steps. Adding friction to onboarding can enhance user commitment.</li></ol><h3>Reflections and Guiding Questions</h3><p>The freemium model may not be suitable for every product, as Equals discovered the hard way. Drawing from their experience, here are some guiding questions for startups contemplating freemium:</p><ul><li>Is your product aligned with successful freemium models?</li><li>Have you considered the true goal of onboarding — user activation?</li><li>Does adding friction to onboarding align with your product’s goals?</li></ul><p>Equals’ rollercoaster journey with freemium serves as a valuable case study for startups grappling with the decision to embrace or resist the allure of free plans. Ultimately, the path to success may involve understanding that what users ask for and what a product needs for sustainability can be two different trajectories.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=18dfff616043" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>