<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Barun on Medium]]></title>
        <description><![CDATA[Stories by Barun on Medium]]></description>
        <link>https://medium.com/@barun_79986?source=rss-972769099cc2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*g0jQEsecJ9plzp9PbmHJWQ.png</url>
            <title>Stories by Barun on Medium</title>
            <link>https://medium.com/@barun_79986?source=rss-972769099cc2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 22:00:17 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@barun_79986/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Mapping Success: Navigating Your Organization’s Decision Tree]]></title>
            <link>https://medium.com/@barun_79986/mapping-success-navigating-your-organizations-decision-tree-8255ab276ab2?source=rss-972769099cc2------2</link>
            <guid isPermaLink="false">https://medium.com/p/8255ab276ab2</guid>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[organizational-culture]]></category>
            <category><![CDATA[decision-making]]></category>
            <category><![CDATA[metrics]]></category>
            <dc:creator><![CDATA[Barun]]></dc:creator>
            <pubDate>Sat, 03 Aug 2024 17:20:51 GMT</pubDate>
            <atom:updated>2024-08-03T17:20:51.459Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jb7V1JuvbB9clZ-U9NSV6w.png" /></figure><p>In any organization, decision-making is not a singular process but a complex web of interconnected choices that shape its trajectory. This web can be visualized as a decision tree, where each node represents a decision point, and the branches signify the possible outcomes or paths an organization might take. Understanding this decision tree is crucial for aligning efforts, setting clear goals, and ensuring that every team member knows their role in the broader picture.</p><h3>Mapping Out the Decision Tree</h3><p>The primary goal of visualizing an organizational decision tree is to identify the various decisions that need to be made and clarify who is responsible for each one. By mapping out these decision points, organizations can:</p><ol><li><strong>Clarify Roles and Responsibilities</strong>: Determine who makes what decisions and at what level. In my experience the DACI framework is quite effective in this. Of course, this roles/responsibilities are fluid, and usually need periodic realignment based on team’s progress towards a goal.</li><li><strong>Align Strategic Goals</strong>: Ensure that decisions made at different levels align with the overall strategic direction. OKRs usually work, if the team is diligent in following through, and constantly re-evaluating it against.</li><li><strong>Identify Key Decision Points</strong>: Highlight crucial nodes where significant strategic choices are made. In many cases, it’s easy to identify the key decision (but the decision itself can be rather complex to make) Example — The market segment you build for. In many cases, the “keyness” of a decision only shows itself in time. Example — choosing the right framework to build your product makes non-linear contribution to your release velocity.</li></ol><h3>Discovering Your Place in the Tree</h3><p>Understanding where you fit into this decision tree is essential for contributing effectively. Different departments or teams may have their own sub-trees, with their specific decision points and responsible individuals. By recognizing your position within the tree, you can better understand your impact on the organization and how your decisions ripple outwards.</p><h3>Navigating Certainty and Uncertainty</h3><p>In any decision tree, there will be areas of high certainty and areas of greater ambiguity:</p><ol><li><strong>High Certainty Nodes</strong>: These are typically at the leaf nodes, where specific, measurable tasks occur. For example, writing a line of code with a known input and output. Metrics can be clearly defined and tracked at these points.</li><li><strong>Uncertainty Nodes</strong>: As we move up the decision tree, decisions become more strategic and less certain. Choices such as selecting a technological framework (e.g., Langchain v/s Build) involve greater ambiguity and require deeper strategic alignment.</li></ol><h3>The Role of People in Decision-Making</h3><p>Uncertainty in decision-making underscores the <strong><em>importance of hiring the right people</em>.</strong> When individuals align with the organization’s values and strategic direction, they can navigate ambiguity more effectively. They bring:</p><ol><li><strong>Strategic Alignment</strong>: An understanding of and alignment with the organization’s goals and values.</li><li><strong>Internal Compass</strong>: A personal orientation that resonates with the organization’s direction, helping them make decisions that support the broader strategy.</li></ol><p>This is why it’s highly important for <strong><em>co-founders</em></strong> in context of startups, and <strong><em>senior organizational leaders</em></strong> in context of big companies to <strong><em>be highly involved in hiring</em></strong>. A 15–30 minute conversation with a potential hire reveals a lot about this person.</p><h3>Increasing Certainty Through Measurement</h3><p>One of the primary goals in managing an organizational decision tree is to create more certainty. This can be achieved through:</p><ol><li><strong>Setting Metrics</strong>: Collaboratively establish metrics from both a top-down and bottom-up approach. This ensures that <strong><em>metrics are relevant and actionable</em></strong> at all levels.</li><li><strong>Measuring Performance</strong>: Regularly track and measure these metrics to provide feedback and improve decision-making processes. Also, <strong><em>all data should be democratized</em></strong>. Anyone within the organization who is thinking about a decision should be able to pull in through various sources. This will help them build an accurate picture, and <strong><em>ground their decision in known facts</em></strong>.</li></ol><p>Understanding and navigating the organizational decision tree is a strategic imperative for any organization. By clarifying decision points, aligning roles and responsibilities, and emphasizing the importance of strategic alignment and measurement, organizations can improve their decision-making processes and drive better outcomes. The journey from uncertainty to certainty is continuous, but with the right people and the right metrics, it is a journey that can lead to sustained success.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8255ab276ab2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Winning Startups: Build, Market, Repeat]]></title>
            <link>https://medium.com/@barun_79986/winning-startups-build-market-repeat-e98d8c641eae?source=rss-972769099cc2------2</link>
            <guid isPermaLink="false">https://medium.com/p/e98d8c641eae</guid>
            <category><![CDATA[marketing]]></category>
            <category><![CDATA[go-to-market]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[engineering]]></category>
            <dc:creator><![CDATA[Barun]]></dc:creator>
            <pubDate>Fri, 28 Jun 2024 23:54:53 GMT</pubDate>
            <atom:updated>2024-06-28T23:54:53.320Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yT3NBEthfjnBVdDgqMUigw.png" /></figure><p>Having a front seat in the development of products in the GenAI space at Fiddler has been a profound experience. As the lead on monitoring solutions for GenAI at Fiddler, I’ve found the journey both wild and inspiring.</p><p>About two years ago, our team sought to strengthen our approach to monitoring unstructured data, which led to the creation of Vector Monitoring. This initiative was heavily influenced by our scientists, and as engineers, we facilitated its transition to the market. We demonstrated it to various customers, and while it resonated during demos, it didn’t achieve significant adoption in production. This showed us that there was excitement, but perhaps we were ahead of our time, as many companies were still entrenched in traditional machine learning approaches.</p><p>The GenAI landscape changed drastically with the advent of technologies like chatGPT, which sparked a tremendous boom. This shift made the research component of our product increasingly crucial, and we invested more resources to enhance it, including adding native support for vector types. A frenzy ensued in the GenAI ecosystem, with companies like LangChain, LlamaIndex, Chroma, and Pinecone emerging. Startups were receiving insane valuations just months after launching, and venture capitalists, reinvigorated by the boom, began investing heavily in AI. Companies experiencing FOMO shifted funds to AI projects, ushering in a period of optimism and rapid innovation. Many infrastructure companies also sprang up to support training, fine-tuning, serving, monitoring, and orchestration.</p><p>The go-to-market strategy during this time was simple: add “AI” to your name, use sparkly emojis liberally to emphasize your AI focus, and if you had the resources, host a hackathon in places like AGI-house or somewhere in San Francisco. It was an era of learning through quick-and-dirty experiments.</p><p>By 2024, the landscape began to mature. The initial frenzy over GenAI was replaced by a focus on real, valuable applications and solving concrete problems. VPs started demanding ROI. Applications that were celebrated or acquired just a year before were now struggling to scale, support users, deploy on-premises, provide robust authentication, adapt to new use cases, ensure security, or justify their costs. These were classic engineering challenges — no longer about inventing new technologies, but about excellent engineers, data scientists, and product managers coming together to solve foundational issues.</p><p>With the influx of VC funding in 2023 and 2024, the competition for visibility in front of potential clients intensified. Now, a well-thought-out, effectively executed go-to-market strategy is crucial for success. The days of relying on AI’s novelty are over; it’s time to deliver practical, robust solutions that meet real-world demands.</p><p>This journey has reinforced a vital lesson: even in the innovative wave of GenAI, the fundamentals are what set a company apart. Strong engineering capabilities and a coherent go-to-market strategy are essential. To succeed, we must build products that not only incorporate cutting-edge technology but are also usable in an enterprise setting and supported by a robust go-to-market motion that ensures their discovery and adoption. This dual focus on technical excellence and market readiness is what will define the next leaders in the GenAI landscape.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e98d8c641eae" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Navigating Dual Roles: My Journey as Product and Engineering Lead]]></title>
            <link>https://medium.com/@barun_79986/navigating-dual-roles-my-journey-as-product-and-engineering-lead-at-fiddler-362d3d4b1756?source=rss-972769099cc2------2</link>
            <guid isPermaLink="false">https://medium.com/p/362d3d4b1756</guid>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[product-management]]></category>
            <dc:creator><![CDATA[Barun]]></dc:creator>
            <pubDate>Tue, 14 May 2024 04:14:03 GMT</pubDate>
            <atom:updated>2024-05-14T04:14:22.180Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JNBkIdNLxbhzBhjQ2-vW2A.png" /></figure><p><strong>Introduction<br></strong>Over the past six months, I’ve had the unique opportunity to serve as both the Product Manager and Engineering Lead at Fiddler. This dual role has been both a challenging and enriching experience, stretching my capabilities and offering profound insights into the intersection of product development and engineering within the rapidly evolving tech landscape.</p><p><strong>Stepping Into Big Shoes<br></strong>After Fiddler’s previous Product Manager left, I assumed their responsibilities, fueled by determination and excitement. My previous experiences had equipped me with robust skills now directed toward driving our initiatives in Generative AI. This shift not only challenged me but also positioned our company at the forefront of the competitive industry.</p><p><strong>Challenges of Dual Leadership<br></strong>Transitioning into this dual role involved a steep learning curve. The task of replacing an excellent PM like Sree was daunting yet exhilarating, requiring high-impact decisions and alignment of diverse teams with varying goals. This role is reflective of a growing trend in the tech industry where leaders are expected to wear multiple hats, navigating complex cross-disciplinary challenges to drive innovation.</p><p><strong>Overcoming Stereotypes and Cultivating Empathy<br></strong>One of the significant challenges was navigating biases often held against engineers in leadership roles. Skepticism about my strategic business capabilities mirrored the broader industry stereotype that engineers lack business acumen. This experience taught me the importance of mutual respect and understanding across different roles. Triumphing over these stereotypes, I led a breakthrough project, which significantly boosted our market standing, demonstrating that engineers can indeed excel in strategic business roles.</p><p><strong>The Emotional Toll of Continuous Engagement<br></strong>The ‘always-on’ nature of the PM role proved to be both rewarding and mentally taxing. The continuous need for alignment and communication often bordered on burnout, affecting creativity and personal well-being. For example, I had a day where I conducted a customer demo at 7 AM PT, and my last meeting ended at 10 PM PT with the India Engineering team. A visual timeline of a typical day could illustrate these demands and help readers appreciate the intensity of the role.</p><p><strong>Leadership Styles and Team Dynamics<br></strong>My leadership style evolved significantly during this dual tenure, emphasizing transparency, patience, and flexibility. Managing defensive reactions and communicating complex ideas clearly under pressure became crucial. These adjustments were not only about personal growth but also about enhancing our team dynamics.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DU249Pxf4ivblAidWn-HvQ.png" /></figure><p>After a particularly challenging conflict, a colleague shared the following feedback, which reflects the impact of these leadership adjustments:</p><p><em>“We will continue to be diligent together. Thanks for being so open, patient, and flexible. I can see why your managers have so much faith in you. I hope you will repay my experience with yours.”</em></p><p>This note underscores the collaborative spirit that we strive to foster within our team. It is through moments like these that we build a stronger, more cohesive team, capable of tackling complex challenges and driving innovation forward.</p><p><strong>Lessons Learned and Skills Honed<br></strong>My tenure has sharpened several key areas:</p><ul><li><strong>Defensiveness:</strong> Managing defensive reactions during heated discussions became crucial.</li><li><strong>Pushing Back:</strong> Learning to say no in a resource-limited environment was vital.</li><li><strong>Communication:</strong> Enhancing my ability to articulate complex ideas clearly was necessary, especially when I developed and communicated a 6-month roadmap to the entire company.</li><li><strong>Writing:</strong> Evolving my writing to convey complex ideas succinctly and persuasively became essential.</li></ul><p><strong>Concluding Thoughts<br></strong>Reflecting on the synergy between Product and Engineering, it’s clear that balancing these roles is about creating a dialogue that drives innovation. This journey has underscored the indispensable relationship between these areas, essential for developing superior solutions that meet customer needs.</p><p><strong>Reflecting on the Synergy Between Roles<br></strong>The tension between Product and Engineering isn’t just about managing roles; it’s about fostering a dynamic conversation that propels innovation. As I reflect on these past six months, I am grateful for every challenge and victory, which have shaped not just a better product but a leader I am proud to be. Looking forward, I am excited to continue this journey, leveraging lessons learned and a passion to innovate and inspire.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=362d3d4b1756" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Code as conversation]]></title>
            <link>https://medium.com/@barun_79986/code-as-conversation-a02df798d9c4?source=rss-972769099cc2------2</link>
            <guid isPermaLink="false">https://medium.com/p/a02df798d9c4</guid>
            <category><![CDATA[code-review]]></category>
            <category><![CDATA[code-newbie]]></category>
            <category><![CDATA[code]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Barun]]></dc:creator>
            <pubDate>Sat, 27 Apr 2024 16:58:18 GMT</pubDate>
            <atom:updated>2024-04-27T16:59:36.375Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ScYqd3kGEeXwepUYbnMlYw.png" /></figure><p>In the world of software development, the way we write code can either pave the way for easy maintenance and collaboration or set the stage for a confusing mess that even the most seasoned developers struggle to untangle. The idea that “code is a conversation” with your future self and your peers encapsulates the essence of why coding standards matter. But what does it really mean to write code not just as a personal diary, but as a clear and accessible piece of communication?</p><h4>The Literary Spectrum of Code</h4><p>Code can range from being a cryptic piece of art, akin to high literature that only a select few can decipher, to a straightforward text that’s as readable as a popular novel. The former might showcase one’s technical prowess, but the latter ensures that the code remains accessible and maintainable by a broader audience. This is especially crucial in a team setting where code readability should always be a priority. There are, however, exceptions to this rule.</p><p><strong>Exceptions to the Rule:</strong></p><ol><li><strong>Established Patterns:</strong> Sometimes, within a specific group or team, certain coding patterns and practices are well understood. In these cases, deviating from general readability standards might be acceptable, provided these patterns are documented and there’s an onboarding process for newcomers to grasp the nuances of such coding styles.</li><li><strong>Performance Optimization:</strong> In scenarios where code performance is critical and optimizations are necessary, readability might take a back seat. For instance, integrating assembly language within Python for performance gains might make the code look intimidating. However, this should be accompanied by comprehensive comments or documentation explaining the rationale and workings of such code sections to aid future developers.</li></ol><p>These exceptions should come with an onboarding plan for developers who might not be familiar with the specific practices or optimizations used. This might include:</p><ul><li><strong>References and Documentation:</strong> For well-established patterns, having references to external documentation or internal wikis that explain these patterns in context can be invaluable.</li><li><strong>Descriptive Comments:</strong> For complex optimizations or unusual code snippets, detailed comments are crucial. These comments should not just describe what the code does, but also why it was necessary to implement it in such a manner.</li></ul><h4>Coding for Accessibility</h4><p>The goal should be to write code that a sophomore CS major could understand. This level of clarity not only aids in maintaining and scaling the project but also serves as a form of live documentation. Unlike static documentation, videos, or external guides that can quickly become outdated, well-written code acts as a self-explanatory guide to how the product functions.</p><h4>The True Value of Readable Code</h4><p>Readable code enhances collaboration, facilitates easier maintenance, and speeds up the onboarding process for new team members. It acts as a real-time, up-to-date documentation of the system, reducing dependencies on external resources that might not keep pace with the rapid changes typical in a software development environment.</p><p>In summary, while there are circumstances under which complex coding practices are justified, striving for readability and clarity should always be a developer’s primary goal. By viewing code as a conversation meant for a broad audience, developers can ensure that their work remains accessible, understandable, and maintainable, no matter how the team or technology evolves.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a02df798d9c4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[WANDB 2024 Insights(art 1): Transforming Search and SQL with AI]]></title>
            <link>https://medium.com/@barun_79986/wandb-2024-insights-art-1-transforming-search-and-sql-with-ai-1f7ca8dfef37?source=rss-972769099cc2------2</link>
            <guid isPermaLink="false">https://medium.com/p/1f7ca8dfef37</guid>
            <category><![CDATA[weights-and-biases]]></category>
            <category><![CDATA[snowflake]]></category>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[search]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Barun]]></dc:creator>
            <pubDate>Fri, 19 Apr 2024 17:29:04 GMT</pubDate>
            <atom:updated>2024-04-19T17:31:44.109Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>WANDB-FC 2024 Insights(Part 1): Transforming Search and SQL with AI</strong></h3><p>The annual Weights &amp; Biases Fully Connected Conference in San Francisco, a cornerstone event for the Generative AI industry, provided a deep dive into the advancements and innovations shaping the future of technology. Key sessions from industry leaders at You.com and Snowflake showcased significant strides in search technologies and SQL language processing. Here’s a comprehensive look at these sessions and their contributions to the field of large language models (LLMs).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fKX54mSJGEOWEiIKw6sPYg.png" /></figure><h3>You.com: Pioneering Next-Generation Search with AI</h3><p>Richard Socher, CEO of You.com, delivered an insightful overview of his journey in AI and NLP over the past decade, focusing on the development and future of AI-powered search technologies.</p><h3>Innovations and Challenges:</h3><ul><li><strong>Search Index Quality:</strong> Emphasizing the critical role of a well-structured search index, You.com has developed a system that drastically improves the precision and comprehensiveness of search results.</li><li><strong>Model Training:</strong> The session highlighted the ongoing challenge of updating training models to keep pace with rapidly changing information, combating issues such as outdated content, misattributions, and broken hyperlinks.</li><li><strong>Complex Reasoning:</strong> The company has tackled the training of LLMs for complex reasoning tasks, employing innovative strategies to enhance effectiveness.</li></ul><h3>Competitive Landscape and Product Development:</h3><ul><li><strong>Against Competitors:</strong> You.com has strategically positioned itself against competitors like Perplexity by prioritizing depth and accuracy of information over mere speed of retrieval.</li><li><strong>Performance Enhancements:</strong> The company noted significant improvements in model performance, especially in accuracy, which is crucial for the reliability of a search engine.</li><li><strong>Handling Long Contexts:</strong> Their system can now handle inputs up to 64,000 tokens, facilitating comprehensive document analysis.</li><li><strong>User Feedback:</strong> Feedback metrics such as clarity, conciseness, comprehensiveness, and personalization play a key role in ongoing product improvements.</li></ul><h3>Future Directions:</h3><ul><li><strong>Research Version:</strong> A new research-focused version of the product is being developed for deeper analysis and validation of search results.</li><li><strong>Technology Integration:</strong> Continued support for integrating new LLMs as they become available ensures the platform remains cutting-edge.</li><li><strong>Personalization Challenges:</strong> The complexities of personalizing search based on past interactions are being actively addressed to enhance the relevance and accuracy of content delivered to users.</li></ul><h3>Snowflake: Revolutionizing SQL Query Processing with LLMs</h3><p>Snowflake’s session provided an in-depth look at their efforts to translate natural language queries into SQL through the use of sophisticated LLMs and advanced data-handling strategies.</p><h3>Development Journey:</h3><ul><li><strong>Phase V0:</strong> The initial model optimization aimed at top performance on the Spider benchmark, involving fine-tuning on a variety of sources including web data and internal query logs.</li><li><strong>Phase V1:</strong> The integration of tool use enhanced the model’s functionality, and highlighted the need for high-quality data annotations to reflect the complexity of real-world SQL query generation.</li><li><strong>Phase V2:</strong> Adjustments made to better accommodate the conversational and interactive nature of SQL query formulation, with a focus on supporting iterative refinement based on user feedback.</li></ul><h3>Insights and Iterations:</h3><ul><li><strong>Instruction Tuning:</strong> Advanced strategies like chain of thought and complex instruction tuning have been vital for improving model performance.</li><li><strong>Semantic Catalog Retrieval:</strong> A semantic retrieval system, powered by a real LLM, effectively navigates extensive and complex schemas.</li><li><strong>System Design:</strong> The success of the product depends not just on LLM modeling but also on a robust end-to-end system design that includes user interface and interaction.</li></ul><h3>Challenges and Continuous Improvement:</h3><ul><li><strong>Conversational SQL:</strong> Creating a conversational SQL system that supports multi-turn interactions remains a challenging ongoing endeavor.</li><li><strong>Annotation Quality:</strong> The session emphasized the critical importance of high-quality annotations to ensure accurate SQL query outputs.</li></ul><h3>Conclusion</h3><p>The insights from the Fully Connected Conference underscore the rapid advancements in the field of Generative AI. Both You.com and Snowflake are at the forefront of transforming how businesses and individuals interact with data and technology, illustrating the profound impact of AI on practical applications. These sessions not only reflect the current state of AI innovations but also chart a course for future developments in the industry.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1f7ca8dfef37" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Applying First Principles]]></title>
            <link>https://medium.com/@barun_79986/when-to-use-first-principles-thinking-1cdc72e47428?source=rss-972769099cc2------2</link>
            <guid isPermaLink="false">https://medium.com/p/1cdc72e47428</guid>
            <category><![CDATA[first-principles-thinking]]></category>
            <category><![CDATA[startup-lessons]]></category>
            <category><![CDATA[thinking]]></category>
            <dc:creator><![CDATA[Barun]]></dc:creator>
            <pubDate>Sun, 14 Apr 2024 23:07:35 GMT</pubDate>
            <atom:updated>2024-04-14T23:23:59.496Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yEnKxqUJrVtiT70FYpBIOg.png" /></figure><p>First principles thinking, a method recently brought into the limelight by Elon Musk and his many followers, involves deconstructing an idea into its fundamental components to uncover “irrefutable” truths. Yet, determining these truths is not straightforward, as biases can greatly skew our perception of what is “irrefutable.”</p><p>This approach stands in stark contrast to “pattern matching” or thinking through analogy, where ideas are adopted because they have been accepted by others, rather than through personal scrutiny. History shows us that unexamined acceptance can lead to stagnation in thought — a concern echoed by philosophers such as René Descartes and Ralph Waldo Emerson. Descartes, the father of modern philosophy, promoted a methodological skepticism, advising to doubt everything not inherently clear and distinct. Emerson, a central figure in transcendentalism, advocated for self-reliance and finding inner joy, emphasizing the need to trust oneself over societal conventions.</p><p>Can first principles thinking be practically applied to every decision in our daily lives? The answer might disappoint those seeking simplicity. Realistically, engaging deeply with every piece of information we encounter is unfeasible. Our modern world bombards us with a vast array of data, making complete, thorough analysis exhausting, if not impossible.</p><p>Most of us tend to rely on digested information for efficiency. It is essential, however, to choose wisely whose opinions to trust and to stay informed about opposing viewpoints. Our selections are often subtly influenced by underlying biases related to race, gender, caste, or religion, which can trap us in echo chambers devoid of diverse perspectives.</p><p>To illustrate, consider how first principles thinking transformed the electric car industry. Innovators, instead of accepting prevailing notions that electric cars were impractical and expensive due to high battery costs, reevaluated the fundamental assumptions about battery technology and vehicle design. This rigorous analysis led to innovations that drastically reduced battery costs and improved car performance, fundamentally changing the automotive landscape.</p><p>Another compelling example is in the field of software development, particularly with the advent of a new video compression algorithm called AV1. Developers faced the challenge of increasing data transmission rates without losing video quality. By applying first principles thinking, they questioned the basic assumptions about pixel patterns and data redundancy in video streams. This led them to develop a method that more efficiently predicts and encodes repeating patterns, significantly enhancing compression rates and video quality at lower bandwidths. This breakthrough not only optimized data transmission but also had a profound impact on streaming services and content delivery networks.</p><p>By understanding when and how to apply this thinking, we can better navigate the complexities of modern life, recognizing when it’s crucial to challenge conventional wisdom and when it’s acceptable to rely on the expertise of others.</p><h3>Strategic Framework for Applying First Principles Thinking</h3><ol><li><strong>Assess Complexity and Need for Innovation:</strong></li></ol><ul><li><strong>Overview:</strong> Employ this approach for intricate challenges where existing methodologies fall short.</li><li><strong>Guidance:</strong> Particularly effective in scenarios that require groundbreaking solutions, like developing sustainable energy solutions or creating new medical technologies.</li><li><strong>Advantage:</strong> Encourages radical innovation by dissecting problems to their fundamental truths and reassembling them in novel ways.</li></ul><p><strong>2. Target High-Impact Opportunities:</strong></p><ul><li><strong>Overview:</strong> Prioritize issues where innovative solutions can drastically enhance performance, sustainability, or quality of life.</li><li><strong>Guidance:</strong> Suitable for areas such as public health, environmental conservation, and critical infrastructure projects.</li><li><strong>Advantage:</strong> Focuses efforts on changes that can significantly alter the status quo for the better, ensuring meaningful progress.</li></ul><p><strong>3. Build Foundational Knowledge:</strong></p><ul><li><strong>Overview:</strong> Essential for mastering highly specialized or evolving sectors.</li><li><strong>Guidance:</strong> Useful when launching into cutting-edge fields or technologies, where a deep and accurate base of knowledge can provide a significant competitive edge.</li><li><strong>Advantage:</strong> Prepares the ground for innovation and ensures adaptability as industries evolve and new technologies emerge.</li></ul><p><strong>4. Optimize Resource Investment:</strong></p><ul><li><strong>Overview:</strong> Evaluate the alignment of resource expenditure with potential strategic gains.</li><li><strong>Guidance:</strong> Critical for ensuring that resource allocation in time, capital, and human effort is justified by the anticipated outcomes, especially in startups and small enterprises.</li><li><strong>Advantage:</strong> Maximizes efficiency and effectiveness, preventing resource wastage and focusing on projects with the highest return on investment.</li></ul><p><strong>5. Resolve Stagnant Challenges:</strong></p><ul><li><strong>Overview:</strong> Break through impasses in situations where conventional tactics have been ineffective.</li><li><strong>Guidance:</strong> Apply when projects are stalled or solutions seem out of reach using standard methods.</li><li><strong>Advantage:</strong> Can rejuvenate stalled initiatives and inspire new directions that are more likely to succeed.</li></ul><p><strong>6. Harness Passion and Expertise:</strong></p><ul><li><strong>Overview:</strong> Utilize personal or collective passion as a driving force for in-depth exploration and persistent problem-solving.</li><li><strong>Guidance:</strong> Most impactful when aligned with personal interests or core business strengths, ensuring prolonged engagement and thorough analysis.</li><li><strong>Advantage:</strong> Sustains motivation and fosters a culture of curiosity and dedication, which are indispensable for innovative success.</li></ul><p><strong>7. Ensure Scalability and Flexibility:</strong></p><ul><li><strong>Overview:</strong> Verify that solutions are scalable and can be adapted for broader application without loss of functionality or efficiency.</li><li><strong>Guidance:</strong> Essential when planning for growth or aiming to impact larger systems, such as national infrastructure, global supply chains, or expansive software platforms.</li><li><strong>Advantage:</strong> Prepares projects for successful expansion and versatile applications, safeguarding against scalability pitfalls.</li></ul><p><strong>8. Prioritize Ethical Considerations:</strong></p><ul><li><strong>Overview:</strong> Integrate ethical deliberations to foster solutions that are not only effective but also socially responsible and acceptable.</li><li><strong>Guidance:</strong> Particularly important in fields impacting human lives directly, such as artificial intelligence, genetic research, and public policy.</li><li><strong>Advantage:</strong> Enhances societal acceptance and trust, ensuring long-term viability and compliance with ethical norms.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1cdc72e47428" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hugging Face playbook]]></title>
            <link>https://medium.com/@barun_79986/hugging-face-playbook-575ed1b78369?source=rss-972769099cc2------2</link>
            <guid isPermaLink="false">https://medium.com/p/575ed1b78369</guid>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[hugging-face]]></category>
            <category><![CDATA[startup-lessons]]></category>
            <category><![CDATA[playbook]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Barun]]></dc:creator>
            <pubDate>Mon, 01 Apr 2024 20:36:10 GMT</pubDate>
            <atom:updated>2024-04-01T20:41:52.676Z</atom:updated>
            <content:encoded><![CDATA[<p>As Hugging Face stands at the forefront of democratizing artificial intelligence (AI), particularly in the realm of natural language processing (NLP), it’s not just their technological innovations that draw attention. Equally compelling is the (very well-known) playbook they’ve utilized to build a company that’s both influential and community-centric. This playbook has parallels in the journeys of other successful tech ventures, highlighting a modern blueprint for building a company in the open-source era.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gQorV-9jw67qD2j6pdH6xQ.png" /></figure><h3>Solving Pain Points with Open-Source Solutions</h3><p>At the core of Hugging Face’s strategy is the identification and alleviation of significant pain points in AI and NLP through open-source solutions. The early landscape of NLP was fraught with challenges such as high entry barriers, the complexity of model development, and the necessity for substantial computational resources. By offering easy-to-use tools like the Transformers library, Hugging Face made it simpler for researchers, developers, and companies to adopt and implement state-of-the-art NLP technologies without the overhead traditionally associated with such tasks.</p><h3>Fostering a Vibrant Community</h3><p>Understanding that the strength of an open-source project lies as much in its community as in its code, Hugging Face has put a significant emphasis on community engagement. They have nurtured a space where users, from novices to experts, can contribute, learn, and collaborate. This approach not only accelerates the development of their tools but also ensures they are widely adopted and improved upon. A thriving community means a constant stream of feedback and contributions, which helps in refining the tools to better meet the users’ needs.</p><h3>Productizing the Open-Source Solution</h3><p>Hugging Face’s approach to productizing their open-source solutions demonstrates a keen understanding of scalability and performance. Through offerings like the Hugging Face Hub, Inference API, and various libraries, they’ve built a platform that not only supports the community’s needs but also provides the infrastructure for deploying AI solutions at scale. This transition from an open-source project to a scalable product allows Hugging Face to maintain its commitment to open-source values while also meeting the needs of more enterprise-level users.</p><h3>Parallel Strategies in Tech Advancements</h3><p>The playbook Hugging Face has employed echoes strategies used by other ventures that have successfully leveraged open-source projects for growth. Companies like Confluent with Kafka and initiatives in monitoring and AI platforms such as LangChain and llamaHub have followed similar paths. These strategies involve building around a core open-source technology, engaging a community for development and support, and then scaling through products and services that cater to a broader audience.</p><h3>The Ripple Effect of Hugging Face’s Approach</h3><p>By adhering to this playbook, Hugging Face has not only democratized access to cutting-edge AI technologies but has also inspired a model for innovation that extends beyond the company. It’s a testament to how open-source principles, coupled with strategic product development and community engagement, can fuel growth in the tech sector. Hugging Face’s journey underscores the potential for open-source projects to transform into sustainable, impactful businesses that drive forward both technology and community.</p><p>It’s clear that Hugging Face’s approach is not just about building tools; it’s about building a community and an ecosystem that sustains and propels the field of AI forward. Hugging Face stands as a model for how open-source innovation can be leveraged to not only create groundbreaking technologies but also to build a company that remains at the cutting edge of its industry. Their journey from solving specific NLP challenges to creating a comprehensive platform for AI development offers a roadmap for future tech ventures aiming to make a similar impact.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=575ed1b78369" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding RAG in Language Models]]></title>
            <link>https://medium.com/@barun_79986/understanding-rag-in-language-models-8b7498cf8dbd?source=rss-972769099cc2------2</link>
            <guid isPermaLink="false">https://medium.com/p/8b7498cf8dbd</guid>
            <category><![CDATA[llmops]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Barun]]></dc:creator>
            <pubDate>Fri, 13 Oct 2023 04:53:31 GMT</pubDate>
            <atom:updated>2023-10-13T15:19:47.133Z</atom:updated>
            <content:encoded><![CDATA[<p>Let’s take a closer look at Retrieval Augmented Generation (RAG) and find out why it’s a useful tool when working with Large Language Models (LLMs).</p><h3>What is RAG?</h3><p>Imagine RAG as a smart librarian for language computer models. Instead of a model trying to recall a huge library of information, RAG brings back useful pieces from documents to create answers that are relevant and correct. It sees LLMs as “reasoning engines” and the (vector) databases holding the documents as the system’s “memory.”</p><h3>Three Ways to Use LLMs</h3><p>When we use LLMs, we generally have three options:</p><ul><li><strong>Building a Custom Model</strong></li><li><strong>Fine-tuning an Existing Model</strong></li><li><strong>Using RAG</strong></li></ul><h3>1. Making Your Own Model</h3><p>Creating a model can be like making a meal without a recipe. It’s usually for special use-cases that normal models can’t cover. For many common uses, like chatting or answering questions, it might be an unnecessary effort.</p><h3>2. Fine-tuning an Existing Model</h3><p>Adjusting a ready model may sound easier but has its own problems. Not all models let you make easy changes, and sending your data to a model provider for fine-tuning might be risky. If you’re fine-tuning an on-prem model, you’ll need computer resources, plus in-house data engineering and deep learning expertise.</p><h3>Why Many Choose RAG</h3><p>Creating or even fine-tuning a model can be too much work in many cases:</p><ul><li>Information can change quickly, meaning you might need to retrain your model often.</li><li>Retraining or adjusting can take a lot of time.</li><li>There might not be enough data for effective fine-tuning or retraining.</li></ul><p>RAG helps to avoid these challenges and still works quite well. RAG is a balanced and smart choice for many applications, striking a good balance between being effective and saving resources.</p><p>The major downsides of a RAG based system are —</p><ul><li>It needs setup time to store document indexes.</li><li>It adds some delay because it has to find relevant documents during the process. For example, a platform like <a href="http://perplexity.ai/">perplexity.ai</a> must look through the internet to bring documents and add them as context to the user’s question. Tools like glean or Elasticsearch may need to search through office apps like JIRA or Confluence to find relevant documents and add context.</li><li>RAG based systems use some tokens in context window to add data from the documents.</li></ul><h3>Key Problems a Good RAG Toolkit Will Tackle</h3><ol><li>Reducing delay during inference.</li><li>Getting the RAG setup to work on a larger scale in production.</li><li>Making the retrieval of “relevant” documents better.</li><li>Making efficient use of context window.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8b7498cf8dbd" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>