<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Harish Bohara on Medium]]></title>
        <description><![CDATA[Stories by Harish Bohara on Medium]]></description>
        <link>https://medium.com/@harishbohara?source=rss-27bce89f776d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ExqgQ0IR8LDmMSAWhOLmFw.jpeg</url>
            <title>Stories by Harish Bohara on Medium</title>
            <link>https://medium.com/@harishbohara?source=rss-27bce89f776d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 02:46:20 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@harishbohara/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Running LLM on your own HW]]></title>
            <link>https://medium.com/@harishbohara/running-llm-on-your-own-hw-fa3d86314f89?source=rss-27bce89f776d------2</link>
            <guid isPermaLink="false">https://medium.com/p/fa3d86314f89</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ollama]]></category>
            <category><![CDATA[chatgpt]]></category>
            <dc:creator><![CDATA[Harish Bohara]]></dc:creator>
            <pubDate>Sun, 19 Nov 2023 13:05:03 GMT</pubDate>
            <atom:updated>2023-11-19T13:16:39.249Z</atom:updated>
            <content:encoded><![CDATA[<p>This article is meant for someone who is a beginner to host LLMs and wants to run an LLM on his/her own HW (own laptop or own machine). Most of this information can also be found on Ollam website and <a href="https://github.com/jmorganca/ollama">github</a>.</p><ol><li>Allow someone to run LLM on a local machine and run</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*emYVn14niKdONdqICllraw.png" /><figcaption>Chal over console</figcaption></figure><p>2. Host ```ollama``` as a server and call it over HTTP to process your request</p><ul><li>This is quite useful where you can host models instead of calling ChatGPT, get results from your own servers</li></ul><h3>Setup</h3><p>Download and install on your local system — <a href="https://ollama.ai/download">https://ollama.ai/download</a></p><h4>Run CLI</h4><p>Once it is installed you can run using the following command</p><pre>ollama run llama2</pre><p>it may take time for the first time (if the model is not downloaded). Once you see the prompt, you can start playing with the prompt.</p><pre>&gt;&gt;&gt; tell me a joke<br>Sure! Here&#39;s a classic one:<br><br>Why don&#39;t scientists trust atoms?<br>Because they make up everything!<br><br>I hope that made you smile! Do you want to hear another one?</pre><h4>Connect to OLLAMA from some code</h4><p>There are times when you want to connect to this Ollama and use it. You can do the following</p><pre>ollama serve</pre><p>Now you can connect to this from your code on the port 11434. Here is a sample (I wanted to try something out — where I can pass some text and ask LLM to extract some information for me)</p><pre>from langchain.callbacks.manager import CallbackManager<br>from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler<br>from langchain.llms import Ollama<br><br>llm = Ollama(<br>    model=&quot;llama2&quot;, callback_manager=CallbackManager([StreamingStdOutCallbackHandler()])<br>)<br><br>prompt = &quot;&quot;&quot;<br>I have given you a json input which you can find inside &#39;&#39;&#39;JSON&#39;&#39;&#39; below. Follow the instruction:<br><br>1. Read Json<br>2. Find the country for user<br>3. Double check if this is actually the country of something else e.g. state <br>4. If this is not the country then try to find it again<br>5. Do not explain how, but only give the result as a json in request format.<br>6. Also extract any date in YYYY-MM-DD formate only.  <br>7. Make sure you do not answer me &quot;Sure, blah blah&quot;. Only focus on the Json response.<br>8. Double check your answer, and make sure this output is just the json.<br>9. Double check and ensure any data is in formate of YYYY-MM-DD<br><br>&#39;&#39;&#39;<br>{<br>  &quot;name&quot;: &quot;John Doe&quot;,<br>  &quot;age&quot;: 30,<br>  &quot;bill_date&quot;: &quot;After 10 day&quot;,<br>  &quot;location&quot;: {<br>    &quot;city&quot;: &quot;New York&quot;,<br>    &quot;state&quot;: &quot;NY&quot;<br>  },<br>  &quot;interests&quot;: [<br>    &quot;running&quot;,<br>    &quot;reading&quot;,<br>    &quot;traveling&quot;<br>  ]<br>}<br>&#39;&#39;&#39;<br><br>Use the following format to output data:<br>{&quot;name&quot;: &quot;&lt;find the name of the country json input&gt;&quot;, &quot;country&quot;: &quot;&lt;find the country from the json input&gt;&quot;, &quot;bill_date&quot;: &quot;&lt;bill date&gt;&quot;}<br>&quot;&quot;&quot;<br><br>llm(prompt)</pre><p>Output</p><pre>Sure, I can help you with that! Based on the JSON input provided, here is the output in the requested format:<br><br>{&quot;name&quot;: &quot;John Doe&quot;, &quot;country&quot;: &quot;USA&quot;, &quot;bill_date&quot;: &quot;After 10 day&quot;}</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fa3d86314f89" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Auto-GPT is an open-source application that demonstrates the power of GPT-4.]]></title>
            <link>https://medium.com/@harishbohara/auto-gpt-is-an-open-source-application-that-demonstrates-the-power-of-gpt-4-bef054ba51c6?source=rss-27bce89f776d------2</link>
            <guid isPermaLink="false">https://medium.com/p/bef054ba51c6</guid>
            <category><![CDATA[auto-gpt]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[chatgpt]]></category>
            <dc:creator><![CDATA[Harish Bohara]]></dc:creator>
            <pubDate>Tue, 02 May 2023 06:34:37 GMT</pubDate>
            <atom:updated>2023-05-02T06:34:37.023Z</atom:updated>
            <content:encoded><![CDATA[<h3>AutoGPT: How It Works with ChatGPT to Help You Accomplish Your Tasks</h3><p><a href="https://github.com/Significant-Gravitas/Auto-GPT">Auto-GPT</a> is an open-source application that demonstrates the power of GPT-4. It uses LLM “thoughts” to achieve any goal set, pushing the boundaries of what’s possible with AI.</p><p>ChatGPT is a new technology that can generate human-like text. But we still have a lot to learn about how to use it effectively. In this blog, I’ll share an important lesson from auto AutoGPT. We’ll see how it can help us work better with these models.</p><ul><li>This blog provides a high-level understanding of how AutoGPT works with ChatGPT as a backend to complete your tasks.</li><li>AutoGPT generates a prompt that includes your question, constraints, commands, resources, evaluations, and response format.</li><li>ChatGPT uses the commands you provide, and AutoGPT maps them to corresponding implementations.</li><li>By running executions in a loop, you can achieve your goals with ChatGPT’s helpful guidance.</li></ul><h3>Let’s see how it starts — high-level view</h3><p>This section will explain how the technology works at a high level. Once you understand this, you will be able to see how to build it yourself.</p><p>Here is an example: I asked AutoGPT to <em>find a Pizza in Bangalore</em>. AutoGPT expanded it in the following internal prompt which is sent to ChatGPT.</p><p>AutoGPT generates a prompt that includes the original question you asked and your goals. It also includes additional information such as constraints, commands, resources, performance evaluations, and response formats.</p><p>By including this information, the AutoGPT (Based on AutoGPT v0.2.2) code ensures that ChatGPT considers all relevant constraints and provides guidance on what steps to take next.</p><pre>You are PizzaBot, a bot to find thin crust pizza in bangalore <br>Your decisions must always be made independently without seeking user assistance. Play to your strengths as an LLM and pursue simple strategies with no legal complications.<br><br>GOALS:<br><br>1. Look for all pizza in bangalore<br>2. check how far is the place from koramangala<br>3. filter pizza with price &lt; 1000<br>4. put it in a file<br>5. done<br><br><br>Constraints:<br>1. ~4000 word limit for short term memory. Your short term memory is short, so immediately save important information to files.<br>2. If you are unsure how you previously did something or want to recall past events, thinking about similar events will help you remember.<br>3. No user assistance<br>4. Exclusively use the commands listed in double quotes e.g. &quot;command name&quot;<br>5. Use subprocesses for commands that will not terminate within a few minutes<br><br>Commands:<br>1. Google Search: &quot;google&quot;, args: &quot;input&quot;: &quot;&lt;search&gt;&quot;<br>2. Browse Website: &quot;browse_website&quot;, args: &quot;url&quot;: &quot;&lt;url&gt;&quot;, &quot;question&quot;: &quot;&lt;what_you_want_to_find_on_website&gt;&quot;<br>3. Start GPT Agent: &quot;start_agent&quot;, args: &quot;name&quot;: &quot;&lt;name&gt;&quot;, &quot;task&quot;: &quot;&lt;short_task_desc&gt;&quot;, &quot;prompt&quot;: &quot;&lt;prompt&gt;&quot;<br>4. Message GPT Agent: &quot;message_agent&quot;, args: &quot;key&quot;: &quot;&lt;key&gt;&quot;, &quot;message&quot;: &quot;&lt;message&gt;&quot;<br>5. List GPT Agents: &quot;list_agents&quot;, args: <br>6. Delete GPT Agent: &quot;delete_agent&quot;, args: &quot;key&quot;: &quot;&lt;key&gt;&quot;<br>7. Clone Repository: &quot;clone_repository&quot;, args: &quot;repository_url&quot;: &quot;&lt;url&gt;&quot;, &quot;clone_path&quot;: &quot;&lt;directory&gt;&quot;<br>8. Write to file: &quot;write_to_file&quot;, args: &quot;file&quot;: &quot;&lt;file&gt;&quot;, &quot;text&quot;: &quot;&lt;text&gt;&quot;<br>9. Read file: &quot;read_file&quot;, args: &quot;file&quot;: &quot;&lt;file&gt;&quot;<br>10. Append to file: &quot;append_to_file&quot;, args: &quot;file&quot;: &quot;&lt;file&gt;&quot;, &quot;text&quot;: &quot;&lt;text&gt;&quot;<br>11. Delete file: &quot;delete_file&quot;, args: &quot;file&quot;: &quot;&lt;file&gt;&quot;<br>12. Search Files: &quot;search_files&quot;, args: &quot;directory&quot;: &quot;&lt;directory&gt;&quot;<br>13. Analyze Code: &quot;analyze_code&quot;, args: &quot;code&quot;: &quot;&lt;full_code_string&gt;&quot;<br>14. Get Improved Code: &quot;improve_code&quot;, args: &quot;suggestions&quot;: &quot;&lt;list_of_suggestions&gt;&quot;, &quot;code&quot;: &quot;&lt;full_code_string&gt;&quot;<br>15. Write Tests: &quot;write_tests&quot;, args: &quot;code&quot;: &quot;&lt;full_code_string&gt;&quot;, &quot;focus&quot;: &quot;&lt;list_of_focus_areas&gt;&quot;<br>16. Execute Python File: &quot;execute_python_file&quot;, args: &quot;file&quot;: &quot;&lt;file&gt;&quot;<br>17. Generate Image: &quot;generate_image&quot;, args: &quot;prompt&quot;: &quot;&lt;prompt&gt;&quot;<br>18. Send Tweet: &quot;send_tweet&quot;, args: &quot;text&quot;: &quot;&lt;text&gt;&quot;<br>19. Do Nothing: &quot;do_nothing&quot;, args: <br>20. Task Complete (Shutdown): &quot;task_complete&quot;, args: &quot;reason&quot;: &quot;&lt;reason&gt;&quot;<br><br>Resources:<br>1. Internet access for searches and information gathering.<br>2. Long Term memory management.<br>3. GPT-3.5 powered Agents for delegation of simple tasks.<br>4. File output.<br><br>Performance Evaluation:<br>1. Continuously review and analyze your actions to ensure you are performing to the best of your abilities.<br>2. Constructively self-criticize your big-picture behavior constantly.<br>3. Reflect on past decisions and strategies to refine your approach.<br>4. Every command has a cost, so be smart and efficient. Aim to complete tasks in the least number of steps.<br><br>You should only respond in JSON format as described below <br>Response Format: <br>{<br>    &quot;thoughts&quot;: {<br>        &quot;text&quot;: &quot;thought&quot;,<br>        &quot;reasoning&quot;: &quot;reasoning&quot;,<br>        &quot;plan&quot;: &quot;- short bulleted\n- list that conveys\n- long-term plan&quot;,<br>        &quot;criticism&quot;: &quot;constructive self-criticism&quot;,<br>        &quot;speak&quot;: &quot;thoughts summary to say to user&quot;<br>    },<br>    &quot;command&quot;: {<br>        &quot;name&quot;: &quot;command name&quot;,<br>        &quot;args&quot;: {<br>            &quot;arg name&quot;: &quot;value&quot;<br>        }<br>    }<br>} <br>Ensure the response can be parsed by Python json.loads</pre><p>This prompt is what is submitted to ChatGPT. Copy and paste it into the model, and you’ll receive a similar response to what I provided earlier.</p><pre>{<br>    &quot;thoughts&quot;: {<br>        &quot;text&quot;: &quot;I need to find all pizza places in Bangalore and filter them by price. I can use Google search to find a list of pizza places in Bangalore and then filter them by price. I can then write the filtered list to a file.&quot;,<br>        &quot;reasoning&quot;: &quot;I need to use Google search to find a list of pizza places in Bangalore and then filter them by price. I can then write the filtered list to a file.&quot;,<br>        &quot;plan&quot;: &quot;- Use Google search to find a list of pizza places in Bangalore\n- Filter the list by price\n- Write the filtered list to a file&quot;,<br>        &quot;criticism&quot;: &quot;I need to ensure that I am filtering the list correctly and that I am writing the correct information to the file.&quot;,<br>        &quot;speak&quot;: &quot;I will use Google search to find a list of pizza places in Bangalore and then filter them by price. I will then write the filtered list to a file.&quot;<br>    },<br>    &quot;command&quot;: {<br>        &quot;name&quot;: &quot;google&quot;,<br>        &quot;args&quot;: {<br>            &quot;input&quot;: &quot;pizza places in bangalore&quot;<br>        }<br>    }<br>}</pre><p>When you use AutoGPT, it generates a prompt that includes thoughts, reasoning, and guidance on what steps to take next. The response generated by ChatGPT is the source of all the information that you see when you run AutoGPT. By leveraging the power of ChatGPT, AutoGPT is able to provide valuable insights and suggestions to help you accomplish your goals.</p><h4>What step to take next</h4><p>In the response above, you’ll notice a section on “Commands.” This section provides guidance on what action to take next and what input is needed for that action. By following the commands provided, you can effectively navigate through the problem and accomplish your goals.</p><p>It’s worth noting that ChatGPT utilizes the commands you provided in the input prompt, along with any hints or additional information you may have included. AutoGPT has a mapping of these command names to the corresponding command implementations. For example, consider the “google” command and its execution:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N59_lpc6fircEXVwpf53bw.png" /></figure><p>In this example, ChatGPT was able to understand the command “google” and provide relevant search results based on the location provided. This showcases the power of using commands with ChatGPT to accomplish specific tasks.</p><h4>What next…</h4><p>Yes, that’s correct! By using a loop that continues to run these executions, you can effectively accomplish your goals by submitting answers to AutoGPT and receiving helpful guidance from ChatGPT. This process can be repeated until your desired outcome is achieved.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bef054ba51c6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Leveraging DS Platforms for high speed deployment]]></title>
            <link>https://engineering.cred.club/leveraging-ds-platforms-for-high-speed-deployment-aab2a9afa769?source=rss-27bce89f776d------2</link>
            <guid isPermaLink="false">https://medium.com/p/aab2a9afa769</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[data-science-workflow]]></category>
            <category><![CDATA[model-repository]]></category>
            <category><![CDATA[platform]]></category>
            <category><![CDATA[deployment-automation]]></category>
            <dc:creator><![CDATA[Harish Bohara]]></dc:creator>
            <pubDate>Tue, 25 Apr 2023 06:09:17 GMT</pubDate>
            <atom:updated>2023-04-25T06:21:06.704Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CkbSh14bePAjTlIk-GDxqw.png" /></figure><p>In 2006, British mathematician Clive Humby declared “data is the new oil.”Like oil, data in raw form — sans insights and analysis has limited use.</p><p>To harness the potential on data, organizations are constantly seeking ways to leverage their data to gain a competitive edge. This is where data science comes into play.</p><p>CRED has evolved from a single product platform to a multi product platform -payments, commerce, travel, investments and more. As the platform evolves, the need to trains and deploys data science models used by various lines of business (LOB) services will also increase. These models help us make informed business decisions based on data-driven insights.</p><p>This blog details the process data science platform team adopted to support the data science team with analysis and model building.</p><p><strong>the role of data science team:</strong></p><blockquote>train models and deploy them in production</blockquote><blockquote>maintain model repository and deployment infrastructure</blockquote><blockquote>ensure model uptime and SLA</blockquote><p>as the team built more solutions they were facing certain challenges:</p><ol><li>distracted from core responsibilities by maintaining model repository and deployment infrastructure</li><li>reduced productivity due to excessive backend maintenance efforts</li><li>difficulty balancing responsibilities and core focus on data analysis and model building</li><li>hindered ability to provide valuable insights and drive business value through data analysis and model building</li></ol><h3>Goal: Enabling Efficient Data Science Workflows</h3><p>As the DS platform team, our goal was to provide a comprehensive platform that solves the challenges faced by our data science team. Our platform shold enable the data science team to focus on their core responsibilities of data analysis and model building by taking care of the backend infrastructure and deployment processes. Specifically, our platform aims to achieve the following:</p><ul><li>enable our data science team to focus on their core responsibilities of data analysis and model building</li><li>provide a full version-based model repository for model experimentation and evaluation (EDA)</li><li>provide a cost-effective platform that streamlines model deployment and ensures SLA management</li><li>significantly reduce the time it takes to deploy a model to production, from one week to a matter of hours</li><li>improving NFRs such as SLA, fault tolerance, resilience, and disaster recovery is critical for project success, enabling teams to create a more reliable system that meets customer and business demands.</li></ul><p>By providing a platform that streamlines and automates the model deployment process, the goal was to increase the productivity and efficiency of our data science team, while also improving the quality of the models they produce.</p><h3>High-Level Design</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yAhElUsBvNna-VPnm-ggwQ.png" /><figcaption>a simplified view of end-end training, building, deployment, and model inferencing</figcaption></figure><h4>Model Repository</h4><blockquote>MLFlow is our preferred platform for model management, and we’ve gone a step further by customizing it to suit our needs.</blockquote><p>We’ve integrated it seamlessly with the rest of our system to create a centralized repository for all our models. This has allowed us to streamline our workflow and deploy high-quality models quickly, driving significant value for our organization.</p><p>Our data science team leverages the platform SDK to train and log models to our MLFlow repository. By doing so, they are able to version and experiment with their models, ensuring that we are always working with the latest and greatest.</p><h4>Model Deployment &amp; Model Serving</h4><p>After logging the model to our MLFlow repository, our data science team can easily deploy it to staging or production using our platform’s one-click deployment feature. This initiates our CI/CD pipelines to build the deployment service, which is then used by our backend service for model inference.</p><p>To minimize risks during software deployment, popular strategies include:</p><blockquote><strong>Blue/Green Deployment</strong>: Maintain two identical models with one active and one inactive. Traffic is redirected to the active model once updated.</blockquote><blockquote><strong>A/B Deployment</strong>: Maintain multiple model versions and route traffic to each version for comparison.</blockquote><blockquote><strong>Canary Deployment</strong>: Direct a small percentage of traffic to the new version while most traffic uses the old version. Gradually increase traffic to the new version and eventually replace the old version.</blockquote><p>This enables us to gradually transition to the new model and ensure that it’s working as expected before rolling it out to the entire system.</p><h4>Data Platform Proxy</h4><p>Our data platform Proxy is the front-end service that enables our backend teams to call a unified endpoint instead of model-specific endpoints. This provides us with several benefits, including the ability to perform percentage rollouts and run multiple versions of models for A/B testing purposes.</p><p>In addition to these benefits, our unified endpoint approach also simplifies model logging by creating a centralized location for input/output data. This helps us detect drift and monitor the performance of our models more effectively, enabling us to make adjustments as needed.</p><p>Overall, our streamlined approach to model management, from logging and deployment to endpoint unification, has enabled us to rapidly develop and deploy high-quality models that drive tangible value for our organization.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fMV4MMYJU6gcc07bl5UFeg.png" /><figcaption>Simplified view to show End-End workflow of training/deployment/inferencing</figcaption></figure><figure><img alt="Platform deployment control panel — used by the data science team to manage models in prod/pre-prod/stage" src="https://cdn-images-1.medium.com/max/1024/1*GLo5g1ajeosAtbIzOiAPcQ.png" /><figcaption>Platform deployment control panel — used by the data science team to manage models in prod/pre-prod/stage</figcaption></figure><h3>Outcome</h3><p>Thanks to our streamlined approach to model management, we have achieved the following outcomes:</p><ul><li>reduced end-to-end deployment time from one week to less than 1–2 hours</li><li>improved productivity by centralizing model deployment, enabling us to have a common metric/monitoring layer</li><li>reduced the cost of management by using a unified endpoint approach and customizing our MLFlow repository to integrate with the rest of our system seamlessly</li><li>a unified platform enables teams to apply best practices across all channels, resulting in faster and more reliable deployment</li><li>centralized management of model deployment metadata by the platform enables effective disaster recovery measures, ensuring business continuity</li></ul><p>Overall, our approach has enabled us to rapidly develop and deploy high-quality models, delivering significant value for our organization.</p><h3>What’s next</h3><p>As the platform scales, we will plan to:</p><ol><li>extend our platform to automate end-to-end batch model training and deployment, enabling us to stay updated with the latest data and make decisions more quickly</li><li>implement a Multi-Armed Bandit (MAB) approach to run multiple models and dynamically converge toward the best-performing model</li><li>explore other innovative approaches and techniques to continually improve our model management capabilities</li></ol><p><strong>Please share your thoughts, questions, and feedback in the comments section below. Incase you are developing or deploying interesting solutions, we would love to hear about it as well.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aab2a9afa769" width="1" height="1" alt=""><hr><p><a href="https://engineering.cred.club/leveraging-ds-platforms-for-high-speed-deployment-aab2a9afa769">Leveraging DS Platforms for high speed deployment</a> was originally published in <a href="https://engineering.cred.club/">CRED Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Drools — Load rules as a string]]></title>
            <link>https://medium.com/@harishbohara/drools-load-rules-as-a-string-55406f7f9c86?source=rss-27bce89f776d------2</link>
            <guid isPermaLink="false">https://medium.com/p/55406f7f9c86</guid>
            <category><![CDATA[dynamic-rules]]></category>
            <category><![CDATA[rule-engine]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[rules]]></category>
            <category><![CDATA[drools]]></category>
            <dc:creator><![CDATA[Harish Bohara]]></dc:creator>
            <pubDate>Sun, 28 Aug 2022 09:42:50 GMT</pubDate>
            <atom:updated>2022-08-28T09:46:15.145Z</atom:updated>
            <content:encoded><![CDATA[<h3>Drools — Load rules as a string</h3><p>This is an example of using the drools rule engine. You can load the rules dynamically (you don&#39;t have to create a file and maybe you read it from DB)</p><h4>Setup a rule to execute</h4><p>We will write a rule: This is a simple rule which will update the result object and mark this filtered if the order is completed.</p><ul><li>dialect is java — I am using the java dialect</li><li>agenda-group — This can be used to control what rules are to be executed. You can have all rules in a single file and ignore a few rules</li><li>NOTE — you may have to change the import of class Result based on your package</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e8440fc07fcd611e4cefb8fba96d9bad/href">https://medium.com/media/e8440fc07fcd611e4cefb8fba96d9bad/href</a></iframe><h4>Java code</h4><p>Here an example code to run the above rules.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/438f3a0bc11a5d998d994b7b688adec2/href">https://medium.com/media/438f3a0bc11a5d998d994b7b688adec2/href</a></iframe><h4>Java maven dependency</h4><p>Use the following dependency in yoru pom.xml</p><pre>&lt;<strong>dependency</strong>&gt;<br>    &lt;<strong>groupId</strong>&gt;io.github.devlibx.easy&lt;/<strong>groupId</strong>&gt;<br>    &lt;<strong>artifactId</strong>&gt;helper&lt;/<strong>artifactId</strong>&gt;<br>    &lt;<strong>version</strong>&gt;0.0.72&lt;/<strong>version</strong>&gt;<br>&lt;/<strong>dependency&gt;<br></strong>&lt;<strong>dependency</strong>&gt;<br>    &lt;<strong>groupId</strong>&gt;org.kie&lt;/<strong>groupId</strong>&gt;<br>    &lt;<strong>artifactId</strong>&gt;kie-ci&lt;/<strong>artifactId</strong>&gt;<br>    &lt;<strong>version</strong>&gt;7.1.0.Beta1&lt;/<strong>version</strong>&gt;<br>&lt;/<strong>dependency</strong>&gt;</pre><h4>Resources</h4><p>Reference code — <a href="https://github.com/kiegroup/drools/blob/main/drools-examples-api/kiefilesystem-example/src/main/java/org/drools/example/api/kiefilesystem/KieFileSystemExample.java">https://github.com/kiegroup/drools/blob/main/drools-examples-api/kiefilesystem-example/src/main/java/org/drools/example/api/kiefilesystem/KieFileSystemExample.java</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=55406f7f9c86" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building systems to deliver customization]]></title>
            <link>https://engineering.cred.club/building-systems-to-deliver-customization-7b0672a65f6d?source=rss-27bce89f776d------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b0672a65f6d</guid>
            <dc:creator><![CDATA[Harish Bohara]]></dc:creator>
            <pubDate>Mon, 25 Jul 2022 09:36:11 GMT</pubDate>
            <atom:updated>2022-08-01T07:08:52.555Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ryqzOZoc404FuvK5UJUzJQ.png" /></figure><p>Since its launch, the CRED app has rapidly scaled to a multi-genre product with various product offerings- from e-commerce to payments, lending to rewards, travel, and investments. Our member base has also grown to 9mn+.</p><p>In this article, we detail how we work to personalize solutions and experiences for our members. The systems we deploy for member segmentation make it possible for us to understand our members, their needs, and preferences so as to customize their interaction with the app.</p><p>CRED is a platform for India’s high-trust community. We create value for our members by equipping them with the right wherewithal to spend, save and grow their money. This is made possible by teams who build premium &amp; customize experiences for reward-worthy individuals.</p><p>Let us understand how this member segmentation is made possible.</p><p><strong>But first, what is a segment in the CRED context?</strong></p><blockquote>A segment defines a set of members who fulfill some criteria. E.g. “<em>store_txn_done</em>” is a segment that is a set of all members who have done a store transaction. A member may belong to one or more segments.</blockquote><p><strong>What is a user attribute in the CRED context?</strong></p><blockquote>A user attribute is a property of a member which is not specific to a business e.g. member city, whether the member has availed “cash feature”; or member card count.</blockquote><p>As mentioned earlier, currently the CRED app has various products and features, and leveraging segments across businesses we can show relevant product offers and features to members who belong to a specific segment.</p><p>Examples:</p><blockquote>if a member uses the bill payment feature regularly, rewards for bill payment can be shown prominently to the member.</blockquote><blockquote>if a member refers friends to the CRED platform, some rewards are unlocked for the member.</blockquote><h4>Getting to know the systems that make this happen</h4><p>In ancient times, an <strong>Obelix</strong><em> </em>was built to commemorate events and honor important people.</p><p>At CRED, Obelix powered the segments and was responsible for managing user segments.</p><p>There are 3 categories of segments:</p><ol><li><strong>Realtime segments:</strong> create/update/delete based on some real-time events <br>- For e.g. member added a card -&gt; member is tagged to segment <em>X_CARD_ENABLED</em> -&gt; this segment allows different businesses to help discover the member</li><li><strong>Batch Segment:</strong> created/updated/deleted by periodic jobs</li><li><strong>Static Segment:</strong> Adhoc segments for members for the specific use case<br>- e.g. add a member to segment “<em>bill_payment_enabled</em>”, this will make the member eligible for a reward on the CRED app</li></ol><h3>Obelix High-Level Design (TLDR)</h3><ul><li>Businesses publish events, which are moved to a single queue “<em>consolidate queue</em>”</li><li>Obelix reads from this “<em>consolidate queue</em>”, and the different event has different business logic in Obelix. This logic will add/remove a segment for a user</li><li>Businesses get user segments from Obelix</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mhZ3-c9FKBrJ82R3" /><figcaption>High-Level Design</figcaption></figure><h4>How backend services/analytics team use segments?</h4><p>User segment is a list of strings e.g. [<em>city_bangalore</em>, <em>last_paid_1_week_ago</em>, <em>card_count_2</em>]. Fundamentally these segments are considered as a boolean flag true/false - if a string exists then it is true, otherwise false.</p><p>Segments like “<em>cash_enabled</em>” indicate if the user has access to the “cash” product. However you will notice that there are segments like “<em>last_paid_1_week_ago</em>”, these are created from daily job and can have other variations “<em>last_paid_2_week_ago</em>”, “<em>last_paid_3_week_ago</em>”, ….</p><p>Most of places, the backend service used a rule engine where rules were defined (outside the code rule) like the following:</p><p>Suppose you want to show cards on the home page, the following logic will be executed to get the cards for the home screen</p><pre>if “user segments”.contains(“city_bangalore”) {<br>    enable_card = true<br>}</pre><pre>if “user segments”.contains(“new_user”) {<br>   new_user_offer= enabled<br>} else {<br>   new_user_offer= disabled<br>}</pre><p><strong>The limitation and issues with the existing system:</strong></p><ol><li>This model worked fine at the beginning as we had limited segments and the requests to create new segments was low. Gradually, the count of segments exploded as “only boolean type segments” were supported.<br>- e.g. one segment for per city, one segment for no. of card - this was due to the way we designed segments as a boolean<br>- Complex segment also contributed to this exploded list e.g. “<em>user_bangalore_1_card</em>, <em>user_bangalore_2_card</em>, <em>use_delhi_1_card</em>, <em>user_delhi_2_card</em>, …”.</li><li>Since Obelix was maintained by a team, it had all the business logic to listen to events and add/update segments. As more business and product teams started using segments, the manual intervention by the platform team became a bottleneck<br>- e.g. during IPL, the platform team got 20-30 new segment requests, and requests were taken in first come first serve mode. This prioritization was a challenge for some teams where the segment they needed was onboarded after N days.<br>- This also made it harder to plan - other backend needs to coordinate the Obelix release so that they can go live</li><li><strong><em>No</em></strong> <strong><em>self-serve capability</em></strong> for businesses</li><li><strong><em>Isolation issue</em></strong> - all the events from different queries are pushed into one “<em>consolidated queue</em>”. If one business sent a burst, it impacted all other teams.</li><li>Since the business logic was sitting inside the Obelix code base, it was hard for everyone to set up Obelix, reproduce issues, and debug it.</li><li>Hard to keep complex segments in sync (composite segments)<br>- E.g suppose you have a segment that depends on 2 values (city and no of cards). <em>user_bangalore_1_card</em>, <em>user_bangalore_2_card</em>, <em>use_delhi_1_card</em>, <em>user_delhi_2_card</em> → Everytime one var changed you need to change more than 1 segments “<em>user_bangalore_1_card</em> -&gt; “<em>user_delhi_1_card</em>”</li><li>Hard to reuse existing data<br>- e.g. if a segment “X” has 3-4 param to build it, it would have listened to 1-2 streams, pulled data from 1-2 services, and materialized it. Other segments would need to pull the same data again to do the calculation for segment “Y”</li></ol><p>At CRED, we are guided by our values of being customer-obsessed, delivering outcomes, and solving elegantly. The challenges in the existing Obelix system prompted us to look for a more elegant solution for customer segmentation. In the section below, we detail the process of making UAS robust and more efficient.</p><h3>Need for UAS (User Attribute Service)</h3><p>As explained above, we found some fundamental limitations in Obelix. Apart from this, we wanted to build more capabilities from the UAS. During the planning stage of building UAS, we listed some primary requirements for UAS:</p><ol><li>A fully self-served platform<br>- Segment/attributes onboarding can be done by respective LOBs (line of business)<br>- Remove dependency on the platform team (Obelix team) in the segment creation/management process</li><li>Responsibility segregation<br>- The platform team will manage the core platform - reduce (or eliminate) platform team involvement in the attributes/segments creation process<br>- LOB business logic should be created and maintained by LOB teams<br>- LOBs to debug business logic in case of issues. Platform team to get involved in resolving platform issues.</li><li>Platform SLAs for LOB<br>- The platform will manage SLA for the core platform - availability and scalability of the core platform<br>- Resource isolation - the burst events of one LOB must not impact other LOB<br> - LOBs will manage business logic and LOB-specific runtime. LOBs manage runtime and plan for burst load. Uptime of LOB runtime is managed by LOB teams - alert/monitoring etc.</li><li>Extend segment definitions to “user attributes” -&gt; allow rich operations — e.g. equal, greater/less than operation, boolean, etc.<br>e.g.operation city=bangalore instead of “<em>city_bangalore</em>” -&gt; this will remove a need for 100+ segments<br>e.g. <em>city=bangalore</em>, <em>card_count = 2</em> -&gt; this removed need for (no of city * no of cards) segment combinations. This can be achieved with only 2 attributes.<br>- Other high-order segments with 3-4 properties had many more combinations. Instead, 3-4 attributes will cover this combination</li><li>Reduce the complexity to keep segments in sync</li><li>Discoverability of attributes/segments</li></ol><h3>UAS High-Level Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QSN-JX7IvCEdkIwQ" /><figcaption>UAS High-Level Architecture</figcaption></figure><h4>LOB</h4><p>LOBs send business/backend events (no change from the LOB side). LOBs are the source of events.</p><h4>UAS LOB Processor</h4><p>The processor is a runtime provided by the UAS team. It provides a config-based framework for a specific one or more input streams, and a Javascript engine to implement the business logic.</p><p>What is this “<strong><em>LOB Runtime</em></strong>” or “I am LOB, what am I expected to do to launch my runtime”?</p><p>As a LOB, you will clone the LOB processor git-repo. Add your config and Javascript for business logic, commit your changes, and the CI/CD will deploy your runtime. This runtime will read events from the stream, run your javascript logic, and post attributes to UAS.</p><p>What capabilities are provided by common platform runtime:</p><ol><li>Alerting/Monitoring LOBs - published to Datadog<br>- End-to-end tracing on Datadog from event received -&gt; event processing -&gt; updating to UAS</li><li>Durability/retry of UAS attributes (once JS provides results)</li><li>Providing common services to LOB scripts e.g. common services like user service, device service, etc are part of platform runtime which can be reused by LOBs</li></ol><p>A snippet of LOB configs and business logic: This is a sample to demonstrate what LOB config and business logic will look like</p><pre>LOB Config:<br>----------</pre><pre>sources:<br>  order_created:<br>    type: kafka<br>    properties:<br>      endpoint: kafka:9092<br>      topic: topic_where_backend_push_order_created_events<br>    namespaces:<br>      store:<br>         script_file: &quot;lob_order_created_logic.js&quot;</pre><pre><br>lob_order_created_logic.js<br>--------------------------</pre><pre>if(event[&#39;event_type&#39;]===&#39;order_completed&#39;) {<br>   store.order_count = store.order_count + 1<br>   store.store_user = true<br>} else if(event[&#39;event_type&#39;]===&#39;order_delivered&#39;) {<br>   store.delivered = store.delivered + 1<br>}</pre><h4>UAS</h4><p>UAS is the data service that stores all attributes in DB. It also publishes all the updates to Change data capture (CDC). It has some additional responsibilities:</p><ol><li>Access control of LOB data</li><li>Schema enforcement</li><li>Service attributes/segments</li><li>Make this data available to the data platform</li></ol><h4>Data platform (existing platform)</h4><p>The data platform pipeline listens from CDC and pushes the data to:</p><ol><li>Realtime analytical DB (Pinot) - for real-time analysis and dashboard (&lt; 10-sec data latency)</li><li>Data lake - for analysts and data science (to use in model training)</li></ol><h3>Responsibilities</h3><h4>UAS platform team runtime:</h4><ol><li>Provide runtime which can take the above configs and JS</li><li>Ensure all events from the configured topic are delivered to the LOB business logic (JS) and executed</li><li>The result of the calculation is sent to UAS and persisted</li><li>Support of runtime deployment via Git</li></ol><h4>UAS platform LOB runtime:</h4><ol><li>Business logic inside JS</li><li>Correctness of config e.g. Kafka topic etc</li><li>Uptime of runtime - e.g. no of containers are correct (based on the LOB throughput), enough CPU/RAM is configured in the deployment configuration</li></ol><h3>Low-Level details</h3><h4>Namespaces:</h4><p>We grouped user attributes into namespaces e.g.store, cash. One LOB can have 1 or more namespaces. This allowed us the following</p><ol><li>Access control at the namespace level<br>- Owner LOB can read/write<br>- The owner LOB can whitelist other LOB to read the attributes from his namespace</li><li>Scheme - Each namespace has an AVRO schema attached to it<br>- Owner LOB defines the schema upfront (can update if it does not break compatibility)<br>- Schema is used to ensure that the data posted from the LOB processor are matching the schema for a given namespace<br>- Schema allows the downstream, CDC consumers to ensure schema and also automates other tooling and platforms.</li></ol><pre>A sample schema:<br>----------------</pre><pre>{<br>    &quot;type&quot;: &quot;record&quot;,<br>    &quot;namespace&quot;: &quot;org.cred.store&quot;,<br>    &quot;name&quot;: &quot;store&quot;,<br>    &quot;fields&quot;: [<br>     {<br>       &quot;name&quot;: &quot;order_count&quot;,<br>       &quot;type&quot;: &quot;int&quot;,<br>       &quot;default&quot;: &quot;0&quot;<br>     },<br>     {<br>       &quot;name&quot;: &quot;delivered_count&quot;,<br>       &quot;type&quot;: &quot;int&quot;,<br>       &quot;default&quot;: &quot;0&quot;<br>     },<br>     {<br>       &quot;name&quot;: &quot;store_user&quot;,<br>       &quot;type&quot;: &quot;boolean&quot;,<br>       &quot;default&quot;: &quot;false&quot;<br>     },<br>  ]<br>}</pre><h3>Results:</h3><ol><li>The upgraded UAS when deployed resolved many limitations of the existing system. LOBs can onboard attributes/segments — <strong><em>self-serve</em></strong><br>- This removed dependency on the platform team and enabled product/LOBs to move faster<br>- Since attributes/segments are created by LOB teams, the backend service depending on those segments is planned by the same LOB</li><li>Attributes reduced the number of segment combination<br>- Now instead of checking for [user segment list contains “<em>city_bangalore”</em>]; the backend can check “<em>city=bangalore</em>”</li><li>Attributes are <strong>discoverable</strong> which enabled the reuse of existing data<br>- e.g. if an attribute exists “<em>store_user=true</em>” then “Cash LOB” can just read the attribute from the “Store” namespace, instead of pulling this data from different sources and knowing the business logic for “<em>store_user</em>”</li><li>LOB <strong>runtime Isolation </strong>— one LOB does not impact other LOB</li><li>Clear <strong>segregation of responsibility</strong> between LOB&lt;&gt;Platfrom — LOB maintains its own business logic, and the platform team manages platform service<br>- This also reduced time to discover issues and time to resolve the issue</li></ol><h4>This is not the end of it. In the upcoming UAS blogs we will detail:</h4><blockquote>What next? The platform on top of UAS<br>- EDP, zero code attributes/segments capability for product/ops teams</blockquote><p>Do comment below to share your thoughts on this.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b0672a65f6d" width="1" height="1" alt=""><hr><p><a href="https://engineering.cred.club/building-systems-to-deliver-customization-7b0672a65f6d">Building systems to deliver customization</a> was originally published in <a href="https://engineering.cred.club/">CRED Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kafka Integration with Spring Boot]]></title>
            <link>https://medium.com/@harishbohara/kafka-integration-with-spring-boot-5636d9f5bc8f?source=rss-27bce89f776d------2</link>
            <guid isPermaLink="false">https://medium.com/p/5636d9f5bc8f</guid>
            <category><![CDATA[kafka]]></category>
            <category><![CDATA[spring-boot]]></category>
            <dc:creator><![CDATA[Harish Bohara]]></dc:creator>
            <pubDate>Fri, 08 Jul 2022 18:39:37 GMT</pubDate>
            <atom:updated>2022-12-23T16:54:48.748Z</atom:updated>
            <content:encoded><![CDATA[<p>This article describes the integration of Kafka with Spring Boot. This is a wrapper on top of the Kafka library which provides additional functionality on top of raw Kafka integration:</p><ol><li>Resilience (using circuit-breaker)</li><li>Config-based Kafka produce/consumer</li><li>Sync/Async mode of sending messages</li></ol><h4>Integration</h4><ol><li>Add properties in your &lt;your properties file name&gt;.properties</li><li>Create the following 2 Java files — <em>MessagingConfigs</em> and <em>MessagingConfigurationBeanFactory</em></li><li>Include dependency in your Gradle or POM file — ‘<em>io.github.devlibx.easy:messaging-kafka:0.0.133</em>’</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d12222d03f047a4a54e1fcb7e229bb85/href">https://medium.com/media/d12222d03f047a4a54e1fcb7e229bb85/href</a></iframe><h4>How to use messaging to send messages to Kafka</h4><p>Here is an example of how yo use <em>IMessagingFactory</em> class to send messages to Kafka.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a3b905cffa3b8c28f7f45c52a3c055c5/href">https://medium.com/media/a3b905cffa3b8c28f7f45c52a3c055c5/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5636d9f5bc8f" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>