<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Srinivas Balivada on Medium]]></title>
        <description><![CDATA[Stories by Srinivas Balivada on Medium]]></description>
        <link>https://medium.com/@srinivasbalivada?source=rss-ca2898e7ff0c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*CUQrm5PqzlPfVqmv.</url>
            <title>Stories by Srinivas Balivada on Medium</title>
            <link>https://medium.com/@srinivasbalivada?source=rss-ca2898e7ff0c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 12:22:03 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@srinivasbalivada/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[A/B Testing — A Theoretical Approach]]></title>
            <link>https://srinivasbalivada.medium.com/a-b-testing-a-theoretical-approach-e99a21e2c318?source=rss-ca2898e7ff0c------2</link>
            <guid isPermaLink="false">https://medium.com/p/e99a21e2c318</guid>
            <dc:creator><![CDATA[Srinivas Balivada]]></dc:creator>
            <pubDate>Sun, 20 Jul 2025 19:03:00 GMT</pubDate>
            <atom:updated>2025-07-20T19:10:40.756Z</atom:updated>
            <content:encoded><![CDATA[<h3>A/B Testing — A Theoretical Approach</h3><p>A/B testing (or split testing) is a powerful experimental technique used by data scientists, product managers, marketers, and engineers to make data-driven decisions. It involves comparing two (or more) versions of a webpage, feature, or product to determine which one performs better with respect to a key business metric. Properly designed A/B tests help organizations optimize user experience, increase conversions, and reduce guesswork.</p><p>This blog post demystifies A/B testing, presenting a step-by-step guide covering the core concepts, statistical foundation, sample size calculation, execution, common pitfalls, and best practices. Drawing from industry standards and the definitive guide <em>Trustworthy Online Controlled Experiments</em> .</p><p><strong>Why A/B Testing?</strong></p><p>Imagine launching a new checkout page on your e-commerce platform hoping to increase sales. How do you know the new design truly helps rather than hurts? A/B testing allows you to empirically validate hypotheses by exposing some users (treatment group) to the new design and others (control group) to the existing one, then measuring the difference in outcomes.</p><p>Key benefits include:</p><ul><li><strong>Causal Inference:</strong> Randomly assigning users minimizes confounders and biases, enabling confident attribution of changes to the tested feature.</li><li><strong>Risk Mitigation:</strong> Testing before full rollout reduces the chance of deploying harmful changes.</li><li><strong>Incremental Optimization:</strong> Continuous experimentation fuels product and business improvements over time.</li></ul><p><strong>Key Concepts in A/B Testing</strong></p><p>Before diving deeper, let’s clarify foundational concepts:</p><ul><li><strong>Control Group:</strong> The baseline or status quo experience without any changes.</li><li><strong>Treatment Group:</strong> The group exposed to the new feature or variation.</li><li><strong>Randomization Unit:</strong> The entity on which random assignment is done — commonly users, sessions, or cookies. Proper randomization ensures unbiased comparisons.</li><li><strong>Metrics:</strong> Quantitative outcome variables used to evaluate impact. Metrics should be <strong>actionable, sensitive, interpretable</strong>, and ideally summarized in a single Overall Evaluation Criterion (OEC) to avoid ambiguity.</li><li><strong>Guardrail Metrics:</strong> Secondary metrics tracked to detect unintended side effects (e.g., page load time, bounce rate).</li></ul><p>For example, when testing a redesigned homepage, the conversion rate might be the primary metric, while page load time acts as a guardrail.</p><p><strong>Statistical Foundations: Understanding Hypothesis Testing</strong></p><p>At the heart of A/B testing lies <strong>hypothesis testing</strong>, a formal method to assess whether observed differences are likely due to chance.</p><ul><li><strong>Null Hypothesis (H0):</strong> No difference between treatment and control (e.g., the mean conversion rates are equal).</li><li><strong>Alternative Hypothesis (H1):</strong> There is a difference (treatment differs from control).</li></ul><p>Other key statistical terms:</p><ul><li><strong>Significance Level (α):</strong> Threshold probability of falsely rejecting H0 (Type I error).</li><li><strong>Power (1 — β):</strong> Probability of correctly rejecting H0 when there is a true effect (sensitivity).</li><li><strong>Minimum Detectable Effect (MDE):</strong> The smallest effect size considered practically significant.</li></ul><p>The p-value obtained from the distribution represents the probability that the difference between the means is zero. As a general guideline, if this probability (p-value) is less than 0.05, we can reject the null hypothesis</p><p>Confidence intervals provide a range around estimated effects quantifying uncertainty. Assuming large sample sizes, the Central Limit Theorem justifies approximating sampling distributions as normal, enabling these inference techniques.</p><p><strong>Sample Size Calculation</strong></p><p>Determining adequate sample size is critical to detect meaningful effects with high confidence while conserving resources.</p><p>For tests comparing two proportions (e.g., conversion rates), a commonly used approximate formula is:</p><p>n=δ216⋅p(1−p)​</p><p>where:</p><ul><li>n is the sample size per group,</li><li>p is the baseline conversion rate (control),</li><li>δ is the absolute MDE (e.g., detecting a 1% increase as 0.01),</li><li>Assumptions: significance level α=0.05, power = 80%.</li></ul><p>The formula captures the intuitive fact that rarer events (low p) or smaller effects (low δ) require larger samples.</p><p>In real-world scenarios, sample sizes might be constrained by campaign reach or available traffic, often necessitating trade-offs and iterative sample tests.</p><p><strong>Steps to Execute a Robust A/B Test</strong></p><ol><li><strong>Define Business Objective and Hypothesis:</strong> Clearly articulate what you intend to measure and why.</li><li><strong>Select Metrics:</strong> Identify primary and guardrail metrics aligned with business goals.</li><li><strong>Choose Randomization Unit:</strong> Commonly individual users or sessions, ensuring independence and sensitivity.</li><li><strong>Determine Sample Size:</strong> Based on baseline metrics, desired MDE, and acceptable error rates.</li><li><strong>Implement Random Assignment:</strong> Use deterministic bucketing (e.g., hashing user IDs) to avoid biased allocations.</li><li><strong>Run the Experiment:</strong> Ensure consistent data collection with automated pipelines.</li><li><strong>Monitor Experiment:</strong> Visualize key metrics with confidence intervals to detect anomalies early.</li><li><strong>Analyze Results:</strong> Apply hypothesis testing to evaluate outcomes.</li><li><strong>Address Multiple Testing:</strong> If conducting many tests or sequential analyses, adjust significance thresholds (e.g., Bonferroni correction).</li><li><strong>Consider Duration:</strong> Avoid premature conclusions by running tests long enough to observe stable trends and minimize novelty effects.</li><li><strong>Review Guardrail Metrics:</strong> Confirm no detrimental side effects occurred.</li><li><strong>Decide and Rollout:</strong> Based on evidence, implement or iterate.</li></ol><p><strong>Common Pitfalls and How to Avoid Them</strong></p><ul><li><strong>Insufficient Sample Size:</strong> Leads to underpowered tests, increasing false negatives.</li><li><strong>Multiple Testing Without Adjustments:</strong> Inflates false positives; correct with Bonferroni or false discovery rate controls.</li><li><strong>Novelty and Primacy Effects:</strong> Initial spikes in metrics that fade over time; requires extended monitoring.</li><li><strong>Imbalanced Groups:</strong> Unequal sample sizes bias estimates; maintain parity.</li><li><strong>Ignoring Guardrails:</strong> Optimizing one metric while negatively impacting others leads to bad overall outcomes.</li><li><strong>Poor Randomization:</strong> Introduces confounders; deterministic hashing or stratification helps.</li></ul><p><strong>Best Practices for Successful A/B Testing</strong></p><ul><li><strong>Use Meta-Analysis:</strong> Aggregate results across experiments to improve statistical power and avoid false discoveries.</li><li><strong>Automate Data Collection and Analysis:</strong> Minimize human errors and speed iteration.</li><li><strong>Visualize with Confidence Intervals:</strong> Help teams interpret significance intuitively.</li><li><strong>Continuously Monitor Metrics:</strong> Detect early signals and unexpected issues.</li><li><strong>Document Experiments:</strong> Preserve institutional knowledge and enable future learning.</li></ul><p><strong>Educate Stakeholders:</strong> Clear communication avoids misinterpretation of statistical results.</p><p><strong>Conclusion : </strong>A/B testing combines scientific methodology with practical business insights, requiring a careful balance of statistical rigor and real-world considerations. Mastering its key concepts, statistical foundations, and operational best practices enables teams to confidently execute data-driven decisions and scale their experimentation programs effectively. By following these guidelines—such as proper hypothesis setting, sample size determination, managing multiple testing, and using guardrail metrics—and continuously adapting them to the organization&#39;s context, companies can foster a culture of sustained experimentation.</p><h4><strong>If you liked this post, go ahead and give it a clap :)</strong></h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e99a21e2c318" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fundamentals on Operationalizing Machine Learning Applications in Azure ML]]></title>
            <link>https://srinivasbalivada.medium.com/operationalizing-machine-learning-applications-in-azure-ml-8c171c0a5ca7?source=rss-ca2898e7ff0c------2</link>
            <guid isPermaLink="false">https://medium.com/p/8c171c0a5ca7</guid>
            <dc:creator><![CDATA[Srinivas Balivada]]></dc:creator>
            <pubDate>Mon, 14 Jun 2021 06:53:58 GMT</pubDate>
            <atom:updated>2021-06-21T11:30:02.220Z</atom:updated>
            <content:encoded><![CDATA[<p>Fundamentals on Operationalizing Machine Learning Applications in Azure ML</p><p>Overview:<br>The data used in project is related to direct marketing campaigns of a banking institution. The primary objective is to this project is to predict whether a customer subscribe a term deposit or not in Micosoft Azure <br>The project is all about creating a optimized model from scratch by using Azure ML and deploy it as a webservice so that it can be consumed . <br>It can be done by two methods <br>1. Using Azure ML Studio <br>2. Using Pipelines</p><p>The architectural diagram, steps and screen recording is available in the</p><p># Architecture Diagram:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/353/1*Xje-rNinmtrZWhoxnvDJIg.png" /><figcaption>Reference:<a href="https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/ai/training-python-models">https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/ai/training-python-models</a></figcaption></figure><p># Process Flow:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*olGRKFvkHCLiMoeoEVeteQ.png" /></figure><p>SP Authentication:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jouwklqa32QL-UMwdghdGg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jouwklqa32QL-UMwdghdGg.png" /></figure><p># Creating Auto ML model and Executing Auto ML Model</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*piirHI02rxE83pyJ40pNXA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sbnDMTpXZTOt8-vuHYsf1A.png" /><figcaption>\\</figcaption></figure><p># Voting Ensemble model is the best model</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ow5VKOyNrB5qc7mrryKWYQ.png" /></figure><p>Deployed as a Webservice</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7UkbA_LY2ysftw2QNBzw0g.png" /></figure><p># Executed logs.py file to view the logs</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RTUElAMTX1ifzb9igGEJIw.png" /></figure><p># Swagger is running on locahost:9000</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2Xwnb4GsXIn6PhzLQ0wAuQ.png" /></figure><p># Endpoint has been created in the link shown in screenshot and tested through posting json type of input</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*HXfH5C5AgW17-ttTATswLQ.png" /></figure><p># Creating a benchmark using bash benchmark.sh and Testing the endpoint after benchmark creation</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/829/1*_NI1sIepZtxe7pwCgBU19w.png" /></figure><p># Python SDK implementation<br>Python SDK Implementation:<br>After creating new notebook and connecting with the compute instance . Pipelines are built to implement a AutoML model on the same dataset. Pipeline has been configured for AutoML and scoring<br>After submitting the pipeline the following screen shot is captured displaying the status of pipeline in SDK and designer studio<br>Pipeline Section in AzureML Studio:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1uPTu98I9bTBjiz3mDEwMQ.png" /></figure><p># Pipeline is created and running on below screenshot</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4LDx9HALHDRpt-OTWPiI0g.png" /></figure><p>#<br>![alt text](<a href="https://github.com/balivada987/nd00333_AZMLND_C2/blob/master/AutoMl12.png">https://github.com/balivada987/nd00333_AZMLND_C2/blob/master/AutoMl12.png</a>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jtMED5ULLYCjDRvaX_9V6A.png" /></figure><p># Pipeline Runs are completed and their screenshot</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mCrlT4o6rssVxmehl0h2nw.png" /></figure><p># Pipeline Endpoints are Displayed in the screenshot</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cFOfdKSw0RAXmdD86dCcVQ.png" /></figure><p># Run Details widget screenshot</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1yyuRjzXf9B7BXrtmaf_bA.png" /></figure><p># Pipeline is scheduled at regular intervals using schedule command</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pN0Hzp9uwOK5oIMdTjjMOw.png" /></figure><p># Screenshot of all the Pipeline Runs since pipeline is scheduled at a frequency of 2 minutes</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oif-tDtZggqunEZwWMiGUg.png" /></figure><p># Screencast Video Link</p><p><a href="https://www.youtube.com/watch?v=-yH1SSQmINk?feature=oembed">Operationalizing ML model in Azure</a></p><p># Areas of Improvement :<br>1.Data Imbalance has been detected as displayed in screenshot . Undersampling and Oversampling can be used to improve the distribution of data <br>2. During sampling stratified sampling can be used</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ijbb9E2wQta8FVfiQspy5g.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8c171c0a5ca7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RAKE an algorithm for key word generation .]]></title>
            <link>https://srinivasbalivada.medium.com/rake-an-algorithm-for-key-word-generation-63ada4dbbde5?source=rss-ca2898e7ff0c------2</link>
            <guid isPermaLink="false">https://medium.com/p/63ada4dbbde5</guid>
            <dc:creator><![CDATA[Srinivas Balivada]]></dc:creator>
            <pubDate>Thu, 30 Jul 2020 06:34:22 GMT</pubDate>
            <atom:updated>2021-06-14T08:40:31.161Z</atom:updated>
            <content:encoded><![CDATA[<h3>RAKE an algorithm for key word Identification</h3><p>The main steps in this algorithm are:</p><ol><li>Split Text data to array of tokens seperated by “-” .</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*8SnwLwUC5MimOxCNWfXLpA.png" /></figure><p>2. Build Cooccurence graph for each word</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*B21-D0XuU6swQdd1IjNlfQ.png" /></figure><p>Degree(word) is the sum of values of a whe word for example:Degree(set)= 2+3+1=6. Frerquency of word is how many times word appeared in corpus so frequency (set) =3</p><p>Now the metric is Degree(set)/frequency(set)=6/3=2</p><p>So for each word degree(word)/freq(word) is calculated and later summed them for concatenated words for example</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*SwEiWQdKEYVotyxh0D7MTQ.png" /></figure><p>for example</p><p>“minimal generating sets” = (Degree(minimal)/freq(minimal) +Degree(generating )/freq(generating) +Degree(sets)/freq(sets)</p><p>So the score for “ minimal generating sets “ is (8/3) +(3/1) +(3/1)= ~8.7 . Similary for other strings of the corpus is as follows. Using these scores the scores are calculated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*a2MxQaeOnoVrl-Gbfo0TFw.png" /></figure><p>Model Performance is calculated through Recall, Precision and F-measure</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QSh8WomqPc93xiGrvy_9Iw.png" /></figure><p>Conclusion: RAKE is one of the highly used method for Key word Identification.</p><p>Tables Reference: <a href="https://www.researchgate.net/publication/227988510_Automatic_Keyword_Extraction_from_Individual_Documents">https://www.researchgate.net/publication/227988510_Automatic_Keyword_Extraction_from_Individual_Documents</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=63ada4dbbde5" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>