<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vikrama Raghuraman on Medium]]></title>
        <description><![CDATA[Stories by Vikrama Raghuraman on Medium]]></description>
        <link>https://medium.com/@vikramaraghuraman?source=rss-4b6977765222------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*JrsmqY7LzLn8DcnSexgIZw.jpeg</url>
            <title>Stories by Vikrama Raghuraman on Medium</title>
            <link>https://medium.com/@vikramaraghuraman?source=rss-4b6977765222------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 23:28:15 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@vikramaraghuraman/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Fuel Price Hike: A Multifaceted Issue]]></title>
            <link>https://vikramaraghuraman.medium.com/fuel-price-hike-a-multifaceted-issue-c08db28e1ea2?source=rss-4b6977765222------2</link>
            <guid isPermaLink="false">https://medium.com/p/c08db28e1ea2</guid>
            <category><![CDATA[crude-oil]]></category>
            <category><![CDATA[fuel]]></category>
            <category><![CDATA[policy]]></category>
            <dc:creator><![CDATA[Vikrama Raghuraman]]></dc:creator>
            <pubDate>Wed, 03 Nov 2021 17:09:41 GMT</pubDate>
            <atom:updated>2021-11-03T17:09:41.206Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*KQJNIc2RQ3CKPrTT.jpg" /></figure><p>Every day India sets a new high level in fuel price. Almost all of the daily consuming products’ price holds the part which is determined by fuel price. Inflation is intimidating India’s economic recovery. The current price hike won’t impact much on demand as fuel is an inelastic demand product, but it has a grave impact on people’s livelihood.</p><h4>How is pricing done?</h4><p>Public sector Oil Manufacturing Companies in India like BPCL, IOCL, HPCL will determine the price of fuel on a daily basis based on the exchange rate, price in the international market, tax rates in India, etc. OPEC pricing is the benchmark for fuel price.</p><p>The advent of COVID-19 made OPEC stop crude production as the demand reached almost nil. Post pandemic, as most of the countries opened their economy faster than expected, the OPEC production phase was maintained slower than expected. Still, the phase is maintained in a way that demand is kept higher than supply. The Middle East countries were the sole source for India in the import. Since it has higher pricing for India than for some European countries, India shifted to import from Latin America and Africa countries a few months back. This move irked the middle east and they imposed heavier charges. Thus, diversification of fuel imports didn’t work well.</p><p>The rise in crude price is not the sole reason behind the fuel price hike. Taxation also has a major role. In 2014–2015 the total central excise duty from petrol and diesel was around 70000 crores. Now in 2020–2021, the total excise duty crossed 2.9L crores. Experts said that since petrol and diesel are not under GST, the government is earning a lot of revenue and using that for welfare schemes, but at the same time, it has bad implications on the economy. During the past one year of struggle with COVID, money has been pumped under PM Garib Kalyan Yojana scheme for free ration. Humongous amount was spent on the vaccination program. To spend here in this dire situation money from taxation from fuel was used. Though this can be a defense, still it had a painful impact on people.</p><p>20% ethanol blending initiative will reduce the fuel cost and also help in reducing vehicular emission and cut down the import of crude. But this is nowhere going to happen in the near future. Even e-vehicle is not in the near future. The only immediate relief from this is through slashing excise duty on fuel.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c08db28e1ea2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Air Pollution: Policies so for and Way out]]></title>
            <link>https://vikramaraghuraman.medium.com/air-pollution-policies-so-for-and-way-out-351218060019?source=rss-4b6977765222------2</link>
            <guid isPermaLink="false">https://medium.com/p/351218060019</guid>
            <category><![CDATA[crisis]]></category>
            <category><![CDATA[pollution]]></category>
            <category><![CDATA[policy]]></category>
            <dc:creator><![CDATA[Vikrama Raghuraman]]></dc:creator>
            <pubDate>Sun, 31 Oct 2021 02:33:51 GMT</pubDate>
            <atom:updated>2021-10-31T02:33:51.148Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mbpDHnmtNGQ5HLhH.jpg" /></figure><p>India is smothering day by day by increasing air pollution. Delhi nowadays breathes the worst air ever which engenders both from inside and outside of Delhi. In this article, let&#39;s see what are the policies implemented so for and what are the way out to improve air quality and facilitate eco-friendly living.</p><p>This article primarily focuses on <strong>four</strong> major key points that can play a pivotal role in the decision-making of making the air clean. They are briefed below.</p><ul><li>Reducing stubble burning through effective monitoring using remote sensing and tech-driven alternatives for stubble burning will help. <strong>Direct Benefit Transfer</strong>(DBT) methods like E-RUPI must be implemented fast to transfer subsidies instantly to benefit farmers.</li><li><strong>“Mission Electric” </strong>must be infused <strong>gradually(</strong>Not suddenly<strong>)</strong> pan India. The transformation from combustion to electric needs infrastructure push. Standardized state policy like the center’s <strong>FAME II(Faster Adoption and Manufacturing of Hybrid and Electric Vehicles) </strong>is needed. Likes of Delhi-Mumbai expressway which reduces travel time must be pitched.</li><li>Banning crackers pan India in one go will create a dreadful impact. Government must spend on R&amp;D to make all crackers comply with the <strong>‘reduction of 30%’</strong> emission cap. Government must reach out to Sivakasi which contributes 90% of crackers pan India.</li><li><strong>Beijing’s model</strong> of overcoming air pollution must be a guiding light for us.</li></ul><p>India is breathing the world’s worst air nowadays. <strong>22 out of 30</strong> most polluted cities are placed in Inda. More than <strong>1M</strong> people face death due to this. A high level of PM2.5, the dangerous fine particle is covering the skies. It is better to immediately have an eye over air pollution. I have plotted a donut chart below which shows top contributors to Delhi’s air pollution(Data is from SwachhIndia).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/560/1*GqpPZAfimL_nrr3KyPPe_w.png" /></figure><p>It is better to focus on the solution rather than the problem. The vision is to alleviate the cause of air pollution through transport and biomass burning first. The below image shows that winter is most polluted in Delhi as stubble burning happens but we mostly point fingers towards Diwali.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/757/1*sZ_NyLfRgbOANH9Y6Sg0dQ.png" /></figure><p>Below are the few issues which we have to set right.</p><h3><strong>1. ADDRESSING CRACKER ISSUE</strong></h3><p>Before Diwali last year, the court banned all traditional crackers and allowed only <strong>‘Green crackers’ </strong>and also restricted the setting off crackers. But despite this ban air of Delhi still got more worse.</p><p><strong>1.1. WHY BANNING IS A CONCERN?</strong></p><p>Since the court order, a plethora of fireworks factories have closed, workers protested the ban, and revenue of the Indian firework industry(the world’s second-largest after China) has nose-dived. Now states like Rajasthan and Delhi have decided to completely ban crackers this year. Banning in one go is cataclysmic in many senses. I have collected the information to showcase the threat that industries face below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/557/1*u0et4bDm1nBDkPaD_xTg7g.png" /></figure><p>8 Lakhs people of Sivakasi will lose jobs in one go if the same ban extends pan India in one go.</p><p><strong>1.2. WAY OUT</strong></p><ul><li>The government can set more R&amp;D teams by itself and help find the most optimal crackers formula which complies with the court rule of reducing 30% pollutants in the chemicals.</li><li>Govt can ease the export of crackers.</li></ul><h3><strong>2. ADDRESSING STUBBLE BURNING</strong></h3><p>Much of the air pollution in Delhi in winter is due to stubble burning in Haryana and Punjab, Severe effect is felt during November, a time when many farmers set fire to leftover rice stalks and straws after harvest since it is the most cost-effective way.</p><p><strong>2.1. WHY PROPOSED SOLUTIONS FAILED SO FAR?</strong></p><p>The government of Punjab distributed 24000 ‘<strong>Happy Seeder</strong>’, an alternative for removing the stubble and helping in the next cropping. It failed as people needed at least 50000 happy seeders to cover their entire farm.</p><p>ICAR has developed a solution named <strong>‘Pusa’</strong>, which can decompose crop residue into manure by accelerating the decomposition process. This is still unpopular among people. The below image shows all the cost-effective alternatives which are available but not effectively used.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/536/1*eVqC2-IeSrCMicC32SAXZQ.png" /></figure><p><strong>2.2. WAY OUT</strong></p><ul><li><strong>Pusa</strong> should be popularized through private players after setting a cap price.</li><li>We should pitch a <strong>direct benefit transfer(DBT)</strong> way for this sort of subsidy transfer like ERUPI can be used after launching.</li><li>Hastening idea of following ISRO protocol to monitor stubble burning through <strong>remote sensing</strong> technology</li></ul><h3><strong>3. ADDRESSING POLLUTION THROUGH TRANSPORT</strong></h3><p><strong>SAFAR</strong> under the ministry of earth sciences has found that vehicles and industries are the biggest contributors to PM 2.5 emissions in Delhi. They contributed <strong>28 percent</strong> of all PM2.5 emissions in Delhi.</p><p><strong>3.1. WHY PROPOSED SOLUTIONS FAILED SO FAR?</strong></p><p>EV adoption plan is already on track. Still, people are reluctant over the infrastructure for maintaining EV.</p><p>Not all states are giving subsidies to match the heavy purchase cost of EVs.</p><p><strong>3.2. WAY OUT</strong></p><ul><li>Govt must facilitate charging booths.</li><li>Few states which are currently suffering from pollution have come front to give EV subsidies remaining states are lethargic. All states must give subsidies to EVs as similar to <strong>FAME II</strong> of the central government.</li><li>“<strong>Mission Electric</strong>” must be gradually infused pan India.</li><li>Like Delhi-Mumbai Expressway we can go for many to reduce commute time of trucks which will reduce pollution.</li></ul><h3><strong>4. FOLLOWING BEIJING MODEL</strong></h3><p>The report, <a href="https://www.unenvironment.org/resources/report/review-20-years-air-pollution-control-beijing"><strong>A Review of 20 years’ Air Pollution Control in Beijing</strong></a>, was compiled by a UN Environment-led team of international and Chinese experts over two years. The study is for 1998 to 2017. “This improvement in air quality didn’t happen by accident. It was the result of an enormous investment of time, resources, and political will,” said Joyce Msuya, Acting Executive Director of UN Environment. The below line chart shows how PM2.5 decreased in Changping over the recent years.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*RrZtJHfLaP7HrAfpSjORYw.png" /></figure><p><strong>4.1. KEY TAKEAWAYS FROM BEIJING</strong></p><ul><li>Energy infrastructure optimization</li><li>Coal-fired pollution control</li><li>Vehicle emission controls</li></ul><p>Thus, there are various major contributors to air pollution, so instead of focusing on minor contributors, we must focus on major contributors. It is not impossible as we already have Beijing which has made this happen.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=351218060019" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Giant Leap in Instant Cross-Border Transaction]]></title>
            <link>https://vikramaraghuraman.medium.com/a-giant-leap-in-instant-cross-border-transaction-8ea3e39669d3?source=rss-4b6977765222------2</link>
            <guid isPermaLink="false">https://medium.com/p/8ea3e39669d3</guid>
            <category><![CDATA[foreign-affairs]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[india]]></category>
            <category><![CDATA[intl-affairs]]></category>
            <dc:creator><![CDATA[Vikrama Raghuraman]]></dc:creator>
            <pubDate>Mon, 20 Sep 2021 11:36:21 GMT</pubDate>
            <atom:updated>2021-09-20T11:36:21.519Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*xjArCxA6GJazYyPDqokFDw.png" /></figure><p>Cash was king, is king, and surely will be a king in near future too, but digital payments are making humongous strides in India. BCG-Google report in 2020 said,” Non-cash payment transactions which today constitute 22 percent of all consumer payments, will overtake cash transactions by 2023”. UPI is already witnessing 300+ crores transactions every month. India has added multiple fintech startups to the unicorn list. PhonePe is obsessed to hold a major market share of around 45% making GPay be at second. In the chase of becoming a cashless society, RBI has shifted gears and decided to expand mobile payments and allow cross-border payments with Singapore. Recently India has also agreed to enable and implement BHIM UPI in Bhutan and also partnered with UAE to accept UPI payments. This article is to elaborate <strong>UPI-PayNow linkage between India and Singapore.</strong></p><p>PayNow is a counterpart of UPI in Singapore. RBI and the Monetary Authority of Singapore came into concurrence to link their instant payment systems. This linkage will aid in making instant, low-cost money transfers directly from one bank to another between IND and SIN. Before praising its benefits, let’s see the present system for foreign exchange.</p><p>Say a father wants to send money from India to his son/daughter studying abroad. As per LRS(Liberalized Remittance Scheme) guidelines, remittance up to USD 2,50,000 per year is allowed. This seems cool, but the meta thing is spread and tax over that money and also the time for processing. Typical time of processing and allowing foreign exchange is 2–3 working days. Let’s say we can bear these timing constraints. As an add-on to this inconvenience, banks will charge spread which is called a card rate, which is nothing but a conversion charge. For instance, if we trade INR with USD, say the exchange rate is 74 Rs per Dollar, banks we approach will buy a dollar at this rate and sell to us at a rate of say Rs.78. This 3–5% commission is called the spread. This spread may look meager but when we look for a huge sum of money, it is hefty. In addition to this spread GST will be applied for the taxable amount of the amount which you are transferring. Transfer of 25L will typically take a spread charge of 1L. Even in Singapore, people are reaching out to Singtel Dash and other firms rather than banks as they charge lower costs than the banks.</p><p>Thus putting all this into mind we can clearly see both countries lack <strong>real-time, low-cost, cross-border</strong> transaction facilities, which led to the tie-up between the two countries. This tie-up will enhance the trade and tourism between the two countries. India expects this to come into effect by 2022.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8ea3e39669d3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Is Type 1 Error More Lethal Than Type 2 Error?]]></title>
            <link>https://vikramaraghuraman.medium.com/is-type-1-error-more-lethal-than-type-2-error-227cbd8940da?source=rss-4b6977765222------2</link>
            <guid isPermaLink="false">https://medium.com/p/227cbd8940da</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[data-analytics]]></category>
            <dc:creator><![CDATA[Vikrama Raghuraman]]></dc:creator>
            <pubDate>Thu, 26 Aug 2021 08:31:34 GMT</pubDate>
            <atom:updated>2021-08-26T08:31:34.353Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/692/1*DPlWmPc88uCW5IqFWEwwiw.png" /></figure><p>The purpose of this article is to clear the lingering question: ”Is Type 1 error more lethal than Type 2 error?”, which pops out of hypothesis testing. We do hypothesis testing to either reject or fail to reject the null hypotheses. We always give some room for error while rejecting the null hypothesis based on the level of significance we have taken. <em>For knowing more about hypothesis testing, read my previous blog on HYPOTHESIS TESTING </em><a href="https://vikramaraghuraman.medium.com/hypothesis-testing-d32ec0607721"><em>here</em></a><em>. </em>We know that we have two types of errors. Let&#39;s clear up the doubts on which error is more dangerous.</p><p>We will consider two cases:</p><p><strong>Case 1</strong>: Type 1 error is more lethal</p><p><strong>Case 2</strong>: Type 2 error is more lethal</p><h4><strong>1. When will Type 1 error be more dangerous?</strong></h4><p>I will take an example for this case. Consider a scenario of giving judgement on an accused. Here the null hypothesis is taken as the <strong>person is innocent. </strong>Here we have just two options either reject the null hypothesis or fail to reject the null hypothesis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/405/1*pLS5ZO_n4rckWDfblLuSRw.png" /></figure><p>The above table shows the decision made by the judge and the original decision. Type 1 error arises when we reject the null hypothesis when it is actually true. Here in our scenario, it means giving a verdict as guilty when he is innocent in reality. Type 2 error arises when we fail to reject the false null hypothesis. In our context, the type 2 error is interpreted as when the judge fails to accuse the guilty person. In this example, the type 1 error is more lethal as we can’t punish an innocent person. Does this the case always? No, let&#39;s see another example.</p><h4>2. When will Type 2 error be more dangerous?</h4><p>Consider an experiment where we wish to find the purity of the water in the river <strong>Ganga.</strong></p><p>Null hypothesis: <strong>Water from Ganga is safe for drinking</strong></p><p>Alternate hypothesis: <strong>Water is not safe for drinking</strong></p><p>We can frame the table for this too, but it is not needed. Here the type 1 error will be rejecting the null hypothesis and concluding that Ganga water is not safe when it is actually safe for drinking. This will make unrest among the people as they have to look for some other means of drinking water. Consider the other scenario where you commit a type 2 error. Here we conclude that the river water is safe for drinking but actually the river water is unsafe. The consequence of this is not simple. People will start using the impure river water for drinking and will get sick and may even face death. Thus, here committing the type 2 error will have worse consequences than the type 1 error.</p><p>Thus we can’t blatantly state that one error is more lethal than another. It depends on the hypothesis which we have taken. Even we can’t reduce both errors simultaneously. To understand this we have to know about the trade-off between the two errors.</p><blockquote>What is a trade-off between type 1 and type 2 errors?</blockquote><blockquote>If we plan to reduce the type 1 error by reducing the level of significance, we clearly make the rejection of the null hypothesis very tough. This shows you can’t even reject the <strong>false</strong> null hypothesis easily. This will increase the chance of type 2 error.</blockquote><p>Thus based on the problem we are working on and knowing the consequence of the types of error, we try to reduce one of the errors that is lethal, but this is done at the cost of allowing another error.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=227cbd8940da" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Confidence Intervals]]></title>
            <link>https://vikramaraghuraman.medium.com/confidence-intervals-9598461faa5?source=rss-4b6977765222------2</link>
            <guid isPermaLink="false">https://medium.com/p/9598461faa5</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[startlingstats]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Vikrama Raghuraman]]></dc:creator>
            <pubDate>Fri, 06 Aug 2021 05:30:06 GMT</pubDate>
            <atom:updated>2021-08-06T05:30:06.993Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/694/1*6ao7Cgh565a_yl0AQBal2A.png" /></figure><h3>1.Introduction</h3><p>Confidence interval is omnipresent in every field. It’s there in health, poll forecasting, etc. You might have come across the poll prediction stating the margin of error. If we are on the mission to find the mean weight of the entire population of India, then the most optimal way is to measure the weight of the entire 1.3 billion. It is not pragmatic to do so. Instead of measuring the weight of the entire population, we measure the weight of randomly selected samples from the population. This is called a point estimate(a sample estimate). Say we found the point estimate(sample mean) as 60Kg. This doesn’t prove that the population means has to be exactly 60Kg. In other words, the probability of having an exact 60Kg as the population mean is literally zero. Here comes the use of confidence intervals. So, instead of saying point estimate(a single number), we use to represent the interval estimate(range of values). We used to say that I am 95% confident that the population mean will be within this range of value. The purpose of this article is to explain the methodology to find the confidence interval and to throw some light on when will traditional confidence interval finding method fails.</p><h3>2. Calculating Confidence Intervals</h3><p>For calculating the confidence interval we must follow four steps:</p><ol><li>Collect samples and find sample estimate(say sample mean)</li><li>Find the standard deviation of the sample</li><li>Find the number of elements in the sample</li><li>Find confidence interval</li></ol><p>If we are in the work of calculating the mean blood sugar level of a certain group(call is as population), firstly, we collect random samples of size n(say 50) from the population. Once we have 50 blood sugar levels, we estimate the mean out of it. Then find the standard deviation of the sample. We are good to go now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/294/1*vUoA3OUyh7PWArTWXBGPfQ.png" /></figure><p>Here,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/223/1*JkqSZwXG7Q3-vqeLR6qyhA.png" /></figure><p>The one strange thing here is the confidence level value. Let us clear the air. This doesn’t simply mean the percentage value. It is not 95% if we want to find the range of values with 95% confidence. It is found from the t-table while having the confidence level(95% in our case) and degree of freedom(sample size-1, 49 in our case) in mind.</p><p>Thus we can find the confidence interval which says that we are 95% confident that the population mean blood sugar level will be in this interval.</p><p>But wait……<strong>Is this way of taking 50 samples out of a huge population optimal? There are some limitations.</strong></p><p>Usually, the traditional method that we have seen makes some assumptions such as the sample size is large(which fails in our case) and assumes the population and sample distribution as normal(which is not always possible).</p><p>So, what&#39;s the way out? Bootstrapping is the way to get rid of these sorts of issues.</p><h3>3. Bootstrapping</h3><p>Bootstrapping is a method of resampling from the samples. This makes us to have more data than before. Also if we use the bootstrapping technique then we can use the central limit theorem to assume a normal distribution. So both the problems of the traditional method are solved.</p><blockquote><strong>What does the central limit theorem state?</strong></blockquote><blockquote>The distribution of sample mean of any distribution will approximate to normal distribution if we calculate sample mean for large number of times.</blockquote><p>Let&#39;s look at the procedure for bootstrapping.</p><p><strong>Step 1</strong>: As usual collect samples from the population</p><p><strong>Step 2</strong>: Calculate estimate(say mean) of that sample</p><p><strong>Step 3</strong>: Now, resample the sample data with replacement</p><p><strong>Step 4</strong>: Find the sample estimate(mean) for the sample found in step 3</p><p><strong>Step 5</strong>: Repeat step 3 and step 4 a large number of times(say 1000 times)</p><p>Yes, we have to resample it 1000 times or even more. It is difficult to get 1000 different samples because it involves human surveying. But it is not difficult to resample one sample 1000 times, as it involves all computer computation.</p><p>Now if we plot all the sample mean from all resamples we will get normal distribution as per the central limit theorem.</p><p><strong>Step 6</strong>: Find the standard deviation for the sample means</p><p><strong>Step 7</strong>: Find the confidence interval as same as before.</p><blockquote><strong>What influences the range of confidence intervals?</strong></blockquote><blockquote>If we are uncertain about the result, the range will be large and if we have a small sample size, we will get a large range of confidence interval.</blockquote><p>So using bootstrapping we can even find the confidence interval for small samples. Bootstrapping is not only helpful in finding confidence interval but also in hypothesis testing and many others.</p><h3>4. Practical example</h3><p>Here we will look into one of the practical examples where the confidence interval is used. Consider a fish seller who wishes to sell all the fish that he grows but is hesitant about when to sell so that he will get more weight. He wishes to find the mean weight of the fishes. Say there are 10000 fish in his pond. He can’t measure the entire population of 10000 fish. Here confidence interval through bootstrapping will help. He can take a sample of say 300 fish and resample it and find the confidence interval. If he finds the interval as [2.5,3.75] in Kg with 95% confidence. Then he can decide whether to sell now or not.</p><h3>5. Conclusion</h3><p>To reiterate, the confidence interval is useful to represent the population mean(estimate) in the form of an interval from the sample mean(estimate). We have also seen a practical application of confidence intervals. I hope this sums up the concepts behind confidence interval.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9598461faa5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Assumptions of Linear Regression]]></title>
            <link>https://vikramaraghuraman.medium.com/assumptions-of-linear-regression-b1a9b2c62470?source=rss-4b6977765222------2</link>
            <guid isPermaLink="false">https://medium.com/p/b1a9b2c62470</guid>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[assumptions]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Vikrama Raghuraman]]></dc:creator>
            <pubDate>Tue, 27 Jul 2021 02:42:31 GMT</pubDate>
            <atom:updated>2021-07-27T02:44:46.203Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/693/1*Acau91Jvg15RLH2EO6A82w.png" /></figure><p>Whenever we come across regression use cases, linear regression is the first algorithm that pops into our minds. Yes, linear regression is phenomenal at multiple instances, but it can’t be applied for all regression problems. Linear regression works under some assumptions. Only if these assumptions satisfy we can apply it to fit the model. The purpose of this article is to give a glimpse of those assumptions.</p><p>In general, there is no certain number of assumptions, but in general, we say four important assumptions.</p><h3>1. Linearity</h3><p>Linear regression will assume that the relationship between the predictor and response variable is linear. How to check? A Scatter plot is the best means to check this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/284/1*kn9ORgp9TXGvaWH-Q14ydQ.png" /><figcaption>Linear relationship</figcaption></figure><p>In the above plot, the distribution is linear so the fit line fits well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/283/1*jKDhPLwRrKOTqs9ESbXZMA.png" /><figcaption>Curvilinear relationship</figcaption></figure><p>In the above plot, the predictor and response variable is not linearly related. The major difference between the polynomial regression and linear regression is in polynomial regression there is no rule that the relationship must be linear, it can be curvilinear too, but in linear regression, we can fit only through a straight line. So, when the data is non-linear, the fit value out of the linear fit model is incorrect.</p><h3>2. Homoscedasticity</h3><p>Homoscedastity means constant variance. This is another assumption made while performing linear regression. Here the assumption is that the variance of the error is constant throughout the X-axis(predictor).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/92/0*ktAldTUu7jEHn7m2.png" /></figure><p>In the above equation, <strong>ε </strong>is the error term. We always assume that <strong>E(ε) =0 </strong>(Expectation or mean of all error is zero)<strong> </strong>and <strong>Variance(ε)=Constant.</strong></p><p>If the above conditions fail then it is heteroscedasticity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/275/1*dmeNFzP2fyh_tE-UQZs4Dg.png" /></figure><p>This plot shows that the error is varying constantly all through the X-value.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*aSIuQeib8CcsTCQcKH2zQg.png" /></figure><p>In the above plot, heteroskedasticity prevails. The variance of the error is not the same all through the x-value, it increases as x increases.</p><blockquote>Why do we care about heteroscedasticity?</blockquote><blockquote>If heteroscedasticity is found then the estimated coefficient value is reliable but the inference got out of standard error and p-value are not reliable and they are uncertain. Explaining this is not the purpose of this article.</blockquote><h3>3. No Multicollinearity</h3><p>Let us understand this with an example. Consider a use case of predicting blood pressure from body weight and body surface area(BSA). Here blood pressure is a response variable. Weight and BSA are predictor variables. By measuring the correlation between the variables we found that BSA and weight are correlated heavily(say r=0.85). This way of correlation between one predictor and another predictor variable is called multicollinearity.</p><blockquote>Why do we care about multicollinearity?</blockquote><blockquote>We usually interpret each and every coefficient by holding other variables constant. This becomes impossible in the case of multicollinearity.</blockquote><blockquote><em>BP=b0+b1(Weight)+b2(BSA)+ϵ</em></blockquote><blockquote>We interpret <strong><em>b1</em></strong> as blood pressure when the weight is increased by one unit keeping BSA constant. This is impossible because when increasing weight by one unit, BSA also increases as they are correlated to each other.</blockquote><h3>4. Normality of Error</h3><p>We assume in linear regression that the error is normally distributed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/309/1*_Hwr3eWpa1Ek4c5q4zI1OQ.png" /></figure><p>In the above plot, we can see more data points are nearer to the fit line than being away. When we have more and more data points nearer to the fit line, this gives the distribution of the error as normal. Linear regression always assumes the error is normality distributed with the mean of zero error.</p><p>Besides these four assumptions, there are a few more assumptions too, but these are the important four. Violating these assumptions may mislead the interpretation of the model output/prediction and sometimes give invalid results too.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b1a9b2c62470" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hypothesis Testing]]></title>
            <link>https://vikramaraghuraman.medium.com/hypothesis-testing-d32ec0607721?source=rss-4b6977765222------2</link>
            <guid isPermaLink="false">https://medium.com/p/d32ec0607721</guid>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Vikrama Raghuraman]]></dc:creator>
            <pubDate>Sun, 18 Jul 2021 19:51:05 GMT</pubDate>
            <atom:updated>2021-07-18T20:05:59.516Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/692/1*qdPgtDq5pvyh5TWL4P2P3g.jpeg" /></figure><p>Hypothesis testing is always a buzzing word in inferential statistics. There is always a perplexity that surrounds this topic, so It’ll be better if we clear the air. Firstly, what is hypothesis testing? It is a significance test that is used to prove that the result that we have obtained is not because of mere chance. Wait, I know it still confuses, so let&#39;s start with an example. Consider a scenario where a bike company advertises that the mean mileage of their bike is 50 Km/Litres. If that figure of 50 seems to be suspicious how do we test that?. The impeccable method is to collect all of that bikes in the market and measure the mileage and taking the mean out of it. But, practicality matters, it is impossible to implement the above method, so what we do is we take a sample of bikes(say 100 bikes) from the entire population randomly and measure the mean mileage and try to reject the claim made by the company or we may fail to reject that claim. In short, it is like telling something about the population from analyzing the randomly picked sample from that population. Now we know what hypothesis testing is superficially, so let’s dive deep.</p><h3>1. Steps Followed in Hypothesis Testing</h3><p>Before going into each and every bit of hypothesis testing, first, let us know the steps involved in hypothesis testing, and then we will clarify each and every step.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/926/1*O8r2SnYjDv50A-jZp-WcTw.jpeg" /></figure><p>Don’t worry if you are not familiar with the terms given above. The purpose of this article is to clarify all these terminologies.</p><h3>2. Declaring Null and Alternate Hypothesis</h3><p>From the initial claim made by the company(refer to the example given in the introduction), we categorize it either in a null hypothesis or alternate hypothesis. The null and alternate hypothesis are always compliment to each other(they are mutually exclusive). If the claim comes with an equal sign then put it as a null hypothesis. I know it’s confusing. Ahh…Let us look at an example - if the researcher claim that the average time of recovery of a new drug is less than 72 hours, then put this statement as a null hypothesis because we get equal to sign here. If we consider μ as a mean, then below is the way how we split the hypothesis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/388/1*GssGo3bWoCQnYuaARo8F3w.png" /></figure><p>Now let us look at another example. College research claims that the mean duration the students spend in a day on social media is less than 3 hours. The below figure shows how to categorize this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/412/1*jpcA3pkQQK5qrxbYimSLbg.png" /></figure><p>Now we can see that our claim is in the alternate hypothesis. So, the claim can either be in null or alternate hypothesis, just include the one with equal to sign in the null hypothesis. Now our first step in the hypothesis testing is done.</p><p>Remember that we can make only two decisions out of hypothesis testing. Either we may<strong> reject the null hypothesis</strong> or we may f<strong>ail to reject the null hypothesis.</strong></p><h3>3. Set the Level of Significance</h3><p>Before understanding what the level of significance is, let us understand the types of error in hypothesis testing. So at the end of the hypothesis testing, we make a conclusion. Whatever we conclude either rejecting the null hypothesis or failing to reject the null hypothesis, we are not 100% sure. You may sometimes reject the null hypothesis which is actually true or you may fail to reject the null hypothesis which is actually false. In both of the scenarios, it creates errors.</p><h4>3.1. Type I error</h4><p>This error arises when we reject the null hypothesis when it is actually true.</p><h4>3.2. Type II error</h4><p>This error arises when we fail to reject the null hypothesis when it is actually false.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/373/1*GPHqHrRETiB1kqcpPzKD3Q.png" /></figure><p>The above image will clear the air about errors.</p><p>Now we are clear about the errors, so let&#39;s dive into the concept of <strong>level of significance. </strong>It is the maximum allowable probability of making a type I error. The usual value that we set for the level of significance is 0.05. If you feel puzzled, then study till the last step of the hypothesis testing(Using decision rule to conclude).</p><h3>4. Collecting Sample Data and Finding Test Statistics</h3><p>The next step after setting the level of significance is collecting a random sample and calculating the test statistics. If we consider the same example where the bike company claims the mean mileage to be 50(remember this example while reading the further part of this article), here we collect the random sample of say 30 bikes and then we find the mean mileage of those 30 bikes. Say we have calculated the mean mileage as 47 and the standard deviation as 5.5.</p><p>The standard deviation that we found is the standard deviation of the sample but not the population, so to estimate it we use the below formula.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/292/1*5lSL86-KO_S_vhx6BYMoBg.png" /></figure><p>We have estimated the standard deviation of the population as 1(approx). Now it is time to plot the distribution considering that the null hypothesis is true(that is by considering the mean as 50).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/673/1*4yBxNDSZ1wgQ-YlJ9itBBg.png" /></figure><p>Here we have taken the mean as 50 and we have plotted 47 with the standard deviation of 1. Wait, why do we use normal distribution when we don&#39;t even know what will the distribution of population?. The answer to this question is the definition of the <strong>central limit theorem. It states that the sample distribution of the sample mean will almost be normally distributed.</strong></p><p>When we standardize the value of 47, we will get -3.</p><h3>5. Finding p-value</h3><p>From the standardized z value, we find the p-value from the z-table. The p-value is around 0.0013.</p><h3>6. Using Decision Rule to Conclude</h3><p>Now we have the p-value of 0.0013. The decision rule is as follow</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/354/1*481x5rJVGz1A4RVsc4chzQ.png" /><figcaption>LOS means level of significance</figcaption></figure><p>Our p-value is less than the threshold so we reject the null hypothesis(claim of the bike company) that the new model bikes give average mileage of 50.</p><p>But, what does the p-value say? It says under the null hypothesis(that is considering the mileage as 50, based on this consideration we have plotted a distribution), the probability of getting the mean value of 47 as mileage is 0.0013(which is less than 1 percent). This probability shows that this case is very rare so we reject the null hypothesis. Here we have set the value of LOS as 5%(0.05), so this means that even if we get a p-value of 0.049, we will reject the null hypothesis. This is the reason sometimes we get type I error. We reject the null hypothesis when there is say 4.9% probability of that happening, yes 4.9% is absolutely rare but not impossible(this gives rise to Type I error sometimes).</p><blockquote>Why is LOS not set to zero?</blockquote><blockquote>If LOS is set to zero, then we can’t reject the null hypothesis. This makes hypothesis testing useless.</blockquote><blockquote>Why does type II error increase with the decrease in type I error?</blockquote><blockquote>If LOS value is low, then rejecting the null hypothesis becomes very difficult because we reject the null hypothesis based on LOS value. This will reduce the type I error(refer to definition above). On the other hand, since it is very difficult to reject the null hypothesis, we fail to reject the null hypothesis even when it is actually wrong. This makes the type II error increase.</blockquote><blockquote>Does failing to reject the null hypothesis means the null hypothesis is true?</blockquote><blockquote>No, It only means that we are not able to provide enough evidence to back up the alternate hypothesis.</blockquote><p>In this way, we do hypothesis testing to reject/ fail to reject any claims made on the population by just taking sample statistics. This article is bit long, but to grasp the entire concept of hypothesis testing, it is worth reading the entire content.</p><h4>Important Questions that we discussed here:</h4><ol><li>Why is LOS not set to zero?</li><li>Why does type II error increase with the decrease in type I error?</li><li>Does failing to reject the null hypothesis means the null hypothesis is true?</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d32ec0607721" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Linear Regression-Deep Dive]]></title>
            <link>https://vikramaraghuraman.medium.com/linear-regression-deep-dive-92129568165c?source=rss-4b6977765222------2</link>
            <guid isPermaLink="false">https://medium.com/p/92129568165c</guid>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[regression]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Vikrama Raghuraman]]></dc:creator>
            <pubDate>Thu, 08 Jul 2021 19:55:25 GMT</pubDate>
            <atom:updated>2021-07-09T02:04:42.794Z</atom:updated>
            <content:encoded><![CDATA[<p>When we simply call <strong>linear regression</strong>, it is customarily tagged to <strong>simple linear regression. </strong>In layman’s terms, linear regression is a model that provides a relationship between two variables. To elaborate, it provides a relationship between one predictor(Independent) variable and another target(dependent) variable. I know one question would have popped up in the back of your mind: <strong>If linear regression is all about the relationship between two variables, then in what way is it different from correlation? Correlation</strong> just gives the strength and direction of a relationship but regression quantifies the nature of the relationship. The spectrum of application of linear regression is wide. It extends its wings all across business, agriculture, healthcare, etc. In the language of food, if data science is on our dining plate then linear regression is a starter. Let’s dive deep into the fundamentals of linear regression.</p><h3><strong>1. Understanding the equation</strong></h3><p>The equation used to relate two variables is given below</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/115/1*4YSSaycMBGQHqzPS_b_cLw.png" /><figcaption>(Refer <strong>terminology alert</strong> section)</figcaption></figure><p>Here, y is the target, response variable, x is the predictor variable, epsilon is the error. This error term comprises both reducible and irreducible errors. Most people <strong>confuse</strong> <strong>residuals with irreducible errors</strong>. residuals are just differences between the actual and predicted value. Irreducible error is one that can’t be reduced however you build the model at your best. It is an error that is a random noise that can also be while collecting the data. For instance, during poll prediction, we pick a sample of people from the entire population, this sample may or may not be representative of the entire population. This sort of error also adds up to an irreducible error that can’t be reduced by optimizing our model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/459/1*g3MEACce201yabERd0CTcg.jpeg" /><figcaption>Scatter plot with fit line</figcaption></figure><p>As shown above, in linear regression, a straight is used to fit the model and predict the results.</p><h3><strong>2. How does the regression model learn?</strong></h3><p>It is always good to see the best fit line in the above image, but how does the model conclude that the above fit line is the best? There is a hardcore maths that will run behind. As an analyst, we must understand the mathematics behind any algorithms, so let’s delve and have some glimpse of maths.</p><p>There are two methods by which the linear regression model finds the best fit line.</p><ol><li>OLS(Ordinary Least Square)</li><li>Gradient Descent</li></ol><p>Before discussing them the core of finding the best fit is to minimize the reducible error(residual).</p><p>Residual=Actual value-predicted value</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/139/1*iPomc40GFkrrUbSd9WojCA.png" /><figcaption>(Y — actual value and hashed Y— predicted value)</figcaption></figure><p>We consider the sum of squared residuals(RSS) as the cost function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/147/1*RXU71XiXLRP1L03kPUnQyQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/112/1*X-5aPJwcEPPPsFLZZr6VTg.png" /><figcaption>(Predicted value)</figcaption></figure><p>Residuals are squared in order to avoid canceling each other. So whatever method we follow to find the best fit, it will try to reduce the RSS in common.</p><p>Based on the methodology of finding the fit line that has low RSS, it can be categorized either as OLS or gradient descent.</p><p><strong>2.1 Maths behind OLS</strong></p><p>Here to find the minimum RSS, we have two parameters to control, b0 and b1. So we need to find the optimal value of b0 and b1 so that RSS is minimum. Here we use argmin function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/217/1*j3mU1o2GLyCNF4kOyt-j-Q.png" /></figure><p>From the acumen of calculus, we know that to find the minimum value of any function we first equate its derivative to zero. Since we have two variables b0 and b1, we do partial differentiation w.r.t both variables and find the optimal b0 and b1 values to fit the line.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/248/1*joSS6HZhelKgHxcujLEO9g.png" /><figcaption>(ybar— mean of y and xbar — mean of x)</figcaption></figure><p>In the same manner, we differentiate with respect to b1 and find it.</p><p>So, in this way of analytical method, we can find the b0 and b1 values for minimum RSS and we can find the optimal fit line.</p><p><strong>2.2 Gradient descent</strong></p><p>It is the most common technique used in multiple algorithms to get our optimal parameters’ value. We can also use this technique to find the optimal b0 and b1 values. It is an <strong>iterative</strong> method. Here the initial value of b0 and b1 is taken in random and through a number of iterations, the optimal values are found. Explaining the maths behind gradient descent is out of the scope of this article.</p><h3><strong>3. Conclusion</strong></h3><p>Here we have discussed the equation of the fit line, learning procedure, and the mathematics behind the learning process. Though the linear regression model is easy to train and run, some people may end up in the wrong conclusion i. e correlation will be taken as causation(<em>refer to terminology alert section</em>). Just because we see a linear relationship between two variables in the regression model, that doesn’t mean one variable caused the other variable.</p><blockquote><strong>Why correlation is not equal to causation? </strong>Let us consider an example, in Canada in a certain time number of deaths increased suddenly. At the same time, the air conditioners sales also increased. Thus analysts found a correlation between the number of deaths and AC sales. This doesn’t mean that increase in AC sales caused the death of people. The true reason behind this was the temperature at that time was extremely terrible, this led to increased AC sales and also the number of people died out of heatstroke because of the scorching sun. This shows correlation doesn’t mean causation.</blockquote><p>So we have to be very careful while interpreting the results of the regression model.</p><h3>Terminology alert!!</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/921/1*D5u06CcdBbzZGaWqhQJdaA.png" /><figcaption>(To be remembered)</figcaption></figure><p><strong>Interview questions that have been discussed here:</strong></p><ol><li>Why correlation ≠ caussation?</li><li>OLS vs GD</li><li>Residuals(reducible) vs Irreducible error</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=92129568165c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>