<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Arnab Dey on Medium]]></title>
        <description><![CDATA[Stories by Arnab Dey on Medium]]></description>
        <link>https://medium.com/@arnab-dey?source=rss-44286b7db55------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*FmQz6J-DyNFF-HEVxVeXuA.jpeg</url>
            <title>Stories by Arnab Dey on Medium</title>
            <link>https://medium.com/@arnab-dey?source=rss-44286b7db55------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 03:24:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@arnab-dey/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Modernization of Healthcare Industry and its Future Applications]]></title>
            <link>https://arnab-dey.medium.com/revolutionizing-health-industry-machine-learning-and-future-7f16a9b70e94?source=rss-44286b7db55------2</link>
            <guid isPermaLink="false">https://medium.com/p/7f16a9b70e94</guid>
            <category><![CDATA[covid19]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[future-technology]]></category>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[patient-care]]></category>
            <dc:creator><![CDATA[Arnab Dey]]></dc:creator>
            <pubDate>Mon, 14 Jun 2021 08:19:54 GMT</pubDate>
            <atom:updated>2021-12-04T17:54:16.218Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*KjWvZjO0l5LbGFDL.png" /></figure><p>With digitalization disrupting every industry, including healthcare, the ability to capture, share and deliver data is becoming a high priority. Machine learning, big data and artificial intelligence (AI) can help address the challenges that vast amounts of data pose.</p><p>Machine learning can also help healthcare organizations meet growing medical demands, improve operations and lower costs.</p><p>At the bedside, machine learning innovation can help healthcare practitioners detect and treat disease more efficiently and with more precision and personalized care.</p><h3>What Is Machine Learning? An Overview</h3><p><a href="https://www.ibm.com/cloud/learn/machine-learning">Machine learnin</a>g is one of the most common forms of AI. It processes and finds patterns in large data sets to enable decision-making. Machine learning applications consist of <a href="https://www.investopedia.com/terms/a/algorithm.asp">algorithms</a>: a collection of instructions for performing a specific set of tasks. The algorithms are designed to learn from the data independently, without human intervention.</p><p>Over time, machine learning algorithms improve their prediction accuracy without requiring programming. A deep dive into what machine learning is reveals <strong>three critical components of algorithms</strong>: representation, evaluation and optimization.</p><p>Representation means that data must be classified in a form and language that a computer can handle. This component sets the stage for the next component, evaluation, to determine whether the data classifications are useful.</p><h3>How Machine Learning in Health Informatics Impacts Healthcare</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/694/1*cNiQQU-_Z_XPU7cPwuo3pA.png" /></figure><p>The following sections discuss three areas in which machine learning in health informatics impacts healthcare.</p><h3>Recordkeeping</h3><p>Machine learning in health informatics can streamline recordkeeping, including electronic health records (EHRs). Using AI to improve EHR management can improve patient care, reduce healthcare and administrative costs, and optimize operations.</p><p>One example includes natural language processing, which enables physicians to capture and record clinical notes, eliminating manual processes.</p><p>Machine learning algorithms can also make EHR management systems easier to use for physicians by providing clinical decision support, automating image analysis and integrating telehealth technologies.</p><h3>Data Integrity</h3><p>Gaps in healthcare information can result in machine learning algorithms making inaccurate predictions, which can negatively impact decision-making in clinical settings.</p><p>Since healthcare data is originally intended for EHRs, the data must be prepared before machine learning algorithms can effectively use it.</p><p>Health informatics professionals are responsible for maintaining data integrity. Activities that health informatics professionals perform include gathering, analyzing, classifying and cleansing the data.</p><h3>Predictive Analytics</h3><p>The combination of machine learning, health informatics and predictive analytics offers opportunities to improve healthcare processes, <a href="https://healthitanalytics.com/features/how-machine-learning-is-transforming-clinical-decision-support-tools">transform clinical decision support tools</a> and help improve patient outcomes. The promise of <a href="https://healthitanalytics.com/features/what-is-deep-learning-and-how-will-it-change-healthcare">machine learning’s changing healthcare</a> lies in its ability to leverage health informatics to predict health outcomes through predictive analytics, leading to more accurate diagnosis and treatment and improving physician insights for personalized and cohort treatments.</p><p>Machine learning can also provide additional <a href="https://www2.deloitte.com/us/en/insights/topics/analytics/predictive-analytics-health-care-value-risks.html">value from predictive analytics by</a> translating data for decision-makers to uncover process gaps and improve overall healthcare business operations.</p><h3>Other Applications of Machine Learning in Healthcare</h3><p>Machine learning can positively impact patient care delivery strategies. For example, it can help clinicians identify, diagnose and treat disease. Applications of machine learning in healthcare can also streamline healthcare tasks and optimize surgery planning, preparation and execution.</p><p>Here are some examples of machine learning applications in healthcare.</p><h3>Disease Identification and Diagnosis</h3><p>Machine learning algorithms can detect patterns associated with diseases and health conditions by studying thousands of healthcare records and other patient data.</p><p>Recent developments in machine learning can help increase healthcare access in developing countries and innovate cancer diagnosis and treatment. <a href="https://www.entrepreneur.com/article/341626"><em>Entrepreneur</em> reports</a> that a deep learning-based prediction model developed at the Massachusetts Institute of Technology can predict breast cancer development years in advance. COVID -19 can also be predicted.</p><p>Additionally, according to an <a href="https://journalofethics.ama-assn.org/article/ethical-dimensions-using-artificial-intelligence-health-care/2019-02"><em>AMA Journal of Ethics</em> article</a>, AI applications in healthcare “can now diagnose skin cancer more accurately than a board-certified dermatologist.” The article points to machine learning’s additional benefits, including diagnostics speed and efficiency and a shorter time frame for training an algorithm versus a human.</p><h3>Medical Imaging Diagnosis</h3><p>CAT scans, MRIs and other imaging technologies offer such high-resolution detail that going through the megapixels and data can challenge even experienced radiologists and pathologists.</p><p>Machine learning has demonstrated its value in helping clinical professionals improve their productivity and precision. Common use cases for <a href="https://healthitanalytics.com/news/top-5-use-cases-for-artificial-intelligence-in-medical-imaging">machine learning in medical imaging</a> include identifying COVID-19, cardiovascular abnormalities, detecting musculoskeletal injuries and screening for cancers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*P7lDIT-6tIyh1vPj.jpg" /></figure><h3>Robotic Patient Surgery</h3><p>Machine learning can use real-time data, information from previous successful surgeries and past medical records to improve the accuracy of surgical robotic tools. The benefits include reduced human error, aid during more complex procedures and less invasive surgeries.</p><p>For example, robots can precisely conduct operations to unclog blood vessels and even aid in <a href="https://www.roboticsbusinessreview.com/health-medical/medtronic-treats-first-us-patients-with-spinal-surgery-robot/">spine surgery</a>. Surgical robotics can also offer more than mechanized assistance to surgeons by planning workflows and executions for surgical procedures.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/0*dxxxCLwFZRnct4xP.jpg" /></figure><h3>Robotic Patient Support Tasks</h3><p>Robots can help <a href="https://www.roboticsbusinessreview.com/health-medical/6-ways-ai-and-robotics-are-improving-healthcare/">augment patient abilities</a> directly. Examples include helping paralyzed patients regain walking ability and performing tasks such as taking blood pressure and providing medication reminders to patients. Robots can even provide companionship to sick and older patients.</p><h3>Personalized Medicine</h3><p>Health data from various sources, including EHRs and genetic data, can help advance personalized care. A health practitioner doesn’t have enough time in a day to analyze all the data to provide <a href="https://www.forbes.com/sites/insights-intelai/2019/02/11/how-machine-learning-is-crafting-precision-medicine/#6056bca05941">precision medicine</a> to patients.</p><p>However, machine learning, with its ability to leverage big data and predictive analytics, creates opportunities for researchers to develop personalized treatments for various diseases, including cancer and depression.</p><h3>What Is the Future of Healthcare Technology?</h3><p>Health informatics professionals stand at the entryway of opportunity, playing a key role in enabling machine learning’s integration into healthcare and medical processes. Their in-depth knowledge of technology and how it can be applied to improve patient care and outcomes offers enormous value to an evolving healthcare industry increasingly reliant on data.</p><p>The deep-learning algorithms of machine learning can trim the time it takes to review patient and medical data, leading to faster diagnosis and speedier patient recovery.</p><p>Machine learning has already proven useful in the current global pandemic. As <a href="https://healthitanalytics.com/news/deep-learning-model-predicts-covid-19-surges-7-days-into-the-future">HealthITAnalytics reports</a>, a deep-learning tool can predict COVID-19 surges in U.S. counties with nearly 65% accuracy. As healthcare organizations seek to integrate machine learning into healthcare and medical processes, a primary responsibility of health informatics professionals — to ensure that healthcare data is reliable — becomes a high priority.</p><p>Various technology-driven healthcare concepts show promise in improving care delivery in the coming years. These innovations will also transform the health informatics professional’s role.</p><p>What’s the future of healthcare technology? Here are examples of technologies that will impact healthcare in the years to come.</p><h3>Virtual Reality</h3><p><a href="https://medicalfuturist.com/5-ways-medical-vr-is-changing-healthcare/">Virtual reality (VR) is changing healthcare</a> by transforming patients’ lives and making it easier to train doctors.</p><p>For example, surgeons wearing special VR headsets can stream operations and provide medical students with a unique view of a surgical procedure. In another example, VR is being used to help speed up recovery in physical therapy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*c-PME8CeC7PiTRXJ.png" /></figure><p>Patients going through physical therapy often endure strenuous physical activities that can feel burdensome. Through VR training exercises with machine learning, recovery programs can be personalized and make physical therapy activities more enjoyable and engaging.</p><h3>Augmented Reality</h3><p>Augmented reality (AR) is among the top three technologies transforming healthcare, <a href="https://medicalfuturist.com/ten-ways-technology-changing-healthcare/">according to The Medical Futurist</a>. Similar to VR, AR applications in healthcare can help better prepare medical students. AR technologies can provide students with opportunities to learn directly from surgeons performing real-life surgeries.</p><p>For example, AR enables medical students to get detailed, accurate depictions of human anatomy without studying real human bodies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*jaZFv5ZObS7pNOkx" /></figure><h3>Wearable Tech</h3><p>From counting steps to monitoring heart rhythms, various types of consumer wearable technologies provide information that can help people become more fit. Other wearable technologies can provide doctors with vital information about patient health, including heart rhythm, blood pressure, temperature and heart rate.</p><p><a href="https://www.pewresearch.org/fact-tank/2020/01/09/about-one-in-five-americans-use-a-smart-watch-or-fitness-tracker/#:~:text=About%20one%2Din%2Dfive%20Americans,smart%20watch%20or%20fitness%20tracker&amp;text=As%202020%20begins%20%E2%80%93%20and%20health,June%203%2D17%2C%202019.">According to Pew Research Center</a>, about 21% of Americans use wearable technologies, such as fitness trackers and smartwatches. As more people embrace wearable technologies, health informatics professionals can help improve the communication and accuracy of data shared between these devices and health information systems that doctors use.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*tTJXXuLEO3PuWPqg.jpg" /></figure><h3>Genome Sequencing</h3><p>Genomic data can help doctors create personalized treatment plans for their patients. Machine learning in health informatics enables genetic mutations to be analyzed much faster and helps in diagnosing conditions that can lead to disease. Genome sequencing, made possible through machine learning applications, can impact cancer diagnosis and treatment and mitigate the impact of infectious disease.</p><p>The first-ever human genome sequencing project cost more than $3 billion. It took more than 13 years to complete, <a href="https://www.weforum.org/agenda/2019/06/today-you-can-have-your-genome-sequenced-at-the-supermarket/">according to the World Economic Forum</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3UkbeMVZO5P01em0.jpg" /></figure><p>Today, individuals can pay less than $600 to have their genome sequenced and get results within a week. As genome sequencing becomes more affordable and machine learning becomes smarter, health informatics professionals can help advance genomic medicine to treat the world’s deadliest diseases.</p><h3>Nanotechnology</h3><p><a href="https://www.nano.gov/you/nanotechnology-benefits">According to the National Nanotechnology Initiative</a>, nanotechnology is defined as “the understanding and control of matter at the nanoscale, at dimensions between approximately 1 and 100 nanometers.”</p><p>Nanotechnology application in healthcare is referred to as nanomedicine. Nanotechnology can help execute tasks such as drug delivery in which molecules, cellular structures and DNA are at work.</p><p>For example, future nanotechnology medicine includes drug delivery methods that “enable site-specific targeting to avoid the accumulation of drug compounds in healthy cells or tissues,” <a href="https://www.engineering.com/DesignerEdge/DesignerEdgeArticles/ArticleID/18561/The-Future-of-Medicine-is-at-the-Nanoscale.aspx">according to Engineering.com</a>.</p><p>In other words, drugs can be delivered to targeted regions bypassing areas in the human system that aren’t affected by diseases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*eTO2YyMQFwRQFm7w.jpg" /></figure><h3>3D Printing</h3><p>According to a <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7224028/">study published in the <em>Journal of Polymers and the Environment</em></a>, 3D printing in biomedicine offers opportunities in the health sector. 3D printing processes allow for the efficient manufacture of drug formulations, implants, prostheses, biosensor devices, and even human tissues and organs. It creates opportunities for personalizing medical treatments, improves healthcare quality, reduces costs and minimizes production risks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*fLMjQGgDgqttCb_k" /></figure><h3>Transforming the Healthcare Industry</h3><p>According to <a href="https://www.itnonline.com/content/artificial-intelligence-healthcare-market-size-and-forecast-report"><em>Imaging Technology News</em></a>, the market for AI in healthcare will expand to more than $31.3 billion by 2025 — a growth of more than 40% since 2018. Individuals seeking to extend their healthcare informatics careers to include machine learning can begin by exploring educational opportunities. This can include enrolling in graduate degree programs in health informatics. With the advanced skills and knowledge they gain in graduate programs, they can help transform the healthcare industry.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/882/0*CTyRpnd6hipiCG0i.png" /></figure><h3>References</h3><ul><li><a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5807064/">National Center for Biotechnology Information, “Machine Learning and Electronic Health Records: A Paradigm Shift”</a>: This study discusses the application of machine learning methods to EHRs and its potential in improving diagnosis and treatment.</li><li><a href="https://www.forbes.com/sites/bernardmarr/2019/11/01/the-9-biggest-technology-trends-that-will-transform-medicine-and-healthcare-in-2020/#613245d472cd"><em>Forbes</em>, “The 9 Biggest Technology Trends That Will Transform Medicine and Healthcare In 2020”</a>: This article highlights some of the biggest and most transformative technology trends in healthcare, including machine learning.</li><li><a href="https://www.himss.org/resources/artificial-intelligence-health-ethical-considerations-research-and-practice">HIMSS, “Artificial Intelligence in Health: Ethical Considerations for Research and Practice”</a>: Though healthcare technology innovation continues to transform medicine, one must consider machine learning’s ethical implications.</li><li><a href="https://www.healthit.gov/topic/health-it-resources/health-it-curriculum-resources-educators">gov, Health IT Curriculum Resources for Educators</a>: More than 10,000 educators around the world have used this resource to access a health IT curriculum.</li><li><a href="https://www.wired.com/wiredinsider/2019/10/from-diagnosis-to-holistic-patient-care-machine-learning-is-transforming-healthcare/"><em>Wired</em>, “From Diagnosis to Holistic Patient Care, Machine Learning Is Transforming Healthcare”</a>: This article provides examples of how machine learning is transforming healthcare.</li><li><a href="https://journalofethics.ama-assn.org/article/ethical-dimensions-using-artificial-intelligence-health-care/2019-02"><em>AMA Journal of Ethics</em>, “Ethical Dimensions of Using Artificial Intelligence in Health Care”</a></li><li><a href="https://www.entrepreneur.com/article/341626"><em>Entrepreneur</em>, “5 Ways Machine Learning Is Redefining Healthcare”</a></li><li><a href="https://www.himss.org/resources/artificial-intelligence-health-ethical-considerations-research-and-practice">HIMSS, “Artificial Intelligence in Health: Ethical Considerations for Research and Practice”</a></li><li><a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6219763/">National Center for Biotechnology Information, “Machine Learning in Medicine: Addressing Ethical Challenges”</a></li><li><a href="https://www.roboticsbusinessreview.com/health-medical/6-ways-ai-and-robotics-are-improving-healthcare/">Robotics Business Review, “6 Ways Robotics and AI Are Improving Health Care”</a></li></ul><blockquote><strong><em>Final Note :</em></strong></blockquote><p><strong>Thanks for reading! If you liked this article, please hit the clap</strong>👏<strong> button as many times as you can. It would mean a lot and encourage me to keep sharing my knowledge.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7f16a9b70e94" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Animated Plots using R Language]]></title>
            <link>https://arnab-dey.medium.com/animated-plots-using-r-language-7d230fa6027e?source=rss-44286b7db55------2</link>
            <guid isPermaLink="false">https://medium.com/p/7d230fa6027e</guid>
            <category><![CDATA[visualization]]></category>
            <category><![CDATA[tutorial]]></category>
            <category><![CDATA[animated]]></category>
            <category><![CDATA[graph]]></category>
            <category><![CDATA[plot]]></category>
            <dc:creator><![CDATA[Arnab Dey]]></dc:creator>
            <pubDate>Mon, 14 Jun 2021 03:41:15 GMT</pubDate>
            <atom:updated>2021-06-14T03:57:47.011Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k9SzJVXFDVoEmqdqZNb1UA.jpeg" /></figure><p>This article describes how to create beautiful animations in R using the <strong>gganimate</strong> R package.</p><p>When telling a data-driven story, few things grab attention more than motion. Our eyes are naturally drawn to bright colors and movement. A good visualization captures the interest of the audience and makes an impression. Fortunately, making such displays has become quite simple with a few easy to use R packages.</p><p>gganimate is an extension of the <strong>ggplot2</strong> package for creating animated ggplots. It provides a range of new functionality that can be added to the plot object in order to customize how it should change with time.</p><p><strong>Key features of gganimate:</strong></p><ul><li><strong>transitions</strong>: you want your data to change</li><li><strong>views</strong>: you want your viewpoint to change</li><li><strong>shadows</strong>: you want the animation to have memory</li></ul><h3>Prerequisites</h3><p>gganimate stable version is available on CRAN and can be installed with install.packages(&#39;gganimate&#39;). The latest development version can be installed as follow: devtools::install_github(&#39;thomasp85/gganimate&#39;).</p><p>Note that, in this tutorial, we used the latest developmental version.</p><p>Load required packages and set the default ggplot2 theme to theme_bw():</p><pre><strong>library</strong>(ggplot2)<br><strong>library</strong>(gganimate)<br>theme_set(theme_bw())</pre><h3>Demo dataset</h3><pre><strong>library</strong>(gapminder)<br>head(gapminder)</pre><pre>## # A tibble: 6 x 6<br>##   country     continent  year lifeExp      pop gdpPercap<br>##   &lt;fct&gt;       &lt;fct&gt;     &lt;int&gt;   &lt;dbl&gt;    &lt;int&gt;     &lt;dbl&gt;<br>## 1 Afghanistan Asia       1952    28.8  8425333      779.<br>## 2 Afghanistan Asia       1957    30.3  9240934      821.<br>## 3 Afghanistan Asia       1962    32.0 10267083      853.<br>## 4 Afghanistan Asia       1967    34.0 11537966      836.<br>## 5 Afghanistan Asia       1972    36.1 13079460      740.<br>## 6 Afghanistan Asia       1977    38.4 14880372      786.</pre><h3>Static plot</h3><pre>p &lt;- ggplot(<br>  gapminder, <br>  aes(x = gdpPercap, y=lifeExp, size = pop, colour = country)<br>  ) +<br>  geom_point(show.legend = FALSE, alpha = 0.7) +<br>  scale_color_viridis_d() +<br>  scale_size(range = c(2, 12)) +<br>  scale_x_log10() +<br>  labs(x = &quot;GDP per capita&quot;, y = &quot;Life expectancy&quot;)<br>p</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*P7fbIKa_auCtE8hC.png" /></figure><h4>Transition through distinct states in time</h4><h3>Basics</h3><p>Key R function: transition_time(). The transition length between the states will be set to correspond to the actual time difference between them.</p><p>Label variables: frame_time. Gives the time that the current frame corresponds to.</p><pre>p + transition_time(year) +<br>  labs(title = &quot;Year: {frame_time}&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*LDDGJufsUBuzeLde.gif" /></figure><p>Create facets by continent:</p><pre>p + facet_wrap(~continent) +<br>  transition_time(year) +<br>  labs(title = &quot;Year: {frame_time}&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*SthF59EIbOQyGQM1.gif" /></figure><h3>Let the view follow the data in each frame</h3><pre>p + transition_time(year) +<br>  labs(title = &quot;Year: {frame_time}&quot;) +<br>  view_follow(fixed_y = TRUE)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*_BqjXAtJWHrisuhw.gif" /></figure><h3>Show preceding frames with gradual falloff</h3><p>This shadow is meant to draw a small wake after data by showing the latest frames up to the current. You can choose to gradually diminish the size and/or opacity of the shadow. The length of the wake is not given in absolute frames as that would make the animation susceptible to changes in the framerate. Instead it is given as a proportion of the total length of the animation.</p><pre>p + transition_time(year) +<br>  labs(title = &quot;Year: {frame_time}&quot;) +<br>  shadow_wake(wake_length = 0.1, alpha = FALSE)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*bVF6UCKRyfCk19pr.gif" /></figure><h3>Show the original data as background marks</h3><p>This shadow lets you show the raw data behind the current frame. Both past and/or future raw data can be shown and styled as you want.</p><pre>p + transition_time(year) +<br>  labs(title = &quot;Year: {frame_time}&quot;) +<br>  shadow_mark(alpha = 0.3, size = 0.5)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*5rpuS8MMnyHkMjfo.gif" /></figure><h3>Reveal data along a given dimension</h3><p>This transition allows you to let data gradually appear, based on a given time dimension.</p><h3>Static plot</h3><pre>p &lt;- ggplot(<br>  airquality,<br>  aes(Day, Temp, group = Month, color = factor(Month))<br>  ) +<br>  geom_line() +<br>  scale_color_viridis_d() +<br>  labs(x = &quot;Day of Month&quot;, y = &quot;Temperature&quot;) +<br>  theme(legend.position = &quot;top&quot;)<br>p</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*bxO4EXejlAQ82N5q.png" /></figure><h3>Let data gradually appear</h3><ul><li>Reveal by day (x-axis)</li></ul><pre>p + transition_reveal(Day)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*fFoGqAzRkUUlYoD2.gif" /></figure><ul><li>Show points:</li></ul><pre>p + <br>  geom_point() +<br>  transition_reveal(Day)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*C6dyacH84opbIuFl.gif" /></figure><ul><li>Points can be kept by giving them a unique group:</li></ul><pre>p + <br>  geom_point(aes(group = seq_along(Day))) +<br>  transition_reveal(Day)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*sCmd41jJb5JakxZD.gif" /></figure><h3>Transition between several distinct stages of the data</h3><p>Data preparation:</p><pre><strong>library</strong>(dplyr)<br>mean.temp &lt;- airquality %&gt;%<br>  group_by(Month) %&gt;%<br>  summarise(Temp = mean(Temp))<br>mean.temp</pre><pre>## # A tibble: 5 x 2<br>##   Month  Temp<br>##   &lt;int&gt; &lt;dbl&gt;<br>## 1     5  65.5<br>## 2     6  79.1<br>## 3     7  83.9<br>## 4     8  84.0<br>## 5     9  76.9</pre><p><strong>Create a bar plot of mean temperature:</strong></p><pre>p &lt;- ggplot(mean.temp, aes(Month, Temp, fill = Temp)) +<br>  geom_col() +<br>  scale_fill_distiller(palette = &quot;Reds&quot;, direction = 1) +<br>  theme_minimal() +<br>  theme(<br>    panel.grid = element_blank(),<br>    panel.grid.major.y = element_line(color = &quot;white&quot;),<br>    panel.ontop = TRUE<br>  )<br>p</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*9Sh8rgUx6GTzKkpc.png" /></figure><ul><li>transition_states():</li></ul><pre>p + transition_states(Month, wrap = FALSE) +<br>  shadow_mark()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*4FlTrn0PMTt8Hh9i.gif" /></figure><ul><li>enter_grow() + enter_fade()</li></ul><pre>p + transition_states(Month, wrap = FALSE) +<br>  shadow_mark() +<br>  enter_grow() +<br>  enter_fade()</pre><h3>Save animation</h3><p>If you need to save the animation for later use you can use the anim_save() function.</p><p>It works much like ggsave() from ggplot2 and automatically grabs the last rendered animation if you do not specify one directly.</p><p>Example of usage:</p><p><strong>Read more</strong></p><p><a href="https://gganimate.com/index.html">gganimate official documentation website</a></p><blockquote><strong><em>Final Note :</em></strong></blockquote><p><strong>Thanks for reading! If you liked this article, please hit the clap</strong>👏<strong> button as many times as you can. It would mean a lot and encourage me to keep sharing my knowledge.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7d230fa6027e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Intro to Natural Language Processing (NLP) : A Beginner's Guide]]></title>
            <link>https://arnab-dey.medium.com/introduction-to-natural-language-processing-nlp-a-guide-1570a6ecad26?source=rss-44286b7db55------2</link>
            <guid isPermaLink="false">https://medium.com/p/1570a6ecad26</guid>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[introduction]]></category>
            <dc:creator><![CDATA[Arnab Dey]]></dc:creator>
            <pubDate>Sat, 12 Jun 2021 06:15:15 GMT</pubDate>
            <atom:updated>2021-07-08T07:36:05.531Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nMhQdot34bjn3uhXzghwoA.png" /></figure><p>Natural Language Processing or NLP, is broadly defined as the automatic manipulation of natural language, like speech and text, by software. Natural Language Processing (NLP) refers to AI method of communicating with an intelligent systems using a natural language such as English. The study of natural language processing has been around for more than 50 years and grew out of the field of linguistics with the rise of computers.</p><p>Processing of Natural Language is required when you want an intelligent system like robot to perform as per your instructions, when you want to hear decision from a dialogue based clinical expert system, etc.</p><p>The field of NLP involves making computers to perform useful tasks with the natural languages humans use. The input and output of an NLP system can be:</p><ul><li>Speech</li><li>Written Text</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*JbQ58utmVAnAQ1G7ueLMAA.png" /></figure><h3>Components of NLP</h3><p>There are two components of NLP as given −</p><h3>Natural Language Understanding (NLU)</h3><p>Understanding involves the following tasks −</p><ul><li>Mapping the given input in natural language into useful representations.</li><li>Analyzing different aspects of the language.</li></ul><h3>Natural Language Generation (NLG)</h3><p>It is the process of producing meaningful phrases and sentences in the form of natural language from some internal representation.</p><p>It involves −</p><ul><li><strong>Text planning</strong> − It includes retrieving the relevant content from knowledge base.</li><li><strong>Sentence planning</strong> − It includes choosing required words, forming meaningful phrases, setting tone of the sentence.</li><li><strong>Text Realization</strong> − It is mapping sentence plan into sentence structure.</li></ul><p>The NLU is harder than NLG.</p><h3>Difficulties in NLU</h3><p>NL has an extremely rich form and structure.</p><p>It is very ambiguous. There can be different levels of ambiguity −</p><ul><li><strong>Lexical ambiguity</strong> − It is at very primitive level such as word-level.</li><li>For example, treating the word “board” as noun or verb?</li><li><strong>Syntax Level ambiguity</strong> − A sentence can be parsed in different ways.</li><li>For example, “He lifted the beetle with red cap.” − Did he use cap to lift the beetle or he lifted a beetle that had red cap?</li><li><strong>Referential ambiguity</strong> − Referring to something using pronouns. For example, Rima went to Gauri. She said, “I am tired.” − Exactly who is tired?</li><li>One input can mean different meanings.</li><li>Many inputs can mean the same thing.</li></ul><h3>NLP Terminology</h3><ul><li><strong>Phonology</strong> − It is study of organizing sound systematically.</li><li><strong>Morphology</strong> − It is a study of construction of words from primitive meaningful units.</li><li><strong>Morpheme</strong> − It is primitive unit of meaning in a language.</li><li><strong>Syntax</strong> − It refers to arranging words to make a sentence. It also involves determining the structural role of words in the sentence and in phrases.</li><li><strong>Semantics</strong> − It is concerned with the meaning of words and how to combine words into meaningful phrases and sentences.</li><li><strong>Pragmatics</strong> − It deals with using and understanding sentences in different situations and how the interpretation of the sentence is affected.</li><li><strong>Discourse</strong> − It deals with how the immediately preceding sentence can affect the interpretation of the next sentence.</li><li><strong>World Knowledge</strong> − It includes the general knowledge about the world.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*P31_zPfgYppem4RKUx-7OQ.png" /></figure><h3>Steps in NLP</h3><p>There are general five steps −</p><ul><li><strong>Lexical Analysis</strong> − It involves identifying and analyzing the structure of words. Lexicon of a language means the collection of words and phrases in a language. Lexical analysis is dividing the whole chunk of txt into paragraphs, sentences, and words.</li><li><strong>Syntactic Analysis (Parsing)</strong> − It involves analysis of words in the sentence for grammar and arranging words in a manner that shows the relationship among the words. The sentence such as “The school goes to boy” is rejected by English syntactic analyzer.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/367/0*Sb9W7pEnekDa-fZC.jpg" /></figure><ul><li><strong>Semantic Analysis</strong> − It draws the exact meaning or the dictionary meaning from the text. The text is checked for meaningfulness. It is done by mapping syntactic structures and objects in the task domain. The semantic analyzer disregards sentence such as “hot ice-cream”.</li><li><strong>Discourse Integration</strong> − The meaning of any sentence depends upon the meaning of the sentence just before it. In addition, it also brings about the meaning of immediately succeeding sentence.</li><li><strong>Pragmatic Analysis</strong> − During this, what was said is re-interpreted on what it actually meant. It involves deriving those aspects of language which require real world knowledge.</li></ul><h3>Implementation Aspects of Syntactic Analysis</h3><p>There are a number of algorithms researchers have developed for syntactic analysis, but we consider only the following simple methods −</p><ul><li>Context-Free Grammar</li><li>Top-Down Parser</li></ul><p>Let us see them in detail −</p><h3>Context-Free Grammar</h3><p>It is the grammar that consists rules with a single symbol on the left-hand side of the rewrite rules. Let us create grammar to parse a sentence −</p><p>“The bird pecks the grains”</p><p><strong>Articles (DET)</strong> − a | an | the</p><p><strong>Nouns</strong> − bird | birds | grain | grains</p><p><strong>Noun Phrase (NP)</strong> − Article + Noun | Article + Adjective + Noun</p><p>= DET N | DET ADJ N</p><p><strong>Verbs</strong> − pecks | pecking | pecked</p><p><strong>Verb Phrase (VP)</strong> − NP V | V NP</p><p><strong>Adjectives (ADJ)</strong> − beautiful | small | chirping</p><p>The parse tree breaks down the sentence into structured parts so that the computer can easily understand and process it. In order for the parsing algorithm to construct this parse tree, a set of rewrite rules, which describe what tree structures are legal, need to be constructed.</p><p>These rules say that a certain symbol may be expanded in the tree by a sequence of other symbols. According to first order logic rule, if there are two strings Noun Phrase (NP) and Verb Phrase (VP), then the string combined by NP followed by VP is a sentence. The rewrite rules for the sentence are as follows −</p><p><strong>S → NP VP</strong></p><p><strong>NP → DET N | DET ADJ N</strong></p><p><strong>VP → V NP</strong></p><p><strong>Lexocon −</strong></p><p>DET → a | the</p><p>ADJ → beautiful | perching</p><p>N → bird | birds | grain | grains</p><p>V → peck | pecks | pecking</p><p>The parse tree can be created as shown −</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*s4zdV31cEw-jxq_y.jpg" /></figure><p>Now consider the above rewrite rules. Since V can be replaced by both, “peck” or “pecks”, sentences such as “The bird peck the grains” can be wrongly permitted. i. e. the subject-verb agreement error is approved as correct.</p><p><strong>Merit</strong> −</p><p>The simplest style of grammar, therefore widely used one.</p><p><strong>Demerits −</strong></p><ul><li>They are not highly precise. For example, “The grains peck the bird”, is a syntactically correct according to parser, but even if it makes no sense, parser takes it as a correct sentence.</li><li>To bring out high precision, multiple sets of grammar need to be prepared. It may require a completely different sets of rules for parsing singular and plural variations, passive sentences, etc., which can lead to creation of huge set of rules that are unmanageable.</li></ul><h3>Top-Down Parser</h3><p>Here, the parser starts with the S symbol and attempts to rewrite it into a sequence of <em>terminal symbols</em> that matches the classes of the words in the input sentence until it consists entirely of terminal symbols.</p><p>These are then checked with the input sentence to see if it matched. If not, the process is started over again with a different set of rules. This is repeated until a specific rule is found which describes the structure of the sentence.</p><p><strong>Merit</strong> −</p><p>It is simple to implement.</p><p><strong>Demerits −</strong></p><ul><li>It is inefficient, as the search process has to be repeated if an error occurs.</li><li>Slow speed of working.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XjeVJjsVkZJghr7eEhQIEQ.jpeg" /></figure><h3>Challenge of Natural Language</h3><p>Working with natural language data is not solved. It has been studied for half a century, and it is really hard.</p><blockquote><em>It is hard from the standpoint of the child, who must spend many years acquiring a language … it is hard for the adult language learner, it is hard for the scientist who attempts to model the relevant phenomena, and it is hard for the engineer who attempts to build systems that deal with natural language input or output. These tasks are so hard that Turing could rightly make fluent conversation in natural language the centerpiece of his test for intelligence.</em></blockquote><p>— Page 248, <a href="https://amzn.to/2tO1cOO">Mathematical Linguistics</a>, 2010.</p><p>Natural language is primarily hard because it is messy. There are few rules.</p><p>And yet we can easily understand each other most of the time.</p><blockquote><em>Human language is highly ambiguous … It is also ever changing and evolving. People are great at producing language and understanding language, and are capable of expressing, perceiving, and interpreting very elaborate and nuanced meanings. At the same time, while we humans are great users of language, we are also very poor at formally understanding and describing the rules that govern language.</em></blockquote><p>— Page 1, <a href="https://amzn.to/2u0JtPl">Neural Network Methods in Natural Language Processing</a>, 2017.</p><h3>From Linguistics to Natural Language Processing</h3><h3>Linguistics</h3><p><a href="https://en.wikipedia.org/wiki/Linguistics">Linguistics</a> is the scientific study of language, including its grammar, semantics, and phonetics.</p><p>Classical linguistics involved devising and evaluating rules of language. Great progress was made on formal methods for syntax and semantics, but for the most part, the interesting problems in natural language understanding resist clean mathematical formalisms.</p><p>Broadly, a linguist is anyone who studies language, but perhaps more colloquially, a self-defining linguist may be more focused on being out in the field.</p><p>Mathematics is the tool of science. Mathematicians working on natural language may refer to their study as mathematical linguistics, focusing exclusively on the use of discrete mathematical formalisms and theory for natural language (e.g. formal languages and automata theory).</p><h3>Computational Linguistics</h3><p><a href="https://en.wikipedia.org/wiki/Computational_linguistics">Computational linguistics</a> is the modern study of linguistics using the tools of computer science. Yesterday’s linguistics may be today’s computational linguist as the use of computational tools and thinking has overtaken most fields of study.</p><blockquote><em>Computational linguistics is the study of computer systems for understanding and generating natural language. … One natural function for computational linguistics would be the testing of grammars proposed by theoretical linguists.</em></blockquote><p>— Pages 4–5, <a href="https://amzn.to/2h6U4qY">Computational Linguistics: An Introduction</a>, 1986.</p><p>Large data and fast computers mean that new and different things can be discovered from large datasets of text by writing and running software.</p><p>In the 1990s, statistical methods and statistical machine learning began to and eventually replaced the classical top-down rule-based approaches to language, primarily because of their better results, speed, and robustness. The statistical approach to studying natural language now dominates the field; it may define the field.</p><blockquote><em>Data-Drive methods for natural language processing have now become so popular that they must be considered mainstream approaches to computational linguistics. … A strong contributing factor to this development is undoubtedly the increase amount of available electronically stored data to which these methods can be applied; another factor might be a certain disenchantment with approaches relying exclusively on hand-crafted rules, due to their observed brittleness.</em></blockquote><p>— Page 358, <a href="https://amzn.to/2uHeERE">The Oxford Handbook of Computational Linguistics</a>, 2005.</p><p>The statistical approach to natural language is not limited to statistics per-se, but also to advanced inference methods like those used in applied machine learning.</p><blockquote><em>… understanding natural language require large amounts of knowledge about morphology, syntax, semantics and pragmatics as well as general knowledge about the world. Acquiring and encoding all of this knowledge is one of the fundamental impediments to developing effective and robust language systems. Like the statistical methods … machine learning methods off the promise of automatic the acquisition of this knowledge from annotated or unannotated language corpora.</em></blockquote><p>— Page 377, <a href="https://amzn.to/2uHeERE">The Oxford Handbook of Computational Linguistics</a>, 2005.</p><h3>Books</h3><ul><li><a href="https://amzn.to/2tO1cOO">Mathematical Linguistics</a>, 2010.</li><li><a href="https://amzn.to/2u0JtPl">Neural Network Methods in Natural Language Processing</a>, 2017.</li><li><a href="https://amzn.to/2h6U4qY">Computational Linguistics: An Introduction</a>, 1986.</li><li><a href="https://amzn.to/2uHeERE">The Oxford Handbook of Computational Linguistics</a>, 2005.</li><li><a href="https://amzn.to/2uzwxDE">Foundations of Statistical Natural Language Processing</a>, 1999.</li><li><a href="https://amzn.to/2uZMF27">Natural Language Processing with Python</a>, 2009.</li></ul><blockquote><strong><em>Final Note :</em></strong></blockquote><p><strong>Thanks for reading! If you liked this article, please hit the clap</strong>👏<strong> button as many times as you can. It would mean a lot and encourage me to keep sharing my knowledge.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1570a6ecad26" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exercises can keep us Healthy and fit During COVID-19 Pandemic]]></title>
            <link>https://arnab-dey.medium.com/how-exercise-keeps-us-healthy-and-fit-during-covid-19-pandemic-21d4e48ac00?source=rss-44286b7db55------2</link>
            <guid isPermaLink="false">https://medium.com/p/21d4e48ac00</guid>
            <category><![CDATA[healthy-lifestyle]]></category>
            <category><![CDATA[healthy]]></category>
            <category><![CDATA[fitness]]></category>
            <category><![CDATA[covid19]]></category>
            <category><![CDATA[exercise]]></category>
            <dc:creator><![CDATA[Arnab Dey]]></dc:creator>
            <pubDate>Sat, 12 Jun 2021 05:32:59 GMT</pubDate>
            <atom:updated>2021-06-14T19:23:23.898Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*ZVGBq5Cr1RJaINM2Qj29sA.jpeg" /></figure><p><strong>Exercise</strong> is any bodily activity that enhances or maintains physical fitness and overall health and wellness. It is performed for various reasons, to aid growth and improve strength, preventing aging, developing muscles and the cardiovascular system, honing athletic skills, weight loss or maintenance, improving health. The <strong>COVID-19 pandemic</strong> has brought this fast-moving world to a standstill. Most of us are working and doing various tasks online. Doing regular exercise can help you to gain more immunity power, reduce your stress and anxiety in this pandemic. It will also boost your health and can keep you away from various diseases.</p><p>Physical activity is an important part of your overall physical and mental health. Regular physical activity over time produces long-term health benefits and reduces the risk of many diseases. It also keeps you in shape so you can enjoy leisure activities and safely perform work and home chores. Overall, it helps maintain an independent, productive, and social life.</p><p>According the recent Physical Activity Guidelines for Americans, all Americans should be active even older adults and those who are disabled or managing a chronic illness. All adults gain substantial health benefits from a combination of activities that include aerobic fitness, flexibility, and muscle-strengthening exercises.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/680/1*JE49WaKEJeWYJGajIuBB2g.jpeg" /></figure><h3><strong>Would you like to:</strong></h3><p>· Decrease your risk of disease?</p><p>· Feel better physically and mentally?</p><p>· Look better?</p><p>· Help avoid injuries?</p><p>· Keep doing activities you enjoy throughout your life?</p><p>Regular physical activity will help you do these things. Physical activity is essential to prevent and reduce risks of many diseases and improve physical and mental health. It can even help you live longer — research from the <em>American Journal of Preventative Medicine </em>indicates that regular exercise can add up to five years to your life.</p><p>Physical activity also keeps you in shape so you can enjoy leisure activities and safely perform work and home chores. It offers great mental and social benefits as well. The Lancet released a series of studies that attribute positive outcomes to physical activity, including “a<a href="http://www.takingcharge.csh.umn.edu/wellbeing/life-purpose"> sense of purpose</a> and value, a better quality of life, improved<a href="http://www.takingcharge.csh.umn.edu/wellbeing/health/sleep"> sleep</a>, and reduced <a href="http://www.takingcharge.csh.umn.edu/wellbeing/health/stress-mastery">stress</a>, as well as <a href="http://www.takingcharge.csh.umn.edu/wellbeing/relationships">stronger relationships</a> and <a href="http://www.takingcharge.csh.umn.edu/wellbeing/community">social connectedness</a>.”</p><h3>What are the health benefits of exercise?</h3><p>Regular exercise and physical activity may</p><ul><li><strong>Help you control your weight.</strong> Along with diet, exercise plays an important role in <a href="https://medlineplus.gov/weightcontrol.html">controlling your weight</a> and preventing <a href="https://medlineplus.gov/obesity.html">obesity</a>. To maintain your weight, the calories you eat and drink must equal the energy you burn. To lose weight, you must use more calories than you eat and drink.</li><li><strong>Reduce your risk of heart diseases.</strong> Exercise strengthens your heart and improves your circulation. The increased blood flow raises the oxygen levels in your body. This helps lower your risk of <a href="https://medlineplus.gov/heartdiseases.html">heart diseases</a> such as <a href="https://medlineplus.gov/cholesterol.html">high cholesterol</a>, <a href="https://medlineplus.gov/coronaryarterydisease.html">coronary artery disease</a>, and <a href="https://medlineplus.gov/heartattack.html">heart attack</a>. Regular exercise can also lower your blood pressure and triglyceride levels.</li><li><strong>Help your body manage blood sugar and insulin levels. </strong>Exercise can lower your <a href="https://medlineplus.gov/bloodsugar.html">blood sugar</a> level and help your insulin work better. This can cut down your risk for <a href="https://medlineplus.gov/metabolicsyndrome.html">metabolic syndrome</a> and <a href="https://medlineplus.gov/diabetestype2.html">type 2 diabetes</a>. And if you already have one of those diseases, exercise can help you to manage it.</li><li><strong>Help you quit smoking. </strong>Exercise may make it easier to <a href="https://medlineplus.gov/quittingsmoking.html">quit smoking</a> by reducing your cravings and withdrawal symptoms. It can also help limit the weight you might gain when you stop smoking.</li><li><strong>Improve your mental health and mood. </strong>During exercise, your body releases chemicals that can improve your mood and make you feel more relaxed. This can help you deal with <a href="https://medlineplus.gov/stress.html">stress</a> and reduce your risk of <a href="https://medlineplus.gov/depression.html">depression</a>.</li><li><strong>Help keep your thinking, learning, and judgment skills sharp as you age. </strong>Exercise stimulates your body to release proteins and other chemicals that improve the structure and function of your brain.</li><li><strong>Strengthen your bones and muscles. </strong>Regular exercise can help kids and teens build strong bones. Later in life, it can also slow the loss of bone density that comes with age. Doing muscle-strengthening activities can help you increase or maintain your muscle mass and strength.</li><li><strong>Reduce your risk of some cancers, </strong>including <a href="https://medlineplus.gov/colorectalcancer.html">colon</a>, <a href="https://medlineplus.gov/breastcancer.html">breast </a>, <a href="https://medlineplus.gov/uterinecancer.html">uterine</a>, and <a href="https://medlineplus.gov/lungcancer.html">lung</a> cancer.</li><li><strong>Reduce your risk of falls. </strong>For older adults, research shows that doing balance and muscle-strengthening activities in addition to moderate-intensity aerobic activity can help reduce your risk of falling.</li><li><strong>Improve your sleep. </strong>Exercise can help you to fall asleep faster and stay asleep longer.</li><li><strong>Improve your sexual health. </strong>Regular exercise may lower the risk of <a href="https://medlineplus.gov/erectiledysfunction.html">erectile dysfunction</a> (ED) in men. For those who already have ED, exercise may help improve their sexual function. In women, exercise may increase sexual arousal.</li><li><strong>Increase your chances of living longer. </strong>Studies show that physical activity can reduce your risk of dying early from the leading causes of death, like heart disease and some cancers.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*UMEuJMu47RZBusQGT33leA.jpeg" /></figure><h3>How can you make exercise a part of your regular routine?</h3><ul><li><strong>Make everyday activities more active.</strong> Even small changes can help. You can take the stairs instead of the elevator. Walk down the hall to a coworker’s office instead of sending an email. Wash the car yourself. Park further away from your destination.</li><li><strong>Be active with friends and family.</strong> Having a workout partner may make you more likely to enjoy exercise. You can also plan social activities that involve exercise. You might also consider joining an exercise group or class, such as a dance class, hiking club, or volleyball team.</li><li><strong>Keep track of your progress.</strong> Keeping a log of your activity or using a fitness tracker may help you set goals and stay motivated.</li><li><strong>Make exercise more fun.</strong> Try listening to music or watching TV while you exercise. Also, mix things up a little bit — if you stick with just one type of exercise, you might get bored. Try doing a combination of activities.</li><li><strong>Find activities that you can do even when the weather is bad.</strong> You can walk in a mall, climb stairs, or work out in a gym even if the weather stops you from exercising outside.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*DxZ1DmpZcUQL_pCI.jpeg" /></figure><p><strong>If you’re new to exercise and not sure where to start, here are some ideas to get you moving:</strong></p><ul><li><strong>Take your workout online:</strong> On YouTube there are endless free exercise videos to try, regardless of your fitness level or the size of your living room. From yoga and strength workouts to Pilates, high intensity interval training (HIIT) and more. Our This Girl Can — Victoria <a href="https://thisgirlcan.com.au/active-at-home/">Get Active @ Home’ </a>page has lots of videos you can try, especially if you are sick of seeing ‘FITSPO’ workouts and you want something that focuses on feeling good.</li><li><strong>Join a local sports club: </strong>Grassroots sports clubs are back up and running now that restrictions on outdoor gatherings have lifted, so now is the perfect time to get back into some organised sport. It doesn’t have to be competitive, there are plenty of social sports programs available for people who want to get active without the pressure of intense competition.</li><li><strong>Go freestyle:</strong> If a structured routine is not your style, get creative and build your own workout — walking or running on the spot for 30 second intervals, do some star jumps, planks, sit ups, push ups, or even burpees. Anything to get your heart rate up a little either at your home or at a local outdoor area such as a park or oval.</li><li><strong>Go solo outside:</strong> Walking, cycling and running are great solo activities and now restrictions have eased, this is the ideal time to do them. We suggest going early in the morning or late in the day, to avoid the busy times in popular areas. It’s also a great time to try your hand at <a href="https://www.vichealth.vic.gov.au/be-healthy/how-to-bushwalk-in-victoria-during-covid-normal">bushwalking</a>, with lots of great tracks and trails to explore in Victoria. Remember you need to carry a mask with you so you can wear it if you can’t physically distance from other people. Wash your hands thoroughly with soap and warm water for at least 20 seconds when you get home.</li></ul><p><strong>WHAT DO EXPERTS RECOMMEND FOR FITNESS?</strong></p><p>The 2008 <a href="http://www.health.gov/PAGuidelines/guidelines/">Physical Activity Guidelines for Americans</a> are designed so people can easily fit physical activity into their daily plan and incorporate activities they enjoy.</p><p>These guidelines recommend that<strong> </strong>each week<strong> </strong>you do<strong> </strong>one of the following:</p><p>· Two and a half hours of moderate intensity<strong> aerobic physical activity</strong> (Examples are walking briskly, water aerobics, ballroom dancing, and general gardening)</p><p><em>OR</em></p><p>· One hour and 15 minutes of vigorous physical activity (Includes racewalking, jogging or running, swimming laps, jumping rope, and hiking uphill or with a heavy backpack)</p><p>Make sure that you do <strong>at least 10 minutes of aerobic activity at a time</strong>. In fact, people who engage in short bursts of physical activity for 10 minutes at a time have been shown to be just as healthy as people who do a regular 30-minute workout every day. Check out Fitness to learn ways to incorporate bursts of physical activity naturally into your daily life.</p><p>In addition, include muscle strengthening activities, such as weight training, push-ups, sit-ups, carrying heavy loads, or heavy gardening, at least two days a week. Exercise specialists, such as the American College of Sports Medicine, also recommend incorporating stretching and other exercises to increase flexibility and avoid injury, as well as exercises to improve balance and reduce risk of falls.</p><p>A word of caution</p><p>While exercise is undoubtedly beneficial, it is not true that the more exercise you do or the harder you work the body, the better the results. Too much or too arduous physical activity can lead to injury. It is essential to maintain a balance between working out the muscles without overdoing it. Know that your body gets stronger during rest and recovery.</p><p><strong>INCLUDE AEROBIC ACTIVITIES</strong></p><p>Aerobic activity improves your cardiovascular health and helps protect against <a href="http://takingcharge.csh.umn.edu/conditions/heart-disease">heart disease</a>. It also improves your physical energy and produces endorphins that improve your mood.</p><p>What you do and how often and hard you exercise are determined by your goals, present fitness level and health, interest, and convenience. It is important to choose an activity you enjoy and that fits your lifestyle and tailor it to your fitness level. This will help you make exercise a habit. It’s also a good idea to choose more than one type of exercise to give your body a more complete workout and to avoid boredom.</p><p>There are two great ways to increase aerobic benefits:</p><p>· Add more vigorous activities</p><p>· Add more time for aerobic activities</p><h3>Add more vigorous activity</h3><p>Instead of doing only moderate-level activities, consider replacing some with vigorous aerobic activities that will make your heart beat even faster. Adding vigorous activities provides benefits in less activity time. In general, 15 minutes of vigorous activity provides the same benefits as 30 minutes of moderate activity.</p><p>Have you been walking for 30 minutes five days a week? On two days, try jogging instead of walking for 15 minutes each time. Keep on walking for 30 minutes on the other three days.</p><p>Would you like to have stronger muscles? If you have been doing strengthening activities two days a week, try adding an extra day.</p><h3>Helpful tips to get started</h3><p>One way to set your goals and monitor your level of physical activity is to know your <a href="http://www.cancer.org/docroot/PED/content/PED_6_1x_Target_Heart_Rate_Calculator.asp">target heart rate</a>. Make your cardiovascular workouts hard enough to break a sweat and get your heart pumping faster.<a href="http://www.takingcharge.csh.umn.edu/activities/how-hard-should-i-work-out"> Learn how hard you should work out.</a></p><p>Walking is one excellent cardio-respiratory exercise that almost everyone can do. Walking 10,000 steps a day can improve health and fitness. (It takes a little more than 2,000 steps to walk one mile.) Many people discover when they begin wearing a pedometer that they only average between 900 and 3,000 steps a day. Learn how to start your own <a href="http://www.takingcharge.csh.umn.edu/activities/starting-walking-program">walking program</a>.</p><p>There are many choices for cardiovascular exercise. Learn some of the <a href="http://www.takingcharge.csh.umn.edu/activities/selecting-activities-are-right-you">pros and cons</a> of each to help you make choices.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*GmvGacjVVxm2wNW9.jpeg" /></figure><p><strong>FIND OUT MORE ABOUT FITNESS FROM REFERENCE LINKS</strong></p><h3>References</h3><p>Aerobics and Fitness Association of America <a href="http://www.afaa.com/">http://www.afaa.com</a></p><p>Activity Bursts Everywhere: <a href="http://abeforfitness.com/">http://abeforfitness.com</a></p><p>America on the Move <a href="http://aom2.americaonthemove.org/">http://aom2.americaonthemove.org</a></p><p>American Lung Association Running Club <a href="http://www.alarc.com/">http://www.alarc.com/</a></p><p>Aquatic Exercise Association <a href="http://www.aeawave.com/">http://www.aeawave.com/</a></p><p>Center for Disease Control <a href="http://www.cdc.gov/physicalactivity">http://www.cdc.gov/physicalactivity</a></p><p>National Institute on Aging <a href="http://www.nia.nih.gov/HealthInformation/Publications/exercise.htm">http://www.nia.nih.gov/HealthInformation/Publications/exercise.htm</a></p><p>National Institute of Health <a href="http://health.nih.gov/topic/ExercisePhysicalFitness">http://health.nih.gov/topic/ExercisePhysicalFitness</a></p><p>Patience Tai Chi Association <a href="http://www.patiencetaichi.com/">http://www.patiencetaichi.com</a></p><blockquote><strong><em>Final Note :</em></strong></blockquote><p><strong>Thanks for reading! If you liked this article, please hit the clap</strong>👏<strong> button as many times as you can. It would mean a lot and encourage me to keep sharing my knowledge.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=21d4e48ac00" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data visualization using Python]]></title>
            <link>https://arnab-dey.medium.com/data-visualization-using-python-15cbcde7e8f7?source=rss-44286b7db55------2</link>
            <guid isPermaLink="false">https://medium.com/p/15cbcde7e8f7</guid>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[python-visualization]]></category>
            <category><![CDATA[data-visualisation]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[graph]]></category>
            <dc:creator><![CDATA[Arnab Dey]]></dc:creator>
            <pubDate>Sat, 12 Jun 2021 04:07:18 GMT</pubDate>
            <atom:updated>2021-06-12T04:10:32.913Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sFaV7PF-Wq25gXo1zo-9PQ.jpeg" /></figure><p><strong>Data Visualization</strong> is the presentation of data in graphical format. It helps people understand the significance of data by summarizing and presenting huge amount of data in a simple and easy-to-understand format and helps communicate information clearly and effectively. It is the discipline of trying to understand data by placing it in a visual context so that patterns, trends and correlations that might not otherwise be detected can be exposed.</p><p>Python offers multiple great graphing libraries that come packed with lots of different features. No matter if you want to create interactive, live or highly customized plots python has an excellent library for you.</p><p>To get a little overview here are a few popular plotting libraries:</p><ul><li><a href="https://matplotlib.org/"><strong>Matplotlib:</strong></a><strong> </strong>low level, provides lots of freedom</li><li><a href="https://pandas.pydata.org/pandas-docs/stable/visualization.html"><strong>Pandas Visualization:</strong></a><strong> </strong>easy to use interface, built on Matplotlib</li><li><a href="https://seaborn.pydata.org/"><strong>Seaborn:</strong></a><strong> </strong>high-level interface, great default styles</li><li><a href="http://ggplot.yhathq.com/"><strong>ggplot:</strong></a><strong> </strong>based on R’s ggplot2, uses <a href="https://www.amazon.com/Grammar-Graphics-Statistics-Computing/dp/0387245448">Grammar of Graphics</a></li><li><a href="https://plot.ly/python/"><strong>Plotly:</strong></a><strong> </strong>can create interactive plots</li></ul><p>In this article, we will learn how to create basic plots using Matplotlib, Pandas visualization and Seaborn as well as how to use some specific features of each library. This article will focus on the syntax and not on interpreting the graphs, which I will cover in another blog post.</p><p>In further articles, I will go over interactive plotting tools like Plotly, which is built on D3 and can also be used with JavaScript.</p><h3>Importing Datasets</h3><p>In this article, we will use two datasets which are freely available. The <a href="https://archive.ics.uci.edu/ml/datasets/iris">Iris</a> and <a href="https://www.kaggle.com/zynicide/wine-reviews">Wine Reviews</a> dataset, which we can both load in using pandas read_csv method.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/411/0*UfHTgIjWTxQd2FLd.png" /></figure><p>Figure 2: Iris dataset head</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/996/0*tkEpt9U54BWV39zK.png" /></figure><p>Figure 3: Wine Review dataset head</p><h3>Matplotlib</h3><p>Matplotlib is the most popular python plotting library. It is a low-level library with a Matlab like interface which offers lots of freedom at the cost of having to write more code.</p><p>To install Matplotlib pip and conda can be used.</p><pre>pip install matplotlib<br>or<br>conda install matplotlib</pre><p>Matplotlib is specifically good for creating basic graphs like line charts, bar charts, histograms and many more. It can be imported by typing:</p><pre>import matplotlib.pyplot as plt</pre><h3>Scatter Plot</h3><p>To create a scatter plot in Matplotlib we can use the scatter method. We will also create a figure and an axis using plt.subplots so we can give our plot a title and labels.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/390/0*cCEioxzpwQgsR2Lr.png" /></figure><p>Figure 4: Matplotlib Scatter plot</p><p>We can give the graph more meaning by coloring in each data-point by its class. This can be done by creating a dictionary which maps from class to color and then scattering each point on its own using a for-loop and passing the respective color.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/390/0*EHdnL50nOpK0X1FY.png" /></figure><p>Figure 5: Scatter Plot colored by class</p><h3>Line Chart</h3><p>In Matplotlib we can create a line chart by calling the plot method. We can also plot multiple columns in one graph, by looping through the columns we want and plotting each column on the same axis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/366/0*fI8B3AJSz6wqi0Ri.png" /></figure><p>Figure 6: Line Chart</p><h3>Histogram</h3><p>In Matplotlib we can create a Histogram using the hist method. If we pass it categorical data like the points column from the wine-review dataset it will automatically calculate how often each class occurs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/405/0*ikRrpFrpqULz_Zt2.png" /></figure><p>Figure 7: Histogram</p><h3>Bar Chart</h3><p>A bar chart can be created using the bar method. The bar-chart isn’t automatically calculating the frequency of a category so we are going to use pandas value_counts function to do this. The bar-chart is useful for categorical data that doesn’t have a lot of different categories (less than 30) because else it can get quite messy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/405/0*8Z4Vlof33xJRNuPX.png" /></figure><p>Figure 8: Bar-Chart</p><h3>Pandas Visualization</h3><p>Pandas is an open source high-performance, easy-to-use library providing data structures, such as dataframes, and data analysis tools like the visualization tools we will use in this article.</p><p>Pandas Visualization makes it really easy to create plots out of a pandas dataframe and series. It also has a higher level API than Matplotlib and therefore we need less code for the same results.</p><p>Pandas can be installed using either pip or conda.</p><pre>pip install pandas<br>or<br>conda install pandas</pre><h3>Scatter Plot</h3><p>To create a scatter plot in Pandas we can call &lt;dataset&gt;.plot.scatter() and pass it two arguments, the name of the x-column as well as the name of the y-column. Optionally we can also pass it a title.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/390/0*9RM5fWRuoIvGHJcu.png" /></figure><p>Figure 9: Scatter Plot</p><p>As you can see in the image it is automatically setting the x and y label to the column names.</p><h3>Line Chart</h3><p>To create a line-chart in Pandas we can call &lt;dataframe&gt;.plot.line(). Whilst in Matplotlib we needed to loop-through each column we wanted to plot, in Pandas we don’t need to do this because it automatically plots all available numeric columns (at least if we don’t specify a specific column/s).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/366/0*uTO85UUlgVS6-6nX.png" /></figure><p>Figure 10: Line Chart</p><p>If we have more than one feature Pandas automatically creates a legend for us, as can be seen in the image above.</p><h3>Histogram</h3><p>In Pandas, we can create a Histogram with the plot.hist method. There aren’t any required arguments but we can optionally pass some like the bin size.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/405/0*utTFNb5Wcu42iWy9.png" /></figure><p>Figure 11: Histogram</p><p>It’s also really easy to create multiple histograms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/609/0*iilLFe9wQajcnwFO.png" /></figure><p>Figure 12: Multiple Histograms</p><p>The subplots argument specifies that we want a separate plot for each feature and the layout specifies the number of plots per row and column.</p><h3>Bar Chart</h3><p>To plot a bar-chart we can use the plot.bar() method, but before we can call this we need to get our data. For this we will first count the occurrences using the value_count() method and then sort the occurrences from smallest to largest using the sort_index() method.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/391/0*1UlpEdqLqrsc0kEJ.png" /></figure><p>Figure 13: Vertical Bar-Chart</p><p>It’s also really simple to make a horizontal bar-chart using the plot.barh() method.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/380/0*ODrVCdqYqMO7PgN1.png" /></figure><p>Figure 14: Horizontal Bar-Chart</p><p>We can also plot other data then the number of occurrences.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/372/0*8Oe4CVbJOjHjvG2_.png" /></figure><p>Figure 15: Countries with the most expensive wine(on average)</p><p>In the example above we grouped the data by country and then took the mean of the wine prices, ordered it, and plotted the 5 countries with the highest average wine price.</p><h3>Seaborn</h3><p>Seaborn is a Python data visualization library based on Matplotlib. It provides a high-level interface for creating attractive graphs.</p><p>Seaborn has a lot to offer. You can create graphs in one line that would take you multiple tens of lines in Matplotlib. Its standard designs are awesome and it also has a nice interface for working with pandas dataframes.</p><p>It can be imported by typing:</p><pre>import seaborn as sns</pre><h3>Scatter plot</h3><p>We can use the .scatterplot method for creating a scatterplot, and just as in Pandas we need to pass it the column names of the x and y data, but now we also need to pass the data as an additional argument because we aren’t calling the function on the data directly as we did in Pandas.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/390/0*3wYY7_xUa1nScvxP.png" /></figure><p>Figure 16: Scatterplot</p><p>We can also highlight the points by class using the hue argument, which is a lot easier than in Matplotlib.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/390/0*lrpb_NiJmpYGqzcT.png" /></figure><p>Figure 17: Scatterplot colored by class</p><h3>Line chart</h3><p>To create a line-chart the sns.lineplot method can be used. The only required argument is the data, which in our case are the four numeric columns from the Iris dataset. We could also use the sns.kdeplot method which rounds of the edges of the curves and therefore is cleaner if you have a lot of outliers in your dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/337/0*TQ8ZLmt0it-b9g36" /></figure><p>Figure 18: Line Chart</p><h3>Histogram</h3><p>To create a histogram in Seaborn we use the sns.distplot method. We need to pass it the column we want to plot and it will calculate the occurrences itself. We can also pass it the number of bins, and if we want to plot a gaussian kernel density estimate inside the graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/391/0*-3orJZj6lwkAODbT.png" /></figure><p>Figure 19: Histogram</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/388/0*yWTB7Dgp4JFj3IL6.png" /></figure><p>Figure 20: Histogram with gaussian kernel density estimate</p><h3>Bar chart</h3><p>In Seaborn a bar-chart can be created using the sns.countplot method and passing it the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/405/0*E8rwighLg_kqARuj.png" /></figure><p>Figure 21: Bar-Chart</p><h3>Other graphs</h3><p>Now that you have a basic understanding of the Matplotlib, Pandas Visualization and Seaborn syntax I want to show you a few other graph types that are useful for extracting insides.</p><p>For most of them, Seaborn is the go-to library because of its high-level interface that allows for the creation of beautiful graphs in just a few lines of code.</p><h3>Box plots</h3><p>A Box Plot is a graphical method of displaying the <a href="https://en.wikipedia.org/wiki/Five-number_summary">five-number summary</a>. We can create box plots using seaborns sns.boxplot method and passing it the data as well as the x and y column name.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/364/0*Wa2umuba_ehUv9rK" /></figure><p>Figure 22: Boxplot</p><p>Box Plots, just like bar-charts are great for data with only a few categories but can get messy really quickly.</p><h3>Heatmap</h3><p>A Heatmap is a graphical representation of data where the individual values contained in a <a href="https://en.wikipedia.org/wiki/Matrix_(mathematics)">matrix</a> are represented as colors. Heatmaps are perfect for exploring the correlation of features in a dataset.</p><p>To get the correlation of the features inside a dataset we can call &lt;dataset&gt;.corr(), which is a Pandas dataframe method. This will give us the <a href="https://www.displayr.com/what-is-a-correlation-matrix/">correlation matrix</a>.</p><p>We can now use either Matplotlib or Seaborn to create the heatmap.</p><p>Matplotlib:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/305/0*o0PMuozg1FTtE8ML.png" /></figure><p>Figure 23: Heatmap without annotations</p><p>To add annotations to the heatmap we need to add two for loops:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/305/0*6KhhvNptI-nTNTrM.png" /></figure><p>Figure 24: Heatmap with annotations</p><p>Seaborn makes it way easier to create a heatmap and add annotations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/418/0*8uZJilpkdOrAxnTy.png" /></figure><p>Figure 25: Heatmap with annotations</p><h3>Faceting</h3><p>Faceting is the act of breaking data variables up across multiple subplots and combining those subplots into a single figure.</p><p>Faceting is really helpful if you want to quickly explore your dataset.</p><p>To use one kind of faceting in Seaborn we can use the FacetGrid. First of all, we need to define the FacetGrid and pass it our data as well as a row or column, which will be used to split the data. Then we need to call the map function on our FacetGrid object and define the plot type we want to use, as well as the column we want to graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/584/0*j55DnmsXeuHrEMlD" /></figure><p>Figure 26: Facet-plot</p><p>You can make plots a lot bigger and more complicated than the example above. You can find a few examples <a href="https://seaborn.pydata.org/generated/seaborn.FacetGrid.html">here</a>.</p><h3>Pairplot</h3><p>Lastly, I will show you Seaborns pairplot and Pandas scatter_matrix , which enable you to plot a grid of pairwise relationships in a dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/637/0*DTeSnM8lsrlnifgL" /></figure><p>Figure 27: Pairplot</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/0*IhVw5qn-TER0UN6x.png" /></figure><p>Figure 28: Scatter matrix</p><p>As you can see in the images above these techniques are always plotting two features with each other. The diagonal of the graph is filled with histograms and the other plots are scatter plots.</p><blockquote><strong><em>Final Note :</em></strong></blockquote><p><strong>Thanks for reading! If you liked this article, please hit the clap</strong>👏<strong> button as many times as you can. It would mean a lot and encourage me to keep sharing my knowledge.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=15cbcde7e8f7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Handwritten Digit Recognition using Machine Learning in Python]]></title>
            <link>https://arnab-dey.medium.com/handwritten-digit-recognition-using-machine-learning-in-python-a22d562a1e1f?source=rss-44286b7db55------2</link>
            <guid isPermaLink="false">https://medium.com/p/a22d562a1e1f</guid>
            <category><![CDATA[digit-recognition]]></category>
            <category><![CDATA[handwritten-digit]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[sklearn]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Arnab Dey]]></dc:creator>
            <pubDate>Tue, 08 Jun 2021 06:37:04 GMT</pubDate>
            <atom:updated>2021-06-12T05:49:41.749Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/560/1*JZF2O7U6rtNKBDsjbTLIVA.jpeg" /></figure><p>Recognizing handwritten text is a problem that can be traced back to the first automatic machines that needed to recognize individual characters in handwritten documents. Think about, for example, the ZIP codes on letters at the post office and the automation needed to recognize these five digits. Perfect recognition of these codes is necessary in order to sort mail automatically and efficiently. Included among the other applications that may come to mind is OCR (Optical Character Recognition) software. OCR software must read handwritten text, or pages of printed books, for general electronic documents in which each character is well defined. But the problem of handwriting recognition goes farther back in time, more precisely to the early 20th Century (1920s), when Emanuel Goldberg (1881–1970) began his studies regarding this issue and suggested that a statistical approach would be an optimal choice.</p><blockquote><strong><em>Hypothesis :</em></strong></blockquote><p>The Digits data set of the Scikit-learn library provides numerous data-sets that are useful for testing many problems of data analysis and prediction of the results. Some Scientist claims that it predicts the digit accurately 95% of the times. Perform data Analysis to accept or reject this Hypothesis.</p><blockquote><strong><em>Prerequists :</em></strong></blockquote><p>Sklearn</p><p>Matplotlib</p><p>Basics of Machine learning</p><blockquote><strong><em>Dataset :</em></strong></blockquote><p>In this project, we are using the Handwritten Digits dataset which is already ready in the sklearn library. we can import the dataset using the below code.</p><pre>from sklearn import datasets<br>digits = datasets.load_digits()</pre><p>Digits dataset is a dictionary that contains data, targets, images, features names, description of the dataset, target names, etc.</p><p>We focus mainly on data and targets. We extract both on different variables.</p><p>main_data = digits[‘data’]<br>targets = digits[‘target’]</p><p>Now we can see our data using the following code.</p><pre>def view_digit(index):<br>    plt.imshow(digits.images[index] , cmap = plt.cm.gray_r , interpolation = &#39;nearest&#39;)<br>    plt.title(&#39;Orignal it is: &#39;+ str(digits.target[index]))<br>    plt.show()</pre><pre>view_digit(17)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/245/1*rhufInSlij3AskcSxTv3JQ.png" /><figcaption><strong><em>Model Planning:</em></strong></figcaption></figure><p><strong>Model Planning</strong></p><p>To see how different models work on different data sizes we are using 3 models Support vector Classifier, Decision Tree Classifier, Random Forest Classifier.</p><ol><li><strong>Support Vector Classifier :</strong></li></ol><p>The objective of the support vector machine algorithm is to find a hyperplane in an N-dimensional space(N — the number of features) that distinctly classifies the data points.</p><p><a href="https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47">More…</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*fOhYupL1BuUVxTFLVN-hFA.jpeg" /><figcaption>Ref:- <a href="https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47">https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47</a></figcaption></figure><p><strong><em>Code :</em></strong></p><pre># import the SVC<br>from sklearn import svm<br>svc = svm.SVC(gamma=0.001 , C = 100.) <br># gamma and C are hyperparameters# Training data = 1790 , Validation data = 6<br>svc.fit(main_data[:1790] , targets[:1790])# predict on test data<br>predictions = svc.predict(main_data[1791:])# check the result <br>predictions , targets[1791:]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*Rwuc4ZOIbq26cC4IU95mfA.jpeg" /><figcaption>SVC Output</figcaption></figure><p>As we can see we use very high data for training and very little data for training and the Support Vector Classifier do a very good job on data and we get 100 % accuracy on test data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/669/1*lebA5opdJVF2kyzgn2ux0w.jpeg" /></figure><p><strong>2. Decision Tree Classifier :</strong></p><p>Decision Tree Classifier is a simple and widely used classification technique. It applies a straightforward idea to solve the classification problem. Decision Tree Classifier poses a series of carefully crafted questions about the attributes of the test record. Each time it receives an answer, a follow-up question is asked until a conclusion about the class label of the record is reached.</p><p><a href="http://mines.humanoriented.com/classes/2010/fall/csci568/portfolio_exports/lguo/decisionTree.html">More Details on Decision Tree…</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*HvggGsrJMuMLYaOKskTa1g.jpeg" /><figcaption>Ref:- <a href="http://mines.humanoriented.com/classes/2010/fall/csci568/portfolio_exports/lguo/decisionTree.html">http://mines.humanoriented.com/classes/2010/fall/csci568/portfolio_exports/lguo/decisionTree.html</a></figcaption></figure><p><strong><em>Code :</em></strong></p><pre># import the Classifier<br>from sklearn.tree import DecisionTreeClassifier# Instanciate Model<br># we can also use criterion = &#39;entropy&#39; both lead us to nearly same <br># resultdt = DecisionTreeClassifier(criterion = &#39;gini&#39;) # fit the data on model<br># Training Set = 1600 , Validation Set = 197<br>dt.fit(main_data[:1600] , targets[:1600])# prediction on test data<br>predictions2 = dt.predict(main_data[1601:])# We use classification materics as accuracy_score<br># import accuracy_score<br>from sklearn.metrics import accuracy_scoreaccuracy_score(targets[1601:] , predictions2)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*CMzzDu0XC0pwKT9NXP2-GQ.jpeg" /><figcaption>Accuracy of Decision Tree Classifier</figcaption></figure><p>Now for this time, we use different sizes of training data and validation data. As we can see Decision Tree classifier performs poor performance on the data. we can increase the accuracy by fine-tuning the hyperparameters of DTC.</p><p><a href="https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html">Hyperparameters of Decision Tree Classifier</a></p><p><strong>3. Random Forest Classifier :</strong></p><p>Random forests is a supervised learning algorithm. It can be used both for classification and regression. It is also the most flexible and easy-to-use algorithm. A forest is comprised of trees. It is said that the more trees it has, the more robust a forest is. Random forests create decision trees on randomly selected data samples, get a prediction from each tree, and selects the best solution by means of voting. It also provides a pretty good indicator of the feature’s importance.</p><p><a href="https://www.datacamp.com/community/tutorials/random-forests-classifier-python">More on Random Forest…</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*7KmEKdMFmFabtjJonxJuEQ.jpeg" /><figcaption>Ref:- <a href="https://www.datacamp.com/community/tutorials/random-forests-classifier-python">https://www.datacamp.com/community/tutorials/random-forests-classifier-python</a></figcaption></figure><p><strong><em>Code :</em></strong></p><pre>from sklearn.ensemble import RandomForestClassifier# n_estimators hyperparameters( default 100 )<br>rc = RandomForestClassifier(n_estimators = 150)# Training Data = 1500 , Validation data = 297<br>rc.fit(main_data[:1500] , targets[:1500])predictions3 = rc.predict(main_data[1501:])accuracy_score(targets[1501:] , predictions3)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/694/1*drFhh6ldmBFr09l6N_NIxA.jpeg" /><figcaption>Accuracy score of Random Forest Classifier</figcaption></figure><p>As we can see Random Forest performs excellently with fewer data compare to both the Decision tree and Support vector. We get a 92 % Accuracy score with Random Forest Classifier.</p><blockquote><strong><em>Conclusion :</em></strong></blockquote><p>As per our hypothesis, we can say with hyperparameter tunning with different machine learning models or using more data we can achieve near 95% accuracy on the handwritten dataset. But make sure we also have a good amount of test data otherwise the model will get overfit.</p><blockquote><strong><em>Code :</em></strong></blockquote><p><a href="https://github.com/arnab132/Recognizing-Hand-Written-digits-using-Machine-Learning-in-Python">https://github.com/arnab132/Recognizing-Hand-Written-digits-using-Machine-Learning-in-Python</a></p><blockquote><strong><em>Connect with me:</em></strong></blockquote><p><strong><em>Github :</em></strong></p><p><a href="https://github.com/arnab132">arnab132 - Overview</a></p><blockquote><strong><em>Final Note :</em></strong></blockquote><p><strong>Thanks for reading! If you liked this article, please hit the clap</strong>👏<strong> button as many times as you can. It would mean a lot and encourage me to keep sharing my knowledge.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a22d562a1e1f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Meteorological Data Analysis using Python]]></title>
            <link>https://medium.com/data-science-and-machine-learning-by-arnab-dey/meteorological-data-analysis-using-python-816a290f76a2?source=rss-44286b7db55------2</link>
            <guid isPermaLink="false">https://medium.com/p/816a290f76a2</guid>
            <category><![CDATA[weather-analytics]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[meteorological-data]]></category>
            <category><![CDATA[python-data-analysis]]></category>
            <category><![CDATA[data-analytics]]></category>
            <dc:creator><![CDATA[Arnab Dey]]></dc:creator>
            <pubDate>Tue, 08 Jun 2021 05:00:21 GMT</pubDate>
            <atom:updated>2021-06-14T10:43:33.981Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/750/0*mU83qf_rs9J3zeNk.jpg" /></figure><p>Data analysis can be described as a process consisting of several steps in which the raw data are transformed and processed in order to produce data visualizations and make predictions. Data analysis is schematized as a process chain consisting of the following sequence of stages:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/434/0*3WtaknT23f-1cerR.png" /></figure><p><strong>Null Hypothesis</strong> means we need to find whether the average Apparent temperature for the month of say April starting from 2006 to 2016 and the average humidity for the same period have increased or not.</p><p><strong>Dataset:</strong> <a href="https://www.kaggle.com/muthuj7/weather-dataset">https://www.kaggle.com/muthuj7/weather-dataset</a></p><p><strong>Code:</strong></p><pre><strong>import</strong> <strong>numpy</strong> <strong>as</strong> <strong>np</strong> <br><strong>import</strong> <strong>pandas</strong> <strong>as</strong> <strong>pd</strong> <br><strong>import</strong> <strong>matplotlib.pyplot</strong> <strong>as</strong> <strong>plt</strong> <br><strong>import</strong> <strong>seaborn</strong> <strong>as</strong> <strong>sns</strong> <br><strong>import</strong> <strong>warnings</strong> <br>warnings.filterwarnings(&#39;ignore&#39;) <br>%matplotlib inline</pre><pre>df = pd.read_csv(&#39;weatherHistory.csv&#39;)<br>df.head()</pre><pre>df.shape<br>df.dtypes<br>df.describe()<br></pre><pre>df[&#39;Formatted Date&#39;] = pd.to_datetime(df[&#39;Formatted Date&#39;],utc= <strong>True</strong>) <br>df[&#39;Formatted Date&#39;]</pre><pre><em># We will check if any Nan value is present or not? </em><br>df.isnull().sum()</pre><pre><em># First we set Formatted Date as our index, because we need every year data of a specific month. </em><br>df = df.set_index(&#39;Formatted Date&#39;)<br>df.head()</pre><pre><em># We need only 2 columns from our Dataset. so will fetched that columns only. </em><br>cols = df[[&#39;Apparent Temperature (C)&#39;,&#39;Humidity&#39;]] <br>df_avg = cols.resample(&#39;MS&#39;).mean()<br><em># &quot;MS&quot; ==&gt; Month starting </em><br><em># We are actually displaying the avg apparent temperature (C) and humidity using mean()</em><br>df_avg.head()</pre><pre><em># We will plot both value with respect to datetime i.e. index column to check variation in data. </em><br>plt.figure(figsize=(12,7))<br>plt.title(&quot;Changes in Apparent Temperature (C) and Humidity with time&quot;)<br>sns.lineplot(data=df_avg);</pre><pre><em># Now we will find particular month == &#39;April&#39; as per question. </em><br><em># we will extract only April month data using index.month == 4. </em><br>data = df_avg[df_avg.index.month==4]<br>data</pre><pre><strong>import</strong> <strong>matplotlib.dates</strong> <strong>as</strong> <strong>dts</strong></pre><pre><em># Now we will subplot the 2 variable with respect to index value. </em><br><br>fig, st = plt.subplots(figsize = (13,7))<br><br>st.plot(data.loc[&#39;2006-04-01&#39;:&#39;2016-04-01&#39;, &#39;Apparent Temperature (C)&#39;],marker=&#39;o&#39;, linestyle=&#39;-&#39;,label=&#39;Apparent Temperature (C)&#39;);<br>st.plot(data.loc[&#39;2006-04-01&#39;:&#39;2016-04-01&#39;, &#39;Humidity&#39;],marker=&#39;o&#39;, linestyle=&#39;-&#39;,label=&#39;Humidity&#39;);<br>st.set_xticks([&#39;04-01-2006&#39;,&#39;04-01-2007&#39;,&#39;04-01-2008&#39;,&#39;04-01-2009&#39;,<br>               &#39;04-01-2010&#39;,&#39;04-01-2011&#39;,&#39;04-01-2012&#39;,&#39;04-01-2013&#39;,&#39;04-01-2014&#39;,&#39;04-01-2015&#39;,&#39;04-01-2016&#39;])<br>st.xaxis.set_major_formatter(dts.DateFormatter(&#39;<strong>%d</strong> %m %Y&#39;))<br>st.set_xlabel(&#39;Month of April&#39;)<br>st.legend(loc = &#39;center right&#39;)<br>plt.title(&quot;Yearwise Variation in Apparent Temperature (C) and Humidity for the Month of April&quot;);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/711/1*Y7W8Sn72G7K9FWjxjMm2IQ.png" /></figure><p><strong>Fig. 1: Screenshot of the Plot of Changes in Apparent Temp and Humidity with time</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/759/1*BHfmD-CppRLdnf0iRv0eig.png" /></figure><p><strong>Fig. 2: Screenshot of Plot between Year-wise Variation in Apparent Temp and Humidity in April month</strong></p><p><strong>Final Conclusion : -</strong></p><p>The Average Apparent Temperature increase for year 2008–09 and then again decrease from 2009–10 to its average level. Then slighlty increase for period 2010–11 and then drop again for year 2011–12. For year 2014–15 it decrease drastically and then come to average level for year 2015–16.</p><p>It observed, for year 2009 the Apparent Temperature is at Highest 14.26(C) and for year 2015 the Apparent Temperature is at lowest 10.63(C).</p><p>There is no any change in average Humidity for given year 2006–2016. The humidity graph line is approximately parallel to the X-axis.</p><p><strong>Reference</strong></p><p><a href="https://www.kaggle.com/muthuj7/weather-dataset">Weather Dataset</a></p><blockquote><strong><em>Final Note :</em></strong></blockquote><p><strong>Thanks for reading! If you liked this article, please hit the clap</strong>👏<strong> button as many times as you can. It would mean a lot and encourage me to keep sharing my knowledge.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=816a290f76a2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-and-machine-learning-by-arnab-dey/meteorological-data-analysis-using-python-816a290f76a2">Meteorological Data Analysis using Python</a> was originally published in <a href="https://medium.com/data-science-and-machine-learning-by-arnab-dey">Data Science and Machine Learning by Arnab Dey</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cyber Security]]></title>
            <link>https://medium.com/cyber-security-an-indian-perspective/cyber-security-f9bcd437714e?source=rss-44286b7db55------2</link>
            <guid isPermaLink="false">https://medium.com/p/f9bcd437714e</guid>
            <category><![CDATA[cyber]]></category>
            <category><![CDATA[india-perspective]]></category>
            <category><![CDATA[arnab-dey]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[security]]></category>
            <dc:creator><![CDATA[Arnab Dey]]></dc:creator>
            <pubDate>Fri, 09 Aug 2019 03:27:40 GMT</pubDate>
            <atom:updated>2021-06-14T10:41:00.233Z</atom:updated>
            <content:encoded><![CDATA[<h3>Cyber Security: An Indian Perspective</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*EK5H_MOwWC1gYoOv.jpg" /><figcaption>Source: Picture given by The Independent on the article</figcaption></figure><p>The rapid growth of the internet and computer technology over the past few years has led to the growth in new forms of crime — dubbed cyber crime — throughout the world. The term “cyber crime” has nowhere been defined in any statute or Act passed or enacted by the Indian Parliament. Today, the world is moving towards a point where everything from banking to stock exchanges and traffic control, telephones to electric power, health care, welfare and education depends on software. This exponential growth, and the increase in the capacity and accessibility of computers coupled with the decrease in cost, has brought about revolutionary changes in every aspect of human civilisation, including crime. It is rapidly evolving from simple e-mail mischief where offenders send obscene e-mail, to more serious offences like theft of information, e-mail bombing to crashing servers, etc. It is sure all of us have encountered unwanted product advertisements. Piracy is also a serious threat that involves illegal reproduction and distribution of software applications, games, movies and audio CDs which is causing a huge monetary loss to entertainment industry worldwide.</p><p>Hacking is also a serious crime by a hacker who is simply a talented computer user who misuses his vast knowledge. Virus attacks that say “I love you” enter our computer and destroy all the computer information are also a serious threat. Computer frauds, thefts and harassment through social networking sites like Twitter and Facebook are also risking our lives.<br>Undoubtedly we cannot single out only one crime that is posing maximum threat but all forms of crimes are equally posing a serious threat to both business and individuals.</p><p>Cyber Crime in the Act is neither comprehensive nor exhaustive. Cases of spam, hacking, stalking and e-mail fraud are rampant although cyber crimes cells have been set-up in major Indian cities. The problem is that most cases remain unreported due to lack of awareness. Capacity of human mind is unfathomable. It is not possible to eliminate cyber crime from the cyber space. However, it is quite possible to check them.</p><p>The home user segment is the largest recipient of cyber attacks as they are less likely to have established security measures in place and therefore it is necessary that people should be made aware of their rights and duties. The fight against cyber crime starts in our very own home. We should not reply any e-mail from unknown persons; learn to report spam mails to the e-mail servers. We should not upload our personal information on social networking sites or our account details on other such sites.</p><p>Also the use of antivirus software can be a great help to fight against viruses and worms. Users must try and save any electronic information trail on their computers, firewalls, use of intrusion detection system, etc. and further making the application of the laws more stringent to check crime. Potentially, everything from generic word processing software to customized computer programmes designed for a specific company could work on a cloud computing system.</p><p>It is a good idea to rely on another computer system to run programmes and store data as it would enable clients to access their applications and data from anywhere at any time. They could access the cloud computing system using any computer linked to the Internet. Data would not be confined to a hard drive on one user’s computer or even a corporation’s internal network.<br>Also, it could bring hardware costs down as the client would not need to buy the fastest computer with the most memory, because the cloud system would take care of the rest. Often, scientists and researchers work with calculations so complex that it would take years for individual computers to complete them. The cloud system would tap into the processing power of all available computers on the back end, significantly speeding up the calculation.</p><p>There will always be new and unexpected challenges to stay ahead of cyber criminals and cyber terrorists but we can win only through partnership and collaboration of both individuals and government. There is much we can do to ensure a safe, secure and trustworthy computing environment. It is crucial not only to our national sense of well-being, but also to our national security and economy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*5ydiXxCiv6Vy23I0.jpeg" /></figure><p>Published in <strong>The Independent </strong>paper<strong> </strong>by Arnab Dey on 16th July 2016</p><p>Reference:</p><p><a href="https://www.theindependentbd.com/printversion/details/51380">https://www.theindependentbd.com/printversion/details/51380</a></p><blockquote><strong><em>Final Note :</em></strong></blockquote><p><strong>Thanks for reading! If you liked this article, please hit the clap</strong>👏<strong> button as many times as you can. It would mean a lot and encourage me to keep sharing my knowledge.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f9bcd437714e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cyber-security-an-indian-perspective/cyber-security-f9bcd437714e">Cyber Security</a> was originally published in <a href="https://medium.com/cyber-security-an-indian-perspective">Cyber Security</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>