<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Akshat Anand on Medium]]></title>
        <description><![CDATA[Stories by Akshat Anand on Medium]]></description>
        <link>https://medium.com/@its-axat?source=rss-f064ddbf5711------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*8zfDn22BQth9Yy1OHT87pg.jpeg</url>
            <title>Stories by Akshat Anand on Medium</title>
            <link>https://medium.com/@its-axat?source=rss-f064ddbf5711------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 08:52:42 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@its-axat/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Why VR can be a great potential for meditation and mindfulness?]]></title>
            <link>https://its-axat.medium.com/why-vr-can-be-a-great-potential-for-meditation-and-mindfulness-6420acd0f3c6?source=rss-f064ddbf5711------2</link>
            <guid isPermaLink="false">https://medium.com/p/6420acd0f3c6</guid>
            <category><![CDATA[metaverse]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[meditation]]></category>
            <category><![CDATA[vr]]></category>
            <category><![CDATA[mindfulness]]></category>
            <dc:creator><![CDATA[Akshat Anand]]></dc:creator>
            <pubDate>Sun, 10 Apr 2022 10:20:25 GMT</pubDate>
            <atom:updated>2022-04-10T10:20:25.139Z</atom:updated>
            <content:encoded><![CDATA[<p>VR has great potential for mindfulness, because of its inherent ability to control the environment and create scenarios to help break the illusion of the Self.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8T3Mpn9J1U__34Zc" /><figcaption>Photo by <a href="https://unsplash.com/@nampoh?utm_source=medium&amp;utm_medium=referral">Maxim Hopman</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>Mindfulness: what it does not add</strong></p><p>Mindfulness meditation is not intended to be a stress reliever or a focus-improving practice. It’s not even intended to be a method of enhancing your mental health. All of these are unintended consequences of the practice.</p><p>It’s not attempting to avoid having thoughts. It is not required to be done while sitting with your eyes closed. Meditation is not a single technique, but rather a collection of activities from various cultures across the world that include gaining knowledge of the nature of consciousness.</p><p><strong>Meditation’s purpose</strong></p><p>Understanding the genuine nature of awareness, that is, recognising what your mind is truly like and who you are, is a key objective of various mindfulness techniques. And, sure, you are most likely incorrect about what you believe your mind is like. The technique is useful because it allows us to untangle the myths and erroneous ideas we have about our minds and identities.</p><p>And you would be saying “What you just stated smells like racketeering to me.”</p><p>I appreciate it. I, too, despise crackpots and dubious scientific claims. We now lack good terminology to express the benefits of meditation, but I will attempt to spell out some realities with the practice. It’s about realising that your ideas and experiences do not originate from you, but rather from this emptiness, as a matter of experience. Those are not your thoughts. Similarly, bodily experiences such as sound, touch, pain, and sight are not truly You; they are only felt in the great open expanse of your mind.</p><p>In actuality, everything you see, think, feel, and live through is not You, or you may conceive of it all as contained in a single consciousness. This is my interpretation of non-duality or the recognition that there is no difference between You and Everything Else. In my experience, they are both the same. It takes practice to achieve this condition of perceiving what you are for real, especially if you wish to retain this level of non-duality for an extended period of time. This condition also provides you with a sense of well-being and relaxation, as well as the ability to choose your ideas. You can always recognise and let rid of negative ideas while retaining positive ones (while not leaving the non-dual state).</p><p>It is also tough to acquire and requires consistent practice.</p><p><strong>This is where virtual reality can truly assist.</strong></p><p>VR is excellent at altering your surroundings. Many meditation methods place a high value on the surroundings. I envision that when used imaginatively, VR may effortlessly go from a “transparent” real-world feed to virtual worlds with diverse qualities in order to examine the many elements of consciousness and then combine them. It also appears to be in line with Douglas Harding’s “headless” studies.</p><p>In general, I believe that VR has the capacity to free a person from the sensation of identifying with his/her ideas by recognising the non-duality of awareness. I want to see amateurs, not academics, experimenting with new ideas and thinking out of the box.</p><p>If you really resonated with my thoughts, I would be grateful if you do <strong>clap</strong> and <strong>share</strong> among your friends and colleagues.</p><p>You can follow me here -</p><p>Linkedin — <a href="https://www.linkedin.com/in/akshatanand1999/"><strong>link</strong></a></p><p>Twitter— <a href="https://twitter.com/akshattweetsnow"><strong>link</strong></a></p><p>Peace!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6420acd0f3c6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Application of Tries and Ternary Search trees]]></title>
            <link>https://medium.com/data-science-community-srm/application-of-tries-and-ternary-search-trees-d38663c5a550?source=rss-f064ddbf5711------2</link>
            <guid isPermaLink="false">https://medium.com/p/d38663c5a550</guid>
            <category><![CDATA[internship-interview]]></category>
            <category><![CDATA[trees]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[coding-interviews]]></category>
            <category><![CDATA[data-structures]]></category>
            <dc:creator><![CDATA[Akshat Anand]]></dc:creator>
            <pubDate>Sat, 19 Dec 2020 16:53:23 GMT</pubDate>
            <atom:updated>2021-11-01T10:58:14.619Z</atom:updated>
            <content:encoded><![CDATA[<p>Learn how you can effectively implement tries and ternary search trees in real-world applications.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*PufS4H85BST1mBLL" /><figcaption>Photo by <a href="https://unsplash.com/@_imkiran?utm_source=medium&amp;utm_medium=referral">Sai Kiran Anagani</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Motivation</h3><p>So the motivation behind learning Tries and Ternary Search trees was my inability to answer one of the questions which were asked in one of my <strong>internship interviews</strong>. The question which was asked as “How you can implement or design a data structure so that you can<strong> search a word in a dictionary</strong>?”, my answer was “We can implement using Hash Tables and discussed the structure and process of finding a particular word by searching the key of the word in the dictionary”, so after this interviewer asked, “How you can avoid collision handling, also the one word with a different meaning?”, I was like speechless because I didn’t know about Tries and Ternary Search trees.</p><p>Ps: If you haven’t read about tree data structure, please refer to this <a href="https://www.geeksforgeeks.org/binary-tree-set-1-introduction/"><strong>link</strong></a></p><p>Update: Got an Offer from the same company as the FTE role, so be consistent and never give up :) .</p><h3>Tries</h3><p>Tries are a tree that <strong>stores strings</strong>. The maximum number of children of a node is equal to the size of the alphabet. Trie supports search, insert and delete operations in <strong>O(L)</strong> time where <strong>L</strong> is the length of a key.</p><p>It is an efficient information re<strong><em>Trie</em></strong>val data structure. Using Trie, search complexities can be brought to the optimal limit (key length). If we store keys in the binary search tree, a well-balanced BST will need time proportional to <strong>M * log N</strong>, where M is maximum string length and N is a number of keys in the tree. Using Trie, we can search the key in O(M) time. However, the penalty is on Trie storage requirements</p><p>A trie is, like other <strong>tree-based data structures,</strong> made up of a set of nodes connected by pointers. These pointers indicate a parent-child relationship between the nodes. Parents are above their children in the tree. Typically the root node of the tree is an “empty” node so that it can point to all members of the alphabet the trie is using to store.</p><p>Unlike other tree-based data structure like the <a href="https://www.tutorialspoint.com/data_structures_algorithms/avl_tree_algorithm.htm"><strong>AVL tree</strong></a>, the trie is not necessarily balanced. It is totally dependent on the contents of the tree.</p><p>Nodes in the trie can hold either single members of the alphabet or the entire word so far. For example, the tree could look like the graphic above, or it could look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/224/1*ofw5m-S2XppleEeU2xxNGQ.png" /></figure><p><strong>So why try Trie?</strong></p><ol><li>With Trie, we can insert and find strings in <strong><em>O(L)</em></strong> time where <em>L</em> represent the length of a single word. This is obviously faster than BST. This is also faster than Hashing because of the ways it is implemented. We do not need to compute any hash function. No collision handling is required (like we do in open addressing and separate chaining)</li><li>Another advantage of Trie is, we can easily print all words in <strong>alphabetical order</strong> which is not easily possible with hashing.</li><li>We can efficiently do a <strong>prefix search</strong> (or auto-complete) with Trie.</li></ol><p><strong>Disadvantages with Trie:-</strong> <br>The major disadvantage of tries is that they need a lot of memory for storing the strings. For each node we have too many node pointers(equal to a number of characters of the alphabet), <strong>if space is concerned, then Ternary Search Tree can be preferred for dictionary implementations</strong>. In Ternary Search Tree, the time complexity of search operation is O(h) where h is the height of the tree. Ternary Search Trees also supports other operations supported by Trie like prefix search, alphabetical order printing, and nearest neighbour search.</p><p><strong>Ternary Search Tree</strong></p><p>A ternary search tree is a special trie data structure where the child nodes of a standard trie are ordered as a binary search tree</p><p><strong>Representation of ternary search trees:</strong><br>Unlike the trie data structure where each node contains <strong>26 pointers</strong> for its children, each node in a ternary search tree contains only <strong>3 pointers</strong>:<br>1. The<strong> left pointer points</strong> to the node whose value <strong>is less</strong> than the value in the current node.<br>2. The<strong> equal pointer points</strong> to the node whose value <strong>is equal</strong> to the value in the current node.<br>3. The <strong>right pointer points</strong> to the node whose value <strong>is greater</strong> than the value in the current node.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/759/1*HGZfY55MHLPCXkzYIGaxkQ.jpeg" /></figure><p>Apart from the above three-pointers, each node has a field to indicate data(a character in case of dictionary) and another field to mark the end of a string.<br>So, more or less it is similar to BST which stores data based on some order. However, data in a ternary search tree is distributed over the nodes. e.g. It needs <strong>3 nodes </strong>to store the word <strong>“INK”</strong>.</p><p>The advantage of using ternary search trees over tries is that ternary search trees are more space-efficient (involve only <strong>three-pointers </strong>per node as compared to <strong>26 in standard tries</strong>). Further, ternary search trees can be used any time a hashtable would be used to store strings.</p><p>Tries are suitable when there is a proper distribution of words over the alphabets so that spaces are utilized most efficiently. Otherwise, ternary search trees are better. Ternary search trees are efficient to use(in terms of space) when the strings to be stored share a common prefix.</p><p><strong>Applications of ternary search trees:</strong><br><strong>1. </strong>Ternary search trees are efficient for queries like “<strong>Given a word, find the next word in the dictionary</strong>” or “<strong>Find all phone numbers starting with 8928”</strong> or “<strong>typing few starting characters in a web browser displays all website names with this prefix</strong>”(Autocomplete feature)”.</p><p><strong>2. Used in spell checks</strong>: Ternary search trees can be used as a dictionary to store all the words. Once the word is typed in an editor, the word can be parallelly searched in the ternary search tree to check for the correct spelling.</p><p>If you want to start practising tree, here is some set of questions:-</p><p>Tree basic question —<strong> </strong><a href="https://onedrive.live.com/?authkey=%21AJt9WdmM7U4qINk&amp;cid=842AECBB531CCEA4&amp;id=842AECBB531CCEA4%218965&amp;parId=842AECBB531CCEA4%218964&amp;o=OneUp"><strong>link</strong></a></p><p>Tree Intermediate question —<strong> </strong><a href="https://onedrive.live.com/?authkey=%21ABhbvz46VB61ano&amp;cid=842AECBB531CCEA4&amp;id=842AECBB531CCEA4%218966&amp;parId=842AECBB531CCEA4%218964&amp;o=OneUp"><strong>link</strong></a></p><p>I hope my shared experience will help someone in the preparation for their interviews. I would be grateful if you do <strong>clap</strong> and <strong>share</strong> among your friends and colleagues.</p><p>You can follow me here -</p><p>Linkedin — <a href="https://www.linkedin.com/in/akshatanand1999/"><strong>link</strong></a></p><p>Github — <a href="https://github.com/cipheraxat"><strong>link</strong></a></p><p>Peace!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d38663c5a550" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-community-srm/application-of-tries-and-ternary-search-trees-d38663c5a550">Application of Tries and Ternary Search trees</a> was originally published in <a href="https://medium.com/data-science-community-srm">DataX Journal</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Open source for synthetic tabular data generation using GANs]]></title>
            <link>https://medium.com/data-science-community-srm/open-source-for-synthetic-tabular-data-generation-using-gans-b3d686d68105?source=rss-f064ddbf5711------2</link>
            <guid isPermaLink="false">https://medium.com/p/b3d686d68105</guid>
            <category><![CDATA[data-generation]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Akshat Anand]]></dc:creator>
            <pubDate>Mon, 09 Nov 2020 04:05:46 GMT</pubDate>
            <atom:updated>2020-11-09T05:02:04.500Z</atom:updated>
            <content:encoded><![CDATA[<p>Learn about the importance of Synthetic Data Generation by using open-sourcedydata-synthetic library</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Kn4OKFb57E_g9DnL" /><figcaption>Photo by <a href="https://unsplash.com/@markusspiske?utm_source=medium&amp;utm_medium=referral">Markus Spiske</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>What is Synthetic data?</strong></p><p>It is any production data that can apply to a given situation that is not obtained by any direct or real measurement. In general words, it is a kind of data that is developed in an artificial manner that has not been collected from real-world events.</p><p>For many reasons and purposes, data is eventually generated by a computer simulation can be seen as synthetic data. They border most of the applications such as physical modeling, such as music generations or flight simulators etc. The final output of such systems resembles the real thing but is fully algorithmically generated.</p><p>In the context of privacy and security protection, the process of the creation of synthetic data is an involved process known as <strong>data </strong><a href="https://en.wikipedia.org/wiki/Anonymity"><strong>anonymization</strong></a><strong>. </strong>In general Synthetic data is used in a variety of fields as a filter for any information that would rather compromise the confidentiality for any particular aspects of the data. Like many times the particular aspects come from the human’s information such as name, IP address, telephone number, credit card number etc.</p><p><strong>Why should we use the synthetic data generation process?</strong></p><p>These data are generated to meet specific needs or conditions which may not be found in the original or real data and are generally used as a theoretical or simulation value, any situation etc. The important use-cases are is to protect the <strong>privacy and confidentiality</strong> of any kind of authentic data</p><p>Generally in real collected data are biased so by synthetic data generation we can <strong>remove the biases</strong> in the data which leads to overfitting of the machine learning/ deep learning models.</p><p>Synthetic data are more often generated to represent the authentic data and helps us to set the baseline of a particular standard.</p><p><strong>Applications</strong></p><p>Synthetic data are used generally in <strong>data mining </strong>processes. Testing and training of fraud detection systems. (a <a href="https://colab.research.google.com/github/ydataai/ydata-synthetic/blob/master/examples/gan_example.ipynb#scrollTo=q-x9saRwHioO"><strong>reference</strong></a> to this its been explained in this blog), making of confidentiality systems and other types of system which is devised using the synthetic data.</p><p>There are some specific algorithms and generators that are designed to create real-world data. This data assists in teaching a system on how to react to certain situations or criteria.</p><p><strong>Synthetic data in Machine learning</strong></p><p>Due to increasingly being used for machine learning purposes, a model is trained on a synthetically generated dataset with intention of <strong>transfer learning</strong> on real data.</p><p>Some advantages of this data are:</p><ul><li>Once the synthetic environment is prepared, we can prepare an<strong> enormous amount of data </strong>as much as it is needed.</li><li>They have<strong> perfectly accurate labels</strong> so the training and testing process they show less error.</li><li>It can be <strong>used as a substitute</strong> if the real data segments contain any confidential data.</li></ul><p>Due to advances in generative models, <strong>GAN</strong>s( Generative Adversarial Networks), leads to the essential idea that one can produce data and then use it for training. This fully generated approach hasn’t been shaped up. But GANs and other adversarial training, in general, show successful results on improving synthetic data generation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*F3fux_r-YOx5cjX-" /><figcaption>Photo by <a href="https://unsplash.com/@mbaumi?utm_source=medium&amp;utm_medium=referral">Mika Baumeister</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Introducing an open-sourced <strong>ydata-synthetic </strong>library, It consists of a set of different GANs architectures developed using Tensorflow 2.0 for Generative Adversarial Networks for synthetic data generation. Generally, it is for regular tabular and time-series data.</p><p>To load this library on your system, use this given command-</p><pre>pip install ydata-synthetic</pre><p>We will be importing important functions for generating data</p><pre>import importlib<br>import sys<br>import pandas as pd<br>import numpy as np<br>import sklearn.cluster as cluster<br>import matplotlib.pyplot as plt</pre><pre>from ydata_synthetic.synthesizers import VanilllaGAN<br>from ydata_synthetic.preprocessing.credit_fraud import *<br>model = VanilllaGAN</pre><p>We will be using this on the credit fraud dataset, to use this library, and the dataset can be found from <a href="http://www.kaggle.com/mlg-ulb/creditcardfraud"><strong>here</strong></a><strong>.</strong></p><pre>data = pd.read_csv(&#39;data/creditcard.csv&#39;, index_col=[0])</pre><p>Here we read and preprocess the original data which we have downloaded from the Kaggle. Generally for generating synthetically data we need to have some amount of data from which we could create a synthetic environment for data generation.</p><pre>data_cols = list(data.columns[ data.columns != &#39;Class&#39; ])</pre><pre>label_cols = [&#39;Class&#39;]<br>print(&#39;Dataset columns: {}&#39;.format(data_cols))</pre><pre>sorted_cols = [&#39;V14&#39;, &#39;V4&#39;, &#39;V10&#39;, &#39;V17&#39;, &#39;V12&#39;, &#39;V26&#39;, &#39;Amount&#39;, &#39;V21&#39;, &#39;V8&#39;, &#39;V11&#39;, &#39;V7&#39;, &#39;V28&#39;, &#39;V19&#39;, &#39;V3&#39;, &#39;V22&#39;, &#39;V6&#39;, &#39;V20&#39;, &#39;V27&#39;, &#39;V16&#39;, &#39;V13&#39;, &#39;V25&#39;, &#39;V24&#39;, &#39;V18&#39;, &#39;V2&#39;, &#39;V1&#39;, &#39;V5&#39;, &#39;V15&#39;, &#39;V9&#39;, &#39;V23&#39;, &#39;Class&#39;]</pre><pre>processed_data = data[ sorted_cols ].copy()</pre><p>Here we create a list of column names and store it to data_cols and assigns label_cols variable the “Class” column, we print the value to get an idea in the dataset. Also, we make the order of the column and store it to sorted_cols, and makes a copy in processed_data.</p><pre>data = transformations(data)</pre><pre>train_data = data.loc[ data[&#39;Class&#39;]==1 ].copy()</pre><pre>print(&quot;Dataset info: Number of records - {} Number of varibles - {}&quot;.format(train_data.shape[0], train_data.shape[1]))</pre><pre>algorithm = cluster.KMeans<br>args, kwds = (), {&#39;n_clusters&#39;:2, &#39;random_state&#39;:0}</pre><pre>labels = algorithm(*args, **kwds).fit_predict(train_data[ data_cols ])<br>print( pd.DataFrame( [ [np.sum(labels==i)] for i in np.unique(labels) ], columns=[&#39;count&#39;], index=np.unique(labels) ) )</pre><pre>fraud_w_classes = train_data.copy()<br>fraud_w_classes[&#39;Class&#39;] = labels</pre><p>Before we going to train the GAN do not forget to apply the required data transformations to ease here we’ve applied a <strong>PowerTransformation as transformations()</strong>. For this example, we will only synthesize the minority class.</p><pre>noise_dim = 32</pre><pre>dim = 128</pre><pre>batch_size = 128<br>log_step = 100<br>epochs = 200+1<br>learning_rate = 5e-4</pre><pre>models_dir = &#39;./cache&#39;</pre><pre>train_sample = fraud_w_classes.copy().reset_index(drop=True)<br>train_sample = pd.get_dummies(train_sample, columns=[&#39;Class&#39;], prefix=&#39;Class&#39;, drop_first=True)</pre><pre>label_cols = [ i for i in train_sample.columns if &#39;Class&#39; in i ]<br>data_cols = [ i for i in train_sample.columns if i not in label_cols ]</pre><pre>train_sample[ data_cols ] = train_sample[ data_cols ] / 10 train_no_label = train_sample[ data_cols ]</pre><pre>gan_args = [batch_size, learning_rate, noise_dim, train_sample.shape[1], dim]<br>train_args = [&#39;&#39;, epochs, log_step]</pre><pre>synthesizer = model(gan_args)<br>synthesizer.train(train_sample, train_args)</pre><p>Below we can try to train our own generators using the available GANs architectures. We can train it either with labels (created using KMeans) or with no labels at all.</p><p>The thing to note that we’ve decided to synthesize only the minority class from the Credit Fraud dataset. For training the GAN model we can choose Vanilla GAN, CGAN, DCGAN etc.</p><p>We will be using <strong>VanilllaGAN from the given library</strong>.</p><p>After the model is trained we can create a synthetic environment for generating data.</p><pre>models = {&#39;GAN&#39;: [&#39;GAN&#39;, False, synthesizer.generator]}</pre><p>Loading the model for data generation.</p><pre># Setup parameters visualization parameters</pre><pre>seed = 17<br>test_size = 492 # number of fraud cases<br>noise_dim = 32</pre><pre>np.random.seed(seed)</pre><pre>z = np.random.normal(size=(test_size, noise_dim))</pre><pre>real = synthesizer.get_data_batch(train=train_sample, batch_size=test_size, seed=seed)</pre><pre>real_samples = pd.DataFrame(real, columns=data_cols+label_cols)labels = fraud_w_classes[&#39;Class&#39;]</pre><pre>model_names = [&#39;GAN&#39;]</pre><pre>colors = [&#39;deepskyblue&#39;,&#39;blue&#39;]<br>markers = [&#39;o&#39;,&#39;^&#39;]</pre><pre>class_labels = [&#39;Class 1&#39;,&#39;Class 2&#39;]<br>col1, col2 = &#39;V17&#39;, &#39;V10&#39;<br>base_dir = &#39;cache/&#39;</pre><pre>#Actual fraud data visualization</pre><pre>model_steps = [ 0, 200, 300]<br>rows = len(model_steps)<br>columns = 5</pre><pre>axarr = [[]]*len(model_steps)<br>fig = plt.figure(figsize=(14,rows*3))</pre><pre>for model_step_ix, model_step in enumerate(model_steps):<br>axarr[model_step_ix] = plt.subplot(rows, columns, model_step_ix*columns + 1)<br>for group, color, marker, label in zip(real_samples.groupby(&#39;Class_1&#39;), colors, markers, class_labels ):</pre><pre>plt.scatter( group[1][[col1]], group[1][[col2]],<br>label=label, marker=marker, edgecolors=color, facecolors=&#39;none&#39; )</pre><pre>plt.title(&#39;Actual Fraud Data&#39;)<br>plt.ylabel(col2) # Only add y label to left plot<br>plt.xlabel(col1)<br>xlims, ylims = axarr[model_step_ix].get_xlim(), axarr[model_step_ix].get_ylim()</pre><pre>if model_step_ix == 0:<br>legend = plt.legend()<br>legend.get_frame().set_facecolor(&#39;white&#39;)</pre><pre>for i, model_name in enumerate( model_names[:] ):</pre><pre>[model_name, with_class, generator_model] = models[model_name]<br>generator_model.load_weights( base_dir + &#39;_generator_model_weights_step_&#39;+str(model_step)+&#39;.h5&#39;)<br>ax = plt.subplot(rows, columns, model_step_ix*columns + 1 + (i+1) )</pre><pre>if with_class:<br>g_z = generator_model.predict([z, labels])<br>gen_samples = pd.DataFrame(g_z, columns=data_cols+label_cols)<br>for group, color, marker, label in zip( gen_samples.groupby(&#39;Class_1&#39;), colors, markers, class_labels ):<br>plt.scatter( group[1][[col1]], group[1][[col2]],</pre><pre>label=label, marker=marker, edgecolors=color, facecolors=&#39;none&#39; )</pre><pre>else:</pre><pre>g_z = generator_model.predict(z)<br>gen_samples = pd.DataFrame(g_z, columns=data_cols+[&#39;label&#39;])<br>gen_samples.to_csv(&#39;Generated_sample.csv&#39;)<br>plt.scatter( gen_samples[[col1]], gen_samples[[col2]],</pre><pre>label=class_labels[0], marker=markers[0], edgecolors=colors[0], facecolors=&#39;none&#39; )<br>plt.title(model_name)<br>plt.xlabel(data_cols[0])<br>ax.set_xlim(xlims), ax.set_ylim(ylims)<br>plt.suptitle(&#39;Comparison of GAN outputs&#39;, size=16, fontweight=&#39;bold&#39;)<br>plt.tight_layout(rect=[0.075,0,1,0.95])</pre><pre># Adding text labels for traning steps<br>vpositions = np.array([ i._position.bounds[1] for i in axarr ])<br>vpositions += ((vpositions[0] - vpositions[1]) * 0.35 )</pre><pre>for model_step_ix, model_step in enumerate( model_steps ):<br>fig.text( 0.05, vpositions[model_step_ix], &#39;training\nstep\n&#39;+str(model_step), ha=&#39;center&#39;, va=&#39;center&#39;, size=12)<br>plt.savefig(&#39;Comparison_of_GAN_outputs.png&#39;)</pre><p>After running this code we can generate the data synthetically.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/617/1*LZ7jQ6Dk9UzGcaKEuyUaBA.png" /></figure><p>After running we can see light blue data is generated on the right-hand side graph. Hence using the given library ydata-synthetic used to generate data using method <strong>synthesizer</strong>.</p><p>Source Code of application - <a href="https://colab.research.google.com/github/ydataai/ydata-synthetic/blob/master/examples/gan_example.ipynb#scrollTo=q-x9saRwHioO"><strong>here</strong></a></p><p>Github repo for the library —<a href="https://github.com/ydataai/ydata-synthetic"><strong> here</strong></a></p><p>Hope you enjoyed this! Do <strong>clap</strong> on the blog and follow me on medium and <a href="https://www.linkedin.com/in/akshatanand1999/"><strong>Linkedin</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b3d686d68105" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-community-srm/open-source-for-synthetic-tabular-data-generation-using-gans-b3d686d68105">Open source for synthetic tabular data generation using GANs</a> was originally published in <a href="https://medium.com/data-science-community-srm">DataX Journal</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to CNN]]></title>
            <link>https://medium.com/data-science-community-srm/introduction-to-cnn-f3aaa8d79891?source=rss-f064ddbf5711------2</link>
            <guid isPermaLink="false">https://medium.com/p/f3aaa8d79891</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[convolutional-neural-net]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Akshat Anand]]></dc:creator>
            <pubDate>Tue, 21 Jul 2020 03:31:01 GMT</pubDate>
            <atom:updated>2020-07-21T13:57:56.250Z</atom:updated>
            <content:encoded><![CDATA[<h4>Learn the basics of the Convolutional Neural Network in 10 minutes.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*c7nRefR5LfmNLcgf" /><figcaption>Photo by <a href="https://unsplash.com/@herfrenchness?utm_source=medium&amp;utm_medium=referral">Clarisse Croset</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>Image classification</strong> has always been a challenging task for all AI developers across all the fields. Convolutional Neural Network has emerged as the data-driven approach for this challenge. The overview of this post will be about the <strong>image representation</strong> and the <strong>layers</strong> that make up the <strong>convolutional neural network</strong>.</p><h3>Image Classification</h3><p>For image classification, we need a neural network that takes the image as an input and predicts the class of that image. For example, let us take an image for prediction of class.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G-w5fmsAtyiq1M4JUlKonw.png" /></figure><p>It looks simple as it predicts the class of the image. But the question is how?</p><h3>Convolutional Neural network</h3><p>To approach image classification, we will use CNN (Convolution Neural Network) that can find and define special patterns in 3D image space. The important thing which stands out CNN from other neural network is that it can look at a group of pixels in the image and can find spatial features which many neural networks look at as an individual point.</p><p>CNN is made up of multiple layers but there are three main types of layers-convolutional, pooling, and fully-connected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*AVCWwwIvuWqSMTpsNtUwzg.gif" /></figure><p>We have four different convolutional kernels, that will produce four different, filtered images as output. The idea is each filter will extract a different feature from an input image and these features will eventually help to classify that image, for example, one filter might detect the edges of objects in that image and another might detect unique patterns in color. These filters, stacked together, are what make up a convolutional layer.</p><h3>High-pass Filter</h3><p>Let’s have a closer look at one type of convolutional kernel: a high-pass image filter. High-pass filters are meant to detect abrupt changes in intensity over a small area. So, in a small patch of pixels, a high-pass filter will highlight areas that change from dark to light pixels (and vice versa). We will be looking at patterns of intensity and pixel values in a grayscale image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/996/1*cXjsArX89J_QNWad4292yQ.png" /></figure><p>For example, if we put an image of a car through a high-pass filter, we expect the edges of the car, where the pixel values change abruptly from light to dark, to be detected. The edges of objects are often areas of abrupt intensity change and, for this reason, high-pass filters are sometimes called “<strong>edge detection filters.</strong>”</p><h3>Convolutional Kernels</h3><p>The filters we will be talking about are in the form of matrices, so-called convolutional kernels, which are just grids of numbers that modify an image. Below there is an example of a high-pass filter, a 3x3 kernel which does edge detection.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/362/0*x_Pyr-aUL_6-Gt-U.png" /></figure><p>You may notice that all the elements in this 3x3 grid sum to zero. For an edge detection filter, all of its elements must sum to zero because<strong> it’s computing the difference or change between neighboring pixels</strong>. If these pixel weights did not add up to zero, then the calculated difference would be either <strong>positively</strong> or <strong>negatively</strong> weighted, which has the effect of brightening or darkening the entire filtered image, respectively.</p><h3>Convolution</h3><p>To apply this filter to an image, an input image, F(x,y), is convolved with the kernel, K. Convolution is represented by an asterisk (<strong>not to be mistaken for multiplication</strong>). It involves taking a kernel, which is our small grid of numbers, and passing it over an image, pixel-by-pixel, creating another edge-detected output image whose appearance depends on the kernel values.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*-aJxR7klOwxl08kT.gif" /></figure><p>We’ll walk through a specific example, using the 3x3 edge detection filter. To better see the pixel-level operations, I’ll zoom in on the image of the car.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*c45tDcjf9Nj8FluL.png" /></figure><p>For every pixel in this grayscale image, we place our kernel over it, so that the selected pixel is at the center of the kernel, and perform convolution. In the below image, I’m selecting a center pixel with a value of 200, as an example.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/854/0*EFN4-ULQGNs5f7LY.png" /></figure><p>The steps for a complete convolution are as follows:</p><ol><li>Multiply the values in the kernel with their corresponding pixel value. So, the value in the top left of the 3x3 kernel (0), will be multiplied by the pixel value in that <strong><em>same</em></strong> corner in our image area (150).</li><li>Sum all these multiplied pairs of values to get a new value, in this case, 175. This value will be the new pixel value in the filtered output image, at the same (x,y) location as the selected center pixel.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*KQaygHBaWwLT2MXG.gif" /></figure><p>This process repeats for every pixel in the input image, until we are left with a complete, filtered output.</p><h3>Image Borders</h3><p>The only pixels for which convolution doesn’t work are the pixels at the borders of an image. A 3x3 kernel cannot be perfectly placed over a center pixel, at the edges or the corners of an image. In practice, there are a few ways of dealing with this, the most common ways are to either <strong>pad this image with a border of 0’s </strong>(or another grayscale value) so that we can perfectly overlay a kernel on all the pixel values in the original image, or to ignore the pixel values at the borders of the image, and only look at pixels where we can completely overlay the 3x3 convolutional kernel.</p><p>Often, there is not a lot of useful information at the border of an image, but if you do choose to ignore this information, every filtered image will be just a little bit smaller than the original input image. For a 3x3 kernel, we’ll lose a pixel from each side of an image, resulting in a filtered output that is two pixels smaller in width and height than the original image. You can also choose to make larger filters. 3x3 is one of the smallest sizes and it’s good for looking at small pixel areas, but if you are analyzing larger images you may want to increase the area and use kernels that are 5x5, 7x7, or larger. Usually, you want an odd number so that the kernel nicely overlays a center pixel.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/974/0*SvvjuXXB3aAQLo2B.png" /></figure><p>Above is another type of edge detection filter; this filter is computing the difference around a center pixel but it’s only looking at the bottom row and top row of surrounding pixel values. The result is a horizontal edge detector. In this way, you can create oriented edge detectors!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3QRSuj-7V8TJBgmv.png" /></figure><p>So, these convolutional kernels, when applied to an image, make a filtered image, and <em>several</em> filtered images make a convolutional layer.</p><h3>Activation Function</h3><p>Recall that grayscale images have pixels values that fall in a range from 0–255. However, neural networks work best with scaled “strength” values between 0 and 1 (we briefly mentioned this in the last post). So, in practice, the input image to a CNN is a grayscale image with pixel values between 0 (black) and 1 (white); a light grey may be a value like 0.78. Converting an image from a pixel value range of 0–255 to a range of 0–1 is called <strong>normalization</strong>. Then, this normalized input image is filtered and a convolutional layer is created. Every pixel value in a filtered image, created by a convolution operation, will fall in a different range; there may even be negative pixel values.</p><p>To account for this change in pixel value, following a convolutional layer, CNN applies an <strong>activation function</strong> that transforms each pixel value.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*L1C0HCSvCS_E3F91.png" /></figure><p>In a CNN, you’ll often use a ReLu (Rectified Linear Unit) activation function; this function simply turns all negative pixel values into 0’s (black). For an input, x, the ReLU function returns x for all values of x &gt; 0, and returns 0 for all values of x ≤ 0. An activation function also introduces nonlinearity into a model, and this means that CNN will be able to find nonlinear thresholds/boundaries that effectively separate and classify some training data.</p><h3>Max pooling layer</h3><p>After a convolutional layer comes to a pooling layer; the most common type of pooling layer is a <em>max-pooling</em> layer. Each of these layers looks at the pixel values in an image, so, to describe max-pooling, I’ll focus on a small pixel area. First, a max-pooling operation breaks an image into smaller patches, often 2x2 pixel areas.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/0*8kfyZftRb1hAQfVO.png" /><figcaption>Zooming in even further on four of these patches</figcaption></figure><p>A max-pooling layer is defined by the patch size, 2x2, and stride. The patch can be thought of as a 2x2 window that the max-pooling layer looks at to select a maximum pixel value. It then moves this window by some stride across and down the image. For a patch of size 2x2 and a stride of 2, this window will perfectly cover the image. A smaller stride would see some overlap in patches and a larger stride would miss some pixels entirely. So, we usually see a patch size and a stride size that are the same.</p><p>For each 2x2 patch, a max-pooling layer looks at each value in a patch and selects only the maximum value. In the red patch, it selects 140, in the yellow, 90, and so on, until we are left with four values from the four patches.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/874/0*6PHcOT5NFsHVq6O5.png" /></figure><p>Now, we might be wondering why we would use a max-pooling layer in the first place, especially since this layer is discarding pixel information. We use a max-pooling layer for a few reasons.</p><p>First, <strong>dimensionality reduction</strong>; as an input image moves forward through a CNN, we are taking a fairly flat image in x-y space and expanding its depth dimension while decreasing its height and width. The network distills information about the content of an image and squishes it into a representation that will make up a reasonable number of inputs that can be seen by a fully-connected layer.</p><p>Second, max-pooling makes a network resistant to small pixel value changes in an input image. Imagine that some of the pixel values in a small patch are a little bit brighter or darker or that an object has moved to the right by a few pixels. For similar images, even if a patch has some slightly different pixel values, the maximum values extracted in successive pooling layers, should be similar.</p><p>Third, by reducing the width and height of image data as it moves forward through the CNN, the max-pooling layer <strong>mimics an increase in the field of view</strong> for later layers. For example, a 3x3 kernel placed over an original input image will see a 3x3 pixel area at once, but that same kernel, applied to a pooled version of the original input image (ex. an image reduced in width and height by a factor of 2), will see the same number of pixels, but the 3x3 area corresponds to a 2x larger area in the <em>original</em> input image. This allows later convolutional layers to detect features in a larger region of the input image.</p><h3>Fully-connected Layer</h3><p>At the end of a convolutional neural network, is a fully-connected layer (sometimes more than one). Fully-connected means that every output that’s produced at the end of the last pooling layer is input to each node in this fully-connected layer. For example, for a final pooling layer that produces a stack of outputs that are 20 pixels in height and width and 10 pixels in-depth (the number of filtered images), the fully-connected layer will see 20x20x10 = 4000 inputs. The role of the last fully-connected layer is to produce a list of <strong>class scores</strong>, and perform classification based on image features that have been extracted by the earlier convolutional and pooling layers; so, the last fully-connected layer will have as many nodes as there are classes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*LOkUTKqOiI_E2bRg.png" /></figure><h3>Summary</h3><p>CNN’s are made of many layers: a series of <strong>convolutional layers</strong> + <strong>activation</strong> and <strong>max-pooling layers</strong>, and at least one, the final fully-connected layer that can produce a set of class scores for a given image. The convolutional layers of a CNN act as feature extractors; they extract shape and color patterns from the pixel values of training images. <strong>It’s important to note that the behavior of the convolutional layers, and the features they learn to extract, are defined entirely by the weights that make up the convolutional kernels in the network</strong>. A CNN learns to find the best weights during training using a process called<strong> backpropagation</strong>, which looks at any classification errors that a CNN makes during training, finds which weights in that CNN are responsible for that error, and changes those weights accordingly.</p><h3>PyTorch Code Example</h3><p>If you’d like to see how to create these kinds of CNN layers using PyTorch, take a look at my Github, <a href="https://github.com/cipheraxat/Deep-Learning-Portfolio/tree/master/intro-to-computervision"><strong>tutorial repository</strong></a>. You may choose to skim the code and look at the output or set up a local environment and run the code on your computer (instructions for setting up a local environment are documented in the repository readme).</p><p>If you liked the tutorial, <strong>do clap</strong> and share among your friends. Follow me on <a href="https://www.linkedin.com/in/akshatanand1999/"><strong>LinkedIn</strong></a> and on <a href="https://github.com/cipheraxat"><strong>Github</strong></a> as well.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f3aaa8d79891" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-community-srm/introduction-to-cnn-f3aaa8d79891">Introduction to CNN</a> was originally published in <a href="https://medium.com/data-science-community-srm">DataX Journal</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Whatsapp Automation with Selenium]]></title>
            <link>https://medium.com/data-science-community-srm/whatsapp-automation-with-selenium-a4fbaff625a0?source=rss-f064ddbf5711------2</link>
            <guid isPermaLink="false">https://medium.com/p/a4fbaff625a0</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[selenium]]></category>
            <category><![CDATA[whatsapp]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[automation]]></category>
            <dc:creator><![CDATA[Akshat Anand]]></dc:creator>
            <pubDate>Tue, 12 May 2020 18:45:00 GMT</pubDate>
            <atom:updated>2020-05-12T18:45:00.463Z</atom:updated>
            <content:encoded><![CDATA[<h4>Send messages to multiple people with just a few lines of Python Code</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*llOQnQMmTKOEk071RCdtKQ.jpeg" /></figure><p>Do you ever wonder sending messages automatically by just running a simple python script? Yes, you heard it right, it can be done easily by using a package like selenium.</p><p>Selenium is a very smart package in python with which developers can automate the browser’s activity. With this, we can make use of Whatsapp-web through the browser and use it to automate message transfers.</p><h3>Basic Requirement-</h3><ul><li>Library/Packages — Selenium , Chromedriver(for web automation).</li><li>IDE — Jupyter Notebook.</li><li>Browser — Opensource Chromium web browser.</li></ul><h3>Procedure-</h3><p><strong>Step 1: Installing Selenium-</strong></p><p>Selenium is used as browser Automation software. It can be installed using the following code on the terminal-</p><pre>$pip install selenium</pre><p><strong>Step 2: Downloading Webdriver-</strong></p><p>We will be using the Chrome web driver for automation. Download the chrome web driver from this <a href="https://chromedriver.storage.googleapis.com/index.html?path=2.25/"><strong>link</strong></a> ( download the appropriate version as per your PC).</p><p>Extract the given file to the given location, as <strong>we will need the location later.</strong></p><p>You can refer to this <a href="https://sites.google.com/a/chromium.org/chromedriver/"><strong>documentation</strong> </a>for any doubts in downloading and setting it up.</p><p><strong>Step 3: Install the open-source Chromium Web Browser-</strong></p><p>Write the following code on the terminal to install the web browser.</p><pre>$sudo apt-get install chromium-browser</pre><p><strong>Github Repo for reference</strong> — <a href="https://github.com/cipheraxat/Whatsapp-Automation"><strong>Link</strong></a></p><p>Now its time for real fun let us write code for -</p><p>The below code helps us to send Whatsapp text messages to the target user.</p><pre>#importing all the necessary libaries</pre><pre>from selenium import webdriver<br>from selenium.webdriver.support import expected_conditions as ec<br>from selenium.webdriver.support.ui import WebDriverWait<br>from selenium.webdriver.common.keys import Keys<br>from selenium.webdriver.common.by import By</pre><pre>import time</pre><pre>browser = webdriver.Chrome(&#39;/usr/bin/chromedriver&#39;)<br>browser.get(&#39;https://web.whatsapp.com/&#39;)<br>wait = WebDriverWait(browser, 600)</pre><pre>target = &#39;&quot;Rohan mathur&quot;&#39; #enter contact name here<br>string = &quot;Message by python!&quot; #target msg<br>x_arg = &#39; //span[contains(@title, &#39; + target +&#39;)]&#39;<br>target = wait.until(ec.presence_of_element_located((By.XPATH, x_arg)))</pre><pre>target.click()</pre><p>The above code imports the specific libraries. We use the browser as the object which uses the functionality of the chrome web driver and ‘<strong>/usr/bin/chromedriver’</strong> is the location of the web driver,</p><p>The wait object does the chrome driver to wait for 600ms. We use this for the web driver to load until the open the chrome driver opens for the automation.</p><p>The target variable assigns the name of the contact we are sending it and the string the content of the message. Now, x_arg is helped to find the particular target contact on the search box on WhatsApp web and at last, it did with function click() to maneuver to the particular contact.</p><pre>input_box = browser.find_element_by_class_name(&#39;_1Plpp&#39;)</pre><pre>for i in range(100):#loops runs for 100 times<br>    input_box.send_keys(string + Keys.ENTER)</pre><p>The input_box finds the text box element (which can be found by right-clicking the mouse button and selecting the particular div class). Finally, we run a loop of a particular number for sending to the target.</p><p>Now when the web driver opens scan the <strong>QR code</strong>…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Kdlzg4lfN6mXlGKRSFqZjw.png" /></figure><p>Open the WhatsApp mobile and click on the <strong>WhatsApp web</strong> and scan the QR code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/259/1*h9UOQ8V_QqhFRLqKOWDbfA.jpeg" /></figure><p>And there you go.</p><p>If you want to send <strong>images</strong> through automation,</p><pre>file_path = &quot;/home/akshat/Downloads/akshat.jpg&quot;#file path</pre><p>This gives the <strong>image/file pathname</strong>.</p><pre>from time import sleep#sending image to whatsapp<br>attachment_section = browser.find_element_by_xpath(&#39;//div[<a href="http://twitter.com/title">@title</a> = &quot;Attach&quot;]&#39;)<br>attachment_section.click()</pre><pre>image_box = browser.find_element_by_xpath(&#39;//input[<a href="http://twitter.com/accept">@accept</a>=&quot;image/*,video/mp4,video/3gpp,video/quicktime&quot;]&#39;)<br>image_box.send_keys(file_path)</pre><pre>sleep(3)</pre><pre>send_button = browser.find_element_by_xpath(&#39;//span[<a href="http://twitter.com/data">@data</a>-icon=&quot;send-light&quot;]&#39;)<br>send_button.click()</pre><p>And here you can send the image to the <strong>target contact</strong>.</p><p>Ps. Please use this script only for educational purposes, I am not responsible if your friends ( or even Whatsapp ) may <strong>block you</strong>.</p><p><strong>Comment</strong> below about your experience!</p><p>And <strong>do clap</strong> and follow our publication if you like the content. Peace!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a4fbaff625a0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-community-srm/whatsapp-automation-with-selenium-a4fbaff625a0">Whatsapp Automation with Selenium</a> was originally published in <a href="https://medium.com/data-science-community-srm">DataX Journal</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Statistical Power and Power Analysis]]></title>
            <link>https://medium.com/data-science-community-srm/statistical-power-and-power-analysis-98cf4e10b064?source=rss-f064ddbf5711------2</link>
            <guid isPermaLink="false">https://medium.com/p/98cf4e10b064</guid>
            <category><![CDATA[probability]]></category>
            <category><![CDATA[analysis]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[power-analysis]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Akshat Anand]]></dc:creator>
            <pubDate>Sun, 12 Apr 2020 01:28:52 GMT</pubDate>
            <atom:updated>2020-04-16T03:22:38.201Z</atom:updated>
            <content:encoded><![CDATA[<h4>Introduction to key statistical concepts such as effect size, statistical power, significance level, and sample size.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jidbi1--elimaTb3B2HT5w.jpeg" /></figure><p>The statistical power by the <strong>hypothesis test</strong> is referred to as the probability of measuring or detecting an effect, only if there are any true effect to be present to detect.</p><p><strong>Power</strong> is the confidence one might have in the conclusion which can be drawn from the results of the given study. It can be used as a tool to estimate the sample size or the number of observations in order to measure an effect in an experiment.</p><blockquote>In this blog we will go through:</blockquote><p>1, A power analysis that is used to estimate the minimum number of sample sizes required for an experiment from the desired significance level, effect size, and statistical power.</p><p>2. The statistical power is the term where the probability of the hypothesis test of finding an effect if there any effect is found.</p><p>3.Lastly, how we can calculate and plot the power analysis in python in order to design an experiment.</p><p><strong><em>Hypothesis Testing(Statistical Hypothesis Testing)-</em></strong></p><p>A hypothesis test makes an assumption about an outcome, which is termed as the null hypothesis. This is often interpreted using a p-value, the probability of observing that the null hypothesis is true or not.</p><p><strong>p-value(p)</strong>- The probability of obtaining a result equal to or more extreme than was observed in the data.</p><p>While interpreting the p-value of a significance test, we can specify a significance level(referred to as alpha(a)). A general value for significance level is 5% and written as 0.05.</p><p>In fact, the p-value is interested in the context of the chosen significance level. This makes the significance test as “<strong>Statistically significant</strong>” if the p-value is less than the significant level which means the null hypothesis is neglected.</p><ul><li><strong>p≤alpha</strong>: (reject h0, different distribution)</li><li><strong>p&gt;alpha</strong>:(doesn’t reject h0, same distribution)</li></ul><p>And,</p><ul><li><strong>Significance level(alpha)</strong>: Boundary for measuring a statistically significant finding while interpreting the p-value.</li></ul><p>Hence we see that p-value is a probability and that can be actually different as the test could go wrong. There can be errors in our interpretations.</p><p>So, there are two types of error:</p><ul><li><strong>Type I error</strong>: It rejects the null hypothesis when there is no significant effect(which we call false positive). Here the p-value is small.</li><li><strong>Type II error</strong>: It doesn’t reject the null hypothesis when there is a significant effect(called as false negative). Here the p-value is large.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DCCmDV7qcOlbVgbPq-E6ZA.png" /><figcaption>source-Reddit</figcaption></figure><p>In this context, we can think of the significance level as the probability of rejecting the null hypothesis if it were true i.e. the probability of making a false positive</p><p><strong><em>Statistical Power</em></strong></p><p>Statistical power or the power of a hypothesis test is a probability that test correctly rejects the null hypothesis i.e. the given probability of a true positive result. It is useful only if when the null hypothesis is eventually rejected.</p><p>The higher the statistical power for an experiment, the lower the probability of making a Type II (false negative) error. That is the higher the probability of detecting an effect when there is an effect.</p><p>Power = 1 — Type II Error</p><p>Pr(True Positive) = 1 — Pr(False Negative)</p><p>While interpreting statistical power, we seek setups that have high statistical power.</p><ul><li><strong>Low Statistical Power</strong>: Large risk of committing Type II errors, e.g. a false negative.</li><li><strong>High Statistical Power</strong>: Small risk of committing Type II errors.</li></ul><p>When the experimental results with having low statistical power will eventually lead to invalid conclusions about the meaning insights</p><h3>Power Analysis</h3><p>Generally, statistical power is related to four parts.</p><ul><li><strong>Effect Size</strong>. The quantified magnitude of a result present in the population. Effect size is calculated using a specific statistical measure, such as Pearson’s correlation coefficient for the relationship between variables.</li><li><strong>Sample Size</strong>. The number of observations in the sample.</li><li><strong>Significance</strong>. The significance level used in the statistical test, e.g. alpha. Often set to 5% or 0.05.</li><li><strong>Statistical Power</strong>. The probability of accepting the alternative hypothesis if it is true.</li></ul><p>A power analysis involves estimating one of these four parameters given values for three other parameters. This is a powerful tool in both the design and in the analysis of experiments that we wish to interpret using statistical hypothesis tests.</p><p>As a beginner, we can start with sensible defaults for some parameters, such as a significance level of 0.05 and a power level of 0.80. We can then estimate a desirable minimum effect size, specific to the experiment being performed. A power analysis can then be used to estimate the minimum sample size required.</p><p>In addition, multiple power analyses can be performed to provide a curve of one parameter against another, such as the change in the size of an effect in an experiment given changes to the sample size. More elaborate plots can be created varying three of the parameters. This is a useful tool for experimental design.</p><h3>Application with python</h3><p>In this section, we will look at the <a href="https://en.wikipedia.org/wiki/Student%27s_t-test">Student’s t-test</a>, which is a statistical hypothesis test for comparing the means from two samples of Gaussian variables.</p><p>The test will calculate a <strong>p-value</strong> that can be interpreted as to whether the samples are the same (fail to reject the null hypothesis), or there is a statistically significant difference between the samples (reject the null hypothesis). The common significance level for interpreting the p-value is 5% or 0.05.</p><p><strong>Significance level (alpha)</strong>: 5% or 0.05.</p><p>The size of the effect of comparing two groups can be quantified with an effect size measure. A common measure for comparing the difference in the mean from two groups is the Cohen’s d measure.</p><p><strong>Effect Size</strong>: Cohen’s d of at least 0.80.</p><p>We can use the default and assume a minimum statistical power of 80% or 0.8.</p><p><strong>Statistical Power</strong>: 80% or 0.80.</p><p>For a given experiment with these defaults, we may be interested in estimating suitable sample size. That is, how many observations are required from each sample to at least detect an effect of 0.80 with an 80% chance of detecting the effect if it is true (20% of a Type II error) and a 5% chance of detecting an effect if there is no such effect (Type I error).</p><p>We can solve this using a power analysis.</p><p>The statsmodels library provides the <a href="http://www.statsmodels.org/dev/generated/statsmodels.stats.power.TTestIndPower.html">TTestIndPower</a> class for calculating a power analysis for the Student’s t-test with independent samples. Of note is the <a href="https://www.statsmodels.org/dev/generated/statsmodels.stats.power.TTestPower.html">TTestPower</a> class that can perform the same analysis for the paired Student’s t-test.</p><p>The TTestIndPower instance must be created, then we can call the <strong><em>solve_power()</em></strong> with our arguments to estimate the sample size for the experiment.</p><pre>#perform power analysis<br>analysis = TTestIndPower()<br>result = analysis.solve_power(effect, power=power, nobs1=None, ratio=1.0, alpha=alpha)</pre><p>The complete example is listed below.</p><pre># estimate sample size via power analysis<br>from statsmodels.stats.power import TTestIndPower<br># parameters for power analysis<br>effect = 0.8<br>alpha = 0.05<br>power = 0.8<br># perform power analysis<br>analysis = TTestIndPower()<br>result = analysis.solve_power(effect, power=power, nobs1=None, ratio=1.0, alpha=alpha)<br>print(&#39;Sample Size: %.3f&#39; % result)</pre><p>Running the example calculates and prints the estimated number of samples for the experiment as 25. This would be a suggested minimum number of samples required to see an effect of the desired size.</p><p>Power curves are line plots that show how the change in variables, such as effect size and sample size, impact the power of the statistical test.</p><p>The <a href="http://www.statsmodels.org/dev/generated/statsmodels.stats.power.TTestIndPower.plot_power.html"><strong>plot_power() function</strong></a> can be used to create power curves. The dependent variable (x-axis) must be specified by name in the ‘<em>dep_var</em>‘ argument. Arrays of values can then be specified for the sample size (<em>nobs</em>), effect size (<em>effect_size</em>), and significance (<em>alpha</em>) parameters.</p><pre># calculate power curves from multiple power analyses<br>analysis = TTestIndPower()<br>analysis.plot_power(dep_var=&#39;nobs&#39;, nobs=arange(5, 100), effect_size=array([0.2, 0.5, 0.8]))</pre><p>The complete example is listed below.</p><pre># calculate power curves for varying sample and effect size<br>from numpy import array<br>from matplotlib import pyplot<br>from statsmodels.stats.power import TTestIndPower<br># parameters for power analysis<br>effect_sizes = array([0.2, 0.5, 0.8])<br>sample_sizes = array(range(5, 100))<br># calculate power curves from multiple power analyses<br>analysis = TTestIndPower()<br>analysis.plot_power(dep_var=&#39;nobs&#39;, nobs=sample_sizes, effect_size=effect_sizes)<br>pyplot.show()</pre><p>Running the example creates the plot showing the impact on statistical power (y-axis) for three different effect sizes (<em>es</em>) as the sample size (x-axis) is increased.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Sp4f3FB9rLIXmCQGBZ980Q.png" /></figure><p>We can see that if we are interested in a large effect that a point of diminishing returns in terms of statistical power occurs at around 40-to-50 observations.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=98cf4e10b064" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-community-srm/statistical-power-and-power-analysis-98cf4e10b064">Statistical Power and Power Analysis</a> was originally published in <a href="https://medium.com/data-science-community-srm">DataX Journal</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How blockchain can help the music industry to grow?]]></title>
            <link>https://its-axat.medium.com/how-blockchain-can-help-the-music-industry-to-grow-84a07a61401f?source=rss-f064ddbf5711------2</link>
            <guid isPermaLink="false">https://medium.com/p/84a07a61401f</guid>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[music]]></category>
            <dc:creator><![CDATA[Akshat Anand]]></dc:creator>
            <pubDate>Mon, 28 Oct 2019 08:07:51 GMT</pubDate>
            <atom:updated>2019-10-28T08:07:51.509Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*f2aLWW1QTl0GTNmQSeLDqQ.jpeg" /></figure><p>Blockchain is revolutionizing the way companies handle digital data, contracts, money, and transactions. For the music business, streamlining these areas could completely transform the relationship between artists, labels, managers, publishing companies, and streaming services — a relationship which has been somewhat rocky over the last few years.</p><p>From Spotify being sued in 2015 for $150 million by a group of artists who claimed that the service had reproduced and distributed their music without permission to Tidal over inflating streaming numbers in 2018, the lack of transparency across the board seems to be a major issue that is holding the industry back. The good news is, emerging technologies are providing new innovative solutions to clear the murky waters of the business. And many labels, both public and privately owned, are listening and adopting.</p><p><strong>Transparent Royalty Payouts</strong></p><p>Currently, the global music industry has an estimated $2.5 billion worth of uncollected royalties. Dubbed the “royalty black box”, there have been more than 46 million instances of unidentified songwriters or unknown copyright owner notice of intents (NOIs) filed with the US Copyright Office by streaming services since April 2016. Why is this? Well, there may be many variables, but it usually starts with the accuracy of metadata. For those who don’t know, record companies, publishers, and streaming service providers have their own separate databases to keep track of who owns the rights to the music and who is owed what money. But none of these databases is connected or is advanced enough to talk to each other. The lack of interoperability combined with the complexity of the way music is being both created and consumed globally, has created an issue where hundreds of millions of dollars are not sent to their rightful owners. A solution to this issue is to create a blockchain-centric distribution model that allows metadata to interact with different systems. Closing the gap between metadata errors and artist payout periods will be collectively important for the music business to move forward.</p><p><strong>Reshape the Way Artists and Labels Think About Advances</strong></p><p>An “advance” is almost always defined as a pre-payment of royalties, whether the advance is paid by a record company to an artist, a publisher to a writer or a merchandiser to an artist. In almost every situation, advances have always come with two negative aspects. First, artists are always on the hook to recoup the advance from dollar one, and second, the financier (typically a label) always takes on a majority of the overhead risk. In today’s business, both labels and artists need to put aside their former understanding of advances and think about them through the lens of blockchain technology. If deployed correctly, blockchain could distribute the risk of advances among numerous stakeholders, resulting in a more favorable sharing of royalties and less up front risk from the financier. Innovations like this could fundamentally shift the way artists think about, and even go about funding their projects and entertainment endeavors.</p><p><strong>New Artist Monetization Models</strong></p><p>Well, according to a report from Information Is Beautiful, it would take on average about 1,117,021 streams a month if you are a signed artist (assuming the label takes a cut 30 per cent cut) and about 180,000 streams if you are an independent artist. Now, add in monthly recurring expenses an artist sees per month such as the cost of making music, touring, and marketing. An artist would likely have to gross more streams than what was mentioned above to feel “comfortable”. Instead of focusing solely on traditional revenue models, artists and their teams need to consider incorporating blockchain technology to develop new revenue pipeline models.</p><p>One area where there is massive opportunity lies in the use of cryptocurrencies. By creating a cryptocurrency that ties to a specific song or album, artists could create a virtual “stock market” where fans could buy into pieces of the song’s rights, while receiving money accrued from the streams. This compound revenue model would be handled by the security of smart contracts built within blockchain technology. Solutions like this are creating new opportunities for artists to monetize their creativity.</p><p>Overall, the music business must scale blockchain technologies if the industry wants to create a more transparent playing field for the creatives of the business.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=84a07a61401f" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>