<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Murli Jadhav on Medium]]></title>
        <description><![CDATA[Stories by Murli Jadhav on Medium]]></description>
        <link>https://medium.com/@murlijadhav007?source=rss-e8254109f8ba------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*YwbuupaCp2V3xpm5</url>
            <title>Stories by Murli Jadhav on Medium</title>
            <link>https://medium.com/@murlijadhav007?source=rss-e8254109f8ba------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 05:45:08 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@murlijadhav007/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[OpenCv For Beginners (Part-I)]]></title>
            <link>https://medium.com/@murlijadhav007/opencv-for-begginers-part-i-af1a32180a9a?source=rss-e8254109f8ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/af1a32180a9a</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[opencv]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[image-processing]]></category>
            <dc:creator><![CDATA[Murli Jadhav]]></dc:creator>
            <pubDate>Tue, 25 Feb 2020 14:23:56 GMT</pubDate>
            <atom:updated>2020-03-10T14:19:21.365Z</atom:updated>
            <content:encoded><![CDATA[<p>Here we are covering the basics to advance of opencv for beginners. In last of the tutorial series we also creating a basic sketch mini project.</p><blockquote>Here we are covering Reading and writing image,GrayScaling ColorSpaces, Drawing Images, Sharpening, Thresholding, Binarisation of Images</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*rj4DmlbX2G4Aq_TfuvnmJg.png" /><figcaption>Image Source= <a href="https://www.pyimagesearch.com/wp-content/uploads/2017/08/faster_for_loop_header.png">Here</a></figcaption></figure><p><strong>What is OpenCV and Why we need it ?</strong></p><p>OpenCv is founded in Intel labs in 1999 by <strong>Gary Bradsky. </strong>The<strong> </strong>First launched in 2000. The OpenCv developed in c++. Now a days we are using the wrapper of python on the top of c++ implementation. OpenCv supports the wide variety of programming languages like java,python. The more information of Opencv found <a href="https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_setup/py_intro/py_intro.html"><em>here</em></a></p><p><strong>Installing OpenCv using Python :-</strong></p><p>we install openCv same as we are installing any libraries in python. The command is</p><pre>!sudo pip install opencv-python</pre><pre>#For Importing use<br>import cv2</pre><pre>#For Displaying the version of OpenCv<br>print (cv2.__version__)</pre><p><strong>Reading and Writing the Images :-</strong></p><p>Here we are reading the images from the local drive. For that use following commands</p><pre>import cv2</pre><pre>#Loading the images from the given path<br>path = cv2.imread(&#39;./images/input.jpg&#39;)</pre><pre>#syntax=imshow(Title_of_image , path)<br>cv2.imshow(&#39;Hello World&#39;, input)</pre><pre>#Closing the open window<br>#here when you click any key then it closes the opened window<br>cv2.waitKey()</pre><pre>#Following command closes all the opened windows<br>cv2.destroyAllWindows()</pre><p><strong>Reading the dimensions and depth of images:-</strong></p><pre>print(input_image.shape)<br>output = (512, 256, 3)<br>#from above output we can see that 512 pixels is a width and 256 pixels is height and 3 is depth(RGB) </pre><p><strong>GrayScaling of the Image :-</strong></p><p>here we are using the cv2.cvtColor() for converting the colour images to grayscale image. For more information about cvtColor() click <a href="https://docs.opencv.org/2.4/modules/imgproc/doc/miscellaneous_transformations.html"><em>here</em></a></p><pre>import cv2</pre><pre># Load input image<br>image = cv2.imread(‘./images/input.jpg’)<br>cv2.imshow(‘Original’, image)<br>cv2.waitKey()</pre><pre># Converting to gray scale<br>converted_im = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)</pre><pre>cv2.imshow(‘Grayscale’, converted_image)<br>cv2.waitKey()<br>cv2.destroyAllWindows()</pre><h3>Introduction to color spaces</h3><p>Mostly color spaces are used for the extracting the RGB values from an image pixels. If you want to perform the color operation or want values of color you will get it using color spaces</p><pre>input_image = cv2.imread(‘input_1.jpg’)<br>#Showing the image<br>cv2.imshow(‘Taj Mahal Image’, input_image)<br>cv2.waitKey()</pre><pre># BGR Values for the first 0,0 pixel<br>B, G, R = input_image[0, 0] <br>print(B, G, R)<br>print(input_image.shape)</pre><pre>cv2.destroyAllWindows()</pre><h3>Drawing on empty image :-</h3><p>If you want to do some operations on image first draw the empty blank image and draw the shapes on it.</p><pre># Creating an empty image<br>image = np.zeros((512,512,3), np.uint8)</pre><pre>#Drawing line on empty image<br>cv2.line(image, (0,0), (511,511), (255,127,0), 5)<br>cv2.imshow(“Blue Line”, image)</pre><pre>cv2.waitKey(0)<br>cv2.destroyAllWindows()</pre><p>For Drawing the circle :-</p><pre>image = np.zeros((512,512,3), np.uint8)</pre><pre>#Drawing Circle on Empty Image<br>cv2.circle(image, (350, 350), 100, (15,75,50), -1) <br>cv2.imshow(&quot;Circle&quot;, image)<br>cv2.waitKey(0)<br>cv2.destroyAllWindows()</pre><h3>Sharpening the image :-</h3><p>If you want to sharpening the image then first you have to create a convolutional matrix. The convolutional matrix is a matrix which contains the values for the performing some operations like sharpening, blurring. If we do element wise multiplication of conv. values and image pixels then we get desired results</p><pre>import cv2<br>import numpy as np</pre><pre>image = cv2.imread(‘input_1.jpg’)<br>cv2.imshow(‘Original’, image)</pre><pre># Create our shapening kernel, we don’t normalize since the <br># the values in the matrix sum to 1<br>kernel_sharpening = np.array([[-1,-1,-1], <br> [-1,9,-1], <br> [-1,-1,-1]])</pre><pre># applying different kernels to the input image<br>sharpened = cv2.filter2D(image, -1, kernel_sharpening)</pre><pre>cv2.imshow(‘Image Sharpening’, sharpened)</pre><pre>cv2.waitKey(0)<br>cv2.destroyAllWindows()</pre><h3>Edge Detection:-</h3><pre>import cv2<br>import numpy as np</pre><pre>#reading the image<br>image = cv2.imread(‘input_1.jpg’,0)</pre><pre>height, width = image.shape</pre><pre># Extract Sobel Edges<br>sobel_x = cv2.Sobel(image, cv2.CV_64F, 0, 1, ksize=5)<br>sobel_y = cv2.Sobel(image, cv2.CV_64F, 1, 0, ksize=5)</pre><pre>cv2.imshow(‘Original’, image)<br>cv2.waitKey(0)<br>cv2.imshow(‘Sobel X’, sobel_x)<br>cv2.waitKey(0)<br>cv2.imshow(‘Sobel Y’, sobel_y)<br>cv2.waitKey(0)</pre><pre>#Printing the sobel edges for both x and y axis<br>sobel_OR = cv2.bitwise_or(sobel_x, sobel_y)<br>cv2.imshow(‘sobel_OR’, sobel_OR)<br>cv2.waitKey(0)</pre><pre>laplacian = cv2.Laplacian(image, cv2.CV_64F)<br>cv2.imshow(‘Laplacian’, laplacian)<br>cv2.waitKey(0)</pre><pre>#Detecting the canny edges<br>canny = cv2.Canny(image,50, 150)<br>cv2.imshow(‘Canny’, canny)<br>cv2.waitKey(0)</pre><pre>canny = cv2.Canny(image,50, 150)<br>cv2.imshow(‘Canny’, canny)<br>cv2.waitKey(0)</pre><pre>cv2.destroyAllWindows()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/532/1*m9LUdrAUTIw3Tt0Ec5UTRw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/524/1*tM-g3YGlixm0b72EZ09mRQ.png" /><figcaption>(A)Taj Mahal Gray Scale Image (B) Taj Mahal After Detecting the canny edges</figcaption></figure><h3>Mini-Project (Live Sketch):-</h3><p>Please refer my github file link for the all the code and the project</p><p>Link :- Click <a href="https://github.com/MurliJadhav/OpenCV-Codes-All">HERE</a></p><p>Also Check my Part-2 of this tutorial bellow</p><p>— — — — — — -Part-2 Soon — — — —</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=af1a32180a9a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Auto Encoders : De-Noising Text Documents]]></title>
            <link>https://medium.com/analytics-vidhya/auto-encoders-de-noising-text-documents-c58d6950dfad?source=rss-e8254109f8ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/c58d6950dfad</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[tensorflow]]></category>
            <dc:creator><![CDATA[Murli Jadhav]]></dc:creator>
            <pubDate>Fri, 31 Jan 2020 16:37:03 GMT</pubDate>
            <atom:updated>2020-05-15T17:09:49.304Z</atom:updated>
            <content:encoded><![CDATA[<h3>Auto Encoders: De-Noising Text Documents</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/0*XeWl7ip-VXp4QXov.png" /><figcaption>Fig:-Auto encoder structure (fig-<a href="https://www.researchgate.net/profile/Xifeng_Guo/publication/320658590/figure/fig1/AS:614154637418504@1523437284408/The-structure-of-proposed-Convolutional-AutoEncoders-CAE-for-MNIST-In-the-middle-there.png">Here</a>)</figcaption></figure><p><strong>What is Auto-Encoders?</strong></p><p>Auto-Encoders is unsupervised learning method.The idea behind the auto encoder is compressing the data in the latent space and then again de-compress the data from latent space for recreating the inputs.</p><p><strong>What is encoders?</strong></p><p>Encoders is a part of neural network which compress the inputs in latent space representation.</p><p><strong>What is decoders?</strong></p><p>Decoders is useful for reconstructing the input from latent space representation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0uTpozegFTpY0VDNAFgZEg.png" /><figcaption>Fig- Encoder and Decoder</figcaption></figure><p>Auto-encoders also useful for dimensional reduction like PCA. If auto-encoders work as PCA then why we need auto-encoders? The auto-encoders is very useful for learning the most silent features.In latent space representation contains very less dimensions than inputs that’s why the auto-encoders learn useful information</p><p><strong>Applications of Auto-Encoders :-</strong></p><ol><li>Image De-Noising</li><li>Dimensionality-Reduction</li><li>Feature Variation</li><li>Image Colourising</li><li>Water mark removal</li></ol><h3>Auto-Encoders for De-Noising Text :-</h3><p>Here we are implementing auto encoders for denoising the text documents. the text document is as follows :-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/375/1*qVkBF75k69fUFel50twjRA.png" /><figcaption>Fig- noisy text document</figcaption></figure><p>The data set is used for this is <a href="https://www.kaggle.com/c/denoising-dirty-documents">here</a>. The dataset includes dirty document on which we have to remove that dirty part from image and clean the whole image. so here we are using auto encoders technique for de-noising the documents</p><h3><strong>Auto-Encoder Denoising Model Code :-</strong></h3><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1e789195a8bace63c8d121c868b5b64a/href">https://medium.com/media/1e789195a8bace63c8d121c868b5b64a/href</a></iframe><p><strong>Model Structure :-</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/593/1*he54FYTH2krfI4nnVUPWqA.png" /><figcaption>Fig= Model Structure</figcaption></figure><p>From above figure see the input layer and last layer shapes both are same. Here middle layers are used for dimensionality reductions and compress all the data in latent space then again we up-sampling/de-compress data for recreating the image</p><p><strong>TensorBoard Output :-</strong></p><p>Here we runs our model for 30 epochs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/489/1*3WQ2tjMlnd4KZTuxvqlQZg.png" /><figcaption>Fig- Epoch Loss (Blue line= Validation, Orange Line=Train data</figcaption></figure><p><strong>Evaluating and prediction :-</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/56344b3f9924621109542ee1d7b56fa6/href">https://medium.com/media/56344b3f9924621109542ee1d7b56fa6/href</a></iframe><p><strong>Predicted Values :-</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/375/1*qVkBF75k69fUFel50twjRA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/375/1*3pKFuwQl5wWBVGnvN7yygQ.png" /><figcaption>Fig= Actual and Predicted values</figcaption></figure><p><strong><em>References :-</em></strong></p><ul><li><a href="https://blog.keras.io/building-autoencoders-in-keras.html">Building Autoencoders in Keras</a></li><li><a href="https://www.kaggle.com/c/denoising-dirty-documents">Denoising Dirty Documents</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c58d6950dfad" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/auto-encoders-de-noising-text-documents-c58d6950dfad">Auto Encoders : De-Noising Text Documents</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A.I. Based Cavity Detection System]]></title>
            <link>https://medium.com/@murlijadhav007/a-i-based-cavity-detection-system-b908e707b20e?source=rss-e8254109f8ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/b908e707b20e</guid>
            <category><![CDATA[fastai]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[cnn]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Murli Jadhav]]></dc:creator>
            <pubDate>Tue, 31 Dec 2019 14:49:11 GMT</pubDate>
            <atom:updated>2020-03-10T14:24:31.416Z</atom:updated>
            <content:encoded><![CDATA[<p>Here we are implementing the cavity detection system using deep learning techniques</p><h3>Business problem and Introduction:-</h3><p>In 21 st century 80–90% people are suffering from dental problems and early research shows that the dental problem is related to the cavity. Nowadays people are so much busy in their work hence they do not care about themselves this leads to health problems including dental problems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KZdRlgjE_VaRx-H76AvkUw.jpeg" /><figcaption>Image Link is <a href="https://www.toothncare.com.au/images/slide_04.jpg">here</a></figcaption></figure><p>Early diagnosis is very important for maintaining dental problems. Dentists use laser-technology for diagnosis of cavity this is very costly and these results are highly negative that dentists do not rely on results instead they relying on visual inspection. Artificial intelligence helps for identifying dental problems and also it predicts better solutions with the help of dentists to the patients.</p><p>Artificial intelligence helps for identifying dental problems and also it predicts better solutions with the help of dentists to the patients.. Artificial Intelligence has more power to save a large amount of data which is useful for individuals need. Artificial Intelligence fill the gap between doctors and laser-techniques</p><h3>Cavity Types:-</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/1*PUGeMXrgs7xJXiftO4TMuQ.jpeg" /><figcaption>Image is found <a href="https://encrypted-tbn0.gstatic.com/images?q=tbn%3AANd9GcRtJD8NvqnfYhiIKk0b0BZyv7MpCEziYPEDJ0ci5Y-uCukBOm5d">here</a></figcaption></figure><p>Here We have to classify the the images as per cavities. There are total 6 types of cavities. based on this six type we have to classify or predict the probabilities of per classes</p><h3>Data Collection :</h3><p>Data will be collected using the following methods</p><p><strong>a.Primary Data:</strong>Data regarding teeth and dental disease are collected from various dentists</p><p><strong>b.Secondary Data:</strong> Data about Artificial intelligence will be collected from research journals and e-resources</p><p>The Collected Data can be stored in our own server with there location (path) in csv format. We collected total upto 300 images of cavities</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r5gmRiRNYn1Mw7X8YLnFZg.jpeg" /><figcaption>Fig:- Collection of various types of Cavity Images</figcaption></figure><ul><li><strong>Note:- </strong>Here i implemented model only for 2 classes i.e, for class-I and class-II</li></ul><p><strong>Techniques Used</strong></p><ol><li>Fast.ai (<a href="https://www.fast.ai/">https://www.fast.ai</a>)</li><li>CNN (Resnet34)</li></ol><h3>Implementation:-</h3><p><strong>Fast.ai:-</strong></p><p>Fastai library is pretty new, but it’s getting an extraordinary amount of traction. It’s making a lot of things a lot easier, but it’s also making new things possible. So really understanding the fastai software is something which is going to take you a long way.<br>So how does it compare? There’s only one major other piece of software like fastai that tries to make deep learning easy to use and that’s Keras. Keras is a really terrific piece of software, we actually used it for the previous courses until we switch to fastai. It runs on top of Tensorflow.</p><p><strong>Importing Fast.ai :-</strong></p><pre>from fastai.vision import *<br>import numpy as np<br>import pandas as pd</pre><p>Making folders for each class:-</p><pre>folders = [&#39;Class-I&#39;, &#39;Class-II&#39;]<br>files = [&#39;Class-I&#39;, &#39;Class-II&#39;]<br>classes = [&#39;Class-I&#39;,&#39;Class-II&#39;]<br>path = Path(&#39;data/Cavity&#39;)</pre><p><strong>Folder Structure must be as followes:-</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/358/1*NuFrkPTSoU7SWSIYOBbKwA.png" /><figcaption>Fig:- Folder Structure</figcaption></figure><p><strong>Getting Image data from folders:-</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/02b01d450419d6b6192fdcf8b463aa72/href">https://medium.com/media/02b01d450419d6b6192fdcf8b463aa72/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/394/1*d0olxWtFlLRP2hNUjM7nNQ.png" /><figcaption>Fig:- Height of image</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/464/1*VmZiI4Us0ruL8menRZEi4g.png" /><figcaption>Fig-Width Of image</figcaption></figure><p><strong>Image Data generators in Fast.ai:-</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dbb24afe1a0fe1c1b038bb3de4e2fbb6/href">https://medium.com/media/dbb24afe1a0fe1c1b038bb3de4e2fbb6/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/732/1*lwfPDmWh11gW7RTFhZ3NTg.png" /><figcaption>Fig-Displaying Some random images from above code with there classes</figcaption></figure><p><strong>Preprocessed Images:-</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/165/1*zAhm7qcRuTCxw_a9kuUqgg.png" /><figcaption>Before PreProcessing</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/165/1*YK_hRVRXrjUc27cga_8BRQ.png" /><figcaption>After PreProcessing</figcaption></figure><h3>Model Architecture(resnet34):-</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/936/1*UngkI_m-8tSXl2y-x4-nSg.png" /><figcaption>Fig Resnet34 Image link <a href="https://miro.medium.com/max/1872/1*Y-u7dH4WC-dXyn9jOG4w0w.png">here</a></figcaption></figure><p>One of the problems ResNets solve is the famous known vanishing gradient. This is because when the network is too deep, the gradients from where the loss function is calculated easily shrink to zero after several applications of the chain rule. This result on the weights never updating its values and therefore, no learning is being performed. With ResNets, the gradients can flow directly through the skip connections backwards from later layers to initial filters.</p><p><strong>Creating the CNN learner:-(resnet34 model)</strong></p><pre>learn = create_cnn(data, models.resnet34, metrics=[error_rate, accuracy])</pre><p>Finding the appropriate learning rate</p><p>Fast.ai gives you the functionality for the finding the optimal learning rate. the code for this as</p><pre>learn.lr_find()<br>learn.recorder.plot()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/557/1*yTJE9_NuYllZUcHy51HK0g.png" /><figcaption>Fig-learning rate vs Loss</figcaption></figure><p><strong>Initialization of the model:-</strong></p><p>In whole process we are using transfer learning. we are applying the weights of the resnet34 model. so its easier to us for classifying the things</p><pre>interp = ClassificationInterpretation.from_learner(learn)</pre><p><strong>Fitting the model:-</strong></p><pre>learn.fit_one_cycle(5)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/557/1*rwOfW1e5EpXThVHA7WNvKQ.png" /></figure><p>here we can say that train loss is decreasing as well as our accuracy is improving per epochs. and also see the time for execution. Fast.ai is very fast as per name and as compared to tensorflow and keras. By using fast.ai we are getting better results than any other techniques. Thats why i am using the fast.ai implementation for blog</p><p><strong>Train and validation loss:-</strong></p><pre>learn.recorder.plot_losses()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/436/1*UHdE_2nUUCm4nGGjndu8hg.png" /><figcaption>Train and validation loss for per batches</figcaption></figure><p><strong>Printing Top Losses/Most confused images:-</strong></p><pre>interp.plot_top_losses(21, figsize=(15,11), heatmap=False)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/1*see2iMiLG3zbpfEAWHuMiw.png" /></figure><p><strong>Exporting the model:-</strong></p><pre>learn.export()</pre><p><strong>Loading the model:-</strong></p><pre>learn = load_learner(path)</pre><p><strong>Predicting the single point:-</strong></p><pre>defaults.device = torch.device(‘cpu’)<br>img = open_image(&#39;Test5.jpg&#39;)<br>img<br>pred_class,pred_idx,outputs = learn.predict(img)<br>pred_class</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/599/1*Psh6BDKf0-DD3M081kmzqg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/483/1*9-FbXOzVDgrhnWJOujkD0Q.png" /><figcaption>Output</figcaption></figure><blockquote>For Full tutorial code you will<strong> find on github </strong><a href="https://github.com/MurliJadhav/Cavity_Detection"><strong>here</strong></a></blockquote><h3><strong>References:-</strong></h3><p>Grace F. Olsen ; Susan S. Brilliant ; David Primeaux ; Kayvan Najaria “An image-processing enabled dental caries detection system” published in 2009 ICME International Conference on Complex Medical Engineering, INSPEC Accession Number: 10615830</p><ul><li><a href="https://docs.fast.ai/vision.learner.html">vision.learner</a></li><li><a href="https://www.appliedaicourse.com/">Applied Roots</a></li></ul><p><strong>Also See my other articles:-</strong></p><ul><li><a href="https://medium.com/@murlijadhav007/introduction-to-similarity-metrics-a882361c9be4">Introduction to Similarity Metrics</a></li><li><a href="https://medium.com/mytake/gradient-descent-why-and-how-e369950ae7d3">Gradient descent: Why and How ?</a></li><li><a href="https://medium.com/analytics-vidhya/pdf-pmf-and-cdf-in-machine-learning-225b41242abe">PDF, PMF and CDF in Machine Learning</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b908e707b20e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to Similarity Metrics]]></title>
            <link>https://medium.com/analytics-vidhya/introduction-to-similarity-metrics-a882361c9be4?source=rss-e8254109f8ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/a882361c9be4</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Murli Jadhav]]></dc:creator>
            <pubDate>Mon, 30 Dec 2019 18:14:15 GMT</pubDate>
            <atom:updated>2020-01-02T04:55:49.383Z</atom:updated>
            <content:encoded><![CDATA[<p>In this blog we are discussing about “ how two sentences are similar”. In that context, we are looking at <strong>Jaccard</strong> <strong>similarity</strong> as well as <strong>cosine</strong> <strong>similarity</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*LWAPg1HoQ8tZ6rUObYlAMA.jpeg" /><figcaption>Fig=Book Shelf (<a href="https://www.google.co.in/url?sa=i&amp;rct=j&amp;q=&amp;esrc=s&amp;source=images&amp;cd=&amp;ved=2ahUKEwi63ObWp5TlAhW-ILcAHaIsBdIQjB16BAgBEAM&amp;url=https%3A%2F%2Funsplash.com%2Fs%2Fphotos%2Fbooks-on-a-shelf&amp;psig=AOvVaw2NaZ_b6KxOami5NNDa29ta&amp;ust=1570886650159181">find_here</a>)</figcaption></figure><p>Before going to see similarity matrix you need to have some knowledge about word embedding. Here i explaining a little bit summary for word embeddings</p><h4>Types Of Word Embedding:-</h4><p>Word embedding is natural language modelling technique for mapping word to vectors of real number. Following are some word embedding techniques:-</p><ol><li><strong>Counter-Vectorizer:-</strong>The idea is to collect set of words , sentences, documents or paragraphs to count occurrence of each word in document. It is also called as one-hot encoding. this is not maintaining the semantic meaning</li><li><strong>TF-IDF:- </strong>TF-IDF stands for term frequency and inverse document frequency.It is often used as information retrieval and text mining. This is not maintaining the semantic meaning</li><li><strong>Word2Vec:- </strong>word2vec is most popular technique developed by google in 2013. It maintain the semantic meaning of the document.</li></ol><h4><strong>Jaccard Similarity:-</strong></h4><p>Jaccard similarity index is also called as jaccard similarity coefficient. It measures the similarity between two sets. The range is 0 to 100%. The more percentage then more similar two word.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/389/1*oTv9dTcg8lmuiB7Cf5dkuA.jpeg" /><figcaption>Fig-Formula for jaccard similarity</figcaption></figure><p>The demo code for calculating jaccard similarity between two sentence is as follows:-</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f29a22642f7f326226e9f313e92bb15e/href">https://medium.com/media/f29a22642f7f326226e9f313e92bb15e/href</a></iframe><p>The above example is a simple code example which is used to find out the similarity between two sentences. If You have large amount of data then its better to use Word-Embending techniques as discussed above.Note that you have done the lemmatization before passing sentence to jaccard similarity.</p><p>If the length of the documents is large and even we get some same words then jaccard distance is not doing well.</p><h4>Cosine Similarity:-</h4><p>The cosine similarity is measure the cosine angle between the two vectors. For cosien we have to convert all sentences to vectors. For converting to vector we can use TF-IDF, Word2Vec.</p><p>The formula for cosine similarity is:-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/426/1*5J8YlnfnZlzFobQC9cGk-w.png" /><figcaption>Fig-Cosine Similarity formula</figcaption></figure><p>The code example is as follows:-</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5fcc5dc0590cf40ab915d121fa42c90b/href">https://medium.com/media/5fcc5dc0590cf40ab915d121fa42c90b/href</a></iframe><p>For better results you can make a vectors using glove models which maintains the semantic meaning. Normalizing the vectors is also better for improvement in results.</p><h4>Major difference between jaccard and cosine similarity:-</h4><ol><li>Jaccard Similarity takes set of unique length of words instead cosine similarity takes whole sentence vector</li><li>If data duplication is not matter then its better to use jaccard similarity else cosine similarity is good for measuring the similarity between two vectors even if the data duplication is there.</li></ol><p>Also see my previous blogs</p><ol><li><a href="https://medium.com/analytics-vidhya/pdf-pmf-and-cdf-in-machine-learning-225b41242abe">https://medium.com/analytics-vidhya/pdf-pmf-and-cdf-in-machine-learning-225b41242abe</a></li><li><a href="https://medium.com/mytake/gradient-descent-why-and-how-e369950ae7d3">https://medium.com/mytake/gradient-descent-why-and-how-e369950ae7d3</a></li></ol><h3>References:-</h3><ul><li><a href="https://www.appliedaicourse.com/">Applied Roots</a></li><li><a href="https://towardsdatascience.com/overview-of-text-similarity-metrics-3397c4601f50">Overview of Text Similarity Metrics in Python</a></li><li><a href="https://medium.com/@adriensieg/text-similarities-da019229c894">Text Similarities : Estimate the degree of similarity between two texts</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a882361c9be4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/introduction-to-similarity-metrics-a882361c9be4">Introduction to Similarity Metrics</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gradient descent: Why and How ?]]></title>
            <link>https://medium.com/analytics-vidhya/gradient-descent-why-and-how-e369950ae7d3?source=rss-e8254109f8ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/e369950ae7d3</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[optimization-algorithms]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[gradient-descent]]></category>
            <dc:creator><![CDATA[Murli Jadhav]]></dc:creator>
            <pubDate>Mon, 30 Sep 2019 12:29:55 GMT</pubDate>
            <atom:updated>2020-01-03T04:50:37.210Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Gradient descent: Why and How ?</strong></h3><h4>An Theoretical approach for gradient descent and its variants</h4><p>The minimising the cost function is very important in machine learning model. Here gradient descent is an iterative technique for minimising the cost function.cost function is error in prediction(Y_Actual-Y_Predicted)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4x7dwg5RCHhJoAI4sLlvlQ.jpeg" /><figcaption>Gradient Descent</figcaption></figure><p>If you see above image the person is on the top of the mountain or middle of the mountain here the person wants to go down with minimum number of steps. The person do this iteratively (with small steps) until he reaches at the bottom of the mountain.</p><p>The gradient descent works same. Here the mountain is the dimension of data, steps is the learning rate and the bottom(goal point) is the cost function with minimal errors.</p><p><strong>Learning Rate:-</strong></p><p>From the above example, the steps which are taken to reach the goal point (optimum point) is called as learning rate</p><p><strong>How to choose learning rate?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*OfYVPAl2ZEBeBHlTfq_glA.png" /><figcaption>Learning rate comparison</figcaption></figure><p>If the learning rate is too high then it bounces to convex function but not reached local minimum efficiently</p><p>If the learning rate is too small then it reached to local minimum but it takes too much time</p><p>We have to choose the learning rate such that the cost function decreases simultaneously. Here plot the graph of cost function vs itearation to check the correct learning rate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*LhuI0ysvsRGfO8wy2iEOQA.png" /><figcaption>Choosing good learning rate over iterations</figcaption></figure><blockquote>In gradient descent we calculate the cost over all data points. and perform updates in weight in each iteration. For example if we have 10,000 points in train then we are calculating the W(weights) and Cost just in single iteration.</blockquote><p>The problem with gradient descent is, Suppose the mountain has two minimum points the first reached minimum point is called as local minimum and second (last) minimum point is called as global minimum.then gradient descent stops the cost function when it reached to local minimum.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/526/1*Z-xJ5TAVGXRHK-4R29Pbfw.png" /><figcaption>minimum and maximum of points [Image Source:<a href="https://www.datasciencecentral.com/profiles/blogs/optimization-techniques-finding-maxima-and-minima">click_here</a>]</figcaption></figure><p>It is always better to optimise the cost function such that it reaches to global minimum. For this we have to use another gradient descent techniques. These are as follows:-</p><ol><li><strong>Batch-Gradient Descent</strong></li><li><strong>Stochastic Gradient Descent</strong></li><li><strong>Mini-Batch gradient Descent</strong></li></ol><h3><strong>1.Batch-Gradient Descent :-</strong></h3><p>It is also called as vanilla gradient descent technique. In this technique we calculates the error at each observation but perform the update only when all observation done evaluating</p><p>Batch gradient descent technique is computationally expensive technique. It is very slow and large data-sets does not load into memory</p><h3><strong>2.Stochastic Gradient Descent:-</strong></h3><p>SGD performs the weight updates on each observation with every iterations. here we first need to shuffle the data-set so that we get completely randomised data set.</p><p>As our data-set is randomised data-set and error and weight updated for each example gradient weights are arriving at global minima instead of stuck in local minima.</p><p>SGD is faster than batch gradient technique. This technique also computationally redundant because it computes each example at a time</p><h3>3.Mini-Batch Gradient Descent :-</h3><p>This technique is a combination of sgd and batch gradient technique.First it split the data-set into small batches and then perform updates on each batches. Here as we take data in batch samples it removes the noise which is variance of the weight update. It is speedy technique as compared to both batch and sgd method.</p><blockquote><em>Also view the next part i.e, Programmatic approach for this technique</em></blockquote><h3>References:-</h3><ul><li><a href="https://www.appliedaicourse.com/">Applied Roots</a></li><li><a href="https://medium.com/mindorks/an-introduction-to-gradient-descent-7b0c6d9e49f6">An introduction to Gradient Descent</a></li><li><a href="https://medium.com/@montjoile/an-introduction-to-gradient-descent-algorithm-34cf3cee752b">An introduction to Gradient Descent Algorithm</a></li><li><a href="https://developers.google.com/machine-learning/crash-course/reducing-loss/stochastic-gradient-descent">Reducing Loss: Stochastic Gradient Descent</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e369950ae7d3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/gradient-descent-why-and-how-e369950ae7d3">Gradient descent: Why and How ?</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[PDF, PMF and CDF in Machine Learning]]></title>
            <link>https://medium.com/analytics-vidhya/pdf-pmf-and-cdf-in-machine-learning-225b41242abe?source=rss-e8254109f8ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/225b41242abe</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[exploratory-data-analysis]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Murli Jadhav]]></dc:creator>
            <pubDate>Tue, 10 Sep 2019 06:01:04 GMT</pubDate>
            <atom:updated>2024-01-17T11:14:33.418Z</atom:updated>
            <content:encoded><![CDATA[<h3>PMF, PDF and CDF in Machine Learning</h3><h4>Random variables and the various distribution functions which form the foundations of Machine Learning</h4><p><strong>Table of contents</strong></p><ul><li>Introduction</li><li>Random Variable and its types</li><li>PDF (probability density function)</li><li>PMF (Probability Mass function)</li><li>CDF (Cumulative distribution function)</li><li>Example</li><li>Further Reading</li></ul><h3>Introduction</h3><p><strong>PDF </strong>and <strong>CDF </strong>are commonly used techniques in the Exploratory data analysis to finding the probabilistic relation between the variables.</p><p>Before going through the contents in this page ,first go through the fundamental concepts like <strong>random variable</strong>, <strong>pmf</strong>, <a href="https://www.analyticsvidhya.com/blog/2023/07/cdf-vs-pdf/"><strong>pdf and cdf</strong></a><strong>.</strong></p><h3><strong>Random variable</strong></h3><p>A random variable is a variable whose value is unknown to the function i.e, the value is depends upon the outcome of experiment</p><p>For example, while throwing a dice, the variable value is depends upon the outcome.</p><p>Mostly random variables are used for regression analysis to determine statistical relationship between each other. There are 2 types of random variable:</p><p><strong><em>1 — Continuous random variable</em></strong></p><p><strong><em>2 — Discrete random variable</em></strong></p><p><strong><em>Continuous random variable:- </em></strong>A variable which having the values between the range/interval and take infinite number of possible ways is called Continuous random variable . OR the variables whose values are obtained by measuring is called Continuous random variable. For e.g, A average height of 100 peoples, measurement of rainfall</p><p><strong><em>Discrete Random Variable:-</em></strong>A variable which takes countable number of distinct values. OR the variables whose values are obtained by counting is called Discrete Random Variable. For e.g, number of students present in class</p><h3>PDF (Probability Density Function):-</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/811/1*5WHytMF-WUpYt8E4uCOFMg.jpeg" /><figcaption><strong>The formula for PDF</strong></figcaption></figure><blockquote>PDF is a statistical term that describes the probability distribution of the <strong>continues</strong> random variable</blockquote><p>PDF most commonly follows the Gaussian Distribution. If the features / random variables are Gaussian distributed then PDF also follows Gaussian Distribution. On PDF graph the probability of single outcome is always zero, this happened because the single point represents the line which doesn’t cover the area under the curve.</p><p>You can find deep insights on PDF and <a href="https://www.analyticsvidhya.com/blog/2023/07/cdf-vs-pdf/">CDF here</a></p><h3>PMF (Probability Mass Function):-</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/432/1*XlERORSJusILdPXUdKel5w.png" /><figcaption>Fig:- Formula for PMF</figcaption></figure><blockquote>PMF is a statistical term that describes the probability distribution of the <strong>Discrete</strong> random variable</blockquote><p>People often get confused between PDF and PMF. The PDF is applicable for continues random variable while PMF is applicable for discrete random variable For e.g, Throwing a dice (You can only select 1 to 6 numbers (countable) )</p><h3>CDF (Cumulative Distribution Function):-</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/375/1*2vXxablEGsq5Y9Cy7Ifmng.png" /><figcaption>Fig:- Formula for CDF</figcaption></figure><p>PMF is a way to describe distribution but its only applicable for discrete random variables and not for continuous random variables. The <strong>cumulative distribution function</strong> is applicable for describing the distribution of random variables either it is continuous or discrete</p><p><strong>For example</strong>, if X is the height of a person selected at random then F(x) is the chance that the person will be shorter than x. If F(180 cm)=0.8. then there is an 80% chance that a person selected at random will be shorter than 180 cm (equivalently, a 20% chance that they will be taller than 180cm)</p><h3><strong>Python example for PDF and CDF on Iris Dataset:-</strong></h3><p>The <a href="https://www.kaggle.com/uciml/iris"><em>iris data set</em></a> contains the following data:-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*wOQL2hj_T8TXjDcY4WhuKw.png" /><figcaption>Fig:- Flower image from iris dataset</figcaption></figure><p>The detailed explanation of iris data-set is <a href="https://www.ritchieng.com/machine-learning-iris-dataset/">here</a></p><p><strong>PDF On Iris:-</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/366/1*64AAHW_zZTFa7q-EsSON1w.png" /><figcaption>PDF for [‘species’]== ‘setosa’ on petal length</figcaption></figure><p><strong>CDF on Iris:-</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/375/1*EeOHeNSz46sI0HbOQgBQJA.png" /><figcaption>CDf of iris_setosa using petal length</figcaption></figure><p><strong>Both PDF and CDF visualisation:-</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/366/1*Nbc9VhJC3YndJLg_ZQD68g.png" /><figcaption>Pdf and Cdf</figcaption></figure><p><em>You will find the detailed explanation with python code on </em><a href="https://github.com/MurliJadhav/Pdf-Cdf/blob/master/EDA_Plotting_iris.ipynb"><strong><em>Github</em></strong></a><strong><em> </em></strong><a href="https://github.com/MurliJadhav/Pdf-Cdf/blob/master/EDA_Plotting_iris.ipynb"><em>Here</em></a><em>.</em></p><h3>References:</h3><ul><li><a href="https://www.ritchieng.com/machine-learning-iris-dataset/">Iris Dataset</a></li><li><a href="https://www.investopedia.com/terms/p/pdf.asp">Probability Density Function (PDF) Definition</a></li><li><a href="https://www.probabilitycourse.com/chapter3/3_2_1_cdf.php">Cumulative Distribution Function</a></li><li><a href="https://math.stackexchange.com/questions/52400/what-is-cdf-cumulative-distribution-function">What is CDF - Cumulative distribution function?</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=225b41242abe" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/pdf-pmf-and-cdf-in-machine-learning-225b41242abe">PDF, PMF and CDF in Machine Learning</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>