<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Chaitali Bhattacharyya on Medium]]></title>
        <description><![CDATA[Stories by Chaitali Bhattacharyya on Medium]]></description>
        <link>https://medium.com/@chaitalibh.cb?source=rss-23acbb4468d0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*SqCxoV7Qgm8zru1E</url>
            <title>Stories by Chaitali Bhattacharyya on Medium</title>
            <link>https://medium.com/@chaitalibh.cb?source=rss-23acbb4468d0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 11:21:39 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@chaitalibh.cb/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[LLM Interpretation:How to Identify Functionally Important Features?]]></title>
            <link>https://medium.com/@chaitalibh.cb/llm-interpretation-how-to-identify-functionally-important-features-be26e03c3965?source=rss-23acbb4468d0------2</link>
            <guid isPermaLink="false">https://medium.com/p/be26e03c3965</guid>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[gpt]]></category>
            <category><![CDATA[llama-3]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Chaitali Bhattacharyya]]></dc:creator>
            <pubDate>Fri, 22 Nov 2024 10:57:17 GMT</pubDate>
            <atom:updated>2024-11-22T10:57:17.817Z</atom:updated>
            <content:encoded><![CDATA[<p>Latest Neurips paper [2024] talks about the current ‘limitaion’ of SAE and how it heavily depends upon the dataset features.</p><p>In this blog I have discussed the paper, <strong>“Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning”</strong> in details.</p><p>In the blog I have tried to explain the details of the main paper. Please look into this <a href="https://cv-chaitali.github.io/bhblogai.github.io/">site</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=be26e03c3965" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deep Image Prior : Explained]]></title>
            <link>https://medium.com/@chaitalibh.cb/deep-image-prior-explained-8a20598976a5?source=rss-23acbb4468d0------2</link>
            <guid isPermaLink="false">https://medium.com/p/8a20598976a5</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Chaitali Bhattacharyya]]></dc:creator>
            <pubDate>Mon, 07 Oct 2024 11:03:32 GMT</pubDate>
            <atom:updated>2024-10-07T11:03:32.550Z</atom:updated>
            <content:encoded><![CDATA[<h3>Deep Image Prior : Explained</h3><blockquote>In the paper of “Deep Image Prior”, they did not claim to introduce a novel method. However, they introduced a fact that CNN architecture holds some prior. Well! before delving deeper into the topic, I’ll discuss few things regarding the problem statement or why we need this.</blockquote><p>Please check the link below for step by step explanation :</p><p><a href="https://chaitalibh.notion.site/DIP-23af41c398a94c89bfc53a95940a816f">DIP | Notion</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8a20598976a5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transformer is All You Need:Part-1]]></title>
            <link>https://medium.com/@chaitalibh.cb/transformer-is-all-you-need-part-1-17b5203c6be9?source=rss-23acbb4468d0------2</link>
            <guid isPermaLink="false">https://medium.com/p/17b5203c6be9</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[transformers]]></category>
            <dc:creator><![CDATA[Chaitali Bhattacharyya]]></dc:creator>
            <pubDate>Mon, 27 Mar 2023 06:42:43 GMT</pubDate>
            <atom:updated>2023-03-27T06:42:43.120Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>The vision Transformer [paper resource: <a href="https://arxiv.org/pdf/2010.11929.pdf">AN IMAGE IS WORTH 16X16 WORDS</a>] was developed by Google Research, Brain Team in 2021 based on Transformer in Natural Language Processing (NLP) [paper resource : <a href="https://arxiv.org/abs/1706.03762">Attention is All You Need</a>]</blockquote><p>What is Transformer in NLP?</p><p>As we got the basic idea of Vision Transformer from NLP Transformer so we need to understand what actually is happening inside this NLP model.</p><p>Let;s take a sentence for an example</p><p>“I am going to school” So, in these kind of tasks they have been using RNN/ LSTM networks which are very popular. RNN works like the figure below. It takes the output of the hidden layer of the previous output and goes to the next layer/ word and it goes on. After finishing the sentence it goes to the decoder and predicts an word according to the first work of the sentence and it’ll go on just like the encoder section. In the case of translation first decoder word would be “main” [hindi] and so on… But in these kind of situations considering the grammatical and other parameters while translating is kind of hard for RNN and LSTM. Even if the model learns the grammar , the meaning or the structure of the sentence will be hard to manage for all the languages.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/353/1*3APuVwYly4scFUT7GniGVA.png" /></figure><p>So, “Attention” comes here to be the savior. In case of attention, at the decoder section it helps the decoder to go through the last decoded word and analyse the word. In this case, if the last decoded word is “going” then instead of going through all the hidden layers of teh encoder section it only pays attention to the “going” word.</p><p>But the question here is how does the decoder know which word to look at?</p><p>well, the output of the decoder is a bunch of keys.. so, these keys would make an index of hidden states. So, basically we don’t need any RNN layer in the decoder section we can just put attention instead of RNN layers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/578/1*t7FfIaNelEfsFOgPw0EjSw.png" /></figure><p>So they proposed this architecture where the input sentence will go inside the input embedding and the target sentence will go inside the output embedding. And all these will happen at once.</p><p>Here, Positional Embedding means the labels of words. There are so many words and we don’t know the sequence and to get the sequence right the positional embedding is important.</p><p>For this input embedding the positional embedding is for each word and for the next word the similar process will be carried out till the l;ast word of the sentence.</p><p>The encoder layer’s work is to map the input sequence in an abstract continuous representation. It consists of two submodules 1. Multiheaded Attention and 2. Fully connected network.\</p><ol><li>Multiheaded attention :</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/229/1*-q65mWBYAlKh7JggTp5eyA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/231/1*opMUtC38z1ie5W5wJc88dA.png" /></figure><p>It works in a specific attention work which is Self-Attention.It means that the model can associate each words of the sentence i.e “school” can have the knowledge about “to” and “going” or “school” might have the intuition of the structure of the other words present in the sentence. To achieve the self attention the input is fed into the three fc layers to create the<strong> query (q), key(k) and value(v)</strong> vectors.</p><p>q X v = scores</p><p>Each word will have corresponding score which will decide where to put the attention. The higher the score the attention will be more.</p><p><strong>After this the score will be divided by the dimension of q and k which will produce scaled scores. After this we need to get the softmax of the scaled scores to get the attention weights(w) the probability value of which can be 0 to 1.</strong></p><p>then, w X v = output vector</p><p>Then the output vector can be fed into the linear layer.</p><p>So for the Multi-headed-attention module the q, k and v should be splitted into n vectors before self attention layer. and each output of self attention layer can be concatenated into one vector and fed into the linear layer.</p><p>After the output there is a residual connection where the multi-head attention layer output is added to the original input. The output of teh residual connection then goes through layer norm. It is fed into the pointwise feed forward network. The output of that is again added to the output of the layer normalization output.</p><p><strong><em>This was all about the Encoder layer …………….</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/195/1*daL-olNhy6gFtnlXg9N8Ng.png" /></figure><p>For the decoding layer the positional embeddings are fed into the multi-headed attention which computes the attention score again. This layer is a bit different from the encoder’s. For an example… the word “going” should not have the information about the words after it..it should have the access to the words before it only. <strong>But how can we get that feature where the model doesn’t create a situation where the word is getting access of the future words? Well.. for that masking is used inside the layer. So, in this section the scaled score is added to the look-ahead mask. The mask is having elements of 0s and — infs. when we add this to the scaled scores we get the matrix which will have the scaled score of the words and for the future words it’ll be -inf. And when we take the softmax of this we’ll get the output where the -inf will be turned into 0.</strong></p><p>For the next multi-headed attention layer, as seen in the figure…there are various connection. one is the output of the input embedding and othe rone is the decoder’s scaled masked score. This will help the model to decide which encoder’s input is worth focusing on. After this we’ll get the output from this layer which will be fed into the feed forward layer which then goes to the final linear layer which has the access of the classifier. The classifier is then fed into the softmax layer producing probability scores.</p><p>I hope it helped a bit.</p><p>So this was it for the overall basic idea of what happens inside the Transformer. However, this was for Language processing. In the next part I’d be trying to show the visual representation of how it works on images [Vision transformer].</p><p><strong><em>In case you are interested in AR projects I have also wrote detailed explanation on Camera Calibration [</em></strong><a href="https://medium.com/@chaitalibh.cb/camera-calibration-with-checkerboard-54f93af742a0"><strong><em>https://medium.com/@chaitalibh.cb/camera-calibration-with-checkerboard-54f93af742a0</em></strong></a><strong><em>]</em></strong></p><p><strong><em>In case you are interested in AR projects I have also wrote detailed explanation on Camera Calibration [</em></strong><a href="https://medium.com/@chaitalibh.cb/camera-calibration-with-checkerboard-54f93af742a0"><strong><em>https://medium.com/@chaitalibh.cb/camera-calibration-with-checkerboard-54f93af742a0</em></strong></a><strong><em>]</em></strong></p><blockquote>REFERENCES:</blockquote><p>These two videos were very helpful:</p><ol><li><a href="https://www.youtube.com/watch?v=iDulhoQ2pro&amp;t=1083s">https://www.youtube.com/watch?v=iDulhoQ2pro&amp;t=1083s</a></li><li><a href="https://www.youtube.com/watch?v=4Bdc55j80l8&amp;t=303s">https://www.youtube.com/watch?v=4Bdc55j80l8&amp;t=303s</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17b5203c6be9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Detailed and Visual explanation: Convolutional Neural Networks]]></title>
            <link>https://medium.com/@chaitalibh.cb/detailed-and-visual-explanation-convolutional-neural-networks-42c89756f593?source=rss-23acbb4468d0------2</link>
            <guid isPermaLink="false">https://medium.com/p/42c89756f593</guid>
            <category><![CDATA[image-classification]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[convolutional-network]]></category>
            <dc:creator><![CDATA[Chaitali Bhattacharyya]]></dc:creator>
            <pubDate>Mon, 06 Feb 2023 06:15:21 GMT</pubDate>
            <atom:updated>2023-02-06T06:15:21.613Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>I think visual explanations are very important to see what is happening inside the layers of a Deep Learning network. It’s also very interesting and fascinating in the way that mathematics (linear algebra or whatever way you see it) creates such amazing visual interpretation. Let’s just get into the topic.</blockquote><p>Here, I am going to talk about the pre-trained model <strong>VGG-16</strong>, one of the most popular CNN models till now. But before that let’s just start with what are <strong><em>the layers included in Convolutional Neural Network (CNN).</em></strong></p><p>There are three main layers in CNN: 1. Convolution Layer, 2. Pooling Layer and 3. Fully connected layer.</p><h3>Convolution Layer</h3><p>The convolution operation is defined as an operation between two functions. Let’s say John is out jogging and he wants to track himself (I know it’s weird but let’s just think like it). So the tracking device is giving an output which is x(t) where t is time. But the result can be noisy due to different issues so an easy way to detect his position he wants the tracker to get more outputs and he also wants to get a weighted average out of it. The more you get observations, the more you get accurate. So, let’s take a weighted function of w(a). where a is the age of the measurement. The purpose is just to get the latest information as older information gets kind of less valid in these cases. Anyways, so now the s(t) gives a better/ more relevant result of John’s position. And this is called convolution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/606/1*CFmcqPm7N2yBpoEF93yCMQ.png" /></figure><p>You can also address/express it as</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/388/1*VMnMo3pzzcfUgK54lYHzHw.png" /></figure><p>For this particular example, the w should be 0 in all negative cases otherwise it’ll look into future possibilities which is kind of invalid anyway. <strong><em>If we look at the equation the x is the input and w is the kernel and the output can be a feature map.</em></strong></p><p>Now, if we see that the t, time is only an integer like (1 second, 2 seconds, 3 seconds), we’ll get discrete convolution. in that case, we can express it as …</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oZzAHhs6wvFo6jpv_Bfpdw.png" /></figure><p>If we use 2D images…let’s say I as an Input, similarly, we need to use a 2D kernel as well..where <em>k is the kernel m and n </em>are the <em>indices of I and i and j </em>are the<em> new indices at position i and j.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7pOTIZaxiPWgN6zYCBPB6Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JNPAaUXEOydJIRwR6zF7Dw.png" /><figcaption>Fig-1</figcaption></figure><p>If you see the fig-1, this is how the kernels work with the image. And the kernel matrix simply multiplies with the image matrix and generates the output. Padding can be added to protect spatial information. Padding is nothing but creating a border around the image with zeros and Stride means the steps / pixel you are skipping the kernel to move.</p><p>If we Input image M,M,B and kernels’s spatial size is N and the stride is S with padding with P, the size of the Ouput will be..</p><p><strong><em>Xout = (M-N+2P/S)+1</em></strong></p><blockquote><strong>In case we are using an image with dimensionality such as hyperspectral or multispectral image, we can use 3D CNN for a better result. In case of 3D CNN the kernel goes in depth as well. If you have an image with 120 bands. The kernel will go through all the 120 bands and perform.</strong></blockquote><h3>Pooling Layer</h3><p>The reason why Pooling layer is used is that one can make a small change without a big effect on the image as it takes the nearest neighbor pixels’ value and take a new number representing all the neighborhood pixels.</p><blockquote>“It means that if we translate the input by a small amount, the values of most of the pooled outputs do not change”…Deep learning book,Ian Goodfellow and Yoshua Bengio and Aaron Courville</blockquote><p>Usually there are 3 kind of pooling.</p><ol><li>Max Pooling : it selects the highest value in the matrix</li><li>Average Pooling : it takes the average of the the elements in the matrix</li><li>Sum Pooling : it takes the sum of the elements in the matrix</li></ol><p>Pooling layer is used when we need to adjust the inputs of varying sizes. Some of the tests have proved that Max pooling layer is useful for most of the cases. But but but….. there is a problem using convolution and pooling together. Let’s say we have a case where we need to protect the spatial information, using pooling will increase the error, and can cause underfitting as well.</p><h3>Fully Connected Layer (FC)</h3><p>Fully connected used to change the output of the previous layers as a grid/vector. One of the advantages is that we can learn features from fully connected network. It also helps to generate more complicated relationship between inputs and outputs.</p><h3><em>Visualization Through the Code</em></h3><p>Here, I have taken pretrained vgg16 model but for visualization I have tried to call the model layers as you can see. You can take any other model of your choice.</p><pre>from keras.models import Sequential<br>from keras.layers.core import Flatten, Dense, Dropout<br>from keras.layers import Conv2D, MaxPool2D, ZeroPadding2D<br>from keras.optimizers import SGD<br>import numpy as np<br>from matplotlib import pyplot as plt<br>from keras.models import Model<br>  <br>model = Sequential()<br>model.add(Conv2D(input_shape=(224,224,3),filters=64,kernel_size=(3,3),padding=&quot;same&quot;, activation=&quot;relu&quot;))<br>model.add(Conv2D(filters=64,kernel_size=(3,3),padding=&quot;same&quot;, activation=&quot;relu&quot;))<br>model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))<br>model.add(Conv2D(filters=128, kernel_size=(3,3), padding=&quot;same&quot;, activation=&quot;relu&quot;))<br>model.add(Conv2D(filters=128, kernel_size=(3,3), padding=&quot;same&quot;, activation=&quot;relu&quot;))<br>model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))<br>model.add(Conv2D(filters=256, kernel_size=(3,3), padding=&quot;same&quot;, activation=&quot;relu&quot;))<br>model.add(Conv2D(filters=256, kernel_size=(3,3), padding=&quot;same&quot;, activation=&quot;relu&quot;))<br>model.add(Conv2D(filters=256, kernel_size=(3,3), padding=&quot;same&quot;, activation=&quot;relu&quot;))<br>model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))<br>model.add(Conv2D(filters=512, kernel_size=(3,3), padding=&quot;same&quot;, activation=&quot;relu&quot;))<br>model.add(Conv2D(filters=512, kernel_size=(3,3), padding=&quot;same&quot;, activation=&quot;relu&quot;))<br>model.add(Conv2D(filters=512, kernel_size=(3,3), padding=&quot;same&quot;, activation=&quot;relu&quot;))<br>model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))<br>model.add(Conv2D(filters=512, kernel_size=(3,3), padding=&quot;same&quot;, activation=&quot;relu&quot;))<br>model.add(Conv2D(filters=512, kernel_size=(3,3), padding=&quot;same&quot;, activation=&quot;relu&quot;))<br>model.add(Conv2D(filters=512, kernel_size=(3,3), padding=&quot;same&quot;, activation=&quot;relu&quot;))<br>model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))<br>model.add(Flatten())<br>model.add(Dense(units=4096,activation=&quot;relu&quot;))<br>model.add(Dense(units=4096,activation=&quot;relu&quot;))<br>model.add(Dense(units=2, activation=&quot;softmax&quot;))</pre><p>We can check it using <em>model.summary()</em>. The loss function is <strong><em>“categorical_crossentropy”</em></strong> First, let’s call the conv layers one by one to check the shape. You can put any layer number (conv) to check the shape.</p><pre>layer = model.layers<br>filters, biases = layer[3].get_weights()<br>print(layer[3].name, filters.shape)</pre><p>And after that if we plot the filters[:, :, :, i-1], we can visualize the images like this. These filters are from layer[3]/4th layer. If you check the 3rd layer you can see filters with lesser depth. The more higher we go the depth gets better. You can see both the gray scale (left) visualization and normal colored(right) visualization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/644/1*RMxS5q1CQHbhQNLYHqAZdw.png" /></figure><p>We can check for others changing the value of x, filters[:, :, x]. You can go ahead and check for each and every filters to analyze the effects on the image. Now we need to resize our image to match the in put size (224,224).</p><pre>from keras_preprocessing.image import load_img, img_to_array<br>img = load_img(&#39;C:/Users/tt/Desktop/SE.jpg&#39;, target_size=(224, 224))</pre><p>After this you can see the changed size to make sure it matches the input size. It will give you this shape if you are using RGB image “<strong><em>shape = (224,224,3)”. {You can upload any image of your choice}</em></strong></p><p>Then we can change the image to an array using img_to_array(img). We can predict the images using fo = model_short.predict(img) and we can store prediction in fo.</p><pre>for feature in fo:<br><br>    fig=plt.figure(figsize=(12, 12))<br>    for i in range(1, columns*rows +1):<br>        fig =plt.subplot(rows, columns, i)<br>        plt.imshow(feature[0, :, :, i-1])<br>    plt.show()</pre><p>After plotting we can analyze how the filters are extracting the features. Similarly, you can play with the variables and check what are their importance on the images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/692/1*-WdjTyLmAft8EV06_tS16g.png" /></figure><p><em>I really hope this helps while I am also having a clearer vision while making these vlogs I hope to learn more. Correct me if you feel like there is something you want change in the vlog so that I can also learn.</em></p><p><strong><em>In case you are interested in AR projects I have also wrote detailed explanation on Camera Calibration [</em></strong><a href="https://medium.com/@chaitalibh.cb/camera-calibration-with-checkerboard-54f93af742a0"><strong><em>https://medium.com/@chaitalibh.cb/camera-calibration-with-checkerboard-54f93af742a0</em></strong></a><strong><em>]</em></strong></p><blockquote>REFERENCES:</blockquote><ol><li>This video will be very helpful incase you are confused about the visualization part <a href="https://www.youtube.com/watch?v=ho6JXE3EbZ8&amp;t=363s">https://www.youtube.com/watch?v=ho6JXE3EbZ8&amp;t=363s</a></li><li>The book I am studying to understand the idea of Deep Learning in depth and I am loving this book <a href="https://www.deeplearningbook.org/">https://www.deeplearningbook.org/</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=42c89756f593" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Camera Calibration With CheckerBoard]]></title>
            <link>https://medium.com/@chaitalibh.cb/camera-calibration-with-checkerboard-54f93af742a0?source=rss-23acbb4468d0------2</link>
            <guid isPermaLink="false">https://medium.com/p/54f93af742a0</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[aruco]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[camera-calibration]]></category>
            <dc:creator><![CDATA[Chaitali Bhattacharyya]]></dc:creator>
            <pubDate>Thu, 26 Jan 2023 12:47:29 GMT</pubDate>
            <atom:updated>2023-01-26T13:01:33.222Z</atom:updated>
            <content:encoded><![CDATA[<p>Camera calibration is the most important part when one wants to detect or calculate the distance between the AR marker/any object and the Camera. In this blog, I am going to share my project and the important study material for the project.</p><p>Keywords: Focal length, Camera matrix, Optical centre, Rotational vector, Translation vector, Distortion, Pinhole camera, Fisheye camera.</p><blockquote><strong>Camera Calibration</strong></blockquote><h3>Camera Parameters</h3><p>Why do we need camera calibration? Well! if you see the <strong>figure-1</strong> [Please don’t judge my drawing skill], The camera is actually capturing the 3-dimensional data and transforming it into 2 Dimensional data. For this transformation, it’s obviously hard to analyse the real-world distance just with a camera. And this is one of the reasons why we need to do the camera calibration. The main aim is to get the camera’s geometrical parameters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Er1RDv9Sq7zSgn9aQsusdg.jpeg" /><figcaption>Figure-1</figcaption></figure><p>Let’s see what parameters we need for camera calibration. There are basically two types of parameters: Intrinsic parameters and extrinsic parameters.</p><p>In this case,</p><p>Intrinsic parameters: Focal length [fx,fy] and Optical center [cx,cy]. Extrinsic parameters: rotation [rx, ry, rz]and translation vectors [tx, ty, tz]. These extrinsic parameters are their to help you understanding the transformation between 3D real-world object to 2D image.</p><p>In the following equation, x = homogeneous form of 2D image points, P = camera matrix (containing the intrinsic parameters) and X = 3D homogeneous point.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_KeIKndw-AEFRmj1kdRRbg.jpeg" /></figure><p>So, this was about a short discussion about which primary parameters are important here.</p><h3>Distortion</h3><p>But there is another parameter which we have to take into consideration…and that is Distortion. Now Distortion can be present in different form. If you see f<strong>igure-2, </strong>you can see two types of distortion. One is Barrel and other one is Pincushion distortion. Now, there are other kinds of distortion which can be present while taking pictures according to what lens you are using. And you can remove the distortion based on the lens type you are using.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/0*ThNRClhtrabCb6Dk.jpg" /><figcaption>Figure-2</figcaption></figure><p>The two popular methods are the Pinhole camera model and the Fisheye camera model. Now, if you are using a camera with more than 135 degrees Field of View (FOV) you have to use Fisheye. In this small project, I have used the Pinhole model as my camera has 120 degrees of FOV. In OpenCV, there is <strong><em>cv.calibrateCamera</em></strong> which returns the distortion coefficient which includes the coefficients K1 through K6 representing net radial distortion while P1 and P2 represent net tangential distortion.</p><p>Let’s get into the method of camera calibration….</p><p>I have used (8x6) checkerboard you can download yours from <a href="https://markhedleyjones.com/projects/calibration-checkerboard-collection">this site.</a> Click 17–25 pictures(from different angles ) from the camera you are going to use for detection. If your project needs high accuracy please click more with various angles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g3fTt73p5arXsEWod2wjKQ.png" /></figure><pre>import cv2 <br>import numpy as np<br># corners of the square blocks (vertical and horizontal)<br>Ch_Dim = (8, 6)<br>Sq_size = 24  #milimeters<br>criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001) </pre><p>After giving the values of my checkerboard I have added the 3D and 2D points as mentioned in the discussion.</p><pre>obj_3D = np.zeros((Ch_Dim[0] * Ch_Dim[1], 3), np.float32)<br>index = 0<br>for i in range(Ch_Dim[0]):<br>    for j in range(Ch_Dim[1]):<br>        3D_obj[index][0] = i * Sq_size<br>        3D_obj[index][1] = j * Sq_size<br>        index += 1<br>#print(obj_3D)<br>obj_points_3D = []  # 3d point in real world space<br>img_points_2D = []  # 2d points in image plane.</pre><p>After this I have looped all my images from image file so that all the files can be read and by using drawChessboardCorners. Before</p><p>image = cv.2cvtColor(img, cv.COLOR_RGB2BGR)</p><p>check if your images are in RGB or BGR and change it accordingly</p><pre>import glob<br>image_files = glob.glob(&quot;location of folder/*.jpg&quot;)<br><br>for image in image_files:<br>    #print(image)<br>    <br>    img = cv2.imread(image)<br>    image = cv2.cvtColor(img, cv2.COLOR_RGB2BGR)<br><br>     <br>    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)<br>    ret, corners = cv2.findChessboardCorners(image, Ch_Dim, None)<br>    if ret == True:<br>        obj_points_3D.append(obj_3D)<br>        corners2 = cv2.cornerSubPix(gray, corners, (3, 3), (-1, -1), criteria)<br>        img_points_2D.append(corners2)<br><br>        img = cv2.drawChessboardCorners(image, Ch_Dim, corners2, ret)</pre><p>Now I have used<strong><em> cv2.calibrateCamera </em></strong>to get the distortion coefficient, Intrinsic and extrinsic matrix. As you can se in code snippet, R_vecs and T_vecs are Rotational and Translation vector respectively and dist_coeff is the distortion coefficient which may give you 5 coeffs/6 coeffs according to your camera and lens and the method you follow.</p><pre>ret, mtx, dist_coeff, R_vecs, T_vecs = cv2.calibrateCamera(obj_points_3D, img_points_2D, gray.shape[::-1], None, None)<br>print(&quot;calibrated&quot;)<br>np.savez(<br>    f&quot;{calib_data_path}/CalibrationMatrix_college_cpt&quot;,<br>    Camera_matrix=mtx,<br>    distCoeff=dist_coeff<br>    RotationalV=R_vecs,<br>    TranslationV=T_vecs,<br>)</pre><blockquote>REFERENCES</blockquote><ol><li><a href="https://www.youtube.com/playlist?list=PL7Fxe7FxChBa8DWnKQ0if4s8eBEOOVOdn">These three videos</a> from Prof. Shree Nayar, Columbia University helped me a lot and it can be a good resource to gain deeper knowledge for the Extrinsic and intrinsic parameters.</li><li><a href="https://docs.opencv.org/4.x/da/d13/tutorial_aruco_calibration.html">https://docs.opencv.org/4.x/da/d13/tutorial_aruco_calibration.html</a></li><li><a href="http://www.cs.cmu.edu/~16385/s17/Slides/11.1_Camera_matrix.pdf">http://www.cs.cmu.edu/~16385/s17/Slides/11.1_Camera_matrix.pdf</a></li><li><a href="https://learnopencv.com/understanding-lens-distortion">https://learnopencv.com/understanding-lens-distortion</a></li><li><a href="https://in.mathworks.com/help/vision/ug/camera-calibration.html">https://in.mathworks.com/help/vision/ug/camera-calibration.html</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=54f93af742a0" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>