<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Olga Gorun on Medium]]></title>
        <description><![CDATA[Stories by Olga Gorun on Medium]]></description>
        <link>https://medium.com/@olgagorun?source=rss-9155bf5ea3a7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*NAhOnKrM-zelzkLm.</url>
            <title>Stories by Olga Gorun on Medium</title>
            <link>https://medium.com/@olgagorun?source=rss-9155bf5ea3a7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 03:32:50 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@olgagorun/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Automatic negative dataset generation in binary image classification. Aspect ratio problem]]></title>
            <link>https://medium.com/@olgagorun/automatic-negative-dataset-generation-in-binary-image-classification-aspect-ratio-problem-4b852556ce1b?source=rss-9155bf5ea3a7------2</link>
            <guid isPermaLink="false">https://medium.com/p/4b852556ce1b</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[case-study]]></category>
            <dc:creator><![CDATA[Olga Gorun]]></dc:creator>
            <pubDate>Sun, 02 Dec 2018 15:55:54 GMT</pubDate>
            <atom:updated>2018-12-02T15:55:54.153Z</atom:updated>
            <content:encoded><![CDATA[<h3>Problem definition</h3><p>It’s known that most of the teams dealing with computer vision problems spend most of their time to gather dataset to work with. We in Anyclip are not an exception. And in general, the question I try to address here is how to reduce the time for this problem.</p><p>The typical problem we solve is a binary image classification of the types A vs ~A. Let’s look at the negative dataset problem. We already have some number of images gathered for different classifications. Most of these images can be used for a negative dataset for any next problem. That would reduce the time for the next dataset preparation approximately twice. But how to choose them in an automatic or semi-automatic way?</p><h3>Initial Solution</h3><p>Let’s say we gathered some images for positive part of a dataset. Could we build some weak classifier based on positives only and use it to choose negatives? There are one class classification/novelty and outlier detection models that are intended to solve a very similar problem: describe some class using instances of this class only. But typically they work with a relatively small number of features and are not used for work with images. There is an excellent blog post by Aniket Bhatnagar describing some approach to exactly this problem: <a href="https://hackernoon.com/one-class-classification-for-images-with-deep-features-be890c43455d">https://hackernoon.com/one-class-classification-for-images-with-deep-features-be890c43455d</a>. The main idea is to extract CNN features on an image from some pre-trained CNN network before the last layer and reduce their number even more with PCA.</p><p>There are two attractive points in this approach:</p><ul><li>Since we need classifier for negatives choice only, the only significant characteristic is <em>negatives precision</em>. So we can easily play with model thresholds to deal with confusion matrix trade-offs.</li><li>Since we don’t need to train deep learning network, only classical ML algorithms, size of positives dataset needed is relatively small. ~1000 or fewer images are enough to be able to get an almost unrestricted number of negative images with precision greater than 90%.</li></ul><p>Here are characteristics dependencies of different algorithms on training set size on “food_5k” dataset (mentioned in the original blog post).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d9x5U43FbYZqI5iODdBb5A.png" /></figure><p>Gaussian mixtures algorithm looks especially promising. It gives excellent results already from a training set containing 600 images only.</p><h3>Images aspect ratio problem</h3><p>Problems appeared when we tried to check the same algorithms on different datasets. Three more subjects were considered: ‘firearms’, ‘alcohol’ and ‘war and terror’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3usqi1Amgv3g6EOPaLiTMQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xzG6m2CAZP2709o0PqARQA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TMML0alaxjO1ZGp9851H6w.png" /></figure><p>It is obvious that algorithms training parameters were tuned for the specific dataset. But how to explain so different results? What’s the difference between these datasets?</p><p>It appears that ‘food_5k’ dataset consists of images relatively close to squares. But the rest of datasets contain images with various aspect ratio. For example, there are many video frames with the aspect ratio of 16:9.</p><p>Deep learning model results are optimized for square images. Actually, Resnet-152 takes a matrix of 3x224x224. If our image size is more than 224x224, the extra part of image pixels simply won’t be connected to the next layer. It explains why the results for “war and terror” subject, where content is spread over all the image, are better than for “firearms” and especially “alcohol”, where interesting content is usually localized in some part of the image and this part of the image can be simply out of area calculated by the network. So right image preparation (resize/crop) is a critical part of the problem.</p><p>So here is the next challenge: how to prepare images “right” before feeding the network to get significant features.</p><p>Here are approaches we considered and corresponding results.</p><h4>Resize to 224x224 exactly</h4><p>Resize an image to 224x224 exactly. In such a case aspect ratio is not preserved and according to our experiments on different problems it gives bad results so it was even not included in experiments</p><h4>Resizing of the biggest side to 224</h4><p>Resizing of the biggest side to 224 with padding of the smaller side, so all the image content will be included to a new square image. This approach gave worse results. Probably it is due to the fact that 224x224 is already a small image. And in case of an image that is far from square one actual smaller side size and corresponding image resolution are too small and many image details don’t exist in such resolution anymore</p><h4>Content-aware image cropping</h4><p>It is also possible to try to change the image itself extracting its meaningful square part. There is a number of content-aware automated image cropping algorithms. We used library <a href="https://github.com/jwagner/smartcrop.js">https://github.com/jwagner/smartcrop.js</a> . It takes into account edges, skin color, high saturation regions and several more aspects. Here the results are controversial. In some subject/algorithm combinations they are a bit better than our base (resize of minimal side to 224), in some — worse. The changes from the base case are not too high. It is probably can be explained by two factors. First of all, subjects, where the content is spread over all image, are less sensitive to any type of crop. On the other hand, the concept of “good cropping” is too context-dependent and so hard for automation. For example, used library tends to preserve image parts containing people and it’s not always the way we need (for example in cases of “firearms” and “alcohol”).</p><p>There are additional interesting algorithms about image summarizing or re-targeting like Bi-directional Similarity (<a href="http://www.wisdom.weizmann.ac.il/~vision/VisualSummary/bidirectional_similarity_CVPR2008.pdf">http://www.wisdom.weizmann.ac.il/~vision/VisualSummary/bidirectional_similarity_CVPR2008.pdf</a>), Seam Carving (<a href="http://www.eng.tau.ac.il/~avidan/papers/imretFinal.pdf">http://www.eng.tau.ac.il/~avidan/papers/imretFinal.pdf</a>) or Visual Retargeting of Natural Images using an “Internal GAN” (<a href="https://www.youtube.com/watch?v=zC8Y4Mhb7Fk&amp;index=5&amp;list=PLNiWLB_wsOg4xrvT-GHU_DK_XyxY1s9OP&amp;t=0s">https://www.youtube.com/watch?v=zC8Y4Mhb7Fk&amp;index=5&amp;list=PLNiWLB_wsOg4xrvT-GHU_DK_XyxY1s9OP&amp;t=0s</a>). But for any algorithm, we can imagine a classification problem when such automated image resize is unsuitable so further experiments in this direction were not continued.</p><h4>Manual cropping</h4><p>Since automated resizing is hard, the next idea is to prepare square images for training and validation dataset manually (actually negatives in validation set can be gotten by simple resize to minimal side). Here results in average are much better, and it is our first good candidate for the optimal type of resize. Though there are algorithm/subject pairs where they are not better than our base resizing type.</p><p>It needs to be noted also that such images preparation involves additional manual work, while the purpose is to reduce the overall time of classification model development.</p><h4>Slicing</h4><p>One more idea is slicing. Indeed we can want to represent one image by a number of slices (say with width and height of 224 px) shifted by some step, and to take convolution network features for every image. Obviously, for positive examples, part of the images of such a set contains content under question and part doesn’t. There are two decision points in such an approach:</p><ol><li>How to choose slice step and whether to resize an image before slicing?</li><li>How to represent such features set as one instance to the following model?</li></ol><p>Regarding slice step, the step of 50% of slice size looks good enough. For resize before slicing the trade-off is the following. When an image is divided into too many slices (say, without resizing at all) it’s both time-consuming and causes to interesting content to be spread into many slices. When an image is resized to minimal logical size (minimal side to 224), details resolution maybe not detailed enough. We tried three variants:</p><ul><li>slicing without resize</li><li>resize of minimal side to 720 before slicing</li><li>resize of minimal side to 224 before slicing</li></ul><p>The second variant gave the best results in most cases.</p><p>Now regarding features sets representation for models. The first idea is to concatenate slices features sets to one larger feature set. But there are two problems. One is that such concatenated set is large. Say, for images with four slices we get to 2048 features. Such a number of features is too large for considered algorithms, and that’s exactly the cause of PCA usage in the initial approach. But even if we succeed to extract meaningful features from such much larger features set, the second question is how to decide which number of slices should be concatenated. Different images from training/validation dataset can have different sizes and a different number of slices but concatenated features set size should be permanent. So we need to find the biggest number of slices at least for training and validation datasets and to decide how to fill missing parts of the concatenated feature set for most of images: whether to complement it with features of one of the slices or some type of aggregation.</p><p>Another possibility is to use some aggregation function on convolution features per feature over slices of given image. We used <em>maximum</em> and <em>average</em> and <em>maximum</em> surprisingly gave pretty good results that are comparable or better than for manual resize case.</p><p>For training and validation positive set it’s not possible to consider slices of one image separately because all we know is that there is interesting content in one or more slices, and only one label (positive) should be given. But for a testing set (for prediction) there is one more option. We can make a prediction for every slice and to classify the image as positive in the case of at least one found positive. This possibility also was checked but didn’t give work better than the case when the same type of features preparation is applied to training, validation and test set.</p><p>There is one more interesting characteristic of slice variant that prevents from giving it definite first place. As stated above the most significant statistic characteristic for the given problem is negative precision. But most resize types give not too low values ~75-~95% also for negative and positive recall and F1. In the case of slicing it’s not always this way. For part of algorithms/subjects negatives recall is decreased significantly (to ~20–30%). Practically it means that we need a bigger unlabeled dataset to get to generate the same amount of negatives.</p><p>Here are the data (at least one variants in prediction are not added here to avoid too cluttered graphs).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ug9miUjbH7gslLwK-Ihugg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y7O4kHNrt-uISESXIaT25w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qjNW8MwQ8noGiJ6H0YVIpw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*828IzQLKixJNteFhbt-hgA.png" /></figure><h3>Conclusions</h3><p>So here are the conclusions:</p><ul><li>Resize type influences negatives precision significantly. Negatives generation precision can vary in ~20%</li><li>Different algorithms react differently to different resize types. The decision of appropriate resize type for one algorithm can be less suitable to a different algorithm</li><li>Different classification problems (subjects) have different sensitivity to choice of resizing type</li><li>The most promising resize types are <em>manual resize</em> and <em>slicing</em>. From variants of slicing <em>resize to 720 + slicing with per feature aggregation (maximum) of features over slices</em> gave the best result. These variants gave the best results for most of algorithm/subject combinations</li><li>Some resize type choices (especially connected to slicing) reduce significantly negatives recall. In such cases we have the trade-off of the accuracy of generated negatives dataset and the number of unlabeled images we may be needed for this. For example, it’s simple to find algorithm and resize type that gives more than 90% precision and more than 20% of recall for every tested subject (for example, in case of<em> One class SVM </em>and <em>slicing</em> precision varies in bounds 0.94%-99% and it’s not only suitable combination). But if the required recall is at least 50%, there is no combination of algorithm and resize type with precision above 90%</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4b852556ce1b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>