<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sam Shleifer on Medium]]></title>
        <description><![CDATA[Stories by Sam Shleifer on Medium]]></description>
        <link>https://medium.com/@sshleifer?source=rss-f0dc85db5885------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*007Sew1HBXEgK4khqr4R0Q.jpeg</url>
            <title>Stories by Sam Shleifer on Medium</title>
            <link>https://medium.com/@sshleifer?source=rss-f0dc85db5885------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 03:22:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sshleifer/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[MixMatch Paper Summary]]></title>
            <link>https://medium.com/@sshleifer/mixmatch-paper-summary-1995f3d11cf?source=rss-f0dc85db5885------2</link>
            <guid isPermaLink="false">https://medium.com/p/1995f3d11cf</guid>
            <category><![CDATA[pytorch]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Sam Shleifer]]></dc:creator>
            <pubDate>Wed, 15 May 2019 00:06:02 GMT</pubDate>
            <atom:updated>2019-05-18T23:18:26.793Z</atom:updated>
            <content:encoded><![CDATA[<h3>MixMatch</h3><p>This is a summary of a recent (and very readable) <a href="https://arxiv.org/abs/1905.02249">paper</a> I enjoyed called “MixMatch: A Holistic Approach to Semi-Supervised Learning”, which presents a new algorithm for training models on a small amount of labeled data and a large amount of unlabeled data that is way more accurate than other approaches. I liberally paraphrase the authors of the paper, which I think is OK — I’m summarizing their work. For the code, I started with this <a href="https://github.com/gan3sh500/mixmatch-pytorch">repo</a> and made modifications to get it closer to working. <a href="https://github.com/sshleifer/mixmatch">This repo</a> is the working product. As of May 18, it has a working DataLoader and Loss function, but I haven’t run experiments or aggressively verified correctness yet. The authors’ tensorflow code is <a href="https://github.com/google-research/mixmatch">here</a>.</p><p><strong>Motivation</strong></p><p>Semi-supervised learning (SSL) relies on using information from unlabeled data to teach the model. Most of the papers referenced by MixMatch tend to use batches of both labeled and unlabeled data in the same training run, as opposed to the pretrain on a language modeling task then finetune on labeled data paradigm of ULMFit, Elmo and BERT. Recent work tries to learn add a term to the loss function that encourages either:</p><p>(a) confident predictions on unlabeled data. Even if we don’t know the label for an example we know that it is unlikely to be half in one class and half in another class. Pseudo-labeling, which takes the model’s most confident predictions on unlabeled data and assumes that the largest predicted probability is the correct label is a form of this.</p><p>(b) consistency regularization: encourage same prediction on slightly perturbed inputs. Predictions shouldn’t change dramatically if inputs only change slightly. This is the idea of another paper Goodfellow worked, “Virtual Adversarial Training”.</p><p>MixMatch also uses Mixup, (TODO: link) a technique where we train a model on combinations of examples. For example, we feed an image that is half cat and half dog (either by overlaying the images or putting them next to each other in the image) and expect the model to produce p(dog)=.5, p(cat)=.5 and the 0 probability for the rest of the classes.</p><p>None of these techniques are mutually exclusive, and the magic of the MixMatch paper is that they find a sensible way to use them all!</p><h3>MixMatch</h3><p>Complication: loss is computed separately for labeled and unlabeled examples, but they are potentially mixed up!</p><h4>(1) Label Guessing for unlabeled examples:</h4><ul><li>Augment each unlabeled example k=2 times, average predictions of each augmented example</li><li>Sharpen the predicted class probabilities using the sharpen function. The resulting distribution is our guess for the labels of this example.</li></ul><pre>def sharpen(preds: np.NDArray, temp=.5) -&gt; np.NDArray:<br>    # as temp goes to 0 this starts to look more like a one-hot distribution<br>    numerators = preds ** (1/temp)<br>    return numerators / (numerators.sum(axis=1))<br>    <br>ub = [augment_fn(X_unlabeled) for _ in range(K)]  # K augmentations<br>qb = sharpen(sum(map(model, ub)) / K, T)  # labeled guesses</pre><h4>(2) Mixup and Match</h4><p>We start with three batches: X a batch of labeled examples, U a batch of unlabeled examples and guessed labels, and W=Shuffle(concat([X, U]) . We use Wto get mixup partners for each example. Here is the code:</p><pre>def lin_comb(a, b, frac_a): return (frac_a * a) + (1 - frac_a) * b<br>C = np.concatenate</pre><pre>Ux = C(ub, axis=0)<br>Uy = C([qb for _ in range(K)], axis=0)<br>indices = shuffle(np.arange(len(xb) + len(Ux)))<br>Wx = C([Ux, xb], axis=0)[indices]  # mixup partners<br>Wy = C([qb, y], axis=0)[indices] # mixup partners</pre><ul><li>Apply mixup to labeled data and entries from Ux and W[:len(X)] yielding X’mixed up examples where at least 50% of the example is from a labeled data point.</li></ul><pre>def mixup(x1, x2, y1, y2, alpha):<br>    &quot;&quot;&quot;Modified to make sure that majority of example comes from x1.&quot;&quot;&quot;                  beta = np.random.beta(alpha, -alpha, x1.shape[0])<br>    beta = np.maximum(beta, 1 - beta)  <br>    return lin_comb(x1, x2, beta), lin_comb(y1, y2, beta)</pre><pre>X, p = mixup(xb, Wx[:n_labeled], y, Wy[:n_labeled], alpha)  # Labeled batch<br>U, q = mixup(Ux, Wx[n_labeled:], Uy, Wy[n_labeled:], alpha)  # Unlabeled batch<br>return  C([X, U], axis=1), C([p, q], axis=1), n_labeled # need to keep track of where labeled examples are for loss fn</pre><ul><li>Make a labeled batch containing labeled examples mixed up with potentially unlabeled examples. Enforce that the labeled example must have larger than 50% of the weight in the averaging of the labels and images.</li><li>Make unlabeled batch yielding mixed up examples where at least 50% of the example is from an unlabeled data point.</li></ul><p>(3) Loss is a weighted combo of squared L2 Loss (MSE) for mostly unlabeled U&#39; , CrossEntropy for X&#39;</p><p>This is where my code (and understanding) breaks: the target associated with an example in X&#39; is mixed up but CrossEntropyLoss expects continuous targets.</p><p>Note: the author mention that “we do not propagate gradients through the guessed labels.” What does that mean? Why even create U&#39; if it does not effect the loss? What I interpret is that you treat the guessed labels as real labels and just back-prop the loss through the part of the computational graph that made the predictions. So if you implement MixMatch on tensors, you need to make sure autograd is turned off before you make qb .</p><h3>Results</h3><p>Experiments are run using a wide Resnet model on cifar-10 and SHVN, with different amounts of labels offered to the models. In the picture below, we see that MixMatch almost reaches the “full supervised” benchmark (training on 5000 labeled images, with significantly fewer labeled examples, and outperforms other SSL techniques by a wide margin:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/554/1*9czAjLudf4qFGzeSfiziHA.png" /><figcaption>Figure 2 from the MixMatch paper</figcaption></figure><p>It also seems like none of the techniques MixMatch uses are superfluous, trying to remove any step hurts metrics considerably and the unlabeled data is obviously helping!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gKV4hnj5qGV84qhmFyAaYA.png" /><figcaption>Table 4: Ablation Study Results from the MixMatch paper</figcaption></figure><p><strong>Next Steps:</strong></p><p>Try to replicate cifar-10 performance with WideResnet.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1995f3d11cf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My Favorite Workflow Hacks for Mac]]></title>
            <link>https://medium.com/@sshleifer/my-favorite-workflow-hacks-for-mac-c90139076992?source=rss-f0dc85db5885------2</link>
            <guid isPermaLink="false">https://medium.com/p/c90139076992</guid>
            <category><![CDATA[productivity]]></category>
            <category><![CDATA[mac]]></category>
            <category><![CDATA[keyboard-shortcuts]]></category>
            <dc:creator><![CDATA[Sam Shleifer]]></dc:creator>
            <pubDate>Tue, 16 Oct 2018 00:08:56 GMT</pubDate>
            <atom:updated>2018-10-16T00:22:25.870Z</atom:updated>
            <content:encoded><![CDATA[<h3>My Favorite Workflow Hacks for Mac/iOS</h3><p>Writing this for the next time I get a new computer, and to help out all the other mac users who have 20th percentile working memories! Another post with more coding focused stuff will come soon.</p><h3>OSX</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UFfDttRMahdl3SKTtAqaSw.png" /></figure><ol><li>[Alfred] (<a href="https://www.alfredapp.com/">alfredapp.com</a>) lets you find and open files + google stuff without thinking about where you want to search. you just search and then think after. If you get the paid version you get extended clipboard which is truly incredible (see screenshot).</li><li>[Spectacle] (<a href="https://www.spectacleapp.com/">spectacleapp.com</a>) this is all the rage at Kensho. let’s you resize windows in mac without touching your mouse.</li><li>[Rescuetime] (<a href="https://www.rescuetime.com/dashboard">rescuetime.com/dashboard</a>) tracks how you spend your time to the website/application level. Will tell you 3hours of netflix, 2 hours of pycharm, and send weekly digest emails. I send these to my coworkers to hold myself accountable.</li><li>[Self Control] (<a href="https://selfcontrolapp.com/">selfcontrolapp.com</a>) This is a classic. Allows you to block websites for a fixed amount of time. I wish it could block for longer than 24 hours!</li><li>[iTerm2] (<a href="https://www.iterm2.com/">iterm2.com</a>) I haven’t used the default terminal in so long that I don’t have much to say here.</li><li>[iStat] (<a href="https://bjango.com/mac/istatmenus/">bjango.com/mac/istatmenus</a>) Not free, but can tell you how much memory and CPU are being used at every moment and over the last few minutes. Replaces `htop`/activity monitor,</li><li>Evernote: I like it because you don’t have to connect to wifi before you write and the search is nice. That said, it’s not perfect with code snippets. Though I haven’t seen anything that’s not markdown based that’s good with code snippets.</li></ol><h3>Mac Keyboard System Preferences</h3><ul><li>Keyboard: Set Key Repeat, Delay Until repeat to far right to make scrolling faster</li><li>Keyboard.: modifier keys, disable Caps Lock</li><li>minimize dock</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c90139076992" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to finetune tensorflow’s object detection models on Kitti self-driving dataset]]></title>
            <link>https://medium.com/@sshleifer/how-to-finetune-tensorflows-object-detection-models-on-kitti-self-driving-dataset-c8fcfe3258e9?source=rss-f0dc85db5885------2</link>
            <guid isPermaLink="false">https://medium.com/p/c8fcfe3258e9</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[object-detection]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Sam Shleifer]]></dc:creator>
            <pubDate>Wed, 01 Nov 2017 02:06:48 GMT</pubDate>
            <atom:updated>2017-11-03T22:09:11.773Z</atom:updated>
            <content:encoded><![CDATA[<p>This took me about 30 hours, hopefully it takes you less!</p><h4>Boredom Warning</h4><p>50% of the post discusses discusses the rather uninteresting process of converting the data found at the KITTI <a href="http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=2d">website</a> into a format where you can finetune tensorflow’s pre-trained object detection models using my <a href="https://github.com/sshleifer/object_detection_kitti">code</a>. The last 50% compares the results of training faster-RCNN and SSD Mobilenet on the dataset, and goes over some theory.</p><h4>TLDR</h4><ul><li>clone the repo <a href="https://github.com/sshleifer/object_detection_kitti">HERE</a>, and follow the instructions in the README</li><li>Fast-RCNN is slower and more accurate than SSD Mobilenet</li></ul><h3>Implementation Details</h3><h4>Get the 2D Object Detection data</h4><p>The Kitti <a href="http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=2d">website</a> has lots of links to different datasets, and it can be hard to know what you actually need: images and annotations, let alone how to organize it in your file system. All you need are the image_2 and label_2 files! *Note, the download takes an hour.*</p><pre><em>source </em>tensorflow_env.sh<br>wget http://kitti.is.tue.mpg.de/kitti/data_object_image_2.zip<br>wget http://kitti.is.tue.mpg.de/kitti/data_object_label_2.zip<br><em>unzip </em>-q data_object_image_2.zip<br><em>unzip </em>-q data_object_label_2.zip<br><em>rm </em>-rf kitti_data<br><em>mkdir </em>kitti_data<br><em>mv </em>training kitti_data<br><em>mv </em>testing kitti_data</pre><p>Now run</p><p>tree kitti_data/training</p><p>and you should get a lot of files and something like the following</p><pre>training<br>  image_2<br>    ├── 00001.jpg<br>    └── ....jpg<br>  label_2<br>        ├── 00001.txt<br>        └── ....txt</pre><p>Now comes the hacky part. To get the data in the format desired by <a href="https://www.google.com/url?sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=1&amp;ved=0ahUKEwiahMfX3rfWAhVFRiYKHR_JAUoQFggtMAA&amp;url=https%3A%2F%2Fgithub.com%2Ftensorflow%2Fmodels%2Ftree%2Fmaster%2Fobject_detection&amp;usg=AFQjCNGzS_wxG1Siq1v53rYOAim2d58PcA">tensorflow</a> , we need to make tfrecords, a terrible data that you need to run a tensorflow session to even inspect. Since the object detection repo has a script for converting from PascalVOC format to tfrecord, I wrote a script that modifies code borrowed from the <a href="https://www.google.com/url?sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=1&amp;ved=0ahUKEwjIjKaT37fWAhVELyYKHdfDD18QFggoMAA&amp;url=https%3A%2F%2Fgithub.com%2Fumautobots%2Fvod-converter&amp;usg=AFQjCNHJko8yi7QLhE-N9vUNxxL-ZJFFmg">vod-converter</a> repo to convert to VOC format, and then the tf code (with some small modifications) to go into tf-record format. All that fun is in create_dataset.py. It also does some other annoying things like convert from png to jpg and take the leading zeroes off the beginning of the filenames.</p><p>Running python create_dataset.py puts 10% of the image, label combos (794 by my count) in validi.tfrecord, and the other 90% in train.tfrecord.</p><p>At this juncture, tree data should give you</p><pre>data<br>├── kitti_map.pbtxt<br>├── mscoco_label_map.pbtxt  # useless<br>├── pascal_label_map.pbtxt  # useless<br>├── pet_label_map.pbtxt     # useless<br>├── train.tfrecord<br>└── valid.tfrecord</pre><h4>Changes to samples/config/*.config</h4><p>I made very few changes to the config files. Mostly I changed eval to run eval_input_reader on all my validation data, and look at the right path: data/valid.tfrecord , of course! I also have a rant about protobuf but I will spare you.</p><h4>What are TFRecords?</h4><p>TFRecords are the “recommended format” for tensorflow, and are roughly equivalent to a list of tf examples. tf examples are roughly equivalent to a more type sensitive list of dicts, where each key (one key per image) in the record is a Feature and each feature has some keys like height that are scalars and some keys like bbox_Xmin that are a list of coordinates with one entry for every bounding box. Here is how to convert a fake list of dictionaries to tf.TFRecord format:</p><pre>images_list = [{&#39;height&#39;: 2,<br>                &#39;objects&#39;: [{&#39;xmin&#39;: 12, &#39;xmax&#39;: 24, &#39;ymin&#39;: 230, &#39;ymax&#39;: 256}, {&#39;another bbox&#39;},...]<br>               ]<br>writer = tf.python_io.TFRecordWriter(&#39;data/valid.tfrecord&#39;)<br>for i, image in enumerate(images_list):<br>    bbox_Xmins = [bbox[&#39;xmin&#39;] for bbox in image[&#39;objects&#39;]]  # list of floats<br>    height = image[&#39;height&#39;]  # an int<br>    tf_example = tf.train.Example(features=tf.train.Features(feature={<br>        &#39;image/height&#39;: dataset_util.int64_feature(height),<br>        &#39;image/object/bbox/xmin&#39;: dataset_util.float_list_feature(bbox_Xmins)<br>    }))<br>    writer.write(tf_example.SerializeToString())<br>writer.close()</pre><h4>What is this magical Tensorflow objection detection repo?</h4><p>Most of the code that I use comes from the tensorflow models/object_detection repo (i guess technically the models repo) which provides:</p><ul><li>code to train 6 different models on any tfrecords, and evaluate performance of the model as it is being trained.</li><li>6 pretrained models for download</li><li>A bunch of utilities to do things like calculate mAP and draw bounding boxes on images.</li></ul><h4>Training and Evaluating at the same time</h4><p>There are scattered posts about how while you train your model you should also run eval.py and tensorboard. The way I do this is run ./train.sh then open a new tab in iTerm and run ./eval.sh and then a third tab and run tensorboard --logdir kitti_mobilenet , and then I go to sleep or read<a href="https://www.amazon.com/dp/B0022VV0RC/ref=dp-kindle-redirect?_encoding=UTF8&amp;btkr=1"> The Corrections</a>. I also had a problem whereby training uses all the memory in my GPU so I added</p><pre>session_config.gpu_options.per_process_gpu_memory_fraction = 0.6</pre><p>to trainer.py . I don’t see a perceptible different in training speed, but haven’t checked that closely.</p><h4>Calculating mAP</h4><p>Inspecting the bounding boxes produced by your model requires “exporting your graph for inference” . Time for another bash script that removes a small amount of redundancy from calling eval.py, which comes with the repo.</p><pre>./freeze.sh samples/configs/faster_rcnn_inception_resnet_v2_atrous_kitti.config <br>rcnn_logs/model.ckpt-{SOME CHECKPOINT NUMBER, FIXME}<br>frozen_model_dir<br># all one line</pre><p>will freeze/export your finetuned model to frozen_model_dir/frozen_inference_graph.pb</p><p>Then you can run jupyter notebook and open kitti_inference.ipynbwhere you can evaluate your mAP by class on train or valid data and look at example bounding boxes. Hopefully the notebook is somewhat self-documenting.</p><h3>Theory and Results</h3><h3>Architectural Differences Between SSD and Faster-RCNN</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/504/0*zKbOItNvzn3Uu1O3." /><figcaption><a href="http://deeplearning.csail.mit.edu/instance_ross.pdf">http://deeplearning.csail.mit.edu/instance_ross.pdf</a></figcaption></figure><p>Proposal based networks, like faster-RCNN, propose a bunch of bounding boxes (regions of interest) using one neural network and then send each proposal to two neural nets: one that does image classification on the proposed region (is it a car or a pedestrian or nothing?) and another that performs bounding box regression (how should I modify the corners of the proposed region to match the ground truth?)</p><p>This process is slow because a huge number of regions are proposed and then evaluated for every image. It is more accurate than the apostmentioned (made up word!) alternative, however, because you get to use a network whose sole purpose is image classification.</p><p>Single shot detectors, like the SSD Mobilenet I trained or the fancier <a href="https://arxiv.org/pdf/1612.08242.pdf">YOLO</a>, go straight from image pixels to bounding boxes by making the same default region proposals for every image, and then predicting bounding box coordinates and class probabilities for each proposal in one neural network.</p><p>Recent-ish improvements to the original R-CNN architecture are focused on cutting down the computational cost of region proposals in order to bridge the speed gap with the SSDs. <a href="https://arxiv.org/pdf/1504.08083.pdf">Fast R-CNN</a> (2015), for example, improved inference speed from 47s per image to 2s per image by computing the feature volume once per image, and then passing those features, and computes the feature volume used by the proposal network once and then passes convolutional features and proposals (generated by the <a href="https://ivi.fnwi.uva.nl/isis/publications/2013/UijlingsIJCV2013/">selective search</a> algorithm) to an RoI pooling layer, which crops the big convolutional feature map to the region included in the proposal, and then resizes the cropped features so that they are consistently shaped.</p><p><a href="https://arxiv.org/pdf/1506.01497.pdf">Faster R-CNN</a> (2016), the architecture I experiment with, supposedly achieves a further 10x speedup by introducing a region proposal network that takes the place of selective search. The training scheme alternates between training RPN to generate boxes and objectness scores (starting from shared convolutional feature volume) that the image detector (which is unchanged from Fast-RCNN) and bounding box regressor can predict accurately.</p><p>The following chart explains the differences between RCNN and its successors, as well as the newer R-FCN, much better than I just did.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wk2qi4pDSFbrilQFrLF0Sg.png" /><figcaption>Transition to less region-wise computation <a href="http://deeplearning.csail.mit.edu/instance_ross.pdf">http://deeplearning.csail.mit.edu/instance_ross.pdf</a></figcaption></figure><h4>My Results</h4><p>So far, I have only tried training SSD Mobilenet V1 and Faster-RCNN-inception-resnet-v2, which were the least accurate (but fastest) and most accurate (but slowest) pre-trained models available from the model zoo. Unsurprisingly, my faster-RCNN was much more accurate and much slower than SSD Mobilenet.</p><p><strong>Caveats:</strong></p><ul><li>These mAP scores (mean Average Precision, an accuracy metric, higher is better) are less friendly than the KITTI leaderboard since average include all categories, not just pedestrian, cyclist and car.</li><li>Hardware: I used a single MSI GeForce GTX 1080 GPU on an Ubuntu box that I built by loosely following <a href="https://blog.slavv.com/the-1700-great-deep-learning-box-assembly-setup-and-benchmarks-148c5ebe6415">this post.</a></li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fairtable.com%2Fembed%2FshrDFak7D1ttrZsw7&amp;url=https%3A%2F%2Fairtable.com%2FshrDFak7D1ttrZsw7&amp;image=https%3A%2F%2Fstatic.airtable.com%2Fimages%2Foembed%2Fairtable.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=airtable" width="800" height="533" frameborder="0" scrolling="no"><a href="https://medium.com/media/a3fe562003f520c1f8b79b1cf9747f0b/href">https://medium.com/media/a3fe562003f520c1f8b79b1cf9747f0b/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fairtable.com%2Fembed%2FshrVx84z0lmgF9S3i&amp;url=https%3A%2F%2Fairtable.com%2FshrVx84z0lmgF9S3i&amp;image=https%3A%2F%2Fstatic.airtable.com%2Fimages%2Foembed%2Fairtable.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=airtable" width="800" height="533" frameborder="0" scrolling="no"><a href="https://medium.com/media/f22bd088528395176270bb076e7a3e6a/href">https://medium.com/media/f22bd088528395176270bb076e7a3e6a/href</a></iframe><h4>More reliable results</h4><p>(note that my Faster-RCNN uses inception v2 feature extractor)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*FzFXMn-GlnfMmUGX." /><figcaption>Huang et al. Speed/Accuracy Tradeoffs for Modern Convolutional Object Detectors. CVPR 2017</figcaption></figure><h4>Example Prediction from my experiments</h4><p>For an example, lets take (the image Faster-RCNN had the most trouble with).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*pyX329ks6sSq0N9s." /><figcaption>Raw Image</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*ADYOoxZ7Jp56rJNm." /><figcaption>Human Labels (provided by Kitti as ground truth)</figcaption></figure><p>Note that the pedestrians in the center of the image are very hard to see!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZFAsmdE-eA8RyDrgBqEt8g.png" /><figcaption>SSD Mobilenet Predictions: Recall error with cyclist in foreground!</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*Hjz-qyUHSnXHUmYA." /><figcaption>Faster-RCNN Predictions</figcaption></figure><ul><li>Most of the mistakes that faster-RCNN makes (of which there are very few) involve small objects in the background, many of which I can’t see without squinting.</li><li>Neither architecture frequently makes mistakes on cars, unless there are huge numbers of cars in the image.</li><li>Faster-RCNN is 9X slower at inference! Maybe I should time inference for more than one image and average, as this is higher than the authors claim.</li><li>Human performance: I certainly cannot do as much object detection as these architectures in the same amount of time, at least not consciously.</li></ul><p>Next Steps:</p><ul><li>Try more models (there are now 6 in the tensorflow model zoo) and try to replicate the mAP vs inference time plot.</li><li>Try to track mAP during eval, rather than after freezing.</li><li>Time inference on many images, not just one.</li></ul><p>References:</p><ul><li>And many more!</li><li><a href="http://deeplearning.csail.mit.edu/instance_ross.pdf">http://deeplearning.csail.mit.edu/instance_ross.pdf</a></li><li><a href="https://arxiv.org/pdf/1506.01497.pdf">https://arxiv.org/pdf/1506.01497.pdf</a></li></ul><p>Thanks for reading, let me know what I screwed up!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c8fcfe3258e9" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>