There’s this autonomous driving challenge called F1tenth that has both a “physical” version (in which you need a physical car to participate, 1/10th the scale) and a “virtual” one where anyone can compete. In this blog post we’ll focus on the virtual race which comes with a gym-like environment called f1tenth_gym which looks like this:

Yea, I know — you can almost smell the burning rubber :)
But actually, the simplicity of this environment (and most gym environments, I suppose) is its greatest strength. You can prototype quickly and test out more ideas. This is the reason I switched to this environment from the CARLA simulator which, although prettier, is sometimes too cumbersome.

The controller that’s driving the car in the short clip above is called Pure Pursuit and the best lap time I got was 52.47s after testing out more than 1000 different parameter combinations. The topic of this blog post is: how I got that lap time down to 44.35s using the same data I collected while searching for optimal controller parameters.

Introduction

The car has two actuators: the steering angle, and the desired speed (and not acceleration, as one might expect from a “normal” car).

I used a Pure Pursuit controller which essentially aims the steering angle at some point that lies further down the road on a pre-defined path, thus giving a pretty easy but quite robust way of choosing the first actuator. (By the way, if you’d like to know more about the Pure Pursuit controller look no further than this lecture.) That distance is often called the lookahead distance and I should note that normally this distance depends on the speed of the vehicle, but in my implementation it’s actually frozen for the entirety of the race, just to make it even simpler.

You still need to come up with the second actuator, speed. The simplest approach: stick to a pre-defined speed unless the centrifugal force exceeds a tire force threshold and if it does, reduce the speed to a value that is right below that point. Obviously, this simplification is bound to fail because: the vehicle’s dynamics are way more sophisticated, and also because although we ask for a particular speed, the car can’t just instantaneously “make it so”. For example: say we’re going 10m/s and our procedure determines that the speed should actually be 5m/s tops. Asking for a particular speed is not the same as magically setting the speed of the vehicle to that value, it takes a while to get there.

We thus arrive at a pretty simplistic controller whose behavior (and overall performance) depends on three parameters:
• lookahead distance | 4.24m
• speed setpoint | 10.62m/s
• tire force max | 13.42N
where the values provided are those for which I got the best lap time, just to give you an idea about their magnitudes.

Motivation

The problem with this controller is that you might find parameters that give you OK-ish results for one set of conditions (a particular corner, let’s say) but not for the whole race track. In particular, the track in which I was interested in has a lot of straights:

Our simple controller can either drive well through the corners, or drive fast in those straights, not both. One would need to do path optimization to be able to go through the corners faster, or add an “if” statement to the code to drive fast in the straights and slow through the corners, or any other trick to make it more swanky. This would increase the overall number of parameters and, honestly, would feel a bit clunky. But also, I had a bigger picture in mind, namely: I wanted to have a procedure that can be generalized to racing against opponents. (In the next blog post, though.)

The idea was to train a model that emulates the behavior of our weak controller for a given set of parameters and then use that model to continuously determine the best parameters. This requires two things: 1) a method of scoring the parameters that will allow us to determine which are “best”, and 2) a procedure for determining the best parameters at a given situation without evaluating all possible combinations of parameter values in real time, during the race.

Methods

Model

The input to the model are three vectors: the state of the vehicle, the path that we’re supposed to follow, and the parameters of the controller. And there are two output vectors: the future trajectory, and the future actuators that are needed for that trajectory to come true.

INPUTS:
The state of the vehicle comprises 5 numbers: longitudinal and lateral speeds, angular speed (Z-coordinate), and current actuators (steering angle and speed). Note that there’s no position nor yaw. This is because we’re already operating in a coordinate system that’s centered on the car (its position is always (0, 0)), with the X-axis shooting forward in the direction in which the vehicle is pointing (so yaw is always 0).

The cent stands for centerline which is the path we’re following. As the name suggests, this is the path that lies exactly in the middle between the race track’s bounds. (Clearly, this path is not the fastest but we’ll worry about that more in the next blog post.)

And finally, the params are the controller parameters (lookahead distance, speed setpoint, and tire force max) that were set while the data was recorded. The idea is to look for better params while driving via an online gradient ascent procedure such that we constantly adapt to the changing conditions.
_________

OUTPUTS:
The trajectory are 150 2D points representing future positions of the vehicle, and the act is a vector containing future actuators: 9 steering angles, and 9 speeds.

We need the trajectory to compute the score later on, whereas the actuators are, well, needed to drive the vehicle. Although I should say: not all of them, I’m only taking the first steering angle and speed and throw away the rest (as you would normally do in a controller like Model Predictive Control, which I used in a previous blog post).
_________

The colorful boxes in the graph depict modules containing one or more fully connected layers, nothing fancy.

During inference, the state and centerline are given, we can’t change them. But the idea is to probe the model for alternative controller parameters that might yield a more promising trajectory.

Score

Initially, I used the term “return” but then I realized it might confuse reinforcement learning savvy readers so I settled for “score”. Remember, we’d like to be able to say how good the current params are (those used as input to the neural network), i.e. we want to assign a score to the controller parameters. I used the following formula to compute the score:

where progress is a function of the trajectory and the centerline and is simply the total distance travelled along the centerline, whereas the penalty is a function of all the distances between the points in the trajectory and all the points in the nearby bounds of the racetrack. See the code for more details.

During inference, while driving the vehicle, I’m estimating the gradient of the score with respect to the controller parameters and I’m doing that by feeding the model a batch of size 25 in which all states and centerline inputs are the same, but the controller parameters differ and these are: the current param values + 24 neighboring values needed for estimating the gradient.

I continuously update the controller parameters as you would normally do in a gradient ascent loop:

This online optimization probably never reaches the optimum, partly because the score landscape constantly changes, and partly because this is a pretty crude approach, but the hope was that it at least adapts quickly enough to new conditions.

Results

This is the graph representation I used to determine if the model drives as intended (see the full video here):

The upper row depicts model predictions (going from left): the future trajectory (150 steps into the future), and 9 future steering angles, and 9 future speeds. The red lines and dots represents the best prediction in a batch of size 25, blue represents the results for the current controller parameters. All those grey lines correspond to suboptimal input params.

The bottom row contains a minimap + values of progress and penalty calculated for the predicted trajectories. Both plots have observation indices sorted according to their progress values so that we can quickly see how big the penalty is for the trajectory that goes the furthest.

Data

I’d like to be able to test out this approach on an actual car (1/10th scale, no need to panic) so I needed to consider two things: safety and how much data is needed.

Safety is ensured as long as any walls are far from the path the car is following and the localization algorithm works correctly. If both are true, I can sample controller parameter values from a fairly wide range and run data collection without additional supervision.

The best lap time I got (44.35s) was when I trained a model on data from 1000 races, each race lasting for 110s simulation time (maybe 7s of real time), so that’s 1000 x 110s = 30.6h, way too long. But I also made experiments where only 32 races were conducted (which is ~1h worth of simulation time) and the best lap time then was 45.49s which is “good enough”. Our real world is a bit more complicated so that 1h of simulation time might translate to just about anything, but at least it gives me some hope it will be attainable.

Inference time

Inference on my workstation (GTX 1080Ti) along with progress and penalty calculations (which are also done on the GPU) takes ~1.5ms. That’s for a batch of size 25, but I also checked how long it takes for a batch of size 512 and it was ~3.1ms. I don’t know yet how fast it will be on a Jetson but, again, this looks promising, even before any additional speedups (I’m thinking of TensorRT, that’s the one I know the best).

Discussion & future work

It’s not lost on me that the lap time reduction wouldn’t be so profound if I used a more sophisticated controller. In particular, an MPC that optimizes the distance travelled along the centerline would probably lead to a model wouldn’t have such a significant lap time reduction (unless, for example, there’s something very wrong with the dynamical model used for the controller). Speeding up the Pure Pursuit controller wasn’t really the point here, I just wanted to know if I’m heading in the right direction, if there aren’t any critical errors in my code.

The fact that the lap time went down is a good sign, and now I can move on to the next step which is the optimization of the “cent” input that’s being used by the model. Currently, that’s not feasible because this feature has a size of 600 (300 2D waypoints), but it can be substantially condensed. Once that’s done, and once I figure out how to effectively sample the space of possible waypoints, I should not only be able to follow a faster path, but I’m also hoping to avoid collisions with obstacles.

The next step after that is avoiding collisions with opponents. For that I would need to at least roughly forecast their future positions. But wait a second… the current model can do just that, all I need to do is make the batch a bit larger, it will still be blazingly fast. (Assuming I know the state of my opponents, but that’s “doable”.) Then I can calculate the penalty using not only the bounds but also the predicted trajectory of the opponents and optimize the score as I’m doing currently.

This is where I am right now and where I’d like to get from here. The project is split into three repos:
* a fork of the f1tenth_gym project
* this repo for training the model
* and this overarching repo (a Python package) containing code used in both of the above projects.

If you enjoyed this post, please hit the clap button below and follow our publication for more interesting articles about ML & AI.

Drive faster with machine learning was originally published in Acta Schola Automata Polonica on Medium, where people are continuing the conversation by highlighting and responding to this story.

After several months of building a model of a self-driving car, I was able to get it to drive around a pond that’s next to my house. Here’s a teaser:

The “what”

It’ll be easier for me to refer to the mobile robot by it’s name, “Karr” (an homage to KARR), simply because it’s shorter than “an autonomous RC car” and less ambiguous.

Karr was strongly inspired by the specification for the F1/10th competition but it doesn’t have a LIDAR (which is too damn expensive) and instead uses a depth camera as it’s primary sensor. It’s main computing unit is the NVIDIA Jetson TX-1 and the chassis is a two-wheel drive Traxxas Slash. The depth camera is mounted on top of a “safety cage” scaffold (built by my Dad — thanks, Dad!) so that: 1) it has a better view point, 2) the hardware as a whole is safer (in case of a rolloever or an unfortunate collision).

Most notably, the computing unit has a GPGPU that supports CUDA which, in particular, let’s you get a shorter inference time for a neural network than on a CPU of, for example, a Raspberry Pi. With 256 Maxwell CUDA cores it actually allowed for the model to yield predictions in ~3.5ms (pre-processing and everything else included)!

Here’s how the world looks like from Karr’s perspective (full movie):

The closer something is, the darker it is in the depth map. Also, although this will not be relevant in this article, a depth camera lets you get point clouds which can then be used e.g. for SLAM (Simultaneous Localization and Mapping). This is how it looks like:

This is a mapping of the aforementioned pond that’s next to my house, and the path around it — that’s the “race track” around which I wanted the neural network model to be able to drive.

The “why”

My ultimate goal is to train a neural network on depth maps in the CARLA simulator so that it can drive Karr in the real world. But that’s for another blog post, in this one Karr was driven by a model trained using the same pipeline I implemented in a previous project (which I tested out in a simulation environment). The only two differences were: 1) the model was simpler, and 2) the data was produced by a richer, more convincing simulator — our reality ;)

Software

The neural network was trained using keras, however it was much easier to install Theano rather than TensorFlow on the Jetson so I used the former as backend.

The depth camera has its own dedicated library, librealsense, with various plugins (e.g. for Unreal Engine 4), APIs (in particular, for Python), and an impressive suite of additional tools, tutorials, and example use-cases. I compiled this library with -DBUILD_WITH_CUDA flag (“looky here”) thus yielding 60FPS, although running with the lowest resolution (which was far more than I needed).

I didn’t mention one more hardware component: an Arduino-like microcontroller, Teensy-LC. The Jetson communicates with this microcontroller which then sends a PWM signal that controls the power of the engine and the steering angle of the front wheels. For that, the microcontroller needed a small program which I wrote using the standard Arduino IDE. The source code can be found here.

Also, I used a wireless game pad for manual control, and no, I didn’t have to program that, but it’s good to mention this now because I wanted to make a general point regarding software: it was getting complicated. There are all these peripherals, each with its own API, each sending messages asynchronously at its own rate, each prone to its own set of errors... you get the idea. And the aim is to bring all these elements together, collect data for training, deploy the model, use its predictions to control the motor and the steering angle, improve, and repeat. Phew.

To manage all that, I used a framework called the Robot Operating System (ROS) which isn’t actually an operating system (strictly speaking) but it’s so rich in functionalities, submodules for all sorts of sensors, tools for messaging, for data visualization, and so much more — that by virtue of its utility it deserves the “operating system” title. If you haven’t heard about ROS before, check out the Wikipedia article, it gives a pretty good overview.

But, roughly, ROS abstracts the various sensors and procedures being calculated as a computational graph in which nodes correspond to processes and edges to topics through which processes get their required data in a form of structured messages. I’ve written those terms in italics because those are terms actually used in ROS (for a reference check out the Concepts ROS wiki page). The messages are passed asynchronously, each at its own rate, with processes being triggered by the arrival of a new portions of data.

I used ROS’es wrappers for communicating with the depth camera, with the game pad, and with the microcontroller. But ROS is not just a repository of wrappers, it’s also (and in my opinion, first and foremost) a development platform. Thus, I used ROS to implement a so-called package for controlling the car, traxxas_control, and if you’re interested, I’ve open-sourced the code.

The traxxas_control package is organized into nodes for controlling the car using: a neural network, a game pad, and a heuristic based on the depth map. I gathered data for training by steering the car myself, and by using the heuristic. In the next section I’m discussing how I actually trained the model and on which data.

Training the model

I trained a neural network model to clone some method of steering the car, human-generated or deterministic. As mentioned above, I used a pipeline developed for a previous project, however I needed to keep the model simple so that the inference on the Jetson was fast, yet complex enough to perform well. The architecture uses a single convolutional layer and predicts steering angle and throttle for 10 steps into the future. The architecture is summarized in the following plot (it’s a bit perplexing, I know, but you can check out the code for a detailed definition):

When driving at low speed (see the movie in the Results section), I used the steer__1 prediction, i.e. steering angle 1 step into the future. The reason is that getting an image from the camera, processing it, producing a prediction, and then communicating that to Karr’s actuators — all this takes time, a prediction for that image is overdue. I’ve calculated the latency and it was roughly the same as the rate at which the images come from the camera. But that’s a rough estimation, I’ll need to account for speed of the car in the future. For medium and high speeds I used steer__4 and steer__8 respectively, they seemed to be handling the car better but I haven’t really made a systematic analysis, it’s completely subjective.

I’m ignoring the model’s throttle predictions for now. It’s not that I don’t trust the model (I don’t), I just wanted to keep the setup simple, focus just on the steering angle.

Also, going back to inference time, I needed to reshape the depth map images from 424×240 to 42×24. This is how the world looks like after reshaping (I’ve also included model predictions):

Not great, not terrible. I wouldn’t want to drive in those conditions, but the lower resolution didn’t deteriorate the quality of the model that much — MSE on the test set was ~0.010 and ~0.009 correspondingly for the decimated and normal resolution variants.

On a different note, I anticipated there will be problems in mimicking the way a human controls the car. In a previous blog post I focused on how different sources of data (human-generated vs. deterministic) influence the quality of the model and concluded that it’s relatively difficult to train a model based on expert data (i.e. generated by a human driver).

But that was in a simulator — what’s the case if we’re operating in reality?

First approach: human-generated data

As it turned out, reality aggravates the problem with training a model on expert data. I was able to get the model to ride around the track, but it required multiple corrections and very low speed. I’m not going to show its performance in this blog post, I think I might work on it a bit more. I’m confident I could gather the right type of data (drive away from the track’s barriers and thus “teach” the model how not to get off the track, like I did in a simulation). But I knew it’ll be easier to train a model based on a more consistent method of steering than that coming from a human, and that’s what I focused on.

The behavioral cloning approach is pretty hard and I’ve already mentioned the reasons “why” in earlier blog post, but suffice it to say that humans base their decision on more than an ML model has at its disposal + they’re rather inconsistent. But I refuse to leave it like that, I’m currently working on this problem and… we’ll see, that’s all I can say for now.

Second approach: heuristic based on depth map

Instead, I used a simple, deterministic procedure that simply steers in the direction in which there’s the most space to go. It’s a variation of a method called “follow the gap”, with a few differences that account for: 1) the fact that the view is constantly shaking, 2) noise in the depth map, and 3) a pretty narrow field of view of the camera.

Roughly, the idea is to take a set of points lying on an elliptical arc, take depth readings in the neighborhood of each point, calculate the average depth in that neighborhood, and then choose the point with the highest average depth. That point corresponds to a steering angle. I stored a decimated image and its corresponding angle, and those pairs (image, angle) were then fed to the model. On the left is a depth map in which the points lying on the arc were replaced by their average neighborhood distance.

Prediction took, on average ~35ms (on the Jetson) using this method. That’s actually not bad and it’s definitely “driveable”. However, a neural network model trained on the labels generated with this heuristic yields a prediction (including communication and pre-processing) in ~3.5ms even though the GPU has just “only” 256 CUDA cores.

This is a neat trick: make a costly computation on a larger machine, use the result as label for a neural network, train it, and then use its predictions for making decisions on a smaller, embedded system.

Results

I’m going to first introduce you to the pond next to my flat around which Karr was driving, then I’ll show three videos of the car driving at: low, medium, and high speed. And then, I’ll include a short video of how the same neural network performed in an entirely different environment.

Driving around the pond

Here’s how the pond looks like on Google Maps:

And here’s a video from the neural network controlling the steering angle and making a single lap around the pond, low speed:

Here’s a lap at medium speed (sorry for the shaky camera, I had to run behind the car):

And here’s one of the turns at high speed:

I didn’t capture the whole lap because I didn’t trust myself to react fast enough to stop the car and avoid any possible collisions, but also: I would look absurdly sprinting around the pond behind a “toy car”.

As a bonus, and to give you an idea of how fast a car like this can go, here’s a short video at maximum speed, on a straight, but still controlled by the neural network:

Driving in a cellar

I used the exact same NN model and tested it out in a completely different environment (full movie):

This is a corridor in my cellar and it’s just one turn, I know, but it’s a 180°, much tighter turn than any of the ones Karr experienced driving around the pond. I had to use two hacks to make it work (can you guess what they were?), but honestly, it was way easier to get to work than I anticipated. And as a result a model trained on data gathered around the pond also works indoors.

I think that’s awesome :)

Acknowledgements

I would like to thank:

• the company Skriware for 3D-printing the blue body parts you could see on top of the Traxxas chassis
• the organizers of the F1/10th competition for providing excellent materials for amateurs like me to follow and build my own autonomous RC car. The upcoming F1/10th competition will take place in New York, in October (we’ll be there!), and the one after that will be in Berlin, probably in April or May 2020. It’s an awesome event, with great atmosphere and people, and you, Dear Reader, should definitely take part in it ;)
• and last but not least— my Dad for building the scaffold to which the camera was attached.

If you enjoyed this post, please hit the clap button below and follow our publication for more interesting articles about ML & AI.

Training a neural network for driving an autonomous RC car was originally published in Acta Schola Automata Polonica on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this blog post I’m presenting a neural network architecture for transforming this type of input:

into this type of output:

A quick note here: the side cameras don’t really cover the whole 360° angle around the car, therefore the model needs to infer the missing regions from those actually provided at input. Also: I got the semantic segmentation for free from the CARLA simulator, this isn’t an output from any model, it’s the ground truth.

Motivation

To reiterate: given four side cameras with built-in semantic segmentation (we’ll discuss that in a moment) we’ll assemble the top-down view from a camera above the car. From this perspective we can immediately see the car’s surroundings, which way the road goes, and the positions of other vehicles on the road. The inspiration for this project was the CARLA Autonomous Driving (AD) Challenge, in which one of the categories is based on input from four side cameras.

If you haven’t heard about the CARLA Autonomous Driving (AD) Challenge, that’s OK, it’s not a prerequisite for this blog post, I’m going to introduce the important parts here anyway.

The objective of the CARLA AD Challenge is to complete a given route (above), provided different types of input data that can be used by the algorithm driving the car. Below are the four categories, or: Tracks, that specify the available input:

Let’s look at Track 2’s description:

Track 2 — RGB cameras: for brave ML teams that want to explore driving with cameras only.

Brave? ML? Yup, that’s us ;)

Now, since this is a simulator, I can gather virtually any type of additional data I can imagine, train a neural network capable of modelling that data, and then use the model’s predictions for controlling the car, for SLAM (Simultaneous Localization And Mapping), or for path planning. In my case, the additional data was the top-down view from a camera mounted 100m vertically above the car, capable of providing ground truth semantic segmentation of the scene — I call it the semantic bird’s-eye view.

Set-up

OK, so if in the CARLA Challenge the available cameras are RGB, without semantic segmentation, why did I utilize it in my model? There are three reasons:

1. It was easier to get a straight answer to the fundamental question: “can it be done?”
2. I figured that once I’ll know the answer to the above question, I’ll be able to come up with a decent semantic segmentation model, especially for the type of synthetic images acquired from a simulated environment.
3. Also, I figured, once I’ll have a neural network capable of mapping four semantic segmentation cameras into a single top-down view in a simulator, and that if I’d have similar side cameras mounted on a real car, and assuming that I’d have a decent model for semantic segmentation of real-world images, I would be able to re-create this semantic bird’s-eye view in the real world. Easy peasy, right? ;)

Although the last reason is admittedly a bit far-fetched, it’s actually the one I personally find the most compelling.

The architecture

My initial thought was to use a fully convolutional autoencoder for the four input images, combine their bottlenecks in some clever manner, and to build a fully convolutional network for assembling the top-down view. But, if you think about it, fully convolutional models map a given part of the input onto a particular part of the output. Sort of like a “1-to-1” mapping. And for our purposes the architecture should allow for any part of the input to affect any part of the output. Let’s call that an “any-to-any” mapping. (See the EDIT below for a retrospective comment.)

To model this “any-to-any” mapping we need somewhere in our architecture a layer that has this magical property of carrying information from any of its input neurons to any of its output neurons. If only such layer existed…

Obviously, the dense layer works fine, and that’s what I used in the end. But before disclosing the architecture, I want to ask you this, dear Reader: have you come across an architecture that generates an image, that would involve convolutional layers, but no dense layers, in which any part of the input has the option of impacting any part of the output? I was thinking about an attention mechanism combined with a convolutional layer that would have this property but this solution would suffer from the same drawbacks the dense layer does.

I should probably explain my seeming animosity towards the dense layer: the number of parameters in my final model is 8,234,179, of which 7,150,080 come from the dense layers! Thus, I have a hunch there is a more natural approach. And it might utilize dense layers, by all means, but in a more clever way than I did. Maybe a factorization module is the way to go? Maybe there’s a recurrent approach? I’m open to your thoughts and suggestions in the comments section.

The model I used for inference has the following graphical representation:

The relevant parts are:

• gray: the four inputs from: FrontSS, LeftSS, RightSS, and RearSS cameras (“SS” stands for Semantic Segmentation, of course);
• orange: the encoder_submodel which is a fully convolutional encoder part of an autoencoder (all inputs share the same encoder model), more about the autoencoder below;
• yellow: a stack of dense layers followed by a Conv2D, then Concatenate, and then Conv2D—one stack per one input type;
• orange: the decoder_submodel which is a fully convolutional decoder part of the same autoencoder from which we extracted the encoder_submodel (again, there is one decoder for all types of input);
• cornflower blue:reconstruction being the final semantic bird’s-eye view.

The inference model was built from building blocks (in particular, the encoder/decoder parts of an autoencoder) of a more complicated model trained using several auxiliary losses. Without going into details: on the one hand, the model learned how to efficiently encode and decode images from the side cameras and the top-down camera using a common autoencoder, and on the other hand, how to reassemble the top-down view from the encoded side cameras’ images. For an implementation check out this notebook.

EDIT: as Bartosz Topolski pointed out, it’s possible to use a fully convolutional architecture that has this “any-to-any” property. You just need to make sure that the filter size of the convolutional layer after the autoencoder’s bottleneck is large enough (with respect to the bottleneck’s size). I’ve checked and he’s right, this is a notebook with a model that doesn’t use dense layers. This solution doesn’t have some of the properties that a “standard” fully convolutional model has (you can’t expect an input of a different shape to be processed correctly), but it stands to prove that the current architecture can be improved. For a 3-class variant (details below) my model had a loss of ~0.062, and Bartek’s had ~0.042 on the test set! And on the 7-class variant mine had ~0.40 and Bartek’s had ~0.36.

Purpose

A top-down view has a number of advantages:

• it condenses relevant information from four images into one;
• it can serve as a building block for localization/mapping algorithms;
• it can be used for path planning (example of which I’m presenting below);
• it allows for training a higher-level reinforcement learning agent that chooses an optimal path rather than atomic actions (a bit longer comment in a moment).

I’m going to show results of a crude path planning procedure at the end of this post. I’ll probably devote a future blog post to the topic of possible applications of the method presented here.

Training and Results

Training

I’ve trained the model on data gathered in the two default towns available in CARLA (Town01 and Town02), 40 episodes per town, each episode 1000 frames long. For validation and testing I used separate episodes but from the same towns. All results shown here were produced on the test episodes.

I’ve gathered images with resolution 300 × 200 (same as in the CARLA AD Challenge) but because the training took a lot of time, I’ve decimated them by selecting every other row/column, and then cropped them so that the shape would be divisible by 8 (requirement of the fully convolutional autoencoder), so the final shape was 144 × 96. In this set-up the training took ~16h on my GTX 1080 Ti. Inference on the same machine takes 4.5ms, however note that in a full setup (from RGB to semantic bird’s-eye view) we would also need some time for producing semantic segmentation for the input images.

Semantic bird’s-eye view

Here’s a compilation of the input side cameras, ground truth, and the predicted top-down view (entire results can be found here):

The rightmost image was constructed by taking an argmax of the predicted class probabilities. This approach has an annoying drawback: the vehicles seem to be reappearing out of nowhere, even though actually the predicted probability for their class is not null. The model was slightly imbalanced and I could have tried calibrating it, and/or instead of using an argmax try to define a better decision making procedure (in terms of a hand-crafted loss matrix like the one you build in decision theory). Also, in the following subsection I’m using a better technique of visualizing the predicted class probabilities. Long story short, the visuals might have been tweaked, but I didn’t want to spend too much time on it.

Instead, I’ve decided to spend some time on utilizing this predicted top-down view for path planning.

Crude path planning

Now, the truth is I only need three classes to be able to drive around and not collide with others: road, vehicles, everything else. Here’s how these results look like (the full video can be found here):

Again, the predicted vehicles materialize very close to our car, but it so happens that because now there are three classes, instead of taking an argmax, I can treat these images as RGB so that we can better see that the model indeed predicts other vehicles.

It’s best to see what I mean:

The white dots denote the waypoints of the path planning procedure. The path is found using a greedy algorithm that scans a range of possible waypoints, takes a sphere centered on a given waypoint, and calculates the “average road” for that sphere. Then, the waypoint with the highest “average road” is chosen, and the procedure is repeated. For implementation see this script, the function is called find_waypoints.

I should emphasize: these dots were added after the data collection process, which is the reason why sometimes the waypoints “can’t make up their mind” and jump from one option to a completely different one. But if the car actually followed the path, such cases would not occur.

The path planning procedure generates waypoints that can then be followed using some standard control procedure, MPC for example (check out my previous blog post to learn more about the MPC).

Discussion

The results for the 7-class variant look a bit trippy, I must say. The primary reason is that the input doesn’t contain the whole information required for reconstructing the top-down view. Heck, the side cameras don’t provide a full 360° sweep around the car. But also: certain classes are rare and/or form shapes that are thin and I’ve noticed that these types of shapes get lost in the encoding process. U-Net, for example, handles such detailed shapes better because it uses skip connections. I don’t quite know how these skip connections would fit into my current model, but it might be worth looking into.

The 3-class variant is less wobbly in part because the location of objects is more “inferable” from the input, and in part because these three classes are admittedly easier to predict. By “inferable” I mean: besides vehicles, there’s very little that can happen in the narrow regions not covered by any of the side cameras, the model can easily guess that.

The data set is available here and the code here, if you’d like to experiment with the architecture and, ultimately, to come up with a better architecture and results. If you encounter problems with the code, please create an Issue in the repository.

Future work

I suspect that the 7-class-model might yield better results if there was additional information about how tall the objects are. So I think it makes sense to utilize the depth map in the top-down camera as part of the output predicted by the model.

Instead of using a single top-down camera, I’m thinking of using several, suspended at different heights. This would result in a Matryoshka-like set of outputs and should allow for predicting the road and other vehicles at greater distances than currently.

I have a hunch that a generative model might do a better job of “guessing” the parts of the output that don’t have their corresponding parts in the input. This applies to the regions not covered by the side cameras as well as those situations in which a nearby vehicle blocks the view.

Because the crude path planning procedure looks promising, I feel tempted to try out a reinforcement learning agent that would choose a path, perhaps the optimal speed, and (potentially) coefficients of a controller. Then, the path would be followed using the controller, as it’s typically done, but the coefficients of the controller would be dynamically changed by the agent. I’m hoping here that when operating on a higher level, the agent would have a lower chance of overfitting to atomic decisions, and instead learn more general laws governing the environment. Sorry if this paragraph is less clear, I’ll probably devote a future blog post where I’ll explain myself a little better.

Thus far I’ve ignored pedestrians (I’ve excluded them from the simulation) and I must either find a better way of modelling rare classes (e.g. weigh their contributions to the loss function, come up with a better architecture) or emphasize their importance by synthetically enlarging their presence in the top-down view. Actually, come to think of it, making every pedestrian morbidly obese seems to be the right way to go.

References

In this work the authors solve a similar problem but using homography:

[1] Deng, L., Yang, M., Li, H., Li, T., Hu, B., & Wang, C. (2018). Restricted deformable convolution based road scene semantic segmentation using surround view cameras. arXiv preprint arXiv:1801.00708

If you enjoyed this post, please hit the clap button below and follow our publication for more interesting articles about ML & AI.

From semantic segmentation to semantic bird’s-eye view in the CARLA simulator was originally published in Acta Schola Automata Polonica on Medium, where people are continuing the conversation by highlighting and responding to this story.

Before we start, the source code for this whole project is available here. If you have questions regarding the code, please create an Issue. And if you’d like to gain access to the data to re-produce the results, leave a comment, and I’ll try my best to put it out somewhere.

EDIT: in a more recent blog post (here) I’ve decided to use a much more lightweight environment than CARLA to train a model that I used to drive *faster* than the algorithm used for driving the car for data collection.

Introduction

In the previous part of this series, I trained models on depth maps (rather than RGB) collected from the CARLA simulator [1]. (There’s a good reason for this and I’ll discuss it at the end of this blog post.) However, while the essence of Part 1 was: how to create your own race track in CARLA and get a neural network to control a car to go around it, the gist of Part 2 is: how the source of data for training neural network models influence performance on the race track.

This time around I’ve used a different car, one that is faster, more responsive and (subjectively) more fun to control —the Audi TT. The default car (Ford Mustang) has lower acceleration and its tires rarely loose traction. And I wanted the model to have a chance to learn on data in which slippage occurs so that it would, at least in principle, “know how to handle” such situations.

I’ve also added three new race tracks. I’ll go through them in more detail below, but what’s important for now is that I used first two race tracks (creatively named: “01” and “02”) for training, and the other two (“03” and “04” , you guessed it) for testing. The measure of a model’s performance was the distance (expressed in laps around the track) accomplished in 10.000 steps of CARLA’s simulation.

Data

I used three methods of collecting data for training the models:

1. The Proportion-Derivative (PD) controller introduced in Part 1;
2. The Model Predictive Control (MPC) which we’ll discuss shortly;
3. The Game Pad (GP), i.e. I trained the model to mimic my driving behavior.

Here are examples of the three methods of gathering data, as recorded on race track “01”.

Now, what’s crucial is that these simulations were run in synchronous mode (the simulation waits for the model to make a prediction), which gave an unfair advantage to the MPC. I’ll go into more details of what an MPC is but suffice it to say that it’s computationally demanding. Notice that in the movie above the average FPS (Frames Per Second) for MPC is roughly 14. So it takes about 70ms to produce a prediction. That’s partly because I implemented the MPC in Python, and partly because it simply is more complex than, for example, the PD controller. Running MPC (and other models for that matter) in asynchronous mode would require more work and I’ll need to look into that, but for now that wasn’t my focus.

Also, to “augment” the data, I’ve added uniform noise to steering and throttle of all the controllers. The values used for training were the clean, unspoiled values returned by a controller, then noise was added and sent to CARLA.

As a side note, the weather is only an aesthetic effect, it did not influence tire friction, nor the depth data on which the neural network models were trained on. I used weather conditions as an indicator of which model was used to control the car. The convention was: sunny→Proportion-Derivative controller, cloudy→Model Predictive Control, rainy→Game Pad, sunny late afternoon→Neural Network.

Control

In this section, I discuss the four methods of controlling the car used in this experiment:

PD controller

This type of control was already introduced in Part 1 of this series, so I’m not going to spend much time on it here. Shortly speaking, the steering angle at time t is chosen so as to counter the deviation from the pre-defined path — the so-called Cross-Track Error, CTE. The steering angle is composed of two terms, one proportional to CTE, and the other proportional to the time derivative of CTE.

This is the most simplistic version of the PD controller and it served nothing other than providing a baseline. Only the steering was being controlled; the throttle was brutally clipped, keeping the velocity at around some chosen value. Not only that, this type of controller doesn’t make use of the available information provided in the pre-defined path, it only looks at the local state and naively tries to correct it.

MPC

The Model Predictive Control is something different entirely. The idea is to predict how the system (in this case: the car) will behave in the next N steps, assuming a simplistic model of the world (in this case: a kinematic model of motion). The MPC returns actuators (steer and throttle) for the next N steps, but only actuators predicted for the next time step are used for control.

The values of the actuators are determined by minimizing a cost function (I’m only including terms relevant for this discussion, the ellipsis denotes “other terms”):

where δ and a are the steer angle and throttle (actually, acceleration), the {cᵢ} are coefficients chosen by the user, and v₀ is the target speed with which we want the car to drive. The type and number of terms differs between implementations, but what’s important is that:

• on the one hand we want to keep CTE small (follow the path), but on the other we want to go with speed v₀;
• the cost function is composed of terms spanning N steps into the future.

The MPC minimizes the cost subject to a set of constraints dictated by the kinematic model, which makes this a nonlinear optimization problem. In the accompanying repo I use the SLSQP (Seqential Least SQuares Programming) method available in the scipy.optimize.minimize function.

Once we find an optimal set of actuators, {(δᵢ, aᵢ)}, only the first pair, (δ₁, a₁), is used to control the car. Now, because the cost includes conflicting terms (follow the path, but go as fast as possible), the trajectory generated by MPC is often similar to that of an apt driver. The car approaches some of the corners wide, without much slowing down, some require the car to really slow down, and some (like chicanes, for examples) are elegantly cut.

I recommend this series of videos for a good yet gentle introduction to MPC.

GP

To control the car, I used the Logitech F310 game pad and the pygame module to pass on the signal from the pad to CARLA. I should note that at first, I tried to go as fast as possible — cut corners, drift through some of the toughest turns, and reach 100km/h on the longest straights. The video provided earlier shows exactly that style of driving. But despite my best efforts, I was unable to train a neural network to mimic this type of driving, or even: to train the model so that it would successfully complete at least one lap on any of the race tracks.

So I cheated. I drove the car in such a way that the model had a chance of learning a certain pattern of behavior. This is how it looked like:

The pattern of behavior that I wanted to “teach” the model was to follow the track along the barriers. Still, fitting a neural network to data produced by a human is hard, and we’ll discuss this a bit more in the Discussion.

The neural network model

I used a singular neural network architecture, which is a composition of two sub-models: an embedder, and a multi-task set of output layers. The embedder is a LeNet-like-submodel (similar to the one used in Part 1) used to yield a feature extraction layer or: an embedding layer:

The second part of the model builds on top of the feature extraction layer. First are the steering angles —each has its own hidden layer, followed by a single prediction neuron. Next, the embedding layer is concatenated with the speed (assumed to be provided as input), and the outputs for the steering angles. Here’s how it looks like in code:

Why like this? Why not build prediction for throttle and steering on top of an embedding layer already concatenated with speed?

When I provided speed as input for both steering and throttle, the model tried to cut corners too sharply:

It seems as if the model learned that low speed = sharp turn. When I tried to tweak the architecture (reduce complexity, in particular) I ran into a different problem: the model didn’t turn the car sufficiently strong when the speed was high. (However, in the Discussion section I mention an article that uses an architecture in which speed is an input for steer predictions, and their results look pretty impressive!)

But speed is important for making predictions of the throttle. Thus, the architecture detailed above is my workaround for this problem. Also, I’ve added the steering angle outputs to the embedding layer in hopes of providing extra data for the throttle, but frankly, I don’t think it helped much. Same as with steering, each throttle got its own hidden layer, followed by a prediction neuron.

Admittedly, a dedicated hidden layer for each of the outputs significantly increases complexity of the model and I needed to keep a wary eye on overfitting. A more natural approach would be to use a single, common hidden layer emb for all steering outputs and a
concatenate([emb, inp_speed] + steer_outputs)
layer for all throttle outputs. And such a model performed fine, but the more complex model managed tight corners better at higher speeds.

I’ve tried to generate an informative visualization of the architecture, but this is what I got using keras:

And I’ve really tried to make it more informative by using other tools, but didn’t get much improvement. So, like in Part 1, I claim that when it comes to explaining the model the code does a better job than visualizations.

Race tracks

I’ve generated three more maps the same way as described in Part 1. I first created race track “02” and on it tested a model build on “01”. It failed miserably; this new track was too tangled up. When I trained on “02” and tested on “01” the model also failed. So I decided to train on (“01”, “02”), and created race track “03” for testing, and the model performed OK. Once I finally decided on the architecture, the training process, etc., I added race track “04” to the test set to avoid selection bias, and the final set-up was: train on (“01”, “02”), test on “03” and “04”.

Results

Before jumping into the actual results, two important pieces of information:

1. This is not a thorough study

What I mean by that is that I chose a particular architecture not by systematically evaluating a set of candidates. For example, the multi-task predictions for 10 steps into the future— that’s something I think stabilizes the predictions, makes the ride smoother. But just before publishing this post, I tested how well a model without those predictions performed on race track “04” and it actually did OK. It got worse results, but I can’t say for sure it wouldn’t achieve similar or even better results if I were to devote enough time tweaking it.

2. I had two knobs for controlling the pace of the car

The correlation between predicted vs. true values of throttle for almost all of the models looked like this:

There is some correlation but this is a mediocre performance. Oftentimes, a car wouldn’t even start going because at the initial state the throttle prediction would be too low. To deal with that, I introduced two coefficients, α and β, such that the throttle prediction p was transformed into (α·p + β) before passing it to the CARLA client. This allowed me to squeeze better performances out of all of the models.

But it also introduced human error into the results. A systematic approach would be to run a grid search for a large number of combinations of α and β, and report the best result acquired in this fashion. However, I only had my home PC to evaluate the models, I had to cut corners (excuse the pun).

The following table sums up the performances of the controllers and the neural network models on the four race tracks:

I need to explain the ⟨NN(.)⟩ model. Remember, the neural network was trained to output the next (steer, throttle), and 10 their values into the future. I’ve mentioned that the more complex model performed better, and that I “needed to keep a wary eye on overfitting”. I figured that I can reduce variance by introducing a low-cost ensemble by simply bookkeeping the predictions for time step t made in the previous 10 steps and then averaging them. That is: I kept predictions for time step t made at moments t-10, t-9, t-8, …, t-1, and t, and then averaged them.

In what follows, I’m focusing on those ⟨NN(.)⟩ models, since they seem to perform at least as well their “pure” counterparts.

The best controller is MPC but that’s no surprise. What’s interesting is that the ⟨NN(MPC)⟩ model performs only slightly better than ⟨NN(GP)⟩ even though GP performs much worse than MPC. It’s interesting to see how these two models control the car on race track “04”:

That’s smooooth! I really like the way the car slows down in critical moments, and rapidly accelerates after managing them. And remember, this is this model’s first time on this race track.

Now, the model trained on the game pad data:

Notice how wobbly the car behaves, and also: how on occasion the model decides to slow down for no apparent reason. There’s something “wrong” with this model, and I comment on it in the Discussion.

And, for completeness sake, let’s see how the ⟨NN(PD)⟩ model does on the same track:

There’s a suitable meme summarizing the performance of the ⟨NN(PD)⟩ model:

Discussion

It seems that the results for ⟨NN(MPC)⟩ are better than for NN(MPC), but in the case of ⟨NN(GP)⟩ vs. NN(GP) the gain is practically nonexistent. I conclude that the reason why ⟨NN(MPC)⟩ performs better than NN(MPC) is because NN(MPC) suffers from overfitting and the ensemble indeed helps reduce the variance of the model. On the other hand, because the ensemble didn’t help the NN(GP) model, it seems that overfitting is not this model’s main problem.

What is, then?

I’ve tried to reduce bias by crafting more and more complex additions to the architecture. There were many of them, too many to list here, but I should note: the reason why the final model is so darn complicated is a corollary of this search for bias reduction.

In the end, I failed to reduce bias, I don’t think that’s why NN(GP) is so wobbly and indecisive.

In his lecture (I highly recommend the whole course!):

Sergey Levine gives two good reasons for why training a model on data collected from a human expert is problematic:

1. Non-Markovian behavior;
2. Multimodal behavior.

The question is: why didn’t it prevent me from successfully fitting a model on the game pad data? There are two main reasons:
1. The test race tracks are not that different from the training ones.
2. I claim (I haven’t checked that, I would need to see how well a model trained on RGB performs) that the depth images help reduce the gap between training and test distributions (check out this lecture’s derivation for more details).
Still, I wonder if any of the architectural decisions (predicting future steps, in particular) helped alleviate at least the “multimodal behavior” problem.

Also, from a reinforcement learning perspective, in [2] we read (IL stands for “Imitation Learning”):

Overall, when using human demonstrations, online IL can be as bad as batch IL, simply due to inconsistencies introduced by human nature

Check out that article and also these materials provided by the good people at Georgia Tech — they are truly worth your while! And also, on the topic of this article, this is their architecture:

It looks like providing speed as input for steering predictions didn’t lead to the problems I’ve experienced. I’m guessing that if I spent more time tweaking the architecture, I would eventually get satisfactory results.

Model selection

Like in Part 1, I struggled with model selection. To illustrate my frustration, let me give you a puzzle.

Here are three scatter plots of the predicted vs. true steering angles, your task is to guess which of the models: NN(PD), NN(MPC), NN(GP), they belong to:

OK, I think we can all agree that NN(MPC) was the best model, NN(GP) was second, and NN(PD) was third. Based on that, it’s easy to guess that C is the NN(MPC) model, that’s the scatter plot with the highest correlation on the test set.

I found it a bit confusing that a model with scatter plot A was able to finish even a single lap. Let me explain: it seems that this model correctly predicts the sign of the steering angle (e.g. if the steer angle was less than zero the model generally returned a value less than zero), but the actual magnitude of the prediction looks a bit more like an output from a call to random.uniform(-1, 0) (actually, it’s not, the points are not distributed uniformly). It seems especially worrisome for critical situations, in which the car needs to perform a sharp turn, and from scatter plot A it would seem that in such situations the model can return a value close to zero.

Yet, scatter plot A belongs to the NN(GP) model, and B to the NN(PD) model. Even though instantaneous predictions are flawed, the NN(GP) model is able to beat corner after corner, and finally the whole track. But more then that, the NN(GP) model in reality (and by “in reality” I mean “in simulation”) performs significantly better than a model with higher correlation between true / predicted steer values — the NN(PD) model.

I should note that I selected the best model based on MSE on the test set (by the way, the test set in this context is not race track “03” and “04”, but rather laps around race tracks “01” and “02” not used for training). Instead, perhaps evaluating the models on whole corners (sequences of steering angles) would be a better strategy? I should look into offline policy evaluation strategies, because my current procedure has glaring flaws.

Future work and why use depth images instead of RGB

A high-level list of ideas I’m planning to explore:
1. Get the network to predict the speed, not throttle, and use a controller for throttle to achieve that speed (might not work if changes in speed require the controller to be able to adapt quickly).
2. Other / better controllers—here’s a good source of implementations; also POLO looks like an interesting avenue, but additionally: I have an idea (partly inspired by POLO) of supplementing the cost function in the MPC with “learnable” terms making the simple kinematic model a bit more realistic.
3. Find better strategies of evaluating the models offline.
4. Run CARLA in asynchronous mode.

And why use depth images rather than RGB? This is my miniature car:

I called it Karr (to pay homage to KARR) and it’s loosely based on the F1/10th specification. Those two sensors in the front—that’s ZED and Intel RealSense D435i—two stereo cameras that provide depth images. The processing unit is the Jetson TX1 with its own GPU, capable of fairly fast inference using neural networks.

My plan is to transfer knowledge from the simulation to the real world and to use a model trained in CARLA to drive around a race track with Karr. Other have already tried this using RGB data (see for example [3] and [4]), but I have a sneaking suspicion that depth images may suffer from a narrower “simulation-to-reality gap”.

But also, depth images, if treated like point clouds, allow for a range of informative linear transformations for augmenting data (like translations and rotations), providing a better representation of the data to a neural network (making use of the bird’s eye view but better), and localizing the car on the track.

I don’t know yet what I’m going to write about in Part 3, but there’s plenty to choose from:
1. Karr and domain adaptation: fighting the simulation-to-reality gap.
2. A more detailed look into CARLA and Unreal Engine 4.
3. Intro to robotics and depth cameras.
If you have a pick I would love to hear from you in the comments below!

References

[1] Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017). CARLA: An open urban driving simulator. arXiv preprint arXiv:1711.03938.
[2] Pan, Y., Cheng, C. A., Saigol, K., Lee, K., Yan, X., Theodorou, E., & Boots, B. (2017). Agile off-road autonomous driving using end-to-end deep imitation learning. arXiv preprint arXiv:1709.07174.
[3] Yang, L., Liang, X., Wang, T., & Xing, E. (2018). Real-to-Virtual Domain Unification for End-to-End Autonomous Driving. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 530–545).
[4] Bewley, A., Rigley, J., Liu, Y., Hawke, J., Shen, R., Lam, V. D., & Kendall, A. (2018). Learning to Drive from Simulation without Real World Labels. arXiv preprint arXiv:1812.03823.

Other cool resources

• This blog post is definitely worth checking — I can’t wait for the next part from Antonin Raffin, I am a fan of everything he’s put out so far.
• This blog post and the accompanying article about depth prediction (aka
  “depth sensing”) using monocular videos.
• Depth sensing at Berkley.
• And finally, definitely check out the CARLA Autonomous Driving Challenge.

If you enjoyed this post, please hit the clap button below and follow our publication for more interesting articles about ML & AI.

Introduction to the CARLA simulator: training a neural network to control a car (Part 2) was originally published in Acta Schola Automata Polonica on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction: the CARLA simulator is a platform for testing out algorithms for autonomous vehicles.
Motivation: I wanted to check out CARLA, build a simple controller for following a predefined path around the track to gather data, and then train a neural network model using that data.
Approach: I used a simple CNN architecture and modeled the steering angle.
Results: the model is able to drive the car around the track by controlling steering.
Project: https://github.com/asap-report/carla/tree/racetrack/PythonClient/racetrack

EDIT: in a more recent blog post (here) I’ve decided to use a much more lightweight environment than CARLA to train a model that I used to drive *faster* than the algorithm used for driving the car for data collection.

CARLA

CARLA is an open-source simulator built on top of the Unreal Engine 4 (UE4) gaming engine, with additional materials and features providing:

• a LIDAR
• a depth map (emulating a stereo camera, e.g.: ZED or Intel’s D435)
• semantic segmentation data
• two (and soon to be three) towns with pedestrians, other cars, traffic lights, speed limits, and realistic layout
• a repository and data for imitation learning
• a repository for testing out reinforcement learning models
• an option of running without display

But more importantly, CARLA is thriving; it has a great community, it’s documented well, and it’s (subjectively) easy to use.

Unreal Engine 4

One of the reasons I became interested in CARLA was the fact that it’s built using UE4. I wanted to hone (or rather: keep alive) my C++ skills, and at the same time tinker with machine learning for autonomous vehicles, so CARLA was the perfect combination of these two worlds.

But something I didn’t anticipate is that UE4 allows for developing quite advanced features without the necessity of implementing them in C++. For that, UE4 has a dedicated visual scripting language called Blueprints. I’ve used Blueprints for the creation of the race track (more details further).

Alternatives to CARLA

There are several open-source simulators for autonomous cars and/or racing, but very few with capabilities comparable to or greater than CARLA’s. Here’s a list of those that I’ve considered (with their respective environments in parentheses):

• Autoware (Gazebo)
• AirSim (UE4 & Unity)
• TORCS (OpenGL)

And two simpler ones:

• the Udacity simulator (Unity)
• Donkeycar simulator (Unity)

In the end, I only considered CARLA and AirSim since I wanted to work with UE4, and chose CARLA because it seemed easier to contact with the community via their discord chat.

The race track

To build the race track I took the Town02 map, deleted all props (as suggested here), and thus prepared the scene for the race track.

The track is composed of RoadMesh elements that look like this:

But to be able to create a smooth track I had to incorporate this element into a blueprint script called Track Generator. The race track is then created like this:

The race track on which I’ve actually rode looks like this:

I wanted to have sections of: tight corners, standard corners, and a couple of long straights. But, full disclosure, I have no idea what makes a track better or worse. For my purposes, this race track was perfect.

An interlude: a tour of the project

I’ve forked the carla-simulator repository, branched off from the stable version (release 0.8.2) into a branch racetrack and created a directory carla/PythonClient/racetrack where I keep all the necessary scripts for reproducing this work.

The two most important scripts are: client_controller.py and train_on_depth.py, where the first one is used for interfacing with CARLA, for controlling the car and for gathering data, whereas the second is used for training neural networks on the collected data.

Before running the client_controller.py script, remember to first initialize CARLA. On my system it requires the following:

cd ~/CARLA
./CarlaUE4.sh /Game/Maps/Racetrack01 -windowed -carla-server

After which, to control the car using a proportion-derivative (PD) controller (more details below), I call:

python3 client_controller.py \
   --speed 45 \
   --controlled_name pd

And the result should look more or less like this:

We’ll get to the train_on_depth.py script in a minute.

The simplest possible controller

Based on the experiences gathered from the Udacity course, I knew one thing: I didn’t want to control the car myself. It would require multiple data collection sessions, would quickly become tedious and time consuming, and I would inevitably make mistakes that would require editing.

To reach my goal (training a neural network capable of going around the track), I started with the simplest controller that would collect the data for me. And it doesn’t get much simpler than the PD controller:

in which the steering angle at a given time t depends on the Cross-Track Error (CTE), and the time derivative of the CTE. The CTE is the distance between the center of mass of the car, and the desired trajectory. Also, the controller needs to be calibrated, i.e. it’s up to us to choose the coefficients kp and kd such that the car drives (relatively) smoothly. (The fact that Medium doesn’t have support for LaTeX is a pain to say the least.)

Admittedly, this is not the simplest possible controller because, as you might have thought, one might try a “P only” controller that ignores the second component. And yes, this is an even simpler controller but the dCTE/dt component plays an important role of stabilizing the output. Without it, the controller quickly becomes erratic:

so it wouldn’t get the car around the track.

Throttle

The steering angle is not the only actuator we have at our disposal, there is also throttle. (And a handbrake, but we’re not going to touch it… yet.) But what we’re actually interested in is not throttle, but the thing it influences — velocity.

In the first iteration of this project I didn’t really care for velocity, I just wanted the car to go forward; not too fast, not too slow. To that end, I used a simple clip_throttle function that keeps the throttle within reasonable bounds, and tries to keep the speed close to the target_speed:

Notice that this results in a wide range of throttle values, which in turn results in high variance of the velocity. But that’s OK; my reasoning was that this “augments” the data set and yields a more generalized model.

Technically, you could take the time and build a separate PD controller for the throttle so that a target speed is kept fairly constant. But this would require its own (kp, kd) parameters, making the model slightly more complicated.

But more importantly, in the second blog post we’ll use a controller that controls both actuators, steering angle and throttle, simultaneously. This approach is called Model Predictive Control (MPC), and is admittedly more complicated, but it provides a trajectory that is much more efficient. And if this didn’t whet your appetite for the second blog post, here’s a video illustrating how an MPC-controlled car drives around the track:

Right away you can notice that the frame rate is much lower. This is because MPC requires some heavy optimization to be done on the fly, and my implementation of this controller is written entirely in Python.

Building a neural network

Partly inspired by this blog post and partly dissatisfied with the performance of a neural network model trained on pure RGB input (done as part of the Udacity Self-Driving Car Nanodegree), I’ve decided to train a model on the depth map data that can be easily collected in CARLA. This is how the world looks like when seen through the depth map lens:

Admittedly, it’s pretty pixelated and a bit unclear. But note that this representation carries 3D information that is essential for driving the car along the track.

I used keras for training the neural network, and used a fixed, singular architecture. I prefer raw code instead of fancy visualization tools, so here’s the definition of the model:

I didn’t spend much time tweaking the architecture or the optimization parameters. I chose “reasonable” hyperparameters, and instead focused on modifying the input and adding auxiliary losses. But more on that in the second blog post.

The model had one job: predict the steering angle for a given depth map frame. It didn’t make much sense to predict throttle because it was independent of what was in front of the car (at least for our PD controller; this will change in the second blog post). So the throttle was also modulated using the clip_throttle function.

To train the model, I ran:

python3 train_on_depth.py -c pd

I had about 90k frames and the net took about 2h to train, but that’s because: 1) the model was over-the-top-complex, 2) I didn’t shrink the input image, and 3) the default number of epochs was 50. However, the MSE on the test set quickly became flat:

And in reality, starting with “Epoch 2” the scatter plot of predicted vs actual steering angles looks pretty much constant:

This is how the model with the lowest MSE on test set managed to drive around the track:

I think the coolest thing is that the neural network model seems to stabilize the oscillations better than the PD controller.

Problems

The biggest problem I haven’t yet solved is model selection. The MSE was insufficient, because it wasn’t always the case that the model with the lowest MSE on the validation set was the best. One way of dealing with this is to validate the model automatically in CARLA, without display. But I would like to have a more general approach, one that I could apply in the real world.

Also, I’ve noticed that models with the lowest MSE failed to steer the car out of tight turns. That is, high (but rare) values of the steering angle were never produced by such models in the simulator (even if on the test set the model seemed capable of yielding those steering angles). So I’m inclined to evaluate the performance of the model in two types of scenarios: normal, and extreme situations. But that’s for the future.

Introduction to the CARLA simulator: training a neural network to control a car (Part 2)

Be sure to check out the second part of this series. It has more content (an euphemism for: “it’s longer”), and takes a look at different sources of data, created by different controllers, including a game pad!

Training a neural network for driving an autonomous RC car

There’s also an “unofficial” Part 3 which utilizes the exact same machine learning pipeline used for this project, but the neural net is steering an actual mobile robot and the racetrack is a path around a pond that’s next to my house.

References

Unreal Engine 4: this course on Udemy is a great intro but only if you’re into C++. Also, it’s recommended by the creators of CARLA.

Race track: For building the race track I followed this tutorial: youtube link. Also, I found this part from the UE4 documentation valuable. Actually, the whole UE4 is documented very well, here’s one of the gems I intend to explore.

If you enjoyed this post, please hit the clap button below and follow our publication for more interesting articles about ML & AI.

Introduction to the CARLA simulator: training a neural network to control a car (Part 1) was originally published in Acta Schola Automata Polonica on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

The F1/10th is a racing competition for autonomous, electric car models. The organizers (University of Pennsylvania) encourage the participants to develop the fastest (yet, reliable) algorithm, capable of beating a track. The hardware parameters are capped by the rules, and most of the work needs to be devoted to software.

Here’s a short summary video from the event:

The winner of the 3rd F1/10th challenge was chosen based on the number of laps made by the model during two timing sessions. The first sessions lasted 5 minutes, while the second one was shorted and took 3 minutes. If the car needed any kind of assistance, or damaged the track during a lap, the team was required to pause the lap, go back to a previous checkpoint, and continue the lap from there. If the number of laps of two teams were the same, the team with the fastest lap was considered better.

The track

This edition’s track was quite challenging, with multiple turns that proved difficult to all of the teams:

The track started with a simple left turn, followed by two consecutive U-turns. The second U-turn (1) was especially difficult — the cars either tried to go back on this turn, or cut the corner too aggressively.

After beating turn (1) there was a long straight that ended with a tricky turn (2). In the beginning, our car experienced a bit of oversteer, and even turned around. Eventually, we limited the speed of the car in the preceding straight, and narrowed the LIDAR’s scan so that the car wouldn’t be lured to go back.

Then, there’s an easy, slow left turn, followed by the longest straight on the entire track, ending in a left turn (3). On the one hand, we wanted the car to go as fast as possible on this straight, but on the other hand not too fast, so as to fit in the (3)rd corner.

Also, the surface on the entire track was somewhat slippery which also added to the difficulty of the race.

Here’s a video from the first round of the competition (our run starts at 43:00):

We finished 16 laps, and the fastest took 18.29 s.

The simulator

The organizers shared a Gazebo simulator for prototyping and testing out models. And although there’s a huge gap between how the car behaves in the simulator vs. how it behaves on the actual track, the simulator proved to be a valuable tool for trying out new ideas.

The team and the car

First off: we had very limited time to prepare for the competition. Keep that in mind, and if you’ll run into question like: “Why did they use RaspberryPi instead of the Jetson?” the likely answer is: “We were in a hurry and had to cut corners (pun intended).”

The team members were:
- Łukasz Sztyber
- Karol Majek
- and myself
Three guys passionate about self-driving cars. And that’s all you need to take part in the competition. OK, that’s not all you need — the car is pretty important too.

We started off with the Erle Rover platform, which came with a RaspberryPi3 as the computational unit, and a very simple RGB camera. We first tried to work with the camera but that led to nowhere (even a simple deep learning line-follower model was challenging to train). For one thing, the inference time was too long. But more fundamentally, it became clear that a deep learning approach would not generalize to new tracks, and in particular: the track we would need to beat at the competition. (Which is not to say that a machine learning approach is a dead end — it’s not. But I think it needs to be used for learning “things” that generalize to other tracks.)

Then Łukasz bought the recommended LIDAR, and even the simplest approaches based on a LIDAR seemed promising. It became clear that it’s possible to build a model without SLAM, without path planning and control. This was crucial since at that point we had very little time remaining before the competition started, and we knew that only simple solutions would be achievable.

Once we became more serious about the competition, Łukasz invested in a platform which he dubbed Dzik (pronounced “j ee k”). We used the RaspberryPi3 from the Erle Rover as the computational unit. The organizers recommended NVidia Jetson TX-1, or TX-2, which was faster, but we wouldn’t have time to set it up, configure, and adapt to the rest of our hardware. We stuck to RaspberryPi3, and allocated what time we had left to improving the solution.

The solution

Our solution was a variant of the “follow-the-gap” approach for steering, combined with a set of conditions determining the speed of the car. For example, if the optimal direction (the “gap”) is straight ahead, and the distance to the obstacle in that direction is beyond a certain distance (which was one of many parameters of the method), then: go fast. How fast exactly was also controlled by a parameter, and needed adjusting.

Also, it wasn’t actually speed that was controlled, but rather, a Pulse-Width Modulation (PWM) signal passed to the Electronic Speed Controller (ESC), which in turn ran the engine at a particular RPM corresponding to that PWM value. Although we had odometry and tested out a PID controller for speed, it turned out that the ESC was able to reach a stable RPM for a given PWM pretty quickly, so there wasn’t much gain from a PID for speed. That’s all pretty technical, but the gist of this whole paragraph is: we had three PWM values, and we assumed that they correspond to three speeds, and that the engine can quickly switch between these speeds.

In addition to the rules regarding speed, there was a set of rules for stopping. It’s impossible to count all the instances during practice in which we were certain that the LIDAR took the final blow, and that we won’t be able to run in the competition. Therefore, our first line of defense against ‘the blackest scenario’ was a set of conditions in which Dzik just stopped.

Karol says that effectively the rules for going slow/medium/fast form a decision tree. We didn’t train its parameters using observable data, but rather fit it to our intuitions. It’s a bit of a stretch, but you could say we adapted reinforcement learning with an experience-driven optimization algorithm :)

But more seriously, we could have used supervised machine learning, and: record several laps (including minor mishaps) by controlling the car ourselves, fit a machine learning model, validate it on another set of recorded laps (paying close attention to how the model behaves on mishaps), and expect to get a better solution then those hard-coded conditions we used. However, we didn’t go this path, and it’s hard to tell how many problems we would have encountered. In particular: how many laps would be needed? How much time would we need to adapt to a new track? And perhaps the most important unknown: how hard would it be to debug such a model? Remember that our final validation is on the actual track, and if we observe that something goes wrong, it’s very important that we know how to improve upon it. But here, what more can we do other than “get more data”? This is a problem in and of itself. And that’s just speed control, I haven’t even mentioned steering.

So in other words: using an adaptive model seems cool, but it has its drawbacks. Still, it doesn’t mean we won’t try it.

Limitations

Fundamentally, even if our method found the optimal steering angle locally, it wouldn’t be able (in general) to yield the optimal route globally.

To follow the optimal route, one needs to solve the problems of: localization, planning, and control. Such an approach is admittedly much more complicated, but it holds a promise of reaching a systematic solution, one that can become increasingly better, as the solutions to the individual sub-problems improve.

But a more important reason for not using the follow-the-gap approach again (at least not in its existing form) is actually more simple. Namely: we would like to try something else.

Strengths

The follow-the-gap approach is fast. The LIDAR (with nominal frequency of 40Hz), produces a new scan every 25ms. The C++ implementation was able to finish its computations in approximately 3ms (on average). That’s quite important during a race, where each stalled millisecond makes the scan increasingly obsolete.

But there’s also another strength: this method is fairly easy to adapt to a new track. The parameters controlling its behavior are intuitive (once you get to know what they actually control), and finding their optimal values for a new track is a matter of several test runs. (Actually, the track in Torino was pretty challenging, so it took 20–30 runs, but still.)

There’s one more strength, but we haven’t really made use of it during this competition. And that is: the follow-the-gap approach can automatically handle moving objects on the road, and avoid them or even overtake them (if it’s an opponent). It may not seem like that big of a deal, but the alternative, systematic approach will probably need more attention in this particular area. And because the time needed to make a decision would inevitably be longer, the algorithm would need to anticipate future positions of the moving objects. So it might get interesting.

The last year’s winners (from Czech Technical University) also used a version of the follow-the-gap approach. And that makes you wonder: perhaps this method is indeed superior to the more systematic ones? Maybe the drawback of following the optimal route is outweighed by the benefit of making locally optimal decisions very quickly? I think the systematic solution is difficult to get right, and that’s one of the main relative advantages of the follow-the-gap approach. But how long will it hold — hard to say.

Plans for the next competition

…are top-secret and I can’t share them here.

But follow the f1tenth.org forum, and meet us at the next competition, and we will gladly share them with you. Right after the race, that is ;)

References

• f1tenth.org

If you enjoyed this post, please hit the clap button below and follow our publication for more interesting articles about ML & AI.

Experiences and thoughts from the 3rd F1/10th competition was originally published in Acta Schola Automata Polonica on Medium, where people are continuing the conversation by highlighting and responding to this story.

Written by Maciej Dziubiński and Marta Stępniewska-Dziubińska.

Disclaimer: this blog post and the whole project was done “for fun” — it doesn’t really serve any purpose other than sharing something we think is a cool project. From the Reader’s perspective, there are two main merits coming from this post:
1. Seeing how FaceNet can be used to recognize people (or, rather, animated characters) in a one-shot manner;
2. Learning how to stitch FaceNet with another “building block”, namely: with the memory module proposed by Kaiser et al.

Additionally, if you haven’t had much experience with TensorFlow, the accompanying repository and its Jupyter Notebook may also be of value to you.

We’ve tried to keep this blog post accessible to the casual Reader, which means that we’re not going through the implementation of the models, the memory module, or any other (overly) technical details. We hoped to capture the Reader’s attention, and if that succeeded, provide him with all implementational details needed to re-create the experiments. So keep that in mind, and check out the implementation if you find this project interesting.

Introduction

OK, so FaceNet is incredibly good at face recognition of real human beings. That’s great and all, but is it capable of recognizing faces across domains: CGI, painted, drawn, etc.? We asked ourselves this critical question, and the short answer is:

The set-up
Imagine that you’re organizing a venue in a VR world (similar to that in “Ready Player One” or in another book adaptation: “The Congress”) where each participant chooses an avatar. Let’s say that, for whatever reason, you would like to register, and then recognize the participants based on their animated faces using some standard face recognition model.

This might be tricky since most face recognition models are trained on actual (human) faces, so even if the task of recognizing avatars is easy for us (humans), it might be hard for a model. The source of the problem is the difference between the domain on which the model was trained on, and the new, animated domain for which we would like to use the model. For more on this topic we refer the Reader to this blog post, but for the purpose of this project it’s only important to note that: it’s hard to know beforehand how a face recognition model will perform on animated faces.

But as it turned out, we didn’t have to do any domain adaptation —FaceNet performs pretty well without any additional work.

FaceNet

is our face recognition model of choice. As mentioned, it performs very well on real faces (reaching an accuracy of 0.99650 on the Labeled Faces in the Wild dataset).

FaceNet builds on the Inception ResNet v1 architecture and was trained on the CASIA-WebFace and VGGFace2 datasets. It’s worth noting that FaceNet’s weights are optimized using the triplet loss function, so that it learns to embed facial images into a 128-dimensional sphere. Images whose embeddings are close in this space are expected to correspond to similar faces. If you want to learn more, we refer you to the paper and if you really, really want to learn more, check out its TensorFlow implementation.

We won’t go into details concerning FaceNet’s architecture, but for the purpose of the discussion below, it’s important to highlight one particular nuance of the Inception architecture. Namely, it has a train phase (True or False), which controls whether Dropout and BatchNorm are turned ON (True) or OFF (False). Dropout is not that big of a deal, but for those of you who haven’t yet fallen into the pitfall of re-using a model with BatchNorm, here’s a good reference. We’ll see that in our case it’s better to turn the train phase OFF.

The memory module

We’ll use the memory module proposed in the Learning to Remember Rare Events paper by Kaiser et al. Roughly, the memory is composed of three elements: 1) a matrix with rows corresponding to embeddings of observations, 2) a vector storing the labels of the observations stored in this matrix, and 3) a vector of ages of those observations.

Point 1) is not entirely true: a single row may also result from combining two (or more) similar observations, which is done auto-magically by the memory module. This will be relevant later on, when we’ll argue that one of the merits of the memory module is that it requires storing fewer embedding, and thus: less memory.

Methods and Data

Data

Our dataset is tiny, comprising 12 characters, each with 8 distinct images (drawn, painted, etc.), which yields 96 images. Here they are:

You might have noticed that the 12 characters are grouped into 6 pairs of similar faces, i.e.:
* Geralt vs. Vesemir (two witchers, as they were “portrayed” in The Witcher 3);
* Malczewski vs. Witkacy (two influential Polish artists);
* Heroin and hero of the Avatar movie (we didn’t really remember their names, so we named them avatar_female and avatar_male);
* Gollum vs. Smeagol (from the movie adaptation of the Lord of the Rings);
* Durotan vs. Hulk (from the Warcraft movie, and the Marvel Cinematic Universe);
* Thade vs. Cesar (from two versions of the Planet of the Apes; Thade from Tim Burton’s adaptation and Cesar from the reboot series).

Note, that Gollum and Smeagol are actually alter egos of the same character so we expected the model to struggle with these two. But on the other hand, humans are pretty much able to discern between the two just by looking at their expressions, so why not put FaceNet to test?

Another note: Witkacy was represented by 4 painted/drawn pictures and 4 photographies. We’ve expected this to be a problem, but as we’ll see in a minute —FaceNet was resilient to this obstacle.

FaceNet

We used the 20170512–110547model. We have also tested out the newer models (20180402–114759 and 20180408–102900) but got better results with the older one.

FaceNet was used to produce embeddings for the 96 images, and if we use the dot product between these embeddings, we can summarize the similarity between the images with a matrix (row and column indexes are grouped by characters):

Because the embeddings are normalized, the dot product is equivalent to the cosine similarity. The dark blocks along the diagonal represent high similarities between faces coming from the same character. But notice also that Gollum and Smeagol together yield an almost homogenous, dark block, which is in line with what we anticipated: these two will pose a challenge to the model.

Memory

In an upcoming blog post we’ll go through the implementation of the memory module, but here we’ll only mention several of its most important features. First: there is a “bookkeeping” aspect of the memory which is not driven by gradient propagation, but rather in a “insert-update-delete” fashion. Second: the gradient of the loss function is propagated to the model providing the embedding (FaceNet, in our case). And lastly: the prediction is done by finding in the memory matrix the best matching observation (in a 1-nearest-neighbor fashion, using the cosine distance), and returning its corresponding label.

K-shot learning and evaluation

We iteratively trained and evaluated the model. In the first iteration, the model was trained on 12 images, one face per character. Next, the model was evaluated on the remainder of the images. On the second iteration, the model was fed another batch of 12 images, then evaluated on the rest, and so on, and so forth. Actually, there’s also a 0-th iteration, where the model starts off completely blind, without seeing any of the characters, which, as expected, results in an accuracy of about 1/12 = 8.33%.

Two models

We will compare two models, both based on the embeddings produced by FaceNet. The first one will utilize the memory module and will be called “FaceNet + memory”. The second one will be simpler: it will store all embeddings and their corresponding labels, and produce a prediction in a 1-nearest-neighbor fashion. We will call this the “FaceNet alone” model.

The “FaceNet alone” model may at first seem remarkably similar to the “FaceNet + memory” model. But it differs in two respects: 1) this model won’t modify the embedding model (i.e. FaceNet), and 2) because it will store all previous embeddings, it will require more, well, memory. The memory module uses a fixed-sized matrix to store the embeddings, and our default number of rows was 32 (less than the number of images). Note that, because the “FaceNet + memory” had a cap on the (RAM) memory, its task was a bit harder. Let’s see how it influenced the performance.

Results

K-shot accuracies

Finally, let’s see the k-shot accuracies for the “FaceNet alone” and the “FaceNet + memory” models:

That’s what we initially saw. But we didn’t expect the memory module to be that much worse, so we asked ourselves: what are we doing wrong?

And it’s not that we had a bug in the code, but rather that we forgot to pay attention when applying transfer learning with a pre-trained model that uses BatchNorm. Once we turned the train phase OFF, the results were as follows:

Consequently, FaceNet’s weights are updated without the correction for internal covariate shift (partially amended by BatchNorm), and without the ensembling provided by Dropout. Is this OK? Can it be done better? We are open to suggestions so if you have one — please, leave a comment.

To see what kind of mistakes the models made, we compiled two confusion matrices:

The rows of the confusion matrix represent actual labels, and the columns denote predictions. Bu default, the 0-shot prediction of the “FaceNet alone” model is a uniform random choice between the labels of all characters. On the other hand, the 0-shot prediction given by “FaceNet + memory” is “geralt” (class 0) for every image, which corresponds to a black column in the confusion matrix.

For details regarding the implementation, the default values of hyper-parameters, and everything else, we refer the Reader to the accompanying repo.

Discussion

We will discuss the “FaceNet alone” solution first, and then move on to “FaceNet + memory” .

FaceNet alone
This approach is doing a pretty good job, with 83% accuracy right off the bat, after seeing each character just once. It even reaches 100% accuracy, which means that it was able to distinguish (among others) between Gollum and Smeagol.

From the evolution of the confusion matrix we see that the model struggles with discerning between Gollum and Smeagol, but has also problems with another pair: Hulk and Durotan. Still, that’s pretty much it, the rest of the character pairs are no match for this simple 1-nearest-neighbor approach based on FaceNet’s embeddings.

FaceNet + memory
Obviously, we would really like to sell this method like this:

but the Reader will benefit more from a self-critical approach.

The accuracy is lower than that of “FaceNet alone” but bear in mind that there’s a bit of variability in the results. If we were really fussy, we would repeat the experiment many times, bootstrapping (or rather: shuffling) the images for each character. Also, we might have spent more time tweaking the hyper-parameters. But we left that as an exercise for the Reader.

The evolution of the confusion matrix indicates that, again, Gollum and Smeagol are a problem, but more surprisingly the model mistakes Durotan (an orc, mind you) for Vesemir (a witcher). But, conversely to “FaceNet alone”, starting with the 3rd shot, there are no misclassification between Durotan and Hulk. This is weird because from what we saw in the similarity matrix, Durotan has relatively low similarity with Vesemir (at least in comparison with Hulk). We therefore suspect that this is an artifact of this particular ordering of the input images.

Now, what can be said for sure is that “FaceNet + memory” required less memory than the “FaceNet alone” model. Roughly three times less memory. But if each character occurred more frequently, the gain would have been even higher. Going a bit further in that direction: the “FaceNet + memory” is prepared for a life-long learning path, batteries included, whereas the “FaceNet alone” would require unlimited memory.

And as the learning path of “FaceNet + memory” progresses, the model adapts to the domain in which it’s being applied. Our dataset is too small to make use of this property, but it might come in handy in a more demanding environment.

There are, however, two limitations of the memory module that we wish to address in the future:
1. The prediction is made based only on the closest observation which might increases the variance of the model (in the sense of the bias/variance tradeoff);
2. During training, a row in the memory matrix may be updated by combining two embeddings: the one already in the memory and another, coming from a new, but similar observation. This “combining” boils down to adding the two embeddings. However, this way we loose information about the distributions of the embeddings.

Conclusions

We evaluated two approaches to k-shot face detection based on embeddings acquired with the FaceNet model. The first one was based on a memory module proposed by Kaiser et al., the second was a simple 1-nearest-neighbor approach.

Our dataset was composed of 96 facial images of 12 (mainly) CGI characters from movies, video games, and art — domains which FaceNet was not originally intended for.

Both models reach an accuracy of 100% after “seeing” each character 7 times, although we stressed that this result may be noisy. However, this proves that the embedding learned by FaceNet is robust and that it can be transfered across domains, e.g. for face recognition based off of portraits done by criminal sketch artists.

If you would like to learn more about the memory module, make sure to check out the paper and the implementation. We’re also working on a blog post in which we go through the implementation using the eager execution in TensorFlow, so stay tuned!

References

• Kaiser, Ł., Nachum, O., Roy A., Bengio S. (2017). Learning to Remember Rare Events. https://arxiv.org/abs/1703.03129

If you enjoyed this post, please hit the clap button below and follow our publication for more interesting articles about ML & AI.

FaceNet with memory vs. synthetic faces was originally published in Acta Schola Automata Polonica on Medium, where people are continuing the conversation by highlighting and responding to this story.

Written by Maciek Dziubiński and Bartosz Topolski.

If you haven’t read the first part of the tutorial, you can check it out here.

tl;dr We continue the development of supervised models for user verification based on behavioral data. In this part, we’ll focus on neural-network-based models, and finalize this series by looking at performances of all four models on the test set.

This tutorial follows this notebook, so it makes sense to open it now, hit “Cell -> Run All”, and then go back and forth between this post and the notebook.

2. Multi-Layer Perceptron

The Multi-Layer Perceptron (MLP) is an example of a wide family of models called neural networks. It’s a simple example, but because of that, it’s worth taking a slightly deeper look into, before moving on to siamese neural networks.

MLP is a stack of subsequent layers of neurons (hence the name). The picture below depicts a small, three-layered network. MLP is a type of the so-called feed-forward architecture, i.e. each neuron takes as input activations of all neurons from the previous layer, processes it in a certain way (we’ll look into that in a moment) and is then used as input by the next layer.

So, what does a neuron “do” exactly? Let’s assume that our input data is a vector x. Each neuron is calculating a dot product of the input vector x and this neuron’s vector of weights w — that is, multiplying each input coordinate by a weight and summing it all up to a single number. Then, a value called bias is added to the dot product, and finally, the result is transformed by some non-linear function, f.

Since there are multiple neurons in each layer, we can represent the layer as a matrix, with columns corresponding to weight vectors of individual neurons. The biases are, on the other hand, represented as a vector, b. Those weights and biases are what makes a model provide (hopefully) valuable predictions, and finding them is the main goal of training a neural network model. Using linear algebra’s notation, we can represent our whole model in just a few simple equations seen next to the visualization of a network:

Thanks to the non-linearity of the activation functions, our model can estimate almost any mapping from inputs to output. That is, if the number of layers and neurons in them are sufficient. The only thing we have to do now is finding the optimal weights and biases for each layer (collectively called parameters).

As in any machine learning problem, we train the model by minimizing some loss function on the training set. We know that the loss function depends on predictions from our model, but using the equations mentioned earlier we can also see that predictions depend (more indirectly) on weights and biases.

Because we can define our loss as a function of the parameters, we can also differentiate it with respect to those variables — and that gives us information about how the loss functions change as we change the parameter values. We can then decrease the loss function iteratively, by making tiny updates to the parameters, proportional to the derivatives of the loss function with respect to those parameters.

However, there is much more to training neural networks, and machine learning in general, than just minimizing the loss function on the training set (although this alone can be a challenge). We would like the model to generalize well to new, unseen data. This is one of the practical differences between neural nets and the previously mentioned XGBoost model — they’re not as good “out-of-the-box”, and typically require more effort to make them generalize well.

Another difference is that we need to standardize the input feature values. This is done using the RobustScalerForPandas (defined in code cell [20]), which is a wrapper that makes sure that the transformed frame is a pandas.DataFrame. This wrapper inherits from the sklearn.preprocessing.RobustScaler, and it’s interesting to check if there are scaling strategies leading to better results.

The MLP model is defined in code cell [21]; its architecture is specified by parameters passed at initialization, but the activation functions and the optimization procedure are hard-coded. Also, note that there’s a considerable class imbalance in our data set, so maybe it might be worth playing with the class_weight argument passed to the model.fit method?

But even without much tweaking of the hyperparameters, the MPL model performs pretty well — we get AUC scores of about 0.90, which is pretty similar to what we got using XGBoost.

3. Simple siamese neural network

Let’s move to code cell [24], where we can see how the input data frame looks like when estimator.only_feature_diffs is False. Note, that the custom_cross_val_auc_score function actually calculates this frame internally, and we’re not going to use the DF_train_all_pairs variable for anything other than seeing how it looks like. Each column (besides the label column) either ends with an “_A”, or with a “_B”. These suffixes signify to which part of the input a given column belongs to in the siamese model:

Input A and Input B correspond to two keystroke dynamics observations. Then, both inputs are processed by a three-layered neural network, whose activations on the last hidden layer (Hidden layer 3) are then used to calculate their absolute difference (L1 merge layer for short), followed by a single hidden layer (One more hidden layer), and finally leading to the Output layer which returns a value between 0 and 1. That’s how we get our estimate of the probability that the two sequences, described by Input A and Input B, come from a single user.

There are two important notes here. First note: the three-layered networks built on the Inputs are actually two “references” to the same network. That is, these two legs share the same weights — if we were to modify any part of the left leg, the right leg would change as well. Second note: the order of observations supplied as Inputs does not matter, because the L1 merge layer performs a symmetric operation. The symmetry refers to the arguments, i.e. any op such that op(A, B) = op(B, A) would also work. In fact, you can play with this op, and use, for example:

def l2_diff(x): return K.pow(x[0] - x[1], 2)

which corresponds to op(A, B) = (A - B)², but be aware that the “-” and “( )²” operations are performed in a vectorized manner.

Note that this time, we set the only_feature_diffs attribute to False since we want the custom_cross_val_auc_score to use a data frame with actual feature vectors, and not an absolute difference between them.

On validation, the simple siamese model seems to perform OK (again, the AUC is ~0.90). But keep in mind, that we still need to measure the performance of the model on the test set.

4. Siamese recurrent neural network

We now move to code cell [30], where we train another siamese model:

but with a twist: instead of a three-layered siamese legs, we use recurrent units, operating on sequences rather than on handcrafted features. We would like these recurrent units to learn features as valuable, or better, than aggregated statistics of dwell and flight times.

Obviously, we might get better results if we merged the activations from the GRU with an MLP working on our standard features (remember? the mean and std of the dwell times, etc.). Or even if we used dwell and flight times sequences themselves. But that’s not what we had in mind — we would really like the model to learn the features directly from the sequence, without any expert knowledge. It so happens that we knew that dwell and flight times are indeed valuable characteristics, but what about other types of behavioral data, e.g. mouse movements?
We know that’s utopian thinking, but we believe that some day we’ll be able to replace the skill of coming up with great features (typically pertinent to one, narrow domain) with the skill of coming up with general-purpose neural network architectures, capable of extracting even better features from (almost) raw data.

The AUC scores on validation are above ~0.80, but slightly worse than previous models. There are two TODOs in code cell [30] with suggestions for improving the model. (I quick thought: “Two TODOs” would be a great name for a band! No? Never mind…)

But that’s not all; you can:

• experiment with a different recurrent units (GRU is hard-coded in the
  RNNSiameseModel._create_base_network method defined in cell [29]);
• change the number of neurons of the recurrent unit (again, it’s hard-coded);
• increase the number of epochs;
• add regularization;
• surprise us with something else entirely.

Results on the test set

OK, we have our best_*_pipe models trained and ready, we can test them on the non-registration data, for user specified in the test_user_names list at the beginning of the notebook.

Scroll to the code cell [31], where you’ll find the test_user_verification function, which takes care of testing, and produces a list of AUC scores for the verification process. You can go through the function to fully understand its mechanics, but what’s important is that:

• this time, we use the registration sequences as “templates”;
• for a new (non-registration) sequence, we pair it up with every registration sequence for that user;
• for each pair we take the model and estimate the probability that it comes from the same user;
• finally, we take the mean of the predictions, and take that as our final prediction of the new sequence belonging to the legitimate user.

Results for our best_*_pipe-s can be produced by running cells from [33] to [36], and behold:

1. XGBoost has the highest mean AUC (0.9880);
2. The MLP model is second (0.9856);
3. Simple siamese model is third (0.9272), but one user had “definitely wrong” predictions;
4. And our most complicated model is fourth (0.8907).

At first glance, it seems that simplicity pays off. But notice that XGBoost is far from simple, and it wouldn’t perform as good if we didn’t know that dwell and flight times are valuable for verifying users. If we used only the aggregated statistics of the first order difference, XGBoost’s mean AUC score would drop to ~0.91, and that’s not that far from the performance of the siamese recurrent neural network.

Now, on the other hand, we can give you a hint that a single change to the RNNSiameseModel can pull the mean AUC score on the test to ~0.94 (without adding dwell/flight times). Can you guess what is it?

And more importantly: can you achieve an even higher score with the RNNSiameseModel?

Parting words

If you’ve managed to go through the whole blog post — congratulations! It was a long one, but hopefully it was worth it.

If you have any questions regarding this blog post, add a comment. But if it’s more code-related, we would appreciate it if you opened an Issue at the workshop repo.

And of course, if you manage to enhance the siamese architectures, you can fork the project, and provide a link to your version in the comments for everyone to see!

About Nethone

Nethone creates A.I.-based anti-fraud and business intelligence solutions that help enterprises from all around the world turn threats into well-informed, profitable decisions. Founded in 2016 by experienced data scientists, security experts, and business executives, Nethone is one of the fastest-growing FDP companies in Central Europe.

If you enjoyed this post, please hit the clap button below 👏👏👏

You can also follow us on Facebook, Twitter and LinkedIn.

Building Supervised Models for User Verification — Part 2 of the Tutorial was originally published in Daftcode Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction to the data set and validation

Written by Maciek Dziubiński and Bartosz Topolski.

tl;dr We’ll develop four supervised learning models for deciding whether a pair of behavioral patterns comes from the same user. Those models will be: 1) XGBoost, 2) MLP neural network, 3) simple siamese neural network, 4) siamese recurrent neural network.

We recently held a workshop at the “PL in ML” conference entitled “User identification based on keystroke dynamics”. The workshop introduced the problem of verifying a user with the help of behavioral data collected during typing in a password on a keyboard. Materials from the workshop are available here but you can run the code right away by the grace of this new, hot service called MyBinder. The idea is that you go through the notebook as you read this blog post, so it pays off to click “Cell –> Run All” pretty much now.

This blog post serves as a high-level intro to the problem, an overview of the models, and a commentary on what more can be done to improve the models. But more importantly: the purpose of the workshop (and this blog post, as a consequence) is to give you, the Reader, a piece of working code that you can play with, modify, enhance, and run as easily as possible.

The set-up

A seamless user verification system would significantly improve the security of our e-accounts, without sacrificing any of their conveniences. The idea is to reinforce the standard user verification system (typically: a password) with a system that verifies whether the user’s behavior matches that of the legitimate owner of an account.

Suppose that you’ve collected behavioral data of every user during registration. Now, some time after the registration someone logs into the account. We would like to be able to say whether the behavioral pattern of this “someone” matches the behavioral pattern of the owner of the account.

We’ll focus on one particular type of behavioral data, namely: the so-called keystroke dynamics, which are sequences of timestamps corresponding to push/release events recorded during typing. We will train four binary classification models, which for a pair of keystroke dynamics will predict whether it was produced by the same user. Using such a model, we’ll check if a new behavioral pattern matches those collected during registration, and based on that we’ll estimate the probability that we are indeed dealing with the account owner.

Keystroke dynamics

In a general sense, keystroke dynamics relates to the detailed timing information describing when each particular key was pressed and released as a person is typing. But here we’ll be working with a less detailed sequence of timing information, namely: we’ll only have access to the timing information, without knowing exactly which key was being pressed/released.

The data set

We’ll be using the Bei Hang data set, but you don’t need to download it, it’s already included in the workshop repo, in the BeiHang directory. There are two sub-sets of the data: DatasetA and DatasetB — the first one was collected in a cyber cafe environment, and the latter in laboratory conditions. We’ll be working with DatasetA since it’s more “realistic”.

The data set is comprised of timestamp sequences of the form:
P1, R1, P2, R2, … , Pn, Rn
where Pi, is the timestamp of pressing the iᵗʰ key, and Ri is the timestamp of releasing the iᵗʰ key. For each user, we have 4 or 5 such sequences collected during registration, and about 20 sequences recorded during log-in attempts. Among those ~20 sequences some came from the actual owner of the account, while other were typed in by imposters.

The data are read into a pandas.DataFrame using the produce_whole_DF function (code cell [3]), so that you can focus more on the models themselves, rather than the process of loading and cleaning the data.

Example keystroke dynamics collected during registration for two users (“zhuzhu” and “qianhai18”) are plotted in code cells [5] and [6]. Additionally, there are also plots for the first order difference of timestamps (calculated using the np.diff function). What I mean by first order difference is a sequence out produced from sequence seq in the following manner: out[n] = seq[n+1] — seq[n], or in a single step: out = np.diff(seq).

Keystroke dynamics features

There are two prominent characteristics of the keystroke dynamics: the dwell and flight times. The dwell time is the duration of a key being pressed, while the flight time is the time period between releasing a key and pressing the next key.

Dwell and flight times are calculated using functions defined in code cell [7] and their values (again, for users “zhuzhu” and “qianhai18”) are plotted in code cells [8] and [9].

Now, these sequences (keystroke dynamics, dwell and flight times) differ in length between users, whereas most supervised models require feature vectors of unified length. So, in order to satisfy these requirements, we’ll need to somehow aggregate (or: summarize) these sequences of variable lengths. We’ll take the mean and the standard deviation of the dwell times, of the flight times, but also of the first order difference. These features are calculated in the extract_features function (code cell [7]), yielding a single feature vector of length=6 for each keystroke dynamics sequence. But notice that within the extract_features there is a comment:
# TODO(1): add more features
TODOs like this one are scattered throughout this notebook, and the number (1, 2, or 3) stands for difficulty, 1 being “easy”, and 3 being “hard/complex”.

Validation

It’s easy to trick oneself into thinking that the model works fine, when in fact its performance is due to some sort of data leakage. Thus, it’s important to split the data into train and test sets according to user names. Otherwise, the performance on the test set would be an over-optimistic estimation of how well the model performs for new, unseen data (in this case: new users).

That said, the Bei Hang data set is not exactly the cleanest, error-free collection. To make sure the results reported on the test set are not prone to defects in the data, I hand-picked a set of users names, test_user_names (code cell [12]), which we’ll use to define the test set, and the train set as its complement. This is not quite legit — I’m using some arbitrary criteria for choosing the train and test sets, rather than focusing on making both of them unbiased samples of the underlying distribution. However, for the purpose of this blog post, let’s assume that the keystroke dynamics are indeed i.i.d. for different users, and that there is no confounding factor causing certain users to have some of their sequences damaged.

But going back, we’ve mentioned in The set up section that we’ll train binary classification models. For that, we’ll pair up the observations, and label them: 1 if the pair belongs to a single user, and 0 otherwise. Training and validation will be performed only on pairs of observations collected during registration — see the DF_registration_train variable in the cell [12]. (BTW, throughout the code the “DF_” prefix indicates that the variable is a pandas.DataFrame.)

Models

We will train four models:

1. XGBoost;
2. Multi-layer perceptron (MLP);
3. Simple siamese neural network;
4. Siamese recurrent neural network.

In terms of what the model takes as input, we are dealing with two very different types of models. The first type (XGBoost & MLP) are models which we’ll supply with absolute differences of features extracted from keystroke dynamics sequences:

and in code calculated simply as: abs_diff = np.abs(a — b). The labels denote whether the two sequences come from the same user. In other words: we’ll train the model on a feature matrix, whose rows denote how two keystroke dynamics sequences differ.

The second type of models (the siamese models) are those that take as input full information describing both observations:

The “Simple siamese neural network” will take two concatenated feature vectors (each describing a keystroke dynamics sequence), and that’s exactly what’s being shown in the image above. However, the “Siamese recurrent neural network” will take two almost raw sequences describing the two keystroke dynamics. The “almost” is because they’re actually going to be the first order differences of sequences of timestamps (recall the np.diff function).

Model training and validation

To train and validate the model we’ll use the custom_cross_val_auc_score function (code cell [15]). What’s “custom” about it is that for each fold it splits the data into train and validation sets, according to user names. This way, if a given user was used for training, it won’t be used for validation.

The first argument of this function, estimator, is an object with two methods: estimator.fit and estimator.predict_proba. Technically, most classifiers available in sklearn implement these methods (but not all have the latter; see for example sklearn.svm.LinearSVC), so it seems that estimator might simply be a bare-bone classifier. However, the estimator object is also required to have additional attributes: estimator.only_feature_diffs and estimator.feature_names. The first one is used to determine whether the classifier will take two feature vectors, one per each keystroke dynamics sequence, or rather their difference. The latter attribute is used to locate which columns of the input frame (DF_registration) are used as features.

But we might need to standardize the data or transform it before passing to the classifier. To retain the attributes expected of the estimator object, we can wrap a list of preprocessing and transformation steps (along with a classification step at the end), with the sklearn.pipeline.Pipeline class. This is pretty standard practice, and if you haven’t encountered it previously — now’s your chance. Then, we can add the estimator.only_feature_diffs and the estimator.feature_names attributes. And that’s exactly what we’re going to do when training our models in code cells: [16], [21], [26], and [29].

1. XGBoost

The xgboost.XGBClassifier is a robust model in that it gives good results out-of-the-box for a wide range of problems, without the need of adjusting the hyper-parameters. We won’t be able to go into details of how XGBoost is trained in this blog post, but for perhaps the best introduction of the algorithm, make sure to check out this source.

For our purposes it suffices to know that XGBoost offers a good base-line for our subsequent, neural-net-based models. If we’ll be able to reach XGBoost’s default performance without much meddling, we’ll consider a model “easy to handle”. We’ll see that, while multi-layer perceptrons are easy to handle, siamese architectures require significantly more care.

As an aside, one drawback of XGBoost is that it’s difficult to interpret and even with the aid of tools like LIME or ELI5, understanding how exactly the model makes its predictions poses a challenge.

The XGBoost model is trained in a code cell [16]. We plot the AUC scores calculated by the custom_cross_val_auc_score as a histogram. And although I do admit a histogram of four values looks a bit silly, it becomes justifiable once you use a higher number of folds (the num_folds argument of the custom_cross_val_auc_score function). As a bonus, we can take a peek into which features were important for the construction of the XGBoost model, using the xgboost.plot_importance function. Sure enough, the means of dwell and flight times were the most relevant features, and it seems that they are quite informative as the model achieves high values of AUC scores of about 0.90 on validation.

For now, we’ll keep the best_xgb_pipe, and use it at the very end of this notebook, to see how well it performs on the test set, along with the rest of the models. You might play with the arguments of the xgboost.XGBClassifier, and get an even better score, but we would advise to leave that spunk for the siamese recurrent neural network.

Summary

We’ve introduced the data set, spent considerable amount of time explaining the loopholes associated with validation, and trained our first model, the XGBoost.

In Part 2, we’ll dive into neural-network-based models, and finally assess all models on the test set.

Read the second part of the tutorial:

Building Supervised Models for User Verification — Part 2 of the Tutorial

About Nethone

Nethone creates A.I.-based anti-fraud and business intelligence solutions that help enterprises from all around the world turn threats into well-informed, profitable decisions. Founded in 2016 by experienced data scientists, security experts, and business executives, Nethone is one of the fastest-growing FDP companies in Central Europe.

If you enjoyed this post, please hit the clap button below 👏👏👏

You can also follow us on Facebook, Twitter and LinkedIn.

Building Supervised Models for User Verification — Part 1 of the Tutorial was originally published in Daftcode Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

tl;dr we take a repo with a somewhat convoluted README, wrap it up using the Anaconda Project, take care of the bookkeeping and publish our Project for everyone to download and run with a single command without worrying about dependencies.

So, you’ve just finished your project — an analysis, a model, a web app, or something else entirely — and you would like to share it with the community. How to do it in the easiest way? In this post, we’ll explore and showcase the Anaconda Project — a (fairly) new tool from Anaconda, that allows you to pack your project along with runnable commands, self-resolving dependencies mechanism, environmental variables, and more.

From the Anaconda Project docs:

Anaconda Project encapsulates data science projects and makes them easily portable. Project automates setup steps such as installing the right packages, downloading files, setting environment variables and running commands.

Project makes it easy to reproduce your work, share projects with others and run them on different platforms. It also simplifies deployment to servers. Anaconda projects run the same way on your machine, on another user’s machine or when deployed to a Server.

Sounds good, but does it work? Let’s give it a try!

Where to get it?

Anaconda Project is included in the standard Anaconda distribution (version 4.3.1 or higher), so to check your version, run:

anaconda-project --version

My initial, out-of-the-box version was 0.6.0, however, to get the latest version (0.8.0 which is used here), run:

conda install anaconda-project --channel conda-forge

Thus, the version of Anaconda Project I was working with was 0.8.0. Also, I used Python 3.6 and worked on Ubuntu 16.04.

Use case

At this year’s PyCon PL, I had a talk about transfer learning, where I showed how to combine several scripts and tools to retrain an artificial neural network and analyze its output. I wanted to identify motorcycle types from images and to assert that the model made its predictions based on motorcycle parts and not on artifacts hidden in the images. You can find the code from that presentation on DaftCode’s GitHub.

The repo contains:

● a script for downloading images from Google Search;

● a runnable wrapper for the retrain.pyscript (from tensorflow/examples/image_retraining);

● an optional inspection step with TensorBoard;

● a Jupyter Notebook (which is the culminating point of the whole project) that:

1) displays patterns of activations in the last hidden layer of the network (in response to the different types of motorcycles),

2) makes predictions for not-yet-seen images (in the test_images/ directory),

3) shows an analysis of which parts of the images led the model to a particular prediction

● and a pretty long README file.

I wanted to show the steps leading to the analysis presented in the notebook to demonstrate how easy it is to adapt (retrain) models in TensorFlow. But the mere length of the README might suggest something different.

The reason is that there’s a fair amount of “bookkeeping” needed for the main analysis: ensuring all required packages are installed, downloading the data and retraining the model.

The idea is to use Anaconda Project to really “get to it” as fast as possible, simply by skimming over the bookkeeping part of the project and running everything with a single command. In this post, I’m going to show how to wrap an existing repo, without modifying it, in such a way that Anaconda Project: clones the repo, takes care of the bookkeeping and runs the final Jupyter Notebook for the User to play with. Alternatively, you might want to modify an existing repo and make it an Anaconda Project. Or, if you start a new project, it would probably be more convenient to make it an Anaconda Project and build it from the ground up, as you go along.

Initializing the Project

Let’s first create a directory for our Project, initialize it and add the repo with the PyCon presentation.

To initialize the Anaconda Project, run:

mkdir pycon_presentation_anaconda_project
cd pycon_presentation_anaconda_project
anaconda-project init

This creates the anaconda-project.yml file, which is the heart of the Project. This is the configuration file that will be modified after each anaconda-project command.

Next, we’ll clone the repo with the PyCon presentation:

git clone --recursive https://github.com/daftcode/PyCon-motorcycle-transfer-learning.git

The --recursive flag will also update the submodule tensorflow within it.

To ensure reproducibility, we need to checkout a particular commit:

cd PyCon-motorcycle-transfer-learning
git checkout 5c00c5b
cd ..

Adding commands

But wait… we can automate the process of updating the submodules and checking out a particular commit. Let’s do that by adding the first command to our project:

anaconda-project add-command clone_repo "git clone --recursive https://github.com/daftcode/PyCon-motorcycle-transfer-learning.git; cd PyCon-motorcycle-transfer-learning; git checkout 5c00c5b"

We’ll be asked what kind of command is this:

Is clone_repo a (B)okeh app, (N)otebook, or (C)ommand line?

and since this is a command ran in the command line, we choose (C). We can also specify the command type with the --type option. Note, however, that in our case --type unix is the equivalent of choosing (C).

There’s no point in running the command (as we have already taken care of the submodules “manually”), but you could do it by typing:

anaconda-project run clone_repo

Because the directory already exists (you did clone it, right?), nothing is going to really happen. But you might also notice that you get an annoying message:

Potential issues with this project:
* anaconda-project.yml: No commands run notebooks PyCon-motorcycle-transfer-learning/presentation.ipynb, ...

and if you don’t like to be reminded about it every single time you run any command, you can edit the .projectignore file and add the following line:

**.ipynb

Next, we’ll add a command that downloads images needed for retraining. For that, let’s create the following script download_images.sh:

cd PyCon-motorcycle-transfer-learning
for type in classic cross cruiser superbike
do
python image_download.py motorcycle $type
done

that calls the image_download.py script in the PyCon-motorcycle-transfer-learning repo. To add this script to the Project’s commands, type:

anaconda-project add-command --type unix download_images "bash download_images.sh"

This command will download 400 images of four classes of motorcycles (100 each): classic, cross, cruiser, and superbike. Let’s run it:

anaconda-project run download_images

Now, there is a time for the workhorse of this Project — the retrain command:

anaconda-project add-command --type unix retrain "cd PyCon-motorcycle-transfer-learning; python retrain_wrap.py"

but in order to run it correctly, we need to add an additional package: tensorflow.

To add this package so that it’s loaded during the run along with other necessary packages, we’ll use the add-packagesfunctionality:

anaconda-project add-packages tensorflow

The retrain_wrap.py script actually calls the tensorflow/tensorflow/examples/image_retraining/retrain.py script, which downloads the necessary pre-trained model (by default the largest MobileNet available), splits the data into train, validation and test sets, computes the “bottlenecks” (activations on the neurons of the last hidden layer) and then takes care of the whole retraining procedure.

However, we should inspect how the retraining went. For example, we might have overfitted the training data and could have gotten a better model by running the procedure in fewer steps. To inspect the retraining we’ll use TensorBoard (for which the retrain.py script conveniently gathers summary statistics):

anaconda-project add-command --type unix inspect_with_tensorboard "cd PyCon-motorcycle-transfer-learning; tensorboard --logdir summaries"

Note, that you’ll need a browser to view the app at: http://localhost:6006. Once you’re done with analyzing the graphs, hit Ctrl+C to terminate TensorBoard.

If you would like to deploy a web app to a server (TensorBoard, for example), it’s possible to customize the default behavior (as long as the app communicates via HTTP); for details see the Anaconda Project reference.

For our final command, we’ll add the presentation.ipynb notebook:

anaconda-project add-command --type notebook presentation "PyCon-motorcycle-transfer-learning/presentation.ipynb"

Again, we’ll need to add one tiny package to be able to run it: lime. But this is a little bit more tricky because we’re interested in the latest version — 0.1.1.23, which I used for this project — whereas the latest version at the conda-forge channel is 0.1.1.18. However, if we go to www.anaconda.org and search for lime, we’ll see that there’s a channel called “viascience” that offers the coveted, latest version:

anaconda-project add-packages --channel viascience lime

Note, that this time we’re telling Anaconda where to look for it (the “viascience” channel).

We can now run the command:

anaconda-project run presentation

open the notebook and run all commands within it.

Putting it all together

OK, let’s list out the commands we’ve defined:

anaconda-project list-commands

The output should look something like the following:

Name                   Description

====                   ===========

clone_repo             git clone --recursive https://github.com/daftcode/PyCon-motorcycle-transfer-learning.git; cd PyCon-motorcycle-transfer-learning; git checkout 5c00c5b

download_images        bash download_images.sh

inspect_with_tensorboard  cd PyCon-motorcycle-transfer-learning; tensorboard --logdir summaries

presentation           Notebook PyCon-motorcycle-transfer-learning/presentation.ipynb

retrain                cd PyCon-motorcycle-transfer-learning; python retrain_wrap.py

retrain                cd PyCon-motorcycle-transfer-learning; python retrain_wrap.py

Great, but… what now? Should we then replace our previous README with several anaconda-project run ... calls? Or can we do better?

For me, better means: shorter, so that my README says literally this: “To run the Project, type: anaconda-project run all”. Frankly, I was expecting there would be a functionality that allows anaconda-project runto run all or several commands in some defined order. But come to think of it, we might simply define an all command like this:

anaconda-project add-command --type unix all "anaconda-project run clone_repo; anaconda-project run download_images; anaconda-project run retrain; anaconda-project run presentation"

(I’ve left out the anaconda-project run inspect_with_tensorboardsince it requires that the User hits “Ctrl + C”, which, unfortunately, kills the rest of the anaconda-project process.)

We can now formulate our README.md file:

# To run the project ...
... type:
```
anaconda-project run all
```
## To inspect the retraining process ...
... type (after running `all`):
```
anaconda-project run inspect_with_tensorboard
```

That’s what I call a short README!

Uploading the Project to Anaconda Cloud

Before we upload the Project, we want to make sure it’s as tidy as possible. We can use the:

anaconda-project clean

command to get rid of the envs/ directory with all the required libraries, but notice it left out the PyCon-motorcycle-transfer-learning repo. To exclude it manually, open the .projectignoreand add the following:

PyCon-motorcycle-transfer-learning

Note, that the clone_repo command will take care of cloning this repo (along with its submodule tensorflow) and checking out the right commit.

Now, once our project is as tidy as it can be, let’s upload it with the following, final command:

anaconda-project upload

Et voila, our project is out there: https://anaconda.org/mdzi and once it’s downloaded, in can be unpacked with:

anaconda-project unarchive pycon_presentation_anaconda_project.tar.bz2

Pretty neat :)

For a final touch, I’ve also uploaded the presentation.ipynb notebook to Anaconda Cloud with:

anaconda upload presentation.ipynb

so that you can see the final (though static) result of the Project right away.

Summary

We now know how to use Anaconda Project to wrap up an already existing repo, add commands and dependencies, so that the User can download the Project and run everything with a single command:

anaconda-project run all

Anaconda Project is easy to work with and really allows you to simplify the flow for an end User.

However, if the set-up required for running your project isn’t complicated, you might simply export your environment (instead of defining anaconda-project commands), write a bash script run.sh packing it all up, so that the README would look something like this:

conda env create -f environment.yml
bash run.sh

But Anaconda Project isn’t just adding commands and specifying dependencies— it allows you to: automatically download large files (that you don’t want to keep in version control), pack up web apps (that communicate with HTTP) and run them on a server, handling environmental variables that contain credentials and adding services like Redis (this feature is in a demo phase, though).

It’s worth to try it out and see how you can use it for your Project!

If you enjoyed this post, please hit the clap button below 👏👏👏

You can also follow us on Facebook, Twitter and LinkedIn.

Wrap Your Repo with Anaconda Project was originally published in Daftcode Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.