VSLAM — Introduction

stands for Visual Simultaneous Localisation and Mapping.

To do anything useful, a robot must walk around . To walk around, it must navigate. To navigate it must know exactly what is the environment.

VSLAM solves this problem using only visual information , no external beacons , just cameras mapping the world and finding the robot’s placeSo , VSLAM helps to create map using environmental keypoints and then we use that map to localise the robot.

There can be different ways to work around VSLAM, one of which we will discuss here:
we will follow below two steps with respective algorithm we will use —

1. Key frame estimation — Binocular depth estimation
2. localisation w.r.t key frame — 3d — 2d PnP (point and perspective) i.e optimisation of pose such that we minimise the projection error.

So the system operates on continuous , two step cycle. First, it anchors itself by estimating reference key points in the environment. Second, as it moves, it tracks those reference key points to localise its new position , before establishing a new anchor and repeating.

Binocular Depth Estimation —

As there will two cameras for robot’s eyes , we can imagine with ours one also, imagine there is star image in front of us and if we try to see it one by one with only one eyes , for the left eye the star will come in right side of frame and for the right eye the star will come in left side of frame.

So considering same will happen with real camera , for estimating the depth of star from the robot :

we know baseline distance between both camera (D) and projective angle between cameras and star , we can calculate the depth.

Get Ckmedium’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

So we are going to repeat this strategy for many key points in an image , and it will give us key frames, it is like pose and collection of points associated with these pose , and each points will have depth.

Localisation via PnP :

So we are calculating the key frames and consecutively the robot moves and now we have to localise w.r.t this key frames

Let us assume there are 3 object in front of robot and they have their respective key frames and depth and then robot moved somewhat sees differently , so we have to solve optimisation problem and eventually there will be two perspective :

1) What the robot see

2) What should robot see

ignore the face of robot, its in opposite direction

so both perspective will be different or somewhat have error, so we need to solve that how will we change the pose which robot sees to that we will make the error as minimum as possible.

So eventually 3d-2d pose optimisation algorithm asks how do i tweak my pose the tiniest bit to make the visual error smaller?

and we can approach this by continuously adjusting the pose variable to minimise the reprojection error to near-zero, the system derives the true physical position of the robot. The pose that minimises the error is true pose.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

So whole thing we did above comes under frontend part of VSLAM, its primary goal is faster data ingestion, Low latency , tracking progression (discrete) but eventually its faces high vulnerability to drift over the period of time.

Measuring relative movement via key frames naturally invites micro-mistakes , even without systemic bias, these small errors accumulate.
Drift is a natural property of the universe as time increases, certainty decays, and the bounds of spatial error expand until the robot is functionally lost.

So to work around the frontend vulnerabilities we have backend system, so the frontend usually deals with pose estimation , the backend deals with joint estimation of history of poses as well as mapping (position of all key points).

So backend is slow and not real time thus takes all the key frames and poses , it remembers what robot has seen in all of those key frames and tries to figure out the relation between key frames to optimise whole thing for single system. It eventually establishes a long term relations , it creates mathematical ‘edges’ that forcibly pull the drifting trajectory into alignment with true reference frame.