<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jan Steckel on Medium]]></title>
        <description><![CDATA[Stories by Jan Steckel on Medium]]></description>
        <link>https://medium.com/@jan.steckel?source=rss-b18596ec6b9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*T7cyA-_ousj_x6iUQGAJKw.png</url>
            <title>Stories by Jan Steckel on Medium</title>
            <link>https://medium.com/@jan.steckel?source=rss-b18596ec6b9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 22:46:01 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jan.steckel/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Can a robot predict what its sonar will hear next?]]></title>
            <link>https://medium.com/acoustic-explorations/can-a-robot-predict-what-its-sonar-will-hear-next-73cdff7086ca?source=rss-b18596ec6b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/73cdff7086ca</guid>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[robotics]]></category>
            <dc:creator><![CDATA[Jan Steckel]]></dc:creator>
            <pubDate>Wed, 11 Nov 2026 08:11:01 GMT</pubDate>
            <atom:updated>2026-11-11T08:11:01.637Z</atom:updated>
            <content:encoded><![CDATA[<p><em>EchoPT, a transformer that predicts the next sonar image from the last few and from the robot’s own motion commands, and what such a prediction is good for</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3MGvSBpNL03InZKzKdUwbQ.png" /><figcaption><em>Three sonar images measured by a simulated robot (left, oldest first) and the next one, as predicted by our model before it was measured (right). Each fan is a top view of the echoes around the robot, out to 5 metres.</em></figcaption></figure><p>You probably know the feeling of walking down a staircase in the dark and reaching for one more step that is not there. The jolt you feel is your brain discovering that its prediction was wrong. Most of the time, you never notice these predictions, because they come true. If they dont, you feel surprise, and minimizing this surprise is what drives your behaviour (see what I did there, active inference people?).</p><p>Language models such as ChatGPT run on the same trick, applied to words: given the text so far, predict the next word. In this post, I ask the same of a sonar. Can a robot predict what its sonar will hear next, from what it heard a moment ago and from how it is about to move?</p><p>In simulation, it can. Together with my colleagues, we built a transformer called EchoPT (for Echo-Predicting Pretrained Transformer) that predicts the next sonar image better than the closed-form methods we had before. And a robot can do useful things with that prediction, such as noticing that its wheels are slipping.</p><h3>Acoustic flow</h3><p>When a robot drives through a static world, the echoes around it move through the sonar image in a way that is fully determined by its own motion. Driving straight ahead, the sideways distance between a reflector and the robot’s path stays the same, so its echo slides along a fixed curve, a flow line. My colleagues and I worked this out more than ten years ago, and it has since steered robots, deblurred sonar images (<a href="https://medium.com/acoustic-explorations/deblurring-sonar-images-by-driving-around-bee8f8c8364e">Deblurring sonar images by driving around</a>) and taught a simulated bat its ears (<a href="https://medium.com/acoustic-explorations/how-a-bat-could-learn-its-own-ears-06d867593457">How a bat could learn its own ears</a>).</p><p>So acoustic flow is already a prediction, written as an equation. But it predicts where a point goes, and in a sonar image every reflector shows up as a blob, the point-spread function of the sensor. Moving each pixel along its flow line bends that blob into a shape the sensor would never produce. The equations also ignore that the emitter and microphones are more sensitive in some directions than in others, so echoes get stronger or weaker as they move across the image.</p><h3>Echoes as tokens</h3><p>A language model chops text into tokens and learns, from enormous amounts of text, which token tends to come next. Nobody labels anything, the text itself is the teacher. This is called self-supervised learning, and readers of this blog will know by now that I am a fan. You should also be a fan of Self-Supervised Learning, because it is awesome.</p><p>In EchoPT, a token is a whole sonar image: a map of how much echo energy comes back from each range and direction in the horizontal plane. (To my purist deep learning friends: this is not 100% true. We use a vision transformer, and do patch embedding of the input data). The model receives the last three images, the robot’s motion commands (forward speed and turning rate) for those steps, and the command for the step it has to predict. It returns the next image. The teacher is the image the sonar measures one step later.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SAdiVLD4D0JuBGFog67LqQ.png" /><figcaption><em>Top: six consecutive sonar images from a simulated drive, seen from above with the robot at the bottom centre. Bottom: EchoPT gets frames 1 to 3 and the motion commands, and predicts what comes next. For two and three steps ahead, it builds on its own earlier predictions, without any new measurement.</em></figcaption></figure><p>Inside, EchoPT borrows from vision transformers. The three images are stacked and cut into patches, small regions of range and direction. In each transformer layer, every patch can look at all the others and pick out the ones that matter for its own prediction. This is called self-attention. A convolutional branch and a small network for the motion commands run alongside, and all three are combined into the predicted image.</p><p>EchoPT has about 9 million parameters, 7.8 million of them in its eight transformer layers. By today’s standards, that is a language model for ants (or, more fittingly, for very small bats). Training on about 63,000 simulated sonar images (three and a half hours of driving) took about 36 hours on one NVIDIA RTX 4090 graphics card.</p><h3>How good is the prediction?</h3><p>We compared EchoPT with two baselines that only use the most recent image. The first shifts it by the distance and angle the robot moved. The second moves every pixel along its acoustic flow line.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wLStPJpwwGlYdWdMyDdZkw.png" /><figcaption><em>One step ahead, seen from above. The shifted image loses the faint reflector at the top left (1), and acoustic flow deforms the bright blob at the centre (2). EchoPT keeps both close to what was measured. This is one example, the statistics follow below.</em></figcaption></figure><p>On 1,000 images from a simulated drive not used in training, we measured how well each prediction matches the real image with the cross-correlation (1 is a perfect match). One step ahead, EchoPT reached 0.97, against 0.86 for acoustic flow and 0.85 for the shifted image.</p><p>Like a language model writing a sentence, EchoPT can keep going by feeding its own predictions back in. This is called auto-regressive prediction, and the errors then add up for every method. Ten steps ahead, EchoPT still reached 0.69, against 0.56 and 0.38. All differences were statistically significant, but the spread over the test images is large, with standard deviations around 0.25 for all multi-step predictions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vkXkTWXFoVErExfQATPFkA.png" /><figcaption><em>How well the predicted image matches the measured one (mean cross-correlation over 1,000 test images), predicting one to ten frames ahead. From Table 1 of the paper.</em></figcaption></figure><h3>Is my wheel slipping?</h3><p>Robots often rely on odometry when navigating, and outdoors, odoemtry has issues, often due to wheel slip. When the wheels slip, the robot does not move the way its commands say, so the sonar images stop matching what the model expected. The difference between prediction and measurement then works as a slip detector.</p><p>Our simulated robot drove with two slip periods: both wheels between 10 and 16 seconds, a single wheel between 30 and 36 seconds. With predictions five steps ahead, all three methods flag the first period. Only EchoPT clearly flags the second.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ySuB8f3hFKAc46yEXjJkVQ.png" /><figcaption><em>The slip signal (how badly the five-steps-ahead prediction misses the measurement) for the three predictors. Shaded: both wheels slip (left), one wheel slips (right). Traced from Fig. 5 of the paper, arbitrary units.</em></figcaption></figure><p>Just after the second slip, both baselines spike while EchoPT stays quiet. The robot was driving along a wall with few other reflectors, where the simple predictions break down, and EchoPT had seen such scenes in training. EchoPT is not free of false alarms either, as the spikes after 40 seconds show for all three methods. And to be fair, our paper itself notes that an inertial sensor might be the better tool for this particular job. But it serves as a downstream task where such a foundation model could be used for.</p><h3>Driving through noise</h3><p>Factories are full of ultrasonic noise, from compressed-air leaks to machining tools (and, as <a href="https://medium.com/acoustic-explorations/when-the-robot-is-louder-than-the-forest-1355a8edae91">When the robot is louder than the forest</a> showed, from the robot itself). In the second experiment, a simulated robot drove down a corridor of round reflectors (think of orchard trees or the pillars of storage racks), steered by our acoustic-flow controller. We then drowned the sonar in noise bursts of 1.2 seconds, so that about 30% of the time its images held nothing useful.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mDjrBWBdD7MXGh0li8oRdQ.png" /><figcaption><em>Fifty runs per condition. With clean data (top), the robot drives a tight path to the goal. With noise bursts (middle), it stops and wanders, and takes much longer. When EchoPT fills in the noisy images with predictions (bottom), path and travel time come close to the clean case.</em></figcaption></figure><p>Without help, the robot stopped whenever its sonar went deaf. With EchoPT’s predictions in place of the noisy images, the controller (which never knew) drove almost as if nothing had happened. It is a bit like crossing a dark room you know well on memory, a second or so at a time.</p><h3>Conclusion</h3><p>Prediction has been the teacher in almost every post on this blog, up to the neural field in <a href="https://medium.com/acoustic-explorations/how-a-bat-could-build-a-3d-model-from-its-own-echoes-83de9e9c48e4">How a bat could build a 3D model from its own echoes</a>. In EchoPT, the prediction itself is what we are after, and the moments where it fails carry the information. This view of perception, as minimising prediction errors, is called predictive processing. Bats inspired us to try it on echoes, but I do not claim that EchoPT says anything about how a bat’s brain works.</p><h3>Further reading</h3><ul><li><strong>EchoPT: A Pretrained Transformer Architecture That Predicts 2D In-Air Sonar Images for Mobile Robotics.</strong> J. Steckel, W. Jansen and N. Huebel. <em>Biomimetics</em> 9(11), 695, 2024 (open access): <a href="https://doi.org/10.3390/biomimetics9110695">https://doi.org/10.3390/biomimetics9110695</a></li><li>Preprint on arXiv: <a href="https://arxiv.org/abs/2405.12573">https://arxiv.org/abs/2405.12573</a></li><li>Code, trained model and a test dataset (MATLAB): <a href="https://doi.org/10.5281/zenodo.11191954">https://doi.org/10.5281/zenodo.11191954</a></li><li>Earlier posts on prediction as the teacher: Deblurring sonar images by driving around, How a bat could learn its own ears, How a bat could build a 3D model from its own echoes (links in the text)</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=73cdff7086ca" width="1" height="1" alt=""><hr><p><a href="https://medium.com/acoustic-explorations/can-a-robot-predict-what-its-sonar-will-hear-next-73cdff7086ca">Can a robot predict what its sonar will hear next?</a> was originally published in <a href="https://medium.com/acoustic-explorations">Acoustic Explorations</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Seven bat heads, and what they do to sound]]></title>
            <link>https://medium.com/acoustic-explorations/seven-bat-heads-and-what-they-do-to-sound-c24bba45b284?source=rss-b18596ec6b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/c24bba45b284</guid>
            <dc:creator><![CDATA[Jan Steckel]]></dc:creator>
            <pubDate>Mon, 09 Nov 2026 17:51:00 GMT</pubDate>
            <atom:updated>2026-11-09T17:53:44.365Z</atom:updated>
            <content:encoded><![CDATA[<p><em>An open dataset of how seven bat species send and hear ultrasound, on the full sphere and from 20 to 160 kHz, computed from micro-CT scans of their heads</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mZrrAtcPzh-Fw7B-s3uJIg.png" /><figcaption><em>Three of the seven heads (Micronycteris microtis, Rhinolophus formosae, Phyllostomus discolor) and below each, how strongly it calls and hears in every direction in front of it, at 60 kHz.</em></figcaption></figure><p>Hold your hand behind your ear and listen to someone talking in front of you. The voice gets louder, and a little brighter. Your hand is now a crude acoustic antenna, and you have just changed the directivity of your own ear.</p><p>Bats take this principle to an extreme. Their ears are large, oddly shaped and often movable, and many species call through their nose, surrounded by a fleshy structure called a noseleaf. Together, the head, the ears and the noseleaf decide in which directions a bat sends its calls and from which directions it hears the echoes, and how that changes with frequency.</p><p>In this post, we look at a simple question: what exactly does a bat’s head do to the sound it sends and hears? I have just released a dataset that answers it for seven species, computed from micro-CT scans of their heads. The data, the head meshes and the code are open, and the paper is under review.</p><h3>Ears and noses as filters</h3><p>In <a href="https://medium.com/acoustic-explorations/how-a-bat-could-learn-its-own-ears-06d867593457">How a bat could learn its own ears</a>, the ears were a filter: depending on where a sound comes from, the head and pinnae boost some frequencies and suppress others. That filter is called the Head-Related Transfer Function (HRTF), and it is how you can tell whether a sound comes from above or below you. A bat has a second filter at the other end of the chain, the emission beam of its mouth or noseleaf. The product of the two is the Echolocation-Related Transfer Function (ERTF), the spatial sensitivity of the complete sonar system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tgnNZhsbnjfJoEvgx1VUdg.png" /><figcaption><em>Micronycteris microtis at 60 kHz, seen from behind the bat looking forward. The left ear hears best towards the lower left, the noseleaf sends a broad beam, and together they form a narrower two-way beam. Colour: level in dB relative to the maximum, with contours at −3 and −6 dB.</em></figcaption></figure><p>If you build sonar systems, this will look familiar. The ERTF is the beam pattern of a transmitter times that of a receiver, except that here the antennas are made of skin and cartilage, and they were shaped by evolution rather than by an engineer (who would have made them less floppy).</p><h3>Why not just measure it?</h3><p>You can measure an HRTF on a live bat, but you have to sample hundreds to thousands of directions over a band of more than 100 kHz, while the animal moves its head and ears (bats are not known for their patience). On a dead specimen, the tissue deforms. Most published measurements also exist only as figures in papers, not as data.</p><p>So we simulate. The seven head meshes come from the archive of the University of Antwerp, made from micro-CT scans of one preserved specimen per species, and were created as part of the EU projects Cilia and ChiRoPing. Three of them (<em>Macrophyllum macrophyllum</em>, <em>Eptesicus fuscus</em> and <em>Myotis daubentonii</em>) are published here for the first time. I cleaned each mesh into a closed surface of up to 58,000 triangles, within a few hundredths of a millimetre of the scanned surface.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CESZWbRh7-Cjr9KxisvpJg.png" /><figcaption><em>The seven heads of the dataset, from three families: three leaf-nosed bats and two horseshoe bats, which call through their nose, and two vesper bats (the last two), which call through their mouth</em></figcaption></figure><h3>Solving for sound on the skin</h3><p>The workhorse is the boundary element method (BEM). Instead of filling the air around the head with a dense voxel grid, you only describe the surface of the head. Every small triangle of skin then acts as a tiny loudspeaker that re-radiates the sound arriving at it, and the method solves for the strength of all of them at once. Humans hearing can be analyzed using the same treatment: Mesh2HRTF, an open-source BEM package, computes personalised HRTFs from 3D scans of people’s heads for spatial audio.</p><p>One trick makes this cheap. By acoustic reciprocity, you can swap source and receiver without changing the transfer function. So instead of placing a loudspeaker in 10,000 directions and listening at the ear, we make the ear canal itself vibrate and compute where its sound goes. After one expensive step per frequency on a GPU, every extra source (both ears, the noseleaf, and alternative positions for each) costs almost nothing. Each species gets up to 281 frequencies from 20 to 160 kHz, on a 2.5-degree grid over the whole sphere.</p><p>The solver agrees with NumCalc, the physically validated solver of Mesh2HRTF, to within 0.13 dB on the same mesh.</p><h3>Where exactly is the ear?</h3><p>Here is a detail that is easy to underestimate. The mesh does not tell you where the ear canal entrance is, or where exactly the nostrils are. Moving the receiver by a millimetre can change the pattern noticeably. So we derived a placement rule from the two species with earlier simulations: put the source where it radiates most strongly forward and is most tucked into the surrounding anatomy, which is where pinna and noseleaf focus sound.</p><p>We then tested that rule on a bat it had never seen, a <em>Myotis daubentonii</em> that Vesterholm and colleagues had measured and scanned. Among 211 candidate positions around the ear canal entrance, the rule picked one at the 98th to 99th percentile of agreement with the measured HRTF.</p><h3>Does it match a real bat?</h3><p>That same measured bat is the hardest test. Our simulation and the independent Mesh2HRTF simulation of Vesterholm and colleagues agree almost perfectly with each other (a pattern correlation of 0.997). Both deviate from the measurement in the same way, with a pattern correlation of about 0.70. Both need the same extra rotation of about 26 degrees between the measurement directions and the scan, after which they reach 0.82. That points to how the measurement and the scan line up, or to a change in the posture of the dead animal, more than to the solver.</p><p>So the claim is that the pipeline performs like the established tool for human HRTFs, not that either simulation is as good as a measurement.</p><p>The two mouth-calling species needed a different approach. Preserved specimens have closed mouths, and a closed mouth has no opening to shape a beam. Their emitter is therefore a simple piston model of the open mouth, calibrated to beams measured in the field, with alternative radii for the wider beams these bats use in the lab and in confined spaces.</p><h3>Ears that move</h3><p>Many bats move their pinnae while they echolocate, and a fixed mesh cannot. So every head comes with a skeleton for its pinnae and noseleaf, like a character in an animated film, and can be posed and simulated again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qy8ub_IIF5IrxF5GDTpnBA.png" /><figcaption><em>Micronycteris microtis with both pinnae (magenta) tilted forward by 20 degrees, at rest, and tilted back by 25 degrees, each computed again with the BEM. Below: the left-ear pattern at 40 kHz.</em></figcaption></figure><p>The effect is large. Tilting the pinnae moves the main lobe of the ear by up to 18 degrees at 40 kHz, and at higher frequencies the strongest lobe can jump elsewhere entirely. Posture matters at least as much as where you put the receiver.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FQ9Kxj9oIEwg%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DQ9Kxj9oIEwg&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FQ9Kxj9oIEwg%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/c9f97134e7cf4b4cfa53a8d13ed80de3/href">https://medium.com/media/c9f97134e7cf4b4cfa53a8d13ed80de3/href</a></iframe><h3>What is in the box</h3><p>For every species: the clean head mesh, the landmarks of ears, nostrils or mouth, the complex transfer functions (magnitude and phase) of both ears and the emitter, alternative source positions, and the skeleton. The files come as HDF5, SOFA (the standard format for HRTFs) and MATLAB, 3.3 GB in total. A small Python module loads a species, interpolates to any direction, rotates the head, and simulates the binaural echoes of point reflectors, and there are MATLAB examples too.</p><h3>Why this matters</h3><p>For robotics, this is the bat version of a sensor datasheet. If you want to simulate what a bat hears in a forest, design a bat-inspired sonar, train a network on realistic echoes, or compare how noseleaves and ears differ between species, you now have seven full-sphere models to start from, without a new animal and without a new scan.</p><p>The specimens behind these meshes are scarce, and scanning and segmenting a head takes a lot of effort. I would rather see these meshes used by many people than sit on a hard drive. If you use them, I would love to hear what for.</p><h3>Further reading</h3><ul><li>Dataset (CC BY 4.0): <a href="https://doi.org/10.5281/zenodo.23729269">https://doi.org/10.5281/zenodo.23729269</a></li><li>Code (BSD-3-Clause): <a href="https://github.com/Cosys-Lab/bat_ertf_dataset">https://github.com/Cosys-Lab/bat_ertf_dataset</a></li><li>Video: <a href="https://youtu.be/Q9Kxj9oIEwg">https://youtu.be/Q9Kxj9oIEwg</a></li><li><strong>Simulated head related transfer function of the phyllostomid bat Phyllostomus discolor.</strong> F. De Mey, J. Reijniers, H. Peremans, M. Otani, U. Firzlaff. <em>J. Acoust. Soc. Am.</em> 124(4), 2008. <a href="https://doi.org/10.1121/1.2968703">https://doi.org/10.1121/1.2968703</a></li><li><strong>What noseleaves do for FM bats depends on their degree of sensorial specialization.</strong> D. Vanderelst, F. De Mey, H. Peremans, I. Geipel, E. Kalko, U. Firzlaff. <em>PLoS ONE</em> 5(8), 2010. <a href="https://doi.org/10.1371/journal.pone.0011893">https://doi.org/10.1371/journal.pone.0011893</a></li><li><strong>3D photogrammetry as a low-cost and non-invasive method for acoustic modelling of animal hearing.</strong> K. K. Vesterholm, F. T. Häfele, F. Figeac, L. Jakobsen. <em>Methods in Ecology and Evolution</em> 17(1), 2025. <a href="https://doi.org/10.1111/2041-210X.70197">https://doi.org/10.1111/2041-210X.70197</a></li><li><strong>Mesh2HRTF:</strong> <a href="https://mesh2hrtf.org/">https://mesh2hrtf.org</a></li><li>Previous posts: <a href="https://medium.com/acoustic-explorations/how-a-bat-could-learn-its-own-ears-06d867593457">How a bat could learn its own ears</a>, <a href="https://medium.com/acoustic-explorations/why-our-ertis-sonar-sees-better-left-right-than-up-down-9767c19b727d">Why our eRTIS sonar sees better left-right than up-down</a></li></ul><p><em>The original head meshes were created as part of the EU projects Cilia (FP6-IST-016039) and ChiRoPing (FP7–215370).</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c24bba45b284" width="1" height="1" alt=""><hr><p><a href="https://medium.com/acoustic-explorations/seven-bat-heads-and-what-they-do-to-sound-c24bba45b284">Seven bat heads, and what they do to sound</a> was originally published in <a href="https://medium.com/acoustic-explorations">Acoustic Explorations</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sharper sonar images by multiplying instead of adding]]></title>
            <link>https://medium.com/acoustic-explorations/sharper-sonar-images-by-multiplying-instead-of-adding-be1bdee003a4?source=rss-b18596ec6b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/be1bdee003a4</guid>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[medical-imaging]]></category>
            <category><![CDATA[signal-processing]]></category>
            <category><![CDATA[sonar]]></category>
            <dc:creator><![CDATA[Jan Steckel]]></dc:creator>
            <pubDate>Sun, 08 Nov 2026 16:31:01 GMT</pubDate>
            <atom:updated>2026-11-08T16:31:01.446Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How delay-multiply-and-sum beamforming wipes away most of the sidelobes of an in-air sonar, runs in real time thanks to a piece of seventeenth-century algebra, and still cannot beat the size of the array</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l8VXED-tJLs7p6W3arBEvw.png" /><figcaption><em>Nine simulated reflectors, placed symmetrically, seen over the whole frontal hemisphere of our 32-microphone sonar. Left half: delay-and-sum. Right half: fifth-order delay-multiply-and-sum with coherence factor, computed from the same microphone signals.</em></figcaption></figure><p>In the post <a href="https://medium.com/acoustic-explorations/deblurring-sonar-images-by-driving-around-bee8f8c8364e">Deblurring sonar images by driving around</a>, a robot learnt to undo the blur in its sonar images by watching how echoes move as it drives. In this post, I look at a much cheaper question: can you make a sonar image sharper by changing the maths of the beamformer, with the same microphones and the same recording?</p><p>The answer is yes and no. Replacing an addition by a multiplication wipes away most of the clutter around every echo, but it does not let the sensor separate two objects that sit close together.</p><h3>Delay-and-sum</h3><p>Our eRTIS sensor records each echo with 32 microphones on an area about the size of a credit card. Beamforming, as I described in the first post, is a spatial filter: you choose a direction to listen in, and the beamformer tries its best to suppress everything else. The oldest way to do it, and the one the sonars built in my group used for many years, is delay-and-sum. An echo reaches each microphone at a slightly different time, so you shift every signal by the delay that belongs to the chosen direction, and add them up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/479/1*B8H_5uKAVGXQJ8YRlSiX7w@2x.png" /></figure><p>In words: for a direction <em>ψ</em>, shift the signal of every microphone <em>i</em> by its delay <em>τᵢ</em>(<em>ψ</em>), and add all <em>N</em> of them.</p><p>Echoes from the chosen direction line up and add up. Echoes from elsewhere partly cancel. The keyword is “partly”. With 32 microphones over a few centimetres, every reflector shows up as a main blob with a haze of sidelobes around it, the point-spread function of the previous post. In my simulation of our array, that haze sits only 15 to 20 dB below the echo, enough for a strong wall to drown a weak branch at the same range.</p><h3>Multiplying instead of adding</h3><p>Delay-multiply-and-sum (DMAS) keeps the delays and changes the sum. It multiplies the aligned signals of every pair of microphones and adds up all these products. To keep the result in the units of the signal, each signal is first replaced by its signed square root: the square root of its size, with its original sign.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/563/1*KK9qae3U5A3W29hzR09bcw@2x.png" /></figure><p>In words: multiply every pair of aligned signals (after a signed square root) and add up the products.</p><p>Why would that help? A product of two signals is positive when they have the same sign, and negative when they disagree. In the direction of the echo, all microphones agree, so all products are positive and pile up. In a sidelobe direction, the signals only partially line up, and the products come with mixed signs and largely cancel. A sum of pairwise products is a sum of correlations between microphones, so energy only survives where the whole array agrees.</p><p>The idea comes from biomedical imaging, where it boosted the contrast of ultrasound images (Matrone and colleagues, 2015). We were at IUS2025, one of the flagship conferences on all things ultrasound. Here, DMAS seems to have been adopted by the medical imaging community as the default algorithm to be used. We adapted it to in-air sonar and took it further, up to fifth order: products of groups of three, four or five microphones, with the matching signed roots. On top comes a coherence factor, a classic weighting that turns a pixel down when the energy across the array does not add up coherently.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jaAODyMWvFo-d7a4HeSdUA.png" /><figcaption><em>A horizontal cut through the point-spread function of the eRTIS array layout, for one reflector straight ahead (simulation, no noise). Multiplying pairs pushes the sidelobes down by roughly 15 dB. The fifth-order version with coherence factor pushes them below the bottom of the plot.</em></figcaption></figure><p>Array processing experts may wonder why we do not use MVDR or MUSIC (my group has adapted MUSIC for 3D in-air sonar too). They need statistics over many measurements, and at a leisurely 343 m/s, sound gives a sonar one measurement per call. DMAS works on a single call, while MVDR or MUSIC need spatial smoothing to get the multiple snapshots. Our arrays are irregular, and therefore, spatial smoothing does not work.</p><h3>Counting pairs</h3><p>There is a price. With 32 microphones there are 496 pairs, and 201,376 groups of five, for every pixel of every image. Remember that (<em>a</em> + <em>b</em> + <em>c</em>)² = <em>a</em>² + <em>b</em>² + <em>c</em>² + 2(<em>ab</em> + <em>ac</em> + <em>bc</em>)? All pairwise products hide inside the square of the plain sum.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/326/1*W7IF1RaB2d6B01_uIlARYg@2x.png" /></figure><p>In words: add up the signed roots, square the total, subtract the sum of the squares and halve the result. That gives all 496 products in one pass over the 32 microphones.</p><p>For groups of three, four or five, the same trick goes by the name of the Newton-Girard identities. They turn the sum over all groups into a handful of sums of powers, and they date from the seventeenth century, which is the kind of head start on a research problem I can only dream of. We implemented this on the GPU and released it as the open-source Cosys-Lab Acoustic Imaging Toolbox.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h60NlVnQRF4JC1Eb2ikjmw.png" /><figcaption><em>How many terms one pixel needs, depending on the number of microphones. Multiplying every pair or every group of five explodes. With power sums, fifth order needs five sums of 32 values each.</em></figcaption></figure><p>For an image of 1000 directions, a desktop RTX 3090 took 20 ms with delay-and-sum and 40 ms with fifth-order DMAS. The small embedded Jetson Orin Nano took 66 ms and 719 ms. So the order becomes a setting you choose per platform.</p><h3>How much cleaner?</h3><p>In the paper’s simulations, the dynamic range of the point-spread function grows from about 40 dB for delay-and-sum to almost 80 dB for fifth-order DMAS. With noisy microphones (a signal-to-noise ratio of 0 dB in each), the signal-to-noise ratio of the image rose from 32 dB for delay-and-sum to 71 dB for fifth order, and 78 dB with the coherence factor. On real data, the difference is plain to see.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a5cIbamOuPWCAeXDy_FTAA.png" /><figcaption><em>One real eRTIS frame of an indoor scene, from the paper, as range against direction. Left: delay-and-sum. Right: fifth-order DMAS with coherence factor. The paper’s colours are converted back to decibels and redrawn here.</em></figcaption></figure><h3>Behind the forest map</h3><p>The forest map of the first post was built from images made with fifth-order DMAS with coherence factor, over 2293 directions covering the frontal hemisphere. The non-linearity also explains a detail from <a href="https://medium.com/acoustic-explorations/when-the-robot-is-louder-than-the-forest-1355a8edae91">When the robot is louder than the forest</a>: a multiplying beamformer turns the smooth noise of the robot into spikes, which is why we removed that noise before beamforming.</p><h3>Conclusion</h3><p>Multiplying instead of adding gives in-air sonar images with far less clutter, at a cost that power sums keep linear in the number of microphones. It does not raise the resolution, which remains a property of the array. Multiplicative beamformers can also create artefacts from cross-terms of clutter, which the paper argues matters less in the sparse scenes of in-air sonar, where the coherence factor suppresses them. And I like that a seventeenth-century identity ended up in the map of a Norwegian forest.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fr9xx3vV_w8Q%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dr9xx3vV_w8Q&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fr9xx3vV_w8Q%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/74e2db0131e8d1394df7c2bb9384e528/href">https://medium.com/media/74e2db0131e8d1394df7c2bb9384e528/href</a></iframe><h3>Further reading</h3><p>This work was done in my group at Cosys-Lab, University of Antwerp.</p><ul><li><strong>Delay-Multiply-And-Sum Beamforming for Real-Time In-Air Acoustic Imaging.</strong> W. Jansen, W. Daems, J. Steckel. IEEE Access, 2026 (open access). <a href="https://doi.org/10.1109/ACCESS.2026.3657901">https://doi.org/10.1109/ACCESS.2026.3657901</a>, preprint <a href="https://arxiv.org/abs/2511.09165">arXiv:2511.09165</a></li><li>Code (MATLAB, CPU and CUDA): Cosys-Lab Acoustic Imaging Toolbox, <a href="https://github.com/Cosys-Lab/Acoustic-Imaging-Toolbox">https://github.com/Cosys-Lab/Acoustic-Imaging-Toolbox</a></li><li><strong>Comparing adaptations of MUSIC beamforming for 3D in-air sonar applications with coherent source-spaces.</strong> R. Kerstens, T. Verellen, J. Steckel. 8th Berlin Beamforming Conference, 2020.</li><li>Previous posts: <a href="https://medium.com/acoustic-explorations/making-three-dimensional-maps-of-the-world-using-sound-not-light-13f778b754ab">Making Three Dimensional Maps of the World using Sound, not Light</a>, <a href="https://medium.com/acoustic-explorations/deblurring-sonar-images-by-driving-around-bee8f8c8364e">Deblurring sonar images by driving around</a>, <a href="https://medium.com/acoustic-explorations/when-the-robot-is-louder-than-the-forest-1355a8edae91">When the robot is louder than the forest</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=be1bdee003a4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/acoustic-explorations/sharper-sonar-images-by-multiplying-instead-of-adding-be1bdee003a4">Sharper sonar images by multiplying instead of adding</a> was originally published in <a href="https://medium.com/acoustic-explorations">Acoustic Explorations</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Can sonar pretend to be lidar?]]></title>
            <link>https://medium.com/acoustic-explorations/can-sonar-pretend-to-be-lidar-d77167290634?source=rss-b18596ec6b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/d77167290634</guid>
            <category><![CDATA[lidar]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[machine-learning-ai]]></category>
            <dc:creator><![CDATA[Jan Steckel]]></dc:creator>
            <pubDate>Wed, 04 Nov 2026 09:16:01 GMT</pubDate>
            <atom:updated>2026-11-04T09:16:01.560Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How we trained neural networks to turn sparse ultrasonic echoes into laser-scanner data, where they fill in the gaps correctly, where they make things up, and why the physics of sound still decides which is which</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g0hBgnikJspERVa70SyEZw.png" /><figcaption><em>One simulated room, seen by two sensors. The thin teal lines are what a lidar measures, the red-and-yellow glow is what an in-air sonar hears: a handful of echoes from the spots that happen to face it.</em></figcaption></figure><p>For about two decades, people have given me the same friendly advice about my robots that use sonar sensors: “just put a lidar on it”. Lidar is the laser scanner on top of most research robots and many self-driving cars, and its scans are dense and sharp, with support in just about every robotics software package. Sonar data looks nothing like that.</p><p>So here is the question of this post, and one that many robotics readers will have asked themselves: if a robot only has a sonar, can it learn to see what a lidar would see? It can, up to a point. A network trained on pairs of real sonar and lidar scans produces lidar-like scans from echoes alone, good enough for SLAM when the wheels help a little. It also makes things up, in ways you can predict from the physics of sound.</p><p>A PhD student in my group led this line of work, together with colleagues at IDLab (University of Antwerp and imec). Consider this the revenge of the sonar people: fine, put a lidar on it, but only while training.</p><h3>Why sonar does not look like lidar</h3><p>The first post on this blog, <a href="https://medium.com/acoustic-explorations/making-three-dimensional-maps-of-the-world-using-sound-not-light-13f778b754ab">Making Three Dimensional Maps of the World using Sound, not Light</a>, explained why sonar data is so thin. Ultrasound has wavelengths of a few millimetres, so most walls act like mirrors. A lidar’s light is scattered back from every bit of a wall (diffuse reflection), but a wall only returns sound from its mirror point, the one spot where your line of sight hits it at a right angle. Corners return strongly, edges weakly, and long stretches of wall stay silent (because of specular reflections).</p><p>On top of that, the sonar’s beams are wide, so every echo is smeared over several degrees, and sound can bounce twice before it comes back. Such a multipath echo shows up as a ghost, at a place where nothing is.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zEtyKIwWIp6DcL_ZLzfScQ.png" /><figcaption><em>Illustration of the same simulated room. Left: a lidar returns points along every visible surface. Right: a sonar (computed with a simple mirror model) hears the mirror point of the back wall, the corners, the pillar, faint box edges, and one ghost echo behind the left wall, from sound that bounced off the wall and the pillar.</em></figcaption></figure><p>If you are used to lidar, or are a seasoned robotics researcher, you might hope to just feed sonar points to your favourite lidar algorithm in ROS2. Dont, that works poorly. In the simulations of <a href="https://medium.com/acoustic-explorations/where-have-i-been-and-what-is-around-me-3ab03fa2cb51">Where have I been, and what is around me?</a>, lidar-style scan matching on sonar often did worse than counting wheel rotations.</p><h3>Learning to connect the dots</h3><p>Turning sonar into lidar is an inverse problem, like reconstructing a CT image or deblurring a photo (the topic of <a href="https://medium.com/acoustic-explorations/deblurring-sonar-images-by-driving-around-bee8f8c8364e">Deblurring sonar images by driving around</a>). Many different rooms could produce the same few echoes.</p><p>A good mental model is a dot-to-dot drawing with most of the dots missing. If you have completed thousands of drawings of offices and corridors, you can still guess where the lines go: a row of glints along a line is probably one wall, and a strong blob with nothing around it is probably a pillar or a table leg. A neural network trained on many pairs of sonar and lidar scans learns exactly that kind of prior about what rooms look like.</p><p>In the first paper (IEEE Access, 2021), we mounted our eRTIS imaging sonar and a Hokuyo 2D lidar on a small TurtleBot and recorded about 5,000 synchronised pairs in hallways and offices. The network is an encoder-decoder in the style of the U-Net, a familiar architecture from image segmentation, with about eleven million parameters. It takes the five most recent sonar images, so it can combine echoes from slightly different viewpoints, and outputs the lidar scan between 0.2 and 5 metres.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p1VXjOKRvA7tkbdYjeZ5VA.png" /><figcaption><em>Real data in four indoor scenes, seen from above with the sensor at the bottom centre and range rings every metre up to 5 m. Top: sonar input. Middle: the lidar scan the network predicts from it. Bottom: the real lidar scan, used only as reference. Fewer reflectors (second column) give a less complete prediction.</em></figcaption></figure><p>The predictions are smoother and less detailed than real lidar. But the walls are where they should be, and so are the closest obstacles, which matter most for a robot that wants to avoid them.</p><h3>Ghost walls</h3><p>The same paper also shows where it goes wrong. We first trained the network on simulated data (about 12,500 samples), from a simulator that left out multiple reflections. Fine-tuning that model on real data did not work well, and the corridor below shows why. In a real corridor, the sonar hears a series of ghost echoes, from sound that bounced more than once before it came back. A network that has never seen a ghost does what most of us would do when we see one: it panics, and draws a wall.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-g6fb9x2QVU2YyHDu6LgPw.png" /><figcaption><em>A real corridor. The model trained on simulation (middle) turns the multipath echoes in the sonar image (left) into walls and blobs that the lidar (right) does not see. Retrained on real data, the network handles the corridor correctly (third column of the previous figure).</em></figcaption></figure><p>That is the general lesson about hallucination in sensor translation. The network fills the gaps with what it has learned rooms usually look like, so the output always looks plausible. Whether it is right depends on whether the training data contained the physics of the situation, such as mirror-like walls, rooms with few reflectors and multipath. My own guess, untested, is that the same happens in rooms that look nothing like the training data.</p><h3>A shared latent space</h3><p>Next, S2L-SLAM (IEEE Sensors, 2021) fed the predicted scans into off-the-shelf lidar SLAM, and R2L-SLAM (2023) did the same with a small mmWave radar.</p><p>Training one separate translator per sensor does not scale, though. LiDAR-BIND (IEEE Robotics and Automation Letters, 2024) gives every sensor its own encoder into one shared latent space, defined by the lidar, and a single decoder that writes out a lidar scan. Think of it as one shared description of the scene that every sensor learns to write in, a bit like the way models such as CLIP put images and text in one space. Two sensors’ descriptions can then simply be added, and if one of them points in a very different direction, that sensor is probably wrong, which gives you a simple anomaly detector. Each sensor only needs to be recorded together with the lidar, never all at once.</p><h3>Consistency over time</h3><p>There was a catch. Every frame was translated on its own, so the predicted scans jittered from one moment to the next as echoes came and went with the viewpoint. SLAM hates that. It tracks the robot by matching each new scan against what it saw just before, and a flickering wall looks like motion.</p><p>LiDAR-BIND-T adds memory: consecutive frames should get similar descriptions, the shift between two predicted scans should match the shift between the real lidar scans, and a small module fuses a window of five frames.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P6d39D7AtPnBBEXk5RYxDw.png" /><figcaption><em>Average position error of Cartographer SLAM on indoor recordings, measured against the trajectory from the wheel encoders. Without wheel odometry, the scan matching relies on the predicted scans alone.</em></figcaption></figure><p>When the SLAM system may also use the wheels, all sonar-based versions stay within 6 to 10 cm of the reference, against 4 cm for real lidar. Keep in mind that the reference trajectory itself comes from the wheel encoders. When the predicted scans have to carry the whole job, the shared-space model averages 1.5 m of error, the temporal version brings that down to 0.5 m, and a dedicated sonar network without the shared space still does better, at 0.2 m. The paper describes this as a trade-off between the flexibility of the shared space and accuracy.</p><h3>So, can sonar pretend to be lidar?</h3><p>Partly. A network can learn the lidar’s view of a room from echoes, convincingly enough for SLAM when the wheels help. What it cannot do is invent information the echoes never carried: a silent wall stays a guess, and a ghost stays a ghost unless the training data taught otherwise. The physics of sound still decides where the network can be trusted, which is why I would always keep the raw echoes next to the pretty lidar-like output.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fjlx1S-tslmo%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Djlx1S-tslmo&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fjlx1S-tslmo%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/73a964f666ad0c954e053b43f78dcc7b/href">https://medium.com/media/73a964f666ad0c954e053b43f78dcc7b/href</a></iframe><h3>Further reading</h3><p>This work was done in my group (Cosys-Lab) together with IDLab, University of Antwerp and imec.</p><ul><li><strong>Predicting LiDAR Data From Sonar Images.</strong> N. Balemans, P. Hellinckx, J. Steckel. IEEE Access 9, 2021 (open access). <a href="https://doi.org/10.1109/ACCESS.2021.3072551">https://doi.org/10.1109/ACCESS.2021.3072551</a></li><li><strong>S2L-SLAM: Sensor Fusion Driven SLAM using Sonar, LiDAR and Deep Neural Networks.</strong> N. Balemans, P. Hellinckx, S. Latré, P. Reiter, J. Steckel. IEEE Sensors 2021. <a href="https://doi.org/10.1109/SENSORS47087.2021.9639772">https://doi.org/10.1109/SENSORS47087.2021.9639772</a></li><li><strong>R2L-SLAM: Sensor Fusion-Driven SLAM using mmWave Radar, LiDAR and Deep Neural Networks.</strong> N. Balemans, L. Hooft, P. Reiter, A. Anwar, J. Steckel, S. Mercelis. IEEE Sensors 2023. <a href="https://doi.org/10.1109/SENSORS56945.2023.10324990">https://doi.org/10.1109/SENSORS56945.2023.10324990</a></li><li><strong>LiDAR-BIND: Multi-Modal Sensor Fusion Through Shared Latent Embeddings.</strong> N. Balemans, A. Anwar, J. Steckel, S. Mercelis. IEEE Robotics and Automation Letters 9(11), 2024. <a href="https://doi.org/10.1109/LRA.2024.3457384">https://doi.org/10.1109/LRA.2024.3457384</a></li><li><strong>Multi-Modal Sensor Fusion in Latent Embedding Space for Robust Autonomous Navigation.</strong> N. Balemans, A. Anwar, J. Steckel, S. Mercelis. IEEE Sensors 2024. <a href="https://doi.org/10.1109/SENSORS60989.2024.10785240">https://doi.org/10.1109/SENSORS60989.2024.10785240</a></li><li><strong>LiDAR-BIND-T: Improved and Temporally Consistent Sensor Modality Translation and Fusion for Robotic Applications.</strong> N. Balemans, A. Anwar, J. Steckel, S. Mercelis. Preprint, arXiv:2509.05728. <a href="https://arxiv.org/abs/2509.05728">https://arxiv.org/abs/2509.05728</a> [CHECK: replace with the 2026 published version]</li><li>Previous posts: <a href="https://medium.com/acoustic-explorations/making-three-dimensional-maps-of-the-world-using-sound-not-light-13f778b754ab">Making Three Dimensional Maps of the World using Sound, not Light</a>, <a href="https://medium.com/acoustic-explorations/deblurring-sonar-images-by-driving-around-bee8f8c8364e">Deblurring sonar images by driving around</a> and <a href="https://medium.com/acoustic-explorations/where-have-i-been-and-what-is-around-me-3ab03fa2cb51">Where have I been, and what is around me?</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d77167290634" width="1" height="1" alt=""><hr><p><a href="https://medium.com/acoustic-explorations/can-sonar-pretend-to-be-lidar-d77167290634">Can sonar pretend to be lidar?</a> was originally published in <a href="https://medium.com/acoustic-explorations">Acoustic Explorations</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How echoes move when you move]]></title>
            <link>https://medium.com/acoustic-explorations/how-echoes-move-when-you-move-2fc81ef75f88?source=rss-b18596ec6b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/2fc81ef75f88</guid>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[sonar]]></category>
            <category><![CDATA[perception]]></category>
            <category><![CDATA[sensors]]></category>
            <dc:creator><![CDATA[Jan Steckel]]></dc:creator>
            <pubDate>Sun, 01 Nov 2026 11:06:01 GMT</pubDate>
            <atom:updated>2026-11-01T11:06:01.588Z</atom:updated>
            <content:encoded><![CDATA[<p><em>When a sonar moves through a still world, every echo moves in a way that is fixed by the sonar’s own motion. My lab has used that to steer robots, and to let sonars and simulated bats learn about themselves.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kfnBoWRdq_O2U4w_5b9uTw.png" /><figcaption><em>Each trail is one reflector, as seen by a sonar that drives forward while turning gently: azimuth from left to right, range from bottom to top. Computed from the acoustic flow equations.</em></figcaption></figure><p>When you are like me, you like to look out of the window of a moving train or car. The poles next to the track or road flash by, the trees further away glide past, and the hills on the horizon hardly move at all. That pattern of motion across your eyes is called optic flow, and animals use it all the time. Honeybees flying through a tunnel keep to the middle by balancing the flow they see on their left and right. Reading this lineage of paper was actually one of the defining moments in my early scientist life, and I can only recommend to look into these.</p><p>A sonar has the same thing, but then for the echoes in the sensing process. When a sonar moves through a world that stands still, every echo changes its range and direction in a way that is completely fixed by the sonar’s own motion. This motion is described by a set of differential equations, and we call this motion acoustic flow. In two earlier posts, it did a lot of the heavy lifting without much of an introduction. So in this post, I look at the question behind it: how do echoes move when you move, and why is that so useful?</p><p>The short answer: two simple equations tell you where every echo will be at the next call. That is enough to steer a robot through a corridor without a map. It also turned out to be a very good teacher for a sonar that has to learn about its own imperfections.</p><h3>An echo passing by</h3><p>Picture a robot with a sonar that drives straight ahead at 1 metre per second. One metre to its right, five metres ahead, stands a pole. At first, the echo of the pole comes back from almost straight ahead, about five metres away. As the robot drives on, the range shrinks steadily, and the direction swings to the right, slowly at first and then faster and faster. When the robot passes the pole, the echo comes from 90 degrees to the right, at exactly one metre.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y1ohPhDWvnbq-PglLuW4YQ.png" /><figcaption><em>A pole one metre to the right of the path of a sonar driving at 1 m/s (left), and what the sonar measures (right). The range drops almost linearly, and the azimuth changes slowly while the pole is far away and fast when it is close.</em></figcaption></figure><p>That is the train window again. The range tells you how far away the pole is, and the way its direction changes tells you how far it is from your path.</p><h3>The flow equations</h3><p>Some maths ahead. Only two equations, I promise.</p><p>For a sonar that faces forward on a robot driving with speed <em>V</em> and turning with turn rate <em>ω</em>, a reflector at range <em>r</em> and azimuth <em>θ</em> moves through the sonar image as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/138/1*G23Rwtg8lATXzxl41qiOyA@2x.png" /></figure><p>In words: the range shrinks at the full speed of the robot for a reflector straight ahead, and does not change at all for a reflector exactly to the side.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/145/1*bQfDG8aXIJGD0t-JF0S6xw@2x.png" /></figure><p>In words: driving pushes every reflector outwards, away from straight ahead, and more strongly the closer it is. Turning shifts all reflectors by the same angle, whatever their range. (Here <em>θ</em> is measured to the right of straight ahead, and <em>ω</em> is positive for a turn to the right.)</p><p>My PhD supervisor (Herbert Peremans, a true genius to whom I owe a great deal) and I first wrote this down for robot control at ICRA in 2014. The robot only needs its own speed and turn rate, which wheel odometry and a gyroscope provide. If you are used to cameras in your robotics systems, this is the sonar version of optical flow, the principle that lets an optical mouse track how far you moved it.</p><h3>Flow lines</h3><p>Here comes the mental model. When the robot drives straight, the quantity <em>r</em> · sin <em>θ</em> never changes. It is simply the sideways distance between the reflector and the robot’s path. So every reflector slides along its own flow line through the range-azimuth image, and that line is labelled by how far the reflector is from your path.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GqYXD1sHGrS5wysW1K2VZw.png" /><figcaption><em>How reflectors move through the sonar image (teal arrows), at 1 m/s. Left: driving straight. Each magenta line collects all reflectors at the same sideways distance from the path. Anything in the shaded zone, closer than 25 cm to the path, will hit the robot. Right: turning to the right at 25 degrees per second. The flow now swirls around the centre of the turn.</em></figcaption></figure><p>This has a lovely consequence. A reflector inside the narrow band around straight ahead (the collision zone) is on a collision course, whatever its range, and you can read that straight from the image, without a map. And when the robot drives along a wall, the echoes of that wall pile up along one flow line. Keep that line where you want it, and you follow the wall.</p><h3>Bats did it first</h3><p>The idea of acoustic flow comes from bats. In experiments by Kathrin Kugler and colleagues in Munich, bats flew through narrow passages whose side walls were made more or less echoic. How the bats flew depended on how strong the echoes from the walls were, even though the distance stayed the same. The authors concluded that sensory flow guides bat flight, much like optic flow guides visual animals.</p><p>Twenty years earlier, David Lee, James Simmons and colleagues proposed that bats could steer using how fast the delay of an echo changes. Divide the range by how fast it shrinks, and you get the time until you hit the object, a quantity Lee called tau. In the flow equations, that is just <em>r</em> divided by −d<em>r</em>/d<em>t</em>.</p><h3>Steering a robot with echoes</h3><p>In the ICRA paper and its follow-up in IEEE Sensors Journal (2017), we turned these flow cues into a layered controller for a robot with a 3D sonar. Simple behaviours (obstacle avoidance, corridor following, handling corners and T-junctions) each look at their own part of the flow field. The controller was validated in simulation first, and then transferred to a real mobile robot.</p><p>My team took this further. In a paper at IPIN 2021, we made the flow model work for any position and orientation of the sonar on the robot, and fused several sonars into one controller. In simulation, ten sensor configurations were each run 15 times, without a single collision. The journal version in Sensors (2022) added tests on a real robot in an office.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5deXvcy8AWwoCSLCnf5cKQ.png" /><figcaption><em>Illustration, not from the papers. A simulated robot starts off-centre in an S-shaped corridor. At every call, it takes the sideways distance r · sin θ of the nearest echoes on its left and right, and turns towards the side with more room. It uses no map and does not know where it is.</em></figcaption></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FslqEtzFBPL0%3Fstart%3D693%26feature%3Doembed%26start%3D693&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DslqEtzFBPL0&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FslqEtzFBPL0%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/14ad27189cb5733b6d5552b7a1eaa9a5/href">https://medium.com/media/14ad27189cb5733b6d5552b7a1eaa9a5/href</a></iframe><h3>Acoustic flow as a teacher</h3><p>Acoustic flow has a second life in my recent work. If the flow tells you where every real echo goes next, anything that goes somewhere else is suspicious. In <a href="https://medium.com/acoustic-explorations/deblurring-sonar-images-by-driving-around-bee8f8c8364e">Deblurring sonar images by driving around</a>, sidelobes give themselves away because they stick to the reflector that caused them instead of following their own flow. A network that has to predict the next images learns to remove them.</p><p>In <a href="https://medium.com/acoustic-explorations/how-a-bat-could-learn-its-own-ears-06d867593457">How a bat could learn its own ears</a>, a simulated bat-like agent uses the 3D version of the same equations, with a head that turns and tilts. If its idea of its own ears is wrong, its predictions of the next echoes fail, and that error is enough to learn the ears.</p><h3>Conclusion</h3><p>Acoustic flow is a small piece of geometry with a long reach. In my lab, it first served to steer robots without a map, and these days it is the teacher that lets a sonar learn about itself. In each case, the sensor’s own motion turns one snapshot into a prediction that can be checked at the next call. I find it hard to think of a cheaper teacher: a gyroscope and some wheel encoders.</p><h3>Further reading</h3><ul><li><strong>Acoustic flow for robot motion control.</strong> H. Peremans, J. Steckel. IEEE International Conference on Robotics and Automation (ICRA), 2014, pp. 316–321. <a href="https://doi.org/10.1109/ICRA.2014.6906628">https://doi.org/10.1109/ICRA.2014.6906628</a></li><li><strong>Acoustic flow-based control of a mobile platform using a 3D sonar sensor.</strong> J. Steckel, H. Peremans. IEEE Sensors Journal 17(10), 2017, pp. 3131–3141. <a href="https://doi.org/10.1109/JSEN.2017.2688476">https://doi.org/10.1109/JSEN.2017.2688476</a></li><li><strong>Adaptive acoustic flow-based navigation with 3D sonar sensor fusion.</strong> W. Jansen, D. Laurijssen, J. Steckel. IPIN 2021. <a href="https://doi.org/10.1109/IPIN51156.2021.9662596">https://doi.org/10.1109/IPIN51156.2021.9662596</a>, preprint <a href="https://arxiv.org/abs/2208.10823">arXiv:2208.10823</a></li><li><strong>Real-time sonar fusion for layered navigation controller.</strong> W. Jansen, D. Laurijssen, J. Steckel. Sensors 22(9), 3109, 2022. <a href="https://doi.org/10.3390/s22093109">https://doi.org/10.3390/s22093109</a></li><li><strong>Echo-acoustic flow affects flight in bats.</strong> K. Kugler, W. Greiter, H. Luksch, U. Firzlaff, L. Wiegrebe. Journal of Experimental Biology 219, 2016. <a href="https://doi.org/10.1242/jeb.139345">https://doi.org/10.1242/jeb.139345</a></li><li><strong>Steering by echolocation: a paradigm of ecological acoustics.</strong> D. N. Lee, J. A. Simmons, P. A. Saillant, F. Bouffard. Journal of Comparative Physiology A 176, 1995. <a href="https://doi.org/10.1007/BF00219060">https://doi.org/10.1007/BF00219060</a></li><li>Previous posts: <a href="https://medium.com/acoustic-explorations/deblurring-sonar-images-by-driving-around-bee8f8c8364e">Deblurring sonar images by driving around</a> and <a href="https://medium.com/acoustic-explorations/how-a-bat-could-learn-its-own-ears-06d867593457">How a bat could learn its own ears</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2fc81ef75f88" width="1" height="1" alt=""><hr><p><a href="https://medium.com/acoustic-explorations/how-echoes-move-when-you-move-2fc81ef75f88">How echoes move when you move</a> was originally published in <a href="https://medium.com/acoustic-explorations">Acoustic Explorations</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why a wall can stay silent to a sonar]]></title>
            <link>https://medium.com/acoustic-explorations/why-a-wall-can-stay-silent-to-a-sonar-f63f7f59f684?source=rss-b18596ec6b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/f63f7f59f684</guid>
            <category><![CDATA[physics]]></category>
            <category><![CDATA[ultrasound]]></category>
            <category><![CDATA[neuroscience]]></category>
            <category><![CDATA[bats]]></category>
            <category><![CDATA[robotics]]></category>
            <dc:creator><![CDATA[Jan Steckel]]></dc:creator>
            <pubDate>Wed, 28 Oct 2026 08:01:03 GMT</pubDate>
            <atom:updated>2026-10-28T08:01:03.902Z</atom:updated>
            <content:encoded><![CDATA[<p><em>At ultrasonic wavelengths, most smooth surfaces reflect sound like a mirror. That makes life hard for robots, and one bat turns it into a hunting trick</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XD8v85xKyXFk96Lk1K04AA.png" /><figcaption><em>A short ultrasonic pulse (computed) leaves its source on the left and hits a smooth, slanted wall. The reflected wave travels off upwards, away from where it came from, so the source never hears this wall.</em></figcaption></figure><p>Picture a room in which every wall is a mirror. The lights are off, and the only light is a torch strapped to your forehead. Where do you see the walls? Only in the one spot where you see your own torch reflected, which is the spot straight in front of you. Turn your head a little, and the wall in front of you goes dark, even though it is still right there.</p><p>That is, roughly, how an in-air sonar experiences a room. Earlier posts on this blog mention in passing that at ultrasonic wavelengths “many surfaces act like almost perfect mirrors”. In this post, I will answer a concrete question: why does a wall in front of a sonar sometimes stay completely silent?</p><p>The short answer is that a wall that looks rough to a laser, is actually very polished to sound. The longer answer involves wavelengths, corners and edges, and a small bat in Panama that uses the very same effect to find insects sitting on leaves.</p><h3>Mirrors are everywhere in echolocation</h3><p>When a sound wave hits a large, smooth surface, it bounces off like light from a mirror: the angle going out equals the angle coming in. This is called a specular reflection. For an echo to come back to a sonar, the sound has to hit the surface at a right angle, so only one spot on a flat wall can echo back: the foot of the perpendicular line from the sensor to the wall that intersects your sensor. Everything else on the wall reflects the call away, into the room.</p><p>Where that spot lies depends on where the sensor is, so when the robot moves, the spot slides along the wall with it. If that spot lies outside the sensor’s field of view, which happens when the robot drives along the wall or looks at it at a steep angle, the wall stays silent, even though it is only a metre away.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-kLKJFth-OYfNuombfHJMA.png" /><figcaption><em>Illustration. Left: a lidar gets a return from every point of the wall it looks at. Right: a sonar driving past the same wall only hears the point where its line of sight meets the wall at a right angle (magenta). That point slides along with the robot, and once it leaves the field of view (shaded), the wall falls silent.</em></figcaption></figure><p>If you are used to lidar in your robotics systems, this is the moment to forget most of your intuitions. A lidar sees a wall from almost any angle, because each laser spot scatters some light back towards the sensor. (Lidar has its own mirror problems, by the way: ask anyone who has tried to map a glass facade.)</p><h3>Rough for light, smooth for sound</h3><p>What decides between mirror and scatterer is the size of the bumps compared to the wavelength. A lidar typically uses near-infrared light with a wavelength of around a thousandth of a millimetre. Our sonar uses calls from 25 to 70 kHz, with wavelengths of about 5 to 14 mm. A painted wall has bumps much smaller than a millimetre. For the laser, those bumps are mountains, and every spot scatters light in all directions (diffuse reflection). For the sound, they are invisible, and the wall acts as a mirror.</p><p>The classic rule of thumb for this is the Rayleigh criterion: a surface reflects specularly when its height variations stay below about one eighth of the wavelength (divided by the cosine of the angle of incidence, for the purists). At 40 kHz, one eighth of a wavelength is about one millimetre. Most indoor walls, floors, doors and desks pass that test easily.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*__A8JYnKM6TjtECA7ASFKg.png" /><figcaption><em>How much sound a bumpy surface sends in each direction, for a call that arrives from 30 degrees to the left of the surface normal (Kirchhoff approximation, averaged over 200 random surfaces, levels relative to a perfect mirror). A smooth surface sends everything into the mirror direction. Only when the bumps reach the size of a wavelength does a noticeable part of the sound come back to the sensor.</em></figcaption></figure><h3>Walls, corners and edges</h3><p>Not everything in a room is a mirror. A corner, where two walls meet at a right angle, bounces the sound twice and sends it straight back to where it came from, over a wide range of angles (a retroreflector). Cyclists know this principle from the reflectors on their bikes, which are made of many small corners. The edge of a door frame or a pillar does something else again: it diffracts sound in all directions, which gives a weak echo from almost anywhere.</p><p>In a single snapshot, a wall, a corner and an edge can all produce one echo, at a certain range and direction, and a simple sonar cannot tell them apart. Billur Barshan and Roman Kuc, and later Lindsay Kleeman and Roman Kuc, showed how to tell them apart in the 1990s, using more than one transmitter and receiver. The way I like to think about it is that you have to watch the echo move while you move, and correlate both motion patterns you observe. The echo of a corner or an edge stays where it is, and the echo of a wall slides along with you.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jrrp4XANqms4c41Em1WiOg.png" /><figcaption><em>The same robot at three positions in front of a wall, a corner and an edge. The echo of the wall comes from a different spot every time. The echoes of the corner and the edge come from the same spot, and the edge echo is much weaker.</em></figcaption></figure><h3>What this means for a robot</h3><p>This one physical fact explains many design choices from earlier posts. In <a href="https://medium.com/acoustic-explorations/making-three-dimensional-maps-of-the-world-using-sound-not-light-13f778b754ab">the first post</a>, I wrote that you cannot simply feed sonar points to a lidar mapping algorithm. Lidar-style scan matching assumes that a wall produces the same points when you look at it again from a slightly different position. With sonar, it produces a different point, or none at all. In my simulations, scan matching on sonar often did worse than counting wheel rotations, which is why RTISSLAM and BatSLAM 2.0 never match single sonar scans against a map.</p><p>It also explains why sonar memory is local. In <a href="https://medium.com/acoustic-explorations/where-have-i-been-and-what-is-around-me-3ab03fa2cb51">the post on pose graphs</a>, a stored BatSLAM 2.0 template was only recognised, in simulation, within about 15 cm sideways and 25 degrees in heading of where it was recorded. And <a href="https://medium.com/acoustic-explorations/how-a-bat-could-build-a-3d-model-from-its-own-echoes-83de9e9c48e4">SonoNERF</a> needed to know from which direction a point was ensonified, because mirror-like surfaces only answer from some directions. Physics does not care about your SLAM pipeline, unfortunately.</p><h3>A bat that uses the mirror</h3><p>Bats have had tens of millions of years longer to deal with this than roboticists. The common big-eared bat, <em>Micronycteris microtis</em>, picks motionless insects, such as dragonflies, off leaves in the rainforest at night. For a long time, this was considered close to impossible with echolocation alone: a silent insect sitting on a leaf is a small echo buried under the large echo of the leaf.</p><p>In a study during my PhD, we looked at how the bat pulls this off . We measured the echoes of a leaf with and without an insect from more than 500 positions on a hemisphere around it, with a biomimetic sonar. Seen head-on, the leaf returns a strong echo that masks the insect. Seen from an oblique angle, the leaf acts as a mirror and sends the call away, while the insect, which is small and irregular, still sends some sound back. From these measurements, we predicted a range of optimal approach angles, more specifically, in the range from 40 to 80 degrees from the leaf normal.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4iC-d8b_9xr1YwHbjR2W5Q.png" /><figcaption><em>Head-on (left), the leaf echoes strongly and the insect’s echo disappears under it. From an oblique angle (right), the leaf mirrors the call away and the insect stands out. The teal band marks the range of approach angles predicted from the leaf measurements.</em></figcaption></figure><p>When doing behavioural experiments on bats, the angles for their approaches fall exactly in the range of angles where the difference between the empty leaf and the full leaf is most salient, as is described in detail in the paper (see end of the post). A nice video of such an approach can be seen below.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FaJdEFgdTpl4%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DaJdEFgdTpl4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FaJdEFgdTpl4%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/d55f643a2c3ac389d4be90666969b302/href">https://medium.com/media/d55f643a2c3ac389d4be90666969b302/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F_h9p5dmUeeI%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D_h9p5dmUeeI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F_h9p5dmUeeI%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/efaf7e51a155124b7e0583436351178b/href">https://medium.com/media/efaf7e51a155124b7e0583436351178b/href</a></iframe><h3>Why this matters</h3><p>For a robot, specular reflection is mostly a nuisance: walls fall silent, echo points slide, and lidar-style algorithms break. The bat shows that the same physics can also be an advantage, if you get to choose where to look from. A sonar robot could do the same, by planning its movements to get the echoes it needs, and that is something I would like to explore in a future post. For now, the lesson is the one that runs through this whole blog: sound is not light, and the algorithms have to respect that.</p><h3>Further reading</h3><ul><li><strong>Bats Actively Use Leaves as Specular Reflectors to Detect Acoustically Camouflaged Prey.</strong> I. Geipel, J. Steckel, M. Tschapka, D. Vanderelst, H.-U. Schnitzler, E. K. V. Kalko, H. Peremans, R. Simon. <em>Current Biology</em> 29(16), 2731–2736, 2019: <a href="https://doi.org/10.1016/j.cub.2019.06.076">https://doi.org/10.1016/j.cub.2019.06.076</a></li><li><strong>Mobile Robot Sonar for Target Localization and Classification.</strong> L. Kleeman and R. Kuc. <em>The International Journal of Robotics Research</em> 14(4), 295–318, 1995: <a href="https://doi.org/10.1177/027836499501400401">https://doi.org/10.1177/027836499501400401</a></li><li><strong>Differentiating Sonar Reflections from Corners and Planes by Employing an Intelligent Sensor.</strong> B. Barshan and R. Kuc. <em>IEEE Transactions on Pattern Analysis and Machine Intelligence</em> 12(6), 560–569, 1990: <a href="https://doi.org/10.1109/34.56192">https://doi.org/10.1109/34.56192</a></li><li>Previous posts: <a href="https://medium.com/acoustic-explorations/making-three-dimensional-maps-of-the-world-using-sound-not-light-13f778b754ab">Making Three Dimensional Maps of the World using Sound, not Light</a>, <a href="https://medium.com/acoustic-explorations/how-a-bat-could-build-a-3d-model-from-its-own-echoes-83de9e9c48e4">How a bat could build a 3D model from its own echoes</a> and <a href="https://medium.com/acoustic-explorations/where-have-i-been-and-what-is-around-me-3ab03fa2cb51">Where have I been, and what is around me?</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f63f7f59f684" width="1" height="1" alt=""><hr><p><a href="https://medium.com/acoustic-explorations/why-a-wall-can-stay-silent-to-a-sonar-f63f7f59f684">Why a wall can stay silent to a sonar</a> was originally published in <a href="https://medium.com/acoustic-explorations">Acoustic Explorations</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Where should a sonar look next?]]></title>
            <link>https://medium.com/acoustic-explorations/where-should-a-sonar-look-next-29e316fec288?source=rss-b18596ec6b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/29e316fec288</guid>
            <category><![CDATA[active-inference]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Jan Steckel]]></dc:creator>
            <pubDate>Mon, 26 Oct 2026 13:52:59 GMT</pubDate>
            <atom:updated>2026-10-26T13:52:59.661Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A simulated robot with a sonar on its arm predicts what each view should sound like, looks where it expects to learn the most about the identity of the object, and stops when it is sure enough. basically, Active inference, but without the pragmatic part</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i4NIzbQo0embmzzNaGbUMQ.png" /><figcaption><em>Rendered from the simulation (an illustration, not a photo). The rings and labels on the shelf are the robot’s current beliefs: where it thinks the objects are, and what it thinks they are.</em></figcaption></figure><p>Remember the board game Guess Who? You do not ask “Is it Bernard?”. You ask “Does your person wear glasses?”, because that question rules out about half of the faces, whatever the answer. Good players pick the question whose answer they cannot predict yet. If you never played it, you missed one of the finest pieces of applied information theory ever sold in a toy shop.</p><p>A robot, or a bat for that matter, that recognises objects with sonar has the same problem to solve, only its questions are viewpoints, and every one of them costs time / energy. In this post, I follow one simulated robot through a recognition task, step by step: what it hears, what it expects to hear, how it decides where to look next, and when it decides it has heard enough.</p><h3>Find the twin</h3><p>A mobile robot with an arm, carrying a 3D sonar like our eRTIS at its end, inspects an unknown object on one shelf, the called the query object. It then drives to a second shelf with three objects and has to say which one is the same kind. The objects are everyday shapes: mug, bottle, bowl, can, box, sphere and torus.</p><p>The robot works blind. It has no camera and no map of the environment. It does not know where the objects are, and its odometry estimate drifts while driving. It finds the shelves and the objects with its own sonar. Everything runs in our physics-based echo simulator. We leave the real-world implementation as an excercise to the reader :).</p><h3>An echo fingerprint</h3><p>Every ping gives the robot a 3D image of where echoes come from, the so-called energyscape. In it, an object is little more than a bright blob. To recognise it, the robot focuses on the object, a bit like turning a microphone towards a speaker, and looks at how the returning echo is spread out in distance and how its spectrum looks. Those two curves are the echo fingerprint of that one view.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dM_EOogYXbWDPpiXgAIiEA.png" /><figcaption><em>Left: one ping of the simulated sonar, the brighter the stronger the echo. Right: focused on one object, its echo fingerprint, as echo strength along the distance around the object (top) and as a spectrum (bottom).</em></figcaption></figure><p>The catch is that the fingerprint depends on where you stand. At ultrasonic wavelengths, smooth surfaces act like mirrors (specular reflection). Seen face-on, a box returns a strong echo. Seen on its corner, most of the sound bounces elsewhere, and what comes back can look like a round object. A single view is rarely enough.</p><h3>Templates</h3><p>So what should a mug sound like from 30 degrees to its left? To answer that, the robot needs a model of how objects produce echoes, a generative model in the proper term. Ours is a library of templates. In 40 simulated rooms where we knew where every object stood, the sonar looked at every object from every direction it could reach. Sorted by class and viewing angle, those views are the templates. We added 120 of the robot’s own earlier runs, so the templates also know what its own, imperfect measurements look like.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*355Ld8L0JOx056mJJwzAvQ.png" /><figcaption><em>Templates: the average echo fingerprint (echo strength along the distance around the object) of every class, for every viewing angle relative to the object. The mug and the box (magenta) change with the angle. The round objects look the same from every side.</em></figcaption></figure><p>The templates show at a glance where looking around pays off. The mug’s handle draws a diagonal trace as it swings past, and the box reflects specularly for every face. A sphere or a can sounds the same from every side, so walking around it teaches you nothing new.</p><h3>Predict, look, compare, update</h3><p>With the templates in hand, the robot keeps a belief about every object: how likely each class is, and how the object is turned. The strategy has three phases. A detailed scan of the query from every reachable direction, a quick look at each test object, and then a loop.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*URWG-0PBEFFPmuSInsjkcA.png" /><figcaption><em>The active-inference loop of the robot. It answers as soon as it is 90 % sure which test object matches, or after 12 extra views.</em></figcaption></figure><p>In the loop, the robot first predicts what each possible next view would show under every hypothesis it still holds. That is Guess Who again. If all hypotheses predict the same fingerprint, the view is useless. If they predict clearly different fingerprints, the view will settle something. The proper name for this score is mutual information, here between the answer (which test object matches) and the echo of the next view. Because a view on the other shelf needs a drive, the robot divides that information by the time it takes to obtain the new view. It takes the best view, compares the echo it gets with the prediction, and updates its beliefs with Bayes’ rule.</p><h3>Watch it work</h3><p>The video shows two complete runs, rendered from the simulation, with a short introduction. You see the robot’s own map and the raw sonar images, and for every step the predicted and the measured fingerprint, the belief update, and how much information the best candidate views promised.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fr8rGO9uLoV4%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dr8rGO9uLoV4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fr8rGO9uLoV4%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/38961c86fd096bbfe310188ee812f247/href">https://medium.com/media/38961c86fd096bbfe310188ee812f247/href</a></iframe><h3>Expected free energy, without preferences</h3><p>What the robot does has a name. It is called <strong>active inference</strong>, the framework of Karl Friston and colleagues. In active inference, a brain (or a robot) carries a generative model of how its observations come about, updates its beliefs with every new observation, and chooses actions that minimise expected free energy.</p><p>Expected free energy weighs two things. An action can be worth taking because it tells you something you do not know yet, the epistemic value. It can also be worth taking because it brings you closer to outcomes you prefer, the pragmatic value. Our robot has no preferred outcomes: no viewpoint leads somewhere it would rather be. Then only the first part is left, and minimising expected free energy means choosing the view that is expected to tell the most about the answer. That is exactly what the robot does.</p><p>So everything is in place: a generative model (the templates), belief updating with every echo, predictions of every possible view, and actions chosen by expected free energy. The robot simply has no pragmatic actions yet.</p><p>Bats seem to do something related when they inspect an object. They change their calls and their approach paths, as if they were collecting the views they need. That is the motivation here, and I do not claim that this is what happens in a bat’s brain.</p><h3>Conclusion</h3><p>A sonar robot can predict what each view of an object should sound like, choose its next view by how much it expects to learn, and stop when it is sure enough. In simulation, the stopping rule was reliable, and keeping track of where things are mattered as much as what they are. The next step is to calibrate the expected information against what views really change, from the steps we logged. Once that works, I would love to try this with a real eRTIS on a real arm.</p><h3>Further reading</h3><ul><li>The video with both runs and the explainers: <a href="https://www.youtube.com/watch?v=r8rGO9uLoV4">Finding an Object’s Twin by Sound</a></li><li>Previous posts: <a href="https://medium.com/acoustic-explorations/why-our-ertis-sonar-sees-better-left-right-than-up-down-9767c19b727d">Why our eRTIS sonar sees better left-right than up-down</a>, <a href="https://medium.com/acoustic-explorations/how-a-bat-could-build-a-3d-model-from-its-own-echoes-83de9e9c48e4">How a bat could build a 3D model from its own echoes</a>, <a href="https://medium.com/acoustic-explorations/how-a-bat-could-learn-its-own-ears-06d867593457">How a bat could learn its own ears</a> and <a href="https://medium.com/acoustic-explorations/where-have-i-been-and-what-is-around-me-3ab03fa2cb51">Where have I been, and what is around me?</a></li><li>Why a wall can stay silent to a sonar [CHECK: link after 28 Oct]</li><li>eRTIS: A Fully Embedded Real Time 3D Imaging Sonar Sensor for Robotic Applications. R. Kerstens, D. Laurijssen, J. Steckel. ICRA 2019. <a href="https://doi.org/10.1109/ICRA.2019.8794419">https://doi.org/10.1109/ICRA.2019.8794419</a></li><li>Active inference and epistemic value. K. Friston, F. Rigoli, D. Ognibene, C. Mathys, T. Fitzgerald, G. Pezzulo. Cognitive Neuroscience 6(4), 2015. <a href="https://doi.org/10.1080/17588928.2015.1020053">https://doi.org/10.1080/17588928.2015.1020053</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=29e316fec288" width="1" height="1" alt=""><hr><p><a href="https://medium.com/acoustic-explorations/where-should-a-sonar-look-next-29e316fec288">Where should a sonar look next?</a> was originally published in <a href="https://medium.com/acoustic-explorations">Acoustic Explorations</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why our eRTIS sonar sees better left-right than up-down]]></title>
            <link>https://medium.com/acoustic-explorations/why-our-ertis-sonar-sees-better-left-right-than-up-down-9767c19b727d?source=rss-b18596ec6b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/9767c19b727d</guid>
            <category><![CDATA[ultrasound]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[signal-processing]]></category>
            <category><![CDATA[sensors]]></category>
            <dc:creator><![CDATA[Jan Steckel]]></dc:creator>
            <pubDate>Sun, 25 Oct 2026 14:44:03 GMT</pubDate>
            <atom:updated>2026-10-25T14:44:03.392Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Beamforming in pictures: how the size of a microphone array, measured in wavelengths, sets how sharply a sonar can tell directions apart, and why our credit-card-sized array sees the world through an oval</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4lzD3q22hwnW8gmr_JcxDA.png" /><figcaption><em>The beam pattern of our eRTIS sensor at 50 kHz, computed from its real microphone positions and seen as the whole frontal view. The bright spot in the middle is the direction the sensor listens to. Note that it is taller than it is wide.</em></figcaption></figure><p>Close your eyes and let someone snap their fingers somewhere around your head. Telling left from right is easy. Telling up from down is harder (try it at your next party, your friends will love it, or look weirdly at you. Its a coin flip to be honest). Your ears sit on either side of your head, so left and right come with differences in arrival time and loudness between the two ears, whereas up and down have to be read from the subtle filtering of your outer ears (more on that in <a href="https://medium.com/acoustic-explorations/how-a-bat-could-learn-its-own-ears-06d867593457">How a bat could learn its own ears</a>).</p><p>Our sonar has the same problem, for a related reason. In the first post of this blog, <a href="https://medium.com/acoustic-explorations/making-three-dimensional-maps-of-the-world-using-sound-not-light-13f778b754ab">Making Three Dimensional Maps of the World using Sound, not Light</a>, I wrote that each detection is a small cone of uncertainty, and that “that cone is wider in elevation than in azimuth for our sensor, because of its credit-card-like aperture (a topic for a later post).” This is that later post.</p><p>The short answer: the microphone array of our eRTIS sensor is 66.5 mm wide and 37.6 mm high, and its beam is about 6 degrees wide left-right and 10 degrees up-down. The ratio of the two widths is almost exactly the ratio of width to height of the array.</p><h3>Listening in one direction</h3><p>The eRTIS, built in my group, has 32 small microphones on a board about the size of a credit card. In the first post, I described beamforming as a spatial filter: you choose a direction in which you want to listen, and the beamformer tries its best to filter out the sound from all other directions.</p><p>How does it do that? An echo from straight ahead reaches all microphones at the same moment. An echo from the side reaches the nearest microphone first and the farthest one a little later. To listen to the side, the beamformer delays every microphone signal by exactly the amount that lines those arrivals up again, and adds them all together. Sound from the chosen direction adds up in step and comes out loud. Sound from other directions arrives out of step, partly cancels, and comes out weaker. This is called delay-and-sum beamforming.</p><p>The catch is in the word “partly”. For an echo only slightly off the chosen direction, the extra delays across the array are much smaller than one period of the sound wave. The signals are still almost in step, and the echo gets through nearly unharmed. How sharp the beam is depends on how far off a direction must be before the arrival times across the array differ by a good fraction of a wavelength.</p><h3>Size, measured in wavelengths</h3><p>That gives the rule of thumb this post rests on: the width of the main beam, in radians, is roughly the wavelength <em>λ</em> divided by the size of the array <em>D</em>. A larger array, or a shorter wavelength, gives a narrower beam. Astronomers know this as the diffraction limit of a telescope, and it is why radio telescopes are so big. Physics does not negotiate on this one. I have tried, and failed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-han0IQ3d-3RuhKyS6QOjg.png" /><figcaption><em>The 32 microphones of the eRTIS (dots), drawn to scale on a credit card. The array spans 66.5 mm horizontally and 37.6 mm vertically. On the right, the wavelength of sound in air at the lowest and highest frequency of our call.</em></figcaption></figure><p>Our sensor emits a sweep from 25 to 70 kHz. In air, that is a wavelength of 13.7 mm at the low end and 4.9 mm at the high end. Measured in wavelengths, the array is about 5 to 14 wavelengths wide, but only about 3 to 8 wavelengths high. In the vertical direction, the array is simply smaller, so the rule of thumb predicts a wider beam there.</p><h3>An oval beam</h3><p>The figure below shows the beam pattern of the real eRTIS layout, listening straight ahead, computed for delay-and-sum beamforming over the frequency band of the call. The bright spot is the main lobe: the directions from which an echo gets through almost unharmed. It is an upright oval.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gqjDjbkUyuaKBCI7XiLElw.png" /><figcaption><em>Left: how strongly the array, listening straight ahead, picks up an echo from every direction in front of it (computed, 25 to 70 kHz). Right: two cuts through the bright spot. At half power, the beam is about 6 degrees wide left-right (teal) and 10 degrees up-down (magenta).</em></figcaption></figure><p>Measured at half power, the main lobe is 5.7 degrees wide in azimuth and 10.0 degrees in elevation, a ratio of 1.76. The array is 1.77 times wider than it is high. For the expert reader: the far-field beam pattern at one frequency is, up to a change of variables, the Fourier transform of the microphone layout, so a layout that is wide in one direction gives a pattern that is narrow in that direction. Optics has the same effect, where a narrow slit spreads light widely.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7ZAF1wNdzNjpZ-qBSA8UjQ.png" /><figcaption><em>Left: the beam narrows as the frequency rises, in both directions. Right: the same layout scaled up or down, at 47.5 kHz. The width in each direction depends on the size of the array in that direction, and the two eRTIS directions sit on the same curve.</em></figcaption></figure><p>The right panel makes the point most directly. Scale the layout down until it is 37.6 mm wide, and its left-right beam becomes about as wide as its up-down beam is now. Nothing about elevation is special. The array is just less tall than it is wide.</p><p>The beam also gets wider away from straight ahead. Seen from the side, the array looks foreshortened, like a coin seen at an angle, as I described in <a href="https://medium.com/acoustic-explorations/deblurring-sonar-images-by-driving-around-bee8f8c8364e">Deblurring sonar images by driving around</a>.</p><h3>Sidelobes and grating lobes</h3><p>You may have noticed that the microphones are not on a neat grid. That is on purpose. If microphones are spaced regularly, more than half a wavelength apart, the array cannot tell some directions apart, and the image fills with copies of each echo, called grating lobes. On the eRTIS, the closest two microphones are 6.8 mm apart, while half a wavelength at 70 kHz is only 2.5 mm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G2RRQZNIS50K7KOvU2qSyw.png" /><figcaption><em>A hypothetical regular grid of 32 microphones with the same aperture (left) produces full-strength copies of the main lobe at about 37 degrees to either side at 60 kHz (grey). The irregular eRTIS layout (magenta) spreads that energy out into lower sidelobes.</em></figcaption></figure><p>Scattering the microphones spreads the energy of those copies over many directions, as lower sidelobes. Sidelobes are the price of having only 32 microphones, and together with the width of the main lobe, they make sonar images blurry, the topic of the deblurring post. The idea of using sparse, irregular arrays for in-air sonar goes back to the broadband 3D sonar of my PhD years [CHECK: OK to place the sparse array in Jan’s PhD work?], and the tidy engineer in me still finds the eRTIS layout slightly upsetting to look at.</p><h3>Conclusion</h3><p>The eRTIS sees better left-right than up-down because its array is 1.77 times wider than it is high, and the width of a beam scales with the wavelength divided by the size of the array in that direction. Every sonar point in our maps carries that oval with it, and the algorithms that build maps from those points have to take it into account.</p><h3>Further reading</h3><p>The figures in this post are computed from the real eRTIS microphone layout (receive array only, delay-and-sum, far field).</p><ul><li><strong>eRTIS: A Fully Embedded Real Time 3D Imaging Sonar Sensor for Robotic Applications.</strong> R. Kerstens, D. Laurijssen, J. Steckel. ICRA 2019. <a href="https://doi.org/10.1109/ICRA.2019.8794419">https://doi.org/10.1109/ICRA.2019.8794419</a></li><li><strong>Ruggedized Embedded Ultrasound Sensing in Harsh Conditions: eRTIS in the Wild.</strong> D. Laurijssen, W. Jansen, A. Aerts, W. Daems, J. Steckel. IEEE Sensors Journal, 2026. <a href="https://doi.org/10.1109/JSEN.2025.3631307">https://doi.org/10.1109/JSEN.2025.3631307</a></li><li><strong>An Optimized Spatial Sampling Strategy for Wide-View Planar Array 3D Sonar Sensors.</strong> J. Reijniers, R. Kerstens, J. Steckel. IEEE TUFFC, 2020. <a href="https://doi.org/10.1109/TUFFC.2020.2964991">https://doi.org/10.1109/TUFFC.2020.2964991</a></li><li><strong>HiRIS: An Airborne Sonar Sensor With a 1024 Channel Microphone Array for In-Air Acoustic Imaging.</strong> D. Laurijssen, W. Daems, J. Steckel. IEEE Access, 2024. <a href="https://doi.org/10.1109/ACCESS.2024.3385232">https://doi.org/10.1109/ACCESS.2024.3385232</a></li><li><strong>RTISSLAM-3D: Full 6-DoF SLAM With an In-Air 3D Imaging Sonar.</strong> J. Steckel, V. Katter, W. Jansen. Preprint, arXiv:2610.22552. <a href="https://arxiv.org/abs/2610.22552">https://arxiv.org/abs/2610.22552</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9767c19b727d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/acoustic-explorations/why-our-ertis-sonar-sees-better-left-right-than-up-down-9767c19b727d">Why our eRTIS sonar sees better left-right than up-down</a> was originally published in <a href="https://medium.com/acoustic-explorations">Acoustic Explorations</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Where have I been, and what is around me?]]></title>
            <link>https://medium.com/acoustic-explorations/where-have-i-been-and-what-is-around-me-3ab03fa2cb51?source=rss-b18596ec6b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/3ab03fa2cb51</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[sonar]]></category>
            <category><![CDATA[robotics]]></category>
            <dc:creator><![CDATA[Jan Steckel]]></dc:creator>
            <pubDate>Sat, 24 Oct 2026 18:04:47 GMT</pubDate>
            <atom:updated>2026-10-24T18:04:47.235Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How a sonar robot could organise its memory in two layers, a pose graph of the places it has been and small scene graphs of what it heard there, and why the physics of echoes asks for that split</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6hMU6yUX_ZCNi0j9YDVZJA.png" /><figcaption><em>BatSLAM 2.0 on a simulated office floor. The estimated path (teal) is drawn with time going upwards, so the loop closures (amber) that tie a place to an earlier visit of the same place connect the laps.</em></figcaption></figure><p>Walk into your kitchen in the dark, and you still know two things: where you are in the house, and roughly where the table and chairs are around you. The first is a global map of places. The second is a local picture of things, which you only need for the room you are in.</p><p>A robot that perceives the world through echoes needs both. In this post, I look at how to organise them: a global pose graph that remembers where the robot has been, and small local scene graphs, hung on its nodes, that describe what is around it. The first layer is built and tested, also on real data. The second exists in simulation, with mixed results. Joining the two is still a plan, and I will be clear about which part is which.</p><p>This post also keeps a promise from the <a href="https://medium.com/acoustic-explorations/making-three-dimensional-maps-of-the-world-using-sound-not-light-13f778b754ab">first post on this blog</a>, in which BatSLAM 2.0 was left as “the topic of a future post”.</p><h3>A string of places</h3><p>A pose graph is the robot’s diary of where it has been. Every node is a pose, linked to the previous one by odometry, which drifts. Recognising a place visited before adds a link between the two visits (a loop closure), and an optimiser pulls the whole chain into agreement.</p><p>For sonar, that recognition step is the tricky one. Two parallel corridors with plain walls sound almost the same for all positions (called perceptual aliassing, the bane of all SLAM algorithm builders), and a single false loop closure can fold two distinct places onto each other, which is called map collapse.</p><p>BatSLAM 2.0, a new version of the original BatSLAM I built during my PhD over a decade ago, therefore never trusts a single matched view. It turns every binaural echo into a cochleogram (the time-frequency image a bat’s inner ear would produce), compares it with stored templates, and accepts a loop closure only after a whole sequence of consistent matches. Even then, the closure enters the graph as a group of links that can be withdrawn again, if new evidence surfaces that contradicts the old one.</p><p>In 12 simulated runs with noisy odometry this gave maps without a single incorrect loop closure. On a real recording, with my lab’s eRTIS imaging sonar turned into two virtual ears by beamforming, it reduced the trajectory error from 3.6 m for the odometry to 0.7 m.</p><h3>Hang the memory on the nodes</h3><p>The key design choice is where to store what the robot heard. In BatSLAM 2.0, every template is anchored to the pose-graph node at which it was recorded. When a loop closure moves the node, the template moves with it.</p><p>This follows from the physics. At ultrasonic wavelengths, many surfaces act like mirrors, so what you hear depends strongly on where you stand. In simulation, a stored BatSLAM 2.0 template is only recognised within about 15 cm sideways and 25 degrees in heading of where it was recorded. Sonar memory is local by nature, so it should live with the pose it was taken from.</p><p>RTISSLAM, the system behind the 3D forest map of the first post, applies the same idea to an occupancy grid. Each keyframe stores its echoes in the robot’s own frame, attached to its node, and the grid is redrawn from the current node poses whenever the graph changes. If you are used to lidar SLAM, note what is missing: the grid never feeds back into the pose estimate. A single sonar scan is too sparse and too viewpoint dependent to match against a map. In my simulations, lidar-style scan matching on sonar often did worse than counting wheel rotations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*44gEiwfA2kSpLcyWGY1gQQ.png" /><figcaption><em>RTISSLAM’s grid of a simulated office just before (left) and after (right) a loop closure. Before, the walls of the revisited corridor appear twice. After, the evidence stored at every node has moved with its node, and the error of the current pose drops from 1.13 m to 0.13 m. Dashed: ground truth, for reference only.</em></figcaption></figure><h3>What is around me?</h3><p>A grid map says where echoes came from, but not what reflected them, and for sonar that matters. A wall is a mirror: it only echoes from the spot where your line of sight hits it at a right angle, and that spot slides along the wall as you move. A corner or a door handle stays where it is. In a single snapshot, a wall, a corner and an edge can produce the same image, a classic result in sonar research. They differ in how their echoes move when the robot moves.</p><p>So the second layer is a local scene graph, built from a short window of pulses and the robot’s own motion within it. And here, we venture in the real of speculation and hand-waving. None of what follows has been proven to work well. My team and I are trying our best to make this all work, but it is all ongoing research. Cutting edge, so to speak.</p><p>Anyway, back to the local scene graph. Its nodes are walls, edges where walls meet, and points, and its links carry the relations: a door handle hangs on its wall, and a corner is where two walls meet. The graph should be learned without labels, in the same spirit as the bat that <a href="https://medium.com/acoustic-explorations/how-a-bat-could-learn-its-own-ears-06d867593457">learned its own ears</a> in post 03: render the graph from the poses the robot will be at a little later, and compare with the echoes it actually hears there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lAIO9dIerKEk1y6EyeliBg.png" /><figcaption><em>Illustration of the architecture. The global pose graph (bottom) holds where the robot has been, linked by odometry and verified loop closures. Local scene graphs (top) of walls, edges and points hang on its nodes. Merging what two nodes saw of the same place is planned, not built.</em></figcaption></figure><p>I tested this layer on simulated office floors, with a simulated eRTIS, and the results are mixed. The model finds about nine in ten of the walls it can hear, but only about three in ten of the walls it proposes are real, and edges and points are harder still. A simple baseline (the last pulse’s echoes, moved along with the robot) even predicts the next few pulses better. A false wall only gives itself away from clearly different viewpoints, which a short window does not have. So, this is all still very wishy-washy.</p><h3>Do bats do this?</h3><p>BatSLAM’s ancestor, RatSLAM, was modelled on the rodent hippocampus, and place cells (neurons that fire when the animal is at a particular location) have been found in bats too. A bat flying through vegetation also has to keep track of the branches right next to it. It is tempting to see the two layers in that, but I do not claim that this architecture is what is implemented in a bat’s brain. The split follows from the physics of echoes and the engineering of SLAM. The bats are the motivation, and the living proof that echoes are enough to find your way (and your way back).</p><h3>Where this stands</h3><p>The pose graph with node-anchored memory is implemented and evaluated, in simulation and on real recordings. The local scene graphs were only tested in simulated offices, with simplified mirror-like physics and no multiple reflections, and merging them across visits is a proposal. I think the split is right, because sonar memory is local and loop closures move everything. Whether a robot can learn good scene graphs from its own echoes is the open question.</p><h3>Further reading</h3><ul><li><strong>BatSLAM 2.0: Sequence-Verified Sonar Place Recognition in a Robust Pose Graph.</strong> J. Steckel. Submitted to IEEE Transactions on Robotics (under review).</li><li>Video of BatSLAM 2.0: <a href="https://youtu.be/VzuYBWAR8kw">https://youtu.be/VzuYBWAR8kw</a></li><li>Code of BatSLAM 2.0: <a href="https://github.com/Cosys-Lab/BatSLAM2">https://github.com/Cosys-Lab/BatSLAM2</a></li><li><strong>RTISSLAM: In-air Imaging Sonar SLAM using a Robust Pose Graph and Occupancy Grid Mapping.</strong> J. Steckel. Preprint, arXiv:2610.18479. <a href="https://arxiv.org/abs/2610.18479">https://arxiv.org/abs/2610.18479</a></li><li><strong>RTISSLAM-3D: Full 6-DoF SLAM With an In-Air 3D Imaging Sonar.</strong> J. Steckel, V. Katter, W. Jansen. Preprint, arXiv:2610.22552. <a href="https://arxiv.org/abs/2610.22552">https://arxiv.org/abs/2610.22552</a></li><li>Previous posts: <a href="https://medium.com/acoustic-explorations/making-three-dimensional-maps-of-the-world-using-sound-not-light-13f778b754ab">Making Three Dimensional Maps of the World using Sound, not Light</a> and <a href="https://medium.com/acoustic-explorations/how-a-bat-could-learn-its-own-ears-06d867593457">How a bat could learn its own ears</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3ab03fa2cb51" width="1" height="1" alt=""><hr><p><a href="https://medium.com/acoustic-explorations/where-have-i-been-and-what-is-around-me-3ab03fa2cb51">Where have I been, and what is around me?</a> was originally published in <a href="https://medium.com/acoustic-explorations">Acoustic Explorations</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When the robot is louder than the forest]]></title>
            <link>https://medium.com/acoustic-explorations/when-the-robot-is-louder-than-the-forest-1355a8edae91?source=rss-b18596ec6b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/1355a8edae91</guid>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[ultrasound]]></category>
            <category><![CDATA[signal-processing]]></category>
            <dc:creator><![CDATA[Jan Steckel]]></dc:creator>
            <pubDate>Sat, 24 Oct 2026 17:50:36 GMT</pubDate>
            <atom:updated>2026-10-24T17:50:36.799Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How we used some clever math to removed the robot’s own ultrasonic noise from the microphones of the eRTIS sonar, before making a single image, so that RTISSLAM could work on real data</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0EXeR0sJylHsSVMzuAvXIA.png" /><figcaption><em>Illustration: a simulated eRTIS image of a made-up scene, raw on the left half and cleaned on the right half. The diagonal streak on the left is interference from a fixed direction, which is what the robot’s electronics did to our real forest data.</em></figcaption></figure><p>If you have ever made a phone call from a car, you know that the person on the other end hears your engine before they hear you. Microphones do not care where a sound comes from. They record everything, and the sound you make yourself is often the loudest of all.</p><p>In the first post on this blog, I showed a 3D map of a forest in Norway, built from the echoes of our eRTIS sonar on a wheeled robot. What I did not show is what the raw data looked like before we could use it. The robot itself was making a lot of ultrasonic noise. This post is about the question that had to be answered first: how do you remove the robot’s own sound from a sonar, without removing the echoes you are after?</p><h3>The simulator is too quiet</h3><p>RTISSLAM, our sonar-based mapping system, was first developed and tuned in simulation. In the simulator, the robot makes no sound of its own, and the microphones only hear the echoes of the emitted call signal. A real recording is different, and on the forest recording two kinds of unwanted sound stood out.</p><p>The first is everything that is fixed with respect to the sensor. The call travels directly from the emitter to the microphones right next to it (the direct path), the sensor rings for a while after that, and parts of the robot reflect the call as well. These signals are by far the strongest in each recording, and they are the same every time.</p><p>The second was a surprise: the robot’s drive electronics emit broadband ultrasound, strongest around 55 kHz and also around 20 kHz, right in the band of our call. In the sonar images, it shows up as radial streaks: energy at all ranges in a few fixed directions, in every image, whether there is an object there or not. For a mapping system that turns bright spots into obstacles, that is a disaster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H-PS6C87L8zRxmoXZzZ-XA.png" /><figcaption><em>The clean-up happens on the 32 microphone signals, before any image is formed. The three dark steps were added for applying our RTISSLAM algorithm on real data.</em></figcaption></figure><h3>Subtracting stationary clutter</h3><p><em>Warning: some maths ahead. If you do not like maths, shame on you. You should like maths. Maths is awesome.</em></p><p>The first step uses a simple observation. As the robot drives, the echoes of the trees and the ground move from one call to the next. The direct path, the ringing and the echoes of the robot itself do not move in unison with robot motion. So if you average many recordings taken along the drive, the moving echoes average out, and what remains is the static background:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uFgvHNzaaRcCzoRkmPj_IA.png" /></figure><p>Here s⁽ᶜ⁾ is the signal of microphone i in recording c, k is the time sample, and we average over C = 300 recordings spread over the whole drive. We then subtract B from every recording. On the forest data, this background holds 93% of the signal power in the first 0.75 m of range, and less than 2% beyond that. So it cleans up the near field and leaves the rest alone. Ground echoes stay in, because they are real.</p><p>This only works if every recording starts at the same moment relative to the call. On the real sensor, a small fraction of the recordings started a little late, so we first shift each one so that its direct path lands on the same sample. A plain subtraction then does the job, and the static clutter is removed (mostly).</p><h3>A fingerprint in 32 microphones</h3><p>The drive interference is harder. It is, in effect, random noise, so it is different in every recording and does not average out. But it has one property we can use: it always comes from the same places on the robot, at fixed positions relative to the array. So at each frequency, it reaches the 32 microphones with the same pattern of delays and strengths. Think of that pattern as a spatial fingerprint. Find the fingerprint, and you can remove everything that matches it.</p><p>To find it, we look at how strongly, and with what phase, each pair of microphones hears the same thing. This is the cross-spectral matrix, one per frequency:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*phT97tMXRMvu5ie9i2sjBA.png" /></figure><p>where S is the vector of the 32 microphone spectra and the E, the expectation, is an average over many short stretches of the recordings. We only use the part of each recording beyond about 1.5 m of range. Echoes are sparse there, while the interference is always on, so the average is dominated by the robot noise.</p><p>Next comes the step that does the real work. An eigenvalue decomposition splits this matrix into independent spatial patterns, the eigenvectors uₙ, each with its energy λₙ:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nWLIBuTM91G0cCEEwWROJg.png" /></figure><p>A source at a fixed position produces one dominant pattern. Echoes from changing directions and sensor noise spread their energy over many weak ones. So we call a pattern to be interference when its energy is more than four times the median, and we take at most four per frequency.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KhvlWkvDPNpnUWrZhR-pDw.png" /><figcaption><em>In a simulation with the eRTIS layout, the interference stands far above the threshold line at 54.5 kHz, and everything else stays below it. On the real forest data, one pattern held 90% of the energy at that frequency.</em></figcaption></figure><p>Collect these patterns in a matrix U, and remove them from every recording with a projection:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nuG9jcpzhPsF4sytlTTITQ.png" /></figure><p>In words: whatever part of the microphone signals looks like the fingerprint is cut away, and the rest passes untouched. The same idea underlies classic subspace methods in array processing like MUSIC and ESPRIT. On the forest recording, between zero and four patterns were removed per frequency, about two on average.</p><h3>Why not clean the image afterwards?</h3><p>A natural question is why we do this on the raw microphone signals, and not on the finished image. We tried that first. The images RTISSLAM uses come from higher-order delay-multiply-and-sum beamforming with a coherence factor, which are non-linear operations. They turn the smooth interference into a spiky pattern along range, so subtracting a noise floor per direction fails.</p><p>The projection, in contrast, is linear and happens before any beamformer. Every beamformer downstream gets the same clean input, and the projector is fitted once per recording (in under a minute of computation) and applied to every call.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LucY-cK-NvK2VDD-DDwx6A.png" /><figcaption><em>From left to right: the raw signals, after subtracting the static background, and after projecting out the interference. Illustration with simulated data, the real eRTIS layout and call, and a made-up scene.</em></figcaption></figure><h3>Did it work?</h3><p>Yes, it did! On the forest recording, the cleaned images show trees as compact echoes, and the radial streaks no longer show up. Every result in the first post, including the 3D forest map, was computed on images cleaned this way.</p><h3>Conclusion</h3><p>Moving from simulation to real data is mostly about the things the simulator did not know about. Here, it was the robot’s own electronics, singing along in exactly the band of our call. The fix did not need machine learning or a new sensor. It needed the observation that a noise source fixed to the robot leaves a fixed fingerprint on the array, and a few lines of linear algebra to remove it.</p><h3>Further reading</h3><ul><li><strong>RTISSLAM-3D: Full 6-DoF SLAM With an In-Air 3D Imaging Sonar.</strong> J. Steckel, V. Katter, W. Jansen. Preprint, arXiv:2610.22552. The preprocessing is summarised in the method section. <a href="https://arxiv.org/abs/2610.22552">https://arxiv.org/abs/2610.22552</a></li><li><strong>RTISSLAM: In-air Imaging Sonar SLAM using a Robust Pose Graph and Occupancy Grid Mapping.</strong> J. Steckel. Preprint, arXiv:2610.18479. <a href="https://arxiv.org/abs/2610.18479">https://arxiv.org/abs/2610.18479</a></li><li>The first post in this series: <a href="https://medium.com/acoustic-explorations/making-three-dimensional-maps-of-the-world-using-sound-not-light-13f778b754ab">Making Three Dimensional Maps of the World using Sound, not Light</a></li><li>Code of the 2D RTISSLAM system: <a href="https://github.com/Cosys-Lab/RTISSLAM">https://github.com/Cosys-Lab/RTISSLAM</a>. The real-data front end with this clean-up is not public yet.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1355a8edae91" width="1" height="1" alt=""><hr><p><a href="https://medium.com/acoustic-explorations/when-the-robot-is-louder-than-the-forest-1355a8edae91">When the robot is louder than the forest</a> was originally published in <a href="https://medium.com/acoustic-explorations">Acoustic Explorations</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>