<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by James Milward on Medium]]></title>
        <description><![CDATA[Stories by James Milward on Medium]]></description>
        <link>https://medium.com/@james.milward?source=rss-529d5fc5f9df------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*kmoKMlMHbw8IrJht4nwafg.jpeg</url>
            <title>Stories by James Milward on Medium</title>
            <link>https://medium.com/@james.milward?source=rss-529d5fc5f9df------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 06:30:16 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@james.milward/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Claude Code as My Co-Engineer: From a Tub of Water to Ultrasonic Flood Prediction in Cumbria]]></title>
            <link>https://medium.com/@james.milward/claude-code-as-my-co-engineer-from-a-tub-of-water-to-ultrasonic-flood-prediction-in-cumbria-9d0c4e652a37?source=rss-529d5fc5f9df------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d0c4e652a37</guid>
            <category><![CDATA[farming]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[iot]]></category>
            <category><![CDATA[climate-change]]></category>
            <category><![CDATA[coding]]></category>
            <dc:creator><![CDATA[James Milward]]></dc:creator>
            <pubDate>Thu, 19 Feb 2026 14:31:08 GMT</pubDate>
            <atom:updated>2026-02-19T14:31:08.052Z</atom:updated>
            <content:encoded><![CDATA[<h3>Claude Code as My Co-Engineer: From a Tub of Water on My Desk to Ultrasonic Flood Prediction in Cumbria</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*dnwqFZqpnObGYGIfli522g.png" /><figcaption>My ultrasonic water velocity sensor — using time of flight principles, enhanced with a computational fluid dynamics simulator (built by Claude Code) and embedded code significantly improved by Claude Code</figcaption></figure><p>If you read my <a href="https://medium.com/@james.milward/deterring-foxes-and-badgers-with-tensorflow-lite-python-raspberry-pi-ring-cameras-ultrasonic-75b3160faa3c">last blog</a>, you’ll know I have a habit of getting myself into strange and unique projects. That last one involved using Python with Machine Learning to train a PyTorch/TensorFlow model so I could detect badgers in my garden. It was complete overkill and went a bit viral - <a href="https://www.bbc.co.uk/news/technology-67053171">BBC News</a> picked it up, I did some radio interviews, and suddenly I was known because of badgers and AI. All very strange.</p><p>Thankfully I was able to develop that concept into deployed AI wildlife cameras for farms in the Lake District and sports clubs in the South. These are currently being trialled to monitor potential bovine TB sources and help deter wildlife safely.</p><p>It was through those farm deployments in the Lake District that I got to know the Lynster Farmers Group. So when Hugh Ward, Project Director of the group, called me at the end of August 2025 with a completely different problem they were facing, I was all ears.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oUoO-3XuVuX4zmxscCb9Uw.jpeg" /><figcaption>Flooding in the Winster Valley — Grange-Over-Sands, Winter 2023</figcaption></figure><h3>The Challenge</h3><p>The Lynster Farmers Group are a collective of 49 farms spread across the Lyth Valley in Cumbria. An extraordinarily beautiful part of the world and one, it turns out, with a serious water problem.</p><p>The community depends on a network of rivers and drainage channels, some of which haven’t been properly maintained in decades. Hugh and the group had just completed a highly ambitious rehabilitation of the River Winster, restoring flow capacity from 60,000 to 660,000 cubic metres per day (11x increase) at the cost of a few hundred thousand pounds.</p><p>But here was the thing. They had limited funding, small sets of manually collected data, one set of flow rate measurements and manual water level readings — courtesy of John Geldard, Chair of the Group, taking a tape measure every day out to one section of the river by his farm. As Hugh put it in an email:</p><blockquote><em>”Without data on flow rate, discharge and levels in the rivers and the drains it is extremely difficult to make informed decisions and to learn.”</em></blockquote><p>They had flood maps showing the problem areas and enlisted help from multiple institutions, including Professor Jon French at UCL in place and ready to build hydrological models of the entire catchment. What they didn’t have was any way to actually measure what the water was doing.</p><p>Having heard this, I decided to do what I always do in situations like this - build something! I thought, ‘how hard can it be to build something that can understand the velocity of rivers?’. Very, very hard it turns out …</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*yJmTWfklRx_G490Q.jpg" /><figcaption>An industrial sensor that measures water surface velocity and level from above — prohibitive at scale at ~£15,000</figcaption></figure><h3>The £10,000 — £30,000 off the shelf problem</h3><p>My first thought was the obvious one … surely you can just buy a cheap sensor - something from a boat for instance, and equip it with some electronics and a SIM card? I tried that, the device had algae build up in no time and became useless.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xg2WM7vVjLCLS3KS1vCy8g.png" /><figcaption>A paddle wheel sensor, used on the hull of boats, suffers from algae build up quickly (left) if not cleaned regularly. Not suitable for permanent river deployments.</figcaption></figure><p>Industrial sensors solve this problem by having no moving parts. They also are priced at at around £10,000 - £30,000 per site when you add a data logger, cellular telemetry, solar power, weatherproof housing, and installation per site. The World Bank benchmarks a typical hydrometric station in a developed country at £20,000 - £35,000 (<a href="https://www.worldbank.org/en/news/feature/2022/10/11/charting-a-course-for-sustainable-hydrological-and-meteorological-networks">link</a>). Over a ten-year lifecycle, a single monitoring point can cost north of £100,000.</p><p>For a farming group that needs sensors scattered across an entire river catchment, that’s financially impossible. Hugh didn’t need one perfect gauging station - he needed twenty affordable ones. Something a community could actually deploy and maintain.</p><p>Time to build.</p><h3>I hired a team of AI Engineers, a.k.a. Claude Code</h3><p>This is the point where a normal person would probably have looked into existing sensor companies or applied for a research grant. Instead, I opened Claude Code and considered a different approach.</p><p>Around September 2025, I’d been using Claude Code for quite a while and experimented with <a href="https://code.claude.com/docs/en/sub-agents">sub-agents</a>. I considered that I could potentially learn this new feature in depth by building an AI team to help me with this problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t6ktSegbYKiwuOzmfbbjPQ.png" /><figcaption>My Fluid Acoustics Physicist meant that I didn’t need to hire a PhD level expert to help me design my sensor</figcaption></figure><p>So I created four specialist agents for the task at hand;</p><ul><li><strong>Fluid Acoustics Physicist agent </strong>— deep expertise in ultrasonic sound in liquid, speed of sound, reflections etc</li><li><strong>Computational Fluid Dynamics agent </strong>— a specialist in a branch of engineering that uses numerical analysis, data structures, and computer algorithms to simulate, analyse, and visualise the behaviour of liquids and gases</li><li><strong>Electronics Engineer agent</strong> — working with me to coordinate the connection of components and help with issues like firmware control of integrated circuits and electromagnetic interference (which could throw readings off)</li><li><strong>Documentation Expert</strong> — able to go through all of the PDFs and documentation supplied with the components I use in this project, helping me write code correctly</li></ul><p>Using my new team, we discussed various approaches and settled on a sensor designed around ultrasonic transducers (an input sensor that can both send and receive sound).</p><p>My physicist taught me everything I needed to know and the physics solution is actually quite elegant. You mount two ultrasonic transducers at an angle to the flow, typically 45 degrees. Fire a pulse from A to B (downstream) and measure how long it takes. Then fire one back from B to A (upstream). If the water is flowing, the downstream pulse arrives slightly faster than the upstream one. The time difference, Δt, is directly proportional to the flow velocity.</p><p>In principle, you’re measuring the difference between a sound wave swimming with the current and one swimming against it. Same physics that makes it harder to shout into the wind.</p><p>Simple, I thought. Not quite.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lh8SKe2iE0OQ6c7TI0tu4Q.png" /><figcaption>I initially selected an ESP32 microcontroller to run my new tool and soldered a collection of components onto Perfboard. I wrote a program in C and C++ to control these components and send test signals. Claude significantly improved it. Final analysis demonstrated that I needed something with a bit more computational horse power so I eventually moved to a Teensy 4.1</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sgpigRumeXgMhtFaDFb9og.png" /><figcaption>My rough 3D printed V1 prototype — two transducers, attached to a pole would be angled in the water. Initial tests in a cheap tub of water on my desk proved that the devices could send and receive ultrasonic sound.</figcaption></figure><h3>Confident AI, bad readings and more learning</h3><p>Getting the theory onto a circuit board was one thing but getting ultrasonic pulses to actually travel reliably through water was another.</p><p>Initially, I was emboldened by my fancy AI team who assured me that this device would work and confidently told me I could move to a river deployment soon. Turns out they were quite wrong as I spent a few months of weekends and evenings with a plastic tub of water sitting next to my desk trying to make sense of the readings I was getting.</p><p>This is where Claude, however, earned its keep. I’m a software architect by trade, I can design distributed systems, write Python, wrangle cloud infrastructure. But I am not an experienced electronics engineer, more of a hobbyist. The gap between understanding <em>*conceptually*</em> that we needed nanosecond timing and actually writing firmware that processes nanosecond timing is a gap that would have taken me years to cross alone.</p><p>Claude bridged this - not by writing code and throwing it at me but by working through the electronics with me, explaining how the integrated circuits work and walking me through oscilloscope waveforms when I bought one specifically to debug this thing.</p><p>Yes, Claude trained me to use an oscilloscope. We’d moved well past “chatbot” territory.</p><p>Next to test this in fast moving water.</p><h3>I built a 3.4m flume down the side of my house</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o2RHCGTHehxGxDqiB7p_lw.png" /><figcaption>My wife loved the new water feature in our garden — a flume to test velocity readings.</figcaption></figure><p>You know that feeling when you’re explaining a project to your partner and you can see the exact moment they start reconsidering their life choices? That moment arrived when I started carrying 3.4-metre lengths of wood down the garden.</p><p>I needed a way to start testing the device in moving water and as I had no rivers near me, the flume made sense. After watching some YouTube videos (thanks <a href="https://www.youtube.com/watch?v=sppaBqpIT-w">FluidsExplained</a>) I designed a long trough with plastic sheeting inside, a submersible pool pump at one end, and the transducer frame mounted in the middle. Get water flowing at a known speed, see what the sensor reads, calibrate accordingly.</p><p>After initially soaking myself due to a badly mounted pump, I ran a few more tests and stumbled on two issues -</p><ol><li><strong>The sensor readings didn’t change — </strong>Noise seemed to dominate</li><li><strong>I had no ground truth</strong> — How fast was the water moving through my device?</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*zcj_3g4grWoJzTWoWjn7Zw.gif" /><figcaption>I needed to understand how quickly I was moving the device so I could calibrate readings. Simple infrared triggers gave accurate timing of the movement of my device between two known points.</figcaption></figure><p>The solution was to place the device on a moving trolley, with two infrared gates that would be triggered by the movement, giving me accurate timing between two points (2 metres). However, something about the design of the housing was disrupting the ultrasonic path. But I couldn’t see the issue as the housing looked fine, the transducers were aligned, the angles were correct.</p><p>I needed to see what the water was actually doing around the sensor.</p><h3>Turbulence</h3><p>I went back to Claude and after trying recommendations of SimScale (a computational fluid dynamics platform used by engineers for exactly this kind of analysis) and other tools, I had more problems:</p><p>1. The learning curve would eat weeks. This is professional CFD software with professional CFD complexity. I didn’t have time to learn something like this and I wanted something Claude and I could interact with, easily.</p><p>2. The simulation costs: roughly <strong>$5,000 to $7,500</strong> for the kind of resolution I needed and frequency of runs.</p><p>To Claude! I prompted Claude with a detailed prompt, defining a tool I needed to solve the problem and the challenges I’d had with other simulation tools. I asked it to create a plan and implement a new flow dynamics tool in Python which should utilise either the graphics card on my MacBook or gaming PC to speed up the simulation. I needed a way of seeing this in both 2D and 3D, so after some time, Claude finished and I fired up this new tool — <strong>I could see the problem instantly.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cQ3P_Ge2S0cLWAs0c_muvQ.png" /><figcaption>Claude Code created a rich simulation tool, utilising the GPU capabilities of my machine to give me fast and accurate simulations. It demonstrated that my design created turbulent vortices around the face of my transducers, effectively stopping the device from working.</figcaption></figure><p>The V1 housing design created turbulent vortices directly in the ultrasonic measurement path. The struts holding the transducers in place were disrupting the flow between them, scattering the ultrasonic pulses and making consistent time-of-flight measurement impossible. The sensor wasn’t broken, the housing was breaking the physics.</p><p>So I went back to my CAD tool and consulted Claude on the best approach to design something that is more hydrodynamic. It identified some issues and after some trial and error, I had a much better design with minimal turbulence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DfsHZMRvvkcOrYrq7Zeqlg.png" /><figcaption>A much improved hydrodynamic housing, reducing turbulence and allowing for better readings. The final version has the wires hidden and hard edges are reduced so stop algae build up.</figcaption></figure><p>I configured Claude to use this new tool via an agent and run a few hundred simulations of the old vs new design. The analysis was clear</p><ul><li>Much smaller obstacles — the old struts are thinner/less hydrodynamic</li><li>Less turbulence (21% vs 45%) — the old design disturbs the acoustic path more</li><li>More flow acceleration (129% vs 92%) — enabling higher resolution readings as long as I could calibrate</li></ul><p>I then ran this myself using particle visualisation on the surface of the model and it was clear that I had improved flow -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LGWgtUogY6XDqSuNhfYl-Q.png" /><figcaption>Claude Code created a rich simulation tool. The 3D visualisation helped tune the new design.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j_4DCzjNogJW5ij8kGn68A.png" /><figcaption>2D simulation mode gave me a clear picture that I now had a turbulence gap due to the new hydrodynamic design of my struts. Note — the line between the transducers (‘chord’) was positioned incorrectly by Claude in this visual.</figcaption></figure><h3><strong>Success!</strong></h3><p>After much trial, error and pain, I’d finally made a breakthrough. The device can comfortably measure water velocity with initial tests showing promising results at velocities as low as 0.25 m/s.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*clZhwg1NenrNQEOf7MV9jw.png" /><figcaption>My V2 prototype — many digital iterations went through my fluid dynamics tool before I arrived on this one. I ended up with asingle 3D printed housing which I can replicate easily and mount in many rivers.</figcaption></figure><p>So let me take stock of this journey, as I don’t think any of this would have been possible 9 months ago (writing this in Feb 2026) based on where AI coding tools were then and given that Claude Code was only released Feb 21st 2025. A phone call about flooding led to:</p><ul><li>Researching deeply complicated acoustics, physics, fluid dynamics and electronics using a team of AI / Claude agents to bolster my knowledge gaps.</li><li>Building out firmware written in C/C++ that orchestrates nanosecond-timing (0.00000001s) to give extremely high accuracy</li><li>A 3D-printed housing design aided by computational fluid dynamics</li><li>A computational fluid dynamics simulator that Claude Code built (with extremely detailed prompting) from scratch in Python running on a consumer GPU — avoiding what would have been significant commercial simulation costs (roughly $5,000–$7,500 for multiple runs). My fluid dynamics tool is no where near as accurate and feature rich as SimScale but it did a pretty good job for what I needed.<br><em>Note — I added this tool as an agent so Claude could run tests on it’s own to create it’s own iterative testing loop</em></li><li>A 3.4-metre calibration flume with a trolley and infrared timing gates</li><li>A few months with a tub of water by my desk</li><li>A hose to the face</li></ul><p>Total component cost per prototype: <strong>roughly £500</strong> — a fraction of the £10,000–£20,000 typically quoted for industrial installations.</p><p>And the thing that made it possible to turn a software architect with a hobby level ability in electronics into someone who could build an ultrasonic flow meter was working with AI as a genuine engineering partner. Not “AI wrote the code for me.” More like: I hired a team of specialists who happened to live in my terminal, and together we figured it out.</p><p>What AI bubble?</p><h3>Next Steps</h3><p>I’m finalising the design to enable long-term deployment in rivers. The current prototype gives velocity readings for a single point in the water column, but the real goal is a multi-sensor configuration. Multiple transducers mounted on a vertical column to build a velocity profile across the river’s cross-section, paired with the existing above-water ultrasonic level sensor for automated depth measurement.</p><p>All of this will be uploaded to my cloud platform via 4G/5G every few minutes, giving real-time visibility of river conditions through dashboards and threshold-based alerts. No more tape measures, no more guesswork.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kjehFPVmf2OG2DH1jAVsaQ.png" /><figcaption>Tim Farron MP, John Geldard (Lynster Farmers Group, Chair) hosting the Catchment Improvement Programme update Feb 2026 — the application of this technology was discussed alongside using AI in the decision making process.</figcaption></figure><p>But the thing I keep coming back to is the density argument. A single perfect gauging station tells you what one stretch of river is doing. For a catchment like the Lyth Valley, with its network of rivers, drainage channels and infrastructure, you need coverage in twenty or thirty places, not one. At £10,000 to £20,000 per site, that’s impossible for most communities. At roughly £500 in components per sensor, it starts to become practical.</p><p>That’s the real opportunity here. Not a better sensor, but a fundamentally different monitoring strategy, where you can afford to instrument an entire catchment rather than relying on sparse, expensive measurement points. That data is what is needed to build hydrological models that will actually help communities like the Lynster Farmers Group make informed decisions about their water infrastructure.</p><p>I’ve spoken to the Farmers Guardian about the challenge facing the group, and recently met with Tim Farron MP, representatives from the Environment Agency, Westmorland and Furness Council and other agencies to discuss how this technology could support the wider catchment improvement programme.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-tqrGZgckKlYlYTlTQZW_g.png" /><figcaption>Chatting to Alex Black from the Farmers Guardian about the challenge facing the Lynster Farmers Group in Cumbria and how the device can help. Behind is a blocked pumping station — real data across the area is needed to highlight this issue and enable farmers to act</figcaption></figure><p>I’m looking forward to seeing how we can apply technologies like this to the real world problems that we’re all facing.</p><p>More to come …</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d0c4e652a37" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deterring foxes and badgers with TensorFlow Lite, Python, Raspberry Pi, Ring cameras & ultrasonic…]]></title>
            <link>https://medium.com/@james.milward/deterring-foxes-and-badgers-with-tensorflow-lite-python-raspberry-pi-ring-cameras-ultrasonic-75b3160faa3c?source=rss-529d5fc5f9df------2</link>
            <guid isPermaLink="false">https://medium.com/p/75b3160faa3c</guid>
            <category><![CDATA[esp8266]]></category>
            <category><![CDATA[raspberry-pi]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[tensorflow-lite]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[James Milward]]></dc:creator>
            <pubDate>Sun, 24 Sep 2023 20:00:23 GMT</pubDate>
            <atom:updated>2023-09-24T20:10:01.068Z</atom:updated>
            <content:encoded><![CDATA[<h3>Deterring foxes and badgers with TensorFlow Lite, Python, Raspberry Pi, Ring cameras &amp; ultrasonic deterrents</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dSSDRZ1v82yv9G4GqufIpA.png" /><figcaption>The first attempt at object identification using SSDLite, MobileNetV2 and COCO left a lot to be desired</figcaption></figure><p>A recent influx of foxes and badgers in my garden and the ensuing damage they caused gave me the perfect opportunity to consider how I could learn and utilise ML to try and reduce the number of unwanted night time visitors we’ve been experiencing.</p><p>I’ve been keeping an eye on TensorFlow and it’s object detection capabilities for quite a while and this seemed like a good opportunity to give it a try. TensorFlow Lite especially stood out as it can be deployed to low powered edge devices like the Raspberry Pi 4 or ESP-32/<a href="https://www.arducam.com/esp32-machine-vision-learning-guide/">ESP-CAM</a> which reduces the requirement for costly hardware running continually to provide object detection capabilities.</p><p>TensorFlow Lite provides several object detection models from the <a href="https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md">Model Zoo</a> and I wanted to find out which one would give a good balance of accurate identification and low-powered computational requirements. The TensorFlow Lite object detection models have really advanced in the last few years and models like SSD MobileNet V2 FPNLite can provide ~80% accuracy on low powered devices, providing significant improvements over more complex R-CNN based models or EfficientNet/EfficientDet.</p><h3>Firstly though, why deter foxes and badgers?</h3><p>I recognise the importance of living harmoniously with our wildlife, so much so that we have an active ground bee hive in our garden which I’m keen on keeping as we desperately need more pollinators and they aren’t causing us any harm.</p><p>However, foxes and especially badgers can cause an immense amount of damage in a short space of time. I have two young children and that poses further issues as this type of wildlife bring nasty insects and diseases along with them.</p><p>So I set myself a goal of building a tool that could deter both badgers and foxes <strong>without</strong> causing them any harm.</p><h3>What about off the shelf solutions?</h3><p>You may be reading this thinking that machine learning is overkill for building a wildlife deterrent. You’re probably right but I love completely insane projects and I’ve tried so many humane solutions to safely stop my unwanted garden guests, all of which failed.</p><p>So what did I try first? Scent marking repellent, natural mixes of citronella/chilli and blocking up the entry points along the fences in the garden. This just resulted in our fluffy friends ignoring my initial attempts and digging more holes. However, out of everything I tried one solution offered a glimmer of hope — solar powered ultrasonic repellents.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QBSjEg3VquhEKR6L.jpg" /><figcaption>One of the many examples of ultrasonic repellents you can purchase on Amazon. I have a few of these throughout our garden which provided mixed results.</figcaption></figure><p>I placed a few of these devices across the garden which successfully reduced the number of nightly visits we experienced. However -</p><ul><li>The deterrent has different modes for badgers and foxes which has to be manually set, resulting in some areas not being covered for one of the types of animals</li><li>They run 24/7, meaning that we trigger them when we’re in the garden which results in us turning them off and periodically forgetting to turn them back on</li><li>They don’t always get fully charged by the solar panel (cloudy days) and can go for days with no charge before we realise that they need plugging in</li></ul><h3>Using Python/TensorFlow/Arduino to improve the ultrasonic deterrent</h3><p>Ultrasonic deterrents work by detecting heat using the onboard passive infrared (PIR) sensor and triggering the speaker to bathe the target in high frequency noise (13.5— 19.5khz for foxes and 19.5–24.5khz for badgers).</p><p>Unfortunately the PIR triggers the speaker every time it detects a heat signature (i.e. me or my family), so I needed to make this more intelligent. This is where TensorFlow comes in as the <a href="https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/">Object Detection API</a> can be utilised to detect animals with a high degree of accuracy.</p><p>If badgers and foxes could be accurately identified, the deterrent could be triggered remotely. Modification of the ultrasonic deterrent would be needed to allow for remote triggering and this will be facilitated by a microprocessor with WiFi capability (ESP8266). Furthermore, the ESP8266 could switch the deterrent to a specific mode to operate at a selected frequency range and accurately deter our targets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5Ubv_QMQ0bL-yuGTq8NA1Q.png" /><figcaption>Initial attempts at passing video from a Ring Stick Up camera to the pre-trained SSD MobileNet V2 320x320 provided poor results</figcaption></figure><p>I started off using a <a href="https://www.youtube.com/watch?v=2yQqg_mXuPQ">YouTube tutorial</a> which walks you through passing images through a pre-trained TensorFlow model based on the <a href="https://cocodataset.org/#home">COCO dataset</a>. I built this out in <a href="https://colab.research.google.com/">Google Colab</a> as you can execute arbitrary python code through a web browser and is especially suited to machine learning as you get free GPU time on Tensor Core GPUs.</p><p>My first results were a little disappointing. I was extremely pleased that I had been able to get some Python together to pass images through TensorFlow but I quickly understood that the publicly available models were simply not trained to identify badgers or foxes, so they ended up being detected as a sink, car, umbrella or bear (if I was lucky).</p><h3>Training my own badger/fox model</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FXZ7FYAMCc4M&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DXZ7FYAMCc4M&amp;image=http%3A%2F%2Fi.ytimg.com%2Fvi%2FXZ7FYAMCc4M%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/b8b43b59207cbfae9fdbc0b378bb87d6/href">https://medium.com/media/b8b43b59207cbfae9fdbc0b378bb87d6/href</a></iframe><p>An evening of Googling led me to discover this fantastic <a href="https://www.youtube.com/watch?v=XZ7FYAMCc4M">YouTube tutorial</a> which walks you through training a TensorFlow Lite object detection model based on your own custom imagery.</p><p>The tutorial stresses the importance of getting a decent set of imagery together for the object you’re looking to detect with TensorFlow. I’d previously been able to get decent imagery from my Ring Stick Up camera which had caught the badgers/foxes in the act of wrecking my garden so off I went to the Ring portal. I spent another evening screen capturing videos until I had a few hundred images of the badgers and foxes which I dumped into Google Drive.</p><p><a href="https://github.com/HumanSignal/labelImg">LabelImg</a> was used to annotate each image and I identified the position in each image of a fox or badger (alongside other garden items/animals), outputting the labelling to a PASCAL VOC XML format which TensorFlow understands. I uploaded this to Google Drive, fired up Colab, configured my training and set the Python script to train a new model for a few hours.</p><p><em>Note — I’d recommend purchasing a Google Colab plan to access the Nvidia A100 Tensor Core GPU’s as this significantly decreased the time it took to train models. Roughly £9 gets you 100 ‘compute units’ and one training session consumed around 15 of these.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hn7voAH7td3ypTMFFVEmxQ.png" /><figcaption>Training the data model and monitoring in TensorBoard — accuracy increased as the model went through 30,000 steps of image training</figcaption></figure><p>After roughly 2.5 hours, the model was looking good and started to show a reduction in improvement/learning so I decided to pull the plug and got my hands on a freshly baked badger/fox TensorFlow Lite model.</p><h3>badger != umbrella</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/397/1*wKDmsLyxmR_EwXk13VP4-w.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/294/1*11fKzBdW02OrlgVugep_vQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/330/1*03NFSD7x7oKlI2j5z2_Itw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/226/1*nw7GoczVeqHQhLssQJS6Jw.jpeg" /><figcaption>240 annotated images and 2.5 hours of training provided a model with decent accuracy</figcaption></figure><p>Success! Running TensorFlow’s object detection API with my newly trained model gave decent results against a random sample set of my images. The overall Mean Average Precision (mAP) of the model was coming out at 62.21% for 2.5 hours of training — not bad. I could improve this by training it against more images but I had enough to get a proof of concept working.</p><p>Next, I needed to progress from still imagery and implement a solution for streaming video from the garden to a local instance of TensorFlow Lite running the newly baked fox/badger model. This would be eventually hosted on a Raspberry Pi 4 but could be initially built and deployed to Docker locally to enable rapid prototyping.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O4Vfvz4-u2U5209jtrASfg.png" /><figcaption>End-to-end concept of the system. MQTT provides lightweight communication and allows for multiple deterrents to be deployed and issued commands from a single Raspberry Pi 4.</figcaption></figure><p>The above concept set a rough architectural basis for how the system would hang together. With this in place, it felt like a good time to bring the software and hardware components of the system to life. Firstly, this contraption needed a decent name and thankfully ChatGPT has a wealth of bad ideas when it comes to product names based on ridiculous concepts …</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/671/1*bPKEmc0znYxlVw3O7Pb_fg.png" /><figcaption>Thanks ChatGPT!</figcaption></figure><h3>Building the Furbinator 3000</h3><h4>Step 1 — Camera streaming via ring-mqtt</h4><p>Now that I had a working detection model and badgers were no longer being identified as an umbrella, I needed to connect a night vision camera which provided streaming capability for TensorFlow to consume.</p><p>The popular Ring camera system in theory has all of the required attributes — high quality night vision imagery and internet connectivity, plus I already had a few installed. Unfortunately Ring has no official API but has a decent <a href="https://www.npmjs.com/package/ring-client-api">unofficial API</a> and the brilliant <a href="https://github.com/tsightler/ring-mqtt">ring-mqtt</a> project has been built on top of this library to provide a solid foundation for streaming video from Ring cameras via RTSP and alerting to motion events via a rich MQTT implementation. Thankfully, this library turned out to be superb with excellent documentation — thanks <a href="https://medium.com/u/7fbf0c249eed">Tom Sightler</a> / <a href="https://github.com/tsightler">tsightler</a>. Download it <a href="https://github.com/tsightler/ring-mqtt">here.</a></p><blockquote>Note — this library is designed to stream RTSP when a motion event is detected and shouldn’t be used for 24/7 feeds as this will cause issues (i.e. breaking the Ring fair usage policy).</blockquote><p>I have configured this to run both the ring-mqtt container and an mqtt-broker container with Docker Compose, the YAML for this is in included in the <em>docker</em> folder of the Python app.</p><h4>Step 2— Building the Python TensorFlow app</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/917/1*je2_vecKcpBy2u_jcK3jQA.png" /><figcaption>The <a href="https://github.com/Jimbwlah/tensorflow-ring-animal-detector">Python tensorflow-ring-animal-detector project</a> I’ve put together to breathe life into the Furbinator 3000</figcaption></figure><p>I built a <a href="https://github.com/Jimbwlah/tensorflow-ring-animal-detector">Python application</a> to provide object detection through the TensorFlow API. This application also facilitates training and testing of new models.</p><p>The app first needs to be configured via <em>constants.py </em>and is then run from <em>src/tf_model_stream_analysis.py. </em>This application does the following -</p><ol><li>Connects to a locally hosted MQTT broker and listens for Ring motion events from the ring-mqtt library<br><em>- see the </em><strong><em>MQTTHandler</em></strong><em> class</em></li><li>Upon a motion event being identified, starts a RTSP stream from the ring-mqtt library. I’ve ingested each stream via a multi-threaded video dispatcher to enable high performance when multiple cameras are used <br><em>— see the </em><strong><em>ThreadedVideoDispatcher </em></strong><em>class</em></li><li>The RTSP stream is passed to OpenCV which captures the stream frame by frame using the FFMPEG codec. I’ve added a customisable FPS limiter to this to reduce the CPU overhead which is especially helpful on the Raspberry Pi <br><em>— see the </em><strong><em>VideoStream </em></strong><em>class</em></li><li>Each video frame is passed to the TensorFlow Object Detection API, which is used in conjunction with the pre-trained model I have built <br><em>— see the </em><strong><em>TFVideoAnalysis </em></strong><em>class</em></li><li>Upon a fox/badger being detected a MQTT message is published to the MQTT broker and a snapshot is taken and saved locally<br><em>- see the </em><strong><em>ClassTracker </em></strong><em>and </em><strong><em>FurbinatorActivator </em></strong><em>classes</em></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/592/1*qD300_GT42-LC-BtuCdtUg.png" /><figcaption>ClassTracker/FurbinatorActivator taking a snapshot of a Badger</figcaption></figure><h4>Step 3— Reducing false positives</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/646/1*66Fw-jMm1bUzGjaCFjx9oQ.png" /><figcaption>Oddly shaped leaves/grass/cobwebs presented multiple false positives — a real issue if we want to reliably trigger an ultrasonic deterrent. That patch of gras does look like a fox too! Argh</figcaption></figure><p>Following some testing I started to notice a number of false positives from the system. Thankfully most of these false positives were triggered from stationary objects such as fallen leaves, grass patterns or debris caught in cobwebs.</p><p>To get around this issue, I decided to combine motion detection with object detection. To detect movement, I compared each frame of the RTSP stream to a previous one to detect any pixels that had changed, largely based on the fantastic work in this <a href="https://towardsdatascience.com/image-analysis-for-beginners-creating-a-motion-detector-with-opencv-4ca6faba4b42">blog</a> -</p><pre># NOTE - Run this inside a while loop which takes individual frames from a<br># threaded cv2.videoCapture() function<br>first_frame = None<br>frame_diff = None<br><br>prepared_frame = cv2.cvtColor(frame_rgb, cv2.COLOR_BGR2GRAY)<br>prepared_frame = cv2.GaussianBlur(src=prepared_frame, ksize=(5,5), sigmaX=0)<br><br># Calculate difference and update previous frame<br>if (first_frame is None):<br>    first_frame = prepared_frame<br>else:<br>    # cv2.absdiff calculates the absolute difference between two arrays<br>    frame_diff = cv2.absdiff(src1=first_frame, src2=prepared_frame) <br>    first_frame = prepared_frame <br><br>    # Dilute the image a bit to make differences more seeable; <br>    # more suitable for contour detection<br>    kernel = np.ones((5, 5))<br>    frame_diff = cv2.dilate(frame_diff, kernel, 1)<br><br>    # Only take different areas that are different enough (&gt;20 / 255)<br>    thresh_frame = cv2.threshold(src=frame_diff, thresh=25, maxval=255, type=cv2.THRESH_BINARY)[1]<br>    cv2.imshow(&quot;Motion Detection Analysis&quot;, thresh_frame)<br><br>    # Use motion contours when an object is detected and correlate<br>    motion_contours, _ = cv2.findContours(image=thresh_frame, mode=cv2.RETR_EXTERNAL, method=cv2.CHAIN_APPROX_SIMPLE)</pre><p>Here we take each frame from a multi-threaded videoCapture() process, convert to black and white (colour adds unnecessary complexity) and apply a Gaussian Blur to smooth it out slightly. An absolute differential of a first and second frame is calculated as a new frame, this frame is dilated to even out any anomalies and then each pixel is converted to either white or black.</p><p>We then find the area that has changes since the last frame (cv.findContours), apply object detection to this frame, iterate over each object detection score for each class detected in the frame and calculate the location of the motion detection vs the location of the object detection -</p><pre># Motion detection inside object detection - determine if it&#39;s near <br># the object that&#39;s been detected. This improves object detection as <br># it can mistakenly recognise stationary grass/leaves/patterns <br># as animals<br>motion_near_object = False<br>if motion_contours != None:<br>    for contour in motion_contours:<br>        (x, y, w, h) = cv2.boundingRect(contour)<br><br>        # Calculate difference between coords of detected object and <br>        # detected motion. ymin, xmin, ymax, xmax are the coords of the<br>        # object detection bounding box<br>        ymin_calc = abs(ymin-y)<br>        xmin_calc = abs(xmin-x)<br>        ymax_calc = abs(ymax-(y+h))<br>        xmax_calc = abs(xmax-(x+w))<br><br>        # Motion detected if difference is no lower than threshold <br>        # (20% of resolution width)<br>        threshold = self.resolution_w/20<br>        if ymin_calc &lt;= threshold and xmin_calc &lt;= threshold and ymax_calc &lt;= threshold and xmax_calc &lt;= threshold:<br>            cv2.rectangle(img=frame, pt1=(x, y), pt2=(x + w, y + h), color=(255, 0, 0), thickness=2) # blue<br>            motion_near_object = True</pre><p>If the detected object is near the motion detection (the max distance I set between the two was 20% of the image width) then display the motion contour and set the <em>motion_near_object </em>flag. This looks a little something like this -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*W3RrKiv9fC2EnS0q.gif" /><figcaption>Note the fox being sometimes detected as a badger and an item on the floor being incorrectly identified as a bird. This is the side effect of a model that’s only been trained for 2.5 hours. This went away with further training of the model.</figcaption></figure><p>The code can be improved, especially the location of the motion detection bounding boxes vs the object detection bounding boxes but for now this works and significantly reduced the number of false positives.</p><p><strong>To recap</strong>, we now have the ability to -</p><ul><li>Stream video from Ring/RTSP cameras to Python</li><li>Analyse the video with both object detection in TensorFlow and motion detection in OpenCV</li><li>Trigger an event with the <em>motion_near_object</em> flag. In the Python app I put together, this publishes the following to MQTT which a modified ultrasonic deterrent can listen for with a bit of Arduino -</li></ul><pre>self.mqtt_client.publish(&quot;foxbadger&quot;, &quot; **strobe&quot;)</pre><h4>Step 4 — A physical prototype</h4><p>Now that events are reliably published to MQTT when a fox/badger is detected, an ESP8266 equipped WeMos D1 Mini Pro was used to listen for this event and control off-the-shelf deterrent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dNMuut6vsM4hWr13.jpg" /><figcaption>WeMos D1 Mini Pro — ESP8266 equipped microprocessor with USB, WiFi and multiple GPIO, ideal for controlling the ultrasonic deterrent</figcaption></figure><p>The design is very simple — when the ESP8266 received a message via MQTT, it does two things</p><ul><li>Sets GPIO4 to HIGH for 22 seconds — This turned the ultrasonic deterrent on. This was pre-set to a mode which filtered through all frequencies and fired the onboard LEDs. (note — a later design will set these as necessary via MQTT)</li><li>Sets GPIO5 to HIGH for 50ms, then LOW for 50ms and repeats for 22 seconds in total — creating a basic strobe</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hG2_S7Bw9Raa09ytXAH7TA.png" /><figcaption>Basic circuit illustrating how the components are connected. NOTE — The IRF520 MOSFET was necessary between the 12v step up converter and 12v LED strip as the EN pin on the step up converter allowed a small amount of voltage to leak when disabled, causing the LEDs to be very dimly lit when in the OFF state. The MOSFET ensured no voltage passed to the LED unless GPIO5 was HIGH. A decent 12v step up converter would eliminate the need for this — mine was obviously very cheap.</figcaption></figure><p>The Arduino code I put together can be found in the Furbinator3000 repo <a href="https://github.com/Jimbwlah/furbinator3000">here</a>.</p><p>The components used are -</p><ul><li>WeMos D1 Mini Pro + Antenna —<a href="https://www.ebay.co.uk/itm/295625321004?chn=ps&amp;_trkparms=ispr%3D1&amp;amdata=enc%3A1Oot227RgQb--v6vtzcQLTA79&amp;norover=1&amp;mkevt=1&amp;mkrid=710-134428-41853-0&amp;mkcid=2&amp;mkscid=101&amp;itemid=295625321004&amp;targetid=1647205089240&amp;device=c&amp;mktype=pla&amp;googleloc=1006596&amp;poi=&amp;campaignid=17206177401&amp;mkgroupid=136851690655&amp;rlsatarget=pla-1647205089240&amp;abcId=9300866&amp;merchantid=752517336&amp;gclid=Cj0KCQjw06-oBhC6ARIsAGuzdw3Ek6n8-wMP2EvbzpBSeKy_WZy9ApIWjxOYUzQPo56GCZdtRVZxFesaArfcEALw_wcB"> buy here</a></li><li>3.3v IRF520 MOSFET Driver Modules (x2) —<a href="https://www.ebay.co.uk/itm/224231058831?chn=ps&amp;_trkparms=ispr%3D1&amp;amdata=enc%3A13MiB4z64QI2j6FCc6RSuCA52&amp;_ul=GB&amp;norover=1&amp;mkevt=1&amp;mkrid=710-134428-41853-0&amp;mkcid=2&amp;mkscid=101&amp;itemid=224231058831&amp;targetid=1647205089240&amp;device=c&amp;mktype=pla&amp;googleloc=1006596&amp;poi=&amp;campaignid=17206177401&amp;mkgroupid=136851690655&amp;rlsatarget=pla-1647205089240&amp;abcId=9300866&amp;merchantid=232684032&amp;gclid=Cj0KCQjw06-oBhC6ARIsAGuzdw2VJF0n5RYT6El6os_r-6ypg735NidalEjyKa6vMEGlTqxoIyXGcTgaAm1bEALw_wcB"> buy here</a></li><li>12v DC-DC converter step-up voltage booster — <a href="https://www.ebay.co.uk/itm/252996434711?hash=item3ae7c33b17:g:-6kAAOSwjKFZO8FO&amp;amdata=enc%3AAQAIAAAA8AasmADT6XmIydNByOtZS%2BxxyteDBDx%2BLFpWWwXw1OLjYM2%2BoWoAe%2F7LfDLVXBidshpEcuAoxzxuCV3w4xnbXmIOR0v55Ohp9uPlaghvIKB5rLDg4M2ZTBNAi%2FzEZhnGkwjmRmiEjwkQQvyQEro1kvgERKKqeO3y1HcTspjc9lgihpBfbnSEoW2d0WuEeMdi%2FSuHN0FPYuKP1C9%2FX2%2FEAVSKd8sDAqdJAElRarOnx8oYfoHOF7fG9O9Y%2BrIzSWx0BdtvsdVUScqLgYJmD21QXYnRIdz1o5BsZqtvd%2BnxMW2iG9MwmmjSHxSw8FhTp3P8ow%3D%3D%7Ctkp%3ABFBMzL3XgNdi">buy here</a></li><li>12v waterproof LED strip — <a href="https://www.ebay.co.uk/itm/374308743706?var=643188024628&amp;hash=item57268a2a1a:g:41UAAOSwyTdjTRq6&amp;amdata=enc%3AAQAIAAAAwLsDqrNA14epeJK%2B5lqmp%2FthtMwX1OS0lZt7J62IV5C8DONaRqxD0E0WD0HrojD%2Fvn3ef9sD58gcNk%2FhopMCzY%2BCL%2F5T2srNmmjKPfozfzSKhgkboFADWRh7TGL4GIgeVF2pT8amzjXm5PqtIer6XN4nQ3bAKCPlH3OPHpJLMTMJisyT1aUF2fxcjdm2kR%2B1eyPGxh00XSYeqmmPTxt%2FxTwQMYNNgEcDtiJ%2FoPDy%2FxQ36cQr2uwO7L31s6c8Pr0xHw%3D%3D%7Ctkp%3ABk9SR5aD6oDXYg">buy here</a></li><li>Modified ultrasonic deterrent — <a href="https://www.amazon.co.uk/gp/product/B0BX9P4Y46/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&amp;psc=1">buy here</a></li></ul><p>I’ve wired these all together on a breadboard and housed them in a box that I 3D printed with some leftover white PETG I had lying about. This has been glued onto the back of the battery compartment and I cut a hole through the cover and battery compartment with a 32mm flat wood bit. I’ll eventually put a PCB together in Fritzing which I can order with components pre-soldered but for now this works. The breadboard is especially useful at this stage as it’s incredibly easy to modify if I have any issues.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zTyp8Cfly8ROBkZqb14vBg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/860/1*N6aZeYzRR4GAG4KNvl3a2Q.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/914/1*AX2h3MvIytW3uVMeLwo3-g.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dkQvdnZziZJIwaCj0HkS4Q.jpeg" /><figcaption>Hideous but it works</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*a_dwQMy2oqSoEcjK.gif" /><figcaption>Testing the strobe, red/white LED and speaker with a manual MQTT publish</figcaption></figure><h3>Deploying the Furbinator 3000</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/554/1*NXoR1sj6M2CCMWAtgI3zpw.gif" /><figcaption>Success! Half way through you can see the Badger consider getting his claws into the lawn but swiftly decides to move on.</figcaption></figure><p>As this was developed within docker, setup of the Raspberry Pi 4 (4GB) with a 32gb SD card and deployment to Ubuntu was trivial. It seems to be handling object detection via the SSD MobileNet V2 FPNLite 640x640 model relatively well. <br><em>Note — I had to disable the directML variant of TensorFlow for this to work.</em></p><p>After a few nights of testing, it looks like the local wildlife have decided to quickly exit the garden when the Furbinator is doing it’s thing. I’m used to watching Badgers ripping up my garden from this angle and I cannot describe how happy this makes me that my garden will no longer look like a battlefield when I’m waking up to have my morning coffee.</p><h4>What’s next?</h4><p>Time will tell as to whether the badgers and foxes get used to this thing or not. I have a second version planned which is equipped with a small amplifier to play mp3’s of dogs barking but for now I’m going to sit back and see what happens. Plus I like my neighbours and I fear the dog barking variant is a sign that I’ve gone completely mad.</p><p>Watch out for a follow up blog on how I deployed this to a Raspberry Pi, improved the model with more imagery and performance of the model on low powered compute.</p><p>Thanks for reading.</p><p>— James</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=75b3160faa3c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>