<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Gilberto Briscoe-Martinez on Medium]]></title>
        <description><![CDATA[Stories by Gilberto Briscoe-Martinez on Medium]]></description>
        <link>https://medium.com/@gibr6485?source=rss-172b511c53b1------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*A_lWy9B8lzFXodEca5dtBQ.jpeg</url>
            <title>Stories by Gilberto Briscoe-Martinez on Medium</title>
            <link>https://medium.com/@gibr6485?source=rss-172b511c53b1------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 10:36:18 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@gibr6485/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Learning Fine Tuning]]></title>
            <link>https://medium.com/@gibr6485/learning-fine-tuning-19f8dece1156?source=rss-172b511c53b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/19f8dece1156</guid>
            <dc:creator><![CDATA[Gilberto Briscoe-Martinez]]></dc:creator>
            <pubDate>Sun, 16 Aug 2026 06:09:54 GMT</pubDate>
            <atom:updated>2026-08-16T06:09:54.254Z</atom:updated>
            <content:encoded><![CDATA[<p>The past two (…oops) weeks I’ve been getting my hands dirty with learning how to fine tune Vision Language Models. My goal was to see how good these models were at identifying the key components of the package induction task I’ve been tackling for the past few weeks. I asked the VLMs a few questions, structured as a JSON: 1) how many boxes did it see, 2) is everything ready to pick, 3) which box should each arm pick, and 4) what is the center (in pixel coordinates) on the screen. The results were bad, very bad (like less than 1% accuracy on any of those questions bad). So I buckled up and dove into the world of fine tuning.</p><p><strong>What is fine tuning and how do you do it?</strong></p><p>Fine tuning is the method of taking a Large Language Model and giving it targeted training examples to improve its success at a specific task you want it to do. The goal is to update the weights of the model to get them to be better at predicting the correct answer with the info you get it.</p><p>However, the weights of these models are massive. Even with the smaller models I’m using of 8B parameters, training them in the same way NVIDIA and Tencent do theirs is unteneble for us mere mortals. Fortunately, highschool linear-algebra is on our side. At a high-level we want to take the equation h=Wx, where h is the output of the LLM, W is its weight matrix, and x is the inputs we give it, and update W to make h more correct when we give it a ceratin x. W is usually massive (those 8b parameters I mentioned earlier), so we can take a shortcut and instead do h=(W+C)x where C is the matrix we are updating. The hack comes in by splitting C into two small matricies, A and B, that, when multiplied together, produce C. This leads to a much smaller number of parameters we have to update to train the model. <a href="https://arxiv.org/pdf/2106.09685">Read more here.</a></p><p>Fortunately I didn’t have to implement this from scratch, as I used<a href="https://swift.readthedocs.io/en/latest/GetStarted/Quick-start.html"> ms-swift. </a>This framework meant I could produce a training dataset from my real runs. This dataset consisted of 375 prompts and answers where the answers were generated in JSON from the simulations. I then ran the training on A100s.</p><p><strong>Results</strong></p><p>With all this…the results weren’t that good. When finetuning Qwen-3 8B, getting the box centers worked wonders, going from less than 1% to 77% success. Counting boxes improved, but is still unusable, going from under 1% to ~33%, and selecting which box to pick actually got worse, going from 73% success to 39% success.</p><p>This leaves me with a couple of open questions: was my dataset good enough, will this improvement transfer if I connect it to an action model for controlling the robots and conveyors, and is using a VLM even worth it with these abysmal results. Only time will tell.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=19f8dece1156" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Collection and Model Training on Google Cloud Agent Platform]]></title>
            <link>https://medium.com/@gibr6485/data-collection-and-model-training-on-google-cloud-agent-platform-e61bcf35d978?source=rss-172b511c53b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/e61bcf35d978</guid>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Gilberto Briscoe-Martinez]]></dc:creator>
            <pubDate>Mon, 03 Aug 2026 22:13:55 GMT</pubDate>
            <atom:updated>2026-08-03T22:14:01.359Z</atom:updated>
            <content:encoded><![CDATA[<p>I ride bikes a lot. In Boulder, there are two parts of a bike ride: the grind up the mountain and the descent down it. When I do research I feel roughly the same; some weeks are super fast like I’m blazing down <a href="https://www.strava.com/segments/7011796">Flagstaff</a>, others are slow and grinding like I’m climbing <a href="https://www.strava.com/segments/8795306">Gold Hill</a>. This week was a Gold Hill. Building on last week’s simulation development, it was time to get to switch gears and start training the AI models. However, before I could start, I needed to collect data. In the end, I deployed 8 concurrent runs to Agent Platform (to avoid the nightmare of multiprocessing with GPU instances) and collected 3.65 hours of sim data to start initial testing. Below I’ll discuss my data collection approach, some discussions on visualization and conversion to the training datasets, and how I am deploying the training runs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/979/1*nSZzVZuHL3au6yiD5cW-IA.png" /><figcaption>A screenshot of the Isaac Sim environment setup for taking commands from a trained model.</figcaption></figure><p><strong>What is Agent Platform?</strong></p><p><a href="https://cloud.google.com/products/gemini-enterprise-agent-platform?hl=en">Gemini Enterprise Agent Platform</a>, or Agent Platform for short, is Google Cloud’s tool for developing custom AI models. For my use case, it is replacing the Terraform code I previously wrote to automatically startup a cloud GPU instance, train a model, upload the checkpoints and automatically delete the instance once it’s all done with one fairly straightforward yaml interface that you start with a short command, shown below.</p><pre>workerPoolSpecs:<br>  machineSpec:<br>    machineType: a2-highgpu-1g<br>    acceleratorType: NVIDIA_TESLA_A100<br>    acceleratorCount: 1<br>  replicaCount: 1<br>  diskSpec:<br>    bootDiskType: pd-ssd<br>    bootDiskSizeGb: 500<br>  containerSpec:<br>    imageUri: &lt;docker image lcoation&gt;<br>scheduling:<br>  strategy: FLEX_START<br>  maxWaitDuration: 2700s</pre><pre>gcloud ai custom-jobs create --display-name=&quot;act-event&quot; --region=us-central1 \<br>    --config=.../train_event.yaml</pre><p>Using Agent Platform, the system spins up your docker image with a <a href="https://docs.cloud.google.com/gemini-enterprise-agent-platform/machine-learning/training/cloud-storage-file-system">premounted cloud storage</a> at /gcs. This means that you can have your training data look as if it is directly included in the image and write the checkpoints directly to cloud storage buckets. Further, shutdown of the system automatically occurs when the training run returns 0 (or an error code).</p><p><strong>Data Collection Deployment</strong></p><p>For my initial testing, I set the goal to collect four sim-hours of pick and place data. In the sim I placed six cameras, 2 above the pick conveyors, 2 above the place conveyors, and 2 on each robot’s wrist. They ran at 640x480 at 30fps. I wrote a short script that launched 8 simultaneous runs to keep the total time it took to collect short. For each run, I randomized where the boxes would be placed on the first conveyor with a unique random seed.</p><pre>set -euo pipefail<br><br>INSTANCES=&quot;${1:-8}&quot;<br>TOTAL_SIM_SECONDS=&quot;${2:-14400}&quot;<br>SEED_BASE=&quot;${3:-1000}&quot;<br>BUCKET=&quot;${4:}&quot;<br><br>SIM_SECONDS_PER_INSTANCE=$(python3 -c &quot;print(${TOTAL_SIM_SECONDS} / ${INSTANCES})&quot;)<br>RUN_ID=&quot;run_$(date -u +%Y%m%dT%H%M%SZ)&quot;<br>GCS_ROOT=&quot;/gcs/${BUCKET}/data_collection/sim/${RUN_ID}&quot;<br><br>SCRIPT_DIR=&quot;$(cd &quot;$(dirname &quot;${BASH_SOURCE[0]}&quot;)&quot; &amp;&amp; pwd)&quot;<br><br>echo &quot;Submitting ${INSTANCES} job(s), ${SIM_SECONDS_PER_INSTANCE}s sim-time each, run_id=${RUN_ID}&quot;<br><br>for ((i = 0; i &lt; INSTANCES; i++)); do<br>  instance_dir=$(printf &quot;instance_%02d&quot; &quot;${i}&quot;)<br>  seed=$((SEED_BASE + i))<br>  gcloud ai custom-jobs create \<br>    --display-name=&quot;sim-collect-${RUN_ID}-i${i}&quot; \<br>    --region=us-central1 \<br>    --config=&quot;${SCRIPT_DIR}/gcp_collection_config.yaml&quot; \<br>    --args=&quot;--sim-seconds=${SIM_SECONDS_PER_INSTANCE},--instance-index=${i},--seed=${seed},--gcs-prefix=${GCS_ROOT}/${instance_dir}&quot;<br>done</pre><p>These runs produced <a href="https://github.com/foxglove/mcap">MCAP</a> files that tracked everything in the scene. I tracked the robot and conveyor states needed for training the models to command them but also the box states for potential future uses so we could reconstruct the scene.</p><p><strong>Dataset Conversion and Episode Definition</strong></p><p>MCAP, while incredibly useful as a data collection tool, is not the dataset used for any model training I am doing. So, I had to convert. For initial testing, I am using lerobot to train a simple ACT model that controls both robots and all of the conveyors at the same time. (This was the simplest jump code wise from controlling one robot and no conveyors previously). An immediate issue was defining an episode from the continuous data of the MCAP. I tested two ideas, a fixed time interval for an episode (6 seconds in this case) and event based episodes where one of the two robots began to do a pick. The shape of each state is 2x6 joints, 2x1 end effector state, 2x5 conveyor state. I then created two datasets using the below pseudocode.</p><pre>INPUT: dir of MCAP files, output root, fps, arms, cameras, conveyors, segment strategy<br><br>1. Collect + sort .mcap files by start time (skip INCOMPLETE)<br>2. Scan all files once to discover camera roles present (color topics)<br>3. Replay messages in log-time order, tracking latest known:<br>     arm state (per arm), move target seen?, conveyor state, camera images<br>   - Emit a &quot;snapshot&quot; whenever ALL required streams have a value<br>     (all arms&#39; state + conveyors + all cameras)<br>   - Sampling: one snapshot per robot state message<br>   - Assign each snapshot an episode id:<br>       fixed:&lt;s&gt; -&gt; new episode every s seconds<br>       event -&gt; new episode when an arm leaves WAITING phase<br>4. Peek first snapshot -&gt; detect image size, camera set, conveyor set,<br>   used to define the LeRobot dataset schema (observation.state/action/images)<br>5. Create empty LeRobotDataset with that schema<br>6. Re-replay full stream through a sliding window (current + N future states):<br>     - build observation.state = window of states, action = future state<br>     - optionally drop near-static frames (--no-still-threshold)<br>     - on episode-id change: save accumulated frames as one episode<br>     - otherwise: add frame (state + action + image) to current episode<br>7. Save trailing episode, log totals (frames written / total), move log into output dir<br><br>OUTPUT: LeRobot-format dataset directory ready for training</pre><p>In the end the fixed episode dataset had 2190 episodes and the event-based has 3559. Both had the some total number of state/action pairs.</p><p><strong>Training Deployment to Agent Platform and Next Steps</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f0a4pssdgeB89prQ-aFvrQ.png" /><figcaption>Loss Curve for Training ACT models with episodes defined in fixed intervals (blue) and episodes defined as one of the two robots beginning a pick (gray). Trained for 10 epochs on 4 hours of simulated data.</figcaption></figure><p>Over the weekend I deployed training runs for both of the datasets on ACT. Immediately we see that event based episodes lead to a lower loss. I will test the models this week. To see how that translates to actual performance. Further I will test different model layouts, including adding Vision-Language models into the system to test their impacts on coordinating all the individual parts of the system.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e61bcf35d978" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[IsaacSim Conveyor Package Induction]]></title>
            <link>https://medium.com/@gibr6485/isaacsim-conveyor-package-induction-e24db5b544f1?source=rss-172b511c53b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/e24db5b544f1</guid>
            <category><![CDATA[state-machine]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[simulation]]></category>
            <category><![CDATA[isaac-sim]]></category>
            <dc:creator><![CDATA[Gilberto Briscoe-Martinez]]></dc:creator>
            <pubDate>Mon, 27 Jul 2026 15:10:39 GMT</pubDate>
            <atom:updated>2026-07-27T15:20:07.143Z</atom:updated>
            <content:encoded><![CDATA[<p>A constant discussion topic between Yaashia and I is how robots never ‘just work.’ That sentiment was ever present this week as I endeavored to make a conveyor package induction simulator. Note this isn’t perfect induction as the packages aren’t randomly stacked (yet). The ultimate goal is to have a test bed where I can quickly train different learning model configurations on how to run 5 individually controllable conveyor belts and two robot arms in a package induction task.</p><p>In the process of setting up the conveyors and the robots there were four distinct sub-problems to tackle: 1) setting up the conveyors in the simulator, 2) indexing the conveyors to move the packages on them, 3) setting up the robots to pick and place them, and 4) setting up cameras for inputs to the models. In each of these, very little ‘just worked’, but in the end there were two robots simultaneously picking packages off of the conveyors.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FMz-B6VjEhVM%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DMz-B6VjEhVM&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FMz-B6VjEhVM%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/0e2cea0b123dc3c124f0b273f93c162b/href">https://medium.com/media/0e2cea0b123dc3c124f0b273f93c162b/href</a></iframe><p>All code referenced below is available here: <a href="https://github.com/HIRO-group/isaac_conveyor_and_robot_control">https://github.com/HIRO-group/isaac_conveyor_and_robot_control</a></p><p><strong>IsaacSim Conveyor Belt Utility Tool</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z3ry4K2H9Zx8QdM5QwpDVg.png" /><figcaption>Screenshot of IsaacSim Conveyor Belt Utility GUI. There are many types of conveyors included in the Builder, I used only the simplest for this setup, but there are many possibilities with it I have yet to explore.</figcaption></figure><p>The IsaacSim <a href="https://docs.isaacsim.omniverse.nvidia.com/6.0.1/digital_twin/warehouse_logistics/ext_isaacsim_asset_gen_conveyor.html">Conveyor Belt Utility</a> tool allows you to easily place conveyor belts in the environment. I used it to create a couple of environments. At first I made a racetrack. The idea was to have the robot transfer boxes from one of the conveyors to the other. But this caused many complexities in the design of the controller as the curved belts had interesting design implementations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AAVbrh57elVRIy-DTxDZfA.png" /><figcaption>Racetrack Conveyor Belt Setup in IsaacSim</figcaption></figure><p>Then I created a five conveyor setup. This is what I ended up going with as it more closely matches the real world evaluation setup I will eventually test on. Note that I did not add the robots into the USD so I could control their spawning and motion planning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qIYnNAiTHNSzkVArSpYnSQ.png" /></figure><p>One caveat is this tool is excellent at placing and connecting a conveyor belt system, but it comes with no built in way of controlling the conveyor belts.</p><p><strong>Conveyor State Machine</strong></p><p>So I had to make a conveyor belt state machine from scratch. Each conveyor has its own state machine that looks ahead and behind it to decide what to do. This is so that we can eventually teach learning models to control the conveyors individually. Every step, each conveyor takes in 3 booleans: (occupied, upstream_occupied, downstream_clear), plus at_stop_position. Occupied looks at the current conveyor, upstream looks before it, and downstream looks after. The stop position is a percentage of the conveyor I hard-coded to keep the packages from touching the downstream conveyor when stopping.</p><pre># conveyor_state_machine.py — what one zone checks every tick<br>def step(<br>    self,<br>    occupied: bool,<br>    upstream_occupied: bool,<br>    downstream_clear: bool,<br>    at_stop_position: bool = True,<br>) -&gt; tuple[ZoneObservation, ZoneCommand]:</pre><p>Each conveyor state machine cycles through the following states:</p><ul><li>Empty: nothing here.</li><li>Ready to Receive:<strong> </strong>upstream has a box coming.</li><li>Waiting to Induct<strong>: </strong>a short handshake delay before committing to run, standing in for a real upstream/downstream signal I don’t have in sim yet.</li><li>Inducting: belt runs, and keeps running <em>past</em> the first moment the box is detected as on the conveyor, until it actually reaches its stop position.</li><li>Idle: this one’s overloaded on purpose: Idle while occupied means &quot;holding, waiting on downstream&quot; (or a robot actively picking from it); Idle while unoccupied short-circuits straight back to Empty instead of getting its own case.</li><li>Advance Item; Passthrough: moving packages between conveyors, split in two so a box that’s still leaving the upstream zone (Advance Item) doesn&#39;t get confused with a box that&#39;s already arrived at the downstream (Passthrough).</li></ul><pre># conveyor_state_machine.py — the Idle → Advance Item → Passthrough handoff<br>elif s == Machine.CONVEYOR_STATE_MACHINE_IDLE:<br>    if not occupied:<br>        self._state = Machine.CONVEYOR_STATE_MACHINE_EMPTY<br>    elif downstream_clear:<br>        self._state = Machine.CONVEYOR_STATE_MACHINE_ADVANCE_ITEM<br><br>elif s == Machine.CONVEYOR_STATE_MACHINE_ADVANCE_ITEM:<br>    if not occupied:<br>        self._state = Machine.CONVEYOR_STATE_MACHINE_EMPTY<br>    elif not downstream_clear:<br>        self._state = Machine.CONVEYOR_STATE_MACHINE_PASSTHROUGH</pre><p><strong>Pick and Place State Machine</strong></p><p>Once the conveyor belts could move a box to a stop and hold it there, I needed a robot to actually pick the box up and set it down on the second loop. I did not want to simulate a real gripper closing on a box, since getting contact physics between a gripper and a box to work reliably was its own problem I did not want to take on for this proof of concept. Instead I used what I ended up calling a magic attach: once the arm gets close enough to the box, I create a fixed joint between the arm and the box in code, so the box moves exactly with the arm from that point on.</p><pre># pick_and_place.py - attach the box to the wrist with a fixed joint<br>joint = UsdPhysics.FixedJoint.Define(stage, self._attach_joint_path)<br>joint.CreateBody0Rel().SetTargets([Sdf.Path(self._wrist_link_path)])<br>joint.CreateBody1Rel().SetTargets([Sdf.Path(box_path)])<br>joint.CreateLocalPos0Attr().Set(Gf.Vec3f(*local_pos0.tolist()))<br>joint.CreateLocalRot0Attr().Set(Gf.Quatf(float(local_rot0[0]), Gf.Vec3f(*local_rot0[1:].tolist())))</pre><p>When the arm reaches the place point, I delete that joint and the box drops onto the belt on its own.</p><p>For the actual arm motion, I used <a href="https://github.com/nvidia-isaac/cumotion">cuMotion</a>, a motion planning library, instead of writing my own inverse kinematics. cuMotion plans the whole path from one pose to the next ahead of time and checks that it does not hit anything along the way, then the arm just plays that path back.</p><pre># pick_and_place.py - plan a collision free path before moving, retry a few times if none is found<br>for attempt in range(PLANNING_RETRIES):<br>    path = self.planner.plan_to_pose_target(<br>        q_initial=q_initial, position=target_position, orientation=orientation<br>    )<br>    if path is not None:<br>        break</pre><p>To pick a box, I rank them by distance to the arm’s base and drop anything farther than the arm’s reach instead of letting it try (as there’s a fun little behavior where it shuts the sim down if it can’t find a planning solution):</p><pre># conveyor_indexer.py - pick the closest reachable box, drop anything too far to reach<br>reachable = [path for path in hit_paths if _distance(path) &lt;= max_reach_m]<br>if not reachable:<br>    return None<br>return min(reachable, key=_score)</pre><p>The pick and place logic itself is its own phase machine, separate from the conveyor state machine:</p><pre># pick_and_place.py - the pick and place phases<br>class Phase(IntEnum):<br>    WAITING = 0<br>    STAGE_FOR_PICK = 1<br>    DESCEND_TO_PICK = 2<br>    ATTACH = 3<br>    LIFT_CLEAR = 4<br>    STAGE_FOR_PLACE = 5<br>    DESCEND_TO_PLACE = 6<br>    DETACH = 7</pre><ul><li>WAITING:arm stays in its default pose until the pick zone reports a box, and that box has stopped moving.</li><li>STAGE_FOR_PICK:moves to a fixed pose above the belt. This one gets used twice per cycle, once on the way down to a box, and again right after lifting one clear, since that same pose is also safe to swing from on the way to the place side.</li><li>DESCEND_TO_PICK:lowers straight down onto the box.</li><li>ATTACH:once the tool is within half a centimeter of the box, a fixed joint is created between the wrist and the box.</li><li>LIFT_CLEAR:lifts straight up until the box clears every other box still left in the pick zone, plus a small margin, since the box being carried is not tracked by cuMotion as an obstacle while it is attached to the robot.</li><li>STAGE_FOR_PLACE:moves to a staging pose above the place belt.</li><li>DESCEND_TO_PLACE: lowers to the place point.</li><li>DETACH:the fixed joint is deleted and the box falls the rest of the way onto the belt.</li></ul><p>From there it goes back through STAGE_FOR_PLACE, then WAITING, and starts over on the next box.</p><p>One problem I ran into only showed up once I added a second arm. Both arms sit on either side of the two conveyor loops and use the same kind of staging pose above their belt, but that pose can be reached by rotating the base joint two different ways, away from the middle of the two arms. By default both arms picked the arc toward each other, which risked collision between the two as I did not want to do live obstacle tracking. I fixed it by giving the second arm the same staging pose reached from the opposite direction, rotated another 180 degrees at the base joint:</p><pre># pick_and_place.py - reach the same staging pose from the other side<br>UR20_PRE_PLACE_JOINT_POSITIONS_AWAY = [<br>    UR20_DEFAULT_JOINT_POSITIONS[0] + math.pi<br>] + UR20_PRE_PLACE_JOINT_POSITIONS[1:]</pre><p><strong>Next Goals</strong></p><p>There is a lot still to do, I need to mimic the camera placement of the real world, add in random package and parcel spawning, and collect the training data. After all of that, I can finally do my actual job of testing and developing learning approaches to understand and complete this task. So stay tuned.</p><p><strong>An Update on Pushing the Big Red Button</strong></p><p>Following up from <a href="https://medium.com/@gibr6485/what-i-did-this-week-7-16-26-14cac3374d43">last week</a>. We can push the button!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F0NIQZLQSKxI%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D0NIQZLQSKxI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F0NIQZLQSKxI%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/febd53ec50a9d0886c7b166645f27493/href">https://medium.com/media/febd53ec50a9d0886c7b166645f27493/href</a></iframe><p>And we can open a container:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FHQFA5ORh8RQ%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DHQFA5ORh8RQ&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FHQFA5ORh8RQ%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/94b82f19bf0dfc013ec8f71c2547ce90/href">https://medium.com/media/94b82f19bf0dfc013ec8f71c2547ce90/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e24db5b544f1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What I did this Week: 7/16/26]]></title>
            <link>https://medium.com/@gibr6485/what-i-did-this-week-7-16-26-14cac3374d43?source=rss-172b511c53b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/14cac3374d43</guid>
            <category><![CDATA[ros2]]></category>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[nasa]]></category>
            <category><![CDATA[motion-planning]]></category>
            <dc:creator><![CDATA[Gilberto Briscoe-Martinez]]></dc:creator>
            <pubDate>Fri, 17 Jul 2026 00:24:24 GMT</pubDate>
            <atom:updated>2026-07-17T00:24:24.416Z</atom:updated>
            <content:encoded><![CDATA[<h3>Prologue</h3><p>I am starting this series to hopefully give my advisor regular weekly updates. I’m hoping the pressure of other people reading this (and I hope it’s more than just my advisor) will keep me committed to publishing these. We’ll see. For now I hope you enjoy this tale of robot misadventure.</p><p>All I want to do is to get a robot to press a button. How hard could that be? As it turns out: very. For the past couple of months I and two of my collaborators (<a href="https://yaashia-g.github.io/">Yaashia Gautam</a> and <a href="https://www.linkedin.com/in/umair14040/">Umair Yousaf</a>) have been working in the offices of the fine folks of <a href="https://picknik.ai/">PickNik</a>, using their <a href="https://github.com/NASA-JSC-Robotics/phoebe_bridgeback">Phoebe Bridgeback</a>, in collaboration with the folks in the <a href="https://www.nasa.gov/reference/jsc-robotics/">Dexterous Robotics Lab</a> at NASA Johnson Space Center. The goal is to apply Compliant Explicit Reference Governors (<a href="https://arxiv.org/pdf/2504.09188">read the paper here</a>) to complex interaction scenarios, and even more complex robots. For now, in the <a href="https://en.wikipedia.org/wiki/KISS_principle">spirit of KISS</a>, we are trying to press a big red button.</p><p>This week, something strange happened. When we were commanding simple movements, from one joint position to another, the robot would freeze then move wildly. See the video below for a tame example, there were many that were worse.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FpYbQIyYk6Os%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DpYbQIyYk6Os&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FpYbQIyYk6Os%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/7ffaf7fb4a7c60807ce19635d6b7a95e/href">https://medium.com/media/7ffaf7fb4a7c60807ce19635d6b7a95e/href</a></iframe><p>To investigate we started to search the communications link through ROS and noticed several issues:</p><ul><li>The topic /joint_states would freeze exactly every 8 seconds</li><li>Both arms would freeze for the exact same amount of time, then exhibit an erratic movement if we were commanding motion before the freeze (we recorded a 28Nm impact on one of the joints…sorry NASA)</li><li>If we attempted to listen to the topics (i.e. using ros2 topic echo ) it would cause a 3 second freeze in motion</li></ul><h3><strong>Theories we tested</strong></h3><p><em>CPU Starvation</em></p><p>We initially noticed that CPU usage was pinned at 100%. Our instinct was that this caused communication drops due to processes queing up. The computer running the Phoebe is quite small, a 4 core, 8-thread i3–9100TE.</p><p>We implemented single threading on our code, which dropped usage from &gt;95% to ~60%, but the issue remained.</p><pre># cerg_pd_demo_dual_hw.launch.py — pin the Python control nodes&#39; math to 1 thread<br>single_thread_env = {<br>    &quot;OMP_NUM_THREADS&quot;: &quot;1&quot;,<br>    &quot;OPENBLAS_NUM_THREADS&quot;: &quot;1&quot;,<br>    &quot;MKL_NUM_THREADS&quot;: &quot;1&quot;,<br>}<br>Node(..., additional_env=single_thread_env, ...)</pre><p><em>Python Garbage Collection</em></p><p>Since the freezes happened every 8 seconds, we had the idea that it may be garbage collection. We added controlled gc.freeze() and gc.disable() code snippets, but found that C++ processes also had the freezes, so we moved on.</p><p><em>Reliable-QoS Writer Blocking</em></p><p>We conducted a synthetic experiment of pub/sub. What we found that our subscriber to \joint_states was treating the topic as a <a href="https://cyclonedds.io/docs/cyclonedds/latest/about_dds/reliable_communication.html">reliable publisher</a>. However, it was actually set to be a best effort publisher; where if there was some loss in packets, this is acceptable.</p><pre>// robot_state_publisher.cpp (Humble) — NOT a reliable reader<br>joint_state_sub_ = this-&gt;create_subscription&lt;sensor_msgs::msg::JointState&gt;(<br>    &quot;joint_states&quot;,<br>    rclcpp::SensorDataQoS(),   // ← BEST_EFFORT<br>    ...);</pre><p>When a packet was lost, this would block communication for 3 seconds while the subscriber waited in vain for the confirmation of a packet that wasn’t properly received. When we set the subscriber to best-effort listening, this problem was dropped. However, we still had the 8 second gaps.</p><pre># cerg_node.py — passive observer must never back-pressure the torque writer<br>self.create_subscription(<br>    Float64MultiArray,<br>    f&quot;/{arm}_forward_effort_controller/commands&quot;,<br>    self._on_commands,<br>    qos_profile_sensor_data,   # was: 10 (= RELIABLE depth 10)<br>)</pre><p><em>Listen to Everything</em></p><p>What finally lead to us cracking this case was simply listening to everything. We setup an audit script to subscribe to any topic that was published while we ran the movement on the robot.</p><pre>self.create_subscription(msg_type, topic, self._cb, qos_profile_sensor_data)<br><br>def _cb(self, _msg):<br>    now = time.monotonic()<br>    gap = now - self.last_mono<br>    if gap * 1e3 &gt; GAP_THRESH_MS:<br>        print(f&quot;{wallclock()} [GAP] {self.topic}: {gap*1e3:.0f} ms hole &quot;<br>              f&quot;(started {wallclock(time.time() - gap)})&quot;)</pre><p>Further we would watch for communication drops across these topics and report when they would freeze.</p><pre>def _watchdog(self):<br>    silent = time.monotonic() - js_watch.last_mono<br>    if silent &gt; 0.3 and not js_watch.hole_snapped:<br>        js_watch.hole_snapped = True<br>        print(f&quot;[HOLE] /joint_states silent {silent*1e3:.0f} ms and counting &quot;<br>              f&quot;— live snapshot: [{psi_snapshot()}] [{cm_thread_snapshot()}]&quot;)<br><br># cm_thread_snapshot →  ps -L -o stat=,wchan= -p &lt;arm ros2_control_node pid&gt;</pre><p>Here is where we found it: sock_alloc_send_psk.</p><pre>[HOLE] /joint_states silent 339 ms and counting — live snapshot:<br>       [cpu:avg10=23.61 io:avg10=0.00]<br>       [cm ... S/sock_alloc_send_pskb:2 ...]     ← two threads blocked HERE</pre><p>sock_alloc_send_pskb is where a UDP sender sleeps because <strong>its socket send buffer is full</strong>. The publisher wasn&#39;t crashed, preempted, or garbage collecting : it was blocked writing into a jammed socket. (I won’t lie, if it wasn’t for Claude, I too wouldn’t know what that was). We found that this message was was popping up exactly every 8 seconds, and was tied to the communication freeze.</p><h3><strong>The Mechanism Behind This Failure</strong></h3><p>Our DDS was configured to have multicast turned off, and there were peers listed in ~/.ros/cyclonedds.xml that no longer existed on the network. So, without multicast each program would do its <a href="https://cyclonedds.io/docs/cyclonedds/latest/config/discovery-behavior.html">discovery announcement </a><strong>every </strong>peer listed. The announcements would repeat forever, as they also function as a heartbeat signal. To keep from being declared dead, each program re-announces comfortably before its expiry runs out, which works out to about every 8 seconds. Since there were peers that no longer existed on the network, each process would have to wait for the Address Resolution Protocol (ARP) to timeout, which takes about 3 seconds. Here we get to the issue: the packets would queue while all the processes waited for ARP to timeout. As soon as we deleted the addressed that were no longer on the network, this issue disappeared. <strong>Commenting out two lines fixed the issue. </strong>Sometimes it really is the simplest fixes to the most complex problems.</p><h3>Lessons Learned</h3><p>A few things stuck with us after this one. First, when you turn off the easy way for programs to find each other on a network and give them a fixed list of addresses instead, that list becomes a promise, and if even one address on it goes stale, every program on the machine ends up quietly jammed up waiting on it, over and over, forever. Second, a good guess that happens to match the timing of the bug isn’t the same as the actual cause; we had the <em>Reliable-QoS Writer Blocking </em>theory that lined up almost perfectly and still turned out to be a coincidence, and the only thing that killed it was going and reading the actual code instead of trusting the clock. Third, if you want to catch something intermittent, you have to look at the system in the act, not afterward. Everything looked completely normal once the freeze was over, and it was only a snapshot taken in the middle of the failure that actually caught the issue. And finally, anything whose only job is to watch and report should be built so it can never itself get stuck or clog the works. A monitor that can jam the system it’s monitoring has failed at the one job it had.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tpz2TCSevp_6sOPCw3C5Gw.jpeg" /><figcaption>Hopefully soon we will be pressing these buttons (and doing much cooler things too!)</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=14cac3374d43" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Proximity Sensor Transformer]]></title>
            <link>https://medium.com/correll-lab/proximity-sensor-transformer-6c8128a3824b?source=rss-172b511c53b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/6c8128a3824b</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[transformers]]></category>
            <dc:creator><![CDATA[Gilberto Briscoe-Martinez]]></dc:creator>
            <pubDate>Thu, 16 Oct 2025 23:32:56 GMT</pubDate>
            <atom:updated>2025-10-16T23:32:56.797Z</atom:updated>
            <content:encoded><![CDATA[<h4>Localizing touch in robotic skins using the transformer architecture</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YVDB--tg_NZjKbO5.jpeg" /><figcaption><em>Fig. 1 GenTact generated, 3D Printed proximity sensors for the Franka Emika FR3 robot arm.</em></figcaption></figure><p>In the <a href="https://hiro-group.ronc.one/">HIRO Group</a>, we are actively exploring additional senses to expand robot ability. We are particularly excited about the <a href="https://www.ncbi.nlm.nih.gov/books/NBK92879/">peripersonal space</a> of robots. Unlocking a robot’s ability to sense the area around them using their whole body will enable many new skills in human-robot interaction, vision-denied environments, and whole-body interactions. My colleague <a href="https://ckohl10.github.io/">Carson Kohlbrenner</a> has developed <a href="https://hiro-group.ronc.one/gentacttoolbox/"><strong><em>GenTact</em></strong></a>, a generative approach to create 3d-printed, whole-body capacitive sensors. These sensors detect conductive objects, such as humans.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PW6I7tE5mp1ulvam-CLwGQ.png" /><figcaption>Transformer Encoder System Diagram for Predicting Sensed Object Position</figcaption></figure><p>For my class with Dr. Nikolaus Correll, we were tasked with developing a transformer from scratch on any data. Thanks to Carson’s work, I used some calibration run data to teach a transformer to predict the 3D position of the sensed object using the sensor in the first figure on the left. I used a transformer encoder-only architecture to go from the time series sequence of raw sensor input to the fixed 3D vector of sensed object position relative to the sensor.</p><blockquote>This article was originally published on <a href="https://gbmartinez.substack.com/?utm_campaign=profile_chips">Gilberto’s Substack</a>!</blockquote><h3>Results</h3><p>To analyze the quality of the transformer’s prediction, I focused on the error between the predicted position and the ground truth, measured using forward kinematics of the robot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ML53MmqrZ36FtnQH.png" /><figcaption>A diagram of predicted positions for a given input. The dataset consisted of multiple static positions of the sensed object. Here we see that there is some stochasticity in the predicted position given multiple readings of the sensor. This is likely mainly due to noise in the sensor.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lKFDQtmV6PvYL4GJ.png" /><figcaption>Graph of prediction error over each axis. We’re interested in the bias that the transformer introduces compared to the ground truth. We will need to further explore the consistent positive z-bias.</figcaption></figure><p>The transformer performed quite well in the X and Y axis error: within an error of 6 mm in ax and 10 mm in y (with some long tail outliers). However in Z, there is a consistent positive bias of up to 25 mm. Further investigation is needed into the cause of this error, but the results overall show great promise of the use of the transformer encoder in future use cases.</p><h3>Code</h3><p>All Code is available here: <a href="https://github.com/ggb367/proximity-transformer">https://github.com/ggb367/proximity-transformer</a></p><h3>Data Cleaning</h3><p>See <a href="https://github.com/ggb367/proximity-transformer/blob/master/HW1-transformer/scripts/data_cleaning.py">data_cleaning.py</a></p><p>The goal of this code is to take bagged ROS2 data and turn it into a usable dataset. To do this, we need to do a bit of processing on the saved code. I make a key assumption that the conductive, capacitive sphere sensed at the object is located where panda_link8 is in the kinematic chain of the URDF.</p><h4>Code Breakdown:</h4><p>We loop over all of the saved mcap bag files in the system:</p><pre>for mcap_path in mcap_paths:</pre><p>We then read the mcap and listen to only two topics, joint_states and sensor_raw:</p><pre>for msg in iter_ros2_messages_decoded(mcap_path, topics=[”/joint_states”, “/sensor_raw”])</pre><p>To iterate over the MCAP I use the mcap ros2 library:<br>This code simply makes a MCAP ROS2 decoder factory and loops over all of the messages in the MCAP bag. it then returns only the selected topics</p><pre>def iter_ros2_messages_decoded(mcap_path: str, topics: list[str]):<br>    “”“<br>    Yield SimpleNamespace(topic, ros_msg, log_time) for decoded ROS 2 messages.</pre><pre>    Per mcap.reader API, supply the ROS2 CDR decoder via make_reader(..., decoder_factories=[...])<br>    and then call iter_decoded_messages() with no decoder args.<br>    “”“<br>    with open(mcap_path, “rb”) as f:<br>        rdr = make_reader(f, decoder_factories=[Ros2DecoderFactory()])<br>        for schema, channel, message, ros2_msg in rdr.iter_decoded_messages(topics=topics):<br>            yield SimpleNamespace(<br>                topic=channel.topic,<br>                ros_msg=ros2_msg,<br>                log_time=message.log_time,<br>            )</pre><p>Sensor_Data</p><p>Read it</p><pre>def decode_sensor_raw(msg) -&gt; Tuple[int, np.ndarray]:<br>    “”“<br>    Returns (timestamp_ns, np.int32[N]) for std_msgs/msg/Int32MultiArray<br>    “”“<br>    t = msg.log_time<br>    m = msg.ros_msg<br>    arr = np.asarray(m.data, dtype=np.int32)<br>    return t, arr</pre><p>Downsample the read rate</p><pre># Downsample /sensor_raw by stride to ~target_hz<br>        if args.target_hz &gt; 0 and len(sr_times) &gt; 1:<br>            duration_s = (sr_times[-1] - sr_times[0]) * 1e-9<br>            approx_rate = len(sr_times) / max(duration_s, 1e-9)<br>            stride = max(1, int(round(approx_rate / args.target_hz)))<br>        else:<br>            stride = 1</pre><p>Joint_states</p><p>We need to downsample the rate and do forward kinematics to find the point that the sensors are sensing</p><pre>for i in range(0, len(sr_times), stride):<br>            t = sr_times[i]<br>            s = sr_vals[i]<br>            j_idx = nearest_time_index(t, js_times)<br>            if abs(int(t) - int(js_times[j_idx])) &gt; max_delta_ns:<br>                continue  # no nearby joint_state for alignment</pre><pre>            ee = fk.fk_xyz(js_vecs[j_idx]).astype(np.float32)  # [x,y,z]<br>            buf_ts.append(int(t))<br>            buf_s.append(s)<br>            buf_ee.append(ee)</pre><p>We then save all of the data</p><pre>if len(buf_ts) &gt;= shard_rows:<br>                flush_shard()</pre><h3>The Transformer</h3><h3>Positional Encoder</h3><p>See: <a href="https://github.com/ggb367/proximity-transformer/blob/master/HW1-transformer/src/models/embeddings.py">embeddings.py</a></p><p>We utilize a simple sinusoidal positional embedding</p><pre>import torch<br>import torch.nn as nn<br>import math</pre><pre>class PositionalEncoding(nn.Module):<br>    “”“<br>    Classic sinusoidal positional encoding (precomputed).<br>    Adds PE to input embeddings.<br>    “”“<br>    def __init__(self, d_model: int, max_len: int = 10_000, dropout: float = 0.0):<br>        super().__init__()<br>        pe = torch.zeros(max_len, d_model)  # [L, D]<br>        position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)<br>        div_term = torch.exp(<br>            torch.arange(0, d_model, 2, dtype=torch.float) * (-math.log(10_000.0) / d_model)<br>        )<br>        pe[:, 0::2] = torch.sin(position * div_term)<br>        pe[:, 1::2] = torch.cos(position * div_term)<br>        self.register_buffer(”pe”, pe[None, ...])  # [1, L, D]<br>        self.dropout = nn.Dropout(dropout)</pre><pre>    def forward(self, x: torch.Tensor) -&gt; torch.Tensor:<br>        # x: [B, L, D]<br>        L = x.size(1)<br>        x = x + self.pe[:, :L, :]<br>        return self.dropout(x)</pre><h3>What the Model Does (at a glance)</h3><ul><li><strong>Input:</strong> padded sensor sequences sensor_pad of shape [B, L] with per-example true lengths lengths of shape [B].</li><li><strong>Encoder:</strong> projects scalar samples → d_model features, adds <strong>sinusoidal positional encodings</strong>, passes through N<strong>encoder layers</strong> (self-attention + feed-forward) with <strong>pre-norm residuals</strong>.</li><li><strong>Pooling:</strong> <strong>masked mean</strong> across time to get a single vector per example.</li><li><strong>Head:</strong> a small MLP regresses to <strong>3D position</strong> ([B, 3]).</li><li><strong>Masking:</strong> a boolean mask marks <strong>PAD</strong> positions so attention and pooling ignore them.</li></ul><h3>Padding Mask</h3><pre>def make_length_mask(lengths: torch.Tensor, L: int) -&gt; torch.Tensor:<br>    # lengths: [B], L: max length<br>    # returns mask: [B, 1, 1, L] with True for PAD</pre><ul><li>Builds a boolean tensor where <strong>True = PAD</strong>.</li><li>Broadcastable to attention scores shaped [B, H, L_q, L_k].</li><li>Used in both attention and mean-pooling to <strong>exclude</strong> padded timesteps.</li></ul><h3>The Transformer Module</h3><p>See <a href="https://github.com/ggb367/proximity-transformer/blob/master/HW1-transformer/src/models/transformer.py">transformers.py</a></p><pre>class Transformer(nn.Module):<br>    “”“<br>    Maps padded sensor sequences [B, L] -&gt; [B, 3].<br>    Steps:<br>      1) scale raw ints<br>      2) linear proj to d_model<br>      3) add sinusoidal PE<br>      4) N encoder layers (self-attn + FFN, pre-norm)<br>      5) masked mean pool over time<br>      6) MLP head to 3D<br>    “”“</pre><h3>Constructor (__init__)</h3><ul><li><strong>Hyperparams:</strong> d_model=256, num_layers=4, num_heads=8, d_ff=1024, dropout=0.1, activation=”gelu”.</li><li><strong>Input scaling:</strong> in_scale multiplies raw readings before projection (simple global scale).</li><li>input_proj<strong>:</strong> nn.Linear(1, d_model) converts a scalar reading per timestep into a d_model-dim vector.</li><li>posenc<strong>:</strong> adds positional encoding so the model knows <strong>where in time</strong> each reading occurs.</li><li>layers<strong>:</strong> num_layers instances of EncoderLayer.</li><li>enc_norm<strong>:</strong> final LayerNorm over the encoded sequence.</li><li>head_global<strong>:</strong> LayerNorm → Linear → GELU → Dropout → Linear to regress [B, 3].</li></ul><h3>Init (_reset_parameters)</h3><ul><li>All nn.Linear weights use <strong>Xavier uniform</strong>, biases initialized to <strong>zeros</strong>.</li></ul><h3>Forward (forward)</h3><p>Inputs:</p><ul><li>sensor_pad: [B, L] (ints/floats, padded to same L)</li><li>lengths: [B] true lengths</li></ul><p>Steps:</p><ol><li><strong>Scale &amp; reshape:</strong><br>x = sensor_pad.float() * in_scale → [B, L], then unsqueeze(-1) → [B, L, 1].</li><li><strong>Project to features:</strong><br>input_proj → [B, L, D].</li><li><strong>Add PE:</strong><br>posenc(x) still [B, L, D].</li><li><strong>Build mask:</strong><br>mask = make_length_mask(lengths, L) → [B, 1, 1, L] (True = PAD).</li><li><strong>Encode:</strong><br>for each EncoderLayer: x = layer(x, src_mask=mask) → [B, L, D].</li><li><strong>Normalize:</strong><br>enc_norm(x) → [B, L, D].</li><li><strong>Masked mean pooling:</strong></li></ol><ul><li>valid = (~mask.squeeze(1).squeeze(1)).float() → [B, L] with 1 for keep, 0 for pad</li><li>pooled = (x * valid[..., None]).sum(dim=1) / clamp(valid.sum(1), min=1) → [B, D]</li></ul><ol><li><strong>Head:</strong><br>global_pred = head_global(pooled) → [B, 3]</li></ol><p>Output:</p><ul><li>pred: [B, 3] — the estimated 3D location.</li></ul><h3>Inside an Encoder Layer</h3><p>See <a href="https://github.com/ggb367/proximity-transformer/blob/master/HW1-transformer/src/models/encoder.py">encoder.py</a></p><pre>class EncoderLayer(nn.Module):<br>    # self_attn (MultiHeadAttention) + ffn (PositionwiseFFN)<br>    # both wrapped by PreNormResidual</pre><p>Forward path:</p><ol><li><strong>Block 1:</strong> x = x + Dropout(SelfAttn(LayerNorm(x)))</li><li><strong>Block 2:</strong> x = x + Dropout(FFN(LayerNorm(x)))</li></ol><ul><li><strong>Pre-norm</strong> (LayerNorm before sublayer) stabilizes training in deeper stacks.</li><li><strong>Self-attention</strong> lets each timestep <strong>see</strong> weighted information from all other (non-padded) timesteps.</li><li><strong>FFN</strong> refines features <strong>per timestep</strong> (no cross-time mixing).</li></ul><h3>Multi-Head Attention</h3><p>See <a href="https://github.com/ggb367/proximity-transformer/blob/master/HW1-transformer/src/models/heads.py">heads.py</a></p><pre>class MultiHeadAttention(nn.Module):<br>    # W_q, W_k, W_v, W_o: linear projections<br>    # d_model divisible by num_heads</pre><ul><li>Projects q, k, v to H heads of size d_k = d_model / H.</li><li><strong>Scores:</strong> scores = (QKᵀ) / sqrt(d_k) → [B, H, Lq, Lk]</li><li><strong>Masking:</strong> scores.masked_fill(attn_mask, -inf) where <strong>True = PAD</strong>.</li><li><strong>Softmax + Dropout</strong> over Lk.</li><li><strong>Context:</strong> attn @ V → merge heads → W_o → [B, Lq, D]</li></ul><p>Here, <strong>self-attention</strong> means q=k=v=x.</p><h3>Position-Wise Feed-Forward (FFN)</h3><pre>class PositionwiseFFN(nn.Module):<br>    # Linear(d_model -&gt; d_ff) -&gt; activation -&gt; Dropout -&gt; Linear(d_ff -&gt; d_model)</pre><ul><li>Applied at <strong>each timestep independently</strong>.</li><li>Activation is <strong>ReLU</strong> or <strong>GELU</strong> (selectable).</li><li>Expands to d_ff, contracts back to d_model for richer nonlinearity.</li></ul><h3>Pre-Norm Residual</h3><pre>class PreNormResidual(nn.Module):<br>    # y = x + Dropout(sublayer(LayerNorm(x)))</pre><ul><li>Improves gradient flow and training stability.</li><li>Keeps the residual path <strong>clean</strong> while normalizing the sublayer input.</li></ul><h3>Training</h3><ul><li><strong>Objective:</strong> regress [x, y, z] from a variable-length proximity sequence.</li><li><strong>Loss:</strong> Smooth L1 (Huber) with manual masking (pads don’t contribute).</li><li><strong>Speed/Stability:</strong> optional <strong>AMP</strong> (mixed precision), <strong>grad clipping</strong>, <strong>AdamW</strong>.</li><li><strong>Data:</strong> Parquet → iterable dataset → padded batches via pad_collate.</li><li><strong>Eval:</strong> per-epoch validation with masked metrics; best checkpoint auto-saved.</li></ul><h3>Trainer Overview</h3><p>See <a href="https://github.com/ggb367/proximity-transformer/blob/master/HW1-transformer/src/trainers/trainer.py">trainer.py</a></p><pre>class Trainer:<br>    def __init__(<br>        model, optimizer, train_loader, val_loader=None, device=”cuda”,<br>        amp=True, grad_clip=1.0, ckpt_dir=None, scheduler=None,<br>        log_every=100, huber_beta=0.01<br>    )</pre><h3>Key responsibilities</h3><ul><li><strong>Device + AMP:</strong> moves model to device; enables <strong>torch.amp</strong> if CUDA is available.</li><li><strong>Criterion:</strong> SmoothL1Loss(beta=huber_beta, reduction=”none”); we apply masking and averaging <strong>manually</strong>.</li><li><strong>Optimization:</strong> integrates optimizer, optional scheduler, and <strong>gradient clipping</strong>.</li><li><strong>Checkpointing:</strong> saves model/optimizer/scaler state; tracks <strong>best</strong> validation loss.</li></ul><h3>Padding &amp; Masks</h3><h3>Length→mask (boolean)</h3><pre>@staticmethod<br>def _build_mask_from_lengths(lengths, L) -&gt; Bool[B, L]:<br>    idx = torch.arange(L)[None, :]<br>    return idx &lt; lengths[:, None]  # True = valid, False = pad</pre><ul><li>This mask is used to <strong>exclude padded positions</strong> from <strong>loss</strong> and <strong>metrics</strong>.</li><li>Works for both <strong>seq→vec</strong> [B,3] and <strong>seq→seq</strong> [B,L,3] training.</li></ul><h3>Loss Computation</h3><pre>def _compute_loss(pred, target, lengths) -&gt; (loss_scalar, valid_count)</pre><ul><li><strong>Seq→vec</strong> (pred: [B,3]): SmoothL1 on all elements, mean over B*3.</li><li>Build valid_mask: [B,L,1].</li><li>Average over <strong>valid</strong> elements only (#valid_positions * 3).</li></ul><p><strong>Why Smooth L1 (Huber)?</strong> It is <strong>robust</strong> to occasional spikes/noise in raw sensor targets and often trains more stably than pure L2 for proximity data. The beta=0.01 sets the quadratic→linear transition.</p><h3>Training Loop</h3><p>see <a href="https://github.com/ggb367/proximity-transformer/blob/master/HW1-transformer/scripts/train.py">train.py</a></p><pre>def fit(self, epochs: int, start_step: int = 0):<br>    for epoch in 1..epochs:<br>        model.train()<br>        for (sensor_pad, ee, _, lengths) in train_loader:<br>            # 1) Move to device<br>            # 2) Forward under autocast (if amp)<br>            # 3) Compute masked SmoothL1 loss<br>            # 4) Backprop (scaled if amp), grad clip, optimizer step<br>            # 5) Scheduler step (optional)<br>        # end epoch: evaluate() + save checkpoints (best + latest)</pre><h3>Stability &amp; speed details</h3><ul><li><strong>AMP:</strong> with autocast(’cuda’, enabled=self.amp) reduces memory and can accelerate training.</li><li><strong>Grad clipping:</strong> clip_grad_norm_(..., grad_clip) avoids exploding gradients (defaults to <strong>1.0</strong>).</li><li><strong>Zeroing:</strong> optimizer.zero_grad(set_to_none=True) for faster clears.</li><li><strong>AdamW:</strong> decoupled weight decay is typically better for Transformers on regression.</li></ul><h3>Evaluation</h3><p>See <a href="https://github.com/ggb367/proximity-transformer/blob/master/HW1-transformer/scripts/evaluate.py">evaluate.py</a></p><pre>def evaluate(self):<br>    model.eval()<br>    with torch.no_grad():<br>        for batch in val_loader:<br>            pred = model(sensor_pad, lengths)<br>            loss, valid_count = _compute_loss(pred, ee, lengths)<br>            # Aggregate weighted loss across batches<br>            # Compute MAE with the same mask logic<br>    return {”val_loss”: weighted_mean, “val_mae”: masked_mae}</pre><ul><li><strong>Validation loss</strong> and <strong>MAE</strong> are computed <strong>only over valid timesteps</strong> for seq→seq.</li><li>If the model outputs [B,3] (seq→vec), metrics reduce over B*3.</li></ul><h3>Checkpointing</h3><pre>def save(self, step, best=False):<br>    torch.save({<br>        “model”: model.state_dict(),<br>        “optim”: optimizer.state_dict(),<br>        “scaler”: scaler.state_dict() if amp else None,<br>        “step”: step,<br>    }, ckpt_dir / (”ckpt_best.pt” if best else f”ckpt_step_{step}.pt”))</pre><ul><li>After each epoch, evaluate() runs; if val_loss improves, ckpt_best.pt is updated.</li><li>A regular step checkpoint is also saved for recovery.</li></ul><h3>Dataset &amp; Loaders</h3><h3>Row-range splitting (deterministic)</h3><pre>class EEDatasetSplit(EEDataset):<br>    def __init__(..., start_row=0, end_row=2**63-1, ...)</pre><pre>    def _iter_file(self, file_path):<br>        # Reads Parquet in record batches, slices by absolute row index<br>        # Yields one example per row: (sensor_np[i], ee_np[i], timestamp)</pre><ul><li><strong>Why row slicing?</strong> Create <strong>train/val splits</strong> by row intervals <strong>without shuffling</strong> files.</li><li><strong>Parquet reading:</strong> batches are iterated with pyarrow, then sub-sliced to [start_row, end_row).</li></ul><h3>Dataloaders</h3><p>See: <a href="https://github.com/ggb367/proximity-transformer/blob/master/HW1-transformer/src/datasets/dataset.py">dataset.py</a></p><pre>train_dl = DataLoader(<br>    train_ds, batch_size=..., num_workers=..., collate_fn=pad_collate, pin_memory=True<br>)<br>val_dl = DataLoader(<br>    val_ds, batch_size=..., num_workers=0, collate_fn=pad_collate, pin_memory=True<br>)</pre><ul><li>pad_collate pads variable-length sequences to the batch max length and returns sensor_pad, ee, ..., lengths.</li><li>Keep num_workers=0 for the validation loader here (iterable dataset safety).</li></ul><h3>Shapes: End-to-End Recap</h3><ul><li><strong>Batch in:</strong></li><li>sensor_pad: [B, L] (padded)</li><li>lengths: [B] (true lengths)</li><li>ee: [B, 3] (seq→vec)</li><li><strong>Model out:</strong></li><li>seq→vec: pred: [B, 3]</li><li><strong>Mask:</strong> _build_mask_from_lengths(lengths, L) → [B, L] → expands to [B, L, 1] for elementwise ops.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6c8128a3824b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/correll-lab/proximity-sensor-transformer-6c8128a3824b">Proximity Sensor Transformer</a> was originally published in <a href="https://medium.com/correll-lab">Toward Humanoids</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>