<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Joshua Dakim on Medium]]></title>
        <description><![CDATA[Stories by Joshua Dakim on Medium]]></description>
        <link>https://medium.com/@joshuasolomon_27618?source=rss-3108d65c9b08------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ZES0_NUGZ8u96vPQMd_4FA.jpeg</url>
            <title>Stories by Joshua Dakim on Medium</title>
            <link>https://medium.com/@joshuasolomon_27618?source=rss-3108d65c9b08------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 16:40:22 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@joshuasolomon_27618/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Teaching a Neural Network to See Inside a Zebrafish Brain]]></title>
            <link>https://medium.com/@joshuasolomon_27618/teaching-a-neural-network-to-see-inside-a-zebrafish-brain-5e7c5e573ca6?source=rss-3108d65c9b08------2</link>
            <guid isPermaLink="false">https://medium.com/p/5e7c5e573ca6</guid>
            <dc:creator><![CDATA[Joshua Dakim]]></dc:creator>
            <pubDate>Mon, 23 Mar 2026 20:46:54 GMT</pubDate>
            <atom:updated>2026-03-24T00:25:25.363Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/956/1*tSjdXiF5YoJHpbMtIykz-w.png" /></figure><p>How I built a 3D nucleus segmentation pipeline using U-Net, MONAI, and PyTorch, and hit a Dice score of 0.88 on a free GPU.</p><h3>Introduction</h3><p>Imagine you’re a neuroscientist trying to map the brain of a zebrafish, a tiny vertebrate whose brain is small enough to image completely, yet complex enough to model how vertebrate neural development works. Now imagine that your microscope has just produced a 3D volumetric scan containing thousands of individual cell nuclei, each one a small glowing blob sitting inside a dark sea of extracellular space.</p><p>Your task: identify and delineate every single nucleus in that volume. Manually.</p><p>That’s not just tedious; at scale, it’s effectively impossible. And yet this kind of annotation is exactly what neuroscientists need to track cell division, study gene expression patterns, and understand how the brain is wired together.</p><p>This is the problem I set out to automate. Using the <strong>NucMM-Z dataset</strong> (Nuclear Morphology and Motility, Zebrafish subset), I built a full 3D binary segmentation pipeline that can look at a volumetric microscopy image and predict, voxel by voxel, where the nuclei are. The final model achieved a <strong>validation Dice score of 0.8824, </strong>well above the 0.7 threshold considered clinically useful, running entirely on a free Google Colab T4 GPU.</p><h3>Objective</h3><p>The goal was straightforward in concept, tricky in execution: given a 3D grayscale microscopy volume, predict a binary mask where every voxel is classified as either <em>nucleus</em> or <em>background</em>. No instance tracking, no cell lineage, just a clean, accurate segmentation that a downstream analysis pipeline could actually use.</p><h3>Approach</h3><h3>The Data</h3><p>The NucMM-Z dataset stores its volumes in HDF5 format (.h5), each patch a 64×64×64 voxel cube cropped from a larger whole-brain scan. The filename itself encodes where each patch came from in the original volume, the three numbers in img_0000_0576_0768.h5 are the Z, Y, and X coordinates of the crop origin.</p><p>I started with 27 training volumes and 27 validation volumes. Before writing a single line of model code, I spent time just understanding the data. The raw intensities are uint8 values (0–255), nuclei show up as bright regions, and, crucially, they only occupy about <strong>4.74% of the total voxel volume</strong>. That last number matters a lot for how you design the loss function.</p><p>The labels in NucMM-Z are <em>instance labels</em>, meaning each nucleus gets a unique integer ID. Since I only needed to know whether a voxel was a nucleus or not, I converted all non-zero values to 1 before any training.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/1*Xnsfvmf7NVcT7WUjnsAJ7w.png" /></figure><h3>Handling 3D Data</h3><p>Working in 3D introduces challenges that don’t exist in standard 2D image classification. Each .h5 file is a volumetric grid, think of it as a stack of CT slices, except acquired via electron or fluorescence microscopy. The imaging physics adds its own wrinkles: the microscope&#39;s point spread function blurs signals (especially along the depth axis), shot noise introduces random intensity fluctuations, and the sheer memory cost of loading full 3D volumes forces you to work with patches rather than entire scans.</p><p>I built a custom PyTorch Dataset class that loads volumes from disk on demand, adds a channel dimension to match PyTorch&#39;s expected (C, D, H, W) format, and binarises the labels at load time. Keeping loading lazy (on-the-fly rather than preloading everything) kept RAM usage manageable throughout training.</p><h3>Transforms and Augmentation</h3><p>This is where <strong>MONAI</strong> (Medical Open Network for AI) really earned its place in the stack. MONAI is a PyTorch-based framework designed specifically for medical imaging, and its transform library saved me from writing a lot of boilerplate.</p><p>For training, I applied a pipeline of: intensity normalisation (zero mean, unit standard deviation), random 6⁴³ spatial crops, random flips along all three axes, random 90° rotations, and small random intensity scaling and shifts. These weren’t arbitrary choices; they reflect the biology. Nuclei have no preferred orientation in 3D space, so flips and rotations are legitimate augmentations. Variable staining intensity is a real imaging artefact, so simulating it during training makes the model more robust.</p><p>Validation transforms applied only deterministic preprocessing, no augmentation, to ensure fair and repeatable evaluation.</p><h3>Architecture: 3D U-Net</h3><p>The U-Net architecture has become the standard for biomedical image segmentation for good reason. The encoder progressively downsamples the volume while increasing feature depth (16 → 32 → 64 → 128 → 256 filters across five levels), and the decoder mirrors this process in reverse. The key ingredient is the <strong>skip connections</strong>: feature maps from each encoder level are concatenated directly to the corresponding decoder level, giving the network access to fine spatial detail that would otherwise be lost during downsampling.</p><p>I used MONAI’s built-in UNet with spatial_dims=3 to get full 3D convolutions throughout, num_res_units=2 for residual shortcuts within each level, and instance normalisation. Total trainable parameters: about <strong>4.8 million, </strong>substantial, but well within what a T4 GPU can handle with 64³ patches at batch size 2.</p><h3>Loss Function</h3><p>The choice of loss function was non-negotiable given the class imbalance. With nuclei making up less than 5% of the volume, a model trained with standard cross-entropy could achieve ~95% pixel accuracy by simply predicting “background” everywhere. That’s useless.</p><p><strong>Dice Loss</strong> directly optimises the overlap between predicted and ground-truth masks, so the model is penalised equally for missing nuclei as for over-predicting them. I paired it with the <strong>Adam optimiser</strong> (initial learning rate 1e-3, weight decay 1e-5) and a ReduceLROnPlateau scheduler that halved the learning rate whenever validation loss stopped improving for five consecutive epochs.</p><h3>Training</h3><p>Training ran for 50 epochs with validation every two epochs, taking roughly 20 minutes on the T4. The learning curve told an interesting story. Dice score climbed steadily through the first 26 epochs, reaching 0.8677 by epoch 26, then hovered and improved slowly in the final stretch. One thing I noticed was that the scheduler’s learning rate reduction kicked in around epoch 40, which coincided with the model pushing past 0.87, suggesting the coarser early convergence had genuinely squeezed out what it could and needed smaller steps to refine the boundary predictions.</p><p>The best checkpoint was saved at epoch 42 with a validation Dice of <strong>0.8824</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1p8ody1tVfUAUwvIAiHxYA.png" /></figure><h3>Results</h3><p>Running the best model against the full validation set gave:</p><ul><li><strong>Mean Dice: 0.8813</strong></li><li><strong>Std: 0.0165</strong></li><li><strong>Min: 0.8500 / Max: 0.9119</strong></li></ul><p>The low standard deviation is notable, it means the model isn’t just performing well on easy volumes and struggling on hard ones. The performance is consistent across the different spatial regions of the zebrafish brain that the patches were drawn from.</p><p>Qualitatively, the predictions looked clean. Nucleus boundaries were sharp, small nuclei were detected, and false positives in background regions were rare. Looking at multi-slice strips through a single validation volume, the segmentation held up consistently across the Z-axis depth, not just on the “middle” slices where nuclei are most visible.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/556/1*KVQcGL0ztfZtEG7zdc3ZYA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/553/1*epVSbEBFSLQdiH1A6r_Gug.png" /></figure><h3>Discussion</h3><p>The result is genuinely solid for this problem. A Dice score above 0.88 on a dataset with ~5% foreground, trained from scratch with 27 volumes and no pretrained weights, demonstrates that the 3D U-Net + Dice Loss combination is robust enough to learn meaningful nucleus representations even from limited data.</p><p>That said, I’d be cautious about over-interpreting the numbers. The validation set is drawn from the same brain region and acquisition conditions as the training set. How well this model transfers to zebrafish brains imaged under different conditions, or to the mouse cortex volumes in the full NucMM dataset, is an open question.</p><p>The augmentation strategy also made a bigger difference than I initially expected. Early experiments without intensity augmentation plateaued around 0.82, adding random scaling and shifting pushed past that plateau, which makes sense given how sensitive microscopy data can be to staining variability.</p><h3>Limitations</h3><p>The most honest limitation is dataset size. 27 training volumes is small. The model compensates with aggressive augmentation, but there’s a ceiling on what that can achieve. Acquiring or simulating more diverse training data would likely push performance further.</p><p>The patch-based approach also has a subtle downside: because training samples are 6⁴³ crops, the model has no global context about where in the brain it is. A nucleus near the edge of a volume looks identical to one in the center as far as the model is concerned. Sliding window inference at test time partially addresses this, but it’s not in scope here.</p><p>Finally, the pipeline outputs a binary mask, it doesn’t separate individual nuclei. For downstream analyses that require instance-level information (counting nuclei, tracking divisions), a post-processing step like watershed would be needed.</p><h3>Applications</h3><p>Automated nucleus segmentation like this feeds directly into neuroscience workflows: counting cells in a region, measuring nuclear morphology across experimental conditions, or seeding instance segmentation pipelines for full cell tracking. The same architecture generalises well to other 3D biomedical volumes, organoid imaging, cardiac MRI, or any task where a small object needs to be found inside a large, mostly-empty 3D space.</p><p>The most natural extension here would be sliding window inference over full-resolution volumes using MONAI’s sliding_window_inference, followed by watershed post-processing to recover individual nuclei from the binary mask. From there, connecting this to a cell tracking pipeline would make it genuinely useful for longitudinal developmental studies.</p><h3>Conclusion</h3><p>What started as a problem of scale, too many nuclei, too little time, turned into a clean demonstration of how modern deep learning tools have made biomedical image analysis genuinely accessible. MONAI’s transforms and architecture abstractions meant I could focus on the problem rather than the plumbing. The physics of the microscopy data shaped the design choices at every step: the Dice loss, the augmentation strategy, and the patch size.</p><p>The zebrafish brain is roughly a million times smaller than a human brain. But the pipeline built here, volumetric U-Net, biologically-informed augmentation, careful loss design, scales to much larger problems. That’s what makes this kind of work worth sharing.</p><h3>References</h3><p><strong>Dataset</strong></p><ul><li>NucMM-Z: Nuclear Morphology and Motility, Zebrafish subset. Available via the NucMM benchmark.</li></ul><p><strong>Libraries and Frameworks</strong></p><ul><li><a href="https://monai.io/">MONAI</a> — Medical Open Network for AI (v1.5.2)</li><li><a href="https://pytorch.org/">PyTorch</a> — v2.10.0</li><li><a href="https://www.h5py.org/">h5py</a> — HDF5 file I/O</li><li><a href="https://numpy.org/">NumPy</a> and <a href="https://matplotlib.org/">Matplotlib</a> — numerical computing and visualisation</li></ul><p><strong>Key References</strong></p><ul><li>Ronneberger, O., Fischer, P., &amp; Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. <em>MICCAI</em>.</li><li>Çiçek, Ö. et al. (2016). 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. <em>MICCAI</em>.</li><li>Lin, T.-Y. et al. (2017). Focal Loss for Dense Object Detection. <em>ICCV</em>. (Conceptual context for handling class imbalance.)</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5e7c5e573ca6" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>