DAMAGESCOPE: How We Built a Dual-Encoder Damage Detection Model That Works Through Clouds
A research implementation breakdown, from a rejected final-year project idea to IEEE Xplore.
Why This Exists
This isn’t a tutorial. It’s a breakdown of every architectural decision we made, why we made it, and what we learned building a satellite-based damage detection model with a limited dataset, no prior team experience in this domain, and two prior rejections before we even started.
If you want the paper, the IEEE Xplore link is at the bottom. If you want to understand what’s inside it, keep reading.
The Problem
Detecting structural damage from satellite imagery sounds straightforward until you hit the real constraints:
- Disaster zones are almost always cloud-covered at the exact moment you need imagery
- Ground truth labels from crowdsourced disaster data are noisy and inconsistent
- Inference needs to be fast enough to be operationally useful, not just academically interesting
Most existing approaches either ignore cloud occlusion or treat it as a preprocessing problem. We treated it as an architectural problem.
The Architecture
Why Two Encoders?
Single-encoder models flatten everything into one representation. That works fine in clean conditions. Under cloud cover, it collapses; the model has no fallback when one signal degrades.
We used a dual-encoder approach:
- EfficientNet-B3 — handles RGB visual features. Efficient compound scaling, strong at texture and structural detail extraction.
- Modified ResNet-34 — handles auxiliary spectral/contextual features. We modified the residual blocks to better capture spatial damage patterns at multiple scales.
Neither encoder alone is the answer. The question is how you fuse them.
Adaptive Cross-Modal Attention Fusion
This is the core of what makes DAMAGESCOPE different.
Instead of concatenating encoder outputs (which treats both signals as equally reliable at all times), we use an attention mechanism that dynamically weights each encoder’s contribution based on input quality.
In plain terms: when cloud cover degrades the visual signal, the model learns to trust the auxiliary encoder more. When imagery is clean, visual features dominate. The fusion is not fixed; it adapts per input.
This is why we retain 78% accuracy under full cloud occlusion, a condition where most models degrade catastrophically.
IBCC — Handling Noisy Labels
Get Tharun Kumar ⚡’s stories in your inbox
Join Medium for free to get updates from this writer.
The BRIGHT dataset, while the best available for our use case, has a known problem: crowdsourced damage labels are inconsistent. Different annotators classify the same damage differently. Training naively on this introduces systematic noise.
We used an Independent Bayesian Classifier Combination (IBCC) to aggregate labels probabilistically before training. Instead of treating each annotation as ground truth, IBCC weights annotator reliability and produces a cleaner consensus label.
This is not a common step in satellite damage detection pipelines. It added complexity but meaningfully improved label quality, which directly impacts model generalization.
The 111× Speedup
This came from a combination of the following:
- Architectural efficiency choices (EfficientNet-B3 over heavier alternatives)
- Inference optimization during Azure deployment
- Pruning redundant computation paths in the fusion layer
We benchmarked this against a standard single-encoder baseline. The 111× figure is not cherry-picked; it’s the result across the full BRIGHT test split.
Dataset — Working With What We Had
We used the BRIGHT dataset, one of the few publicly available, annotated satellite imagery datasets focused on building damage detection.
It’s limited. We knew that going in. The architecture had to compensate; hence, the attention fusion and hence the IBCC preprocessing. Working with a constrained dataset forces better architecture decisions than working with unlimited data does.
What’s Coming — V2
V2 is currently in development:
- Edge deployment via my open-source torchloop library (PyPI)
- Decentralized, live damage sensing on-device inference with network coordination
- xBD dataset integration significantly larger, global coverage (access requested)
- Larger parameter model for improved generalization
The goal with V2 is moving from research artifact to something operationally deployable in actual disaster response pipelines.
What We Learned
Three things that actually mattered:
1. Architecture beats data volume until it doesn’t. Attention fusion let us squeeze performance out of a limited dataset. But V2 needs xBD because there’s a ceiling on how far good architecture takes you without scale.
2. Label quality is upstream of everything. Most teams would’ve trained on raw BRIGHT labels and accepted the noise as baseline. IBCC preprocessing was one of the highest-ROI decisions we made.
3. Rejected ideas don’t die. They wait. This project was shelved twice. The architecture we ended up with wasn’t possible in the second year when the idea first appeared; we needed time, more papers, and a deadline to force it into shape.
📄 Full paper on IEEE Xplore: https://ieeexplore.ieee.org/document/11518794/
🔧 torchloop library: https://pypi.org/project/torchloop/
💻 GitHub: https://github.com/Tharun007-TK/disaster-detection
