Understanding and Processing the Brain Tumor Segmentation (BraTS2020) Dataset

A hands-on guide for data scientists entering medical image segmentation

Example of a brain tumor from the BraTS 2020 training dataset.

Most deep learning practitioners encounter image data in a familiar form: JPEG photos, PNG frames, or RGB tensors with values in [0, 255]. Medical imaging data breaks nearly all of those assumptions. When you open a brain MRI for the first time, you are faced with 3D volumetric data, physically meaningful spatial coordinates, scanner-dependent intensity ranges, and a domain where a mislabeled voxel is not a benchmark number — it represents real tissue in a real patient.

This article walks through the BraTS2020 dataset — the Brain Tumor Segmentation benchmark — from directory structure to preprocessed tensor. By the end, you will understand not just what to do with this data, but why each preprocessing decision exists.

1. What Is BraTS and Why Does It Matter?

BraTS — the Brain Tumor Segmentation challenge — is an annual medical imaging competition that has been running since 2012. It provides the largest publicly available dataset of multi-modal brain MRI scans with expert-annotated tumor segmentation masks. The 2020 edition contains 369 training cases and 125 validation cases, each contributed by multiple clinical institutions across multiple countries.
The clinical problem being solved is glioma segmentation.

Gliomas are the most common primary brain tumors, and their accurate delineation in MRI is critical for:

• Surgical planning: surgeons need to know where the tumor ends and where healthy tissue begins
• Radiotherapy targeting: the radiation dose must be precisely aimed at the tumor tissue
• Treatment response monitoring: comparing pre- and post-treatment volume measures whether therapy is working

Manual segmentation by a neuroradiologist takes 1–4 hours per patient and suffers from inter-rater variability. Automated segmentation is therefore not an academic exercise — it is a clinical necessity.

2. Dataset Structure and File Format

2.1 Directory Layout

After downloading from the Synapse platform (which does not currently host BraTS 2018–2020 data), BraTS2020 unpacks into a consistent structure:

MICCAI_BraTS2020_TrainingData/
  BraTS20_Training_001/
    BraTS20_Training_001_flair.nii
    BraTS20_Training_001_t1.nii
    BraTS20_Training_001_t1ce.nii
    BraTS20_Training_001_t2.nii
    BraTS20_Training_001_seg.nii    ← ground truth mask
  BraTS20_Training_002/
  ...
  BraTS20_Training_369/
  name_mapping.csv
  survival_info.csv

Each case gets its own directory containing exactly 5 NIfTI files. The validation set is identical in structure but omits the _seg.nii file — ground truth labels are withheld for the leaderboard.

2.2 The NIfTI Format

NIfTI (Neuroimaging Informatics Technology Initiative) is the standard file format for neuroimaging data. Unlike DICOM — which stores one 2D slice per file — NIfTI stores the entire 3D volume in a single file, along with metadata in a header.

The two pieces of NIfTI metadata you must understand are:

• The data array: a 3D (or 4D) array of intensity values. In BraTS, this is always the shape of (240, 240, 155)—representing 240 mm left-right, 240 mm front-back, and 155 mm bottom-top of the head.
• The affine matrix: a 4×4 matrix that maps voxel indices to physical coordinates in millimeters. This is what tells you that voxel [0,0,0] corresponds to a specific point in 3D space relative to the scanner.

Simplified representation of NIfTI file storage

Loading a NIfTI file with nibabel is straightforward:

import nibabel as nib
import numpy as np

img = nib.load('BraTS20_Training_001_t1.nii')
vol = img.get_fdata()        # numpy array, shape (240, 240, 155)
affine = img.affine          # (4, 4) voxel-to-world transform
voxel_size = np.sqrt((affine[:3,:3]**2).sum(axis=0))  # [1. 1. 1.] mm

3. The Four MRI Modalities

Unlike RGB images, where three channels always mean red, green, and blue, the four BraTS channels each represent a physically distinct type of MRI acquisition. Understanding them is not optional — the model learns to use each channel for different tumor features.

Image illustrating the four MRI modalities (T1, T2, T1C, FLAIR) alongside their corresponding
ground truth (GT) segmentations.

Why T1ce is the Most Important Modality

In healthy brain tissue, the blood-brain barrier (BBB) prevents most molecules from crossing from blood vessels into brain tissue. When a tumor grows, it disrupts this barrier. Gadolinium, the injected contrast agent, leaks through the disrupted BBB and accumulates in active tumor regions.
This is why T1ce has a maximum intensity of 1845 compared to T1’s 678 — the gadolinium creates an artificially elevated signal in the enhancing tumor region. That bright region corresponds directly to label 4 (enhancing tumor) in the segmentation mask, which is the most clinically significant class.

Why You Need All Four

No single modality captures the full tumor extent. The three tumor regions (defined in Section 5) are each best visible in a different modality:

• Edema — best seen in T2 and FLAIR (fluid appears hyperintense)
• Necrotic core — best distinguished in T1 (dark center of the tumor mass)
• Enhancing tumor — only visible in T1ce (requires the contrast agent)

Providing the model with all four channels simultaneously lets it learn cross-modality features — patterns that are only visible when comparing T1 to T1ce, or T2 to FLAIR. Single-modality segmentation of BraTS is significantly worse across every metric.

4. Data Exploration: What the Numbers Tell You

Running a systematic exploration script on the first training case reveals several facts that directly shape preprocessing decisions:

import nibabel as nib
import numpy as np
from pathlib import Path

case_dir = Path('MICCAI_BraTS2020_TrainingData/BraTS20_Training_001')
modalities = ['flair', 't1', 't1ce', 't2']

for mod in modalities:
    vol = nib.load(case_dir / f'BraTS20_Training_001_{mod}.nii').get_fdata()
    brain = vol[vol > 0]
    print(f'{mod:6s}  max={vol.max():.0f}  brain_mean={brain.mean():.1f}  
           non_zero={100*(vol>0).mean():.1f}%')

seg = nib.load(case_dir / 'BraTS20_Training_001_seg.nii').get_fdata().astype(np.uint8)
print('Labels:', np.unique(seg))
for l in np.unique(seg):
    print(f'  Label {l}: {(seg==l).sum():,} voxels ({100*(seg==l).mean():.2f}%)')

The output for case 001:

flair   max=625   brain_mean=173.0   non_zero=15.0%
t1      max=678   brain_mean=354.3   non_zero=15.0%
t1ce    max=1845  brain_mean=417.3   non_zero=15.0%
t2      max=376   brain_mean=114.7   non_zero=15.0%

Labels: [0 1 2 4]
  Label 0:  8,716,021 voxels (97.63%)
  Label 1:     15,443 voxels (0.17%)
  Label 2:    168,794 voxels (1.89%)
  Label 4:     27,742 voxels (0.31%)

Key Observation 1: Identical Non-Zero Masks

All four modalities report exactly 15.0% non-zero voxels. This is not a coincidence — BraTS has pre-processed all scans to share a common brain mask. Voxels outside the brain are set to zero in all modalities.
This is an important preprocessing gift: you can compute the brain mask from any single modality and apply it to all four.

Key Observation 2: Wildly Different Intensity Ranges

FLAIR peaks at 625, T1ce peaks at 1845 — a 3× difference in the same patient in the same session. Across different patients and different scanners, these ranges vary even more dramatically.
A value of 400 in one patient’s T1 carries no meaning relative to 400 in another patient’s T1.
This is the fundamental difference between MRI and photography. Photograph pixel values represent absolute light intensity. MRI voxel values are relative — they depend on the scanner manufacturer, magnetic field strength, coil type, acquisition parameters, and patient anatomy. You cannot normalize MRI by dividing by 255, and you cannot use ImageNet mean/std. Each modality of each scan must be normalized independently.

Key Observation 3: Extreme Class Imbalance

The segmentation labels reveal a severe imbalance:

The background constitutes 97.63% of all voxels. A model that predicts background for every single voxel achieves 97.63% pixel accuracy — and is clinically worthless. This imbalance is the primary reason why cross-entropy loss alone is insufficient for medical segmentation tasks, which we address in Section 6.

5. BraTS Label Convention and Evaluation Regions

5.1 The Missing Label 3

BraTS2020 segmentation masks use the label set {0, 1, 2, 4} — there is no label 3. This is a historical artifact of how the challenge was originally defined. Your model’s output layer, however, will use indices 0 through n_classes — 1, so you must remap label 4 to label 3 before training:

seg = nib.load(seg_path).get_fdata().astype(np.uint8)
seg[seg == 4] = 3   # remap enhancing tumor: BraTS label → model index

# After remapping:
# 0 = background
# 1 = necrotic core (NCR/NET)
# 2 = peritumoral edema (ED)
# 3 = enhancing tumor (ET)   ← was label 4 in the raw file

Forgetting this remap is a silent bug. Your model will have 4 output channels (0–3), but will never see label 3 in the training data if you don’t remap. The enhancing tumor class — the most clinically critical — will not be learned.

5.2 The Three BraTS Evaluation Regions

The BraTS challenge does not evaluate per-class Dice directly. Instead, it evaluates three compound regions derived from the four labels.
Each region represents a clinically meaningful tumor compartment:

Computing these regions from the model output is straightforward:

pred = model_output   # (H, W, D) integer array with values {0, 1, 2, 3}

whole_tumor    = (pred > 0)              # labels 1+2+3
tumor_core     = np.isin(pred, [1, 3])   # labels 1+3
enhancing      = (pred == 3)             # label 3 only

6. Preprocessing: Z-Score Normalization

Why Not Min-Max or Global Normalization?

The standard computer vision approach—dividing by 255 or applying ImageNet statistics—fails catastrophically on MRI. The reason is that MRI intensity values are not calibrated.
The same tissue imaged on two different scanners produces different numbers. Even the same scanner on the same patient produces slightly different numbers on different days.

Get Farah Abu Hamad’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Min-max normalization is equally problematic because MRI volumes frequently contain bright artifact voxels — extreme outliers caused by motion, metal implants, or scanner noise — that would compress the entire meaningful intensity range into a tiny interval.

Z-Score Normalization Within the Brain Mask

The standard solution, used in nearly all competitive BraTS submissions, is per-modality Z-score normalization restricted to non-zero (brain) voxels:

def normalize_modality(vol: np.ndarray) -> np.ndarray:
    """
    Z-score normalization restricted to brain voxels.
    Background (zero) voxels are left at zero.
    """
    brain_mask = vol > 0
    if brain_mask.sum() == 0:
        return vol   # edge case: empty volume

    mu  = vol[brain_mask].mean()
    std = vol[brain_mask].std() + 1e-8   # epsilon avoids division by zero

    normalized = np.zeros_like(vol)
    normalized[brain_mask] = (vol[brain_mask] - mu) / std
    return normalized

Two design choices here deserve attention:

• Brain-mask restriction: Computing mean and std over all voxels including background zeros, would artificially skew both statistics. A volume that is 85% zeros has a mean dominated by those zeros, not by brain tissue intensity. By restricting to non-zero voxels, the normalization captures the actual brain signal distribution.
• Independent per-modality normalization: Each of the four modalities is normalized separately. Normalizing across modalities would implicitly assume their intensity scales are comparable, which they are not — T1ce intensities are inflated by contrast agent, while T2 intensities reflect entirely different physics.

After normalization, each modality has mean ≈ 0 and std ≈ 1 within brain voxels, and exactly 0 in background voxels. This shared scale is what allows the model to meaningfully compare activations across channels.

Bounding Box Crop and Resize

After normalization, the (240, 240, 155) volume is mostly empty space — only 15% of voxels are brain. Feeding this directly to a 3D U-Net wastes GPU memory on background. The standard approach is to:

• Crop to the tight bounding box of non-zero voxels, removing surrounding air
• Resize to a fixed target shape (commonly 128×128×128) using trilinear interpolation

import torch.nn.functional as F
import torch

def crop_to_brain(vol: np.ndarray) -> np.ndarray:
    coords = np.array(np.where(vol > 0))
    if coords.shape[1] == 0:
        return vol
    mins = coords.min(axis=1)
    maxs = coords.max(axis=1) + 1
    return vol[mins[0]:maxs[0], mins[1]:maxs[1], mins[2]:maxs[2]]

def resize_volume(vol: np.ndarray, target=(128,128,128)) -> np.ndarray:
    vol = crop_to_brain(vol)
    t = torch.from_numpy(vol).unsqueeze(0).unsqueeze(0).float()  # (1,1,H,W,D)
    resized = F.interpolate(t, size=target, mode='trilinear', align_corners=True)
    return resized.squeeze().numpy()

7. Class Imbalance and Why Dice Loss Is Needed

The Problem with Cross-Entropy

Standard cross-entropy loss treats every voxel equally. With 97.63% background voxels, a model minimizing cross-entropy will quickly learn to predict background everywhere—this achieves low loss while being completely useless for tumor detection. The tumor classes, especially the necrotic core at 0.17% of voxels, contribute almost nothing to the gradient signal.

The Dice Coefficient

The Dice similarity coefficient (DSC) measures the overlap between prediction and ground truth as a fraction of their combined size:

Dice is computed per class independently, which means background imbalance does not dilute the gradient signal for small tumor classes. If the model predicts zero enhancing tumor voxels, its Dice score for that class is 0.0—a clear, strong gradient regardless of how many background voxels were correctly predicted.

Soft Dice Loss for Training

The discrete Dice coefficient is not differentiable. For backpropagation, we use a soft version that operates on predicted probabilities rather than hard labels:

import torch
import torch.nn.functional as F

class DiceLoss(torch.nn.Module):
    def __init__(self, smooth=1e-5, ignore_background=True):
        super().__init__()
        self.smooth = smooth
        self.ignore_background = ignore_background

    def forward(self, logits, targets):
        # logits:  (B, C, H, W, D)  raw model output
        # targets: (B, H, W, D)     integer class labels
        num_classes = logits.shape[1]
        probs = F.softmax(logits, dim=1)

        # one-hot encode targets
        targets_oh = F.one_hot(targets.long(), num_classes)
        targets_oh = targets_oh.permute(0,4,1,2,3).float()

        start = 1 if self.ignore_background else 0
        dice_scores = []
        for c in range(start, num_classes):
            p = probs[:, c]
            t = targets_oh[:, c]
            intersection = (p * t).sum()
            dsc = (2*intersection + self.smooth) / (p.sum() + t.sum() + self.smooth)
            dice_scores.append(dsc)

        return 1 - torch.stack(dice_scores).mean()

Combined Dice + Cross-Entropy

In practice, the best results on BraTS come from combining both losses. Dice loss handles the class imbalance; cross-entropy provides stable per-voxel gradients and helps the model learn boundary details:

loss = 0.5 * dice_loss(logits, masks) + 0.5 * cross_entropy(logits, masks)

This 50/50 weighting is a widely used baseline. The Dice weight can be increased if small structures are underperforming; the CE weight can be increased if boundary sharpness is insufficient.

8. The Full Preprocessing Pipeline

Combining all the above into a single pipeline for one patient:

import nibabel as nib
import numpy as np
import torch
import torch.nn.functional as F
from pathlib import Path

MODALITIES = ['flair', 't1', 't1ce', 't2']
TARGET_SHAPE = (128, 128, 128)

def load_case(case_dir: Path):
    case_id = case_dir.name
    volumes = []
    for mod in MODALITIES:
        vol = nib.load(case_dir / f'{case_id}_{mod}.nii').get_fdata().astype(np.float32)
        vol = normalize_modality(vol)         # z-score within brain mask
        vol = resize_volume(vol, TARGET_SHAPE) # crop + resize to 128^3
        volumes.append(vol)

    seg = nib.load(case_dir / f'{case_id}_seg.nii').get_fdata().astype(np.uint8)
    seg[seg == 4] = 3                         # remap label 4 -> 3
    seg = resize_volume(seg, TARGET_SHAPE).astype(np.uint8)

    images = torch.from_numpy(np.stack(volumes)).float()  # (4, 128, 128, 128)
    mask   = torch.from_numpy(seg).long()                 # (128, 128, 128)
    return images, mask

This pipeline is applied identically at training time and inference time. The only addition during training is optional augmentation (random flips, intensity jitter) applied after normalization but before returning the tensor.

Conclusion

Understanding the BraTS dataset before modeling is not optional — it shapes every subsequent decision. The key takeaways from this exploration are:

• 3D volumetric data requires 3D-aware models; 2D slice-by-slice approaches discard spatial context across the depth axis
• Four modalities are complementary — no single channel captures the full tumor extent, and each channel contributes different feature information to the model
• Z-score normalization per modality within the brain mask is the correct approach — not global normalization, not min-max, and not cross-modality statistics
• The label 4 → 3 remap is a silent bug waiting to happen; always verify unique labels before training
• 97.63% background makes cross-entropy loss alone insufficient — Dice loss or a combined objective is required
• Evaluation uses three compound regions (WT, TC, ET), not raw per-class Dice — always report the correct metrics