Why data quality matters more than network architecture in map AI

Evidence level: Simulation or model

Why it matters: When geospatial deep learning pipelines underperform, the standard response is to swap architectures — a reflex that the broader computer vision community has reinforced through years of benchmark competitions structured around fixed, curated datasets. In the domain of geographic mapping, however, training labels are rarely curated: they are drawn from administrative cadastral registers that prioritize legal boundaries over geometric precision, and their hidden inaccuracies propagate silently into model evaluation. If performance rankings among architectures shift depending on which labels serve as the reference, then optimizing architecture choice against noisy labels is not just inefficient — it is actively misleading.

How Reference Data Alters AI Structure Performance in Map Segmentation

The assumption that neural network architecture is the dominant driver of segmentation performance on orthophoto maps is widespread and, as this study demonstrates, empirically fragile. Karwowska and Tusznio evaluated 15 distinct network topologies against a newly compiled dataset of 14,378 geographic samples drawn from across Poland, constructed from Web Map Service orthophotos and vector annotations sourced from the national Land and Buildings Register (EGiB). The architectural scope of the comparison was deliberately broad: tested frameworks spanned traditional convolutional networks, transformer-based architectures, and instance segmentation models in the YOLO-Seg family — covering the three major paradigms currently applied to remote sensing segmentation.

Performance was assessed against two reference standards simultaneously: the official cadastral annotations as extracted from EGiB, and a separately curated, manually verified test set in which label errors had been corrected by human annotators. This dual-reference design was the study’s most consequential methodological choice; it allowed the authors to hold the images constant while varying only the standard against which model outputs were judged. The results were unambiguous: performance hierarchies among the 15 architectures shifted substantially depending on which label set served as the reference, meaning that the architecture ranked highest under cadastral evaluation was not necessarily the highest-performing model when measured against ground truth.

Comparing Label Refinement Against Hyperparameter Optimization

Having established that label quality distorts performance rankings, the study proceeded to quantify the magnitude of that distortion relative to conventional model improvement strategies. Fine-tuning models on superior manual annotations — that is, replacing cadastral training labels with the corrected set — produced measurable gains in Intersection over Union (IoU) on the manually labeled test benchmark. Conventional hyperparameter optimization, applied to models trained on the original cadastral labels, yielded a substantially smaller performance gain by comparison; the statistical significance of this difference was confirmed by a Wilcoxon signed-rank test at a threshold of p < 0.001, ruling out chance as an explanation for the gap.

The practical implication is direct: engineering effort spent adjusting learning rates, batch sizes, and regularization schedules against a noisy label set returns less performance per unit of effort than investing the same resources in label correction. The diminishing returns of hyperparameter tuning under label noise are not surprising in principle — the optimization landscape is being shaped by a corrupted objective — but quantifying this effect across 15 architectures and nearly 15,000 geographic samples gives the finding unusual empirical weight.

Why Dataset Precision Beats Architectural Tweaks

The finding reframes the standard workflow of geospatial AI development. Network architecture selection and hyperparameter tuning are both downstream of label quality: a model cannot learn to draw precise building boundaries if its training signal encodes imprecise ones, and no amount of architectural sophistication compensates for a systematically noisy answer key. The analogy is apt — training a segmentation model on cadastral labels is like asking a student to master geometry against a textbook in which many of the printed coordinates are shifted by a meter or two. Practicing harder against the same flawed reference does not improve the underlying skill; obtaining a corrected reference does.

The authors accordingly recommend elevating reference data quality to a primary design consideration, on par with architecture selection rather than subordinate to it. For the geospatial community, this has resource implications: manual label verification is labor-intensive, and the EGiB cadastral data used here is representative of the administrative vector sources that most national-scale remote sensing projects rely on. The study’s dataset of 14,378 manually verified samples from Poland constitutes a rare open benchmark in this space — one that makes the performance gap between cadastral and verified labels directly measurable for the first time at nationwide scale.

Unexplored Aspects of Architecture Testing and Regional Distribution

Two significant gaps limit the immediate generalizability of these findings. The names and exact configurations of the 15 evaluated architectures are not disclosed in the source, precluding direct comparison with published benchmarks and preventing practitioners from mapping the results onto their own toolchains. Additionally, the geographic distribution of the 14,378 samples within Poland is not characterized — an omission that matters because Polish orthophotos vary considerably in acquisition date, resolution, and urban density across regions, and because cadastral label quality is itself spatially uneven: older registry entries in rural municipalities tend to carry larger geometric errors than recently updated urban parcels. Without this distributional information, it is unclear whether the observed label-quality effects are uniform across the country or concentrated in specific regions.

Verdict: Across 15 architectures and 14,378 Polish geographic samples, correcting training label errors improves segmentation IoU more than hyperparameter optimization does — a finding that reorders the standard priorities of geospatial model development and argues for treating reference data quality as a first-order engineering constraint rather than a background assumption.

Based on reporting from Scientific Reports. This is an independent explainer, written in our own words with AI assistance; Scientific Reports has not reviewed or endorsed it. Read the original for the full details.