In this episode I caught up with Justin Guthrie to discuss Infra-Bench CLS, a global benchmark testing seven foundation model backbones on critical infrastructure classification using satellite imagery. We explore how the dataset combines OpenStreetMap labels with Sentinel-1 and Sentinel-2 imagery, and what it takes to compare models with different input requirements using linear probing and fine-tuning.
Justin explains the surprising result that DINOv3, pretrained on natural images, achieved the best overall performance in the benchmark, ahead of the Earth observation models tested. We also discuss why airports, train stations, and data centers are easier to classify, while power infrastructure remains challenging at 10-meter resolution. Finally, we cover lessons for designing fair benchmarks, experiments using just 30% of the training labels, and next steps towards higher-resolution imagery and segmentation.
📖 Paper
💽 Dataset
🖥️ Code
🖥️ Website
📧 Academic inquiries: j.guthrie@northeastern.edu
📧 Business inquiries: justin@enodia.us
Bio: Justin Guthrie is a data scientist who applies AI and Earth observation to understand critical infrastructure and how it holds up under stress. He is a Research Data Analyst at Northeastern University’s Defense Industrial Base Institute (DIBI), a researcher in Northeastern’s Sustainability and Data Sciences (SDS) Lab and at George Mason University, and is a co-founder and GIS Lead of Enodia, an infrastructure resilience startup in Portland, Maine. His research focuses on how infrastructure fails and how those failures spread. His work includes InfraBench CLS, a global benchmark of seven Earth observation foundation models for infrastructure mapping (under review at Big Earth Data) and an investigation of Google AlphaEarth Foundation’s precomputed embeddings in a second-author work which is in publication at IEEE TGRS. He is also a co-first author on a nationwide transmission tower detection study (in progress). With the NASA Disasters group, he uses satellite nighttime lights data to verify coordinated utility power shutoffs, testing whether satellites can track outages independently of utility reports. When not working, you can find Justin at a metal show, relaxing in a cafe, or walking around in Maine’s nature.
🚀 TIMELINE
00:13 – About Justin: Data science, Earth observation, and infrastructure research.
01:05 – Infra-Bench CLS: Why benchmark foundation models on critical infrastructure?
03:13 – Building the dataset: OpenStreetMap labels and Sentinel-1 and Sentinel-2 imagery.
06:03 – Why start with classification rather than detection or segmentation?
07:13 – Selecting foundation models for the benchmark.
08:55 – Comparing models: Input modalities, linear probing, and fine-tuning.
10:37 – Engineering and compute: Google Colab and research clusters.
11:56 – Benchmark results: DINOv3’s surprising performance.
15:55 – Class-level results: Airports, train stations, and data centers.
17:30 – Power infrastructure: Resolution limits and ambiguous substation labels.
21:49 – Next steps: Segmentation, higher resolution, and better experimental design.
25:08 – Practical advice for researchers and infrastructure practitioners.
27:22 – Limited labels: Experiments using 30% of the training labels.
28:56 – Accessing the paper and dataset.
29:25 – Following Justin’s work and getting in touch.
