satellite-image-deep-learning

satellite-image-deep-learning

Share this post

satellite-image-deep-learning
satellite-image-deep-learning
New discoveries #27
Copy link
Facebook
Email
Notes
More
User's avatar
Discover more from satellite-image-deep-learning
Newsletter on deep learning with satellite & aerial imagery
Over 12,000 subscribers
Already have an account? Sign in

New discoveries #27

Meta global canopy map, Deep Learning with SAR, TerraTorch, TUE-CD & ChatEarthNet datasets, Clay Challenge, Field Area Segmentation Competition, and The Planetary Computer Hub is retiring

Robin Cole's avatar
Robin Cole
May 29, 2024
8

Share this post

satellite-image-deep-learning
satellite-image-deep-learning
New discoveries #27
Copy link
Facebook
Email
Notes
More
Share

Welcome to the 27th edition of the newsletter, now reaching 9,780 subscribers! 🔥 This edition highlights two significant trends in our domain: the growing influence of Big Tech on remote sensing, illustrated by Meta’s release of a foundational model and the use of ChatGPT for dataset annotation, as well as the accelerating development of foundational and multimodal models.

Both trends highlight the immense compute power required to train large models, a resource that remains scarce. As Stanford’s Fei-Fei Li recently noted, “Stanford’s NLP Lab has only 64 GPUs; academia is falling off a cliff in terms of AI resources.” This suggests that future research in this direction may be driven by the largest organisations with the financial resources to command vast computing power.

In the coming years, we can expect profound changes in the organisations that influence the remote sensing domain, with an even faster pace of innovation. Traditional industry players may face challenges, while new entrants will emerge. Whether you embrace these changes or not, it’s undoubtedly an exciting time to be in this industry!

Meta release a global canopy map

Meta and World Resources Institute have released a global map of tree canopy height at a 1-meter resolution, allowing the detection of single trees at a global scale. This is significant since existing global forest maps are derived from low to medium resolution satellite imagery (10 or 30 meters) which do not accurately quantify dispersed forest systems such as agroforestry, drylands forests, and alpine forests, which together constitute more than a third of the world’s forests. 

Meta has set a goal of achieving net zero emissions across its activities by 2030 which will largely be achieved by reducing corporate emissions. Meta will address any residual emissions through carbon removal, including through forest-based carbon removal. The release of this global map will enable the monitoring and verification of carbon sequestration projects.

From a technical perspective, the most challenging aspect of this work is developing a model that performs well globally. This was achieved using Self Supervised Learning (SSL) to create a high resolution foundation model for earth observation. Specifically the DINOv2 model was pre-trained solely on unlabelled satellite images (0.5m RGB imagery from Maxar). The canopy height model was fine tuned on the SSL architecture by utilising a modest sized sample of LIDAR ground truth data from the United States (sample shown below).

This work sparked a significant response on social media, leading to numerous interesting exchanges among remote sensing experts about the merits of this dataset. In the further reading I have highlighted one post that effectively summarises the opinions expressed online. However, it is clear that many challenges remain in producing high-quality products on a global scale.

Overall I think this work is a great showcase for the application of foundational models, and I anticipate we will see further refinements from Meta to address some of the outstanding quality issues with this dataset. Given the significant computational resources required for large-scale pre-training of foundational models, I hope companies with access to these resources will explore ways to involve remote sensing experts more deeply in their research. This collaboration could lead to improved outcomes for everyone.

  • 🖥️ Meta official post

  • 💻 Code

  • 📖 Paper

  • 📖 Further reading: Earth Blox dataset review

Deep Learning with SAR

Follow up my recent video on this topic (linked below) I have published an article on Medium on the topic of deep learning with SAR. The article begins with a brief introduction to SAR, it’s unique imaging & detection capabilities capabilities, and an overview of some applications for SAR. I then progress to more technical content covering the preparation of SAR data for deep learning, model architectures, and emerging trends. Finally I provide information on getting started with SAR, linking to some SAR datasets and pre-trained weights in TorchGeo.

Most of my experience is with optical data, and writing this article was an opportunity to expand my understanding of SAR. I encourage everyone to try writing on topics they are unfamiliar with but curious about, as a way to learn about the topic and educate others at the same time. If there other topics you would like articles on, please let me know!

  • 📖 Article: Deep Learning with SAR

  • 📺 YouTube video: Machine learning with SAR at ASTERRA

TerraTorch

TerraTorch is a newly release library from IBM based on PyTorch Lightning and TorchGeo. It’s purpose is to provide a flexible fine-tuning framework for Geospatial Foundation Models, which can be interacted with at different abstraction levels. This release provides a semantic segmentation task and a pixel-wise regression task for fine tuning the Prithvi model, and examples are provided both as Jupyter notebooks, and using configuration files.

  • 📖 TerraTorch documentation

  • 📰 Prithvi in the 19th Newsletter

  • 📺 YouTube video on TorchGeo

TUE-CD dataset

A change detection detection for building damage estimation after earthquake. It consists of bi-temporal images from the Maxar WorldView-2 satellite which capture before/after a devastating magnitude 7.8 earthquake which struck the southern part of Turkey bordering Syria at 4.17 a.m on Monday, 6 February 2023. 1656 image patches with the size of 256×256 are prepared and cover the most severely affected areas, and 2338 destroyed buildings are labeled.

  • 💽 TUE-CD Dataset

ChatEarthNet dataset

ChatEarthNet is a new image-text dataset, providing high-quality natural language descriptions for global-scale satellite data. It utilises Sentinel-2 data for its global coverage as the foundational image source, with semantic segmentation labels from the European Space Agency's WorldCover project to enrich the descriptions of land covers. This dataset has significant potential for both training and evaluating vision-language geo-foundation models for remote sensing.

  • 💽 ChatEarthNet Dataset

Clay Challenge

As featured in Newsletter 25, the Clay nonprofit organisation has a clear objective to create the most capable foundational model by developing in the open. In collaboration with Clark University and Development Seed, Clay have launched the AI For Earth Challenge 2024, which consists of seven tasks intended to highlight the operational value of foundation models. Solutions must be fully open-source and trained only on fully open data. Winners will receive up to $10,000 in compute resources on AWS to be used before end of year to scale up and improve their solution.

  • 🖥️ Challenge website

  • 🗓️ Submission deadline: Sep 1, 2024

Field Area Segmentation Competition

Satellite data has many applications in the agricultural sector, including identifying crop types and understanding growth conditions. The foundation of these applications is the technology to precisely detect field areas. In this competition, participants are challenged to perform instance segmentation on images of agricultural lands around the world, captured by the Sentinel-2 satellite.

  • 🖥️ Competition website

  • 💰 Prize money: $12,000 USD

  • 🗓️ Deadline: Mid July

  • 🎙️ Relevant listen: Field boundary detection with Segment Anything

The Planetary Computer Hub is retiring

The Planetary Computer Hub enables users to access a compute environment hosted on Azure, near to a multitude of geospatial datasets. However it will now be retired on the 6th of June, 2024. I understand this is due to a change of security policy at Microsoft. Fortunately, the datasets will remain accessible. The closure of this platform is a significant loss for our community, but fortunately there are a variety of alternatives listed in the comments on the announcement below.

  • 📣 Announcement

Thanks for reading satellite-image-deep-learning! Subscribe for free to receive new posts and support my work.

Hayat Khan's avatar
Shouzo Tanaka's avatar
Miguel Ibáñez's avatar
Nathan Kiner's avatar
Madan Kumar Y's avatar
8 Likes
8

Share this post

satellite-image-deep-learning
satellite-image-deep-learning
New discoveries #27
Copy link
Facebook
Email
Notes
More
Share

Discussion about this post

User's avatar
New Discoveries #29
Open Buildings 2.5D Temporal Dataset, Detecting Looted Archaeological Sites from Satellite Image Time Series, Global Prediction of Aboveground Biomass…
Oct 3, 2024 • 
Robin Cole
14

Share this post

satellite-image-deep-learning
satellite-image-deep-learning
New Discoveries #29
Copy link
Facebook
Email
Notes
More
1
Deepness QGIS plugin
With Marek Kraft
Dec 19, 2024 • 
Robin Cole
11

Share this post

Copy link
Facebook
Email
Notes
More
12:56
Building Damage Assessment
With Caleb Robinson
Jan 8 • 
Robin Cole
10

Share this post

Copy link
Facebook
Email
Notes
More
17:10

Ready for more?

© 2025 Robin Cole
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share

Copy link
Facebook
Email
Notes
More

Create your profile

User's avatar

Only paid subscribers can comment on this post

Already a paid subscriber? Sign in

Check your email

For your security, we need to re-authenticate you.

Click the link we sent to , or click here to sign in.