New discoveries #27
Meta global canopy map, Deep Learning with SAR, TerraTorch, TUE-CD & ChatEarthNet datasets, Clay Challenge, Field Area Segmentation Competition, and The Planetary Computer Hub is retiring
Welcome to the 27th edition of the newsletter, now reaching 9,780 subscribers! 🔥 This edition highlights two significant trends in our domain: the growing influence of Big Tech on remote sensing, illustrated by Meta’s release of a foundational model and the use of ChatGPT for dataset annotation, as well as the accelerating development of foundational and multimodal models.
Both trends highlight the immense compute power required to train large models, a resource that remains scarce. As Stanford’s Fei-Fei Li recently noted, “Stanford’s NLP Lab has only 64 GPUs; academia is falling off a cliff in terms of AI resources.” This suggests that future research in this direction may be driven by the largest organisations with the financial resources to command vast computing power.
In the coming years, we can expect profound changes in the organisations that influence the remote sensing domain, with an even faster pace of innovation. Traditional industry players may face challenges, while new entrants will emerge. Whether you embrace these changes or not, it’s undoubtedly an exciting time to be in this industry!
Meta release a global canopy map
Meta and World Resources Institute have released a global map of tree canopy height at a 1-meter resolution, allowing the detection of single trees at a global scale. This is significant since existing global forest maps are derived from low to medium resolution satellite imagery (10 or 30 meters) which do not accurately quantify dispersed forest systems such as agroforestry, drylands forests, and alpine forests, which together constitute more than a third of the world’s forests.
Meta has set a goal of achieving net zero emissions across its activities by 2030 which will largely be achieved by reducing corporate emissions. Meta will address any residual emissions through carbon removal, including through forest-based carbon removal. The release of this global map will enable the monitoring and verification of carbon sequestration projects.
From a technical perspective, the most challenging aspect of this work is developing a model that performs well globally. This was achieved using Self Supervised Learning (SSL) to create a high resolution foundation model for earth observation. Specifically the DINOv2 model was pre-trained solely on unlabelled satellite images (0.5m RGB imagery from Maxar). The canopy height model was fine tuned on the SSL architecture by utilising a modest sized sample of LIDAR ground truth data from the United States (sample shown below).
This work sparked a significant response on social media, leading to numerous interesting exchanges among remote sensing experts about the merits of this dataset. In the further reading I have highlighted one post that effectively summarises the opinions expressed online. However, it is clear that many challenges remain in producing high-quality products on a global scale.
Overall I think this work is a great showcase for the application of foundational models, and I anticipate we will see further refinements from Meta to address some of the outstanding quality issues with this dataset. Given the significant computational resources required for large-scale pre-training of foundational models, I hope companies with access to these resources will explore ways to involve remote sensing experts more deeply in their research. This collaboration could lead to improved outcomes for everyone.
💻 Code
📖 Paper
📖 Further reading: Earth Blox dataset review
Deep Learning with SAR
Follow up my recent video on this topic (linked below) I have published an article on Medium on the topic of deep learning with SAR. The article begins with a brief introduction to SAR, it’s unique imaging & detection capabilities capabilities, and an overview of some applications for SAR. I then progress to more technical content covering the preparation of SAR data for deep learning, model architectures, and emerging trends. Finally I provide information on getting started with SAR, linking to some SAR datasets and pre-trained weights in TorchGeo.
Most of my experience is with optical data, and writing this article was an opportunity to expand my understanding of SAR. I encourage everyone to try writing on topics they are unfamiliar with but curious about, as a way to learn about the topic and educate others at the same time. If there other topics you would like articles on, please let me know!
TerraTorch
TerraTorch is a newly release library from IBM based on PyTorch Lightning and TorchGeo. It’s purpose is to provide a flexible fine-tuning framework for Geospatial Foundation Models, which can be interacted with at different abstraction levels. This release provides a semantic segmentation task and a pixel-wise regression task for fine tuning the Prithvi model, and examples are provided both as Jupyter notebooks, and using configuration files.
TUE-CD dataset
A change detection detection for building damage estimation after earthquake. It consists of bi-temporal images from the Maxar WorldView-2 satellite which capture before/after a devastating magnitude 7.8 earthquake which struck the southern part of Turkey bordering Syria at 4.17 a.m on Monday, 6 February 2023. 1656 image patches with the size of 256×256 are prepared and cover the most severely affected areas, and 2338 destroyed buildings are labeled.
ChatEarthNet dataset
ChatEarthNet is a new image-text dataset, providing high-quality natural language descriptions for global-scale satellite data. It utilises Sentinel-2 data for its global coverage as the foundational image source, with semantic segmentation labels from the European Space Agency's WorldCover project to enrich the descriptions of land covers. This dataset has significant potential for both training and evaluating vision-language geo-foundation models for remote sensing.
Clay Challenge
As featured in Newsletter 25, the Clay nonprofit organisation has a clear objective to create the most capable foundational model by developing in the open. In collaboration with Clark University and Development Seed, Clay have launched the AI For Earth Challenge 2024, which consists of seven tasks intended to highlight the operational value of foundation models. Solutions must be fully open-source and trained only on fully open data. Winners will receive up to $10,000 in compute resources on AWS to be used before end of year to scale up and improve their solution.
🗓️ Submission deadline: Sep 1, 2024
Field Area Segmentation Competition
Satellite data has many applications in the agricultural sector, including identifying crop types and understanding growth conditions. The foundation of these applications is the technology to precisely detect field areas. In this competition, participants are challenged to perform instance segmentation on images of agricultural lands around the world, captured by the Sentinel-2 satellite.
💰 Prize money: $12,000 USD
🗓️ Deadline: Mid July
🎙️ Relevant listen: Field boundary detection with Segment Anything
The Planetary Computer Hub is retiring
The Planetary Computer Hub enables users to access a compute environment hosted on Azure, near to a multitude of geospatial datasets. However it will now be retired on the 6th of June, 2024. I understand this is due to a change of security policy at Microsoft. Fortunately, the datasets will remain accessible. The closure of this platform is a significant loss for our community, but fortunately there are a variety of alternatives listed in the comments on the announcement below.