New discoveries #18
New discoveries #18
TreeFormer, HSAT, Geo-Tiles for Semantic Segmentation, Ben-ge dataset, Global Precipitation Nowcasting, AI generated satellite images competition
Welcome to the 18th edition of the newsletter. I am delighted to announce that the newsletter continues to grow and now has 6,376 subscribers. First of all, a big shout-out and special thanks to the sponsors of this newsletter edition, Heimdal Satellite Technologies Ltd (HSAT) 🙏 If you'd like to both support this newsletter and enhance the visibility of your business, service, event, or competition, sponsoring an upcoming edition would be a brilliant choice. To explore sponsorship opportunities, don't hesitate to email me 📧 Thank you for considering this opportunity 🙏
July is typically a time when many of us in Europe take a summer break, and I personally have spent an enjoyable two weeks in Southern Italy. However my time there coincided with an unprecedented heatwave, which led to severe wildfires and highlights the urgent and stark reality of climate change.
As a direct witness to these conditions, I'm reminded of the importance of our work in the field of remote sensing. We have an opportunity to make a real contribution to addressing the pressing challenge of climate change. Let's use these events to underscore the value of remote sensing data, and amplify its significance to the wider community. Together let's take our knowledge and turn it into action, helping to create a safer, more sustainable world.
TreeFormer
This paper introduces TreeFormer, a semi-supervised, transformer-based model for tree counting in aerial and satellite imagery, which can help with forest management. Unlike traditional methods, TreeFormer requires fewer pre-annotated images for training, reducing the annotation costs.
TreeFormer uses a "pyramid" structure, where it starts from a broader context and gradually drills down to finer details, extracting features at multiple scales. It employs contextual attention to focus on relevant features, and a tree density regressor to estimate tree counts. A pyramid learning strategy is used to incorporate both labeled and unlabelled images in training, ensuring local tree density consistency and accurate ranking of tree counts.
Additionally, a "tree counter token" is introduced to regulate the network's global tree counting capability. TreeFormer was evaluated on standard datasets from Jiangsu and Yosemite, and a new KCL-London dataset. Results showed that TreeFormer outperforms other semi-supervised methods and even fully-supervised methods with equal labeled data.
HSAT: Do You Need Ground Truth Data?
HSAT has developed a unique ground truth collection platform that continually gathers data from Brazil to Bangladesh. Our platform, Tessa, is designed to collect data globally at an astonishing pace. Every week, our team visits thousands of sites, maps locations, and labels the data. Our method is over 90% lower cost and 10 times faster than traditional methods.
How Fast? Just last month, we surveyed over 9,000 fields in Thailand , India and Pakistan in under three weeks. We visited every field, photographed it, mapped it, and linked the picture to the polygon. Then we obtained satellite data and weather data for every polygon. All within 20 days.
Where Do We Operate? Our reach is global. We have dedicated teams consistently collecting data in countries like Argentina, Brazil, Kenya, Rwanda, Sudan, Turkey, India, Pakistan, Thailand, and many more.
Applications of Our Data? The data we collect is used for a wide variety of purposes—from powering machine learning models to enhancing logistics understanding.
Examples:
Pakistan: During the we surveyed 2,000 fields in just 10 days to gauge the extent of the damage. We then revisited the exact same locations two weeks later to monitor the impact of the floods
Within 5 days of the earthquake in Turkey, our team was on the ground assessing our factories to determine their capacity to supply essential goods.
Over the past few years, we've surveyed tens of thousands of fields worldwide, creating a comprehensive database of crops
In Need of Rapid and Precise Data? 📧 data@hsat.info
Geo-Tiles for Semantic Segmentation
Deep learning models often require large satellite images to be chipped or 'tiled' into multiple, smaller images suitable for the model's input size. The new GeoTiles tiling scheme was developed to manage this process more effectively. By using the geo-information of the raster data, GeoTiles ensures consistent spatial tile extent, taking into account variables such as differing sensor types, recording distances, and latitudes. It offers users refined control over tile granularity, tile stride, and image boundary alignment, and empowers a user to perform tile-specific data augmentations during training. An accompanying study attests to the system's efficacy, showcasing enhanced results when used with state-of-the-art semantic segmentation models.
Ben-ge dataset: Extending BigEarthNet with Geographical and Environmental Data
Many Earth observation datasets for deep learning include only a couple of modalities at most. The Ben-ge dataset supplements the BigEarthNet-MM dataset by compiling freely and globally available geographical and environmental data. A paper released with the dataset showcases the value of combining different data modalities for the downstream tasks of patch-based land-use/land-cover classification and land-use/land-cover segmentation. Ben-ge is freely available and expected to serve as a test bed for fully supervised and self-supervised Earth observation applications.
Global Precipitation Nowcasting
The paper Global Precipitation Nowcasting of Integrated Multi-satellitE Retrievals for GPM introduces a deep learning architecture for nowcasting of precipitation almost globally every 30 min with a 4-hour lead time. The architecture fuses a U-Net and a convolutional long short-term memory (LSTM) neural network and is trained using data from the Integrated MultisatellitE Retrievals for GPM (IMERG) and a few key precipitation drivers from the Global Forecast System (GFS). The authors found that a regression network works well for light precipitation (<1.6 mm/hr), while a classification network excels in predicting extreme precipitation (>8 mm/hr). This paper expands our understanding of how modern deep learning can enhance the quality of extreme precipitation nowcasting.
LinkedIn Group passes 3k members
I am a regular user of LinkedIn, and share my discoveries on that platform to a dedicated group called the satellite-image-deep-learning group. Since most people already have a LinkedIn account, it is straightforward for anyone with a LinkedIn account to join the group and membership recently passed the 3k members milestone. My hope is that this will become a vibrant location to connect the readership of this newsletter. Please check it out at the link below:
Competition: distinguish real and AI generated satellite images
Recent breakthroughs in generative AI technology have blurred the lines between AI-generated imagery and authentic images to such an extent that differentiation has become challenging. This evolving technology, particularly applied to satellite imagery, is now revolutionising the way we generate maps and model three-dimensional spaces. The objective of this competition, hosted by Solafune and generously sponsored by Stability AI - the masterminds behind Stable Diffusion, is to develop a model capable of distinguishing synthesised satellite-like images from genuine satellite images.
🖥️ Webpage
🗓️ Start: 7 July 2023, End: 5 Sep 2023
💰 Prizes: $10,000 (1st place $ 6,000)
Poll
In the previous poll I asked about the status of remote vs in-office work in our sector. The majority of roles are now hybrid, with an approximately equal split between majority remote (24%) and majority in office (27%). Almost a third of roles (30%) are fully remote, whilst only a fifth (19%) are fully in-office. I expect this is quite a large shift vs pre pandemic, and highlights the range of flexible working arrangements on offer from employers now. In this poll I want to get a broad understanding of the readership of this newsletter
Consulting with Robin
If you need expert guidance on any of the following topics, I’m available for hourly video call consulting:
Applying machine learning and deep learning techniques to satellite and aerial imagery, including dataset selection, model training, and deployment.
Understanding the physics of remote sensing imaging systems.
Building data processing pipelines in the cloud.
Building your brand and community for technical products.
Personal career development.
As an experienced consultant, I offer customised advice and practical solutions to help you achieve your goals in these areas. To discuss this service please email me 📧