New discoveries #6
New discoveries #6
SatMAE, WeatherFusionNet, SatlasPretrain & SRSDD datasets, Lance
Welcome to the 6th edition of the newsletter. I am delighted to share that the newsletter now has 1642 subscribers 🥳 Please note that this edition of the newsletter does not have a sponsor. As a sponsor, you'll receive a shout-out in the opening statement and a dedicated section in the newsletter, reaching a wide audience in the community. If you're interested in gaining visibility for your business or service, sponsoring a future edition of the newsletter is an excellent way to achieve this. For more information on how to sponsor the newsletter, please email me 📧
SatMAE: Pretraining Transformers for Temporal and Multi-Spectral Satellite Imagery
Fresh from NeurIPS 2022 is the paper "SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery". Unsupervised pre-training methods have been shown to enhance performance on downstream supervised tasks, and SatMAE is a pre-training framework for temporal or multi-spectral satellite imagery based on Masked Autoencoder (MAE). In addition to encoding the spatial position of patches, SatMAE also encodes temporal and spectral information within the positional embedding. This approach yields strong improvements over previous state-of-the-art techniques, both in terms of supervised learning performance on benchmark datasets (up to 7%), and transfer learning performance on downstream remote sensing tasks, including land cover classification (up to 14%) and semantic segmentation.
Authors: Yezhen Cong, Samar Khanna, Chenlin Meng, Patrick Liu, Erik Rozi, Yutong He, Marshall Burke, David B. Lobell, Stefano Ermon.
WeatherFusionNet: Predicting Precipitation from Satellite Data
WeatherFusionNet achieved 1st place in the NeurIPS 2022 Weather4Cast challenge. The challenge was for participants to produce high resolution short-term predictions of rainfall from low resolution satellite data. WeatherFusionNet uses three different networks to process the satellite data; extracting rain information from the current frames (sat2rad), predicting future satellite frames (PhyDNet), and combining the input sequence directly to predict rainfall (U-Net). Using this approach, WeatherFusionNet can predict severe rain up to eight hours in advance. Note that the three models are not trained end-to-end due to memory requirements, but the authors propose to do this in the future.
Authors: Jiří Pihrt, Rudolf Raevskiy, Petr Šimánek, Matej Choma
SatlasPretrain dataset
SatlasPretrain is a large-scale pre-training dataset for tasks that involve understanding satellite images. Regularly-updated satellite data is publicly available for much of the Earth through sources such as Sentinel-2 and NAIP, and can inform numerous applications from tackling illegal deforestation to monitoring marine infrastructure.
SRSDD dataset
A High-Resolution SAR Rotation Ship Detection Dataset. All data are from GF-3 Spotlight (SL) mode with a 1-m resolution and each image has 1024 × 1024 pixels. Compared with other existing datasets, the dataset contains multiple categories, namely a total of six categories of 2884 ships
Lance
Lance is an open source python tool for ‘Blazing fast exploration and analysis of computer vision data using SQL and DuckDB, backed by an Apache-Arrow compatible data format’. Lance offers functionality for dataset versioning, debugging, and model reproducibility. Given the familiarity of SQL to many people it is exciting to see a tool which uses it to work with computer vision datasets
Poll
Last week I asked ‘Which ML framework do you use the most?’, and the clear winner was Pytorch, with almost half the votes. Keras/Tensorflow was slightly behind and Sikit learn trailed. The result correlates with what I have observed in the literature at least; a tendency towards Pytorch usage, at least amongst researchers. This week I am interested to know if you have a machine/deep learning model in production?
Consulting with Robin
If you need expert guidance on any of the following topics, I’m available for hourly video call consulting:
Applying machine learning and deep learning techniques to satellite and aerial imagery, including dataset selection, model training, and deployment.
Understanding the physics of remote sensing imaging systems.
Building data processing pipelines in the cloud.
Building your brand and community for technical products.
Personal career development.
As an experienced consultant, I offer customised advice and practical solutions to help you achieve your goals in these areas. To discuss this service please email me 📧
Congratulations that the newsletter is growing. Thanks for sharing these discoveries Robin!