Welcome to the 6th edition of the newsletter. I am delighted to share that the newsletter now has 1642 subscribers 🥳 Please note that this edition of the newsletter does not have a sponsor. As a sponsor, you'll receive a shout-out in the opening statement and a dedicated section in the newsletter, reaching a wide audience in the community. If you're interested in gaining visibility for your business or service, sponsoring a future edition of the newsletter is an excellent way to achieve this. For more information on how to sponsor the newsletter, please email me 📧

SatMAE: Pretraining Transformers for Temporal and Multi-Spectral Satellite Imagery

Fresh from NeurIPS 2022 is the paper "SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery". Unsupervised pre-training methods have been shown to enhance performance on downstream supervised tasks, and SatMAE is a pre-training framework for temporal or multi-spectral satellite imagery based on Masked Autoencoder (MAE). In addition to encoding the spatial position of patches, SatMAE also encodes temporal and spectral information within the positional embedding. This approach yields strong improvements over previous state-of-the-art techniques, both in terms of supervised learning performance on benchmark datasets (up to 7%), and transfer learning performance on downstream remote sensing tasks, including land cover classification (up to 14%) and semantic segmentation.

Authors: Yezhen Cong, Samar Khanna, Chenlin Meng, Patrick Liu, Erik Rozi, Yutong He, Marshall Burke, David B. Lobell, Stefano Ermon.

• 🖥️ webpage

• 💻 Github

• 📖 paper

• 🖥️ blog post on transformers

WeatherFusionNet: Predicting Precipitation from Satellite Data

WeatherFusionNet achieved 1st place in the NeurIPS 2022 Weather4Cast challenge. The challenge was for participants to produce high resolution short-term predictions of rainfall from low resolution satellite data. WeatherFusionNet uses three different networks to process the satellite data; extracting rain information from the current frames (sat2rad), predicting future satellite frames (PhyDNet), and combining the input sequence directly to predict rainfall (U-Net). Using this approach, WeatherFusionNet can predict severe rain up to eight hours in advance. Note that the three models are not trained end-to-end due to memory requirements, but the authors propose to do this in the future.

Authors: Jiří Pihrt, Rudolf Raevskiy, Petr Šimánek, Matej Choma

• 💻 Github

• 📖 paper

Satlas dataset

Satlas is a large-scale, multi-task dataset for benchmarking and improving remote sensing image understanding models. The goal with Satlas is to: label everything that is visible in satellite imagery. Satlas contains 10x more image pixels than the largest existing dataset, and spans seven modalities. It is 2x more than the largest existing dataset.

• 💻 Website

• 🗓️ release date: January 2023

SRSDD dataset

A High-Resolution SAR Rotation Ship Detection Dataset. All data are from GF-3 Spotlight (SL) mode with a 1-m resolution and each image has 1024 × 1024 pixels. Compared with other existing datasets, the dataset contains multiple categories, namely a total of six categories of 2884 ships

• 💻 Github

• 📖 paper

Lance

Lance is an open source python tool for ‘Blazing fast exploration and analysis of computer vision data using SQL and DuckDB, backed by an Apache-Arrow compatible data format’. Lance offers functionality for dataset versioning, debugging, and model reproducibility. Given the familiarity of SQL to many people it is exciting to see a tool which uses it to work with computer vision datasets

• 💻 Github

• 📖 Building a time machine: seamless ML dataset versioning with Lance

Jobs & Events

Do you have a job or event you would like to promote here (first post is free)? Let me know!

Weekly poll

Last week I asked ‘Which ML framework do you use the most?’, and the clear winner was Pytorch, with almost half the votes. Keras/Tensorflow was slightly behind and Sikit learn trailed. The result correlates with what I have observed in the literature at least; a tendency towards Pytorch usage, at least amongst researchers. This week I am interested to know if you have a machine/deep learning model in production?