New discoveries #10
New discoveries #10
ViTs for SITS, MAFAT challenge, Satellite Burned Area Dataset & TorchGeo 0.4.0
Welcome to the 10th edition of the newsletter. I am delighted to announce that the newsletter continues to grow and now has over 3.1k subscribers! First of all, a big shout-out and special thanks to the sponsors of this newsletter edition, the MAFAT challenge 🙏 If you're interested in gaining visibility for your business or service, sponsoring a future edition of the newsletter is an excellent way to achieve this. As a sponsor, you'll receive a shout-out in the opening statement and a dedicated section in the newsletter, reaching a wide audience in the community. For more information on how to sponsor the newsletter, please email me 📧
ViTs for SITS: Vision Transformers for Satellite Image Time Series
Time series sequences of satellite images can be used for crop type classification since seasonal variations specific to particular crop types can be detected. This information can then be used to estimate crop yields and monitor changes in land use over time. The transformer neural network architecture is a type of deep learning model that uses the self-attention mechanism. Transformers initially demonstrated breakthrough results in natural language processing (NLP) tasks, but more recently have been applied to vision and other tasks, and are the subject of intensive research efforts. The paper ViTs for SITS: Vision Transformers for Satellite Image Time Series introduces the temporo-spatial vision transformer (TSViT) architecture. The TSViT incorporates novel design choices that make it suitable for time series tasks. TSViT crop classification and segmentation models are trained and evaluated on Sentinel 2 datasets and achieves state of the art (SOTA) results by a significant margin (shown in the image above). This is an exciting step towards high accuracy and low cost & automated crop mapping using remote sensing imagery.
Authors: Michail Tarasiou, Erik Chavez, Stefanos Zafeiriou
MAFAT challenge update
The “MAFAT Satellite Vision Challenge” — Satellite Imagery Object Detection Competition, has officially started 🌟 The goal is to train oriented object detection models that can identify objects from various classes - such as airplanes, vehicles & vessels. The competition provides a large dataset of diversified satellite images which have a range of resolutions (0.4m to 1.3m), look angles/azimuths and imaging conditions (night, day & seasonal variations). The dataset itself comprises of a labelled dataset (3.3Gb), but also a 5x larger unlabelled dataset (16.6Gb). Winning entrants will undoubtably use both datasets, and the unlabelled dataset presents the opportunity for using semi and unsupervised approaches to training models. The competition website provides the links to the dataset, as well as Jupyter notebooks showing exploratory data analysis (EDA) and the baseline solution using mmrotate (note that you need to be signed in to the platform to see these resources).
💰 prizes: $45,000 in total!
🗓️ End date: 27th April
Satellite Burned Area Dataset
The paper A Dataset for Burned Area Delineation and Severity Estimation from Satellite Imagery introduces a new dataset for Burned Area Delineation and Severity Estimation containing data from Sentinel1 and Sentinel2-L2A (both pre-fire and post-fire data). The dataset coverage is Europe, being mainly Spain, Italy, Portugal + some records on other countries. A baseline Unet model is provided for image segmentation. Thanks to Luca Colomba for alerting me to this dataset on the Discord channel for datasets.
TorchGeo 0.4.0 Release
TorchGeo is a PyTorch library similar to torchvision, but providing datasets, samplers, transforms, and pre-trained models specific to geospatial data. TorchGeo is one of the most exciting projects being developed in this domain, and recently received major upgrades in the 0.4.0 release. These include:
New datasets: Sentinel-1, SpaceNet 6 & Cloud Cover Detection
Performance improvements & bugfixes for datamodules
Support for downloading models from the PyTorch Hub
Object detection is now supported 🥳
Improved documentation
Checkout the release notes and video on TorchGeo below:
Github organisation
My well known Github repository github.com/robmarkcole/satellite-image-deep-learning has been renamed to techniques now lives at the new satellite-image-deep-learning organisation on Github 🌟 Additionally I created several new repositories in order to improve the overall structure of information and improve navigation. These resources exist for the benefit of the whole community, and I encourage fresh contributions to keep on improving them ✍️
Video with Roboflow
I caught up Piotr Skalski, ML Growth Engineer at Roboflow to chat about processing satellite and aerial images 📸 🛰️ We also dived into the community work I am doing here at satellite-image-deep-learning.com. This is quite an easy listed so do check it out!
Discord community
The satellite-image-deep-learning continues to grow, and now has 180 people on the server! To learn more see this page. Hope to talk soon!
How many papers are published with code?
I wrote a small app to query the Arxiv and uncover papers that are accompanied with code on Github. A short experiment indicated that less than 5 % of papers are accompanied with code. Furthermore in many cases the Github repositories were placeholders, and the authors apparently never got around to populating them with usable code. I personally consider that publishing code alongside a paper will dramatically improve the reproducibility of the work. However there is evidently not much motivation to do so. I hope that by raising attention to this issue, more paper authors will consider publishing code alongside their papers 🤞
Poll
If you are either (a) reading a paper or (b) publishing a paper, how important is it to have code published alongside the paper?
One would expect that there is a natural linkage between recognizing the state of, say, a crop or sea state or atmospheric state from appropriate satellite (etc) imagers. The common question is how is the image entropy organized with state. A useful tool when so engaged is the concept of entropic forces in evaluating dynamics.