Mission Critical - Satellite Data is a Distinct Modality in Machine Learning (2024-02-02T00:00:00.000000Z)

TL;DR

This position paper argues that satellite data constitutes a distinct modality for machine learning research and that it must recognize it as such to advance the quality and impact of SatML research across theory, methods, and deployment.

Abstract

Satellite data has the potential to inspire a seismic shift for machine learning -- one in which we rethink existing practices designed for traditional data modalities. As machine learning for satellite data (SatML) gains traction for its real-world impact, our field is at a crossroads. We can either continue applying ill-suited approaches, or we can initiate a new research agenda that centers around the unique characteristics and challenges of satellite data. This position paper argues that satellite data constitutes a distinct modality for machine learning research and that we must recognize it as such to advance the quality and impact of SatML research across theory, methods, and deployment. We outline critical discussion questions and actionable suggestions to transform SatML from merely an intriguing application area to a dedicated research discipline that helps move the needle on big challenges for machine learning and society.

Authors

Esther Rolf

2 papers

Konstantin Klemmer

2 papers

C. Robinson

2 papers

TL;DR

Abstract

Authors

References117 items

Multi-Region Transfer Learning for Segmentation of Crop Field Boundaries in Satellite Images with Limited Labels

Exploring Masked Autoencoders for Sensor-Agnostic Image Retrieval in Remote Sensing

Meta-learning to address diverse Earth observation problems across resolutions

Better, Not Just More: Data-centric machine learning for Earth observation

SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery

CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoders

Satellite Remote Sensing for Environmental Data Justice: Perspectives from Anti-Prison Community Organizers on the Uses of Geospatial Data

Geographic Location Encoding with Spherical Harmonics and Sinusoidal Representation Networks

GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization

Learning Tri-modal Embeddings for Zero-Shot Soundscape Mapping

SSL4EO-S12: A large-scale multimodal, multitemporal dataset for self-supervised learning in Earth observation [Software and Data Sets]

“Do you collect data to give to the university or do you do the work to benefit people?”: Indigenous Data Sovereignty in Environmental Contexts

Sat2Cap: Mapping Fine-Grained Textual Descriptions from Satellite Images

OpenMapFlow: A Library for Rapid Map Creation with Machine Learning and Remote Sensing Data

SSL4EO-L: Datasets and Foundation Models for Landsat Imagery

GEO-Bench: Toward Foundation Models for Earth Monitoring

Revisiting pre-trained remote sensing model benchmarks: resizing and normalization matters

Fairness and representation in satellite-based poverty maps: Evidence of urban-rural disparities and their impacts on downstream policy

CSP: Self-Supervised Contrastive Spatial Pre-Training for Geospatial-Visual Representations

Evaluation Challenges for Geospatial ML

Considerations for AI-EO for agriculture in Sub-Saharan Africa

Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning

Earth Observation and Artificial Intelligence: Understanding emerging ethical issues and opportunities

SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understanding

LAION-5B: An open large-scale dataset for training next generation image-text models

SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery

Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift

Dynamic World, Near real-time global 10 m land use land cover mapping

Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review

Machine learning-based global maps of ecological variables and the challenge of assessing them

An Empirical Study of Remote Sensing Pretraining

Nearest neighbour distance matching Leave‐One‐Out Cross‐Validation for map validation

Resolving label uncertainty with implicit posterior models

Predicting Out-of-Distribution Error with the Projection Norm

An Artificial Intelligence Dataset for Solar Energy Locations in India

Towards operational validation of annual global land cover maps

SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning

Active Learning for Improved Semi-Supervised Semantic Segmentation in Satellite Images

Spatial cross-validation is not the right way to evaluate map accuracy

SPATE-GAN: Improved Generative Modeling of Dynamic Spatio-Temporal Patterns with an Autoregressive Embedding Loss

Predicting with Confidence on Unseen Distributions

Estimation of root zone soil moisture from ground and remotely sensed soil information with multisensor data fusion and automated machine learning

SoilGrids 2.0: producing soil information for the globe with quantified spatial uncertainty

Priority list of biodiversity metrics to observe from space

Toward a Collective Agenda on AI for Earth Science Data Analysis

Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data

Temporal Cluster Matching for Change Detection of Structures from Satellite Imagery

Can digital technologies reshape rural microfinance? Implications for savings, credit, & insurance

The Importance of High–Quality Data for REDD+ Monitoring and Reporting

Assessing out-of-domain generalization for robust building damage detection

Geography-Aware Self-Supervised Learning

Advancing AI for Earth Science: A Data Systems Perspective

The CARE Principles for Indigenous Data Governance

Indigenous Data Sovereignty in the Era of Big Data and Open Data

A generalizable and accessible approach to machine learning with global satellite imagery

Urgent and critical need for sub-Saharan African countries to invest in Earth observation-based agricultural early warning and monitoring systems

Addressing the need for improved land cover map products for policy support

Towards delivering on the Sustainable Development Goals using Earth observations

Are Labels Always Necessary for Classifier Accuracy Evaluation?

Measuring Robustness to Natural Distribution Shifts in Image Classification

Auxiliary-task learning for geographic data with autoregressive embeddings

Language Models are Few-Shot Learners

Multi-Scale Representation Learning for Spatial Feature Distributions using Grid Cells

Satellite Image Time Series Classification With Pixel-Set Encoders and Temporal Self-Attention

Building Damage Detection in Satellite Imagery Using Convolutional Neural Networks

Key issues in rigorous accuracy assessment of land cover products

SEN12MS - A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusion

Presence-Only Geographical Priors for Fine-Grained Image Classification

Tackling Climate Change with Machine Learning

Human-Machine Collaboration for Fast Land Cover Mapping

Learning to Interpret Satellite Images in Global Scale Using Wikipedia

Machine learning for data-driven discovery in solid Earth geoscience

Domain Adaptation for Convolutional Neural Networks-Based Remote Sensing Scene Classification

Bigearthnet: A Large-Scale Benchmark Archive for Remote Sensing Image Understanding

Deep learning and process understanding for data-driven Earth system science

The spatial leave-pair-out cross-validation method for reliable AUC estimation of spatial classifiers