Mission Critical - Satellite Data is a Distinct Modality in Machine Learning

Published in

International Conference on Machine Learning(2024)

External Links:

Generate Graph

TL;DR

This position paper argues that satellite data constitutes a distinct modality for machine learning research and that it must recognize it as such to advance the quality and impact of SatML research across theory, methods, and deployment.

Abstract

Satellite data has the potential to inspire a seismic shift for machine learning -- one in which we rethink existing practices designed for traditional data modalities. As machine learning for satellite data (SatML) gains traction for its real-world impact, our field is at a crossroads. We can either continue applying ill-suited approaches, or we can initiate a new research agenda that centers around the unique characteristics and challenges of satellite data. This position paper argues that satellite data constitutes a distinct modality for machine learning research and that we must recognize it as such to advance the quality and impact of SatML research across theory, methods, and deployment. We outline critical discussion questions and actionable suggestions to transform SatML from merely an intriguing application area to a dedicated research discipline that helps move the needle on big challenges for machine learning and society.

Authors

Esther Rolf

2 papers

Konstantin Klemmer

2 papers

C. Robinson

2 papers

References117 items

Multi-Region Transfer Learning for Segmentation of Crop Field Boundaries in Satellite Images with Limited Labels

Exploring Masked Autoencoders for Sensor-Agnostic Image Retrieval in Remote Sensing

Meta-learning to address diverse Earth observation problems across resolutions

Better, Not Just More: Data-centric machine learning for Earth observation

SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery

Mission Critical - Satellite Data is a Distinct Modality in Machine Learning

Published in

International Conference on Machine Learning(2024)

External Links:

Generate Graph

TL;DR

Abstract

Authors

Esther Rolf

2 papers

Konstantin Klemmer

2 papers

C. Robinson

2 papers

References117 items

Multi-Region Transfer Learning for Segmentation of Crop Field Boundaries in Satellite Images with Limited Labels

Exploring Masked Autoencoders for Sensor-Agnostic Image Retrieval in Remote Sensing

Meta-learning to address diverse Earth observation problems across resolutions

Better, Not Just More: Data-centric machine learning for Earth observation

SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery

Hannah Kerner

1 papers

CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoders

Satellite Remote Sensing for Environmental Data Justice: Perspectives from Anti-Prison Community Organizers on the Uses of Geospatial Data

Geographic Location Encoding with Spherical Harmonics and Sinusoidal Representation Networks

GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization

Learning Tri-modal Embeddings for Zero-Shot Soundscape Mapping

SSL4EO-S12: A large-scale multimodal, multitemporal dataset for self-supervised learning in Earth observation [Software and Data Sets]

“Do you collect data to give to the university or do you do the work to benefit people?”: Indigenous Data Sovereignty in Environmental Contexts

Sat2Cap: Mapping Fine-Grained Textual Descriptions from Satellite Images

OpenMapFlow: A Library for Rapid Map Creation with Machine Learning and Remote Sensing Data

SSL4EO-L: Datasets and Foundation Models for Landsat Imagery

GEO-Bench: Toward Foundation Models for Earth Monitoring

Revisiting pre-trained remote sensing model benchmarks: resizing and normalization matters

Fairness and representation in satellite-based poverty maps: Evidence of urban-rural disparities and their impacts on downstream policy

CSP: Self-Supervised Contrastive Spatial Pre-Training for Geospatial-Visual Representations

Evaluation Challenges for Geospatial ML

Considerations for AI-EO for agriculture in Sub-Saharan Africa

Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning

Earth Observation and Artificial Intelligence: Understanding emerging ethical issues and opportunities

SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understanding

LAION-5B: An open large-scale dataset for training next generation image-text models

SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery

Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift

Dynamic World, Near real-time global 10 m land use land cover mapping

Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review

Machine learning-based global maps of ecological variables and the challenge of assessing them

An Empirical Study of Remote Sensing Pretraining

Nearest neighbour distance matching Leave‐One‐Out Cross‐Validation for map validation

Resolving label uncertainty with implicit posterior models

Predicting Out-of-Distribution Error with the Projection Norm

An Artificial Intelligence Dataset for Solar Energy Locations in India

Towards operational validation of annual global land cover maps

SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning

Active Learning for Improved Semi-Supervised Semantic Segmentation in Satellite Images

Spatial cross-validation is not the right way to evaluate map accuracy

SPATE-GAN: Improved Generative Modeling of Dynamic Spatio-Temporal Patterns with an Autoregressive Embedding Loss

Predicting with Confidence on Unseen Distributions

Estimation of root zone soil moisture from ground and remotely sensed soil information with multisensor data fusion and automated machine learning

SoilGrids 2.0: producing soil information for the globe with quantified spatial uncertainty

Priority list of biodiversity metrics to observe from space

Toward a Collective Agenda on AI for Earth Science Data Analysis

Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data

Temporal Cluster Matching for Change Detection of Structures from Satellite Imagery

Can digital technologies reshape rural microfinance? Implications for savings, credit, & insurance

The Importance of High–Quality Data for REDD+ Monitoring and Reporting

Assessing out-of-domain generalization for robust building damage detection

Geography-Aware Self-Supervised Learning

Advancing AI for Earth Science: A Data Systems Perspective

The CARE Principles for Indigenous Data Governance

Indigenous Data Sovereignty in the Era of Big Data and Open Data

A generalizable and accessible approach to machine learning with global satellite imagery

Urgent and critical need for sub-Saharan African countries to invest in Earth observation-based agricultural early warning and monitoring systems

Addressing the need for improved land cover map products for policy support

Towards delivering on the Sustainable Development Goals using Earth observations

Are Labels Always Necessary for Classifier Accuracy Evaluation?

Measuring Robustness to Natural Distribution Shifts in Image Classification

Auxiliary-task learning for geographic data with autoregressive embeddings

Language Models are Few-Shot Learners

Multi-Scale Representation Learning for Spatial Feature Distributions using Grid Cells

Satellite Image Time Series Classification With Pixel-Set Encoders and Temporal Self-Attention

Building Damage Detection in Satellite Imagery Using Convolutional Neural Networks

Key issues in rigorous accuracy assessment of land cover products

SEN12MS - A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusion

Presence-Only Geographical Priors for Fine-Grained Image Classification

Tackling Climate Change with Machine Learning

Human-Machine Collaboration for Fast Land Cover Mapping

Learning to Interpret Satellite Images in Global Scale Using Wikipedia

Machine learning for data-driven discovery in solid Earth geoscience

Domain Adaptation for Convolutional Neural Networks-Based Remote Sensing Scene Classification

Bigearthnet: A Large-Scale Benchmark Archive for Remote Sensing Image Understanding

Deep learning and process understanding for data-driven Earth system science

The spatial leave-pair-out cross-validation method for reliable AUC estimation of spatial classifiers

Evaluation of Using Sentinel-1 and -2 Time-Series to Identify Winter Land Use in Agricultural Landscapes

Combining Sentinel-1 and Sentinel-2 data for improved land use and land cover mapping of monsoon regions

Model Cards for Model Reporting

Synergistic Use of Radar Sentinel-1 and Optical Sentinel-2 Imagery for Crop Mapping: A Case Study for Belgium

CVM-Net: Cross-View Matching Network for Image-Based Ground-to-Aerial Geo-Localization

Tiling and Stitching Segmentation Output for Remote Sensing: Basic Challenges and Recommendations

You Only Look Twice: Rapid Multi-Scale Object Detection In Satellite Imagery

Tile2Vec: Unsupervised representation learning for spatially distributed data

Land cover mapping at very high resolution with rotation equivariant CNNs: towards small yet accurate models

Unsupervised Representation Learning by Predicting Image Rotations

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Functional Map of the World

Deep learning in remote sensing: a review

Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure

Estimating the prediction performance of spatial models via spatial k-fold cross validation

Attention is All you Need

Toward Seamless Multiview Scene Analysis From Satellite to Street Level

SoilGrids250m: Global gridded soil information based on machine learning

Predicting Ground-Level Scene Layout from Aerial Imagery

Combining satellite imagery and machine learning to predict poverty

Domain Adaptation for the Classification of Remote Sensing Data: An Overview of Recent Advances

DETER-B: The New Amazon Near Real-Time Deforestation Detection System

U-Net: Convolutional Networks for Biomedical Image Segmentation

100

Spatial leave‐one‐out cross‐validation for variable selection in the presence of spatial autocorrelation

101

Good practices for estimating area and assessing accuracy of land change

102

The role of satellite remote sensing in climate change studies

103

Machine Learning that Matters

104

ImageNet: A large-scale hierarchical image database

105

‘Copernicus’

106

Artificial intelligence to advance Earth observation: a perspective

107

TIML: Task-Informed Meta-Learning for crop type mapping

108

CropHarvest: A global dataset for crop-type classification

109

Constructing a Visual Dataset to Study the Effects of Spatial Apartheid in South Africa

110

Mapping crops at global scale! What works and what doesn’t?

111

A guide for collecting and sharing ground reference data for machine learning applications

112

Supplementary Information-Large Scale High-Resolution Land Cover Mapping with Multi-Resolution Data

113

A New Synergistic Approach for Monitoring Wetlands Using 1 Sentinels-1 and 2 data With Object-based Machine Learning 2 Algorithms 3

114

Getting ready for nisar—and for managing big data using the commercial cloud

115

Mission Critical – Satellite Data is a Distinct Modality mission%5CField_boundary_delineation ___AAAI_2023_AI2SE-camera-ready.pdf

116

, pre-trained transformers

117

Model Evaluation for Geospatial Problems ∗

Field of Study

Computer Science

Journal Information

Name

ArXiv

Volume

abs/2402.01444

Venue Information

Name

International Conference on Machine Learning

Type

conference

URL

https://icml.cc/

Alternate Names

ICML
Int Conf Mach Learn