microsoft/satclip

What it solves

SatCLIP creates a single, reusable representation (a vector) for any coordinate on Earth. Instead of feeding raw latitude/longitude to a model, you can use SatCLIP's location embedding as input to many different tasks — like predicting temperature, income, or biome type. It learns what a place is like by looking at satellite images of that place, so the embedding captures meaningful geographic and environmental features.

How it works

SatCLIP is trained with contrastive learning, the same idea used in OpenAI's CLIP model. The training data consists of pairs: a satellite image and the coordinates where that image was taken. Two encoders are trained together: an image encoder that turns the satellite image into a vector, and a location encoder that turns coordinates (longitude/latitude) into a vector. During training, the model learns to make the two vectors similar for matching pairs and different for non‑matching pairs. After training, you only need the location encoder: give it any coordinates and it outputs a vector that summarises the location. The model was trained on a new dataset called S2‑100K, which contains 100,000 multi‑spectral satellite images (256×256 pixels, 12 bands, 10 m resolution) sampled from Sentinel‑2.

Who it’s for

Researchers and developers working on geospatial machine learning tasks, such as predicting environmental variables, classifying regions, or localising images. It is also useful for anyone who needs a pretrained location embedding without building their own from scratch. The project provides Python code, pretrained models, and example notebooks.

Highlights

• Pretrained models available for three vision backbones (ResNet‑18, ResNet‑50, ViT‑B/16) and two spatial‑resolution settings.
• Trained on the S2‑100K dataset of 100,000 multi‑spectral satellite images, released by the authors.
• Evaluated on a wide range of downstream tasks: regression (temperature, precipitation, income, housing, elevation, population density) and classification (country, biome, ecoregion, species).
• Open‑source with training code, command‑line interface, and notebooks for easy use.