OlmoEarth Embeddings: Custom Vector Exports for Earth Observation

OlmoEarth Studio now enables the computation and export of embedding vectors—compact numerical representations of Earth-observation data generated by open-source OlmoEarth foundation models. These embeddings allow users to perform complex geospatial analysis, such as similarity searches and land-cover segmentation, without requiring extensive labeled datasets or massive compute resources.

Computing Embeddings in OlmoEarth Studio

OlmoEarth Studio provides a managed workflow to generate embeddings based on specific user-defined parameters. Users can configure the following settings via the Studio UI or API:

Area of Interest: Custom polygons for imagery acquisition and tiling.
Time Span: Selection of 1 to 12 monthly periods.
Encoder Variants: Three model sizes are available: Nano (128-dim, 1.4M params), Tiny (192-dim, 6.2M params), and Base (768-dim, 89M params).
Spatial Resolution: Options include 10, 20, 40, or 80 meters per pixel.
Imagery Sources: Support for Sentinel-2 L2A, Sentinel-1 RTC, or a combination of both.

Outputs are delivered as Cloud-Optimized GeoTIFFs (COGs) where each band represents one embedding dimension. To optimize storage, vectors are stored as signed 8-bit integers (int8) ranging from -127 to +127, with -128 reserved for nodata. Floating-point vectors can be recovered using the dequantize_embeddings function in the olmoearth_pretrain library.

Downstream Applications for OlmoEarth Embeddings

Because OlmoEarth embeddings organize locations with similar surface characteristics into similar vectors, they can be used for several high-impact geospatial tasks using simple mathematical operations.

Similarity Search

By computing the cosine similarity between a query pixel's embedding and all other pixels in a region, users can create heatmaps of landscape similarity. For example, using OlmoEarth-v1-Tiny, the model can distinguish urban fabric and road corridors from agricultural parcels without any prior labels.

Few-Shot Segmentation

OlmoEarth embeddings enable high-accuracy land-cover mapping with minimal training data. In a test case over Ca Mau, Vietnam, a linear classifier (logistic regression) trained on only 60 labeled pixels (20 per class for mangrove, water, and other) achieved a weighted F1 score of 0.84. The results indicate that the embeddings already internalize ecological distinctions, meaning accuracy saturates quickly even as more labels are added.

Change Detection

Comparing embeddings from two different time periods using per-pixel cosine distance allows for the immediate identification of surface changes. This method was demonstrated by detecting the Park Fire burn scar in Butte County, California, by comparing monthly Sentinel-2 embeddings from September 2023 and September 2024.

Unsupervised Exploration

Principal Component Analysis (PCA) can be used to reduce embedding dimensions to three (mapped to R/G/B) to visualize the model's internal structure. In a test over Flevoland, Netherlands, PCA visualization reproduced the polder parcel grid and distinguished crop types and water bodies without any explicit training on those categories.

Implementation and Performance Optimization

OlmoEarth embeddings serve as a cost-effective entry point for geospatial analysis. For users requiring higher performance than what is possible with frozen embeddings and linear probes, OlmoEarth Studio supports Supervised Fine-Tuning (SFT), which involves training a task-specific model head on custom labels.

Technical Limitations

Embedding quality is subject to the quality of the input imagery. Factors such as persistent cloud cover, atmospheric artifacts, or missing observations during the composite period can negatively impact the resulting vectors.

Availability

The source code, model weights, and research paper are publicly available. Custom embedding exports are available to users of OlmoEarth Studio, and instructions for computing embeddings using the public models are available in the rslearn documentation.