OlmoEarth v1.1 release notes / what's new

OlmoEarth v1.1 release notes / what's new

OlmoEarth v1.1 reduces compute costs by up to 3x

Hugging Face and AllenAI have released OlmoEarth v1.1, a new family of Earth observation models designed to significantly lower the computational overhead of processing satellite imagery. OlmoEarth v1.1 achieves up to a 3x reduction in compute costs across pre-training, fine-tuning, and inference while maintaining performance levels similar to the original OlmoEarth v1 release.

Efficiency gains through token sequence length reduction

Compute costs in transformer-based models scale quadratically with token sequence length. To improve efficiency, the OlmoEarth team focused on reducing the number of tokens generated from remote sensing data.

In the original OlmoEarth v1, Sentinel-2 imagery was processed using resolution-based patches. For a given spatial patch, the model created a unique token per timestep per resolution (10m, 20m, and 60m). This meant a Sentinel-2 input with two timesteps would yield six tokens per patch.

OlmoEarth v1.1 optimizes this by collapsing these resolutions into a single token. This approach produces three times fewer tokens, leading to material savings in computation. While naively merging tokens typically causes performance drops—such as a 10 percentage point drop on the m-eurosat kNN benchmark—the OlmoEarth team implemented modifications to the pre-training regimen to merge tokens without impacting performance.

Model family and accessibility

OlmoEarth v1.1 is released as a family of models to allow users to select a size that fits their specific compute budget. The release includes weights and training code for the following model sizes:

  • Base
  • Tiny
  • Nano

Implications for developers and researchers

For Developers

OlmoEarth v1.1 enables more affordable and frequent planet-scale map refreshes. Developers using the original OlmoEarth v1 models can expect significant speedups during fine-tuning and inference, provided the model performs well for their specific task, although some regressions have been noted in the technical report.

For Researchers

To isolate the effect of methodological changes, OlmoEarth v1.1 was trained on the same dataset as OlmoEarth v1. This allows researchers to study the impact of changes in tokenization and pre-training algorithms without the influence of dataset variations.

Resources

Sources