Open-Source Earth AI That Understands Images Like ChatGPT Understands Text 
A revolutionary open-source platform called Clay is transforming the way we process Earth imagery — using the same core idea behind ChatGPT: transformers.
Created by rocket scientist-turned-entrepreneur Bruno Sánchez, Clay is an Earth-focused AI that interprets aerial imagery with unmatched speed and accuracy, harnessing the power of transformer architectures traditionally used for text, image, and audio processing.
How It Works
Clay processes high-resolution Earth imagery (satellite, drone, plane) to detect objects such as forests, solar panels, boats, and more. It doesn’t rely on human labels or supervision. Instead, it uses Masked AutoEncoders, which force the AI to learn by context and compression — much like how ChatGPT learns relationships in text.
Clay’s embeddings — high-dimensional vector representations of Earth images — are the real game-changer. Once generated, they can be reused for multiple tasks, from locating solar panels to tracking construction growth, all within milliseconds via a vector database.
“You only need to create them once. That’s the power of embeddings — universal pre-compute for most answers.”
Better Than Benchmarks
Clay isn’t focused on leaderboards. Instead, it shines in real-world use cases, offering:
- Fast object identification when the target dominates the image
- Minimal hallucinations, thanks to high-quality, public-domain datasets
- Scalability through embeddings, not constant retraining
And thanks to ergodicity — the fact that Earth presents all patterns somewhere at any time — Clay can generalize new changes (like desertification) without retraining, just by regenerating embeddings.
Beyond Vision: Clay Meets Text
Clay’s team is now exploring multi-modality. They’re working to align Clay’s visual embeddings with text labels using OpenStreetMaps. The result? A universal semantic layer where “forest” in text maps to clusters of trees in imagery — potentially revolutionizing search and classification.
Sustainable by Design
Given the environmental toll of AI training, Clay is intentionally:
- Open-source and non-profit
- Designed to minimize compute usage by leveraging precomputed embeddings
- Positioned to unify the AI-for-Earth ecosystem, reducing redundant models
“We’re not a geospatial company. We’re an answers company.”
How to Use Clay
Besides developer-focused embedding APIs, the team built Clay Explore — a clickable map interface to visually search and extract geospatial insights. Still, future UIs might include spreadsheet plugins or chat-based interfaces.
Meanwhile, Sánchez’s startup, LGND, is commercializing services around Clay, providing tools and services without compromising the project’s open ethos.
For deeper insights into AI, geospatial tech, and the future of Earth data, subscribe to the Orchestrate All the Things Newsletter.
ENJOY & HAPPY LEARNING! 
!