The satellite industry is at a pivotal moment, with thousands of spacecraft in low Earth orbit (LEO) and the need for real-time telemetry data to manage and optimize these fleets. However, the sheer volume and complexity of this telemetry data present a significant challenge: the cardinality wall. This issue is not just about scaling up existing systems; it's about fundamentally rethinking how we approach telemetry infrastructure to support the scale, distribution, and context preservation required for modern space operations.
The cardinality wall refers to the point at which the volume and complexity of telemetry data become so great that traditional ground system databases and architectures begin to fail. This is particularly problematic for constellations with millions of distinct telemetry streams in real-time, as these systems were never designed to handle such scale.
One of the key challenges is the metadata associated with each telemetry stream. Every combination of spacecraft ID, subsystem, component, sensor identifier, orbit segment, mission phase, and software configuration creates a unique stream, leading to high cardinality. This makes it difficult for traditional databases to scale incrementally alongside the fleet, as they struggle with the sheer volume of data and the need to maintain indexes and ensure data consistency.
The issue is further compounded by the need for long-term telemetry retention. Satellite programs often store data for years or decades, which requires systems to support both real-time ingestion and large-scale historical analysis. This is a complex task for general-purpose databases, and as cardinality and retention pressures increase, systems begin to fail, forcing operators to simplify the data just to keep things running.
One example of this challenge is Loft Orbital, which operates microsatellites and mission infrastructure in LEO. As its platform scaled, Loft needed to handle more than 500 million telemetry measurements per day, with ingestion rates reaching 10 million measurements every 10 minutes. Earlier approaches built on relational databases struggled to keep up with both the volume and structure of the data, limiting visibility into system performance. By moving to a time series-oriented architecture, Loft was able to ingest high-frequency telemetry, maintain full context across missions, and deliver faster access to both real-time and historical data.
However, the temptation to remove tags, downsample signals, or shorten retention windows to simplify the data is a dangerous tradeoff. This can lead to the loss of context, which is crucial for engineers to correlate events across subsystems and for machine learning systems to predict component failures. As telemetry infrastructure becomes increasingly mission-critical, the loss of context can have significant implications for operational visibility, anomaly response, and mission resilience.
Breaking through the cardinality wall requires a shift in mindset. Teams need to recognize when their current approach has reached its limits and identify where cardinality is already impacting operations. This could manifest in delayed anomaly detection, slow replay of historical events, or gaps in data during peak ingest periods. By focusing on these pressure points, teams can make more effective changes rather than attempting a full system overhaul.
One effective strategy is to decouple parts of the telemetry pipeline rather than replacing everything at once. Separating high-throughput ingestion from analytical workloads can stabilize real-time monitoring while longer-term changes are planned. Additionally, revisiting strategies that depend on throwing away data to stay operational can help preserve full context and avoid blind spots that may surface later during anomaly investigations or design reviews.
In conclusion, the cardinality wall is a significant challenge for the satellite industry, but it also presents an opportunity to rethink and redesign telemetry infrastructure to support the scale, distribution, and context preservation required for modern space operations. By focusing on the pressure points and making strategic changes, teams can break through the cardinality wall and ensure the reliability and resilience of their LEO operations.
