Low level how does this compare to Victoriametrics, you can get the high cardinality with clickhouse and other columnar dbs but the tradeoff is more ram usage, slower queries, io, etc
Our architecture is built around columnar design, and we use Apache Arrow for in-memory columnar processing and Apache Parquet for durable columnar storage on S3-compatible object storage. In Parseable, every labels stay as columns in the data instead of becoming a large long-lived per-series index like many TSDBs.
Also, one thing we’ve been thinking about a lot is how observability changes as agents become part of day-to-day engineering workflows. They're not just another service, they produce traces, tool calls, prompts, intermediate decisions, errors, costs, and sometimes sensitive business context.
Observing them matters just as much as observing any other system. But it is equally important to decide where that telemetry data should reside. Our view is that teams should be able to keep these observability data close to them: in their own object storage, under their own retention, access, and compliance controls.
How does the 100M active series deployment looks like? How many ingestors are there? What's each instance size? How big is the querier so that it can query across a metric with millions of active series?
This looks super interesting. Question about the scale, I thought Thanos and some other Prometheus variants can handle about 100 million active time series. I would have expected your solution to scale to billions. Have you not pushed it past 100 million or am I maybe missing something.
Fair question. 100M isn't a ceiling, it's what we have seen in that deployment. We have not run a billion series test yet.
The reason we think it scales differently - labels are just columns in Parquet, so there is no per series index that grows with cardinality. In that deployment one label alone has ~2.5M distinct values among 500+ labels, which would be painful for an index based TSDB but here is just a high cardinality column. What drives cost for us is ingestion rate (data points/s) and how much data a query has to scan for a particular time range not series count. Ingest scales horizontally by adding ingestors, and queries prune by time partition and column stats.
A billion series benchmark is on our list, and we'll publish the numbers when we run it.
100M active time series is good information, but what's the data rate for each time series it can handle? One update per minute or 10 per second? There's a factor of 600 difference there. Neither is obviously insanely the wrong update rate.
scrape interval is 15s and sustained ingestion we have seen is ~3M samples/sec that is ~300 TB/day of raw ingest payload, when stored on object store as parquet, the data gets compressed to 99% which makes it 3 TB/day. The 100M figure is total unique series seen over time. For a sense of per metric cardinality, one metric that has the highest cardinality label (2.5 M distinct values) shows ~6M active series per hour.
Low level how does this compare to Victoriametrics, you can get the high cardinality with clickhouse and other columnar dbs but the tradeoff is more ram usage, slower queries, io, etc
Hi HN,
This is Yash, founding team at Parseable (https://github.com/parseablehq).
We've built an open source observability data lake using Rust, that handles high-cardinality data at around 100M time series in production (https://www.parseable.com/blog/how-parseable-handles-100-mil...)
Our architecture is built around columnar design, and we use Apache Arrow for in-memory columnar processing and Apache Parquet for durable columnar storage on S3-compatible object storage. In Parseable, every labels stay as columns in the data instead of becoming a large long-lived per-series index like many TSDBs.
Also, one thing we’ve been thinking about a lot is how observability changes as agents become part of day-to-day engineering workflows. They're not just another service, they produce traces, tool calls, prompts, intermediate decisions, errors, costs, and sometimes sensitive business context.
Observing them matters just as much as observing any other system. But it is equally important to decide where that telemetry data should reside. Our view is that teams should be able to keep these observability data close to them: in their own object storage, under their own retention, access, and compliance controls.
How does the 100M active series deployment looks like? How many ingestors are there? What's each instance size? How big is the querier so that it can query across a metric with millions of active series?
This looks super interesting. Question about the scale, I thought Thanos and some other Prometheus variants can handle about 100 million active time series. I would have expected your solution to scale to billions. Have you not pushed it past 100 million or am I maybe missing something.
We haven't yet tried pushing it to the scale of billions yet. The max that we've gone to is 150-180 million.
Fair question. 100M isn't a ceiling, it's what we have seen in that deployment. We have not run a billion series test yet.
The reason we think it scales differently - labels are just columns in Parquet, so there is no per series index that grows with cardinality. In that deployment one label alone has ~2.5M distinct values among 500+ labels, which would be painful for an index based TSDB but here is just a high cardinality column. What drives cost for us is ingestion rate (data points/s) and how much data a query has to scan for a particular time range not series count. Ingest scales horizontally by adding ingestors, and queries prune by time partition and column stats.
A billion series benchmark is on our list, and we'll publish the numbers when we run it.
100M active time series is good information, but what's the data rate for each time series it can handle? One update per minute or 10 per second? There's a factor of 600 difference there. Neither is obviously insanely the wrong update rate.
scrape interval is 15s and sustained ingestion we have seen is ~3M samples/sec that is ~300 TB/day of raw ingest payload, when stored on object store as parquet, the data gets compressed to 99% which makes it 3 TB/day. The 100M figure is total unique series seen over time. For a sense of per metric cardinality, one metric that has the highest cardinality label (2.5 M distinct values) shows ~6M active series per hour.
How’s this any different then Iceberg?