Reflections from real conversations with EMS and BESS software teams
If you work in energy management or battery storage, you’ve probably seen it happen. A system that ran fine during early tests starts buckling once it’s rolled out across multiple sites. Metrics lag, ingestion slows, dashboards freeze. Operators complain about gaps or inconsistencies. What changed?
Not the batteries. The data.
Over the past year, I’ve spoken with a lot of EMS and BESS teams — some managing single sites, others scaling across dozens. Different stacks, different use cases, but the same root problem keeps coming up: the data infrastructure that was “good enough” at pilot scale just can’t keep up in production.
Here’s what I’ve learned from those conversations.
The Data Pileup Is Real — and So Are Its Consequences
Battery systems generate an enormous amount of telemetry: voltage, current, temperature, state of charge, inverter output, alarms — all at sub-second intervals, across hundreds of devices per site.
That adds up fast. One team I spoke with was ingesting 400,000 data points per second at a medium-sized site. Their largest deployments? Over 1.5 million points/sec.
At that scale, the database becomes the bottleneck. Ingestion starts to lag. Queries take longer. And when something goes wrong, the backlog is a nightmare to recover from.
Storage Costs Can Spiral Quickly
This is often the biggest concern once telemetry systems move from pilot to production. At scale — across dozens of high-resolution signals, multiple assets, and long retention windows — data volumes explode.
For example, a single site ingesting 1 million data points per second can generate over 80 billion records per day. Even with modest retention targets (e.g., 6 months of raw data), that can translate into petabytes of uncompressed storage — and tens of thousands in monthly infrastructure cost.
Teams frequently underestimate the cost impact of retaining raw data for analytics, safety auditing, and performance tracking. Infrastructure bills rise fast, and engineering teams face pressure to cut corners:
- Downsampling critical metrics
- Shortening retention periods
- Limiting granularity or tags
These compromises reduce the system’s visibility and long-term value.
Efficient compression, fine-grained retention controls, and storage-aware query execution aren’t optional — they’re essential from day one.
Most Systems Don’t Handle Edge–Cloud Well
Nearly every BESS operator needs to buffer data locally and sync it to the cloud later — whether because of unreliable connectivity, latency requirements, or both.
But most databases weren’t designed for that. What teams end up with is a patchwork of message queues, file transfer jobs, and custom scripts — all stitched together to keep things from falling apart.
It works, until it doesn’t.
Lightweight, Container-Friendly Deployments Matter More Than Ever
A lot of EMS providers are trying to shrink their infrastructure footprint. I’ve seen teams deploying entire stacks on a single VM or edge node, trying to make things work with 2–4 cores and a few gigabytes of RAM.
Most time-series databases can’t handle that. They assume you’re running a large centralized instance with room to spare. One engineer told me,
Our database uses half the system memory just sitting there. Before we even ingest a byte.
That’s not sustainable when you’re deploying to 50+ sites.
General-Purpose Tools Only Get You So Far
InfluxDB, Prometheus, Timescale — they all work for simple use cases. But once you need high-frequency ingestion, real-time queries, multi-site sync, and long-term retention… the cracks start to show.
You can keep patching things together. Or you can step back and ask:
Is this really the right tool for what we’re doing?
That’s where a few of the teams we’ve worked with landed. And it’s when they started looking for something purpose-built.
What We’ve Seen Firsthand
We’ve helped EMS teams ingest over a million data points per second. We’ve worked with companies syncing site telemetry across edge and cloud, and others trying to deploy full EMS stacks into containers with less than 4 GB RAM.
We recently spoke with a platform team working on a novel battery system — different from the usual lithium-ion approach. But their challenges were the same: high-frequency telemetry, edge processing, and the need to sync data reliably across environments.
Their engineer said it best:
The data itself isn’t hard. What’s hard is keeping it reliable, queryable, and synced — without building five other systems to support it.
Join Us Live: EMS Data Infrastructure in Action
At TDengine, we’ve been working with EMS and BESS companies to solve these problems without duct tape. To help EMS and BESS teams evaluate telemetry infrastructure at real-world scale, we’re hosting a webinar and two focused workshops:
- Webinar: Scaling EMS Telemetry Systems
A high-level walkthrough of utility-scale battery data challenges and architecture patterns, with lessons from real-world deployments. - Hands-on Workshop 1: Performance Under Load
Run high-ingestion benchmarks (up to 1M pts/sec) to observe ingestion speed, compression, and sync behavior under stress. - Hands-on Workshop 2: Architecture & Deployment
Explore containerized and VM deployments, retention tuning, and modular integration approaches for scalable rollouts.
Each session is practical, hands-on, and designed for teams building or upgrading EMS stacks.
