Scaling High-Frequency Sensor Data with FastAPI and PostgreSQL: 200Hz Without Falling Over
The upLYFT physiotherapy platform streams motion data from BLE wearables at 200Hz+ per device — every joint angle, every gait cycle, live, while a clinician watches. The first architecture sketch looked like every tutorial: sensor → HTTP POST → insert row → dashboard poll. That design dies at roughly one device.
What shipped instead sustains 500+ concurrent sessions at sub-100ms sensor-to-dashboard latency on Azure. This article is the architecture, the numbers, and the mistakes that cost us the most time.
Rule 1: The Ingest Path Does Almost Nothing
At 200Hz, a device produces a reading every 5 milliseconds. Any work you do synchronously in the ingest path — validation beyond the trivial, ML inference, database writes — is work done 200 times a second, per device. The only jobs of the hot path are: receive, timestamp, buffer, acknowledge.
@app.websocket("/stream/{session_id}")
async def stream(ws: WebSocket, session_id: str):
await ws.accept()
buf = session_buffers[session_id] # in-memory ring buffer
async for frame in ws.iter_bytes():
buf.append(decode(frame)) # decode + buffer: microseconds
# NO db write, NO inference here — consumers handle thatEverything expensive happens in consumers reading from that buffer: a batch writer flushing to PostgreSQL, and the kinematics pipeline computing joint angles and gait symmetry on windows of readings — which is also where our ML classification (92%+ movement accuracy) runs.
Rule 2: Batch Every Database Write
PostgreSQL is spectacular at bulk writes and miserable at 200 single-row inserts per second per device. The batch writer flushes each session buffer on whichever comes first: 250ms elapsed or 500 readings accumulated. One multi-row INSERT (or COPY for bigger batches) replaces hundreds of round-trips.
- At 50 devices × 200Hz, per-reading inserts mean 10,000 transactions/second — connection pool exhaustion and WAL pressure long before CPU limits.
- The same load batched at 250ms is ~200 bulk inserts/second — PostgreSQL barely notices.
- Partition sensor tables by time (daily or weekly) and index with BRIN, not B-tree — sensor data arrives in time order, which BRIN exploits at a fraction of the index size.
Rule 3: Pick Transport by Loss Tolerance, Not Fashion
We ran WebSockets first: ordered, reliable, works through hotel WiFi and hospital proxies. For the highest-rate live streams we added a UDP path, because when a reading is superseded 5ms later, TCP retransmitting a lost one just delays every reading behind it — head-of-line blocking is the real enemy of "real-time".
Rule 4: Offline Is a Feature, Not a Failure
Devices disconnect mid-session — elevators, dead zones, battery saves. The firmware buffers on-device and replays on reconnect with sequence-numbered, idempotent writes; the server deduplicates on (session, sequence). Data loss became a decision we made per data class — live display frames are droppable, clinical summary windows are not — instead of an accident we discovered in production.
The Numbers That Matter
- 200Hz+ sustained ingestion per device over BLE → WebSocket/UDP
- Sub-100ms sensor-to-dashboard latency, end to end, on Azure
- 500+ concurrent sessions on a deliberately boring stack: FastAPI, PostgreSQL, Redis, Docker
No Kafka, no Flink, no dedicated time-series database — not because those tools are bad, but because this scale does not need them yet, and every component you do not run is a component that cannot page you at 3am.
When You Outgrow This
This architecture has clear limits: past a few thousand concurrent devices, you will want a proper message broker between ingest and consumers; past a year of hot data, a time-series database earns its keep. The point is to buy those upgrades with measurements, not fear. If you are architecting a sensor-heavy product and want a second pair of eyes on the pipeline, this is exactly the work I take on — the full case study is on my IoT and real-time data engineering services page.
Frequently Asked Questions
Can FastAPI handle high-frequency sensor data?
Yes — comfortably. Running FastAPI with async WebSocket endpoints, in-memory buffering, and batched PostgreSQL writes, my production system sustains 200Hz+ per device across 500+ concurrent sessions with sub-100ms end-to-end latency. The framework is rarely the bottleneck; per-reading database writes and synchronous processing in the request path are.
Should I use WebSockets or UDP for sensor streaming?
WebSockets by default: ordered, reliable, firewall-friendly, and browser-compatible. Switch the hot path to UDP when your data is high-rate and loss-tolerant — live motion streams where the next reading arrives in 5ms make retransmitting a lost one pointless. Many production systems run both: UDP for the live stream, WebSockets for control and events.
Do I need TimescaleDB or a dedicated time-series database?
Not as early as vendors suggest. Plain PostgreSQL with batched inserts, time-based partitioning, and BRIN indexes handled clinical-grade 200Hz workloads in my deployments. Move to a dedicated TSDB when retention policies, continuous aggregates, or multi-year history become painful — not before measuring.
Code, architecture patterns, and recommendations in this article come from real projects but are shared as-is, without warranty — validate them against your own requirements before production use. See the Terms of Use.
Available for Consulting
Let's build something
that matters.
I take on a select number of project-based consulting engagements per quarter — from architecture reviews and LLM pipeline audits to full production builds.
80+ clients · 14+ production systems · Remote / Islamabad