Scaling High-Frequency Sensor Data with FastAPI and PostgreSQL: 200Hz Without Falling Over
The upLYFT physiotherapy platform streams motion data from BLE wearables at 200Hz+ per device. Every joint angle, every gait cycle, live, while a clinician watches. The first architecture sketch looked like every tutorial: sensor → HTTP POST → insert row → dashboard poll. That design dies at roughly one device.
What shipped instead sustains 500+ concurrent sessions at sub-100ms sensor-to-dashboard latency on Azure. This article is the architecture, the numbers, and the mistakes that cost us the most time.
Rule 1: The Ingest Path Does Almost Nothing
At 200Hz, a device produces a reading every 5 milliseconds. Any work you do synchronously in the ingest path (validation beyond the trivial, ML inference, database writes) is work done 200 times a second, per device. The only jobs of the hot path are: receive, timestamp, buffer, acknowledge.
@app.websocket("/stream/{session_id}")
async def stream(ws: WebSocket, session_id: str):
await ws.accept()
buf = session_buffers[session_id] # in-memory ring buffer
async for frame in ws.iter_bytes():
buf.append(decode(frame)) # decode + buffer: microseconds
# NO db write, NO inference here: consumers handle thatEverything expensive happens in consumers reading from that buffer: a batch writer flushing to PostgreSQL, and the kinematics pipeline computing joint angles and gait symmetry on windows of readings, which is also where our ML classification (92%+ movement accuracy) runs.
Rule 2: Batch Every Database Write
PostgreSQL is spectacular at bulk writes and miserable at 200 single-row inserts per second per device. The batch writer flushes each session buffer on whichever comes first: 250ms elapsed or 500 readings accumulated. One multi-row INSERT (or COPY for bigger batches) replaces hundreds of round-trips.
- At 50 devices × 200Hz, per-reading inserts mean 10,000 transactions/second: connection pool exhaustion and WAL pressure long before CPU limits.
- The same load batched at 250ms is ~200 bulk inserts/second: PostgreSQL barely notices.
- Partition sensor tables by time (daily or weekly) and index with BRIN, not B-tree: sensor data arrives in time order, which BRIN exploits at a fraction of the index size.
Rule 3: Pick Transport by Loss Tolerance, Not Fashion
We ran WebSockets first: ordered, reliable, works through hotel WiFi and hospital proxies. For the highest-rate live streams we added a UDP path, because when a reading is superseded 5ms later, TCP retransmitting a lost one just delays every reading behind it: head-of-line blocking is the real enemy of "real-time".
Rule 4: Offline Is a Feature, Not a Failure
Devices disconnect mid-session: elevators, dead zones, battery saves. The firmware buffers on-device and replays on reconnect with sequence-numbered, idempotent writes; the server deduplicates on (session, sequence). Data loss became a decision we made per data class (live display frames are droppable, clinical summary windows are not) instead of an accident we discovered in production.
The Numbers That Matter
- 200Hz+ sustained ingestion per device over BLE → WebSocket/UDP
- Sub-100ms sensor-to-dashboard latency, end to end, on Azure
- 500+ concurrent sessions on a deliberately boring stack: FastAPI, PostgreSQL, Redis, Docker
No Kafka, no Flink, no dedicated time-series database, not because those tools are bad, but because this scale does not need them yet, and every component you do not run is a component that cannot page you at 3am.
When You Outgrow This
This architecture has clear limits: past a few thousand concurrent devices, you will want a proper message broker between ingest and consumers; past a year of hot data, a time-series database earns its keep. The point is to buy those upgrades with measurements, not fear. If you are architecting a sensor-heavy product and want a second pair of eyes on the pipeline, this is exactly the work I take on. The full case study is on my IoT and real-time data engineering services page.
The full platform is in the upLYFT case study. If your backend struggles with sensor data, my IoT data engineering work usually starts with an audit of the ingest path.
Frequently Asked Questions
Can FastAPI handle high-frequency sensor data?
Yes: comfortably. Running FastAPI with async WebSocket endpoints, in-memory buffering, and batched PostgreSQL writes, my production system sustains 200Hz+ per device across 500+ concurrent sessions with sub-100ms end-to-end latency. The framework is rarely the bottleneck; per-reading database writes and synchronous processing in the request path are.
Should I use WebSockets or UDP for sensor streaming?
WebSockets by default: ordered, reliable, firewall-friendly, and browser-compatible. Switch the hot path to UDP when your data is high-rate and loss-tolerant: live motion streams where the next reading arrives in 5ms make retransmitting a lost one pointless. Many production systems run both: UDP for the live stream, WebSockets for control and events.
Do I need TimescaleDB or a dedicated time-series database?
Not as early as vendors suggest. Plain PostgreSQL with batched inserts, time-based partitioning, and BRIN indexes handled clinical-grade 200Hz workloads in my deployments. Move to a dedicated TSDB when retention policies, continuous aggregates, or multi-year history become painful, not before measuring.
Code, architecture patterns, and recommendations in this article come from real projects but are shared as-is, without warranty. Validate them against your own requirements before production use. See the Terms of Use.
Taking on new engagements
Need this built
to survive production?
I take on project-based engagements, from architecture reviews and LLM pipeline audits to full production builds, plus fractional CTO work.
6 P@SHA ICT Awards · 23 products shipped · 80+ clients