Computer Vision · Edge AI · Mobile
Aegis Car Tracker
On-Device Follower Detection for Drivers
My roleI built the detection, tracking, plate reading and incident pipeline, and exported the models to run inside the phone app.
The situation
A rear camera sees hundreds of cars on a normal drive. Almost all of them are traffic. One of them might be following you, and a dash-cam will record it without ever saying so.
The brief was a phone app and a rear Wi-Fi camera that notice that one car, and present it as evidence a person can check in seconds, not as a scary red box.
- Everything runs on the phone. No video goes to a server.
- No fabricated data: only show what the pipeline really computes. No make, model, speed or distance, because an uncalibrated camera cannot know them.
- It has to work at night, on a cheap ESP32 camera streaming over its own Wi-Fi.
- iOS and Android need different model formats, and the phone's frame rate drops while it reads plates.
Architecture decisions I made
- D1Score following over time, not per frameA detector finds vehicles; a tracker gives each one a stable id; a scorer weighs four signals over a rolling 10 seconds: is it behind you (a gate that zeroes the score otherwise), how continuously it stayed visible, how steady its distance is, and whether it survived your turns.Trade-off Nothing can fire before 11 seconds: 8 seconds of presence, then the score must hold above 0.6 for 3 more. That delay is the point. A car behind you for 4 seconds is traffic.
- D2Port the tracker and scorer to pure DartThe tracker, follow scorer and plate fusion run inside the Flutter app as a pure Dart package, with parity tests against the Python reference so both give the same answer on the same input.
- D3Read plates from the vehicle crop and fuse the readsThere is no separate plate detector. OCR reads the vehicle crop on the device, and repeated reads of the same car are fused into one consensus plate. The app shows how many reads agreed, never a made-up percentage.
- D4One model, exported for every targetYOLO11 trained on 41,640 road images, exported as CoreML with built-in NMS for iOS (matching PyTorch at 0.963 box IoU), INT8 TFLite for Android, and ONNX and OpenVINO for edge and CPU. A distilled 3.4 MB model runs at 65 FPS on a CPU.
In practice
One follower, from first frame to eventillustrative example, real schema
- DetectThe rear camera's frame arrives over Wi-Fi. The detector finds each bus, car, motorbike or truck and its box.
- TrackThe tracker keeps the same id on the same car across frames. If it vanishes for more than 1.5 seconds, its clock starts again.
- ScoreAfter 8 seconds of continuous presence the car becomes a candidate, and its follow score updates every frame.
- FireThe score crosses 0.6 and holds for 3 seconds. One event fires, with the reason behind the score.
{ "trackId": 7, "continuousSeconds": 12.4, "score": 0.71, "reason": { "position": 1.0, "presence": 1.0, "sizeStability": 0.94, "turnCorr": 1.0 }, "plate": null, "plateVotes": 0 } - EvidenceThe app raises the alert, saves an incident clip, and adds the plate once enough reads agree. Otherwise it says "plate not read".
What I took away
- Turn correlation is the one signal that separates a car on the same road from a car following you. Without the compass, the motorway is full of false followers.
- A weighted score is not a probability. Show it as a band, and show the evidence behind it.
- Test on the real camera at night. The darkest clip still found vehicles in 78% of frames, but only real footage could prove that.
Questions about this project
What problem did Aegis Car Tracker solve?
A driver being followed rarely notices until it is too late, and dash-cams only record; they do not warn. The client needed a system that recognises a follower in real time, on the phone, without sending video to a server.
Who built Aegis Car Tracker, and how does it work?
I did (Qalab Hassnain Agha). I built the detection, tracking, plate reading and incident pipeline, and exported the models to run inside the phone app. How it works: A YOLO11 detector trained on 41,640 road images, a multi-object tracker ported to pure Dart so it runs inside the app, a follow-scorer that weighs how long and how closely a vehicle stays behind, and plate reading on the vehicle crop with multi-read fusion. Models are exported for every target: CoreML with built-in NMS on iOS, INT8 TFLite on Android, ONNX and OpenVINO for edge and CPU. Stack: YOLO11, BoT-SORT tracking, CoreML · TFLite INT8 · ONNX, On-device plate OCR, Flutter, ESP32 Wi-Fi camera, Python.
What results did Aegis Car Tracker achieve?
Delivered with a full client handoff in October 2026. Car AP50 0.822 (mAP50 0.677 across four classes). A distilled 3.4 MB model runs at 65 FPS on a CPU. Validated on recorded day and night drives.
Can I hire Qalab to build something similar to Aegis Car Tracker?
Yes. Contracts from $25 an hour, full-time remote from $3,000 a month, fixed-price prototypes from $600 and builds from $2,000, and fractional CTO retainers from $800 a month. Book a free 30-minute call and I will tell you honestly whether and how I would build it.
Gallery
