CapCam: a phone on a cap as a head tracker for games

I wanted to look around in BeamNG and Assetto Corsa by moving my head, but TrackIR is expensive hardware and camera-based solutions need light and LEDs. A phone has a gyroscope, Wi-Fi and can be strapped to the brim of a cap. Opentrack accepts data over UDP and feeds it to games through the TrackIR protocol, so only the phone side was missing.

Problems came quickly. The gyroscope drifts in yaw, because fusion without a magnetometer has no reference to north. The first samples after the sensor starts are stale. The phone can lie flat on the brim, stand upright on the forehead or sit sideways on a helmet, and every position swaps the axes. Android treats the requested sensor rate as a hint, not a guarantee. A screen glowing under the brim is a nuisance, but without it there is no way to press “recenter”. And finally: users type in an address from “whatismyip” or have a VPN on, and the packets never arrive.

I treated drift as a property, not a bug to compensate: rather than fighting physics, recentering is cheap and always at hand. The orientation source is TYPE_GAME_ROTATION_VECTOR (gyroscope plus accelerometer), so pitch and roll stay tied to gravity, and yaw is zeroed with a quaternion offset about the Z axis. For the first 500 ms the tracker returns identity and does not set “forward” until the fusion settles.

I described the wearing position with one model instead of manual axis swaps: the direction the screen faces times the edge that is “up” gives 12 mountings. The head-to-device matrix comes from a cross product, the quaternion from Shepperd’s method, and head orientation is the device’s rotation in the world composed with the head’s rotation relative to the device. The UI keeps three presets, a “phone turned 180°” switch and a custom position.

• Protocol: the 48-byte opentrack packet (six little-endian f64: x, y, z, yaw, pitch, roll) to port 4242, plus a custom 36-byte format with a quaternion, sequence number and timestamp to port 4243 for mods. One DatagramPacket reused, the whole sensor→tracker→UDP pipeline on a single HandlerThread at display priority with a low-latency WifiLock.
• The rate limiter keeps a schedule of “due times” rather than the gap since the last send: a sensor at 101 Hz under a 100 Hz limit does not lose every second sample. A quarter-interval slack, no catching up after a pause. The panel warns when the real rate falls below 80 percent of the chosen one.
• Axis signs matched to opentrack (positive yaw is right) through the decomposition R = Rz(−yaw)·Rx(pitch)·Ry(roll), with a test for continuity across ±180°.
• Black-screen mode: an overlay covers everything including the system bars, brightness 0.01, FLAG_KEEP_SCREEN_ON, and the volume buttons intercepted in onKeyDown recenter with the phone on your head. A tap reveals the UI for 10 seconds.
• The confirmation vibration did not arrive when the user had touch haptics off: vibration attributes switched to USAGE_HARDWARE_FEEDBACK.
• Network detection: ConnectivityManager with NetworkCapabilities recognises a VPN and compares the subnet of the typed address with the phone’s, refreshing every 3 seconds. The message says outright what the PC address should start with.
• Per-axis sensitivity 0.5–3× and a “side boost”: past a threshold (20° by default) the sensitivity rises smoothly and quadratically, so the centre stays calm while a far glance turns the camera further than you can turn your head without losing sight of the screen.

Confirmed in BeamNG.drive, Assetto Corsa, American Truck Simulator and Richard Burns Rally. Sending at up to about 200 Hz with 60 and 100 Hz options for battery, an APK of about 1.2 MB after R8, Android 8.0 and up. To measure jitter and loss I wrote a UDP monitor in Node that reports p50 and p99 inter-packet gaps.