How it works
Three architectures failed before this one
NaviGuider was built by an explicit loop: specify, plan, implement, unit-test, field-walk, capture failures, diagnose, fix, pin with a regression test. The design that survived is best explained by the ones that did not.
What was tried, and why it was replaced
Generation 1
Region-scored depth heuristic
Superseded
Ratio of near pixels in the left, centre, and right thirds of the raw depth map, speaking stop, move left, or move right.
Failed within the first indoor walks. The floor is the largest near surface in every frame, so ordinary ground triggered constant stops. A region ratio has no concept of terrain: it cannot tell a wall from a curb from a descending staircase, and cannot remember anything.
Generation 2
Calibrated global ground plane
Abandoned
A per-user chest-mount calibration step and a single RANSAC-fitted ground plane, with hazards classified as deviations from it.
Abandoned for two reasons that shaped everything after. Per-user calibration is a usability defect — mount height varies between people and within a single walk. And one plane is the wrong model of a street: real pavement is crowned, sloped, and stepped, so residuals grow with distance and read as phantom bumps.
Generation 3
World-anchored terrain grid
Active
Depth back-projected through camera intrinsics and the ARKit world transform into a persistent top-down 7.5 cm grid, fused from plane anchors, mesh, and depth-RANSAC, with multi-pass per-cell classification.
Active. Camera height is estimated continuously with a moving average instead of being calibrated once. Every later capability — small steps, stairs, people, drop-off truthfulness, planning — is a refinement of this grid.
Generation 3+
Semantic object layer
Active, with authority limits
YOLO11n running on the Apple Neural Engine, fused with LiDAR geometry so labels annotate the grid's hazards.
Adopted under one safety rule: camera semantics never clear LiDAR occupancy. A label can name an obstacle and add conservative cost. It can never say the ground is fine.
Generation 4–5
GuideSpace and the research stack
Shadow, default off
World-anchored 10 cm vertical evidence bins, body slices, body-swept candidate motions, dynamic tubes, curb-latched topological boundaries, multi-hypothesis route belief, maneuver witnesses, attention budgeting, consented route memory.
Implemented in shadow mode only. Every component publishes a typed result or a typed refusal into synchronized replay, and none of them can steer, speak, or grant crossing permission.
Making “drop ahead” trustworthy
Drop ahead is the highest-priority thing the app can say, so its false positives are the most corrosive. Field walks produced five distinct ways to generate a false one, each needing its own countermeasure.
The LiDAR coverage boundary
Floor returns die at two to three metres under grazing incidence, so every column eventually reads floor, then nothing — indistinguishable to naïve logic from a cliff edge that follows you and re-announces forever.
Fix · Drop promotion is capped by measured floor coverage range, not field-of-view geometry.
Discarded low-confidence depth
ARKit marks grazing-range floor low-confidence even where depth is dense, and the sampler was throwing it away, starving coverage.
Fix · Low-confidence samples may reinforce floor only, within 0.12 m of the fitted plane. They can extend the ground; they can never mint a hazard.
Shallow potholes
A 12 cm pothole is a step down, not a cliff.
Fix · Down-steps under 20 cm whose far side returns to floor within 45 cm are demoted. A true drop must span four cells across three columns — a real edge crosses the corridor.
Sensor blanks on dark or wet pavement
LiDAR returns vanish on absorptive surfaces, mimicking missing floor.
Fix · A camera-luma veto compares the candidate patch against confirmed floor. Same brightness and texture means continuous pavement. Critically, the veto fails toward warning: at night, out of frame, or in replay it never suppresses.
Sloped and crowned roads
A single plane misreads legitimate road geometry as a series of steps.
Fix · A local quadratic ground surface, fitted near the traveler instead of across the whole scene.
Frame-rate flip-flopping
Early versions changed command at frame rate as detections came and went.
Fix · A command must persist in 8 of the last 12 frames — about 400 ms — before it commits. Under an idealised independence assumption that is a false-command rate of 5.8 × 10⁻⁴ even at a pessimistic per-frame error rate of 0.2, at the cost of 270 ms of delay.
E2Unit and synthetic tests establish the intended gates and developer observations motivated the fixes. They do not prove a zero false-alert rate. The stronger operating rule is that unknown stays unknown: the system slows, stops, or asks for a scan when risk cannot be resolved.
Navigation without a compass
A chest mount sits next to magnets, which makes the magnetometer unusable — a mundane physical constraint that invalidates the obvious approach to heading. NaviGuider fuses GPS course with AR displacement instead, and gates the audio homing beacon on confidence so it goes quiet rather than pointing you somewhere plausible.
One class of bug here is worth naming because it is so easy to ship: a coordinate-sign error that mirrored steering, sending the traveler left when the corridor opened right. It is now pinned by a regression test. Conventions need tests.
Performance, and why the phone froze
An early real-time build froze the target iPhone within half a second of launch. Three separate faults were interacting:
- Per-frame CPU-side image conversion allocated about 11 MB and forced a GPU readback on the main thread for every 1920 × 1440 frame. RealityKit now renders the camera feed directly on the GPU.
- The pipeline ran synchronously on ARKit’s delegate queue, so a slow frame made ARKit retain a dozen in-flight frames, exhaust its fixed pool, and stop the camera. The delegate queue now does only constant-time work, and at most one frame is ever in flight.
- Fixing the freeze unmasked a latent integer-overflow crash that had been unreachable while the app was frozen — a reminder that in a failing system, later bugs hide behind earlier ones.
A thermal governor steps intake from 30 to 20 to 15 Hz under thermal pressure while stretching hazard-persistence windows in wall-clock time, so throttling makes the system more conservative, never twitchier.
Crossings
The crossing subsystem is implemented for shadow recording only and cannot reach output. Its entry gate is closed because validated pedestrian-signal, crosswalk, curb-ramp, and roadway models do not exist yet.
One field finding explains the severity of that gate: a vehicle-facing green light could be heard as permission to cross. The feature is disabled, and the rule is now explicit — vehicle green must never authorise road entry. The phrase “safe to cross” is not in the app’s vocabulary, and neither is an unqualified “go.”
Where the computation happens
Live perception and guidance run on the device, with no network connection required. Optional research reports are consent-gated and separated from operation — see privacy and the research program.
What this costs you
Continuous camera, depth, and on-device inference is expensive. Active navigation draws heavily on the battery and warms the device, and sustained use in heat will hit thermal limits. Development centred on one phone model and one mount family; a controlled latency, energy, and thermal benchmark across devices has not been completed, so no universal frame-rate or latency figure is claimed.