The route, and everything in the way of it.
Terrain, obstacles, and spoken turn-by-turn directions for blind travelers, on one chest-mounted iPhone.
In engineering evaluation. Not yet on the App Store. Read the limitations.
It starts quiet.
One spoken line, then nothing. Speech is kept for names, route instructions, and urgent stops.
Spoken“Head north on Bryant. Sidewalk is clear.”
A bench becomes a pulse.
Left ear means move left. The ticks quicken as it nears, the way a parking sensor does. Nothing is said.
SpokenNothing.
People are not terrain.
Someone crossing is tracked as a moving thing with a predicted path, never written into the map as an obstacle.
Spoken“Person crossing, slow down.”
Unknown stays unknown.
Depth stops returning at 3.6 m on a wet, absorptive surface. NaviGuider does not guess. It says so, and slows.
Spoken“Surface unclear ahead. Slowing guidance.”
A curb, measured.
A 0.14 m step down, held in world coordinates so it stays where it is when you turn. Both sides, quickening.
SpokenNothing.
At the crossing, the call stays yours.
NaviGuider tells you what it measured: the curb, the step down, the lanes ahead. It does not read signals or traffic, so it never says go. Your cane, your ears, and your training make that call.
Sound is a loan. Haptics pay it back.
Speech takes seconds to carry what a pattern carries in a tenth of a second, so continuous guidance moved to stereo pulses — the interface a driver already knows from a parking sensor.
But pulse audio occupies a channel blind travelers depend on. So the encoding was built from the start to move to hardware: side becomes which foot, rate becomes pulse rate, tone becomes motor site. Swapping vibration for sound changes the transducer, not the code and not the traveler’s mental model.
How the cues are designedE0The haptic shoes are a proposal. No hardware has been built or evaluated, and prototyping them is item eight on the paper’s future-work list.
Drag the hazard
- Side
- Left earMeans move leftShoes: Which foot
- Rate
- 0.88 sHazard at 2.2 mShoes: Pulse rate
- Tone
- 440 HzObstacleShoes: Motor site
Left and right need headphones. Sound stays off until you turn it on, and it also speaks the spoken lines in the walk above.
Two jobs, one voice
Getting somewhere and not walking into something are usually different products. NaviGuider is both, in one voice.
Splitting them across two apps means the traveler arbitrates between them mid-street, which is exactly the wrong place to do it.
Routing that survives the last hundred metres
GPS walking routes get you to the block. Body-relative turn cues take it from there — and they work without a usable magnetometer, because a chest mount sits next to magnets that make one useless. Heading comes from fusing GPS course with AR displacement instead.
- Walking routes from MapKit and CoreLocation
- An audio homing beacon, gated on confidence
- A* local planning through the corridor ahead
- On-demand descriptions of what is around you
- Guidance continues offline once a route is active
Terrain, not pixel statistics
The first prototype counted near pixels in the left, centre, and right of the depth map. The floor is the largest near surface in every frame, so ordinary ground triggered constant stops. What replaced it classifies terrain per cell in world coordinates, and remembers a hazard after it leaves the field of view.
- Obstacles
- ≥ 30 cm
- Curbs and steps
- 10–30 cm
- Small steps
- 4–15 cm
- Drop-offs
- ≥ 15 cm
- Ascending and descending stairs
- as a structure, not a surface
- Moving pedestrians
- retyped before map ingest
E2Test-backed logic, including targeted look-alike cases — a curb versus a trash can, a staircase versus a wall, a pothole versus a drop-off. Detection precision and range error in the field have not been measured.
Six technologies, and what each one changed
Electronic travel aids have been studied for sixty years, and reviews keep reaching the same verdict: they stay in the laboratory, they need dedicated hardware, and they cost more than the people who need them can pay. Four of these six pieces did not exist on a consumer phone five years ago.
LiDAR scene depthApple ARKit
Measured distance, not inferred distance. A curb is 0.14 m down because the sensor measured it, which is what lets a cue be specific instead of hedged. Six decades of electronic travel aids could not put a depth sensor in a traveler's pocket at consumer prices.
256 × 192 depth map at up to 60 Hz, with per-pixel confidence
Visual–inertial odometryARKit world tracking
Hazards can be remembered in world coordinates instead of frame coordinates. A curb stays where it is when you turn your head, and the terrain grid accumulates evidence across a walk rather than starting over every frame.
Camera + IMU fusion, centimetre-scale relative accuracy, gravity-aligned transforms, plane anchors
YOLO11n on the Neural EngineCore ML · Apple Neural Engine
Object recognition runs on silicon physically separate from the CPU and GPU, so naming what is ahead costs nothing that real-time geometry needs. A larger YOLO11x teacher auto-labels collected field data offline.
2.6 M parameters, FP16, square 640 input, embedded NMS, pinned to the ANE
World-anchored terrain gridNaviGuider
The third architecture generation, and the one that survived. Depth is back-projected into a grid fused from plane anchors, mesh, and depth-RANSAC, so terrain is classified per cell against a local ground model rather than one global plane.
Persistent top-down 7.5 cm grid, multi-pass per-cell classification
GuideSpace vertical evidenceNaviGuider · research layer
Risk becomes body-relative. An overhang at head height and a bollard at knee height are different problems, and a configured body volume can be swept through candidate motions to test whether the traveler actually fits.
World-anchored 10 cm vertical bins, free-space rays, traveler-relative body slices
Distance-encoded pulse audioAVFoundation
The interface a driver already knows from a parking sensor, carrying continuous guidance without speech. Critically, its three channels map one-to-one onto the planned haptic shoes, so swapping sound for vibration changes the transducer, not the code or the traveler's mental model.
Stereo side, pulse rate, and tone — three independent channels
Five rules, each one bought with a bug
Every capability was walked outdoors on real sidewalks, slopes, stairs, and streets. Failures were recorded as replayable depth clips, diagnosed to a root cause, fixed, and pinned with a regression test so they could not silently return.
Unknown stays unknown
Treating a missing LiDAR return as a drop created phantom cliffs; treating it as free space would have been more dangerous. Missing data gets its own state, and unresolved risk produces slow, stop, or scan — never an unsupported all-clear.
Authority is architectural, not a threshold
Camera semantics may annotate occupied evidence and add conservative cost. They may never clear LiDAR occupancy. That is a structural rule enforced by invariant tests, not a confidence number someone can tune.
Local terrain models beat global ones
A single fitted ground plane failed on crowned and sloped pavement, and absolute step heights failed against cross-slope. Real streets need local surfaces and local height discontinuities.
Moving people need a temporal policy
Pedestrians were being written into the static hazard map, leaving phantom obstacle trails behind anyone who walked away. People are now retyped before the map ingests them.
Safety timing is measured in seconds, not frames
Frame-count dwell windows changed behaviour under thermal throttling. Promotion, retention, and recovery are expressed in elapsed time so a hot phone gets more conservative, never twitchier.
Missing depth read as a cliff. A global floor model failed on sloped pavement. People walking away left phantom obstacle trails. A coordinate-sign error inverted steering. Each one is in the paper with its root cause and its fix. Read the failure catalog
The affordability gap is structural
Smart canes, ultrasonic wearables, and camera glasses run from hundreds to several thousand dollars. A trained guide dog costs tens of thousands to raise. Any solution that needs dedicated hardware inherits a dedicated-hardware price — which is the argument for building this on a phone people already own.
- 2.2 billion
- people live with a vision impairmentWHO World report on vision, 2019
- 63%
- of surveyed travelers with visual impairments had been injured while navigating outdoorsEl-Taher et al., IEEE Access, 2023
- 37%
- fewer obstacle contacts in a randomized trial of wearable collision warning with 31 blind and low-vision participantsPundlik et al.
- 89%
- of visually impaired people live in low- and middle-income countries, where dedicated hardware is out of reachAckland, Resnikoff & Bourne, 2017
What we can prove, and what we cannot
The current verification run passes 1,518 of 1,519 tests across 134 Swift test files, with 1 skipped — one environment-gated Supabase integration test — and none failed. Invariant tests enforce the structural rules: that camera semantics never clear LiDAR occupancy, that route confidence never revives a rejected path, that shadow components never reach the guidance bus.
A test can show that code obeys a specification. It cannot show that the specification covers every street, or that a traveler understood the resulting cue. So every claim on this site carries the grade of evidence behind it.
- E0
- ProposalProposed. No implementation claim.
- E1
- Code or design reviewDesigned to behave this way.
- NaviGuider today
- E2
- Test or replay verificationThe implementation obeyed the stated cases.
- NaviGuider today
- E3
- Developer device observationSeen during engineering use. Not a study.
- NaviGuider today
- E4
- Supervised participant studyA participant outcome claim.
- E5
- Independent validationGeneralises beyond our team and routes.
Everything NaviGuider currently claims sits at E1 to E3. The last two rows are where it is not yet.
Affordable Smartphone-Based Assistive Navigation Using LiDAR and Haptic Feedback
Forty-seven pages on the architecture, the algorithms, the failure analysis, and a staged evaluation framework — deliberately not on clinical efficacy, because that has not been measured. The whole manuscript is readable in the page.
Noah Hellyer and Aryaveer Sheoran · Cherry Creek High School, Greenwood Village, Colorado · Regeneron International Science and Engineering Fair
Not on the App Store yet
NaviGuider is in the engineering phase. Release waits on a staged sequence: adversarial tests, a measured obstacle-course benchmark, engineering walks with a spotter, and only then supervised work with blind and low-vision participants under approval and O&M supervision. The download button goes live when that does.
NaviGuider is an experimental assistive travel aid. It is not a safety system and not a medical device. It does not detect every obstacle, does not read every crossing signal, and does not know the state of traffic. Grazing-angle LiDAR coverage is short and nonuniform; transparent, reflective, dark, wet, rain-covered, and snowy surfaces remain difficult. The object detector is COCO-bound. Development centred on one phone model and one mount family. The crossing subsystem is shadow-only and cannot reach output. Guidance degrades with sensor obstruction, low light, weather, and thermal limits, and battery use is high during active navigation. Descriptions on this page reflect a system under development; capabilities may change.