Research paper
Affordable Smartphone-Based Assistive Navigation Using LiDAR and Haptic Feedback
47 pages on architecture, algorithms, failure analysis, and a staged evaluation framework — deliberately not on clinical efficacy, because that has not been measured.
Read it here, in full
The complete manuscript — every figure, the full failure catalog, the evaluation protocol, and the reference list. Nothing on this site says anything the paper does not.
- Authors
- Noah Hellyer and Aryaveer Sheoran
- Institution
- Cherry Creek High School, Greenwood Village, Colorado
- Adult sponsor
- Ethan Dusto
- Venue
- Regeneron International Science and Engineering Fair
- Category
- Systems Software / Biomedical Engineering
- Length
- 47 pages
47 pagesPDF366 KB
Opens in your device’s own PDF reader, where search, zoom, and VoiceOver page navigation all work properly.
What the paper reports
An experimental chest-mounted smartphone navigation aid that combines LiDAR depth, ARKit visual–inertial odometry, an on-device YOLO11n detector, local terrain classification, route context, and distance-encoded audio pulses prototyping a planned haptic-shoe interface.
Its central argument is that the failure catalog, the conservative evidence rules those failures produced, and the separation of active from experimental safety layers are the project’s most transferable contributions — more so than any individual capability.
Paper, abstract and section 1
Two asymmetric authority rules
Camera semantics may annotate occupied evidence but may never clear LiDAR occupancy. Route preference may rank only candidates already admitted by geometric, topological, and crossing constraints.
A shadow research stack
Every research component publishes a typed result or a typed refusal into synchronized replay and remains shadow-only. It cannot steer, speak, or grant crossing permission.
Verification, stated exactly
1,518 of 1,519 simulator tests passed across 134 test files, 1 skipped (one environment-gated Supabase integration test), none failed. Compiler warnings remain, so this is not reported as a warning-free build.
What is not claimed
No participant data and no sustained-device evidence. The paper reports architecture, algorithms, failure analysis, and an evaluation framework rather than clinical efficacy.
The failure catalog
Every capability was walked outdoors. Failures were recorded as replayable clips, diagnosed to a root cause, fixed, and converted into automated tests so none could silently return. A representative selection follows; the full table is in the PDF.
Paper, section 7
| Observed failure | Root cause | Resolution |
|---|---|---|
| Missing depth announced as a drop-off | No LiDAR return was treated the same as measured empty space | Missing data became its own state; drop promotion capped by measured floor-coverage range |
| Phantom bumps on ordinary pavement | One globally fitted ground plane cannot represent crowned or sloped streets | Local quadratic ground surfaces, fitted near the traveler |
| Receding walkers left obstacle trails | Person cells entered the static hazard memory; there was no receding class | Leader-following rule; person cells retyped before the map ingests them |
| Steering was mirrored | A coordinate-sign error inverted the across-track axis | Pinned by a regression test — conventions need tests |
| The phone froze half a second after launch | Synchronous work on ARKit's delegate queue exhausted the frame pool | Constant-time delegate queue, at most one frame in flight |
| Wall-avoidance turns amplified drift | Half-field openness counted unobserved cells as open | Course-first turns; corridor probes trust only positively observed floor |
| Heading was unusable | Chest-mount magnets made the magnetometer useless | GPS-course and AR-displacement fusion; beacon gated on confidence |
| The detector hallucinated at night | A COCO-trained model run on sensor-noise frames | Luma-tiered stand-down; the dusk band requires LiDAR corroboration |
| A vehicle-facing green light could be heard as permission | Signal colour lacked pedestrian-signal and intersection context | Feature disabled. Vehicle green must never authorise road entry |
| The attention budget could silence an emergency | Budget rejection ran before the emergency short-circuit | Safety objections stay first; emergency and health cues bypass attention economics |
| Hysteresis changed under thermal load | Promotion and retention were counted in frames while frame rate varied | Safety timing converted to elapsed-time windows |
| A near minor obstacle could hide a farther severe drop | Nearest-first early return ended command evaluation before severity arbitration | Open audit finding — not yet resolved |
Evidence layers
The paper requires every claim to state which layer it rests on. This site uses the same labels, because the distance between “the tests pass” and “it helped a traveler” is the entire remaining project.
Paper, table 5
E0
Proposal
Proposed. No implementation claim.
E1
Code or design review
The architecture is designed to behave this way.
E2
Unit, synthetic, build, or replay verification
The tested implementation obeyed the stated cases.
E3
Developer physical-device observation
Observed during engineering use. Not a controlled study.
E4
Supervised participant study
A participant outcome claim, within the tested operating domain.
E5
Independent or longitudinal validation
Generalisation beyond the original team and routes.
NaviGuider currently holds evidence at E1 to E3. The last three rows are where it is not.
Relation to prior work
The paper is explicit that a first-of-kind claim would be inappropriate. NaviSense demonstrates a contemporary multimodal combination of mobile vision, AR, LiDAR, and audio–haptic feedback with blind and low-vision participants, and other recent work reports participant outcomes using scene understanding.
What NaviGuider contributes instead is narrower and more specific: terrain-specific failure modes, source-authority rules, body-height evidence representation, and a reproducible incident-to-regression workflow.
Planned human-participant evaluation
The engineering phase makes the hypothesis testable. The protocol, modelled on the collision-warning randomized trial that motivates the approach, proceeds only through staged safety gates:
- Deterministic adversarial tests and a measured obstacle-course benchmark.
- Ordinary, non-blindfolded engineering walks by trained staff with a spotter.
- Blind and low-vision participant work — only after SRC/IRB approval, accessible consent, predefined stopping rules, and orientation-and-mobility supervision.
The design is within-subjects and counterbalanced: each participant keeps their usual mobility aid, and is measured with it alone against it plus NaviGuider. Primary outcomes are environmental contacts and adjudicated near misses per 100 m, wrong turns and overshoots, experimenter interventions, and successful completion. Completion time and an accessible workload measure are guardrails, not goals. Unsupervised street-crossing permission is outside the study domain.
Audio-only, haptic-only, and combined conditions may be added — but only after the shoe hardware is independently verified. More on taking part is on the research program page.
Acknowledgments and AI assistance
The paper states its tooling plainly, and so should this page: generative AI coding assistants were used for implementation suggestions, refactoring, test generation, debugging, design critique, documentation, and research-record synthesis. The student researchers defined the research questions and requirements, reviewed generated material, ran and interpreted the tests, and remain responsible for the code, citations, safety decisions, data, and manuscript. No external funding was received, and no human participants were involved in the work reported.