For blind and low-vision (BLV) athletes, working has historically required a bodily tether — whether or not it’s a human information or a painted observe line. In the present day, we’re excited to share how we’re taking steps in the direction of altering that with the Operating Information agent, an accessibility agent that makes use of real-time environmental understanding to assist low-vision athletes run. It marks a large leap from easy path-following to superior, real-time spatial reasoning. As we work to excellent this expertise, our purpose is easy: unassisted independence for each runner.
A hybrid structure for uncompromising security
Constructing on our earlier work with Project Guideline, the Operating Information agent makes use of a chest-mounted Pixel 10 Professional smartphone to view the trail forward and information the person by way of auditory suggestions. As a result of high-speed actions demand excessive belief, we constructed a hybrid, dual-path structure:
- On-device segmentation: Operating completely offline on the Pixel 10’s customized silicon, this mannequin ensures ultra-low latency security. It delivers fast “STOP” alerts and steering cues — heard as directional ticking sounds — so runners preserve a dependable sense of course even with no mobile connection.
- Gemma 4’s superior reasoning: Leveraging Gemma 4 E4B, this path handles complicated multimodal inputs (picture and textual content) for high-level scene understanding completely on machine. To maintain latency low, we use Smarter Body Choice. As a substitute of processing each body, the mannequin solely analyzes “high-entropy” frames — like sudden terrain adjustments or new obstacles — delivering sooner, extremely related teaching.
