# Blind Users Built 37 of Their Own AI Helpers in Two Months, From Uber Finders to Chess Checkers

In a new ASSETS 2026 study, five blind and low-vision co-designers asked an AI coding agent for the tools they needed and built 37 of them. Here's what they made, what users say, and why accessible sites are agent-ready sites.

Canonical: https://robauto.ai/blog/blind-users-build-ai-helpers

Robauto Team · Published Nov 3, 2026 · Last updated Nov 3, 2026

In a new ASSETS 2026 study, five blind and low-vision co-designers asked an AI coding agent for the tools they needed and built 37 of them. Here's what they made, what users say, and why accessible sites are agent-ready sites.

Five blind and low-vision co-designers spent two months doing something that used to require a software team. They described the help they wanted in plain language, an AI coding agent wrote the tool, and they tested it on their own phones. By the end of the study they had built 37 camera-based assistive tools through 68 rounds of improvements, including a car finder for spotting their Uber and a reader that sorts real mail from junk.

The research, called ProgramAT, appears at ASSETS 2026, the ACM SIGACCESS conference on accessibility, and it is one of several new studies this fall showing AI becoming a genuinely personal helper for people who are blind or have low vision. Here is what the researchers found, what users themselves say, and why the lesson matters for every brand with a website.

## **What the ProgramAT co-designers actually built**

ProgramAT pairs a mobile app with an AI coding environment (GitHub Copilot fitted with extra scaffolding) and a compute server. A user types or speaks a request such as "tell me the weight on my kitchen scale in grams," the coding agent writes the code, and the finished tool runs on the phone's camera. Each tool talks back through speech, audio tones, vibration, or a mix, depending on what the user asked for.

The study ran in two phases: two co-designers explored the system first, then five built and refined tools over four more weeks, keeping diary entries and joining weekly check-ins. The team collected more than 24 hours of interview data along the way. Unlike general-purpose describer apps, the authors note, ProgramAT generates and runs task-specific code instead of relying on fixed features.

Tool a co-designer builtWhat it doesWhat they did beforeSource

Car identifierHelps find the right Uber more reliablyCalling a visual-interpreter service or a sighted friend[ProgramAT paper](https://arxiv.org/abs/2607.21760)
Mail sorterTells relevant mail from junkTaking repeated photos with a describer app[ProgramAT paper](https://arxiv.org/abs/2607.21760)
Kitchen scale readerReads the weight in gramsA dedicated talking scale[ProgramAT paper](https://arxiv.org/abs/2607.21760)
Bus stop detectorConfirms whether a bus stop is there, and whereMaps for most of the trip, then asking someone[ProgramAT paper](https://arxiv.org/abs/2607.21760)
Chess move validatorChecks moves during a gameStreaming the board to a sighted friend[ProgramAT paper](https://arxiv.org/abs/2607.21760)
Clockface food descriptionSays where food sits on a plate using clock positionsFeeling around with a fork[ProgramAT paper](https://arxiv.org/abs/2607.21760)
Room name identifierReads room signs aloudRunning a hand along the wall to find braille signs[ProgramAT paper](https://arxiv.org/abs/2607.21760)

The researchers say some of these tools meet needs that no commercial assistive product covered, with the Uber finder and a hand-gesture interpreter as examples. Co-designers started with everyday tasks and moved toward more personal and playful ideas as they got comfortable, including a tool to find one co-designer's plush dog. The paper is candid about limits too: model capability, conflicting instructions, and hallucinations still made creation a back-and-forth conversation rather than one-shot magic. That honesty is part of why the result is encouraging. People stayed in charge, and the agent did the typing.

## **Two more studies point the same direction**

### **Charts that talk back for low-vision readers**

A second ASSETS 2026 paper tested ChartAEye, which lets low-vision users share their magnified view and point of interest with an AI assistant while they explore a chart. Twelve participants compared a generic assistant with one designed around low-vision needs and lived experience. Eight of the 12 preferred the tailored version, whose answers far more often included spatial guidance (71.2% versus 7.8%) and explained what visual elements meant (84.8% versus 29.1%).

### **Videos you can ask questions about**

A third project, presented in the ICMI 2026 companion proceedings, built a Multimodal Agent Video Player that adds a conversational accessibility mode to video. After 18 design sessions with blind and low-vision users, the team found people wanted independence and agency over how they watch, not just extra features. In a follow-up with eight participants, the chat interface built trust, and when the model made a mistake, talking openly about its limits helped repair that trust.

## **Users are already relying on AI every day**

This is not just lab work. The American Foundation for the Blind surveyed 1,735 U.S. adults, and its August 2026 report on AI as assistive technology shows how mainstream these tools have become for blind and low-vision people.

FindingNumberSource

Blind and low-vision respondents who use AI visual description79%[AFB, Innovation for Access](https://afb.org/research-and-initiatives/ai-series/innovation-access)
Of those users, share who use it at least daily51% (another 36% weekly)[AFB](https://afb.org/research-and-initiatives/ai-series/innovation-access)
Most common use: reading text93% of blind and low-vision users[AFB](https://afb.org/research-and-initiatives/ai-series/innovation-access)
All describer users rating output extremely or mostly accurate65% (8% plus 57%)[AFB](https://afb.org/research-and-initiatives/ai-series/innovation-access)
Users who would pick AI over a human reader for sensitive text if the tool doesn't save or share images60%[AFB](https://afb.org/research-and-initiatives/ai-series/innovation-access)
Low-vision users preferring a tailored chart assistant8 of 12[ChartAEye, ASSETS 2026](https://doi.org/10.1145/3797867.3841222)

The same report is clear about what still needs work: 21% of blind and low-vision describer users said an AI mistake had hurt them, often on high-stakes items like tax forms or medication labels that weren't accessible to begin with. The AFB's recommendations are practical: improve accuracy, let users control detail, protect privacy with on-device options, and follow current Web Content Accessibility Guidelines. Every one of those is a chance for builders and brands to make things better.

## **The Robauto takeaway: your site is being read by AI on someone's behalf**

Notice what all of these tools do. They take something built for eyes (a sign, a chart, a scale display, a web page) and turn it into words or actions for a person. The more a source is already made of clear text and labeled parts, the less the AI has to guess. The errors people reported often came from material that was hard to read in the first place.

That is the same job an AI shopping or research agent does when it visits your website. Good accessibility and good agent readiness overlap a lot, which makes this a two-for-one opportunity:

- **Put key facts in real text.** Prices, hours, sizes, return windows, and contact routes should not live only inside images or PDFs.

- **Write meaningful alt text** for product and instructional images, so both screen readers and AI describers say something useful.

- **Label buttons and form fields** clearly, so a screen reader user or an agent knows what "Submit" will actually do.

- **Use real headings and structured data** so a page's outline and product details can be read without guessing.

- **Publish clear policies** in plain language, since users and agents both need to verify details before acting.

Brands that do this help disabled customers directly and make it easier for AI assistants to recommend and use them accurately. It's one of the rare upgrades where doing right by people and doing well in AI search point the same way.

## **Agent-ready summary**

```
`{
"topic": "AI agents as personal assistive technology",
"studies": [
{"name": "ProgramAT", "venue": "ASSETS 2026", "co_designers": 5, "duration": "two months", "tools_built": 37, "iterations": 68, "source": "https://arxiv.org/abs/2607.21760"},
{"name": "ChartAEye", "venue": "ASSETS 2026", "participants": 12, "preferred_tailored_assistant": "8 of 12", "source": "https://doi.org/10.1145/3797867.3841222"},
{"name": "Multimodal Agent Video Player", "venue": "ICMI 2026 Companion", "design_sessions": 18, "follow_up_participants": 8, "source": "https://doi.org/10.1145/3776591.3833853"}
],
"survey": {"org": "American Foundation for the Blind", "respondents": 1735, "blv_using_ai_description": "79%", "daily_users": "51%", "source": "https://afb.org/research-and-initiatives/ai-series/innovation-access"},
"brand_takeaway": "Real text, alt text, labeled controls, headings, structured data, and clear policies help both screen-reader users and AI agents.",
"cta": "https://robauto.ai/scan"
}`
```

## **See how readable your site is to AI**

Want to know whether AI assistants can read and use your site as easily as these co-designers' tools read a kitchen scale? [Scan your site free at robauto.ai/scan](https://robauto.ai/scan) and get a clear list of what to fix first.

## Sources

- [Bespoke Visual Assistance: What and How do Blind and Low-Vision People Create with Agentic Programming? (ProgramAT)](https://arxiv.org/abs/2607.21760)
- [ProgramAT, ASSETS 2026 (ACM)](https://doi.org/10.1145/3797867.3828973)
- [See What I Mean: ChartAEye, ASSETS 2026](https://doi.org/10.1145/3797867.3841222)
- [Multimodal Agent Video Player, ICMI 2026 Companion](https://doi.org/10.1145/3776591.3833853)
- [American Foundation for the Blind, Innovation for Access (Aug 2026)](https://afb.org/research-and-initiatives/ai-series/innovation-access)

## Where can an agent get this data live?

**Robauto MCP — Agent Intelligence & Brand Discovery Infrastructure.** A public, stateless Streamable HTTP endpoint at `https://mcp.robauto.ai/mcp`, callable with no key. Setup and the full tool list are in the [Robauto MCP directory](https://robauto.ai/mcp-directory).

[All blog posts](/blog)

---

(c) 2026 Robauto, Inc. — support@robauto.ai
Machine surfaces: https://robauto.ai/llms.txt · https://robauto.ai/llms-full.txt · https://robauto.ai/.well-known/api-catalog