Expanding Accessibility with AI

Expanding Accessibility with AI

Aaron Gustafson

Hey there. I’m Aaron Gustafson and after spending over two dozen years in the trenches as an accessibility-focused web developer, I was drawn to the potential of AI to benefit people with disabilities at scale.


Ioana, please introduce yourself…

Ioana Tanase

IT*– the human Inclusive AI & Accessibility PM @Microsoft Full time Dyslexic You can find me on LinkedIn *Ioana Tanase

What does AI have to do with accessibility?

I think the first question many people have coming into a session like this is what exactly does AI have to do with accessibility.

I get it… much of the focus has historically been on disease detection, weather pattern analysis, and such. Lately the news is dominated with generative AI and the text, images, audio, and video it’s producing. While there are some incredibly valuable uses of AI, I honestly think a lot of the headlines we see are a lot flashy demos without much substance. To me, the real potential of these technologies is its ability to amplify human capabilities in a way that bridges the gaps we’ve designed into our systems and built environments that keep people with disabilities from being able to do the things they want or need to do every day.

However, this scale is a double edged sword as it can do incredible amounts of harm if you aren’t careful.

AI is excellent at repetitive tasks

When I look at the current generation of AI systems, Idon’t see intelligence, per se. Or perhaps I should clarify: I don’t see much in the way of independent intelligence.

Current gen AI systems are predictive machines that are really efficient at identifying patterns and repeating them, with a bit of variability thrown in to keep their products from being monotonous.

I don’t say this to disparage these systems. There is absolutely value in AI’s ability to rapidly do mundane, repetitive work. These capabilities make it well-suited to supplement our own abilities and fill the gaps between what we can do on our own and what we need to be able to do to accomplish our goals.

AI can provide guidance

Much like Search Engines did over the last 25 years, AI systems are enabling us to surface lessons we’ve learned in the past and disseminate that information at a scale and speed we’ve never witnessed before. It’s not to say they don’t product inaccurate information — they absolutely do — but properly grounded and prompted systems can produce reliable enough results to help accelerate our own understanding of topics to solve problems… whether in code, design, writing, or otherwise.

AI enables people with disabilities to empower themselves on their own terms

And unlike traditional user experiences, the improvisational nature of many of these generative AI systems is allowing us to move past software than is designed to epower people with disabilities to software that enables people with disabilities to design their own solutions and empower themselves on their own terms.

Let’s look at a few examples…

Use Case: Improved Descriptions

One of the first places we started to see generative AI put to use was as a complement to image recognition systems for generating alternative text. Historically the results have been pretty crude, but they are getting better.

We’re starting to see some major improvements in this space too, where generative models interpret and translate complex images like charts to unlock the data held within them, which is a boon to people who are blind or who have other vision disabilities, but also for people who are unable to extract meaning from information dense images for any reason. Imagine a future where we could ask an AI to answer questions we have about a chart or graph!

Then there are tools — like Seeing AI — that help translate the world around us to address the real time needs of people who are Blind or low-vision…

Use Case: Adaptive Content (1 of 2)

Beyond describing interfaces or the world around us, AI can help adapt content to make it more usable to us.

A great example of this is using a LM to distill long documents into key points or takeaways. This is helpful for people with dyslexia, people who are cognitively taxed, and people for whom there are never enough hours in a day.

Use Case: Adaptive Content (2 of 2)

Similarly, AI can adapt content into plan language, helping civilians access the meaning of obtuse legalese and people who are low literacy to access information in other modalities like audio.

And while we are still getting used to having dark & light modes on our devices, AI would certainly be able to adapt the colors of documents, charts, and so much more to make them more accessible to us as individuals.

Use Case: Adaptive Input (1 of 4)

AI can also help immensely when it comes to input as well. We’ve had speech to text systems for a long time, which has been a tremendous boon to people with mobility disabilities.

Use Case: Adaptive Input (2 of 4)

With the growth in AI those systems — which are in every OS these days — are becoming even more powerful tools for people with disabilities.

Use Case: Adaptive Input (3 of 4)

And projects like the Speech Accessibility Project — a collaboration between Amazon, Apple, Meta, Google, and Microsoft — are working to improve speech recognition for people with impacted speech, such as folks with Down Syndrome, Parkinson’s, Cerebral Palsy, ALS, and stroke. AI STT tools are helping people show up as themselves and be more easily understood by their loved ones, care givers, and others.

And while we often think of Generative AI as an output, it’s also an amazing tool for input as well, enabling people with disabilities to express themselves clearly with fewer steps as Ioana shares in this video…

Use Case: Adaptive Input (4 of 4)

Use Case: Improved Communication (1 of 4)

Finally, AI enables people with disabilities to show up as themselves and communicate in an authentic way.

Use Case: Improved Communication (2 of 4)

It’s enabled us to build better Augmentative and Alternative Communication tools. Ones that aren’t as physically and mentally exhausting to use and that reduce the time it takes to compose responses in conversations.

Use Case: Improved Communication (3 of 4)

Finally, AI is giving people without speech the ability to communicate in an authentic way, whether that’s through high quality AI-driven synthetic voices or custom made clones of the voices people had before a disease like ALS made it impossible for them to speak. We have been working with Team Gleason, an ALS support organization, to do that very thing, and I’d like to share a video to demonstrate the impact AI is having in one family’s life…

Use Case: Improved Communication (4 of 4)

With great power comes great responsibility

AI has the power to exponentially improve our lives, but that same compounding power can do real harm if not managed properly.

Now I’ll hand it over to Ioana to talk about our responsibilities when it comes to developing AI systems.

What is Responsible AI?

Responsible AI Principles

Risk mitigation layers

RAI Homework

What’s the next (accessible) stop on our journey?

Trends and opportunities

Innovation at the forefront

A secret innovation formula:

aka.ms/InnovationToolkit

Step 1 – Connect with your passion Step 2 – Identify the wave Step 3 – Think about your customers Step 4 – Craft a vision statement Step 5 – Validate your prototype or solution

Here’s a few other things to get you started

Learn more about Accessibility at Microsoft Microsoft.com/accessibility/resources

Get your Accessibility Fundamentals badge aka.ms/AccessibilityFundamentals

Explore Copilot scenarios aka.ms/ThisIsMyCopilot

Get tech support aka.ms/eDAD

Thank you!