AI-based Apps and Accessibility Functions on Smartphones

Assistive technologies are increasingly part of mainstream use, like audio books, predictive text or text production apps. I recently interviewed Rosie Arcuri, who has a great deal of knowledge of these technologies to learn more about the potential of smartphones as an assistive technology in the classroom. It would be beneficial, she says, if all people were aware of these technologies.

Rosie feels that there are still a lot of negative perceptions about the use of smartphones in the classroom, but that attitudes are starting to change. Assistive technologies are a game-changer for students who need them. Rosie told me a story of a student with a learning disability whose mother would read all of her required readings out loud. This student burst out crying when she discovered Speak Screen, one of the assistive technologies described below. The feature increased her autonomy, reduced the burden on her family and for the first time she could also text her friends without the support of her mother.

Rosie Arcuri of the Adaptech Research Network presented a wide range of assistive technologies that are shipped with most smartphones and accessibility apps that are available for download.

Pre-installed accessibility functionality in smartphones

Accessibility is an important focus of smartphone manufacturers and software development both in Apple’s iOS and the Android operating system. Most smartphones and tablets ship with a variety of accessibility functions pre-installed. The Adaptech Research Network has compiled a list with a variety of accessibility functions for both iOS and Android.

As Rosie mainly uses an iPhone, this article will cover functionality included within iOS. She mentioned in passing that the Android accessibility features have greatly improved in the last few years and provide almost a similar level of options. However, as many Android-based smartphone manufacturers have varying designs and software implementations, the experience can vary from device to device.

A screen capture from Apple’s iOS showing functions from the “Accessibility” category of settings.

VoiceOver, Zoom and Magnifier are examples of functions that are helpful for users with varying degrees of visual impairment. Students with visual impairments or those with dyslexia may also benefit from the “Display and Text Size” option and modifying the spacing of text within the settings of the phone.

All smartphone users might find the “Spoken Content” function useful. It is a text-to-speech function that includes two main options. When activated, the “Speak Selection” function (also known as “Select to Speak” on Android) adds a button to the options on any selected text which allows the user to have the selected text read back to them. The function highlights each word as it is being read to help maintain focus on the text. Rosie points out that this function can be used on any document that is accessible, such as a PowerPoint presentation or a PDF file (using text, not images). She says that students really like this functionality since they have their smartphones with them all the time and they can use it to listen to texts while they are in public transit.

A screen capture from Apple’s iOS showing the “Speak Selection” function. When a paragraph is selected, users can tap on “Speak” to have the paragraph read to them.

There is also the “Speak Screen” function that allows users to swipe down the screen with two fingers, which will result in the entire text being read aloud. Rosie says that this function is interesting for articles or textbooks when students don’t want to select each paragraph of the text. Users can also set the reading speed to the level which they prefer.

There are many more functions for people with a variety of needs. For those with motor control related difficulties, you can set the sensitivity of the phone. For those with hearing impairment, there is functionality to interface with hearing aids, many of which now have bluetooth integrated into their hardware.

Downloadable apps for accessibility

Seeing AI is an app created by Microsoft that turns pictures into text by using Artificial Intelligence (AI); it then detects the text and converts it to audio. It features a “Short Text” mode that can read a brief amount of text – like a business card or book cover – in real time.

The “Document” option uses the camera of the Smartphone to take a picture of a page of text and provides audio prompts to the user to optimize capture of the document. At the time of writing, this app is only available for iOS. Office Lens, another Microsoft app that converts a physical document into text, is available on both iOS and Android.

These apps use AI to do a sort of optical character recognition and convert the picture to text and then to audio.

You have the option to save a text to PDF or Word and upload to Cloud-based storage apps. This, Rosie points out, is useful for students with executive functioning difficulties who can use this functionality to stay organized.`

Rosie mentioned another recent Adaptech article which identified other AI-based features used by students. Some of the notable features are:

• dictation
• word prediction
• auto-captions
• chatbots

Another app that Rosie likes is the Google App on the iPhone. When viewing a website, it places an icon next to the website address bar that activates a function to read web pages out loud.

A screen capture of a Profweb story within the Google App. Notice the text-to-speech button to the right of the website address, which launches the text-to-speech controls that appear at the bottom of this screen capture.

Parting thoughts

Rosie understands why teachers might ask for students to stow their phones away during class time, but reminds me that not everyone using a phone in the class is a distracted learner. There are so many ways that smartphones could be used as a tool in class for all students, and it is important to keep an open mind.

Rosie also mentions that between 10% and 20% of college students have a disability and only a relatively small percentage of students with disabilities seek out the assistance of their college’s accessibility centre. In her experience, it may only be half of students that have access or choose to use these services. Furthermore, these tools have the potential to assist other populations like English as a Second Language students.

“While assistive technologies can be useful to all students, for those with accessibility needs, they are crucial for overcoming barriers. I think the more we normalize it the more people can benefit from what is available. It can lead to more independence and that can carry on from school to the workplace. It has the potential to really help students in the future.”

Thank you Rosie for your enlightening and insightful tour of accessibility functions and apps!