eCanvas – A Voice Controlled Assistive Canvas

Overview

In August 2018, a team led by Dr Ong Lay Teen, a Senior System Engineer at GovTech, visited Cerebral Palsy Alliance Singapore (CPAS) to discuss how the Sensor Platforms and Internet of Things (SIOT) team could develop new assistive technologies to aid Persons with Disabilities (PWDs). The team was introduced to Elaine, a 19-year-old who was born with cerebral palsy, a lifelong physical disability that affects both her movement and posture. Elaine is a mouth painter due to limitations in her mobility and requires the assistance of volunteers at CPAS to move her canvas and dispense paint for her. Inspired and moved by Elaine’s passion for painting, the team developed the eCanvas – a voice-controlled device that would allow Elaine to paint seamlessly despite her physical limitations. Leveraging neural network technology, the eCanvas project was specifically designed to be voice-controlled due to Elaine’s strong verbal capabilities.

What is eCanvas?

The eCanvas is a voice-controlled assistive technology that provides greater independence for PWDs with artistic ambitions or who enjoy art as a recreational or social activity. It consists of a motorised aluminium frame that clamps onto a drawing canvas and allows users to move it within their reach. A microphone connected to the eCanvas processes voice commands via a pre-trained neural network engine, allowing the canvas to move or dispense paint on the palette accordingly.

While developing the eCanvas, multiple site testing and alterations were carried out to perfect the device. Initially designed to only move up and down, the SIOT team improvised upon everyday items and repurposed soap dispensers and a turntable to act as a paint dispenser and palette respectively. The SIOT team also built a custom system - powered by a convolutional neural network – to recognise specific keywords instead of a wide variety of words. Elaine’s voice was also added to the mix when training the device to improve its receptiveness to voice commands.

The eCanvas is one of many SIOT products developed as part of the Smart Nation Sensor Platform (SNSP) and is aligned with GovTech’s focus on inclusiveness and aim to develop technology for the public good.

Why Should eCanvas be Adopted?

The SIOT team designed the eCanvas with inclusiveness in mind as technology can be used to empower minority groups in society that are often overlooked, such as PWDs. This empowers PWDs to contribute to society and achieve their dreams. As Singapore digitalises and progresses towards our Smart Nation plans, it is pertinent for all Singaporeans of all abilities to be included through transformative technologies such as the eCanvas.

The benefits of the eCanvas include:

Assistive touch controls: For individuals with limited verbal abilities but varying levels of upper body movement control, the assistive touch button of the eCanvas enables them to manoeuvre the equipment easily for painting.

Speech/motor therapy uses: The eCanvas can serve as a form of speech or motor therapy as it allows PWDs to work on their speech or mobility through art which has therapeutic properties.

Seamless user experience: With the eCanvas, there is no need to navigate through levels of voice commands like in a phone menu as the options are parallelised onto a single layer. This means that the voice control system interface creates a seamless user experience as only a specific command is required for a desired action to occur.

How Do You Use eCanvas?

Users with zero upper-body mobility but possess strong verbal capabilities.

The eCanvas adopts a voice-command control where users can use a set of verbal commands to move the eCanvas (Eg., “Up”, “Down”, etc.) and adjust their drawing area independently. Numbered commands are also attached to each paint colour for easy dispensing of paint. For instance, users only need to say “One” and the eCanvas will dispense the corresponding paint colour from the respective dispenser.

Users with some level of upper-body mobility but are unable to verbalise well.

Instead of voice commands, users can adopt a sensitive touch button to carry out the same tasks as described in the first scenario. To adjust their drawing area on the canvas, users can move the touch button in the desired direction. For example, to move the canvas downwards, users will toggle the touch button in the downwards direction and vice versa for other directions. The touch button is sensitive enough that users with limited upper body mobility can move the button easily.

Success Stories

Beyond recreational, social and therapeutic uses, the eCanvas has benefitted PWDs financially as well. PWDs who are professional painters have indicated that with the eCanvas, mouth painting is now more seamless as this has improved their income generating abilities.

eCanvas has also been deployed in the Rainbow Centre, a social service agency dedicated to helping young PWDs. The eCanvas currently helps lower-primary school students with different forms of physical limitations to carry out therapeutic activities such as arts and craft.

What’s Next?

The SIOT team plans to deploy eCanvas across various social service agencies in Singapore.

Prior to the COVID-19 pandemic, numerous social service agencies have indicated their interest in adopting eCanvas and although COVID-19 has impacted the scale of events and activities, follow ups will be done with all social service agencies who have indicated their interest.

The eCanvas project also represents the start of GovTech’s assistive technology plans, with more products and prototypes for PWDs currently in the pipeline. This aligns with our goal of creating a more inclusive society through inclusive technologies!

Contact Us

Meet the Team!

eCanvas Contributors (2018-2019):

• Quek Yang Boon, Director
• Tony Varghese, Product Manager
• Liew Hua Peng, Lead System Engineer
• Sivaguru Sivagnanam, Associate Systems Engineer (Oct 2019 – Jun 2020); Intern (Jan – Aug 2019)
• Kaitlyn Ng, Intern, NUS, Bachelor’s in Computer Science

Last updated 19 August 2022

---