eCanvas – A Voice Controlled Assistive Canvas | Singapore Government Developer Portal
Have feedback? Please

Overview

In August 2018, a team led by Dr Ong Lay Teen, a Senior System Engineer at GovTech, visited Cerebral Palsy Alliance Singapore (CPAS) to discuss how the Sensor Platforms and Internet of Things (SIOT) team could develop new assistive technologies to aid Persons with Disabilities (PWDs). The team was introduced to Elaine, a 19-year-old who was born with cerebral palsy, a lifelong physical disability that affects both her movement and posture. Elaine is a mouth painter due to limitations in her mobility and requires the assistance of volunteers at CPAS to move her canvas and dispense paint for her. Inspired and moved by Elaine’s passion for painting, the team developed the eCanvas – a voice-controlled device that would allow Elaine to paint seamlessly despite her physical limitations. Leveraging neural network technology, the eCanvas project was specifically designed to be voice-controlled due to Elaine’s strong verbal capabilities.

What is eCanvas?

Fig 1: eCanvas
Fig 1: eCanvas

The eCanvas is a voice-controlled assistive technology that provides greater independence for PWDs with artistic ambitions or who enjoy art as a recreational or social activity. It consists of a motorised aluminium frame that clamps onto a drawing canvas and allows users to move it within their reach. A microphone connected to the eCanvas processes voice commands via a pre-trained neural network engine, allowing the canvas to move or dispense paint on the palette accordingly.

While developing the eCanvas, multiple site testing and alterations were carried out to perfect the device. Initially designed to only move up and down, the SIOT team improvised upon everyday items and repurposed soap dispensers and a turntable to act as a paint dispenser and palette respectively. The SIOT team also built a custom system - powered by a convolutional neural network – to recognise specific keywords instead of a wide variety of words. Elaine’s voice was also added to the mix when training the device to improve its receptiveness to voice commands.

The eCanvas is one of many SIOT products developed as part of the Smart Nation Sensor Platform (SNSP) and is aligned with GovTech’s focus on inclusiveness and aim to develop technology for the public good.

Why Should eCanvas be Adopted?

The SIOT team designed the eCanvas with inclusiveness in mind as technology can be used to empower minority groups in society that are often overlooked, such as PWDs. This empowers PWDs to contribute to society and achieve their dreams. As Singapore digitalises and progresses towards our Smart Nation plans, it is pertinent for all Singaporeans of all abilities to be included through transformative technologies such as the eCanvas.

The benefits of the eCanvas include:

Assistive touch controls: For individuals with limited verbal abilities but varying levels of upper body movement control, the assistive touch button of the eCanvas enables them to manoeuvre the equipment easily for painting.

Speech/motor therapy uses: The eCanvas can serve as a form of speech or motor therapy as it allows PWDs to work on their speech or mobility through art which has therapeutic properties.

Seamless user experience: With the eCanvas, there is no need to navigate through levels of voice commands like in a phone menu as the options are parallelised onto a single layer. This means that the voice control system interface creates a seamless user experience as only a specific command is required for a desired action to occur.

How Do You Use eCanvas?

Users with zero upper-body mobility but possess strong verbal capabilities.

The eCanvas adopts a voice-command control where users can use a set of verbal commands to move the eCanvas (Eg., “Up”, “Down”, etc.) and adjust their drawing area independently. Numbered commands are also attached to each paint colour for easy dispensing of paint. For instance, users only need to say “One” and the eCanvas will dispense the corresponding paint colour from the respective dispenser.

Users with some level of upper-body mobility but are unable to verbalise well.

Instead of voice commands, users can adopt a sensitive touch button to carry out the same tasks as described in the first scenario. To adjust their drawing area on the canvas, users can move the touch button in the desired direction. For example, to move the canvas downwards, users will toggle the touch button in the downwards direction and vice versa for other directions. The touch button is sensitive enough that users with limited upper body mobility can move the button easily.

Success Stories

Fig 2: An adult mouth-painter painting with the eCanvas
Fig 2: An adult mouth-painter painting with the eCanvas

Beyond recreational, social and therapeutic uses, the eCanvas has benefitted PWDs financially as well. PWDs who are professional painters have indicated that with the eCanvas, mouth painting is now more seamless as this has improved their income generating abilities.

eCanvas has also been deployed in the Rainbow Centre, a social service agency dedicated to helping young PWDs. The eCanvas currently helps lower-primary school students with different forms of physical limitations to carry out therapeutic activities such as arts and craft.

What’s Next?

The SIOT team plans to deploy eCanvas across various social service agencies in Singapore.

Prior to the COVID-19 pandemic, numerous social service agencies have indicated their interest in adopting eCanvas and although COVID-19 has impacted the scale of events and activities, follow ups will be done with all social service agencies who have indicated their interest.

The eCanvas project also represents the start of GovTech’s assistive technology plans, with more products and prototypes for PWDs currently in the pipeline. This aligns with our goal of creating a more inclusive society through inclusive technologies!

Contact Information

For enquiries, email info@tech.gov.sg.

Meet the Team!

Ong Lay Teen, Team Lead, Lead R&D/Systems Engineer

Ong Lay Teen, Team Lead, Lead R&D/Systems Engineer
Lay Teen is the team leader of the Enabling (Assistive) Tech Team and a Lead R&D/Systems Engineer at GovTech. She has over 20 years of international research and development experience in the industry and academia. Her specialisation includes advanced algorithm development, signal processing and RF/Microwave Engineering. She received her Ph.D. in Engineering from Cardiff University, UK, Master of Science from the National University of Singapore (NUS) and Degree in Engineering from Nanyang Technological University (NTU). Lay Teen is passionate about learning, exploring, and sharing advanced technologies from/with research institutions and industry. Her current focus is on leading the team in developing Enabling/Assistive Technology that enable people of special needs to live independently and have a more fulfilling life.

Benjamen Lim, Associate R&D Engineer

Benjamen Lim, Associate R&D Engineer
Benjamen is an associate R&D engineer at GovTech. Benjamen graduated from Northwestern University in the United States with a master’s and bachelor’s in mechanical engineering, specialising in robotics. He is passionate about design and prototyping ideas, from embedded programming to mechanical design, and is excited to contribute to our Smart Nation by using these skills to benefit others.

Lee Si Min, Associate R&D Engineer

Lee Si Min, Associate R&D Engineer
Si Min graduated from NTU with a degree in Computer Science. She enjoys developing assistive technology applications which enable users to live fulfilling and independent lives.

Tan Yi Ming, Associate R&D Engineer

Tan Yi Ming, Associate R&D Engineer
Yi Ming graduated from NTU with a degree in Electrical and Electronic Engineering. He specialises in Intelligent Systems & Control Engineering. Yi Ming is passionate about developing assistive solutions that can help empower and enable users with special needs to live their lives independently.

Esmond Ng, Associate R&D Engineer

Esmond Ng, Associate R&D Engineer
Esmond graduated from NTU with a degree in Mechanical Engineering, with a specialisation in Mechatronics and Robotics Systems. Driven by curiosity and passion, Esmond enjoys challenging himself in the field of technology and improving the quality of life of individuals. Apart from work commitments, Esmond enjoys watching movies to be inspired.

eCanvas Contributors (2018-2019):

  • Quek Yang Boon, Director
  • Tony Varghese, Product Manager
  • Liew Hua Peng, Lead System Engineer
  • Sivaguru Sivagnanam, Associate Systems Engineer (Oct 2019 – Jun 2020); Intern (Jan – Aug 2019)
  • Kaitlyn Ng, Intern, NUS, Bachelor’s in Computer Science

Last updated 02 December 2021


Was this article useful?
Send this page via email
Share on Facebook
Share on Linkedin
Tweet this page