- Dr Jonathan Duckworth, Mr Ross Eldridge, Ms Nina Bennett - RMIT University
- Professor Peter H Wilson, Dr Nick Mumford, Australian Catholic University
- Professor David Shum, Professor Patrick Thomas, Griffith University, Dr Gavin Williams, Epworth Hospital
- Australian Research Council
- Australia Council for the Arts
Elements is an interactive tabletop environment designed to support movement assessment and rehabilitation for individuals with an acquired brain injury. Brain injuries often result in impaired hand and reach function, which is profoundly disabling for those affected. The Elements system provides the patients with an intuitive workspace that affords basic gestural control. Elements integrates digital media software, tangible user interfaces (TUIs), augmented feedback, and artistic expression to enhance patient’s motor and cognitive skills. The results of clinical evaluations indicate that patients relearned movement skills, increased their level of engagement, motivation and self-esteem, and importantly day-to-day functioning also improved.
Audio visual augmented feedback
The Elements design provides patients with a suite of goal directed and exploratory game-like applications for composing with sounds and visual feedback that promote artistic activity. Each of the environments provides (multimodal) augmented feedback (AF) designed to reinforce the actions performed by the patient. This takes the form audiovisual effects that are correlated with the users own motion. AF serves learning by providing the user with additional knowledge of the outcomes of their actions to assist future movement planning. Patients can discover new ways of relating to their body and relearn their movement capabilities in a guided and self-directed fashion.
Natural user interfaces for upper-limb interaction
A key design feature of Elements is its capacity to integrate and support the manipulation of physical objects in ways that are natural to the user’s body and their environment. TUIs placed on the tabletop are the primary means for users to control features and events in the workspace. The shape, scale, texture, color and physical weight of each TUI affords meaningful interaction and has a simple aesthetic that is intrinsically appealing. The design assists patients to relearn object manipulation and placement skills that are essential in daily actions like lifting a cup or placing a food container.
Assessment and Patient Feedback
As an assessment tool, Elements is designed to tracks three aspects of arm movement during goal-directed tasks: movement speed, accuracy of TUI placement on cued targets, and movement efficiency (defined as the deviation from straight-line pathways between targets). These variables are recorded in real time for later analysis and can be fed-back to patients graphically to track progress. Patients can also view performance results for individual tasks as well as a video-replay of movement trajectory for any task trial. This provides patients with highly detailed (and motivating) information on their movement performance and progress; in this way, the feedback facilitates learning.
Flexible design for other populations with movement impairment.
Elements represent one of very few interactive digital media interventions that has successfully been evaluated for brain injury in a clinical setting, and that has shown positive and generalisable therapeutic effects on motor performance, cognition, and everyday function. Further evaluations of Elements in the field of paediatric disability indicate significant treatment effects were seen in children with fairly intractable conditions affecting movement including Childhood Stroke and Cerebral Palsy. These studies show that the Elements design makes a significantly contribution to the treatment of a wider range of patients and age groups with neurological impairments.
Technology integration for ease of use
For therapists administering Elements, our aim was to make the system design user friendly and intuitive. This goal is vital because poor usability limits the acceptance of technology by rehabilitation professionals. The user interface is displayed on the touch-screen, and operated with simple touch, swipe or drag actions. We use commercially available hardware to provide a simple “plug and play” operation. The screen integrates a computer processor, multi-point touchscreen, and marker-based visual tracking via cameras embedded in the unit below the surface of the screen. Elements is a compact and self-contained system, which does not require complex configuration or calibration.