Reflex Arc: gesture recognition can take you places

The brief

JISC TechDis are the UK advisory service on technology and inclusion, supporting the education, training and skills sectors.

Back in 2011 they asked Jonathan to judge a gesture recognition contest. Not happy with simply judging, Jonathan put together a team to win it.

The brief? To create a digital application using gesture recognition, to help learners with disabilities who lack independence due to an inability to communicate by speech or due to lack of motor control.

Our approach

Working with two companies from his BBC days – Reflex Arc and Gamelab UK – Jonathan put together a proposal for the contest, securing two phases of funding to create a sign-language recognition system to help autistic & learning-disabled Makaton users make an easier transition into independent living and employment.

Leading the project’s user-research, Hassell Inclusion worked with specialist colleges across the UK to investigate the user-contexts in which sign-recognition would be useful for students who use Makaton. The user-research identified ways of motivating our target users to use the evolving recognition technologies being created by Reflex Arc and Gamelab UK, resulting in gameplay concepts to engage users with signing in familiar scenarios.

When we started working with Hassell Inclusion to help students with communication difficulties, I never thought it would give me experience and skills that would help me create innovative mainstream VR experiences for the likes of Reebok. This is where thinking differently has got us. It’s been quite some journey...

The outcomes

The resulting uKinect sign language eLearning games enabled these young people, along with their new non-signing colleagues, to learn workplace-specific sign vocabularies to help them transition from college to work environments.

The games used an engaging animated avatar called Boris who demonstrated and responded to signs via an innovative Kinect sign-language recognition system.

As well as being useful for these learners, the ‘Boris Games’ also demonstrated what was possible through sign-language recognition and won the TIGA Games Industry Award for Best Education Initiative in 2013 and was runner up for the Japan Prize 2015.

 

 

 

 

 

The legacy

Over the next few years the innovations that we created in response to the needs of these initial groups of people enabled us to create similar breakthroughs for other groups.

Through Microsoft, whose Kinect technology was at the heart of our gesture recognition, we worked with:

  1. Guide Dogs, who were looking for ways of making mobility training more fun for blind and partially-sighted young children. Working with a school for vision-impaired kids in North London, we created the ‘Nepalese Necklace’ games which took their body and spatial awareness exercises, and made them controls for motivational 3D audio-games, turning ‘boring’ exercises into fun activities. The games transformed the kids’ feelings towards their mobility training and also improved their concentration on the rest of their schoolwork. They also won our team Guide Dogs Partner of the Year award.
  2. A group of stroke outpatients in Shepherds Bush, London, who were looking for motivational games to help them with their rehabilitation. We were inspired by a project which used Kinect games to help stroke rehabilitation, but wanted to create specific games which used movements specifically recommended by their therapists. If they could use these games in their homes to drive their recovery of function without requiring occupational therapists and physio-therapists to be present to guide and motivate them, this would free up therapists to concentrate on higher levels of care, making more effective use of their valuable time. We secured funding from InnovateUK and, over the course of the next 6 months, we proved that we could do this. Our project was voted best presentation in its category at the Collaboration Nation event showcasing all Innovate UK funded projects in 2014.

Since then more work by Gamelab UK and Reflex Arc has taken the results of this innovation further into rehabilitation, adding VR components, and beyond…
 

Intrigued? Check out our services...

Contact Us About Our Services
  • Innovation Programme

    Key: Continually Evolve

    A programme to help your digital teams think differently by engaging with disabled people's needs, inspiring them to deliver innovations in your proposals and products.

  • Accessibility Support Community

    Key: Continually Evolve

    Become part of our Accessibility Support Community to keep up to date with advances in accessibility standards and assistive technologies.

  • Accessibility ROI Benchmarking

    Key: Measure Effects

    A programme which builds on the Accessibility ROI Workshop, supporting you to introduce mechanisms to measure and analyse your key return on investment elements.

  • Accessibility ROI Workshop

    Key: Measure Effects

    A workshop that looks at the different elements of return on investment from accessibility, and how to identify and measure which are most important for your organisation. For example: value of increasing customers, positive PR, minimising customer complaints, and the value of being able to sell services into markets where accessibility is a procurement requirement.

  • Leaders in Accessibility

    Key: Embed Strategy

    A comprehensive programme for key staff who you want to become digital accessibility advocates within your organisation. Delivers training and support for implementing your accessibility strategy across its component parts: responsibility, competence and confidence, suppliers, policies and governance.

  • Accessibility Strategy Review

    Key: Embed Strategy

    We offer strategic review and analysis for organisations with multiple digital products, and for those with a single product. Both include a strategic review of the product(s), accessibility policies and processes, and provide recommendations and support for implementation.

  • Inclusive User Testing

    Key: Enable Process

    We provide two options, we can do inclusive user testing for you, or add testing with disabled people to the user testing you’re already doing. And we can also train your user testers to do this themselves.

  • We provide two options, we can do our prioritised WCAG Compliance Audit, or our unique Live Accessibility Audit. This interactive review with your team finds accessibility issues in your key customer journeys, allows you to document them in your bug-tracking system and even be fixed in the room. It’s quicker and more empowering for your product teams. And we can also train your QA staff to do full audits themselves.

  • Expert Accessibility Support

    Key: Enable Process

    To build on your training, and help you with applying it on your projects we provide embedded expert support, via: on-call email response over the course of a project, or scheduled remote or on-site surgeries for your team to drop in and talk through issues with our specialists.

  • Accessibility Maturity Benchmarking

    Key: Expand Awareness

    We provide two ways of benchmarking your current level of accessibility maturity: benchmarking your organisation’s accessibility processes, and benchmarking a key product to check their results

  • A set of courses to empower and equip key members of the product team - for product managers, business analysts, designers, developers, content authors, QA testers, auditors, user testers and more. Choose which are most important to you, to ensure you deliver accessibility every time.

  • A short workshop to help you build the case for, and put inclusion and accessibility right into the heart of your digital products, understand what it takes to embed this across your organisation, and how to use the benefits of digital accessibility to meet your organisation’s business goals. You can attend a public workshop, or we can deliver a bespoke workshop on your premises.