Speaker
Description
O. Beltramello 1 , A. Crivellaro 2 , P. Fua 3 , V. Lepetit 4 ,
1 CERN, Geneva, Switzerland
2 S&H, Milano, Italy
3 EPFL, Ecole Polytechnique Federale de Lausanne, Switzerland
4 Graz Technical University, Austria
The goal of this project is to develop a daily life assistance system for the visually impaired in uncontrolled
environments. Currently, Computer Vision techniques have been playing on a limited role in assisting the visually impaired, mainly due to the challenging requirements for making the system practical.
Among the many, the system needs to run on mobile devices, while performing complex pattern recognition tasks in uncontrolled environments. Moreover, the design of the Human Computer Interfaces (HCI) is particularly complex, since it should be adapted to the way visually impaired people perceive the world.
Today, the outcome of the research in Augmented Reality carried on within the EDUSAFE Project, as well as recent advances in large scale vision based recognition systems, allow us to enable new research directions and applications.
Our system will provide a mobile application (Android), helping the user analyze the surrounding environment. The user will provide vocal inputs asking questions about the surroundings; the system will process the queries and provide the requested information by analyzing pictures and videos taken from a wearable embedded camera. This setup will keep the user’s hands free, a critical requirement for usability, and will also ensure a longer battery life by avoiding excessive use of the phone camera. Some example queries that our system will be able to assess are: “Is there a free seat on this bus ? Bring me there!”; “ Bring me to the closest crosswalk.”; “ How many people are there in the line in front of me? “ and so on.
For this, we will need to improve state of the art in several domains, such as:
VISION:
a novel 3D Tracking framework is required, able to deal with hundreds of object categories by coupling Pattern Recognition with Navigation and Mapping methods. While the 3D Tracking successfully developed for the EDUSAFE European Project would provide a sound theoretical basis, an entirely new framework is required for scalability.
EMBEDDED COMPUTING :
we need to bridge the gap between the massive computational resources
required by modern Pattern Recognition techniques and the limited capability of mobile platforms.
Moreover, we will develop a miniaturized, wearable camera module able to seamlessly stream images to
mobile platforms in real time, using low power consumption wireless techniques such as the Low Power
Bluetooth standard.
INTERFACES/USABILITY:
a voice recognition system will be put in place for interpreting the
user’s voice commands and high level queries, working in noisy, uncontrolled environments. As for the
output, the most suitable way of conveying information for users with different degrees of inability will be
sought. Furthermore, though mainly addressed for the visually impaired, our system will be straightforwardly adaptable to assist individuals with other kinds of disabilities, such as autistic patients, for navigation and analysis of unknown environments.
Summary
The goal of this project is to develop a daily life assistance system for the visually impaired in uncontrolled
environments.
