Smart Glasses for Visually Impaired

There are more than 285 million people in the world living with sight loss which has a significant impact on their daily lives. Over 85% of these individuals have some remaining vision. Recently, there has been an interest in developing smart glasses, which seek to provide these people with additional information from the nearby environment through stimulation of the residual vision. The aim is to increase the information level regarding the close environment using depth and/or image edges.

We have been developing smart glasses, with which a user can interactively capture a full 3D map, segment it into objects of interest and refine both segmentation and 3D parts of the model during capture, all by simply exploring the space and ‘painting’ or ‘brushing’ by a handheld laser pointer device onto the world. These enhanced images are then displayed to the user on head-mounted AR-glasses, hence stimulating the residual vision of the user.

Our group is involved in the winning entry (‘People's Choice’) for the Google Impact Challenge 2014.


The Semantic Paintbrush: Interactive 3D Mapping and Recognition in Large Outdoor Spaces
Miksik O.*, Vineet V.*, Lidegaard M., Prasaath R., Nießner M., Golodetz S., Hicks S.L., Perez P., Izadi S. and Torr P.H.S.
In Proceedings of the 33nd annual ACM conference on Human factors in computing systems (CHI) 2015, Seoul, South Korea
* Joint first authors
PDF | Show BibTex | Show Details

Incremental Dense Semantic Stereo Fusion for Large-Scale Semantic Scene Reconstruction
Vineet V.*, Miksik O.*, Lidegaard M., Nießner M., Golodetz S., Prisacariu V.A., Kahler O., Murray D.W., Izadi S., Perez P. and Torr P.H.S.
In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) 2015, Seattle, USA
* Joint first authors
PDF | Show BibTex | Show Details