Baidu Eye is a head-mounted device used to assist people in shopping malls. As a research intern in Baidu's Institute of Deep Learning, I was responsible for developing a light algorithm that allows wearer to circle around things in his view using his pointing finger in order to select a specific object from a large scene. The selected item was then sent to Baidu's image search API for object recognition.
This can be used for searching image of products, banners, logos, and even clipping a portion of an advertisement.
Below are some of the main challanges: