Using Camera-Equipped Mobile Phones for Interacting with Real-World Objects 

Michael Rohs, Beat Gfeller 

Abstract 

The idea described in this paper is to use the built-in cameras of consumer 
mobile phones as sensors for 2-dimensional visual codes. Such codes can be 
attached to physical objects in order to retrieve object-related information 
and functionality. They are also suitable for display on electronic screens. 
The proposed visual code system allows the simultaneous detection of multiple 
codes, introduces a position-independent coordinate system, and provides the 
phone's orientation as a parameter. The ability to detect objects in the 
user's vicinity offers a natural way of interaction and strengthens the role 
of mobile phones in a large number of application scenarios. We describe the 
hardware requirements, the design of a suitable visual code, a lightweight 
recognition algorithm, and present some example applications.