ETH Zurich |
Department of Computer Science |
Institute for Pervasive Computing |
In this project we evaluate the feasibility of using the CCD cameras built into off-the-shelf mobile phones to act as sensors for 2-dimensional visual codes. The codes can be attached to physical objects and act as a key to access object-related information and functionality. The use of mobile phones is interesting in this scenario, because mobile phones are ubiquitously available devices providing constant wireless connectivity, and models with integrated cameras are getting more and more popular. Using the integrated camera as a sensor thus offers a natural way of detecting objects in the user's immediate surroundings. More background information is given in the references section. A non-technical German introduction into the project is available here.
Test application for recognizing visual codes. Written by Beat Gfeller and Michael Rohs.
|
|
|
|
Visual codes can be created here, by entering a decimal or hexadecimal number into the text box below the image.
The following videos demonstrate some of the interaction possibilities of the visual code system that are enabled by using the position of the camera focus, the rotation of the code in the image, and the amount of tilting of the code plane relative to the image plane as additional input parameters. Each code introduces a code coordinate system with its origin at the upper left edge of the code and one unit corresponding to a single code bit element. The code coordinate system is independent of the orienation of the code in the image. The recognition algorithm is able to map points in the image plane to points in the code plane and vice versa.
With visual codes, areas in printed documents can be linked to online content. Paper documents thus become interactive media. Imagine a cinema magazine, in which physical hyperlinks connect words and images to movie trailers, background information about the actors, and the local cinema's booking system. Such physical hyperlinks can be conveniently selected with a camera-phone that is in reach of its user most of the time.
In newspapers, visual codes can be used to store or rate newspaper articles, to get background information related to advertisements, and for updates on information which quickly gets obsolete, like stock quotes and weather forecasts. If physical hyperlinks are associated with points and regions on the page, the camera-phone can follow the hyperlink, fetch the additional information and display them. In the video, the camera-phone retrieves the current snow conditions for various regions of Switzerland by moving the camera focus across the corresponding table entries on a newspaper page. Since the rotation of the code in the image is provided as an additional parameter, multiple aspects can be associated with a single point or region. As shown in the video, when the camera is rotated by a certain amount, the display changes to indicate the current temperature instead of the snow depths.
|
Scanning the visual code of an ETHOC ID triggers the phone's WAP browser and shows the associated information.
|
|
Automatic dialling of phone numbers that are encoded in visual codes. Phone number codes can be created with the form shown above in the following way: Phone number 0041 1 632 08 59 has to be encoded as 0xf004116320859.
|
|
|
|
Dialling of phone numbers that are encoded in visual codes.
|
|
Allows to set the current phone profile (general, pager, silent, etc.) by scanning a menu associated with a visual code.
|
|
|
Shows the orientation of the recognized code on the bottom of the screen in the following way:
|
|
Symbian version of the classic Pong game. The green ball bounces back from the walls. The red slider is controlled by the orientation of the phone relative to the background. Moving the phone to the right moves the slider to the right - and vice versa. It implements a kind of "virtual mouse" with the integrated camera. A video that shows how this might be used for interaction with large-scale displays is available here (1.4 MB, wmv).
|
Draws a 3-dimensional wireframe of a house which is rotated along the x-axis and y-axis depending on the orientation of the phone. The integrated camera is used to compute the orientation.
|
Displays a tram map of Zurich. Moving the phone updates the part of the map which is displayed.
|
Tracks the tip of a pen as it is moved in front of the camera.
|
Note: These demo applications have been developed for Symbian v6.1 and tested on a Nokia 7650. Except for the Recognizer_v70s.SIS which has been tested on a Nokia 6600, they will not run on Symbian v7.0s and later, because the camera API has changed. Updated versions for Symbian v7.0s will be available soon.
The visual code and its recognition algorithm was designed and implemented by Beat Gfeller and Michael Rohs. If you have any questions regarding this project, please feel free to contact Michael Rohs or Friedemann Mattern.
Top of page.
ETH Zurich |
Department of Computer Science |
Institute for Pervasive Computing |