Finger Tracker

Usage constraints

The closer the environment is to an "ideal" one, the more robust the system will be (i.e. it will make few errors, there will be less jitter, and latency will be lower).

field of view

The default service setting (see the service configuration panel) is 1.5mm per camera pixel, which corresponds to a 480x360 mm interactive surface. The tracker will work correctly within 1.0mm/pixel and 2.5mm/pixel, with similar performance.

This setting can be roughly approximated.

lighting conditions

Ideally lighting is mixed between office and sunlight. More importantly it is perfectly stable.

In practice, a moderate amount of variation is tolerable (such as created by clouds passing in front of the sun, or sunlight changing between the morning and the afternoon).

background

Even though parasite objects (coffee cups, pens) will not distract the tracker, it is recommended to keep the interactive surface as uniform as possible.

Background adaptation is disabled in the service. It is not robust enough to allow corect detection beyond a few minutes when the scene is modified substantially.

projector contrast

Projected information is actually background information for the tracker; since it typically varies greatly, it should be kept hidden from the camera.

Keep the projector contrast low enough to reach a 1:1 ratio between scene illumination by the projector and by the ambient lighting. Robustness generally increases when the camera is slightly over-exposed (enough to make the projected information disappear while keeping the users' hands visible).

performance

On a modern computer (CPU over 1GHz), the tracker should provide usable results (25Hz processing, ca. 80ms overall latency).

Programs

Below is a list of the programs provided with the GML that create a Finger Tracker service instance.

GML-Monolith-Simplified-FingerTracker

Launches a Frame Grabber service, a Simplified Calibrator service, and the Finger Tracker service. Connection between the three is performed automatically at launch.

GML-Monolith-FingerTracker

Launches a Frame Grabber service, a Automatic Calibrator service, and the Finger Tracker service. Connection between the three is performed automatically at launch.

GML-Component-FingerTracker

Launches only a Finger Tracker service instance.

GML-Demo-FingerTracker-Simple

This sample program connects to a Finger Tracker service and a Simplified Calibrator. Its interface displays colored blobs over the users' fingertips.

GML-Demo-FingerTracker-Basic

This sample program connects to a Finger Tracker service and a Automatic Calibrator. Its interface displays colored blobs over the users' fingertips.

GML-Demo-Shuffler

This sample program connects to a Finger Tracker service and a Automatic Calibrator. Its interface displays a set of photographs that can be translated, scaled and rotated using one or more fingers.

Communicating with the service

To start receiving events from the service, you must

1. Open a TCP connection to the relevant service port.
2. Send an empty message to the service.
3. Start sending and receiving messages.

Messages are formatted as plain ASCII text, with carriage return plus line feed line termination (DOS convention, crlf). All messages are concluded by an empty line, i.e. by the crlf sequence. Each message consists of one or more lines, the first of which is a header line that can usually be ignored safely upon reception. It consists of:

• the magic word BIP/1.0;
• a unique ID for the emitting peer (e.g. a random number), formated as an 8-digit hexadecimal number;
• the index of the message since the connection started (starting from zero), using the same format;
• the length, in bytes, of the message body, excluding the line terminator immediately following the header, the the line terminator that concludes the message, using the same format.

Two example messages:

BIP/1.0 A47F64A1 00000000 0000000¬
¬
BIP/1.0 A47F64A1 00000001 000000D¬
hello, world!¬
¬

The header line is followed by zero or more events, one per line. Each event consists of a name, written between angle brackets, and zero or more pieces of meta-information, separated by space.

Example message containing an event:

BIP/1.0 A47F64A1 00000A14 0000025¬
<Motion> 173 172.15 240.36 1319768891¬
¬

For more details and communication using UDP or multicast UDP, refer to the BIP specification.

Channels

The finger tracker will only run (i.e. start emitting tracking events) if both the calibration and grabber channels are linked to a peer, and the calibration matrix is known.

Note that in the monolithic version of the finger tracker, this is done automatically.