Vision-Based Interaction Services

Foreword

In GML, interaction services are opaque processes that emit or absorb events from a network connection. They implement BIP, a straightforward, progressive protocol that is specifically designed for interaction services.

Using a GML service in your application simply means

1. launching the program bundled with GML that instanciate the service you need, e.g. GML-Monolith-HeadTracker.app on MacOS;
2. open a TCP socket to the appropriate port of the computer hosting the service;
3. start receiving user input events in plain text format.

Running a service

Some services require other services to run; for instance the Finger Tracker service requires a Calibration service and a Frame Grabber service. For ease of use a set of programs are distributed with GML:

• monoliths launch all the dependancies of a particular service, and create the appropriate connections between services. This is the simplest approach for end-users.
• components launch only one service. The user may have to establish connections between dependant services.

The documentation for each specific service lists the programs that provide the service.

Configuring a service

Whenever launching a program bundled with GML, the instantiated services will be materialized by a control interface that allows the user to:

• inspect and modify the service parameters;
• inspect the connections to the services' channels;
• monitor any video / graphical feedback the service offers about its behavior.

Any setting configured via this interface will persist across launches of the containing program.

Inspecting the service parameters. Columns show the parameter names, default values, current values, and an entry box used to set the parameters to new values.

Inspecting the service channels. Displayed are the names of the channels, the direction of the information flow for these channels, and the ports for the available transports.

Here no multicast group is displayed because the service doesn't implement communication over multicast UDP.

Available services

Finger Tracker

Tracks fingertips in a video stream and send mouse-like events to clients. Can be used as a multi-pointer input device by applications.

2D-2D Calibrator

Establishes mappings between display coordinates and camera coordinates in "augmented surface"-type scenarios.

Frame grabber

Captures frame from a digital camera or image files and allows peers to access them.