Pointable | An In-Air Pointing Technique to Manipulate Out-of-Reach Targets on Tabletops

Part-II of my Master's thesis (Remote Multitouch: In-Air Pointing Techniques for Large Display Interactions) where we were looking at how well in-air gestures can work for data manipulation tasks on a large horizontal/tabletop display. The ergonomics of a horizontal display, along with the position of a user (generally seated) changes the design constraints for any interaction techniques we wanted test.

perspective-based or image-plane pointing technique for a tabletop display

In Part-I (MultiPoint) we looked at how freehand pointing compared to laser pointers for large vertical displays; in this case, it made more sense to compare perspective based pointing to multitouch input on a tabletop display. The task for a participant was similar to MultiPoint, and involved a docking task where participants moved + rotated a block from a start location to a destination (i.e. dock).

Interaction maps for each technique (left column: multitouch, right column: perspective based pointing). Darker shades represent more manipulations in that location. Solid square shows dock location. Dashed diamonds show initial target configurations for the largest target size. All three target sizes had common centers. Participants were seated at the bottom edge.

While we knew that multitouch input was good for interacting with objects/elements within reach, we were interested in looking at techniques that could augment multitouch input on tabletop displays. To that end, Pointable was designed to realize the following goals: to augment touch, minimize modality switches, support in-place manipulation, and be unobtrusive.

Here's a video of the system in action:

Software: C# with WPF
Hardware: Vicon Motion Capture cameras, designed and built custom markers used to track hand and head position

Excerpt from the paper published in ITS'11:

Pointable is an in-air, asymmetric bimanual manipulation technique, which augments touch input on a tabletop to more easily interact with distant content. The dominant hand points and acquires remote targets, while the non-dominant hand scales and rotates the target without the need to drag the target closer, i.e. Pointable allows users to perform in-place manipulation. However, if users prefer direct-touch for scaling and rotation transforms, they can use Pointable just as a tool to move content to and from a distant area of the tabletop. Switching from using Pointable to using direct-touch is simply a matter of placing a finger- tip of the dominant hand on the tabletop.

Related Publication(s)

Amartya Banerjee, Jesse Burstyn, Audrey Girouard, and Roel Vertegaal. 2011. Pointable: an in-air pointing technique to manipulate out-of-reach targets on tabletops (ITS ’11)