Wrist Device Makes Controlling Gadgets a Snap

Microsoft's Kinect technology lets gamers kick a virtual soccer ball or ride a digital skateboard using natural movements rather than a controller. Yet it can detect and interpret only full-body motion—it is relatively clueless about interpreting the movements of smaller body parts such as fingers. Now Microsoft researchers are developing a wrist-worn sensor to control video games—along with mobile phones, tablets, computers and TVs—with a flick of the wrist or snap of the fingers.

Like Spider-Man's web shooters, the aptly named Digits straps to the inside of the wrist. To track hand movement, it uses aninfrared (IR) camera, laser, light-emitting diode (LED) and an inertial measurementunit. The laser projects an IR line across the inside of the hand, intersecting the fingers and thumb as they bend inwards and marking the distance. As the hand closes, the laser intersects the fingers nearer to the palm and records those measurements as well.

By triangulating the positions of the camera, laser and different points on the hand that the laser hits, the device can reconstruct a virtual hand that mimics movement of the hand wearing the Digits device, to within 0.1 millimeter. This virtual hand would appear onscreen when the wearer is using it with a mobile phone or some other device
A Digits wearer could control different features on mobile devices—for instance, switching mp3 files by mimicking the turning of a dial, adjusting volume by moving a hand up or down to manipulate an invisible slider, or answering phone calls via a thumbs-up gesture—without even removing those devices from a pocket or handbag.

The researchers' goal is to reach a level of fidelity that captures the full dexterity of the human hand, enabling fine-grained interactions with electrical devices down to the individual finger, says David Kim, a researcher in Microsoft Research's Sensors and Devices Group.* Kim, also a member of the Digital Interaction Group at Newcastle University's Culture Lab, and several colleagues presented the Digits research last week at the ACM Symposium on User Interface Software and Technology in Cambridge, Mass.

"This is body-worn and could be taken anywhere," Kim says. "So you could be interacting in your physical world—working, cooking or reading—and then spontaneously interact with virtual content whenever you want to."

The researchers began working on Digits in January, building a working version by the following month that could perform the basic functions and a more complete model in April. They attribute their ability to turn out a prototype so quickly to their experience working on other natural user interface projects including KinectFusionand Holodesk.

Although the prototype is too big and bulky to wear all the time, the researchers say it is possible to shrink Digits to watch size. The main challenge to miniaturizing the 125-gram device to the size of a wristwatch is using smaller components—the Web camera housing, the laser and the LED. The researchers made the housing for the latest prototype using a 3-D printer, which means creating a new one could be done quickly and cheaply. Miniaturizing the other components would depend on finding a smaller laser and LED out on the market. The researchers are also considering the possibility of a pop-up version of the device that rests flat on the wrist when not in use. This would likely require the development of more customized components.

Also, the researchers have to expand the scope of hand movements that Digits can capture. "Right now we can't cover the entire range of movement," Kim acknowledges. "If you straighten your fingers and stretch them too far backward the laser would not illuminate your fingertips, which means the infrared camera would not be able to see them. One work-around for this would be to have a second set of components on the back of the wrist to cover the whole range of finger movement." In the prototype, one finger also can occlude other fingers, blocking the view for the laser and camera, says Otmar Hilliges, also a researcher in Microsoft Research's Sensors and Devices Group.*

It is also unclear whether Digits functions properly in direct sunlight, or if sunlight would interfere with the IR laser—Microsoft has yet to test this scenario. Given how often smart phones and other gadgets are used in the light of day and the technology's other limitations, it may be a while before hand signals are the de facto move for controlling wireless gadgets.

A Digits wearer could control different features on mobile devices—for instance, switching mp3 files by mimicking the turning of a dial, adjusting volume by moving a hand up or down to manipulate an invisible slider, or answering phone calls via a thumbs-up gesture—without even removing those devices from a pocket or handbag.

The researchers' goal is to reach a level of fidelity that captures the full dexterity of the human hand, enabling fine-grained interactions with electrical devices down to the individual finger, says David Kim, a researcher in Microsoft Research's Sensors and Devices Group.* Kim, also a member of the Digital Interaction Group at Newcastle University's Culture Lab, and several colleagues presented the Digits research last week at the ACM Symposium on User Interface Software and Technology in Cambridge, Mass.

"This is body-worn and could be taken anywhere," Kim says. "So you could be interacting in your physical world—working, cooking or reading—and then spontaneously interact with virtual content whenever you want to."

The researchers began working on Digits in January, building a working version by the following month that could perform the basic functions and a more complete model in April. They attribute their ability to turn out a prototype so quickly to their experience working on other natural user interface projects including KinectFusionand Holodesk.

Although the prototype is too big and bulky to wear all the time, the researchers say it is possible to shrink Digits to watch size. The main challenge to miniaturizing the 125-gram device to the size of a wristwatch is using smaller components—the Web camera housing, the laser and the LED. The researchers made the housing for the latest prototype using a 3-D printer, which means creating a new one could be done quickly and cheaply. Miniaturizing the other components would depend on finding a smaller laser and LED out on the market. The researchers are also considering the possibility of a pop-up version of the device that rests flat on the wrist when not in use. This would likely require the development of more customized components.

Also, the researchers have to expand the scope of hand movements that Digits can capture. "Right now we can't cover the entire range of movement," Kim acknowledges. "If you straighten your fingers and stretch them too far backward the laser would not illuminate your fingertips, which means the infrared camera would not be able to see them. One work-around for this would be to have a second set of components on the back of the wrist to cover the whole range of finger movement." In the prototype, one finger also can occlude other fingers, blocking the view for the laser and camera, says Otmar Hilliges, also a researcher in Microsoft Research's Sensors and Devices Group.*

It is also unclear whether Digits functions properly in direct sunlight, or if sunlight would interfere with the IR laser—Microsoft has yet to test this scenario. Given how often smart phones and other gadgets are used in the light of day and the technology's other limitations, it may be a while before hand signals are the de facto move for controlling wireless gadgets.

A Digits wearer could control different features on mobile devices—for instance, switching mp3 files by mimicking the turning of a dial, adjusting volume by moving a hand up or down to manipulate an invisible slider, or answering phone calls via a thumbs-up gesture—without even removing those devices from a pocket or handbag.

The researchers' goal is to reach a level of fidelity that captures the full dexterity of the human hand, enabling fine-grained interactions with electrical devices down to the individual finger, says David Kim, a researcher in Microsoft Research's Sensors and Devices Group.* Kim, also a member of the Digital Interaction Group at Newcastle University's Culture Lab, and several colleagues presented the Digits research last week at the ACM Symposium on User Interface Software and Technology in Cambridge, Mass.

"This is body-worn and could be taken anywhere," Kim says. "So you could be interacting in your physical world—working, cooking or reading—and then spontaneously interact with virtual content whenever you want to."

The researchers began working on Digits in January, building a working version by the following month that could perform the basic functions and a more complete model in April. They attribute their ability to turn out a prototype so quickly to their experience working on other natural user interface projects including KinectFusionand Holodesk.

Although the prototype is too big and bulky to wear all the time, the researchers say it is possible to shrink Digits to watch size. The main challenge to miniaturizing the 125-gram device to the size of a wristwatch is using smaller components—the Web camera housing, the laser and the LED. The researchers made the housing for the latest prototype using a 3-D printer, which means creating a new one could be done quickly and cheaply. Miniaturizing the other components would depend on finding a smaller laser and LED out on the market. The researchers are also considering the possibility of a pop-up version of the device that rests flat on the wrist when not in use. This would likely require the development of more customized components.

Also, the researchers have to expand the scope of hand movements that Digits can capture. "Right now we can't cover the entire range of movement," Kim acknowledges. "If you straighten your fingers and stretch them too far backward the laser would not illuminate your fingertips, which means the infrared camera would not be able to see them. One work-around for this would be to have a second set of components on the back of the wrist to cover the whole range of finger movement." In the prototype, one finger also can occlude other fingers, blocking the view for the laser and camera, says Otmar Hilliges, also a researcher in Microsoft Research's Sensors and Devices Group.*

It is also unclear whether Digits functions properly in direct sunlight, or if sunlight would interfere with the IR laser—Microsoft has yet to test this scenario. Given how often smart phones and other gadgets are used in the light of day and the technology's other limitations, it may be a while before hand signals are the de facto move for controlling wireless gadgets.

http://www.youtube.com/watch?feature=player_embedded&v=Tm2IuVfNEGk

                                         
                                                  To track hand movement, Digits uses an infrared (IR) camera, laser, LED and an inertial measurement unit. The laser projects an IR line across the inside of the hand. The laser projects an IR line across the inside of the hand, intersecting the fingers and thumb as they bend inwards and marking the distance.