ith AR and VR rapidly developing in both wearable comfort and visual quality, researchers of the technology continue to work on input solutions which will feel more natural than holding controllers.
Recently, a group of technology researchers announced a new product which hopes to combat uncomfortable wearables, named FingerTrak, which is a wristband-based solution that uses thermal cameras to track hand movements in 3D, abstracting 20 finger joint positions from contours on the wearer’s wrist.
FingerTrak was developed by Cornell University’s SciFi Lab with assistance from University of Wisconsin, Madison researchers. The FingerTrak technology uses a deep neural network to stitch together input from three or four miniature thermal cameras mounted around the wrist, which collectively capture entire hand poses.
The camera is able to generate silhouettes, which estimate fingertip and joint positions through backbone and regression networks. The results aren't quite perfect, but they have the potential to be used for some types of AR/VR input. If this technology continues to develop the other potential applications of FingerTrak could be used in mobile health, the early detection of degenerative diseases such as Alzheimer’s and Parkinson’s sign language translation and human-robot interaction and control.
The researchers who developed FingerTrak suggested that the wrist contours alone are “enough to accurately predict the entire hand posture,” which suggests that the whole sensing system could be worn on the wrist, rather than requiring gloves, rings, or other techniques that have previously been made.
A recent video provided by the developers demonstrated FingerTrak’s hand motions being tracked and translated into movements for a bionic hand, as well as enabling a computer to detect when a user is writing, drinking coffee and interacting with a phone.
However, it is still unclear as to whether the system is able to rapidly track specific gestures, for example, what a person is writing and the researchers suggest that FingerTrak’s average angular error rate ranged from 6.46 to 8.06 degrees during testing, depending on the background it was tested against. The FingerTrak technology, until further developed, may be best supplemented rather than replaced by higher accuracy finger tracking solutions, at least for certain applications.
FingerTrak is due to be presented in September at the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing.
More from
Latest News
category
