FingerIO allows you to interact with mobile devices by writing or gesturing on any nearby surface by turning a smartphone or smartwatch into an active sonar device. Credit: Dennis Wise, University of Washington
As mobile and wearable devices such as smartwatches grow smaller, it gets tougher for people to interact with screens the size of a matchbook. That could change with a new tech developed by Un of Washington computer scientists and electrical engineers called FingerIO. It tracks fine-grained finger movements by turning a smartphone or smartwatch into an active sonar system using the device’s own microphones and speakers.
Because sound waves travel through fabric and do not require a line of sight, users can even interact with a phone inside a front pocket or a smartwatch hidden under a sweater sleeve. FingerIO can accurately track 2D finger movements to within 8mm, which is accurate to interact with today’s mobile devices.
Using FingerIO, one could use the flick of a finger to turn up the volume, press a button, or scroll through menus on a smartphone without touching it, or even write a search command or text in the air rather than typing on a tiny screen. FingerIO turns a smartwatch or smartphone into a sonar system using the device’s own speaker to emit an inaudible sound wave. That signal bounces off the finger, and those “echoes” are recorded by the device’s microphones and used to calculate the finger’s location in space.
Using sound waves to track finger motion offers several advantages over cameras – which don’t work without line-of-sight, and radar that require both custom sensor hardware and greater computing power.
UW researchers employed a type of signal typically used in wireless communication – Orthogonal Frequency Division Multiplexing – and demonstrated that it can be used to achieve high-resolution finger tracking using sound. Their algorithms leverage the properties of OFDM signals to track phase changes in the echoes and correct for any errors in the finger location to achieve sub-centimeter finger tracking.
To test their approach, the researchers created a FingerIO prototype app for Android devices and downloaded it to an off-the-shelf Samsung Galaxy S4 smartphone and a smartwatch customized with two microphones, which are needed to track finger motion in two dimensions. Today’s smartwatches typically only have one, which can be used to track a finger in one dimension.
The researchers asked testers to draw shapes such as stars, squiggles or figure 8s on a touchpad next to a smartphone or smartwatch running FingerIO. Then they compared the touchpad tracings to the shapes created by FingerIO’s tracking.
The average difference between the drawings and the FingerIO tracings was 0.8 centimeters for the smartphone and 1.2 centimeters for the smartwatch.
Next steps for the research team include demonstrating how FingerIO can be used to track multiple fingers moving at the same time, and extending its tracking abilities into 3D by adding additional microphones to the devices. http://www.washington.edu/news/2016/03/15/smartwatches-can-now-track-your-finger-in-mid-air-using-sonar/
Recent Comments