Abstract
Completing tasks on smartwatches often requires multiple gestures due to the small size of the touchscreens and the lack of sufficient number of touch controls that are easily accessible with a finger. We propose to increase the number of functions that can be triggered with the touch gesture by enabling a smartwatch to identify which finger is being used. We developed MagTouch, a method that uses a magnetometer embedded in an off-the-shelf smartwatch. It measures the magnetic field of a magnet fixed to a ring worn on the middle finger. By combining the measured magnetic field and the touch location on the screen, MagTouch recognizes which finger is being used. The tests demonstrated that MagTouch can differentiate among the three fingers used to make contacts at a success rate of 95.03%.
Prototype
MagTouch can be implemented using any smartwatch equipped with a 9 degrees of freedom IMU (accelerometer, gyroscope, and magnetometer) and a ring with a permanent magnet. The finger in below shows our implementation of MagTouch that used an unmodified LG Urbane smartwatch.
How MagTouch Works
When a hand wearing a magnet ring touches a smartwatch, the magnetic field and the touch location are determined by the contact finger, as illustrated in the figure above. It shows a user wearing a smartwatch on the left hand and a magnet ring on the middle finger of the right hand touching the screen. The positions of each fingertip relative to the center of the hand do not change when touching a surface. Therefore, it is possible to identify the touching finger from the touch location and the position of the center of the hand.
For example, if the touch point is at the left side of the right hand, then the contact was made with the index finger. We attached a magnet to a hand to get information about the hand’s location using the magnetometer. Figure b and c illustrates that the relative position of the magnet ring shifts when a user touches the smartwatch with a different finger. This leads to changes in the magnetic field vector at the smartwatch’s magnetometer.
Alone, magnetic field data is not sufficient for our purposes. Magnetic field data can be replicated by different contact fingers, as Figure d and e, unless the touching hand moves. For such cases, touch location data is used to identify the fingers accurately.
Challenge of Implementing MagTouch and How We Solved It
It is necessary to examine the physical model underlying MagTouch to understand the challenges. The magnetic field measured by the internal magnetometer of the smartwatch is the sum of the magnetic field from the magnet ring and the ambient magnetic field. The magnetic field from the magnet ring is determined by the position and the orientation of the ring relative to the smartwatch.
This in turn depends on three main variables: which finger touches the screen, the contact point on the screen, and the relative orientation of the right hand to the left hand. The figure a above illustrates these dependencies. The goal of MagTouch is to infer which finger is touching the screen from only the touch location and the magnetic field measured by the smartwatch magnetometer. This is a challenging inverse problem because of two unknown variables: the ambient magnetic field and the relative orientation of the right hand. Figure b illustrates the inverse problem.
The MagTouch method handles the ambient magnetic field using a sub-module called Computational Ambient Magnetic field Eliminator (CAME). CAME measures and saves the ambient magnetic field when the magnet ring is at distance from the smartwatch, i.e., when the magnetic field from the ring is negligible.
For the relative orientation of hands, we used data-driven approach. We used machine learning to handle variations of relative posture of a user’s two hands.
For more detail, please see our paper.