Completing tasks on smartwatches often requires multiple gestures due to the small size of the touchscreens and the lack of sufficient number of touch controls that are easily accessible with a ﬁnger. We propose to increase the number of functions that can be triggered with the touch gesture by enabling a smartwatch to identify which ﬁnger is being used. We developed MagTouch, a method that uses a magnetometer embedded in an off-the-shelf smartwatch. It measures the magnetic ﬁeld of a magnet ﬁxed to a ring worn on the middle ﬁnger. By combining the measured magnetic ﬁeld and the touch location on the screen, MagTouch recognizes which ﬁnger is being used. The tests demonstrated that MagTouch can differentiate among the three ﬁngers used to make contacts at a success rate of 95.03%.
MagTouch can be implemented using any smartwatch equipped with a 9 degrees of freedom IMU (accelerometer, gyroscope, and magnetometer) and a ring with a permanent magnet. The finger in below shows our implementation of MagTouch that used an unmodiﬁed LG Urbane smartwatch.
How MagTouch Works
When a hand wearing a magnet ring touches a smartwatch, the magnetic ﬁeld and the touch location are determined by the contact ﬁnger, as illustrated in the figure above. It shows a user wearing a smartwatch on the left hand and a magnet ring on the middle ﬁnger of the right hand touching the screen. The positions of each ﬁngertip relative to the center of the hand do not change when touching a surface. Therefore, it is possible to identify the touching ﬁnger from the touch location and the position of the center of the hand.
For example, if the touch point is at the left side of the right hand, then the contact was made with the index ﬁnger. We attached a magnet to a hand to get information about the hand’s location using the magnetometer. Figure b and c illustrates that the relative position of the magnet ring shifts when a user touches the smartwatch with a different ﬁnger. This leads to changes in the magnetic ﬁeld vector at the smartwatch's magnetometer.
Alone, magnetic ﬁeld data is not sufﬁcient for our purposes. Magnetic ﬁeld data can be replicated by different contact ﬁngers, as Figure d and e, unless the touching hand moves. For such cases, touch location data is used to identify the ﬁngers accurately.
Challenge of Implementing MagTouch and How We Solved It
It is necessary to examine the physical model underlying MagTouch to understand the challenges. The magnetic ﬁeld measured by the internal magnetometer of the smartwatch is the sum of the magnetic ﬁeld from the magnet ring and the ambient magnetic ﬁeld. The magnetic ﬁeld from the magnet ring is determined by the position and the orientation of the ring relative to the smartwatch.
This in turn depends on three main variables: which ﬁnger touches the screen, the contact point on the screen, and the relative orientation of the right hand to the left hand. The figure a above illustrates these dependencies. The goal of MagTouch is to infer which ﬁnger is touching the screen from only the touch location and the magnetic ﬁeld measured by the smartwatch magnetometer. This is a challenging inverse problem because of two unknown variables: the ambient magnetic ﬁeld and the relative orientation of the right hand. Figure b illustrates the inverse problem.
The MagTouch method handles the ambient magnetic field using a sub-module called Computational Ambient Magnetic ﬁeld Eliminator (CAME). CAME measures and saves the ambient magnetic ﬁeld when the magnet ring is at distance from the smartwatch, i.e., when the magnetic ﬁeld from the ring is negligible.
For the relative orientation of hands, we used data-driven approach. We used machine learning to handle variations of relative posture of a user's two hands.
For more detail, please see our paper.