Leap Motion full hand gesture

By | 02/04/2014

After figuring out how to handle the Leap Motion’s light sensitivity issues, I started producing the actual code for my finn.no app.

The app functionality specification was rather simple: swipe a hand left or right to navigate backwards or forwards through a slideshow, respectively. Digging into the Leap Motion’s Unity3D API, I began to suspect the built-in gesture functionality wouldn’t be adequate, since the app would be exposed to the general public with little or no guidance available.

The API comes with four different gestures defined:

  • KeyTap
  • ScreenTap
  • Circle
  • Swipe

The last one, Swipe, sounded like it would be perfect for my needs. Regrettably, it’s been named a bit inaccurately; a finger swipe is what triggers it. Any finger on a detected hand doing a quick swipe, to be more precise.

I used this Swipe gesture in the first prototype for the app, but upon testing at the physical installation location, it became clear that it was too easy to misuse. It would mostly detect the swipes just fine if the user had at least one finger separate from the rest of the hand, but had trouble if the finger in question was a thumb or the hand was clenched or all fingers gathered together.

A good rule of thumb for any type of development is that if something can be used wrong, someone will eventually do it. Probably sooner than you think.

With this in mind, I did some more research into how I could get the Leap Motion to recognise a hand swipe, rather than a finger swipe. After a nerve-wrecking experience with a Unity3D plugin that didn’t turn out as expected, I found that the best way to implement this particular gesture was by not using the API’s Gesture class at all (!)

The solution was using Motions instead. Here’s the basic description (from the Leap Motion docs) of what Motions are:

The Leap Motion software analyzes the overall motion that has occurred since an earlier frame and synthesizes representative translation, rotation, and scale factors.

IE: if everything that the Leap Motion is currently tracking is moving in one direction, the resulting Motion translation shows you which direction this is. If you move one hand to the left, you’ll get a vector indicating this direction and the magnitude of the movement.

With this information in hand, all I had to do was gather this information across a number of frames, and define what magnitude of movement across these frames were enough to constitute a gesture.

Here’s some sample, simplified code(C#) – with descriptive variable names and all:

string checkForSwipe()
{
    Vector motionSinceLastFrame = 
        LeapControllerThingy.Frame().Translation(LeapControllerThingy.Frame(1));
    if (motionSinceLastFrame.x > 8.0f)
    {
        continuousFramesThatRightMovementWasDetected++;
    }
    else
    {
        continuousFramesThatRightMovementWasDetected = 0;
    }
    if(motionSinceLastFrame.x < -8.0f)
    {
        continuousFramesThatLeftMovementWasDetected++;
    }
    else
    {
        continuousFramesThatLeftMovementWasDetected = 0;
    }

    if(continuousFramesThatLeftMovementWasDetected >= 
        numberOfContinuousUnidirectionalFramesNeededForGesture)
    	{
    return swipeDirection.Left.ToString();
    	}
    else if(continuousFramesThatRightMovementWasDetected >= 
        numberOfContinuousUnidirectionalFramesNeededForGesture)
    {
    return swipeDirection.Right.ToString();
    }
    else
    {
    return swipeDirection.None.ToString();
    }
}

Leave a Reply

Your email address will not be published. Required fields are marked *