Most “true” multi-touch system out there right now, such as Microsoft Surface or Reactable, use a projector and a vision system built around Frustrated Total Internal Reflection (FTIR). This requires a big machine or setup to allow enough distance for the projector to display the image on the screen. The bigger the screen, the farther away the projector needs to be. Of course, you can offset this if your projector has really good optics and can focus-in closer, but most commercial products out there bottom out at around 2′ or 3′ feet.

So how do the iPhone and iPod Touch do it?

Certainly, these devices are way too thin to be doing any kind of optical trickery, so the answer (I think) must lie in a tried-and-true technology most of us have been using for years. I suspect it’s resistive touch screen panels. This is what is used on ATMs and interactive kiosks.

I’ve had a chance to play around with a couple of iPhones lately, and it struck me that all the gestures are either one (tap) or two fingers (pinch). This makes perfect sense, since you really can’t hold on to the device and perform a three-finger gesture. That would be like trying to hit ctrl-alt-tab-shift-F12 on your keyboard using only one hand.

You don’t need multi-touch. It should be more like bi-touch or duo-touch.

I’ve been racking my brain on this one for months, but the final clue came yesterday with this graphic shown on the Apple website. As you can see, it shows two points being touched over two “mystery” layers on top of this cool grid that has two bumps on it corresponding to the touch points. Eureka!!!

If all you need is two touch points at the most, you just throw-on two resistive “layers” over the LCD and you can use either hardware or software techniques to ignore the extra touch points. If the “top” layer senses one finger, the “bottom” layer agrees. If you put down a second finger, the “top” layer ignores it and the “bottom” layer can take over the tracking for it. Shear genius, Apple.

I’m not saying that, yes, this is how Apple is doing it… but at least, this is probably how I would have done it. It’s a tried-and-true technology, it’s inexpensive, and it’s really all you need to do hardware-wise to build such a great product around.

Yet another example of Apple’s incredible vision, innovation, and ability to use current technology and materials in very creative ways. I guess there’s no more need for a transitional product like the “WidePod”.

This means the Zune is done, by the way.

iPod Touch and iPhone are trademarks of Apple Inc. Image is Copyright © 2007 Apple Inc. Zune is a trademark of Microsoft.

I’ve been looking around for information on how to build a setup to do my own multi-touch and interaction experimentation. I recommend you go over to see the reactable project that the “Interactive Sonic Systems” team is working on at the Universitat Pompeu Fabra in Barcelona Spain.

These examples gave the idea to start out with a simple interface experiment. What if you had a circular window that you can manipulate with two fingers? Moving the touch points around would in turn move the circle, and the size of it would depend on how close or far apart your fingers are. Inside the circle you would see that part of an image or video that is “behind” it. It would look something like this:

Building a working multi-touch table will require some fancy fabrication and a chunk of money. so, in order to not get discouraged while that part of the plan comes to fruition I’ve decided to start with the fun part, which is the software. I can always adapt it later once the hardware is done.

When one takes-on a fancy visualization or interaction experiment like this, I recommend you begin by working out a solid mathematical model on paper before you dive into the code. Doing this will save you countless headaches and dead-ends during the coding and will prevent you from giving up too soon. Trust me, I’ve been there. And so, here’s what we have…

Problem: The interface consists of a circular object that is positioned exactly between two control points. The distance between their centers determines its diameter, and moving them changes both the scale and position of the circle relative to the control points.

This lead me to work out the following mathematical model:

It may look scary, but this is good old algebraic geometry from highschool. We know at all times the coordinates of both control points, so we’ll call them X1,Y1 and X2,Y2. To determine the diameter of the circle we are going to draw between them we use the basic 2D distance formula,which we are calling d. And you guessed it, the exact x,y position of the circle is the mid-point between our two controls. I was originally going with some super-fancy trigonometric voo-do, but then I dusted-off the old math book and found this little nugget. The midpoint between two known points is merely the average of their coordinates. That formula is described by C. Having a solid model like this will take care of negative values in the coordinate system by itself. They will cancel each other out. This will save you mad “if” statements.

And here is the first prototype done using Flash Actionscript:

Taking it to the next level, this example is closer to what we are trying to do:

And finally, for those of you who happen to have a webcam, try this one!

If you have any examples, links, or anything else, feel free to drop a comment or two.

Special thanks to Gabriel “George Castanza” Ruiz. Hope this launches his Hand-modeling career.