See-through 3D computer with gesture controls gives us a glimpse of the future

By Zach Epstein

Some believe a future full of massive, gesture-controlled computer displays like the ones seen in Twentieth Century Fox’s Minority Report are an inevitability, and a prototype PC designed by an intern with the Microsoft Applied Sciences Group may be among the first steps in making that future a reality. Massachusetts Institute of Technology Ph.D student and MIT Media Lab research assistant Jinha Lee recently set out to change the way we interact with desktop computers. While progress has been made with 3D display technology, 3D has not yet proliferated in the personal computing space and Lee wants to change that. The end result of his work is a fascinating desktop computer with a transparent 3D display and a unique gesture-based interface that could change the way we use computers.

“Computer scientist Ivan Sutherland once called a computer display ‘a looking glass into a mathematical wonderland,’ and I have always aspired to walk in this wonderland to interact with those abstract beings,” Lee wrote on his blog. Lee’s creation, built in cooperation with researchers and engineers at the Microsoft Applied Sciences Group, does just that.

Lee’s PC uses a transparent display with 3D technology and gesture recognition. Gestures are performed behind the angled display rather than in front of it as with most gesture-based control systems, and a front-facing camera system tracks the user’s head so the 3D perspective shown on the computer’s display adjusts appropriately when the user shifts his or her position. The machine’s operator can also control the computer with a standard keyboard and trackpad positioned behind the display.

A video showcasing Lee’s 3D computer follows below.

See Through 3D Desktop, from Jinha Lee on Vimeo.

The user interface on Lee’s PC shows stacks of windows in a 3D environment. The computer tracks the user’s hands on all three axes, allowing the operator to grab items shown on the screen and interact with them in all three dimensions by moving virtual objects along the x, y, or z axis. This innovative combination of solutions opens a wide range of possibilities for PC software and is far more advanced and versatile than touchscreen technology, which only provides two dimensions that the user is able to interact with.

“Users can casually open up the See-Through 3D Desktop and Type on the keyboard or use a trackpad as in traditional 2D operating environment,” Lee explains. “Windows or files are perceived to be placed in a 3D space between a screen and the input plane. The user can lift up his hands to reach the displayed windows and arrange them in this 3D space.”

Lee has not addressed the possibility of a production version of his see-through 3D desktop PC.