Communications of the ACM
Engineers Develop Real-Time, 3-D Teleconferencing Technology
Nik Karpinsky with Iowa State University's 3-D teleconferencing prototype displaying an image of Zeus.
Credit: Nik Karpinsky
Nik Karpinsky quickly tapped out a few computer commands until Zeus, in all his bearded and statuesque glory, appeared in the middle of a holographic glass panel mounted to an office desk.
The white statue stared back at Karpinsky. Then a hand appeared and turned the full-size head to the right and to the left. Yes, it was quite clear, Zeus really was pictured in 3-D.
And there it was from one computer work station on the second floor of Iowa State University's Howe Hall to another down on the first floor: 3-D teleconferencing that's live, real-time, and streaming at 30 frames per second.
"Four years ago, this would not have been possible," says Karpinsky, an Iowa State doctoral student in human computer interaction who's been working day and night to make the technology a reality.
Part of the problem is the complexity of the technology, says Song Zhang, Iowa State's William and Virginia Binger Assistant Professor of Mechanical Engineering, an associate of the U.S. Department of Energy's Ames Laboratory, and the leader of the 3-D imaging project.
"There are a lot of skills involved," Zhang says. "You have to do programming, optical engineering, hardware, software, and networking."
To make it all work, Karpinsky and Zhang had to solve three big technical problems: capturing the 3-D images, transmitting the images, and displaying the images.
"I was originally worried about transmission," Karpinsky says. "But we had to focus on all three."
The result of successfully combining those technologies is a proof-of-concept prototype that Karpinsky and Zhang call "Portal-s."
It all starts with a projector that shines a light straight at a teleconferencer, in this case, that bust of Zeus. There's a camera to the right of the projector and one to the left, both angled toward the subject. The cameras record two images of the light as it's distorted by the subject. The images are combined to create a single 3-D image.
That optical hardware is networked and connected to a standard computer with a graphics card. The computer combines, processes, and compresses the images. (And it really compresses them — from 700 megabits per second to less than 14 megabits per second.)
The compression allows transmission of 3-D images to another computer, even over wireless networks.
The idea, Karpinsky says, is for the projectors to become the eyes of the teleconferencing system: "What the projector sees is what you see."
Karpinsky and Zhang see a bright future for the technology they've developed with the help of support from the U.S. National Science Foundation and Iowa State's Virtual Reality Applications Center.
Zhang says the next steps include developing and testing applications for smart phones. He thinks the technology is only a few years away.
"In the future, we can do all of this 3-D video conferencing on the phone," he says. "These phones are powerful enough to do all the computation."
Zhang also wants to develop the 3-D teleconferencing technology for use in powerful virtual reality environments such as Iowa State's C6, a six-sided room that surrounds users with 100 million pixels of 3-D images.
All of these 3-D developments, Zhang says, are coming far faster than he expected.
"When Nik first proposed this idea to me," he said, "I never believed we could reach this level by now."
Karpinsky won't be part of the continuing research and development work at Iowa State. He graduates this semester and will move to Washington state to work for Microsoft. He'll also work for a startup imaging company called Phasica3D that spun out of Iowa State research.
No entries found