Presence Examples

A virtual world is taking shape in research labs technology

 

From The Los Angeles Times
(http://www.latimes.com/business/20010205/t000010695.html)

 

A VIRTUAL WORLD IS TAKING SHAPE IN RESEARCH LABS TECHNOLOGY:

The Internet of the future will surround users with sights, sounds, even smells.

 

By KAREN KAPLAN, Times Staff Writer

February 5, 2001

 

One day last fall, virtual reality pioneer Jaron Lanier leaned across a desk and looked directly into the eyes of his colleague Bob Zeleznik. Well, almost.

 

Lanier actually was looking at a computer-generated image of Zeleznik in an experimental "telecubicle," a  half-real, half-virtual work space that connects two offices and makes them seem as if they are one.

 

That enabled Lanier, sitting in a lab at the University of North Carolina at Chapel Hill, to sit face to face withZeleznik, a Brown University computer graphics researcher who was 540 miles away in Armonk, N.Y. In addition to seeing each other as computer-generated creations sharing the same office space, Lanier and Zeleznik also could see--and even rearrange--certain pieces of virtual office furniture that existed only in cyberspace.

Though still crude, the telecubicle offers a glimpse of the Internet of the future--one that will be liberated from computer monitors and infused with the essential senses of sight, sound and touch.

The goal is to create realistic digital worlds where computer-generated avatars will become realisticstand-ins for actual people, surround-sound audio systems will emulate real-life noises and force-feedback technology will reveal the shape and texture of physical objects from across a computer network.

Researchers around the country already are developing the basic building blocks to create what is being dubbed the fully immersive Internet.

 

They are hoping that in 10 to 15 years, the Internet no longer will be confined to computer desktops, two-way pagers, wireless phones and other digital gadgets. Instead, Internet users will be able to literally surround themselves with the global computer network.

 

"We are born, we live and we're going to die in a 3-D environment," said Max Nikias, director of USC's Integrated Media Systems Center, which is working on technologies for immersive environments. "It's more natural to interact this way." An encounter in the immersive Internet will "give you all of the cues that really would be present if you were with the person physically," said Lanier, who is now chief scientist at Advanced Network & Services, a nonprofit Internet research group based in Armonk, N.Y. "Once you can do that, a whole world of body language and gestures becomes operable that expands the possibility for really communicating well."

An immersive Internet also could enable individuals to explore other communities and cultures and even create new ways for them to interact with information on the World Wide Web. "Future Web sites won't just be text but also software that allows you to download information and synthesize it in 3-D," Nikias said. "For instance, with ImmersiNews, you could put a reader in Kosovo."

By the same token, groups of engineers meeting in cyberspace to discuss a new power plant wouldn't have to rely on two-dimensional blueprints. Instead, they could walk together through a virtual version of the proposed plant.

Researchers Grappling With Sensory Input

 

For the National Science Foundation, the federal agency that has invested more than $16 million in immersive Internet research at USC alone, it's simply common sense to expand the amount of sensory information available on the Net. "It gives people more input," said NSF Program Director Mita Desai. "Right now if I write you a letter, you just know the text but you don't hear the tone of my voice. If we talk on the phone, you know whether I sound angry or happy. If you could see me, you could see my expression at the same time. The more sensory input, the better it is." About a dozen American universities are doing research on various aspects of immersive technologies, including USC, the University of North Carolina, the University of Pennsylvania and Brown University.

The researchers are grappling with some of the most delicate problems in computer science. Even the most basic pieces of the immersive Internet are fraught with pitfalls.

At USC's Integrated Media Systems Center, researchers are developing techniques to create 3-D facial models from a pair of two-dimensional pictures. The pictures are used to identify geometric deformations (such as the bulges in the cheek created by a smile) as well as the face's texture (wrinkly or smooth) to re-create a lifelike image.

Graduate student Doug Fidaleo is analyzing video images to determine what goes on beneath the skin to make facial models more realistic. He also is studying the details of facial expression so that the models can provide clues about a person's mental state. Such touches are designed to convey some of the most subtle elements of communication--the sly smile, the sarcastic eye roll, the doubtful raised eyebrow. Fidaleo also is teaching computers to recognize facial expressions in humans. "Expression recognition gives the computer a higher level of knowledge of how you're feeling," he said. "It helps with the computer-human interface." The problem is that human senses are so refined that even the slightest computer errors are easily noticed.

It's Hard to Replicate Shapes and Textures

 

Consider the seemingly simple problems of synchronization (making sure an avatar's mouth is moving at the same time his words are spoken) and latency (the delay between when an avatar is spoken to and when it responds).

 

"Even if there are a few microseconds of delay, people can notice," Nikias said. "It can't be more than 50 microseconds. You need to have it right for things like eye contact."

 

Even those problems are simple compared with the task of duplicating the sense of touch.

 

Researchers are working on a force-feedback technology called haptics, which is designed to enable people to feel the shape and texture of virtual objects.

 

Advances in haptics could allow blind people to feel objects that others can see. A museum Web site could allow visitors to feel the shape of an ancient Greek urn, for example, and an e-commerce site could let shoppers feel the texture of a fur-trimmed wool coat.

 

But they have been struggling to replicate even some of the most basic sensations.

 

"Creating devices that can faithfully measure and reproduce touch is really challenging, and making the Internet powerful enough to transmit that information is very ambitious," Lanier said.

 

Commercially available devices enable users to poke with a stylus or grab with a glove to get a rudimentary feel for the shape of an object depicted on a computer screen.

 

But Joao Hespanha, an assistant professor of electrical engineering systems at USC, said trying to duplicate such simple textures as "soft" or "spongy" will take years or even decades.

 

"The device has to apply a force on you that mimics the force you would get in real life," he said. "To make an object squishy is far more difficult because the model changes when you touch it. A rigid object is easier."

 

Whether digital tricks will ever be good enough to seduce people into thinking that these virtual environments are real is a matter of much debate.

 

Some researchers believe that it's just a matter of time before advances in technology will catch up to the awesome--though finite--abilities of the human sensory system. Others, like Lanier, think human senses learn from technology to continually stay at least one step ahead of the pretenders.

 

"If you go back 100 years, there were experiments where subjects were unable to distinguish a recorded opera singer from a real singer behind a curtain," he said. "In the early days of cinema, people ran out of the theater when a train was coming right at the camera. Our experience with technology has caused us to learn and to hear better."

 

Even the most optimistic people doubt that digital systems will ever truly mimic the sensations of touch, smell and taste. (For starters, what kind of user interface could even transmit flavors and odors?)

 

There is some reason for hope. The sense of sound--dynamic sound that comes from many

directions at once--already has been replicated to an acceptable level.

 

Chris Kyriakakis, an assistant professor at USC's Signal & Image Processing Institute, has developed a system to provide virtual surround sound so that a single audio stream coming over the Internet sounds as if it's coming from a full array of speakers.

 

Basically, Kyriakakis uses a set of filters to expand a single audio stream into multiple channels.

 

Each channel is manipulated so that it sounds as if it is emanating from a different place. To create the filters to reproduce the sound in a concert hall, for example, he outfits a real auditorium with as many as 24 microphones and records a musical performance.

 

Then he uses a process called adaptive filtering to compare the sound waves captured by the various microphones. Those comparisons produce a digital filter so that in the future, an audio stream from a single microphone can be expanded into multiple channels.

 

 A similar process can turn a single pair of speakers into a virtual set of surround-sound speakers.

 

Once all this immersive technology comes to pass, some people probably will still prefer the Internet and other communication technologies, such as videoconferencing, the way they are today, said Desai, the NSF program director.

 

"Maybe for some people, immersipresense is too much," she said. "Those people won't be shut out of the Internet. This is just in addition to what's already there."