About Virtual Reality.



An Article by M. Fernando García Cabello
Summer of 2011
 
English translation by Jorge Aguirre León

Introduction.

Since his dawn, man has sought to evade the reality around him, with the sole purpose of living new experiences that were beyond his reach. Reading a book, listening to a story or watching a good movie are but different examples of this evasion, they transport us to another reality providing us fun, leisure and satisfaction throughout the journey.

This is the foundation that holds giant industries such as television, cinema, video games, the publishing and audiovisual industry overall. Therefore, the making of new forms of evasion and entertainment, with certain acceptance in society, can generate a large economic impact in the industry, particularly in the above mentioned.

Reality.

Reality is perceived as the result of the brain’s interpretation of the stimuli received from the external world through the senses. This process, like any other cognitive process, is influenced by the experiences and capabilities of each individual.

Thus, the perception of reality depends on the following:

  • Stimuli that arrive from the outside world.
  • Reception of those stimuli through the senses.
  • Their transmission to the brain.
  • Brain interpretation process, based on the experience and capabilities of each individual.

The alteration of any of those elements inevitably produces an alteration in reality.

Human being has always dreamed of living in an alternate reality.

Human being has always dreamed of living in an alternate reality.

Processes of reality alteration.

It is possible to alter reality and, thus, create artificial realities. As a matter of fact, our brain does it on a daily bases, when we dream. This is made possible by the following procedures:

  • By altering the brain process of interpretation of stimuli, through, drugs and psychiatric or psychological methods.
  • By creating artificial stimuli, which are perceived by the senses and interpreted as real stimuli.

The first procedure is hardly controllable nowadays, since the knowledge of the brain’s processing system is not sufficiently advanced. On the other hand, the effectiveness of the second one depends directly on the channels used to transmit the mentioned stimuli, often being compensated by the user’s imagination.

This last approach is used by the majority of entertainment channels that we know today and of course, by virtual reality.

The Virtual Reality.

Virtual reality is a technological system that aims to produce a perception of reality, which is artificially synthesized by the usage of computers and other devices, that allows a user to have the feeling of being present in it..

The main advantages of this system compared to more traditional ones are:

  • The experience is completely immersive. The user is isolated from the reality that surrounds him, being completely wrapped by the new reality.
  • The new reality is fully interactive, allowing the user to participate in it and make feedback.

Virtual reality’s history is extensively documented and is as vast as the history of computing and the dream of living new experiences. However, the concepts that we know nowadays became popular in the mid-80s, partly thanks to the work of the visionary scientist Jaron Lanier, to the pioneer devices developed by his company VPL Research and the success of some literary work and science-fiction movies from that decade.

Covers from 'Neuromancer' (book) and 'Tron' (movie), milestones in history and in the popularization of virtual reality.

Covers from ‘Neuromancer’ (book) and ‘Tron’ (movie), milestones in history and in the popularization of virtual reality.

Operating mode.

The operating mode is as follows:

  • The computer generates one scene or a three dimensional world, that serves as an alternate reality and in which the participating users are fully immersed.
  • The scene is transmitted to the user through a specialized device, usually by surrounding visual and auditory stimuli.
  • The user interacts with the scene, again, through specialized devices. This obligates the computer to have the capacity to manage the scene in real time.

The effectiveness of the deception, and thus, the effectiveness of the system, depends on:

  • The computer’s capacity of generating and managing in real time complex scenes with high level of audiovisual realism.
  • The immersive capabilities of the devices that transmit stimuli to the user, specially the visual and auditory.
  • The degree of naturalness in the different forms of interaction with the alternate reality that the specialized devices offer the user.
  • The degree of intrusiveness of the devices, which must be low in order to favor the overall experience.

Therefore, one can conclude that the degree of realism and immersion that a virtual reality system offers, is directly dependent on the capabilities and technological level of the equipment used to generate, manage, transmit and interact with the alternate reality.

Some virtual reality devices.

Apart from one or more computers, which are used to manage the scene and the system in general, certain specialized devices to transmit and interact with the new reality are required.

Virtual Reality Glasses (vr-glasses / vr-helmet).

It is the most important device of the system. It has two purposes:

  • Isolate the user from the real world.
  • Wrap the individual in the new reality through the transmission of visual stimuli.

The device includes two separate screens through which the scene is transmitted stereoscopically. The computer sends an image with a slight offset to each eye synchronously, thus creating a practically perfect three-dimensional impression.

The evolution of these devices in recent years has been remarkable in every respect, both in quality, accuracy and resolution of the displays, as well as comfort, weight and cost of the devices.

Comparative images of the different models of virtual reality glasses: To the left the model included in Virtuality's system, which was very popular throught the 90's. To the right, model Vuzix Wrap VR, launched august 2011. The latter includes two 852x480 point screen, a high-precision head-tracker and stero sound system.

Comparative images of the different models of virtual reality glasses: To the left the model included in Virtuality’s system, which was very popular throught the 90′s. To the right, model Vuzix Wrap VR, launched august 2011. The latter includes two 852×480 point screen, a high-precision head-tracker and stero sound system.

Head-Tracker

In addition to the virtual reality glasses often a device that collects the position and orientation of the user’s head is used. This way, the computer can synchronously update the scene with the movement of the user’s head.

Although these devices are often integrated in the glasses, they can be acquired independently if high accuracy is required.

Head-tracker model TrackIR 4 Pro, by Natural Point. This model, similar in size to a webcam, is often used in low cost flight simulation systems.

Head-tracker model TrackIR 4 Pro, by Natural Point. This model, similar in size to a webcam, is often used in low cost flight simulation systems.

Alternatively and to reduce costs, these devices can be replaced by a webcam with real image recognition software

Devices for interaction with the alternate reality.

Due to the user’s isolation from reality in virtual reality systems, the usual human-computer interaction devices, such as they keyboard or the mouse, are not valid in these systems. To fill this gap interaction devices with alternate reality have been developed over time, with more or less success. Such devices are usually oriented to capture the position and orientation of different body parts, through which the user interacts with the system.

Interaction gloves (data-globe).

Much of the real world we do, we do with hands and fingers. This is the conceptual basis of this device, the most famous and popular for interaction with a virtual reality system. It consists of a glove that captures the position and orientation of the hand and fingers and translates the data to the computer. This way, the individual is able to pick up and manipulate objects in the virtual world and therefore make interaction in the most natural way possible.

Just as with the other devices, the technological evolution of these types of gadgets over the years has been remarkable in many ways. Some of the most advanced models even include feedback systems, so that the user can feel the virtual objects by touch.

On the left, data-globe included in Virtuality's system. On the right, the model CyberGlobe from CyberGlobe II Systems, far more modern, comfortable and accurate. It's frequently used in industrial systems.

On the left, data-globe included in Virtuality’s system. On the right, the model CyberGlobe from CyberGlobe II Systems, far more modern, comfortable and accurate. It’s frequently used in industrial systems.

Although it seems like the most natural and accurate to interact with the alternate reality, the system is intrusive, not too comfortable and definitely not consolidated in the market.

Body-Tracker.

Undoubtedly, the optimal and most natural way to interact with the virtual world is through the entire body, not just through the hands. With this in mind, numerous devices have been developed over time from full suits to tracking systems with cameras and sensors of various kinds. However, until recently, these devices were extremely intrusive and not comfortable at all for continuous use and only had capacity in certain professional fields.

Various body tracking systems. On the left the motion capturing suit MVN BIOMECH, on the right, model MVN, both from XSens.

Various body tracking systems. On the left the motion capturing suit MVN BIOMECH, on the right, model MVN, both from XSens.

Prime Sense Systems and Microsft Kinect.

This trend has changed drastically in the last year, thanks to a new technology developed by Prime Sense. This company, of Israeli origin, specialized in natural human-machine interaction, offers a body-tracking device that is not intrusive, accurate and affordable.

The operating mode of the system is as follows:

  • Reality is captured through a small device that includes, among others, a sensor / infrared projector and a CMOS camera.
Prime Sense's capture device. On the same device various sensors (IR and CMOS) and the specialized chips that process information captured in real time are included.

Prime Sense’s capture device. On the same device various sensors (IR and CMOS) and the specialized chips that process information captured in real time are included.

  • The information captured by the sensors is processed in real time through a specialized chip of own design. The result is a depth image, which includes three-dimensional information and volumes. The result is similar to the data set obtained through a three-dimensional scanner using LIDAR sensors, but at a much lower cost and in real time.
Various examples of depth images generated with Prime Sense technology.

Various examples of depth images generated with Prime Sense technology.

  • This depth image, is treated again by a specialized chip in real-time, to obtain information about the three-dimensional position and orientation of the various parts of the body, creating a full virtual skeleton, which is available to the control system.
Full process schematic of the body-tracking capture system developed by Prime Sense.

Full process schematic of the body-tracking capture system developed by Prime Sense.

The result is spectacular: it works really well, it’s accurate, it does not require installation of any uncomfortable device on the body and it’s affordable.

Representation of three-dimensional skeleton obtained by the system.

Representation of three-dimensional skeleton obtained by the system.

This system has become really popular over the last year as it has been licensed and marketed successfully by Microsoft for their video game console, Xbox 360, it was initially known as Project Natal and finally as Kinect.

Kinect device, video game console Xbox 360 from Microsoft, and their advertising image.

Kinect device, video game console Xbox 360 from Microsoft, and their advertising image.

Other capturing systems.

Although its use and design are not oriented towards virtual reality, the market is full of devices that can be used as a tracker and therefore as interaction devices with the alternate reality. Examples of these devices are:

  • Last generation mobile phones, containing accelerometers, gyroscopes and compass, more than enough sensors for a basic and effective tracking.
  • Controller for Nintendo Wii and PS3 consoles.
  • Etc.

 

Visual system and system control.

An effective system of virtual reality requires a lot of processing power to run the control algorithms, visual generation and interaction with the alternate reality. Specifically in the field of vision, it must be capable of representing complex scenes with high level of realism, having to maintain a more than acceptable rate of frames per second, with the addition that each frame must generate two different images, one for each eye.

Este hecho, que años atrás si era un verdadero problema, no lo es en la actualidad. The power offered by mid-range processors on the market is more than enough to manage the system in general. On the other hand, specialized graphics processors fully relieve the main CPU from heavy processes, generating the three-dimensional scene and allowing a fine level of realism and complexity, at a very low cost.

On the left visual system from video game 'Hunter' by Activision, from the 90's. On the right, the visual system 'Unreal Engine', used nowadays. The differences in quality and realism are but remarkable between these real-time systems.

On the left visual system from video game ‘Hunter’ by Activision, from the 90′s. On the right, the visual system ‘Unreal Engine’, used nowadays. The differences in quality and realism are but remarkable between these real-time systems.

In this respect it is worthy to note that the development and most important advances in this direction are strongly linked to the world of video game and entertainment in general.

Conclusions. Present and future of virtual reality.

The concept of virtual reality has been many years in gestation, waiting for the right technology for mass popularization and implementation. After years of experimental developments, passion and enthusiasm from their researchers and followers, spectacular successes and spectacular failures, it’s time has come.

Today’s technology allows this, since:

  • Processors and visual systems offer more than enough power to generate, maintain and manage the system, in all its aspects.
  • The capabilities offered by virtual reality goggles available today are very high: they have good visibility, high resolution, are comfortable, and do not produce any visual fatigue.
  • The capabilities and variety of interaction and tracking devices allow interaction with the system in a natural, accurate way with a very low learning curve.

And all this at a very low cost, if they are mass produced, as in the case of Microsoft Kinect.

These facts, along with the popularization of the Internet and social networks, whose link with virtual reality can give a new, revolutionary and explosive focus on the concept of virtual relationships and leisure in general, are just a few touches of bright future provided by this technology.

Big technology brands have realized this and the facts bear this out. Nintendo led the way with its successful Wii console, Microsoft followed with its revolutionary Kinect and soon, Sony will present a model of virtual reality glasses for PlayStation 3. This latest release will certainly be a milestone in the spread of virtual reality and become a turning point in the advancement and evolution of forms of digital entertainment.

Prototype viewer 'Personal 3D Viewer' or HMZ-T1 by Sony.

Prototype viewer ‘Personal 3D Viewer’ or HMZ-T1 by Sony.

In conclusion, in the present, the union of a midrange PC, Vuzix Wrap 1200 VR glasses, a Kinect device and an Internet connection offers some truly amazing possibilities in all aspects, at a very low cost. This is surely something that is worth looking into; to invest, develop and spread to the rest of the community.

In the future, only imagination will put limits to this technology.

Next: The project basis