The project basis.

An Article by M. Fernando García Cabello

Summer of 2011

 

English translation by Jorge Aguirre León

Background.

Nowadays, the resources associated with virtual reality are really scattered. The situation is as follows:

• The user has a device that has a very limited number of resources, and applications and sometimes it has to be modifed by installing drivers or patches of questionable stability and operability.

• The application’s developer are limited to supporting a few devices and use private, exclusive, and often poorly documented APIs and SDKs. Furthermore, each software has to design and implement their own user interface with the alternate reality.

• Device manufacturers do not have any standard or platform for which to develop their devices and drivers.

This situation does not favor the spreading of this technology, which will fail to consolidate and be implemented in most households as a new digital entertainment center until:

• The experience offered is far superior to anything available nowadays.

• It becomes accessible, comfortable and easy to use.

• A more or less abundant catalog of software and applications is made available.

• A centralized platform that serves as a link to all the users or communities is made available.

• The devices are mass produced and reach affordable prices.

 

The concept.

The aim is to create a single centralized virtual reality content consumption platform, that offers:

To the final user:

• A new concept of leisure and digital center, with a high degree of social component.

• A unique entrance to a rich catalog of services, applications and products of all kinds, adapted to virtual reality and related to leisure and productivity.

• A unified user interface, consolidated and efficient, used by all applications.

• Access to the same universe of services, regardless of the platform, available devices, and the machine used by the user.

To the developer:

• A set of resources, documentation, APIs and SDKs that allow focus on the development of applications and experiences themselves, delegating to the platform everything to do with the hardware, devices, user interfaces and electronic commerce.

• A showcase centralized for the distribution and sale of its applications and virtual reality products.

To the device manufacturer:

• A direct channel of distribution of its products.

• A consolidated platform of products and applications compatible with their devices.

• A consolidated platform for which to develop their devices and drivers.

• A showcase for new and potential customers for their products.

To the rest of the market, a stranger to technology and virtual reality:

• A new advertising channel to market their products.

In conclusion, this is to follow the successful path taken by other well-known content consumption platforms, such as:

• iOs + AppStore + iTunes.

• PlayStation Network.

• Xbox LIVE.

• Android + Android Market.

But in our case, linked exclusively to devices, content and experiences consumed through virtual reality.

Mode of use.

The mode of use is very simple. The user connects his virtual reality devices, system startup and optionally authenticated. At this point the system presents, through a three-dimensional user interface or main menu all the possibilities of the universe of services, applications and experiences available.

User interface and entrance to the system: The Gate.

Today, all of the user interfaces available for different operating systems and platforms are essentially two-dimensional. This is because they are viewed through a monitor or flat screen. In virtual reality systems, and due to the predominant use of glasses for viewing in three dimensions, the concept changes dramatically. On the one hand, this interface must necessarily be three dimensional and on the other hand it has to be easily handled by the different tracking devices available. Therefore, it is considered a very important part of the success of the project the analysis and design of an effective three-dimensional user interface. A priori, the interface must be:

• Atractive.

• Easy to handle.

• Powerful: it has to exploit the possibilities offered by the three dimensions so that the experience is far superior to traditional interfaces.

• And above all transparent, so that the user completely forgets he’s using a computer, giving much more importance to the experience and to the detail than to the technology itself. This, which may seem obvious, is something that only certain devices have achieved in recent years – like the iPad and the iPhone – and has been an extremely important factor in their success.

Although there are no consolidated precedents of this kind of user interfaces, there are numerous imaginative designs from science fiction films and numerous experimental designs that could serve as inspiration for the first sketches.

The user interface will be initially used by the main application, or entrance to the system. This application, called The Gate, is the gateway to all the possibilities the universe of services, applications and experiences available in the system. The Gate acts similarly to the desktop in a traditional operating system or as iTunes and the main screen on the iPhone or iPad devices.

Finally, the user interface must be available to the developer community for it’s use and integration into all applications designed for the platform.

Some basic services.

To make the system fully operational, some basic applications or services that offer the user a sufficient number of experiences to be satisfied, have continuity and even want to invest to get more experiences, must be available from the start. This catalog of applications and initial services, will be gradually supplemented by third party applications.

Most of the proposed applications below offer services already available in traditional operating systems, however the possibilities offered by the inclusion of a three-dimensional interface can elevate the experience to a much higher level.

Image Viewer, photo albums and stereoscopic and holographic images.

There are obvious possibilities to navigate, search, sort and display images on a three-dimensional immersive environment. In fact there have been numerous experiments on this issue on traditional systems. However, the use of these applications on a traditional monitor or display is an impediment for everyday use. Only through the use of virtual reality such applications can show all their capacities and be exploited by the user in depth, in a natural way.

To this the stereoscopic image display must be added. Lately, the market’s number of devices capable of performing this type of three-dimensional photographs has multiplied and virtual reality offers the most natural environment for viewing.

On the other hand, the viewer must be compatible and able to view and manage images from the most popular platforms for storage and distribution of images in the cloud, Flickr, Picasa, etc.

The holographic image.

By linking a depth image and a traditional image a volumetric image, three-dimensional image or hologram is obtained. This technology, which has not yet been sufficiently exploited and has been like science-fiction for years, finds in virtual reality the perfect environment for exploitation, as image, video or telepresence.

Support for visualization and exploitation of such images on the viewer is proposed.

Viewing the following video to appreciate the immense possibilities offered by this new technique is recommended.

Three-dimension videos and movies viewer.

Using virtual reality goggles allows simulating the projection of videos and movies on a virtual surface of a much larger size than televisions and conventional video systems. This fact, paired with the growing popularity of films shot in three dimensions and the possibility of seeing holographic videos makes virtual reality the ideal environment for watching videos and animations.

Also, as with the viewer, the possibilities to navigate, search, sort and view videos on a three-dimensional immersive environment are enormous.

In the same way, the viewer must be compatible, and able to view and manage videos from popular platforms for storage and distribution of videos in the cloud such as YouTube, Yahoo! Video, etc.

Digital jukebox.

The digital jukebox is presented as a complement to the rest of the media universe offered by the platform. Its use must be very similar to the display of images and videos when browsing, searching, sorting and viewing albums and music in general.

On the other hand, it seems very attractive to include a display that wraps the user in an immersive three-dimensional universe of relaxing psychedelic images generated by the system to the rhythm of music.

Similarly, the integration of certain popular music services like Shazam, receiving internet radio stations into the application would be very useful.

Web Browser. The network has a new dimension.

A system of this kind would be quite incomplete without the inclusion of an integrated web browser in it. This web browser, based on some known free software kit, like Gecko or Web Kit, must be designed to facilitate browsing, visualization and management of multiple pages or sites using its three-dimensional user interface.

The network acquires a new dimension.

Currently, all standards on which the publication and display of websites is based have been designed to display content in two dimensions. This concept is sound and works well for viewing on traditional systems. Virtual reality opens up new and exciting ways for this matter.

To enhance and maximize the web experience on virtual reality the analysis, design and implementation of a high-level descriptive interactive three-dimensional environment is proposed.

The scheme must allow the creation of interactive websites in three dimensions easily, through HTTP protocol, the use of a specific subset of XML tags, and existing customer different languages ​​- JavaScript, Python, etc. There are in this regard some priors, like VRML, which is very outdated and obsolete, or the more recent WebGL language, which is too complex and low level for what is sought. It is through the use of virtual reality where this kind of language can be accommodated.

The possibilities are huge in all aspects – including advertising – and its use is ideal for small content delivery applications that do not require a lot of power or specific code.

Finally, to promote the use of this technology, it would be ideal to make available to developers and web site designers an application that allowed the design and construction and debugging of such three-dimensional pages visually, obviously using virtual reality for it.

Maps.

There is no better environment to display maps than virtual reality. The universe has three dimensions and the maps we handle are usually no more than projections made to transform a three-dimensional reality to a flat plane.

The experience of sailing and flying through surrounding three-dimensional can be fantastic and at the same time accurate. Man has always dreamed of flying freely throughout the world and this may be the closest experience to reality. In fact, over the last few years very popular three dimensional map display systems have emerged in the market, which we all handle very naturally, and which virtual reality can lift up to the highest level.

Productivity: email, calendar, contacts and document viewer.

It is unclear that the use of virtual reality could improve or optimize the environments and processes. What is obvious is that it is a very important sector for the success of the platform. Therefore it is considered necessary to further analyze and design user interfaces optimized in this respect.

The system should include at least support for:

• Email, including the ability to attach experiences and scenes in 3D, etc..

• Calendar.

• Contacts.

• Document Viewer: in PDF format, Microsoft Office, OpenDocument, etc..

• File browser.

• Advanced System notifications.

• Etc.

 

Social networks and telepresence.

The importance and the changes introduced by the social networks in society over the past few years is clear. The possibility of sharing ideas, events, images, news or a simple line of text instantly with a limited group of people or an entire community, has opened amazing possibilities. These possibilities have to be necessarily present in the system, through agreements with large enterprises, enhancing it using immersive three-dimensional environments and expanding their possibilities.

In this regard, one of the areas where virtual reality can bring big changes is in direct communication between two or a group of people who are geographically distant. This can be done in a conventional manner by video conference, with the obvious improvement of it’s management in a three dimensional environment. However it is in the following and innovative forms of communication where virtual reality can bring dramatic changes:

• Communication via avatar or virtual puppet: where the user is represented by a more or less realistic three-dimensional model and whose resemblance to the person in question is chosen by the user. This avatar, which internally has a virtual skeleton, imitates the user’s behaviour according to different tracking systems. Avatars can be situated in any three-dimensional environment chosen by the users.

• Communication via telepresence or holograms: This form of communication is totally experimental and innovative and can be a real revolution in XXI century personal communications. It involves using various volumetric images or holograms for communication. Since these images are captured in real time, they can be filtered to remove the background static, transmitted over the network and placed in a shared three-dimensional environment. As a result, each person visualizes the others in all their dimensions. The possibilities are amazing in all aspects: the holograms can interact with each other, exchange objects or information, touching, etc.

A system that implement both forms of communication, as well as implementing other more traditional such as video-conference or chat is proposed. Any user that is in the system must be able to use this communication system with extreme ease.

The Experience Store and E-commerce.

The system will offer users a centralized store of products, services, experiences and applications, much along the lines of other popular platforms. This store, called ‘The Experience Store” (TXS) – will complement the basic services offered by the system and will be the showcase for different developers and manufacturers. Obviously TXS will have a totally immersive three dimensional user interface.

The idea is not new, but is a concept that really works, because:

• It simplifies the work for end users who know where to look, it is easy and safe to shop and compare products.

• It unifies all users in a unique showcase, potential customers for developers, manufacturers and advertisers of all kinds of products.

The platform has to provide a very solid foundation from which to distribute all kinds of applications in numerous sectors, to which virtual reality can bring a new and revolutionary view. Some of these sectors are:

• Education: It’s been proven that a more practical, interactive and visual teaching reduces learning time. Living a virtual experience of historical fact, see and touch a molecule or a cell, explore geography can be unique experiences. Virtual reality can even shorten learning time in a remarkable way in exact sciences.

• Medicine: there are numerous background of closed and private virtual reality systems that operate on many fields of medicine. Some of these are:
  • Diagnosis.
  • Preoperative planning.
  • Education and testing.
  • Guided surgery.

• Psychology and psychiatry: in this sense, this technology is especially powerful, there even exist specialized clinics in this regard.

• Simulation: any environment can be simulated by virtual reality, reducing formation costs, avoiding or minimizing it on real environments. Some sectors are:
  • Flight Simulation.
  • Simulation of industrial or military vehicles.
  • Simulation of marine vehicles: navigation, fishing, transport, etc.
  • Simulation of ground vehicles in general.
  • Simulation of any industrial control process.

• Industrial and professional sector: technology and three-dimensional immersive environments offers greater control possibilities than traditional systems and broadens the possibilities in all sectors.
  • Remote control and monitoring systems.
  • Aviation in general.
  • Unmanned Aerial Vehicles (UAVs) and remote control.

• Leisure: consumption of content and media, news, weather, shopping and more.

• Video-games: the scope in this sector is so large and obvious that it does not deserve a wider development. As an indication, the video game industry today moves more capital than any other entertainment industry and will surely mean much of the business volume of the platform.

• Travelling: the possibilities are broad: virtual tours, tourist guides etc.

• Social networks and telecommunications.

 

Architecture and general outline.

It is somewhat complex and precipitated to define the architecture of the system in detail in such an early stage of the project. Only a few concepts will be listed:

• The system initial is proposed to be multiplatform.

• Building and marketing a unique system, consisting of a computer, virtual reality glasses and a tracking system like Microsoft Kinect is not ruled out. The base system could include some kind of open source operating system, which would be integrated on system layer – The Gate – which would load on startup by default.

• The system must function regardless of the available virtual reality devices, provided they meet some minimum requirements.

• The application for entering the system – The Gate – must implement a synchronization system, which maintains locally all data necessary to operate the system without the need for internet connection. This includes downloaded applications, files of a certain size, etc..

• The user, once logged in the centralized system (and mandatory prerequisite) is uniquely identified.

• The application for entering the system – The Gate – may be customized by the users according to their preferences. These data, in addition to locally stored will be synchronized with the central server.

• The centralized system saves all necessary data from the user so that he can start the system, and access his data, regardless of the machine he is using.

 

API and SDK.

For the success of the project is especially important to have a broad base of developers for the platform. This requires providing the community an attractive system, full of possibilities, and at the same time a platform where they can monetize. In this respect the SDK must offer at least:

• Access to the user interface API. This enables the implementation of applications regardless of the available devices.

• Access to the visualization API.

• Access to the e-commerce API.

• Access to the system’s cloud-computing system linked to the user account.

• Access to the communications and telepresence API.

• Access to the basic services API, this includes: images, multimedia, maps, productivity and notifications.

 

Conclusions and next steps.

The project is very ambitious in all aspects, both in necessary resources for development, infrastructure requirements, promotion and patrner finding. However it is project for the future, proposed at the right technological time and in a few years it can make a real revolution in the way we understand computation in the XXI century.

Next: The Project Today