Simulation as mass media
Real time event simulation discussion paper.
A discussion and set of examples showing the broad utility of capturing physical events in real time to simulate them online.
This paper is a result of further thinking about the potential application of movement capture systems. If you too are interested in speculation on this subject please take a look at my previous posts Dance Sampling Vision, Exertainment Vision and Product Fiction.
What is this "real time event simulation" - is it the same as virtual reality?
The focus of virtual reality (VR) is interaction by the user through immersion within a simulated space. At this time a considerable amount of hardware is required to enable this immersion - to become an active participant within a simulation. The cost of this hardware and the inconvenience of setup has meant that at this point VR is in the hands of very few users.
The scope of real time event simulation (RTES) is considerably less than that of VR. In RTES the user is more of a viewer than a participant. The users ability to interact within simulated space is limited to that of an audience member, eliminating the need for immersion into the simulated space and thus the associated (expensive and inconvenient) hardware.
Essentially RTES is a one way broadcast of physical events over IP - not dissimilar to an audio or video streaming however in this case the stream is fed into a simulation space, with the IP stream driving the objects within that space to replicate the event being captured in real time eg a basketball game or even the people, luggage and vehicles within an airport.
It is speculated that the one way nature of RTES will enable it to proceed VR as a mass media channel as the costs associated with the system setup are absorbed by the content provider and likely returned in the form of advertising revenue (in the case of basketball) or operational efficiencies (in the case of an airport) - leaving the content consumer with no upfront investment other than a computer and an Internet connection.
RTES System Outline
On exploring the idea of RTES and visioning what such systems would look like across different applications similar patterns emerge. Every RTES system can be broken down into three main parts:
- Data capture
- IP conversion
- Media player
The requirements to capture the data of a physical event vary considerably between application types. In some cases the capturing of positioning in all three dimensions is required (such as in most ball sports) to provide an adequate simulation to users/audience members. However in other applications position of objects across x and y axes is sufficient - such as in motor racing where cars and motor cycles rarely leave the ground.
Data capture is not limited to the current positional information of objects. Almost every application includes the addition of data from pressure, temperature and other sensors. The inclusion of these additional sensors is driven by the applications requirement to provide users/audience members with an effective simulation of real physical event.
This processor compresses the data coming from multiple positional transponder and other sensor sources into a single low band-width IP stream suitable for Internet broadcast with off the shelf IP streaming tools. The speed and amount of compression achieved by this conversion process is a significant factor in both the latency between the actual physical event and its online simulation and the cost of broadcasting the even and as such it is expected that this process will utilise considerable processing power to add value to the IP stream (further compressing it) and delivering it to streaming server half a second of less after the even horizon.
Value that the conversion process can add to the IP stream differs on the application. In the case of a basketball the IP conversion process could add value by assigning ball possession status to players by applying logic to ball position, ball sensor and player position data.
It is expected with the emergence of RTES streams that players to take advantage of this new form of media will proliferate. Unlike players for audio and video where the quality of the experience is dominated by bandwidth of the stream RTES feeds are unique in that the experience will dictated by the media player code-base. Some media players will be designed to play all kinds of RTES feeds - reducing the visual authenticity but providing the utility of being able to view many different simulated spaces. Others will specialise to provide a highly authentic experience of a specific simulation space.
It is also speculated that some players will provide recording facilities to enable extra features such as Tivo style replay and data analysis both during and post event horizon.
Probably the best way to illustrate RTES and its potential is with examples. To facilitate readers with different interests I have broken these examples up into broad categories: entertainment, industry, governance & defence.
The potential for RTES in the entertainment arena - especially sports is considerable.
Professional sports are always looking for new ways to bring their content to audiences with television current the preferred medium to do this. At this time the Internet has not been a major player in projecting games to audiences. Video and audio streaming, while infinitely better than what they were just a few years ago are still relatively bandwidth intensive making them expensive to operate and consume. As a result the business of streaming sports video and audio has not taken off.
RTRS offers the potential to compress the bandwidth associated with streaming complex physical events considerably.
Example 1 - NBA Basketball
- All players and referees would be fitted with a positional transponder with unique identifiers
- Referees would be given PDA to input referee decisions and game clock start and stop points
- The ball would be fitted with a positional transponder and touch sensor
- Backboards of goals would be fitted with touch sensor
- Rings of goals would be fitted with ball pass through sensor
- Court would be surrounded by sensors to capture positional transponder data
The IP conversion system would collate all player, referee, ball and court sensor data to create the IP feed. Logic would be included to assign ball possession and goal scorer. It is expected the bandwidth for such a feed would be no greater than that required for low resolution audio streaming.
It is speculated that the types of players designed to consume this simulation space would come in three main types:
This type of player would be for the die hard fan, a person who lives for the sport. This type of application would be installed on the users hard drive and most likely paid for. It would be capable of turning the feed into a highly detailed simulation of the game with avatars with photographic replication of player faces, uniforms and game venue details. Features of a high resolution player could include:
- Integration into radio broadcasts for audio commentary or even possibly the automatic generation of commentary using a voice synthesizer and feed data.
- An engine to record player statistics and show them along side the game as it is running
- User camera and playback control - Tivo like functionality with the added ability to specify the view angle
Such a player would probably be a purchased application from a sports game specialist vendor such as EA.
This type of player would be for your run of the mill fan who likes to keep up with what's going on but has other more important things to do. This type of application would integrate into the users portal such as a Gadget in iGoogle. The game would be presented within the portlet window possibly top down view only with minimal game statistics such as current score and game time.
This type of player would be designed for team coaches and other parties interested in using the IP stream to support during game tactics and strategy and post game time analysis and coaching focus. It is likely such applications would be developed in house and even possibly include other data feeds not available to the general public e.g. player heart and body temperature sensor data.
Example 2 - Formula 1 Racing
- All cars (including pace car) would be fitted with positional transponders with unique identifiers
- Race governors will be given a PDA to input referee decisions and race start and stop points
- All car team managers would be given a PDA to input pit-stop calls
- Race track would be surrounded with sensors to capture transponder data
- Car telemetry data such as remain fuel, engine temp, tire and oil pressure could be integrated into system
Formula 1 is full of strategy, being able to view fuel and tire data on individual cars would give viewers the ability to make their own predictions on the team tactics. A Formula 1 track is quite large and the cars move quite quickly making the sensor net to capture the car positions quite a challenge to develop. However it is expected that cars are already fitted with sophisticated transponders and telemetry capabilities.
Both high and low resolution players (as described in the proceeding basketball example) would be applicable to Formula 1. However it is expected that Team manager already have more than enough data to support decision making which would likely make a professional player redundant.
Unlike entertainment where the payoff for content providers is advertising revenue, in all the other cases the RTRS business model would be based on operational efficiencies.
Example 3 - Airport
- All vehicles (trucks, buses, planes, cars, forklifts etc) would be fitted with positional transponders with unique identifiers
- All persons (staff and travellers) would be fitted with position transponders with unique identifiers
- Airport would be surrounded with sensors to capture transponder data
- Radar, check-in, customs and immigration systems could be integrated into the system
This system potentially could have a very large number of items to track over a wide area. As such it is expected that the update/refresh speed would would slower than that of a sports application probably in the order of once a second.
Kiosk player - for fitting throughout the airport to help travellers and family members locate places and items within the airport.
Admin player - for recording and analysis of airport traffic to optimize staffing levels and other processes.
PDA player - for hand held usage by staff, travellers and family members
Example 4 - Open Cut Mine
- All personnel and vehicles would be fitted with positional transponders with unique identifiers
- Load carrying vehicles would be fitted with pressure sensors to monitor ore weights
- Mine would be surrounded with sensor to capture transponder and sensor data
The system would automatically connect drivers to vehicles and loads helping to