Communications of the ACM
Home/Magazine Archive/September 1999 (Vol. 42, No. 9)/Image Generation Systems in Virtual Training Platforms/Full Text
Communications of the ACM
Image Generation Systems in Virtual Training Platforms
Computer-image generation in VR environments is an important component of Distributed Mission Training systems. In fact, this is one of the limiting factors of performance that is technologically challenging in most synthetic virtual training environments where visual cues play a critical role. In spite of this, computer-based training has significantly evolved by integrating computer-generated imagery at all levels of the education process. In virtual training environments, image generation systems receive positional inputs from a human or model operator, compute and display a perspective image of a 3D database on image screens, and the operators use these visual cues and references for further action. Three principal issues arise in this process: image complexity in terms of the size of data sets involved, image clarity (both desired and currently possible), and the speed of computation and display.
In fast-moving training environments such as flight simulators, the fields of view are usually large, change at variable rates depending on altitude and position, and require images of different levels of clarity. Comparatively, in slower environments such as automobiles and other ground-based motion devices, the fields of view are smaller, change at slower rates, and require images of high clarity. The fields of view and required clarity define the level of detail in the terrain involved, and the consequential database size; the rates of change determine the computation and display speeds required. There are clear tradeoffs along these dimensions in the choice of an appropriate image-processing system besides the cost. In this article, we present the configuration of a novel image generation (IG) technology for fast-moving training environments developed and implemented in a DMT prototype. We also discuss some of the challenges in the next stages of evolution of this technology, and highlight critical R&D frontiers in the wide range of virtual training applications of DMT where visual systems are crucial.
An IG system is usually driven on the backend by an IG database server with frontend geometric and display processors. The backend processor is a critical determinant of overall system performance since it is responsible for maintaining very large image databases and fast information retrieval especially in high-speed training platforms. The geometric processor converts 3D database elements retrieved by the backend into 2D perspective imagery. (A conceptual architecture of an IG system is shown in Figure 1.)
The computational speed of this processor translates to the latency in image updates in the visual system of the training platform. In fast-action platforms, this speed is critical, and the technology of geometric processors has considerably improved over the last decade in this regard. The realism in virtual training platforms is directly related to this latency: the smaller the latency, the greater the realism. Hence, the quality of training is directly affected by the speed and efficiency of the database server and the geometric processor. The function of the display processor is to position, align, and integrate the final image on the display screens by mapping the output of the geometric processor into the appropriate pixels. The data output at this stage is digital, and is converted into analog signals that drive the display devices. The resolution achievable by the display processor and the end devices determine the image clarity. They should be chosen according to the requirements of a training scenario.
IG applications over the last decade have become increasingly sophisticated with multichannel image-processing capabilities. In such systems, multiple channels of image development and displays are created by adding more servers for database, geometric, and display processing. While multichannel presentations have almost become the norm in the simulator training of Air Force pilots, they are currently being used in training commercial airline pilots as well. The introduction of the additional hardware at the various stages of image processing enables multiple presentations of scenarios (for instance, at the trainee, instructor, and observer consoles), partitioning the display field into screen segments where each segment is uniquely driven by a chain of IG processors in order to lower imaging latencies in fast training platforms, and the possibility of creating 360° field of vision through multiple projections.
A Current IG Technology For Fast-Action Platforms
The field-of-view, brightness, and contrast of practically every visual system available today is far less than what the trainees see in the real equipment of fast-moving platforms. Most importantly, the resolution of current display systems is at least an order-of-magnitude less than that required. Because of this, current visual systems do not provide a trainee with adequate visual definition to identify other components in the field of vision in virtual platforms. For example, components such as other vehicles, roads, and bridges at realistic ranges need better visual definitions for effective training in virtual systems. Without the proper definition and realism in the visual display, many training tasks cannot be adequately performed.
To solve this problem for DMT, a three-pronged approach to developing advanced visual systems is needed. The three key areas are low-cost, deployable visual displays that encompass the training platforms and provide full-field-of-view imagery; super high-resolution display components; and rapidly generated, high-fidelity visual and sensor database systems. Current visual systems are limited by the previous generation design and business decisions. Although there is a significant effort currently under way in the IG industry to optimize all the parameters available to current display technologies, their fundamental limitations impede their implementation into the high-end systems of the future. But new technologies are becoming available that could lead to major advancements. Taking advantage of these new technologies and applying them in innovative ways will be the key to producing high-performance visual displays of the future.
For DMT environments to meet any training or testing need, the IG systems must provide the visual resolution for recognizing objects at realistic ranges and image refresh rates that can support the speed of fast-moving vehicular training platforms.
In this regard, the Air Force Research Laboratory is spearheading the development of an IG system for fast-action training platforms. This system has been developed in conjunction with the IG industry, and implemented in the prototype virtual training platforms of the DMT system at the laboratory. The IG system is integrated with a state-of-the-art visual display system known as Mobile Modular Display for Advanced Research and Training (M2DART). A high-level architecture of this IG technology is shown in Figure 2.
The new IG system consists of a Target Generation Unit (TGU) and Overlay Processor Unit (OPU) in addition to the usual components of an IG system. The TGU and OPU provide necessary inset video and background video format conversion as well as timing synchronization. The OPU converts video from an external background IG system and TGU inset video into a format that will be displayed on the special purpose, full color, super high-resolution, microlaser-based projector system. A goal of this system is to resolve 20/20 visual acuity area-of-interest inset models at 5120 x 4096 video format. The imagery provided by this IG is viewed from the M2DART display system.
M2DART is a rear-screen, real-image, display system with eight flat projection screens integrated together to display eight channels of full-color, full-field-of-view imagery. Due to the relatively small surface area on these screens in comparison to large dome displays, the imagery is significantly brighter with much improved contrast. Our current experience with M2DART indicates it can be a cost-effective solution to many currently unfulfilled virtual platform training requirements.
Various display system manufacturers are producing their own versions of the M2DART. The modularity and flexibility of the design of the M2DART allows it to remain an excellent testbed for many important technology insertion projects such as super high-resolution projection systems, refined screen materials, and advanced image generators. Continued enhancements and upgrades to the M2DART will most likely include modifications to the display geometry such as the use of curved surfaces instead of the current flat-panel facets, and the development of low-cost, high-quality collimating rear projection screens. The M2DART appears to be a solid design approach from which future DMT visual displays should be based.
Challenges and Emerging Solutions
For DMT environments to meet any training or testing need, the IG systems must provide the visual resolution for recognizing objects at realistic ranges and image refresh rates that can support the speed of fast-moving vehicular training platforms. An analysis of M2DART system reveals that achieving this level of acuity requires a projector system capable of projecting 5120 x 4096 pixels. At a minimum, to produce high-quality dynamic images, the image generation update rate and the display refresh rate must both be 60 Hz, noninterlaced. This represents an order-of-magnitude increase over what is considered high-resolution projection technology. Clearly, a CRT-based projector system could not provide this level of resolution.
An alternate emerging technology for providing such resolution is laser-based projection. These displays are known as direct-write systems in that small, solid-state, eye-safe lasers are modulated and then scanned directly onto a screen. No display phosphors or LCD elements are involved. Early prototypes have demonstrated many benefits that could substantially improve DMT visual displays. Most importantly, laser-based displays can achieve the spot size required for very high resolutions. Other benefits include improved display contrast, excellent display brightness (that could approach daylight), a significantly improved color gamut, and a large depth of focus. Newly developed red, green, and blue solid-state laser sources, each operating at a primary color, provide the most efficient, high-brightness projection light source for display applications to date. The combination of solid-state lasers, parallel-scan architectures, and new high-speed spatial light modulators that modulate the video into the laser beams, make a 5120 x 4096, 60 Hz, noninterlaced projector possible. This projection technology also shows promise for developing lightweight, high-resolution helmet-mounted displays in mobile, virtual task-training platforms.
Other new technologies, such as commercial PC-based graphics, could be used to improve IG capability and at the same time drive down cost. PC-based 3D graphics have made significant strides and now accelerate many aspects of graphical visualization. Competition is driving advancements at a rapid rate. In some cases, the graphics capabilities of PC-driven systems are nearly doubling every six months. While these are some of the current solutions, the real challenge is to capitalize on these new technologies and develop super high-resolution visual system components that include super high-resolution projectors (5120 x 4096, 60 Hz, noninterlaced video), lightweight high-resolution displays, and the associated image generators with parallel, high-speed digital interfaces.
Conclusion
DMT is a demanding virtual training environment that requiring advanced levels of IG and display technology are still developing. Major advances in the capabilities of visual systems such as resolution, contrast, and brightness need to be achieved. Development of IG databases in any team-training scenario is time-consuming. However, the silver lining is that new technologies are rapidly emerging to produce giant leaps in these directions. While microlaser systems are revolutionizing the high-end IG systems, commercial PC-based graphics will soon accelerate DMT visual systems with greatly reduced costs.
References
1. Corbin D., Amm, D.T., and Corigan, R.W. Grating light valve and vehicle displays. In Proceedings of the 5th Annual Flat Panel Display Strategic and Technical Symposium (Detroit, Sept. 1998).
2. Corrigan, R.W. Scanned linear architecture improves laser projectors. Laser Focus World (Jan. 1999).
3. Fink, C., Bergstedt, B., Flint, G., Hargis, D., and Peppler, P. Microlaser-based projection display for simulation. In Proceedings of the 18th Interservice/Industry Training Systems and Education Conference (Orlando, Dec. 1996).
4. Hargis, D. and Earman, A. Lasers replace conventional technology in display designs. Laser Focus World (July 1998).
5. Peppler, P. and Gainer, J.C. A full-color, high-resolution, laser projector for a flight simulator. In proceedings of ITEC Conference (Hague, The Netherlands, Apr. 1994).
6. Pierce, B.J. and Geri, G.A. The implications of image collimation for training size and motion-related rasks in a flight simulator. In proceedings of Human Factors and Ergonomics Society, Chicago, Oct. 1998).
7. Spaulding, B. and Peppler, P. Low-cost, PC-based, DIS & HLA simulator visualization system. In proceedings of Advanced Simulation Technologies Conference (San Diego, Apr., 1999).
8. Wight, D., Best, L. and Peppler, P. M2DART Visual Display, A real image simulator display system. In proceedings of Aerosense Conference (Orlando, Fla., Apr. 1999).
©1999 ACM 0002-0782/99/0800 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 1999 ACM, Inc.
No entries found