Communications of the ACM
Computer-supported cooperative work in design
A Java 3D-Enabled Cyber Workspace
In today's global business, the decentralization of organizations has become a major success factor. Outsourcing, joint ventures, and cross-border collaborations have led to work environments geographically distributed across organizational and national boundaries. During the last decade, the Web has gained broad acceptance in academic and commercial areas, used by many as a medium to share data, information, and knowledge. Today, it is widely used for development of collaborative applications supporting dispersed working groups and organizations because of its platform, network and operating system transparency, and its easy-to-use interfacethe Web browser.
In addition to Web technology, Java has brought about a fundamental change in the way applications are designed and deployed. Java's "write once, run anywhere" model has reduced the complexity and cost traditionally associated with producing software on multiple, distinct hardware platforms. With Java, the browser paradigm has emerged as a compelling way to produce applications for collaboration over the Internet. As decentralization of business grows, a large application potential of this work is anticipated. It can be utilized in such areas as design verification, real-time monitoring and control, as well as navigation in augmented reality.
Since 1993, shortly after the debut of the Web, a number of methods and frameworks have been proposed for Web-based collaborative systems [3, 58]. Most of the frameworks are developed for collaborative design and project management, while a few focus on conflict resolutions during collaboration [1, 4]. In terms of technologies used in the existing systems, HTML and Java applets are widely adopted for developing the client-side user interfaces, in addition to ActiveX and Virtual Reality Modeling Language (VRML) [8], while server-side technologies including JavaServer Pages (JSP), Java servlets, as well as Extensible Markup Language (XML) are quickly obtaining attention for new system developments. To facilitate a viable collaborative environment, application servers must engage users in a 3D graphical interaction in addition to the dialogue-like data sharing, since remote users, including designers, engineers, and managers need active and visual help to coordinate their efforts in a distributed environment. Reducing network traffic and increasing system performance are the other concerns in collaborative system development.
Despite the accomplishments in many areas, a shared collaborative system or workspace for real-time distributed environments (remote control, real-time monitoring, online inspection, and collaborative troubleshooting, to name a few) are still missing from the literature. Our approach toward building a Java 3D-enabled cyber workspace targets this area and tries to engage a dispersed working group in the shared workspace, allowing the team members to share real-time data through a set of interactive J3D models. Different from VRML, J3D gives end users high-level constructs for creating and manipulating 3D geometry using a separate behavior model. Instead of camera images, the physical world with common interests (such as a product, a device, or even a production line) can be represented by a set of J3D models with behavioral control nodes embedded. Once being downloaded from an application server, these J3D models are rendered by the computers on the client-side, but remain alive by connecting with the real world through message passing, such as real-time sensor signals, control commands transmissions, and so forth. This approach is being designed into a framework with open architecture.
Architecture and Framework
As new classes of collaborative applications emerge for the Web environment, there is increasing pressure for full multimedia capabilities (audio, video, and 3D graphics) to become seamlessly integrated into the browser paradigm. Users now demand a collaborative workspace that is easy to follow, operate, and maintain. In addition to software innovations, today's technologies have made accelerated 3D graphics and multimedia-capable hardware affordable for such advanced Web applications. However, most multimedia Web applications require a large amount of data being transmitted through the network before it can be shared. The obvious conservation of the Internet bandwidth makes itself the bottleneck for such multimedia applications. It becomes unpractical, especially when real-time data constraints are encountered.
Targeting the distributed, real-time monitoring and control in manufacturing sectors, a framework with high efficiency for cyber collaboration is carefully examined. The framework is being designed to use popular client-server architecture with secured session control. The proposed solutions for meeting both the user requirements for rich data sharing and the real-time constraints are:
- Using interactive J3D models instead of bandwidth-consuming camera images for visualization;
- Choosing lightweight swing components for client-side graphical user interface implementation; and
- Encapsulating major control logic on the server side by JSP and Java servlets.
As shown in Figure 1, the architecture of our framework is based on the View-Control-Model (VCM) design pattern by dividing itself into three tiers: Presentation, Application, and Data. J3D is used to model the physical world that is of common interest. A J3D model may reside in both the Data and Presentation tiers. Once downloaded to a standard Web browser, the J3D model is rendered by using local CPU on the client side and thus can provide a look-and-feel of its physical counterpart at fast speed. The global behavior of the J3D model is driven by real-time sensor signals. The sensor signals are generally much smaller in data size when compared with camera images. They are collected by and broadcast from the application server to the client machines via the cyber workspace. On demand, an end user can send requests (or commands) for other services (or instructions) to the application server, or write messages to other team members participating in the same collaborative cyber workspace for discussion and troubleshooting.
The mid-tier application server handles major security concerns, such as session control and user access control, among others. A central session manager is designed to look after the issues of user authentication, active session control and synchronization among users, sensitive data logging, and user access right authorization. All transactions need to pass through the session manager for consistent status monitoring. For the same security reasons, a physical device is accessible only by the SignalCollector and Commander residing on the application server. The SignalCollector is responsible for sensor data collection from any physical devices connected and registered in the server. It is multithreaded and utilizes the concept of pushlet developed by Just van den Broecke [11]. Based on the Publish-Subscribe design pattern, the SignalCollector maintains a list of subscribers (the clients interested in receiving sensor data from the same device) for efficient data distribution. Another server-side component, called a DataAccessor, is designed to separate logical and physical view of data. It encapsulates complex Java DataBase Connectivity (JDBC) and SQL codes and provides standard methods for accessing data (J3D models, historical data of the devices, or XML documents). The historical data is found helpful for trouble-shooting, while XML is used for data communication in future extensions.
Although the global behaviors of J3D models are controlled by the server based on real-time sensor signals, users still have the flexibility of monitoring the models from different perspectives, such as selecting different J3D models, changing viewpoint, and zooming, through a J3DViewer at the client side. Authorized users can submit control commands through CyberController to the application server. The Commander at the server side then takes over the control for physical device manipulations. StatusMonitor, another client-side module, provides end users with a view of runtime status of the controlled device. The table summarizes the major functionality of each component within the proposed framework.
J3D and its Scene Graph
J3D is considered to be a mid- to high-level, fourth-generation 3D API [2]. What sets a fourth-generation API apart from its predecessors is the use of scene-graph architecture for organizing graphical objects in the virtual 3D world. Unlike the display lists used by the third-generation APIs (such as VRML, OpenInventor, and OpenGL), scene graphs hide much of the rendering details from the programmer while offering opportunities for more flexible and efficient rendering.
Enabled by the scene-graph architecture, J3D provides an abstract, interactive imaging model for behavior and control of 3D objects. Because J3D is part of the Java pantheon, it assures developers ready access to a wide array of applications and network support functionality [9]. Figure 2 illustrates a scene graph for a Direct Drive testbed. This testbed consists of a linear slide and a brushless rotary motor with encoder feedback.
J3D differs from other scene-graph-based systems in that scene graphs may not contain cycles. Thus, a J3D scene graph is a directed acyclic graph. The individual connections between J3D nodes are always a direct relationship: parent to child, (shown in Figure 2). The scene graph contains a complete description of the entire scene, or virtual universe. This includes the geometric data, the attribute information, and the viewing information needed to render the scene from a particular point of view.
As shown in Figure 2, the Virtual Universe object provides a grounding for the entire scene graph. All J3D scene graphs must connect to a Virtual Universe object to be displayed. A scene graph itself starts with the BranchGroup (BG) nodes (although only one BG node in this case). A BG node serves as the root of a subgraph, or branch graph, of the scene graph. The TransformGroup (TG) nodes inside of a branch graph specify the position, the orientation, and the scale of the geometric objects in the virtual universe. Each geometric object consists of a Geometry object, an Appearance object, or both. The Geometry object describes the geometric shape of a 3D object. The Appearance object describes the appearance of the geometry (color, texture, and material reflection characteristics). The behavior of a J3D model is controlled by Behavior nodes, which contain user-defined Java codes and state variables. Sensor signal processing can be embedded into the Java codes of a J3D model, for remote monitoring of its physical counterpart in the real world. Once applied to a TG node, the so-defined behavior control affects all the descending nodes. In this example, the two movable objectsLinear Slide and Rotary Motor are both controlled by two behavior nodes, one for online control/monitoring and the other for offline simulation/animation.
Cyber Workspace Implementation
In the near future, distributed design, manufacturing, and facility management may easily be conducted through an effective collaborative workspace. As shown in Figure 3, such a collaborative workspace prototype has been implemented on top of the proposed framework, through a Java applet-based graphical user interface. A collaborative team member can share real-time data with others while performing his or her own duty through this portal without leaving the browser.
The shared cyber workspace eliminates network traffic by replacing camera images with J3D models. By choosing a viewpoint carefully, an end user can even use the "walk-through" capability of 3D models to perform internal inspections or trouble shootings of physical devices otherwise impossible by using optical cameras. The largely reduced network traffic also makes real-time monitoring, control, inspection, and troubleshooting practical for users on relatively slow hookups, such as modem and low-end wireless connections. This is especially useful for field engineers or shop floor managers who want to access rich information with cellular phone lines.
The functionality of the shared cyber workspace is largely determined by the control and decision-making logics embedded in the application server by means of servlets. Responding to end users' requests, these servlets should be able to search and provide solutions effectively. Complete server-side functionality of the shared workspace is currently being developed at our lab.
Conclusion
J3D is used as one of the enabling technologies in developing the framework. A shared collaborative workspace is implemented on top of the framework for remote monitoring, control, and troubleshooting. It is proven, through the example shown in Figure 3, that using J3D models instead of camera images can reduce network traffic significantly. Participating in the workspace, users can not only reduce network traffic by real-time interactions with quick response, but also obtain more flexible control while still enjoying intuitive visualization of the real world. As an extension to the framework, XML may slash the efforts needed to define the control logics and to communicate between various physical devices, software modules, and Web browsers. In the future, open-architecture devices (like OpenPLCs and OpenCNC) will have Web servers and Java virtual machines embedded. This will make the cyber workspace function more efficiently for sensor data collection, real-time data transmission, and synchronous communication and collaboration.
As decentralization of business grows, a large application potential of this work is anticipated. It can be utilized in such areas as design verification, real-time monitoring and control, and navigation in augmented reality. For example, if combining J3D models properly with camera images, the so-called augmented reality can be applied to Web-based, multiparty collaborations, such as in construction, where a virtual structure is merged into a real scene of landscape for design verification. It can also be extended to remote inspection even if the physical counterpart of the model is invisible or nonaccessible (for example, inside of a nuclear power plant or outside of a space station). Once it is recognized as a promising technology, J3D will be supported by major CAD system venders. It is quite likely independent software suppliers will build authoring systems that generate J3D applications as output [10]. This will make J3D model creation easier, faster, and more practical.
The benefits brought by the J3D-enabled cyber workspace include reduced network traffic, quick response, interactive control, increased flexibility for real-time monitoring, and remote inspection, as well as Web-based synchronous collaboration. The cyber workspace may allow a collaborative team to work as closely and productively as if they are under the same roof.
Despite the accomplishments in many areas, a shared collaborative system or workspace for real-time distributed environments are still missing from the literature.
References
1. Adelson, B. Developing strategic slliances: A framework for collaborative negotiation in design. Research in Engineering Design 11, 3 (Nov. 1999), 133144.
2. Barrilleaux, J. 3D User Interfaces with Java 3D. Manning Publications Co., Greenwich, CT, 2001.
3. Caldwell, N.H.M. and Rodgers, P.A. WebCADET: Facilitating distributed design support. In IEE Colloquium on Web-based Knowledge Servers, London, U.K., 1998.
4. Cooper, S. and Taleb-Bendiab, A. CONCENSUS: Multi-party negotiation support for conflict resolution in concurrent engineering design. J. Int. Manuf. 9, 2 (Apr. 1998), 155159.
5. Huang, G.Q. and Mak, K.L. Web-based collaborative conceptual design. J. Eng. Design 10, 2 (June 1999), 183194.
6. Numata, J. Knowledge amplification: An information system for engineering management. Sony's Innovation in Management Series 7, Sony Corporation, Japan, 1996.
7. Rodgers, J.L. and Salas, A.O. Toward a more flexible Web-based framework for multidisciplinary design. Advances in Engineering Software 30, 7, (July 1999), 439444.
8. Roy, U. and Kodkani, S.S. Product Modeling within the framework of the World Wide Web. IIE Transactions 31, 7, (July 1999), 667677.
9. Sowizral, H., Rushforth, K., and Deering, M. The Java 3D API Specification. 2d Ed., Addison-Wesley, Boston, MA, 2001.
10. Sun Microsystems, The Java 3D API. Technical White Paper, July 1997.
11. van den Broecke, J. Pushlets, Part 1: Send events from servlets to DHTML client browsers. JavaWorld (March 2000); www.javaworld.com/jw-03-2000/jw-03-pushlet.html).
Figures
Figure 1. Framework of a collaborative cyber workspace.
Figure 2. Scene-graph architecture of J3D for a direct drive testbed.
©2002 ACM 0002-0782/02/1100 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2002 ACM, Inc.
No entries found