Communications of the ACM

The study found that group-support-system-supported teams performed best, finding the most defects.

A second GSS feature is tangible group memory, as all computer-based input in the meeting process is preserved and made available to participants during the meeting (and later). It may mitigate the process loss associated with failure to process contributions if group members lose their focus and miss the contributions of their fellow group members. Group memory may also help counter information overload, as the availability of meeting information improves participants' ability to filter and review information generated earlier. Moreover, group memory may reduce sidetracking by helping the entire group stay focused on the task at hand. Group memory may also help increase the process gain associated with synergy.

Group process support. To help us learn more about how GSS group memory and electronic communication affect a software inspection meeting process, our controlled laboratory experiment and study, which involved college students majoring in IS, compared the performance of software inspection teams under three different conditions: unsupported; group memory; and GSS (see the table and the sidebar "How the Study was Done"). In all, 30 groups of students participated, including 10 groups under each condition. The unsupported condition involved a face-to-face inspection that lacked group memory or electronic communication channel support. The group memory condition was similar to the unsupported condition, except it provided group memory support in the form of large sheets of white paper for recording and displaying the group's findings. Lastly, the GSS condition provided group memory and an electronic communication channel. Group memory was provided through an electronic interface of shared windows of text (the team's findings) appearing on each participant's screen; the windows on each machine were refreshed automatically with each new submission. As part of the GSS system, a parallel electronic communication channel allowed each member to communicate simultaneously, sharing only textual information. Prior to conducting the study, we hypothesized that GSS teams would perform best since we expected the GSS support features to boost inspection team performance, except for information overload, which we expected to increase due to the electronic communication channel [7].

Prospects are promising for reengineering the traditional inspection process by using a group support system to implement larger inspection teams and conducting asynchronous inspection meetings.

To gain more insight into these findings, we surveyed the participants regarding potential sources of process loss and gain (see Figure 1). We wanted a better sense of the specific ways the group support features in the group memory and GSS conditions may have helped—or hindered—the inspection meeting process. Statistically significant effects across meeting conditions were found for all but one of the sources of process loss and gain studied. We conducted Scheffe's tests (confidence level of 0.05) to determine which pairs of group means were statistically different. The significant findings (those for production blocking were not) are outlined in the following paragraphs:

 Domination

As we hypothesized, the unsupported groups reported significantly more domination by one or more individual members than the GSS groups. This finding was consistent with our observations of the groups and review of audio recordings of the sessions; dominant members tended to emerge in the unsupported groups, resulting in less equitable participation. In the GSS groups, the electronic communication channel reduced the potential for domination, and participation among GSS team members was more even. The increased participation by all members appeared to contribute to the superior performance by the GSS groups. While perceived domination was a bit more notable for the group memory groups compared to the GSS groups, we found no significant statistical difference between these groups.

 Sidetracking

We found sidetracking to be significantly lower for the GSS and group memory groups than for the unsupported groups. The difference between them was not significant, suggesting that group memory—whether supplied by manual or electronic means—probably plays an important role in limiting loss due to sidetracking. Observations of the groups and a review of the audio recordings of the sessions suggest the group memory approaches used in the study helped reduce sidetracking by keeping group members more focused, as they were more likely to note new defects instead of being sidetracked by a single given defect.

 Information overload

While the GSS support features may have helped reduce the sources of process loss, the survey data also indicated that GSS participants had a significantly stronger sense of information overload than those in the other two meeting conditions. We hypothesized this finding, expecting the parallel communication feature of the GSS would allow all participants to contribute simultaneously, while contributions under the other meeting conditions would be submitted in a more digestible sequential fashion. Another factor that may have contributed to a perception by survey participants of information overload was the number of contributions submitted during an inspection meeting. In addition to detecting and reporting more distinct defects than teams in other treatment groups, the GSS teams also generated more repeat defects, or the same defect submitted by more than one group member. While they were not counted in the performance results, they did add to the amount of information GSS participants had to process. While we anticipated that the group memory feature of the GSS approach might mitigate the information overload effect, the net effect of the GSS approach was clearly greater perceived overload.

 Failure to process contributions

We hypothesized that the group memory features of the group memory and GSS approaches would result in lower perceptions of this source of process loss for the teams using group memory. However, this perception was not the case for the GSS teams, as their perceptions regarding the failure to process contributions were significantly higher compared to both the unsupported teams and the group memory teams.

The GSS interface might have played a role in these findings. Although the GSS participants reported satisfaction with the interface, it was not always possible for them to see all the defects generated by their teams simultaneously due to the size and resolution of the computer screen; for example, as a screen filled with text, it would be necessary to scroll within the window to see each contribution. The combination of this factor and the parallel communication aspects of the GSS may have caused participants to feel it was more difficult to process the contributions of their fellow group members.

 Synergy

Although the GSS participants reported favorable synergistic effects, the comparative findings for synergy showed the unsupported participants perceived significantly more synergy than the GSS participants. Unstructured interviews with participants after the study indicated that this finding might have resulted from their being more comfortable with the face-to-face environment of the unsupported condition and its resulting sense of synergy. In light of the overall performance results, this finding was somewhat surprising; if synergy was indeed greater for the unsupported teams, the stronger performance of the GSS groups could be attributed more to reduced process loss than to increased process gain, or synergy.

In addition to process loss and gain issues, it is also important to consider how study participants felt about using nontraditional meeting approaches, as the implementation success of any innovation depends in part on the perceptions of those using it (see Figure 2). For all conditions, participants reported a strong feeling of satisfaction with the meeting process, and their acceptance of the findings was high. While there was no significant difference among groups with respect to satisfaction with the process, the unsupported groups reported a significantly higher mean for acceptance of findings than the GSS groups. Members of the teams that interacted through the spoken word, including those in the unsupported and group memory categories, would occasionally ask questions to better understand the justification for noting a particular defect. Since GSS team members generally did not use their electronic communication channel to clarify the contributions of others, failure to clarify may explain why the GSS teams reported relatively less acceptance of the findings.

Although the support features provided by the GSS approach in the study had an overall positive effect on performance, we also recognize the need for improvement. For example, while the findings relating to the perception of process loss and gain show that the GSS approach reduced perceptions of certain sources of process loss (such as domination and sidetracking), perceptions increased for other types of process loss (such as failure to process contributions and information overload); perceptions were lower for synergy. Additional refinements to the GSS approach may enhance the balance of group process loss and gain in inspection meetings; for example, an improved GSS interface might make it easier for users to process contributions and promote synergy.