Communications of the ACM
Communications of the ACM
The Centrality and Prestige of CACM
SNA techniques "discover patterns of interaction between social actors in social networks" [6]. Common SNA procedures include information flow analysis (to determine the direction and strength of information flows through the network), calculation of centrality measures (to determine individual roles within a network), hierarchical clustering (to uncover cliques whose members are fully or almost fully connected), block modeling (to discover key links between different subgroups in a network), and calculation of structural equivalence measures (to identify network members with similar characteristics) [2, 6].
In the context of a citation network, SNA allows us to examine relationships between journals, identify roles played by individual journals, and identify cliques or subgroups of journals representing particular streams of research. Our study builds on prior SNA studies [see 1, 5] by presenting evidence of CACM's prominence within a relatively large network incorporating 120 journals that have appeared in previous IS journal ranking studies. CACM was found to be the top-ranked publication in the network based not only on the normalized number of citations received and information source criteria, but also on measures of local and global centrality.
Methodology
Journal selection. Ideally, one would include all journals used by IS researchers when specifying the IS journal network. However, many IS journals are not currently indexed by ISI's Science Citation Index or Social Science Citation Index, and the resources required to collect manual data on these other journals is prohibitive. Thus for this study, we began with a subset of 125 journals listed on the ISWorld Journal Rankings Web page (which presents a composite ranking based on eight broad-based subjective and objective studies conducted between 19952005; see www.isworld.org/csaunders/rankings.htm). One significant departure we made from previous studies was to include IEEE and ACM Transactions and ACM SIG publications as individual entities within the network. This allowed us to track the actual citation patterns, contributions, and relationships associated with each individual publication. All ACM Transactions and SIG publications that were recorded in the 2003 Journal Citation Reports (JCR) were included in the study. IEEE Transactions appearing in JCR were filtered for their level of IS-specificity.
Only 91 of our initial 125 journals were indexed as "Cited Journals" in 2003. Focusing on the "Citing Journal" reports allowed us to capture data on an additional 36 journals (including individual IEEE/ACM Transactions and SIG publications) cited by the base 91, for a total of 127 journals. Citations made by the 36 non-indexed journals were tabulated manually, using either electronic or print copies of their 2003 articles. Journals that could not be located, or that had ceased publication prior to 2003, were removed from the list, leaving a final group of 120 journals.1
Many of the journals on the final list are not considered "IS-specific." However, a journal that is not a strong publication outlet for IS-related articles can still have a strong influence on the field. Thus the approach was to cast a wide net and allow SNA to identify which journals actually have a strong influence on various subcommunities or cliques of IS research.
Data standardization. After eliminating self-citations (including 157 self-citations for CACM, or 36% of its total citations within the network in 2003), we normalized the data to account for citation differences due to journals having different reference list criteria, longer articles, more frequent publishing cycles, or more articles per issue. Thus each cell in the normalized data matrix represents the proportion of a given journal's total network citations that were made to other journals in the network, with each "receiving" journal's final normalized score being the sum of these proportions [1, 3]. It is possible that some bias could be introduced by this method, particularly when citing journals make very few citations and a disproportionate percentage of these go to a single journal. Consistent with prior ranking studies, CACM overwhelmingly received the highest normalized score for any cited journal in the network (see Table 1).
Data Analysis
Information flow. Network citation data was analyzed using UCINET 6 and Netdraw 2.34. The journals were classified into the broad categories of IS, computer science, management/professional, and operations research based on classifications used in prior IS journal ranking studies.2 Node size in the network diagrams is a function of each journal's normalized ranking, while arrow direction represents information flowing from cited journals to citing journals. The full network (see Figure 1) is split fairly evenly, with journals allied more closely to computer science (blue) on one side, and journals allied more closely with business disciplines, including IS (red), on the other. CACM occupies a central position in the full network, bridging the gap between the more technical computer science journals and the more business-oriented IS journals. MIS Quarterly, Information Systems Research, and Journal of MIS, which are usually considered the most prestigious "pure IS" journals, all occupy the dense central region of the red cluster.
We use spring embedding, a method that positions network actors graphically based on their pairwise geodesic distances [1], to analyze the direction and strength of network information flows. At a threshold of 0.10 (meaning the proportion of Journal A's total citations that were made to Journal B is >= 0.10), clusters of related journals begin to emerge, as well as journals that serve as connections between clusters (see Figure 2). A "pure IS" cluster emerges (upper right-hand corner), which cites MIS Quarterly heavily but is largely isolated from other journals in the network. On the other hand, CACM continues to be a source of information to journals from both the computer science and IS camps, but is no longer an important source of information to the professional/managerial and operations research journal clusters.
Increasing the threshold from 0.10 to 0.15 does not greatly impact the view of CACM's relationships to other journals, although it does lead to a clearer overall delineation of network relationships (see Figure 3). By combining the information from these three flow diagrams, we infer that the journals giving the highest proportion of their overall citations to CACM in 2003 had the following research foci:
- Software engineering, data management, and computer systems;
- E-commerce;
- Collaboration and group support systems;
- Human-computer interaction;
- IT's role in society; and
- Emerging areas for IS research and development.
Information flow analysis can also be used to identify which network journals are the most important sources of information for CACM itself. Only two journals (MIS Quarterly and Harvard Business Review) received more than 10% of CACM's total citations within the network in 2003. Other journals receiving at least 4% of CACM's network citations include (in descending order) IEEE Computer, Sloan Management Review, Management Science, Information Systems Review, and IEEE Software, all of which are highly respected publications within their respective disciplines.
Information flow analysis also identifies information sources (journals that receive citations from more journals than they cite) and information sinks (journals that make citations to many different journals, but are not cited by as many in return). Overall, 52 journals were classified as information sources and 67 as information sinks. CACM received the second-highest information-source ranking in the network (net degree of 52, compared to a net degree of 55 for Harvard Business Review). Overall, CACM cited 49 journals in 2003 and was cited by 101. These scores are obviously highly sensitive to restrictions on reference lists or infrequent publication cycles. Some scores might also be affected by JCR's practice of truncating most journal-to-journal citation counts less than two.
Prominence measures. An actor's prominence in a social network can be based on either centrality (visibility due to "extensive involvement in relations") or prestige (visibility based on "the extensive relations directed at them") [4]. In a journal citation network, it is more appropriate to assess prominence based on measures of prestige, rather than centrality.
Network actors can exhibit prominence due to having more direct ties with other actors (degree centrality/prestige), shorter path lengths to other actors (closeness centrality/prestige), or structurally advantageous positions between other actors (betweenness centrality/prestige) [2]. Degree is a localized measure providing information on an actor's immediate neighborhood, whereas closeness and betweenness are global measures.
Prominence measures are best interpreted in light of the overall level of network centralization. Centralization measures report a network's degree of centralization as a percentage of that of a perfectly centralized star network [2]. Thus the higher the network centralization, the more closely the network resembles a perfect star network with one central actor, indicating unequal distribution of power (or prestige) within the network. While the centralization of the IS journal network varies depending on the measure used, it tends to be moderate at best (ranging from 21.50% for betweenness measures to 66.45% for degree measures).
Freeman degree prestige is a common method of determining journal rankings, including our normalized rankings in Table 1. Based on symmetric (reciprocated) citation patterns, CACM receives the highest-degree centrality ranking, counting 46 (38.66%) of 119 journals as being in its immediate neighborhood. Using asymmetric (in-directed) citation patterns, which are a measure of prestige rather than simple centrality, CACM again receives the highest ranking, with citations from 101 (84.87%) of the 119 journals, 30 greater than its nearest competitor.
A more insightful measure of degree prestige in a citation network is the Bonacich power index, which discriminates between citations received from more popular journals versus less popular journals (based on their respective degrees). The top 25 journals based on the Bonacich power index are shown in Table 2. There are several interesting differences between the Bonacich and Freeman degree results. Using the Freeman method, a large percentage of the top 25 journals have a management/professional orientation (including five of the top 10 journals). However, when using the Bonacich power method, many of these journals drop off the list, being replaced by more traditional IS journals. This implies that while management/professional journals are prestigious within their immediate neighborhood, their prestige does not carry over to the network as a whole. The Bonacich measure, however, is quite sensitive to the selected attenuation factor (0.6, in this case).
CACM's top ranking, using the Bonacich power index, indicates the journals ranked close to it likewise have a high degree of prestige. It is also instructive to examine the types of journals not citing CACM in 2003; for the most part, these are academic and practitioner journals from the management discipline.
While it is possible to calculate closeness and betweenness centrality measures in citation networks, information centrality is considered a more appropriate measure to use in such cases, since information exchange between journals does not always follow the shortest path. Information centrality takes into account the actual strength of ties between actors, and thus indicates the relative drop in network efficiency brought about by removing a particular actor. Results for the top 25 journals appear in Table 2, and are highly correlated with the Freeman degree rankings. This makes sense, since in a citation network where the top journals tend to be widely cited, the removal of these journals will obviously cause a serious disruption to information flow. CACM is the most prominent journal in the network based on information centrality, once again highlighting its importance in dispersing knowledge throughout the network.
Limitations
This study only examined citations made by network journals in a single year (2003). However, Spearman rank correlation tests indicate statistically significant correlations between the normalized rankings in our single-year study and several past multiyear ranking studies, increasing our confidence in the results.3 It is possible that the age of cited journals could impact the results, since older journals have more published articles available to be cited. In addition, the journal classifications (IS, CS, Mgmt/Prof, and OR) used here, while based on prior studies, have unclear boundaries. However, one of SNA's advantages is that it can in fact uncover subtle, unrecognized relationships between journals, and thus can aid in the development of more accurate classification schemes in the future.
Future Directions
The Internet has amplified the power of many existing networks and sustains a large collection of new networks (for example, the open source movement). Increasingly, there is recognition that network analysis can tell us a great deal about the relationships between people and between entities. Search engines such as Google exploit the linkages within a network to determine the implicit ranking of pages. The same analytic method, as used in this article, enables us to reveal the importance of journals in the network of computing-related scholarship. Thus, this research provides guidance to scholars in assessing the importance of a particular outlet in a specific academic publication network.
In the age of the Internet, networks are dynamic and new technologies threaten the equilibrium of existing relationships. For example, journal reputations tend to be long-standing, but what is the effect of a potential reshaping of search strategies? In the days of paper-based journals, searching was often confined to those journals with the highest reputation. Now, facilities such as the ACM Portal and Google Scholar support searching across a wide range of journals. Electronic searching, compared to manual searching, means scholars can focus on finding highly relevant articles in a broad domain rather than restricting themselves to a small set of journals. As a result, we might see changes in journal relationships as a consequence of these new tools, and more broadly, we are likely to see changes in the nature of networks as a result of the adoption of new technologies for linking people and accessing information.
References
1. Biehl, M., Kim, H., and Wade, M. Relationships among the academic business disciplines: A multi-method citation analysis. Omega 34, 4 (Aug. 2006), 359371.
2. Hanneman, R.A. Introduction to Social Network Methods. University of California, Riverside, CA, 2001.
3. Holsapple, C.W. and Luo, W. A citation analysis of influences on collaborative computing research. Computer Supported Cooperative Work 12, 3 (Sept. 2003), 351366.
4. Knoke, D. and Burt, R.S. Prominence. In Burt, R.S. and Minor, M.J., Eds., Applied Network Analysis: A Methodological Introduction. SAGE Publications, Beverly Hills, CA, 1983.
5. Nerur, S., Sikora, R., Mangalaraj, G., and Balijepally, V. Assessing the relative influence of journals in a citation network. Commun. ACM 48, 11 (Nov. 2005), 7174.
6. Xu, J. and Chen, H. Criminal network analysis and visualization. Commun. ACM 48, 6 (June 2005), 101107.
Footnotes
1A complete list of these journals and their associated abbreviations is available from the authors on request.
2Details on how we determined our classifications and the studies upon which these classifications are based are available on request.
3A complete list of the 26 studies to which we compared our normalized rankings and the results of the corresponding correlation tests is available on request.
Figures
Figure 1. Information flow (full network).
Figure 2. Information flow (spring embedding, 0.10 threshold).
Figure 3. Information flow (spring embedding, 0.15 threshold).
Tables
Table 1. Normalized ranking scores for top 25 cited journals.
Table 2. Most prestigious/central journals in the IS journal network.
©2008 ACM 0001-0782/08/0100 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2008 ACM, Inc.
No entries found