Communications of the ACM

Communications of the ACM

Is Information Systems a Reference Discipline?

By Pairin Katerattanakul, Bernard Han, Alan Rea
Communications of the ACM, May 2006, Vol. 49 No. 5, Pages 114-118
10.1145/1125944.1125950
Comments

The notion of information systems (IS) being an applied discipline drawing upon other disciplines has been held as conventional wisdom by IS researchers for the past two decades [2]. IS has been conceived as growing from other foundational fields, such as computer science, management science, and organization science [4]. Recently, this conventional wisdom has been challenged by the new perception that IS is a reference discipline for others. Evidence supporting this new perception includes: established IS journals, settled academic conferences, distinct subject matter and perspectives in IS research, an established international society, other well-developed networking channels [2], and usage and recognition of the value of IS research [1].

In addition, IS researchers maintain that IS pervades diverse areas and has much to offer researchers in other disciplines [2]. This claim is supported by two well-known IS articles [5, 10] frequently cited by numerous publications in many disciplines. Unfortunately, no published literature has examined knowledge contributions of IS research to other disciplines. Thus, to provide more objective evidence in support of this new perception, we conducted research using citation analysis to examine how frequently the articles published in major IS journals are cited by publications in other disciplines.

Although citation analysis is not new in business research, most past research might better be termed "reference analysis" since the unit of analysis was the references in, rather than the citations of, an article [3]. Examining the references provided in a journal is useful for identifying what disciplines influence that journal, but not for identifying that journal's contributions to other disciplines. Who we reference indicates where our knowledge base comes from [9]; conversely, where our research is cited provides insights into its contributions. Therefore, this study employs citation analysis to examine the citations made to the articles published in major IS journals to assess knowledge contributions of the IS discipline.

Six major IS journals: Communications of the ACM (CACM), European Journal of Information Systems (EJIS), Information Systems Journal (ISJ), Information Systems Research (ISR), Information & Management (I&M), and MIS Quarterly (MISQ), were selected to highlight the contributions made by the IS discipline. The selection process was carefully done by following the same approach employed in another study on the quality ranking of computing journals based on citation analysis [8]. A brief description here highlights general characteristics of each journal:

• CACM. This flagship publication of the Association for Computing Machinery (ACM) was established in 1958. CACM is published 12 times a year and covers a rich assortment of feature articles that examine both technical and non-technical subjects related to computing, information processing, and evolving technologies.
• EJIS. Published since 1992, this journal is aimed at both the theory and practice of information systems. Specifically, it focuses on technology development, implementation, management, strategic planning, and policymaking. EJIS publishes research articles contributed by academic researchers, and case studies/ reflective articles by practitioners.
• ISJ. Published since 1991, this journal focuses on the study of information systems. ISJ publishes articles with a wide-range and interdisciplinary nature since it promotes integration of technology with management and other disciplines such as psychology, philosophy, and sociology. ISJ is known for publishing research using qualitative methodology.
• ISR. Published since 1990, this is the representative journal of the Institute for Operations Research and the Management Sciences (INFORMS). ISR focuses on theory, creative research, and novel use and design, development, and implementation of information systems that involves organizational, economical, and social ramifications.
• I&M. Published since 1977, this journal is aimed at knowledge dissemination of applied research in information systems in order to provide guidelines and insights into the successful undertaking of information technology initiatives. I&M primarily serves information professionals of the organizations that design, develop, manage, and use information systems applications.
• MISQ. Published since 1977, this journal is aimed at academic research that is critical to appropriate decision making in regard to the acquisition, deployment, and use of information technology. MISQ has a strong focus on the managerial aspect of knowledge in IT, and its impact on modern organizations and human societies.

From the six selected IS journals, a list of 1,120 target articles published in these journals from 1995 to 1998 was compiled. This timeframe was chosen because citation data for some selected journals (such as EJIS and ISJ) was only available from 1995. Then, a second list of the citations made to each target article was compiled. To ensure a reasonable citation history for analysis, all citation data was collected from 1997 to 2000 since the modal elapsed time between IS article publication and citation was found to be approximately two years [7].

Citations were drawn from two well-known citation sources: Social Science Citation Index (SSCI) and Science Citation Index (SCI), which cover a broad range of publications. Altogether 4,668 citations were made to the 1,120 target articles. These citations were made from the articles published in 586 citing journals. Then, using the journal categories provided in SSCI and SCI with some proper modifications, these 586 citing journals were further classified into 15 major disciplines. Finally, for each selected IS journal, the numbers of citations made to its articles by the 586 citing journals was recompiled and aggregated with respect to each discipline.

Table 1 reports the number of citations, the number of target articles, and the average citations per target article for each selected IS journal. The target articles published in CACM receive more than half of the citations (2,668 citations or 57.2%) collected in this study. For citations per target article, on average each target article receive 4.17 citations. MISQ has the highest citations per target article (8.01) followed by ISR (5.83), CACM (4.46), EJIS (3.07), I&M (2.18), and ISJ (1.95), respectively.

Table 2 reports how often each selected IS journal was cited by each discipline. From a total of 4,668 citations made to the target articles in this study, the highest proportion (43.9%) of these citations were from the publications in the IS discipline. Thus, approximately 56% of the collected citations are attributed to other disciplines. Computer science (28%), management (7.6%), engineering (5.8%), sociology (2.6%), library and information science (2.3%), and medicine and health care (2.2%) account for approximately 48% of the citations. Nearly 8% of the citations are attributed to other disciplines (education, psychology, economics, and marketing).

---

We conducted research using citation analysis to examine how frequently the articles published in major IS journals are cited by publications in other disciplines.

---

More than half (ranging from 57.6% to 68.9%) of the citations made to target articles published in five selected journal: EJIS, ISJ, ISR, I&M, and MISQ, are from the articles published in IS journals. However, only 27.3% of the citations made to target articles published in CACM are from the articles published in IS journals. In contrast, publications in computer science journals moderately cite (ranging from 6.4% to 18.4%) the target articles published in EJIS, ISJ, ISR, I&M, and MISQ, while 43% of the citations to CACM target articles are from computer science journals.

Another interesting finding reported in Table 2 is that European IS journals (such as EJIS and ISJ) are cited by fewer disciplines (only 12 and eight disciplines respectively). Possible reasons for this result include the limited availability of European IS journals in U.S. universities and the relative newness of these IS journals (EJIS and ISJ started in the early 1990s). However, a firm explanation can only be supported with additional investigations.

To examine whether particular IS journals are popular to any specific disciplines, we considered the percentage that each selected IS journal contributes to each discipline (see Table 3). Considering that target articles published in CACM receive the highest proportion of citations (2,668 citations or 57.2%), it is not surprising that, among the six selected journals, CACM has the highest percentage of contribution to virtually every discipline. For CACM, contributions to each discipline range from 12.5% to 87.8%. Publications in computer science, library and information science, education, engineering, medicine and health care, and others cite CACM more frequently (87.8%, 87.7%, 85.5%, 78.4%, 75.7%, and 85.3%, respectively) than any other selected journals. Similarly, publications in communication, psychology, sociology, marketing, and other businesses cite CACM at moderately higher percentages than other selected journals. Thus, CACM is quite a popular knowledge source for a variety of disciplines.

The citations in accounting and economics publications for each selected IS journal are comparably low (ranging from 0 to 3 citations). Thus, none of the selected IS journals seems to be a major knowledge source for these two disciplines. However, it is worth noting that each of these disciplines cites the selected journals less than 10 times. For publications in management, CACM, ISR, and MISQ are three major knowledge sources; whereas the other three selected journals (EJIS, ISJ, and I&M) also noticeably contribute to management publications. Finally, the percentage that IS publications cite target articles published in CACM and MISQ (35.5% and 22.3% respectively) make these two journals the main knowledge sources for IS publications. ISR and I&M (15% and 14.5% respectively) are considered significant knowledge sources for IS articles as well; whereas the European IS journals EJIS and ISJ (9.1% and 3.5% respectively), also noticeably contribute to IS publications.

Back to Top

Conclusion

Any discipline striving toward scientific maturity is justifiably concerned about the utility of its knowledge and the rate at which such knowledge is disseminated across the scientific community [3]. The IS discipline is no exception. This study employs citation analysis to examine the contributions of IS knowledge—where and how often it is cited. Results from this study provide strong evidence that the IS discipline has become a reference discipline for others. That is, IS research published in IS journals is frequently cited by other disciplines, even those fields that previously served as reference disciplines for IS (such as computer science, management, and organization science). The frequent citations made from other disciplines to IS research also suggest that IS research contributes to advancing the body of scientific knowledge—as the intended purpose of publications in academic journals is to impart knowledge to others, furthering the advancement of scientific achievements [11].

We believe that further investigation of IS knowledge contributions is warranted since the results of this study are an "average" across target articles published in six selected IS journals during four-year period 1995–1998. In addition, an examination of the citation patterns for all major IS journals would more clearly reflect knowledge contributions within and across disciplines of IS research and would also shed more light on the maturity of IS discipline.

Finally, despite its extensive use, citation analysis is not without its drawbacks. Citation data available in SSCI and SCI is by no means complete. However, these indices have the most extensive citation coverage available for more than 7,000 journals. Study results have shown that as few as 150 journals account for half of what is cited and a core of only approximately 2,000 journals account for about 85% of published articles and 95% of cited articles [6]. Thus, the citation data available in SSCI and SCI should provide a valid picture of the knowledge contributions of journals.

Back to Top

Back to Top

Back to Top

Back to top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2006 ACM, Inc.

---

No entries found