Communications of the ACM

Virtual extension

How a Service-Oriented Architecture May Change the Software Development Process

By Marc N. Haines, Marcus A. Rothenberger
Communications of the ACM, August 2010, Vol. 53 No. 8, Pages 135-140
10.1145/1787234.1787269
Comments

Software development practices have evolved substantially during the past decade. As so called "agile" approaches have gained more acceptance and applications have become progressively more distributed in terms of their physical execution and the development of components, the service-oriented approach to IT architecture has become an important alternative to traditional^a software development.^1,9 Another impetus for the trend to a Service-Oriented Architecture (SOA) is provided by enterprise system vendors as they are incorporating the service-oriented paradigm into their products. Substantial efforts related to open standards (such as Web service standards) and open source products (such as open source enterprise service bus, development tools) are further driving a service-oriented approach for information systems.

A key question is whether SOA adopters are going to be ready for this change and whether they can provide a technical and an organizational environment in which SOA-related technologies can be leveraged to their full potential. There is some indication that currently this may not be the case. In fact, some organizations that have embarked on SOA-related projects early have experienced disappointments. As with other technology waves, the important question is not whether SOA is inherently a good or a bad idea, but rather how it can be done right in a given context. This article tries to answer this question with respect to the software development process.

While much of the literature, both in academia and industry, has focused on business implications of SOA,³ technological realization, architectural issues,^4,10 and implementation guidelines,⁷ few publications have addressed the impact of SOA on the software development process and its methodology. As with any organizational change, modifications to software development processes or practices entail switching cost. Therefore, individuals as well as organizations are inclined to stay with "proven" methodologies,^5,12 although adjustments based on task requirements and technology characteristics should be key drivers for the methodology choice⁶ and are needed to help adopters leverage the full potential of SOA.

This article examines the differences and discusses which parts of development process and methodology may require adjustments to effectively leverage a SOA. It presents the results of a field study suggesting changes to software development practices that are necessary to accommodate the unique properties of the service-oriented approach.

Back to Top

Service-oriented Architecture

Service-oriented Architecture (SOA) is defined in the OASIS reference model as "a paradigm for organizing and utilizing distributed capabilities that may be under the control of different ownership domains."⁸ It can also be viewed as an architectural style for building software applications that uses available services in a network,¹⁰ where the service interface along with its description is a critical part.^b A SOA should provide flexibility regarding the technical execution environments behind the service interfaces, as well as flexibility for creating new interfaces and new composite services. However, once a service interface is established and used by potentially a multitude of consuming applications, changes to the interface need to be kept to a minimum to avoid disruptions in the consuming applications and provide continuous business operations.

Development in the context of SOA entails two major tasks: One is establishing a service infrastructure (such as, core services, management, and security); the other is creating composite applications (or services) based on the service infrastructure.³ Both types of development have different characteristics. The infrastructure can be viewed as the combination of the interface layer and the attached execution environments. Figure 1 illustrates both aspects of SOA-based development. The services may be used by a multitude of client applications and the execution environment may also change, as long the interface contracts are not violated. Consequently the interface layer, which encompasses securing, monitoring, and managing the services, is the pivotal element and requires a holistic, global view and very rigorous development with a long-term horizon.

A typical SOA scenario consists of composite applications constructed from services that may be provided internally or by a third party. Thus, functionality provided by the service may reside at an external location and outside of a domain of direct control. Services frequently rely on the public Internet. Also, contrary to other component-based technologies,¹¹ services are usually not purchased as a software package from a third party and implemented within the organization. This has potential consequences for the software development process not only from a technical, but also from a managerial perspective (such as inter-organizational arrangements and trust issues).

Standardization is perhaps the key underlying enabler for a modern SOA, solving many of the interoperability issues that plagued previous distributed technologies (such as CORBA) and allowing vendors to create tools that automate important steps in the software development process that previously required a manual programming effort. Therefore, some development activities, particularly those related to the connectivity aspect of distributed application development are likely to be deemphasized.

The unique features of SOA, and specifically technologies related to Web services, merit an investigation into a number of development activities, as certain activities may need to be added, emphasized, or rearranged, while others may need to be deemphasized. Not making such needed adjustments may reduce the overall benefits of SOA. For example, an organization that aims to reduce the overall time-to-market of new business services can realizes time reductions in the implementation phase, as SOA-based composite applications effectively reuse existing IT capabilities. However, if testing was not appropriately adjusted, synchronization problems between development and quality assurance may diminish the gains made in the implementation phase. Negative consequences can also arise from a lack of attention to service interface and message design. This can lead to interoperability challenges and additional mediation work, which then adversely impact productivity gains and cost reduction goals.^c

Back to Top

The Field Investigation

Since the unique properties of SOA suggest differences in the software development environments that deploy this paradigm, this study seeks to investigate the practical realities and needs of software development in the context of SOA, and provide insights into necessary modifications to the systems development process based on the analysis of the empirical evidence.

The key data source for this field study is a set of semi-structured interviews with software developers and IT managers from five organizations. The interviews were guided by a set of questions that specifically address the issue of software development in the context of SOA-based on Web services.^d The purpose of the mostly open-ended questions in the interview structure is to help the researcher to consistently address key issues identified in the literature and prior research, while still maintaining the opportunity for eliciting new and unexpected insights from the participants. Participants subsequently reviewed summaries of the interviews and provided feedback on the correct representation of the content. The information obtained in the interviews was supplemented with insights acquired from company documents and Web sites. Table 1 summarizes the research methodology employed.

Individuals from one consulting firm and from four field companies participated in the study. The consultant is nationally recognized for enterprise integration and has been involved in the development of SOAs based on Web services for several years. All companies participating in this study employ Web services and have established or are in the process of moving to a SOA; they are from the insurance, the financial services, and the entertainment industry, as well as from the judicial branch of state government. The participants are working either as enterprise architects, including one IT director, or as senior developers involved in projects related to SOA and Web services. While the companies reported pockets of agile development, we observed overall two typical baseline scenarios for software development practices prior to the introduction of SOA. The majority of software was developed using either a customized methodology based on the Rational Unified Process (RUP) or a controlled waterfall-style process. Table 2 provides a summary of the nature of SOA and Web service use in the four field settings.

Back to Top

Software Development Process Changes

According to the shared views of all participants, there are significant differences between software development in the context of SOA with Web services, compared to previous software development. While some attributed the differences to general SOA principles and a focus on the application interface, others mentioned issues related to using open standards and web services (such as, assessing viability and maturity of standards).

The interviews confirmed the dual nature of software development within the context of SOA. As one participants pointed out, "there are two sides to Web services." Building the service infrastructure and using the services for composite applications were viewed as substantially different tasks. Building the service infrastructure requires a rigorous—and perhaps more waterfall-style—approach focusing on long-term planning to achieve well-designed and reusable services. On the other hand, composite application development may in many circumstances benefit from using a more agile, yet rigorous, software development approach. The results suggest that software developers should consider these as two distinct tasks with potentially different requirements.

All participants identified some aspect of the software development process that needed to be adjusted in the context of SOA with Web services. The comments relate to the overall process, as well as to specific activities. Below is a presentation of the key findings as they relate to the five major phases^e of software development.

Planning. There was agreement among the participants that this phase differs substantially, particularly if no prior experience with service-oriented development exists in an organization. The consultant pointed out that this is where 90% of the projects fail, as it is a labor intensive, often boring and difficult activity. According to a participant, there is less certainty in a service-oriented environment, as many aspects are less predictable and harder to control as they are executed by other business units or external parties. This makes, for example, the estimation of service consumer projects more difficult. Another participant pointed out that planning has to be more holistic to effectively develop "enterprise services" that can be leveraged throughout the organization. It becomes essential that effective communication channels among the stakeholders that participate in a SOA initiative are established. An important effort in the planning stage is the assessment of standards, since current SOA efforts are typically leveraging a combination of several open standards. This includes determining which particular sets of standards are of interest, viable, and supported by each potential vendor.

Analysis. The participants suggested that the analysis phase requires the least modification. However, development in a service-oriented context requires at least as much rigor and discipline as in previous architectures, which is at times ignored. One participant pointed out that traditional analytics is one of the most important skill. "Create the future vision, and then map out how you are going to get there. That is where people are falling today. They jump right into the new architecture, because it is fun." Another participant pointed out the importance of ensuring in the analysis phase that the common data models and schemas have all the required data available, as SOA requires a more global perspective. On a positive note, the participants agreed that development toolsets are improving and are now better supporting the analysis activities for SOA-based development.

Design. The participants observed substantial differences in this phase. "[Design] is where the most radical change is occurring," according to the consultant. The three key issues in the design phase are the initial contract interface using WSDL, the granularity of the services, and the accessibility and performance of remote services.

Due to their pivotal role in a SOA, it is important to define interfaces well up-front as the cost for changes increases in later phases. While this increase of costs is true for most software development contexts, it is even more pronounced in a SOA context, because of the distributed nature and the often lesser control over each component. When moving from an existing SOA based on proprietary technology to standard-based SOA using Web services, the design of the backend systems (execution environment) may not change. However, the development of the interfaces will change dramatically as WSDL and other standards needed for the service contract must be incorporated. Despite the availability of tools that support the generation of interfaces, this requires a solid understanding of these standards not only from a technology perspective, but also from standard maturity and tool availability perspectives.

Service granularity has already gained much attention in the literature.^2,8 While coarse-grained services are generally considered good in a SOA context, there are important trade-offs. Services that are too fine-grained and require many roundtrips to complete a business transaction may cause performance problems, while services that are too coarse-grained limit reusability and may result in the wrong applications being built.

The location and level of service distribution was stressed by the consultant who cautioned that building composite applications with "hundreds of services and 20 of them being remote" is not workable in the short run, as the tools for managing these services are currently not available. Considering the maturity of the technology and related tools at the time of the research, incorporating too many remotely located services in a composite orchestration would be considered bad design.

Implementation. Business agility is one of the key promises that are touted by SOA proponents. This includes the ability to quickly develop and deploy new or improved functionality without disrupting other business interactions. Interoperability should improve, and consequently it should be easier to provide new services internally or to business partners that leverage open SOA standards. Thus, SOA should have a very positive impact on implementation and deployment.

However, it is important to understand the tools that support the implementation and deployment. Based on the comments made by participants, implementation tools are improving, but there is still a need for further deployment automation. With an increasing number of services, their management becomes a critical issue, as any service may be a potential single point of failure in a composite application. Because of the distributed nature of a SOA, implementation and deployment entails a high level of coordination between the parties that are involved.

As the parties tend to be relatively independent from each other, they are likely to be at different maturity levels in terms of Web services technology. This may entail providing multiple channels for services interaction (such as, some business partners may be able to handle specific Web service standards, while others may not; units may prefer different interfaces, such as REST vs. SOAP). Business partners that lack the necessary SOA expertise may initially even require hands-on implementation support for applications from the services provider, as one participant pointed out.

Testing. Testing is also considered a stage with potentially substantial changes in the context of SOA and Web services. In a SOA, more work on integration testing needs to be done. Test plans also need to ensure that services work in different environments with different types of service consumers (such as, portal applications versus mobile device applications). The test plans need to anticipate most of the client application types that are likely to use the service.

The two sides to service-oriented development, the development of the services (and related infrastructure) and the development of the applications that leverage services may occur simultaneously, making it necessary to use two test environments. The service developers need to be able to quickly generate test clients. The client application developers need to be able to run their applications against test services. There are also some Web service specific testing activities that need to be considered, such as checking the compliance of services with the Web service standards or Web services interoperability organization (WS-I) Profiles.

Consequently it is important to have a highly coordinated process and to consider testing throughout the life-cycle. Tools that help automating the process play an important role in effective testing by addressing not only the functional issues, but also security and performance.

Back to Top

Discussion

The field interviews provided several examples of differences that the participants are experiencing in their SOA and Web services related software development activities. Most differences occur in development activities typically associated with planning and design. But other phases, particularly implementation and testing, require noteworthy differences as well. Table 3 summarizes the findings of this study with regards to the phases of the software development life-cycle. It also shows that "traditional" structured analysis and design skills are still very important. Some aspects of service-oriented development may in fact require even more discipline and a more holistic mindset than previous development scenarios.

To address the organizational aspects of these differences, companies have designated new employee roles for education, promotion, and tracking of standards relating to SOA concepts, developer support, security, service management, and services cataloging. With regards to the development process, they have emphasized the importance of several new or changing activities, such as planning and interface design. However, there is little indication that organizations are explicitly considering a concerted change effort that addresses the differences in the development process. Only few participants noted that their organizations may be looking into new methods or mentioned that more agile approaches would benefit some aspects of SOA development. Consequently, there appears to be the potential for a gap between the existing development process and a development process that is well suited for SOA and Web services-based software development.

Composite applications built on services need to quickly respond to changing requirements and have potentially short life spans, thus they need to have short development cycles. Services, or at least their interface, on the other hand, may be used by multiple applications, need to be well-designed, and may not be easily changed or retired without breaking client applications. From this perspective agile development approaches should generally constitute a good fit for composite client applications, while the services and supporting infrastructure may require approaches that emphasize planning and rigorous design up-front. Of course, task and technology are not the only determinates and other influences, such as regulations and documentation requirements, cannot be neglected and may lead to a different choice.

Back to Top

Conclusion

Based on the experiences shared by the participants, there are important differences between the traditional software development process and software development in the context of SOA with Web services. Organizations appear to take measures to support some of these changes. These measures are currently focusing on the organizational structure, such as new roles or organizational units, but not much on adjustments of the software development process.

We argue that adjustments to the process are also necessary to provide a good fit for SOA development. Not accommodating the unique properties of the SOA architecture in the development process would be like building a new highway system, but leaving the same speed limits and regulations. While faster and more flexible transportation would be possible, the structures and processes of the old system are inadequate and may therefore lead to a slowdown and missed opportunities.

The findings of the study have provided an overview of critical areas of change and are not specific to a particular methodology. This ensures a high degree of applicability to other organizational settings. Subsequent research may be able to focus on a specific phase or a specific methodology in a given organizational context using a single in-depth case study or a quantitative field survey. The recommendations made in this study were obtained from the analysis of SOA adopters. SOA scenarios typically exhibit a number of common underlying characteristics (such as, highly distributed functionality, decoupled architecture, and standard interfaces), which are not exclusive to SOA. While our study provides insights for the context of SOA, it is left to future studies to relate software development changes to these underlying characteristics and to examine if some recommendations, made here for the context of SOA, also apply to other development paradigms.

Back to Top

Back to Top

Back to Top

Back to Top

Back to Top

Back to top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2010 ACM, Inc.

---

No entries found