What Is Integration
What Is Integration
net/publication/220672562
What is integration?
CITATIONS READS
91 31,989
1 author:
Thomas R. Gulledge
George Mason University
125 PUBLICATIONS 1,536 CITATIONS
SEE PROFILE
All content following this page was uploaded by Thomas R. Gulledge on 27 June 2014.
What is
What is integration? integration?
Thomas Gulledge
George Mason University, Fairfax, Virginia, USA
Abstract 5
Purpose – This paper aims to provide a clarification of the meaning of the term integration.
Design/methodology/approach – A taxonomy of integration definitions derived from the
academic and trade literature is developed, analyzed, and documented.
Findings – Integration is a word that is commonly used when discussing enterprise applications.
The term integration is inserted in technical papers, e-mail messages, correspondence, proposals, and
even causal conversations. After many years of project work, and many misunderstandings and failed
meetings and workshops, it can only be stated that the word has multiple and misunderstood
meanings. For technical papers (research and trade), the term must be provided with context, or it is
impossible to have a meaningful conversation. Next, multiple alternative definitions (that are valid in
the literature for the appropriate context) are presented and explained in some detail.
Research limitations/implications – The paper is not exhaustive, since new definitions of
integration may exist or may emerge.
Originality/value – The main contribution of the paper is that it yields clarity on a key term that is
frequently used in information systems research. The paper is useful to any researchers or
practitioners who are focused on enterprise system implementation.
Keywords Integration, Interface management, Applications, Information systems, Research
Paper type General review
Figure 1.
Oracle advertisement for
Big I
i.e. “No matter how well you do it, systems integration can never get all of your data
together in one place”[1].
Ellison has a point – interfaces are expensive[2], and they can comprise a
significant portion of the cost of any enterprise system implementation project. And
also, there are the problems with complexity and managing scope integrity across
multiple data sources (Gulledge and Sommer, 2004). Consider Figure 2 from an
unnamed company. Figure 2 shows a situation that is described in the literature as
“systems integration;” i.e. the interfacing of systems together so they can pass
information across a complex technology landscape. We call this type of integration a
form of “Little i,” and we note that this form of Little i (point-to-point interfaces) is an
expensive proposition. Data must be constantly harmonized and cleansed across
multiple data sources, and any changes to one system can lead to complex and costly
re-testing or even re-design and coding of interfaces.
Clearly, we have presented two extremes, and by and large both have been rejected
by large organizations world wide. Most organizations do not want to include all of
their data in one application (e.g. Oracle, SAP, Microsoft, etc.) for a number of different
reasons, but at the same time, no one wants the problems that are associated with
implementations like that shown in Figure 2.
There are other options. In fact there are many options, and that is the point of this
paper. All of the options (including the two above) are called integration. So what is
integration? As one might guess, it depends on the context, and the usage must be
qualified. Big I may not achievable, and it may not even be appropriate. If Little i is
appropriate, what type of Little i is appropriate, given the situation and the state of
What is
integration?
Figure 2.
Interfacing systems
components to define an
enterprise solution
emerging technologies? This paper addresses those questions, and it also categorizes
the most used forms of Little i in the context of enterprise system implementation. This
categorization and associated discussion is essential, or it is impossible to have a
meaningful discourse about application integration.
Integration – Big I
To establish a baseline, the following definition is proposed for integration.
Integration (Big I) – integration implies that all relevant data for a particular
bounded and closed set of business processes is processed in the same software
application. Updates in one application module or component are reflected throughout
the business process logic, with no complex external interfacing. Data are stored once,
and it is instantaneously shared by all business processes that are enabled by the
software application.
This is a rather comprehensive and restrictive definition that revives memories of
first generation enterprise resource planning (ERP). The business process implications
of Big I are discussed in some detail by Gulledge and Sommer (2003).
To preserve clarity throughout this paper, the above definition will always be
referred to as “Big I.” Big I is definitely the goal of management, especially for
mundane business processes. This implies “one source of truth” for those business
processes that are enabled by core ERP solutions. The concept is simple: if all data are
stored once and shared, then integrity issues are less likely to occur. The TCO is
significantly less, since interfaces across application components are not required.
Furthermore, complexity is significantly reduced.
IMDS Figure 3 shows how Big I relates to Little i for a simple example related to US Army
106,1 Logistics.
In this example, Army Logistics processes are scoped with the SAP solution as Big
I; i.e. there is no interfacing across the SAP components. However, some of the logistics
business processes flow outside of the Army. In this case, we indicate the
transportation processes that are part of the end-to-end logistics business processes,
8 but they fall outside of the Army, and they are managed by the US Transportation
Command (TRANSCOM). The systems that support this segment of the end-to-end
process are not SAP, and they are not even owned by the army. This is a classical
composite application[3] and some form of Little i is must be implemented in order to
preserve the integrity of the business process logic[4].
Figure 3, even though a simple picture, shows much about integration. First, it
suggests that large and complex organizations are unlikely to place all of their
business processes in a single application. While assertions of Figure 1 are accurate,
there are at least two reasons why single instance ERP will not occur in most firms:
(1) the internet opened more options for Little i; and
(2) the culture and control of the internal and external system integration
communities will not allow such consolidation.
Like it or not, given the current state of technology, we are going to have to live with is
a mixture of Big I and Little i, at least as long as the current trends continue. The reality
of this situation is reinforced by the fact that the larger software providers are
“opening” their products and making them more flexible for mix and match
Figure 3.
An example of Big I and
Little i in the same
enterprise
opportunities with Little i. This is evidenced by such products as the Oracle Data Hubs What is
and SAP NetWeaver technologies. While it is true, just as Figure 1 shows, that the TCO integration?
could be reduced by moving to Big I, most organizations do not have the flexibility nor
the desire to do that.
However, this does not mean that Big I is dead. There will always be pockets of
Big I; connected by Little i, to other pockets of Big I. This is not a technical assertion,
but is directly related to common sense. For example, one would never “rip” a product 9
like SAP core ERP apart and then interface it back together again. This is self inflicted
pain, and it can be avoided by just implementing the product the way it was intended
to be implemented[5]. Preserve the integrity of the product by implementing Big I
whenever possible, and use Little i to include those components that cannot be included
in the integration domain. One would never dream of separating financials from
materials in an SAP implementation, and then interface it back together again. Or even
worse, it makes even less sense to stand up independent SAP solutions in different
divisions of a company, operating as a family or fiefdom, with the absence of an
enterprise orientation.
We will revisit implementation options later, but before doing that, we must further
explore the options for Little i. The choice of a particular little i technology has
significant implications for the types of mix and match options that are available for
consideration.
Integration (Little i)
As previously mentioned, all forms of Little I are some form of interfacing, even though
they are loosely called “system integration.” Much has been written on the subject, so
we only focus on those types of Little i that are most relevant for the implementation of
enterprise systems:
.
point-to-point integration;
.
database-to-database integration;
.
data warehouse integration;
.
enterprise application integration (EAI);
.
application server integration; and
.
business-to-business (B2B) integration.
Point-to-point integration
This is the most expensive form of integration. Point-to-point integration is the pair
wise development of interfaces among systems. The data model of the target and
source system are known, and someone (e.g. a system integrator) develops the code for
passing information back and forth. Sometimes accelerator products are used, a good
example being the IBM MQSeries of middleware products that are now included as a
part of WebSphere. MQSeries does require writing code at both the source and target
system. The approach to point-to-point integration is well known, most frequently
involving changing both applications to use a middleware layer, by rewriting the
transaction handling code to communicate across the two applications. The traditional
model of interaction is through remote function calls.
The largest problem with point-to-point integration is shown in Figure 4, a situation
that Schäfer (2002) attributes to a customer situation.
IMDS
106,1
10
Figure 4.
Example of point-to-point
integration
Database-to-database integration
This form of Little i, requires the sharing of information at the database level; hence,
providing interoperable applications. The basic replication solution leverages features
built into many databases to move information between databases as long as they
maintain the same schema information on all sources and targets. There are companies
that provide middleware to accelerate this process. Database and replication software are
provided by companies such as Pervasive’s Integration Architect and DataMirror’s
Constellar Hub that permit moving information among many different database products
with different schema. Figure 6 shows the conceptual layout for this form of Little i.
While this integration procedure may work well for database applications, it does
not work so well for enterprise applications. Most enterprise applications have
What is
integration?
11
Figure 5.
Point-to-point integration
from defense financial and
accounting services to the
US Navy Pilot SAP
implementations
Figure 6.
Conceptual layout for
database-to-database
integration
IMDS multi-tiered architectures, where even though the applications reside at a separate tier,
106,1 the business process logic is “bound” to the master data. So, if one simply passes
information at the database level, it is easy to create data integrity problems.
Enterprise software vendors typically publish application program interfaces (APIs)
that allow interfacing at the application level, and it is best to use these APIs. If you
update the database without using the APIs, then you are violating the Big I that is
12 engineered into the product, and integrity problems are a likely result. See that
Anonymous (1999) article in enterprise development where some of these difficulties
are discussed within the context of interfacing with SAP’s R/3 product. For enterprise
implementations, this form of Little i should be avoided.
Figure 7.
Conceptual view of data
warehouse integration
If the integration is at the database level, the same problems associated with What is
database-to-database integration that were mentioned above still apply. If the integration?
integration is at the application level, then data warehouse integration is similar to
point-to-point integration, and the problems with that approach also apply. This form
of integration is quite popular, even though it is expensive to maintain. The reason that
data warehouse integration is popular, is that it allows all parties involved to maintain
their individual stove-piped environments while sharing selective data in a neutral 13
environment. In short, one is trading Big I for autonomy. An example of a large data
warehouse integration effort in the US Army is shown in Figure 8. The logistics
integrated database (LIDB) contains aggregates information from many stand-alone
systems, with the objective of providing enterprise-level analytics. As the figure
indicates, the input data are aggregated from many sources, and output data are
pushed to many sources. Constant cleansing and harmonization is required in order to
avoid integrity problems.
Many enterprise solutions, like those from SAP and Oracle, use data warehouse
solutions for reporting and enterprise analytics. However, this static view of enterprise
data are not the same as Big I. Even if the concept is extended to include a federated
query capability with the data warehouse being a virtual repository of metadata, this is
still no substitute for Big I. However, the big problem, as previously mentioned, is the
maintaining of business process logic at the data warehouse level. While this option
preserves organizational autonomy, it is indeed costly. The data that are pushed into
the warehouse must be constantly monitored for quality, and any changes in any one
of the target or source systems create significant testing and/or additional coding
problems.
Figure 8.
A conceptual view of the
LIDB
IMDS Enterprise application integration
106,1 EAI is the sharing of data and business process logic across hetero/homogeneous
instances through message-oriented-middleware (MOM). EAI may be managed by
packaged vendors (e.g. SAP and Oracle) or through solutions provided by third party
vendors (e.g. IBM, WebMethods, etc.). EAI is sometimes called application-centric
interfacing. EAI is used to connect multiple systems at the application or database
14 levels, using a form of middleware that is sometimes called a broker. The middleware
moves information in and out of multiple systems, using pre-engineered “connectors.”
The connectors are a source of competitive advantage for EAI software providers,
because if a connector already exists for the target and source application, the cost of
interface development can be reduced.
The problems associated with point-to-point integration are reduced by adopting a
hub and spoke model for sharing information. The EAI Middleware allows one to write
a single interface between each application and the middleware, instead of individually
connecting each application to every other application. An example of a hub and spoke
architecture is shown in Figure 9.
Once the information is extracted, it is sent to a central server using some sort of
messaging system, where the information is processed and routed to the target system.
If there is a gap in required business process logic, the logic can be created on the
central server for execution. In theory, any-to-any document swap is possible,
considering the business process logic in the source and target systems.
Using “connectors,” the EAI software processes messages from packaged
applications, databases, and custom applications using a queuing engine. When an
Figure 9.
Hub and spoke
architecture for enterprise
application integration
event occurs (e.g. a transaction in an ERP package or a database table update), a What is
message is published to the queue about the event. Subscribers to queue access the integration?
event envelope, analyze the content, and if it is intended for processing in the target
system, the envelope contains everything necessary for recreating the event in the
target system. The queuing engine ensures that all events are processed in the correct
sequence, ensuring transactional integrity.
Many companies provide pre-packaged EAI solutions, and the market is extremely 15
competitive. The hub and spoke model using connectors has been operational for many
years, and the products have reached a mature level. However, we note that EAI is still
interfacing, and while this is a significant improvement over point-to-point integration,
EAI can be costly to implement and costly to maintain. The main benefits flow from
being able to use “partially configured” connectors, while leverage industry
partnerships which yield certified interfaces. Tremendous consolidation has
occurred in recent years in companies that provide EAI solutions as the larger
software providers have moved in to provide EAI solutions that interact with their Big
I products. For example, SAP now supports EAI as part of its NetWeaver[7] solution,
where previously SAP had used third party providers like IBM and WebMethods to
provide EAI capabilities.
It is also important to note that EAI is typically used inside the enterprise, as
opposed to across the enterprise. For this reason EAI is sometimes called
application-centric interfacing. The objective is to interfaces processes and share
data within the enterprise. The inter-enterprise model falls under a class of solutions
that are called Business-to-Business eCommerce, and this form of interfacing will be
discussed in a later section.
16
Figure 10.
Application server
integration concept
the interfaced applications or components are tightly coupled to each other by sharing
methods. By our assessment, application server integration is Little i, but given the
limits of current technology it is the best approximation that we can provide to
Big I. This is because the data integrity checks and business logic bound to the objects
are always shared, and therefore, never circumvented.
The SAP example is not unique. Most of the major software vendors have a similar
strategy. For example, Figure 11 shows the Oracle strategy for application server
integration.
The key component of Figure 11 for our discussion is in the right-center of the
figure. The Oracle Application Server manages the shared objects and during runtime
“Top Link manages persistence between java objects and database tables.” At the
conceptual level the integration approaches pursued by Oracle and SAP are similar.
The widely accepted disadvantage of using this application server integration is
that significant changes may have to be made to all source and target applications to
expose them around a common set of services (you may have to write object
wrappers[8]). For newer applications, most code is designed with a service-orientation,
alleviating some of the problems with older legacy systems. For a more comprehensive
discussion of the SAP application server offering, see Kangas (2005).
Business-to-business eCommerce
We include this section to clarify other sources of confusion. Independent companies
often execute electronic transactions with each other. While on the surface, this is
similar to point-to-point integration, it is different. B2B connectivity is the passing of
What is
integration?
17
Figure 11.
Application server
integration within Oracle’s
service-oriented vision
Most organizations have needs for B2B connectivity, as well as other forms of
integration. Internal business processes and systems require enterprise integration
efforts, where external business transactions are processed through B2B eCommerce.
Notes
1. This quote appears in Ferguson and Callaghan (2002). 19
2. Some indication of the cost is provided in Gulledge et al. (2004).
3. See Woods (2003).
4. These business process extensions are often called “architectural alignment,” and we discuss
these issues in detail Gulledge et al. (2005).
5. See Gulledge and Sommer (2004).
6. This figure was taken from a presentation by Farrell (2001).
7. SAP’s EAI solution is know as the exchange infrastructure (XI).
8. Wrapper is a common term in the software development literature. See the paper by Sivan
and Venkatavaradan (2005) for the context of usage in this paper.
References
Anonymous (1999), “Integrate with R/3”, Enterprise Development, Vol. 1 No. 8, pp. 25-6.
Farrell, J. (2001), ERP Financial Interfaces, (Power Point Presentation), Defense Financial and
Accounting Services, August.
Ferguson, R.B. and Callaghan, D. (2002), “Tools to ease integration”, eWeek, Vol. 19 No. 15, p. 12.
Gulledge, T.R. and Sommer, R. (2003), “Public sector enterprise resource planning”, Industrial
Management & Data Systems, Vol. 103 No. 7, pp. 471-83.
Gulledge, T.R. and Sommer, R. (2004), “Splitting the SAP instance: lessons on scope and business
processes”, Journal of Computer Information Systems, Vol. 44 No. 3, pp. 109-15.
Gulledge, T.R., Simon, G. and Sommer, R. (2004), “Analyzing convergence alternatives across
existing SAP solutions”, Industrial Management & Data Systems, Vol. 104 No. 9,
pp. 722-34.
Gulledge, T.R., Hafez, W., Ledwon, M. and Svensson, C. (2005), “Solution architecture alignment
for logistics portfolio management”, International Journal of Services and Standardsin
press.
Kangas, M. (2005), Introduction to the SAP Web Application Server, SAP AG, Walldorf.
Linthicum, D.S. (2001), “Where EAI meets B2B”, Software Magazine, Vol. 21 No. 2, pp. 22-5.
SAP AG (2004), Enterprise Services Architecture: An Introduction, SAP AG, Walldorf.
Schäfer, T. (2002), SAP Exchange Infrastructure Architecture Overview, SAP AG, Walldorf,
(Power Point Presentation), (December).
Sivan, S.S. and Venkatavaradan, R. (2005), “Design guidelines: building web service wrappers for
an XML-based system”, DevX, available at: www.devx.com/enterprise/Article/27882
(accessed 19 April).
Woods, D. (2003), Packaged Composite Applications: An O’Reilly Field Guide to Enterprise
Software, O’Reilly Publications, Sebastopol, CA.
Zeng, A.Z. and Phatak, B.K. (2003), “Achieving information integration in supply chain
management through B2B e-hubs: concepts and analyses”, Industrial Management & Data
Systems, Vol. 103 No. 9, pp. 657-65.
IMDS Further reading
106,1 CrossWorlds Software, Inc. (1999), Enabling the Real-time Enterprise: Enterprise Application
Integration with CrossWorlds Software, CrossWorlds Software (Now IBM), Burlingame,
CA.
Joch, A. (2005), “Modern design”, Oracle Magazine, May-June.
20 Corresponding author
Thomas Gulledge can be contacted at [email protected]