(1)APNOM 2005

Methodology of performance evaluation of integrated service systems with

timeout control scheme

Akira Kawaguchi and Hiroshi YamadaNTT Service Integration Laboratories, NTT Corporation

9-11, Midori-cho 3-Chome, Musashino-shi, Tokyo 180-8585, JapanPhone: +81 422-59-2965 Fax: +81 422-59-5671

E-mail: {kawaguchi.akira, yamada.hiroshi}@lab.ntt.co.jp

AbstractMost enterprises introduce EAI (enterprise application integration) systems to improve

business processes and decrease costs. In EAI systems, the broker server can automatically orchestrate several application tasks based on the designed BP (business process) model. The BP is the workflow that defines the integrated service. The integrated service consists of several application tasks. The workflow model is defined as the order of the application tasks to be invoked. In many commercial EAI software programs, the workflow model is designed using the graphical editor, where we can model workflow using the UML (unified modeling language)-based diagram. The designed workflow model is translated into an XML-based language, which is like a BPEL (business process executionlanguage). The broker server interprets and installs the workflow model that was translated into the XML-based language and operates as the service broker.

We are developing a methodology to evaluate the performance of such an EAI integrated system. Our evaluation method is based on a simulation using the hierarchical object-oriented simulation tool, OPNET. We consider network, application traffic, and workflow layer models. To simulate these three models concurrently, we are developing new simulation modules and expanding the functions of the simulation modules on OPNET. In the expanded model, we can configure the timeout control scheme. In this paper, we show the trends in application integration technologies briefly, explain our proposed method to evaluate the performance of the EAI system with the timeout control scheme, and present a simple case study using the developed simulation modules. The proposed method enables us to evaluate the service performance of the above EAI systems.

Keywordsenterprise application integration (EAI), business process (BP), broker server, performance evaluation, simulation, service-oriented architecture (SOA)

235

(2)APNOM 2005

Introduction• EAI (enterprise application integration)

– Broker server orchestrates several applications based on BP (business process) configuration.

• BP is workflow that defines integrated service.• There is a complicated correlation between BP, AP (application), and

network.– SOA (service oriented architecture)

• SOA is a BP-centric architecture.

ClientsAP Servers

Broker server

Service request

Service provision

Orchestrating applications

Application control

BP configuration

Example of EAI structure

1. Introduction1. IntroductionMost enterprises have introduced several application systems to improve the efficiency of each

application individually. These application systems worked separately and individually. However, for complete achievement of services required by customers, outputs of several application systems are generally needed. In most cases, an output from the application system may become the input data for another application system. The service operator may need to manually enter the output data from the other application system because the application system was not integrated well. Operator errors in entering the output data manually cause some service delay. Therefore, most enterprises are introducing EAI systems. Initial EAI systems needed an adapter to connect with other application systems and to transform the data format. Developing the adapter software was expensive and it was generally difficult to scale up the integrated system when using the adapter software solution.

A recent EAI system adopts XML and Web Services technologies to resolve the problems of the above system using the adapter. These technologies change the concept of the application integrated architecture. The new concept is called “SOA (service oriented architecture)”[1], which is a business-process-centric (BP-centric) architecture. In SOA, first, the workflow model is designed using UML-based language. The broker server can interpret the designed UML-based workflow models and can work as the service broker. That is, when the broker server receives a request, it forwards the request to the appropriate application server based on the designed workflow model. When the broker server receives a response from the invoked application server, the broker server may send the next request to another application server or respond to the requester.

There are two types of communications between the broker server and application servers: synchronous communication and asynchronous communication. In this paper, we consider synchronous communication between the broker server and application servers. In synchronous communication, the sessions between the broker server and application server and those between the requester and the broker server are established. When the requester receives a response message from the broker server, then the established session is disconnected. While in asynchronous communication, message queuing communication is adopted. The broker and application servers have message queues. For example, when the broker server receives a request message, it interprets the request, creates the appropriate request message, stores it in the message queue, and disconnects from the requester. The request that is stored in the message queue is immediately dequeued and sent to the appropriate application server queue. When the application server is idle, it immediately dequeues the request message, processes it, and enqueues the response message. The response message may be forwarded to a message queue in another application server or in the broker server. Therefore, the service is divided into several tasks. In each task, request and response messages are exchanged via the message queuing communication.

236

(3)APNOM 2005

Performance issues of service integration architecture planning

• To implement integrated service on network systems, engineers should plan architecture considering the following.– Optimal workflow (BP layer)– Traffic flow of individual AP (AP

layer)– Network infrastructure and

location of AP and data (Network layer)

– SLA as the integrated service and individual APs

BP

AP，Middleware

Network

IT system environment

Service request

Provide service

Service structure

22. . Performance issues of service integration architecture Performance issues of service integration architecture planningplanning

The service operator or system manager should consider the following to design and evaluate the performance of application integration systems. (i) Optimal workflow (Business process layer)(ii) Traffic flow pattern of individual application (Application layer)(iii) Network infrastructure and location of application and data (Network Layer)(iv) SLA (service level agreement) for the integrated service and individual applications

In an EAI system, application traffic is generated based on the designed workflow model in the broker server, so there is a complicated correlation between the traffic generation pattern and designed workflow. For example, in some workflow scenarios, requests from many applications may concentrate on the same database server. That causes service delays. In such a case, the service manager may change the workflow, increase the number of servers, or upgrade the database server.

The traffic flow pattern of individual applications needs to be modeled accurately. There are many multi-tier applications, where traffic flow traverses several application servers and database servers. Compared with a simple client-server communication model, the path of the traffic flow of the multi-tier application is very complicated.

Network modeling is also important. The route of the traffic flow depends on the network infrastructure and its routing architecture. For example, the load-sharing scheme may be configured in some network nodes. In such a case, the network delay is smaller than that in the network without the load-sharing scheme. Network delay influences application performance. Therefore, network modeling needs to be as accurate as possible.

It has been necessary to consider business process, application, and network layers to design integrated service systems that satisfy the required SLA. The service manager should compare the performance measures for several scenarios.

237

(4)APNOM 2005

Related work and EAI tools• Related work

– Erikson et al. (2000)• proposed that the UML (unified modeling language) is the best tools

– Korthaus et al. (1998)• proposed the BOOSTER method

– Aoyama (2002)• described a framework of business-driven web services

– Pooley et al. (1999)• summarized past approaches to software-performance engineering and

proposed some ideas– Balsamo et al. (2003)

• proposed a simulation-based approach• Commercial EAI tools

– BizTalk (Microsoft), WebSphere (IBM), and WebLogic (BEA systems)• Adopting XML-based language like BPEL

• Our methodology expands UML-based software performance engineering to cover distributed and integrated applications.

33. . Related work and EAI toolsRelated work and EAI toolsIn developing and deploying application software in a wide network environment, performance

engineering plays an important role. In performance engineering, many research activities were performed. Our approach uses UML-based modeling techniques.

There are many types of diagrams in UML modeling: use case, activity, sequence, class, and physical diagrams. The UML is useful for designing the business process as well as developing the application software. The use case diagram provides the basis for the definition of workloads in the system. The physical diagrams provide the mapping onto computing and storage devices. They are essential to the defining of contention and the quantification of available resources. The order of the application tasks to be invoked is shown in the activity diagram. In particular, we use the activity and sequence diagrams in UML for our methodology. The activity diagram is useful for us to model the workflow of the orchestrated service. The communication path of the application messages is modeled using the sequence diagram. The sequence of messages among the requester, broker server, and application servers are shown in the sequence diagram.

The following work also used UML-based modeling in performance engineering. Erikson showed that UML is currently the best tool we have that encompasses information, business systems, and technical architecture (2000)[2]. Korthaus proposed the BOOSTER (Business-Object-Oriented Software Technology for Enterprise Reengineering) method as a multilevel approach to business-object-based system development (1998)[3]. Aoyama described a framework for the creation of business-driven web services (2002)[4]. Pooley et al. summarized past approaches to software-performance engineering and proposed some ideas on the exploitation of UML designs in performance modeling (1999)[5]. Balsamo et al. proposed a simulation-based approach to the performance modeling of software architectures specified in UML (2003). They developed a UML performance simulator, which is called “UML-Ψ”[6]. Most of the above-cited research is focused on the performance of software running on a single computer or a simple client-server application. Our methodology expands UML-based software performance engineering to cover distributed and integrated applications. Furthermore, as mentioned, an EAI tool can orchestrate several applications to implement a workflow. In our methodology, the UML activity diagrams are set up in the OPNET simulator. The transport and network layers protocols are also considered.

Most commercial EAI tools, for example, BizTalk (Microsoft)[7], WebSphere (IBM)[8], and WebLogic (BEA Systems)[9], adopt a graphical editor to model the workflow. The graphical models are translated into an XML-based language like BPEL[10] and installed in the broker server.

238

(5)APNOM 2005

• What is the measure of service performance?– Important performance measure

• Total service completion time (BP completion time)– Elapsed time from start to

end of workflow.– Individual AP response

time is important too.• Ratio of completions to requests

– Number of completed and aborted services and their ratios• Rollback completion time

– Processing completion time for maintaining data consistency, which is called “rollback” scheme

» During rollback processing, user access to the database is blocked.

Performance measures

pipeline schemepipeline scheme

start end

job job job

start end

job

fanfan--out schemeout schemejob

job

4. Performance measures4. Performance measuresIn the SLA, the total service completion time is an important performance measure as well as

individual application response time. The total service completion time is calculated as the end time minus the starting time of the workflow. Let us consider that a business process is achieved by the pipeline and fan-out scheme workflow models. In the pipeline scheme, applications are processed in order; while in the fan-out scheme, several applications are processed in parallel. In this case, it seems that the total service completion time of the fan-out scheme is shorter than that of the pipeline scheme. However, if several applications that are processed in parallel are allowed to access the same database server with a low CPU processing speed, this database server may become congested with requests. Then, the total completion time of the pipeline scheme may be shorter than that of the fan-out scheme. Therefore, the workflow process strongly influences the performance of the total service completion time.

The numbers of completed or aborted service requests are also important performance measures. In the workflow model, the timeout value for the application response time may be defined. If the application response time is longer than the defined timeout value, the broker server determines that this service transaction should be aborted. When the abortion occurs, the broker server sends a message to the appropriate database servers to maintain the consistency of the data stored in the database. It is called the “rollback” scheme. The rollback completion time is also an important measure. While the rollback scheme is processed, the database may be locked and does not accept any request. The rollback completion time is required to be as short as possible.

The application response time depends on the application traffic patterns as well as the network and server performance. That is why we should consider the business process, application traffic, and network layers when we consider the above performance measures.

239

(6)APNOM 2005

GEN

SW

FANOUT

PROC PROC

SYNCH_BAR

PROC

SINK

UML Activity diagram OPNET Workflow node

Methodology of evaluating integratedservice performance

Table mapping elements of UML to attributes of OPNET workflow

Figure of same workflow expression by UML diagram and OPNET workflow node

Workflow modeling

5. Methodology5. Methodology of evaluating integrated service performanceof evaluating integrated service performance5.1. Overview of workflow modeling5.1. Overview of workflow modeling

We developed a methodology to evaluate the performance of an EAI integrated system[11]. We are expanding the functions of the simulation modules. Our evaluation method is a simulation based on the hierarchical object-oriented simulation tool, OPNET[12]. Using OPNET, we develop the network, application traffic, and workflow layer models. To simulate these three models concurrently, we need to develop some simulation modules. In [11], we reported the development of the fundamental simulation modules to simulate the business and application processes in network models. We expanded the functions of these developed simulation modules. The timeout and rollback schemes are added to the fundamental simulation modules. Here, we consider the hub-and-spoke architecture as an integrated service architecture. The broker server communicates with several application servers based on the designed workflow model. The designed workflow is interpreted in the broker server. A request message is received at the broker server, and a new request message is created in the broker server and forwarded to the appropriate application servers based on the workflow. When the workflow is completed, the broker server responds to the requester. In the workflow model, the timeout and rollback schemes are considered. When the rollback process is needed, a rollback message is generated.

In workflow modeling, we create the workflow model according to the designed UML-based business process model. We developed several modules that simulate UML functions. The workflow model is created by using several process modules in OPNET modeling. The “GEN” process module corresponds to “Start” in UML. In this module, a WPE (workflow process entity) is generated when a request is received at the broker server. The “PROC” process modules correspond to “Activity” in UML. When the WPE is sent into this process module, the configured application is invoked in the broker server. The “SW” process module corresponds to “Decision” in UML. When the WPE is sent into this “SW” process module, the next phase is decided by a defined policy and the content of the WPE. The “FANOUT” and “SYNCH_BAR” process modules correspond to “Synchronous bar (start)”and “Synchronous bar (end)” in UML. When the WPE is sent into the “FANOUT” process module, the necessary number of WPEs are copied and sent to activity modules that are directly connected. The “SYNCH_BAR” process waits for all WPEs that were processed in the prior “PROC” modules. The “SINK” process module corresponds to “End” in UML. When the WPE is sent into this “SINK”process module, a completion signal is sent to the broker server and the WPE is deleted.

In application traffic modeling, we model the communication sequence between the broker server and application servers using the UML sequence diagram. The message sizes and the intervals between consecutive generated messages are also modeled.

In network modeling, we create a network model using router, switch, and link simulation objects in a virtual computing environment.

240

(7)APNOM 2005

Creating models in virtual environment to simulate integrated application systems

Workflow

Broker server

BPAP Binding

• Create integrated application system models in virtual IT system environment– “Workflow” node:

• We model UML-based workflow process model. – “BPAP Binding” node:

• We configure names of applications that are executed in target activity modules of workflow process.

– “Broker server” node: • Traffic sequence patterns of invoked application are

configured.• Sequence diagram between broker server and

application servers is configured.• Probability distribution of request message size and

time to create request and response messages.

5.2. Creating models in virtual environment to simulate integrat5.2. Creating models in virtual environment to simulate integrated ed application systemsapplication systems

We create simulation models in a virtual computing environment that simulate actual integrated application systems. In a virtual environment these are network, application traffic, and workflow models. In the developed simulation modules, we need to perform the following configuration to evaluate the performance of an orchestrated service. In this paper, we consider the hub-and-spoke architecture.

In a “Workflow” node, we model the workflow model by using the OPNET Node Editor. We combine process and link modules to create the workflow model. This model is created according to the UML activity diagram that describes the business process. The process modules are connected by link modules to set up the same topology as that in the activity diagram. The OPNET Node Editor is flexible enough that we can implement almost any activity diagram. We define the application name and timeout threshold values in each activity module. In workflow modeling, we consider the fan-out, synchronization, and switch schemes.

Each process module represents the function of an element of the activity diagram. In a “BPAP Binding” node, we configure the names of applications that are executed in the target activity modules in the designed workflow model.

In the broker server node, traffic flow patterns of invoked applications are configured. Those are defined by using the UML sequence diagram between the broker server and application servers. The probability distribution of the message size and time to create the request message are configured.

241

(8)APNOM 2005

Control schemes• Implemented control schemes

– Timeout• When the application response time is longer than the defined

timeout value, we determine that this application needs to beaborted.

• Application response time is measured in broker server.– Reject mode

• When consecutive timeouts occur during configured short period, broker server mode changes from “normal mode” to “reject mode”.

• In reject mode, the broker server rejects service requests to the congested application server.

– Rollback function• To maintain data consistency, broker server supports the rollback

scheme.• Rollback message is sent to appropriate application servers.

5.35.3. . Control schemes Control schemes In this paper, we consider the timeout control scheme. When the broker server invokes an

application, it starts the timer. The timeout value is configured. When the application response time is longer than the configured timeout value, we define that the application needs to be aborted.

When consecutive timeouts occur during a short period whose length is configured, the status of the broker server changes from “normal mode” to “reject mode”. In reject mode, the broker server rejects service requests sent to the congested application server. When many timeouts occur during a short period in an application server, it seems that the target application server is down or heavily congested. Then, the broker server decides to reject all or some portion of requests sent to the target application server. In the next case study, the broker server rejects all requests.

When a request abort occurs, the broker server must maintain the consistency of the data. The broker server has the following simple rollback scheme to deal with aborted requests. An aborted request may change the data in one database, but not in another database. This is the cause of inconsistent data.

The broker server manages application server names to which the broker server needs to send rollback messages when an abortion occurs. When the abortion occurs, the broker server immediately sends a rollback message to the appropriate application servers and stops sending new requests to the congested application server. In our model, the distribution of the rollback message size is configured, and the processing time of the rollback message is dependent on the message size, the number of current jobs in the server, and server processing speed. When rollback processing is completed in the server, a rollback complete message is sent back to the broker server. A further study would be to implement a more accurate and sophisticated rollback scheme.

242

(9)APNOM 2005

WPE (workflow process entity)

format

Process module（corresponds to each activity model）

WPE

Workflow node

NW model

Case study (1)• Network Model

– Hub-and-spoke architecture

Workflow node content (fan-out workflow)

Brokerserver

BPAP_Binding BP_Editor_Pipeline BP_Editor_Fanout

6. Case study6. Case study6.1. Network model6.1. Network model

In this case study, we consider the network model shown in this slide. A hub-and-spoke architecture is considered. A fan-out workflow model is also shown in this slide. When the brokerserver receives a request (for example, LAN_01 node), it notifies the “GEN” process module in the workflow node that it has received the request. In the “GEN” process module, a WPE is created and sent to the “PROC_1” process module. The WPE consists of the following information: bp_id, abort_flag, clone_id, bp_start_time, and synch_start_time. The bp_id is a unique transaction identifier. When the service is aborted, an abort_flag is set and the workflow node notifies the broker server that the service should be aborted. The bp_start_time is used to calculate the total service completion time.

When the WPE enters the activity process module, the name of the application that should be invoked is determined. The workflow process notifies the broker server about the application name to be invoked. Then, the broker server invokes the specified application. When the broker server receives a response from the application server, it notifies the activity module (for example, PROC_1) in the workflow node about receiving the response. Then, the WPE is sent to the next process module.

Requests for the integrated service are generated according to the following stochastic process. The request-generating process has “ON” and “OFF” modes. The start of the first period in ON mode is governed by a uniform distribution across the simulation interval from 20 to 320 sec. During periods in ON mode, the PDF (probability density function) of the intervals between consecutive requests from each requester is an exponential distribution with a mean of 30 sec. The period in ON mode is 300 sec. Requests are not generated during periods in OFF mode. The period in OFF mode is 600 sec.

We considered three multi tiered applications. These applications are integrated in the broker server. The traffic parameters of the application message are configured in the OPNET application task utility node model. We deliberately invoked applications in PROC_2 and PROC_3 that access the same server, which processes each application. Conflicts of requests sent to the same server will occur because the invoked applications in the PROC_2 and PROC_3 phases in the workflow are processed in parallel. This is the cause of many timeouts and abortions.

243

(10)APNOM 2005

• Workflow model– Fan-out workflow– Three applications (AP_1, AP_2, and AP_3) are invoked.– Timeout scheme is configured in PROC_2. Transaction is aborted when

response time of AP_2 is longer than timeout threshold (45 sec).– Rules of the mode (“reject” and “normal” modes) transition are

determined. – “Rollback” scheme is defined.

Structure of workflow

When abortion occurs in this process module, rollback message is sent to these servers.

Configured timeout scheme

Case study (2)

(AP_1)

(AP_2)

(AP_3)

WPE

66..2.2. Workflow model Workflow model In this workflow scenario, three applications, AP_1, AP_2, and AP_3 were invoked in the PROC_1,

PROC_2, and PROC_3 process modules, respectively. AP_2 and AP_3 were invoked in parallel and the synchronization process was performed in the “SYNCH_BAR” process module. When the “SYNCH_BAR” process module received responses from AP_2 and AP_3, the WPE is sent to the next “SINK” process module.

In the AP_2 application, the timeout control scheme was configured. The timeout threshold was set to 45 seconds. If the broker server cannot receive the requested application response until the timeout threshold has expired, then the broker server sends an abort message to the requester. Furthermore, if two consecutive timeouts occur within 10 seconds, the broker server determines that the application server on which the AP_2 is running is heavily congested. Then, the broker server changes from normal mode to reject mode. In this model, the reject mode expires in 30 seconds and the broker server changes from reject mode to normal mode. In reject mode, all requests are rejected by the broker server.

The broker server maintains the status that indicates which phase the request message is processed in. In this model, when a timeout occurs, the broker server verifies the requests that start a process but are not completed. Then, the broker server sends the rollback request messages to the application servers on which application AP_1 or AP_3 are running. These application servers respond to the broker server when the rollback processes are complete.

244

(11)APNOM 2005

Numerical results• Comparison of total service performance• Trade-off between the number of service abortions and service

completion time

○: (a) without control scheme△: (b) only timeout configuration□: (c) reject mode is implemented×: average of (a)＋: average of (b)◇: average of (c)

Elapsed time of simulation (sec)

Serv

ice

com

plet

ion

time

(sec

)

Table of service performance

Service completion time

66..3.3. Numerical results Numerical results Here, we consider service completion time, percentage of completed service requests, aborted

service requests, and rejected service requests in reject mode with respect to all the requests. When there is no control scheme, the service completion time is longer than that of other schemes.

In particular, when the request-generating processes of several requesters are in ON mode, request messages are concentrated on the application server. Therefore, when the control scheme is not installed, the completion time is longer than that of the control scheme. When reject mode is implemented, the service completion time is the shortest; while the percentage of service completions without reject mode is the highest. When reject mode is implemented, the completion rate is about 80%.

There is a trade-off. If the service completion time has a higher priority than the service completion percentage in the SLA, the reject-mode scheme is useful to maintain the service level of the service completion time. The parameters of the reject mode configuration should be tuned using this evaluation method considering the required SLA on the integrated service completion time and the service completion percentage.

245

(12)APNOM 2005

Conclusion• We proposed methodology to evaluate performance

of integrated service system with timeout control scheme. – Network, application traffic, and workflow layer models

• Broker server invokes appropriate AP based on BP

– Performance measure• Focused on service completion time, percentages of service

completions, abortions, and rejected requests performed by the reject mode control.

– Strongly dependent on specifications of above three layer models.

• Future work– Develop message-queuing communication– Adoption of “Co-simulation” technology

77. . ConclusionConclusionWe proposed a method based on the simulation to evaluate the performance of application

integration systems with the timeout control scheme. In this method, network, application traffic, and workflow layer models are needed. The total service completion time is as important as the individual application response time as a performance measure. The service completion time is strongly dependent on the above three layer models. In the proposed simulation method, the three layer models are simulated concurrently. We expanded the developed simulation module and added the timeout and rollback schemes.

The service integration architecture will be changed to the ESB (enterprise service bus) architecture using a message queuing communication scheme. We will develop simulation modules to simulate message queuing communication. In the developed method, the three layer models are simulated on the same computer. To evaluate large and complicated workflows, it may be useful to perform network simulations and workflow simulations on different computers. The application of the co-simulation technology is for further study.

ReferencesReferences[1] D. A. Chappell, “Enterprise Service Bus”, O’reilly, June 2004.[2] H. Erickson and M. Penker, “Business Modeling with UML”, John Wiley & Sons, Inc., January 2000. [3] A. Korthaus and S. Kuhlins, “BOOSTER process – a software development process model integrating business object technology and UML”, The First International Workshop on the Unified Modeling Language UML’98, Computer Science 1618, pp.215-226, Springer-Verlag London, 1998.[4] M. Aoyama, “A Business-Driven web service creation methodology”, Proceedings of the International Workshop on Web Services Engineering, pp.255-228, February 2002.[5] R. Pooley and P. King, 1999. “The Unified Modeling Language and Performance Engineering”, IEEE Proceedings on Software, Vol.146, No.1, pp.2-10, February 1999. [6] S. Balsamo and M. Marzolla, 2003. “A simulation-based approach to software performance modeling”, Proceedings of the 9th European Software Engineering Conference, pp. 363-366, AMC Press NY, 2003. [7] Microsoft BizTalk Server, http://www.microsoft.com/biztalk/.[8] IBM WebSphere, http://www.ibm.com/software/websphere/.[9] BEA WebLogic, http://www.bea.com/content/products/weblogic/.[10] OASIS, http://www.oasis-open.org/.[11] H. Yamada and A. Kawaguchi, “A method for the performance analysis of integrated application service”, Proceedings of the ICETE2004, pp275-280, August 2004.[12] OPNET, http://www.opnet.com/.

246