06636737

8/19/2019 06636737

http://slidepdf.com/reader/full/06636737 1/6

An Approach to Carry Out Consistency Analysis on

RequirementsValidating and Tracking Requirements through a Configuration Structure

Padmalata Nistala

Business Systems and Cybernetics Center

Tata Consultancy Services

Hyderabad, India

[email protected]

Priyanka Kumari

Business Systems and Cybernetics Center

Tata Consultancy Services

Hyderabad, India

[email protected]

Abstract — Requirements management and traceability have

always been one of grand challenges in software development

area. Studies reveal that 30- 40% of software defects can be

traced to gaps or errors in requirements Although several models

and techniques have been defined to optimize the requirements

process, ensuring alignment and consistency of elicited

requirements continues to be a challenge. All software

engineering standards and methodologies recognize the

importance of maintaining relationships among the software

elements for traceability. We have leveraged the structured

relationships among the requirement elements to come up with

an approach to systematically carry out consistency analysis of

requirements for software systems.

The framework has multiple models: a multi layered

requirement model, a configuration structure to link and track

the requirement items, a consistency analysis method to identify

the inconsistencies in the requirements and a consistency index

computation to indicate the level of consistency in overall

requirements of the software system. This approach helps to

validate the requirements from both completeness and

correctness perspectives and also check their consistency in

forward and backward directions. The paper outlines the

framework, describes the encompassing models and theimplementation details from pilot of the framework to an

industry case study along with results.

Index Terms — Requirement alignment, requirement

configuration structure, requirement consistency analysis,

requirement inconsistency matrix, requirement consistency index

I.

I NTRODUCTION

According to ISO/IEC 15288 system‟s definition [1], “A

system is a combination of interacting elements organized to

achieve one or more stated purposes”. As the software system

transforms from a set of needs to a final product through

various forms, at each stage there is a high possibility of loss of

quality due to defects in transformation of configuration items

from previous phase. Thus, an understanding of system model

[1] is important not only to understand “what” is being

developed but also to understand “how” it is being developed.

Though a lot of traceability tools and research is trying to

overcome the problem in software industry, managing the

requirements to perform traceability continues to be a primary

issue. Kannenberg and Saiedian in their article [2] reflects the

problem of requirement source traceability without which it is

difficult to determine the incorporation of requirement into

system and thus unable to trace whether system [3] is being

developed according to the requirement or not.

J. Dick [4] suggested solution for the stated problem by

maintaining requirements linking from one level to another

which helps in impact analysis and derivation of a need.

Management and traceability of the requirements should help

in achieving the product to meets the stated requirements. The

concept of traceability [5] is not only limited to tracing therequirement but also on the alignment of customers goals and

requirements to system/software specification which will help

in achieving the stated objective. Tomoyuki [7] has proposed

traceability concept to maintain and demonstrate the

relationships between requirements and software artifacts such

as design models, source code, and test cases.

Standards like SWEBOK [7], ISO/IEC [1], PDTR 30103[8]

have defined the importance of relationships between different

configuration items but they have not defined any

layout/framework on achieving traceability or alignment

through structured relationship between configuration items.

Few book on requirements engineering [3] focuses on how

high level requirements getting transformed into low-levelrequirements depending upon the relationships between layers

of information. The established Deming‟s principle of process

step and product quality [9] also supports correlation between

process step and product quality. These established concepts

requires to creating a relational structure to check the quality at

each process step. Though these literatures emphasize on some

kind of relationship but they do not exactly focus defining the

structure and related necessary granularity while forming the

structure. They lack a robust mechanism for creating a

structured relationship between the artifacts.

In other disciplines such as manufacturing and systems

engineering we can find models, well established principles

and practices for formulating and tracing the product parts andcomposition. The modern architecture strongly believes in

strong relationship between form or structure [10, 11] of the

product to the purpose or function. Similarly, the concept of

Product Breakdown Structure (PBS) [12, 13, 14] in

manufacturing industry is the breakdown of a product into its

required components. This helps in providing a clear picture of

the product and helps in handling the complex parts with easy

978-1-4673-5765-4/13/$31.00 c 2013 IEEE RE 2013, Rio de Janeiro, Brasil

Industry Track

320

8/19/2019 06636737


visuals prints of the components. The relationships and

traceability of product parts is visually transparent.

Various traceability tools presently used in software

industry [3,15] maintain document level linkages focusing on

completeness. Few other tools [15, 16] support some kind of

relation but do not have the adequate granularity and holistic

coverage of the relations.

The approach of the proposed framework is to perform

consistency analysis to correctly and completely track the flow

of requirements with various layers. Here we propose to

leverage the relationships among the requirements

configuration items and propose a model for requirements

alignment and traceability. The research problems taken are

To define guidelines for identification of requirement

elements or configuration items for software systems

To define a framework for requirements tracking through a

requirement configuration structure using the relationships

among requirement configuration items

To conduct consistency analysis on the overall system in

terms of completeness and correctness and identify the

gaps at each phase

To apply this framework for an industry case of softwaresystem and illustrate the applicability of the framework

and the no of inconsistencies in requirements found

through this model.

In the subsequent sections, we describe the proposed

framework - the requirement configuration structure, its

application for requirements traceability of a software

application in a vertical business unit, the result achieved,

followed by the conclusion.

II. FRAMEWORK

Here in this framework, the complete value chain of

software requirements process is considered in the form of key

configuration layers. The complete view of framework with its

encompassing layers, configuration items and their structural

relationships is depicted in Fig 1.

The key components of the framework are:

a multi layered requirement model to ensure the

alignment of requirements right from the business

goals to the software specifications. It comprises of

four key players: business layer, process layer,

requirements layer and specification layer.

a configuration structure to link and track the

requirement items for each layer. This provides a

visual representation of the requirements flow across

various layers. a consistency analysis method to identify the

inconsistencies in the requirements structure. A

configuration inconsistency matrix is populated with

missing linkages of requirements in both forward and

backward directions.

a consistency index computation to indicate the level of

consistency in overall requirements of the software

system.

A.

Layers of the Model

In order to realize the value from the software systems, the

context and business perspective has to be integrated to the

software requirements [17]. Also, to address the leakage issues/

gaps at requirement phase, we initiated the requirement

configuration with the business layer. The key configuration

layers in the framework are FOUR.

1) Business Layer: Here the broad business purpose of the

software system as outlined by the customer is captured in the

form of business goals and objectives. This layer brings the big

picture of the business - the purpose of the software system and

is typically stated by the customer in the RFP (request for

proposal).

2)

Process Layer: The key business processes to realize the

goals and objectives are also identified here. This is a crucial

layer and it is required to identify the end to end chain of

business processes to realize the business goals and objectives.

3)

Requirements Layer: This layer identifies the key

business requirements of the system and is typically the starting

point for many software systems.

Figure 1. Requirement Configuration Structure Framework

321

8/19/2019 06636737


4)

Requirement Specification Layer: This layer describes

the analyzed requirements in the form of specifications. The

requirement specification could be in any notation depending

on the life cycle process e.g. SRS for structured model; Use

cases for OO model and User stories for agile model.

Each layer will have a set of defined configuration items. As

a guideline, we have listed the key requirement configuration

items widely followed in software development.

Business Layer: Business Goals and/or Objectives. Process Layer: Business Process (Id + Name) and Sub

Process.

Requirement Layer: Business Requirements

(Requirements group + Requirements Id + Requirements

Description).

Specification Layer: Use Case (Id + Name).

Depending on the context of the client, the life cycle model

and its architecture, the individual configuration items can be

tailored within each layer.

B. Configuration Structure

This composition of requirement configuration items for

each of these layers along with their relationships helps in

creating the requirement configuration structure (RCS). It

provides guidance in identifying and linking configuration/

components item at the business, process and requirement

layers. Fig-1 depicts the layout of the model “Requirement

Configuration Structure”.

The configuration items at one layer are linked to

corresponding items at another layer to create requirement

structure of the software product/application under

development. The framework suggests compartmentalizing

requirements to the processes it serves and similarly to all other

elements of the configuration item.

C. Consistency Analysis Method

The consistency or track of each relationship from theabove created structure is validated on two dimensions. The

relationship consistency analysis involves:

Completeness analysis – Completeness property is “the

degree to which the set of functions covers all the specified

tasks and user objectives” [18]. Completeness analysis of

RCS involves checking whether for each configuration

item in a layer, are there corresponding configuration

item(s) in forward and backward directions and overall

whether the coverage of the configuration items across the

structure is complete. This analysis helps to establish the

end to end track of requirements in RCS.

Correctness analysis – Correctness property is “the degree

to which a system or component is free from faults in itsspecification, design, and implementation” [19].

Correctness analysis of RCS involves checking for faults

or errors in transformation between any two configuration

items in the structure. It validates for each relationship of

RCS, do the next configuration item correctly achieves the

objectives of the previous configuration item or not. This

analysis is to validate the correctness of requirement items

across layers in RCS.

ISO/IEC/IEEE 24765 defines consistency as “the degree

of uniformity, standardization, and freedom from

contradiction among the documents or parts of a system or

component”. In our framework, we extend this definition

and propose a requirement to be “consistent” when both

completeness and correctness properties across its path in

the structure are true and maintained. The requirement is

“inconsistent” if one or more gaps found in RCS either due

to completeness or correctness issues.The inconsistencies found in analysis are populated into a

configuration inconsistency matrix (CIM). The template of

CIM is given in Table-1. X and Y denote the configuration

items among which there is an inconsistency in relationship;

the direction is forward/backward; inconsistency type is

“Missing” for a completeness gap or “Erroneous” for a

correctness gap; analysis describes the gap and count is

summation of the inconsistent configuration items.

TABLE 1. Configuration Inconsistency Matrix

Process Process for which the related inconsistency exist

Config Item XX is a configuration item in the requirement

configuration structure

Config Item YY is a configuration item in the requirement

configuration structure

Direction Relationship: Forward/ Backward

Inconsistency TypeType of relationship inconsistency:

Missing/ Erroneous

Analysis Analysis description

Count No of inconsistencies in the relationship

CIM provides a tool to consolidate all the inconsistencies in

the structure defined above and analyse the data. The numbers

of inconsistencies as found from the CIM are plotted againstthe business process as “inconsistency graphs”. This

information readily gives indication on the completeness and

correctness of requirements realization and gaps in

transformation at each layer. These graphs visually depict the

inconsistency in quality for the requirements.

D.

Requirement Consistency Index

Requirement Consistency Index (RCI) quantifies the

percentage consistency of the requirements. In the consistency

analysis, a requirement is considered “consistent” when both

completeness and correctness across its path in the structure are

maintained and “inconsistent” if one or more gaps found in

analysis. As an initial measurement, RCI considers each

consistency issue with equally weightage resulting in anaverage value for the metric.

Requirement Consistency Index = Percentage consistency

of the requirements.

RCI =A/(B+C)

where,A =Number of Consistent Requirements.

This is computed as ∑Xi where i denote a requirement

varying from 1 to n; Xi is consistency value of the

322

8/19/2019 06636737


Figure 2. Requirement configuration structure for the project

requirements.

B = Total number of elicited requirementsC = Number of missing requirements identified

III. CASE STUDY

This framework has been piloted for a critical business

system for a large insurance client of TCS. The framework was

implemented to carry out the due diligence exercise for

requirements, find out the inconsistencies and validate the

alignment and traceability of elicited requirements.The actual implementation of this requirement consistency

analysis was carried by two researchers who were also

involved in designing the framework with the knowledge help

of business analyst. The implementation period was two

months which resulted in the formation of requirement

configuration structure and the consistency index which were

also validated by the SMEs.

The pilot application was an “eRegistration System” which

provides different methods of automatically registering external

customers and maintaining and accessing their accounts. The

existing registration process is enhanced with implementation

of security framework for few processes. The new application

also integrates the Management Information (MI) reportcreation. The following section details the framework

deployment for the case.

A.

Layers of the Framework

The first step is identifying the layers and configuration

items. The framework implementation for the registration

system is illustrated in Fig 2 and described below for each layer.

1)

Business Layer: The key configuration items in this

layer are business objectives. The inputs for this layer have

primarily been taken from the delivery proposal report, process

diagrams/flow charts. Customer has identified three high level

business objectives: customer should be automatically

registered though automatic registration and enhancing the

security framework for the existing registration service and to

enhance the interface of registration service with other

connected application.

2)

Process Layer: The above objectives were transformed

into seven end to end business processes (P1 to P7). Automatic

registration while customer is buying a product; registering the

employee of the organization where the insurance client have

enrolled their insurance products; registering customer through

internal staffs; enhancing registration process with the security

framework and password reset functionality and providing

management information are few of the identified business

processes. Sub-process: Sub-processes are identified by the

decomposition of the main processes.

3) Requirements Layer: Requirements layer has been

developed with the business system analyst group. The above

business processes (P1-P7) were linked to applicationrequirements taken from requirements catalogue which is a

client provided artifact. The requirement is defined with

specific requirement group and ID.

4) Requirement Specification Layer: In this layer, use case

documents were provided by customer. Configuration items

(Use Case) for this layer were properly linked to the

corresponding process and requirements.

323

8/19/2019 06636737


Table 2: Configuration Inconsistency Matrix for the project

Process C.Item X C.Item Y Direction Type Analysis Count

P2:Bulk

Registration

Process2-1: Bulk Registration

for product customer

Requirements Forward Missing 1)

Requirements missing regarding security

admin tool main menu for Bulk

Registration as part of tool.2)

Requirements missing regarding bulk

registration file upload.

1

P2:BulkRegistration

Process2-1: Bulk Registrationfor product customer

Requirements Forward Missing 1)

Requirements not adequately definedregarding generation of activation code.

1

P2:BulkRegistration

Requirements:9 Use Case Forward Missing 1)

Use Case missing for confirmation andgeneration of activation code, bulkregister file upload, output generation.

1

P3:Consumer

Self Elected

Registration

Requirements Use Case: Self

Registration

Backward Missing 1)

Password creation and confirnation

requirements missing for self registration

1

P5:Login and

manage account

Process5-1: Consumer Login/

Logout

Requirements Forward Erroneous 1)

Requirements errorneous in nature –

requirements states that customer has to

answer two security question whereascustomer has to asnwer three security

questions.

1

B.

Configuration Structure

The above figure-2, illustrates the requirement

configuration structure (RCS) created for „eRegistration

System” linking all the configuration items, taking an

illustration of “bulk registration” process to fulfill the objective

- “to provide automatic registration facility to the customer”.

The process is divided into two sub-processes “bulk

registration for product customer” and “bulk registration for

workplace customer” All the requirements and use cases are

correctly mapped to these sub-processes as shown in figure-2.

The first row in figure shows the missing use case when going

“forward” from requirements to specification layer. Thus, it

identifies the completeness issue with use case.

Similarly, the sub-process (P2-1) has a requirement on

generation of action code but the listed requirements are

insufficient and do not adequately satisfy the objectives of

activation code generation process. This is a “correctness”related problem with requirements.

C.

Consistency Analysis Method

Based on the RCS structure created, each relationship is

analyzed for consistency in terms of completeness and

correctness as described in the framework. The inconsistencies

found in RCS are logged into configuration inconsistency

matrix (CIM). The table-2 shows a view of the configuration

inconsistency matrix (CIM). The analysis is carried out through

tracking the flow and validating the information. The

inconsistencies listed in CIM have been validated with the

SMEs.

D.

Requirement Consistency Index

Figure-3 depicts the summary of the requirement

inconsistencies as analyzed from configuration inconsistency

matrix (CIM). It also segregates the data based on correctness

and completeness related problems. It can be seen from the

graph that the number of inconsistencies originated from

requirements are very high.

Figure 3. Requirement Inconsistency Graph

The identified seven processes have severe gaps in realizingthe objectives of the eRegistration software system.

Requirements for key processes like bulk registration and auto

registration were completely missing.

Similarly many use cases were missing and are not

specified.

Total no. of requirements for the case is 50 and the No of

consistent requirements 24. Applying the formula, the

requirement consistency index was calculated as 48%.

TABLE 3: Requirement Inconsistency Index for the project

Requirement Consistency Index (RCI) 48%

The low value of index readily gives an indication on the

quality of the overall requirements and the leakage issues

within requirements. It validates the statistics that the

requirements are poorly managed in software industry. The

gaps in requirements are hardest to find but could be unearthed

in this framework through establishment of linkages among the

requirement elements.

324

8/19/2019 06636737


The deployment of this framework helped in identifying

many configuration items which were missing and required for

realizing the business needs into software system.

IV. CONCLUSION

The case study results show that significant requirement

inconsistencies can be found through the requirement

configuration structure framework. Requirement consistency

analysis brings out not only the “missing configuration items” from completeness analysis but also “erroneous configuration

items” in the RCS from the correctness analysis. Both forward

and backward traceability for requirements are covered. The

consistency track of complete requirements value chain from

business objectives, business processes and requirements until

the use cases can be maintained, validated and analyzed

through this framework. Computation of Requirement

consistency index for overall requirements provides a

quantifiable measure on the consistency of requirements and

acts as a requirement quality indicator. This method will

benefit the projects to have a structured approach towards

requirements management through the configuration structure

and can go a long way in containing the requirement defects

and improving phase containment.Future work involves considering the weightage of

requirements into metrics computation, further variations in

requirement scenarios such as change scenarios, reverse

engineering scenarios etc. There are plans to customize the

internal tooling platform include computation of these metrics

and implement for multiple projects.

ACKNOWLEDMENT

The authors would like to acknowledge the management of

Tata Consultancy Services for providing an opportunity to

design and validate the framework. We would like to thank Mr

MGPL Narayana, Vice President, TCS for his encouragement

to write this paper, Mrs Lakshmi Suchetha for showing theinterest for the implementation of the framework.

R EFERENCES

[1] “Systems Engineering Guide for ISO/IEC (System Life Cycle

Processes) 15288” , July 2002 .

[2] A. Kannenberg, H. Saiedian, “Why Software Requirements

Traceability Remains a Challenge,” 14 Crosstalk - Journal ofDefense Software Engineering, July/August 2009,www.stsc.hill.af.mil

[3] E. Hull, K. Jackson, J. Dick, “ Requirements Engineering”,2nd

ed., Springer, 2009, ch-9

[4] J. Dick, “Rich Traceability”, Proc. 1st International Workshop

on Requirements Traceability, Edinburgh, Sept 2002

[5] J. Cleland-Huang, O. Gotel, A. Zisman, “Software and Systems

Traceability” 1st ed., Springer, 2012

[6] T. Arao, E. Goto, T. Nagata, “Business Process Oriented

Requirements Engineering Process” Proc. 13th IEEEInternational Conference on Requirements Engineering (RE‟05),IEEE computer society, Dec. 2005,

[7]

IEEE Standard Committee, “Guide to the Software EngineeringBody of Knowledge (SWEBOK)”, ver 3,IEEE ComputerSociety, 2004

[8] ISO/IEC PDTR 30103: Software and systems engineering –

Lifecycle processes – Framework for product qualityachievement

[9] William W Scherkenbach, „Deming‟s Road to continual

improvement‟, SPC Press Inc., Knoxville, Tennessee (615 5845005) 1991, p26-38

[10] Louis H. Sullivan,“The Tall Office Building Artistically

Considered", Lippincott's Magazine, ver 57, March 1896

[11] S. Dionidis, "A Tale of Four Kernels", Proc. 30th International

Conference on Software Engineering (ICSE „08). Leipzig,Germany: Association for Computing Machinery, May 2008,

pp. 381 – 390

[12]

D. Svensson and J. Malmqvist, “Strategies for Product StructureManagement at Manufacturing Firms”. Journal of Computingand Information Science in Engineering, 2002, pp-50-58

[13] Product Breakdown Structure :

http://productbreakdownstructure.com/

[14] Product Breakdown Structure

http://www.professionalpractice.asme.org/ProductMgmt/Systems/Product_Breakdown_Structure.cfm

[15] A. Castor, R. Pinto, C. Silva, and J. Castro,“Towards

Requirement Traceability in TROPOS” , Centro de Informática,Universidade Federal de Pernambuco, Brazil , 2004

[16] P. Zielczynski, “Requirements Management Using IBM

Rational RequisitePro”, IBM press , 2007

[17] Padmalata Nistala, Supriya Kummamuru & MGPL Narayana“An Approach to Understand and Elicit Requirements UsingSystemic Models: Ensuring a Connect From Problem Context to

Requirements” CSER 2013,Conference on Systems Engineering2013, Georgia Institute of Technology, Atlanta, Mar 2013

[18] ISO/IEC 25010:2011 Systems and software engineering--

Systems and software Quality Requirements and Evaluation(SQuaRE)--System and software quality models ,

[19] ISO/IEC/IEEE 24765:2010 Systems and software engineering--

Vocabulary

325

Documents

06636737