View
219
Download
0
Category
Preview:
Citation preview
8/19/2019 06636737
http://slidepdf.com/reader/full/06636737 1/6
An Approach to Carry Out Consistency Analysis on
RequirementsValidating and Tracking Requirements through a Configuration Structure
Padmalata Nistala
Business Systems and Cybernetics Center
Tata Consultancy Services
Hyderabad, India
Nistala.padma@tcs.com
Priyanka Kumari
Business Systems and Cybernetics Center
Tata Consultancy Services
Hyderabad, India
Priyanka.kumari@tcs.com
Abstract — Requirements management and traceability have
always been one of grand challenges in software development
area. Studies reveal that 30- 40% of software defects can be
traced to gaps or errors in requirements Although several models
and techniques have been defined to optimize the requirements
process, ensuring alignment and consistency of elicited
requirements continues to be a challenge. All software
engineering standards and methodologies recognize the
importance of maintaining relationships among the software
elements for traceability. We have leveraged the structured
relationships among the requirement elements to come up with
an approach to systematically carry out consistency analysis of
requirements for software systems.
The framework has multiple models: a multi layered
requirement model, a configuration structure to link and track
the requirement items, a consistency analysis method to identify
the inconsistencies in the requirements and a consistency index
computation to indicate the level of consistency in overall
requirements of the software system. This approach helps to
validate the requirements from both completeness and
correctness perspectives and also check their consistency in
forward and backward directions. The paper outlines the
framework, describes the encompassing models and theimplementation details from pilot of the framework to an
industry case study along with results.
Index Terms — Requirement alignment, requirement
configuration structure, requirement consistency analysis,
requirement inconsistency matrix, requirement consistency index
I.
I NTRODUCTION
According to ISO/IEC 15288 system‟s definition [1], “A
system is a combination of interacting elements organized to
achieve one or more stated purposes”. As the software system
transforms from a set of needs to a final product through
various forms, at each stage there is a high possibility of loss of
quality due to defects in transformation of configuration items
from previous phase. Thus, an understanding of system model
[1] is important not only to understand “what” is being
developed but also to understand “how” it is being developed.
Though a lot of traceability tools and research is trying to
overcome the problem in software industry, managing the
requirements to perform traceability continues to be a primary
issue. Kannenberg and Saiedian in their article [2] reflects the
problem of requirement source traceability without which it is
difficult to determine the incorporation of requirement into
system and thus unable to trace whether system [3] is being
developed according to the requirement or not.
J. Dick [4] suggested solution for the stated problem by
maintaining requirements linking from one level to another
which helps in impact analysis and derivation of a need.
Management and traceability of the requirements should help
in achieving the product to meets the stated requirements. The
concept of traceability [5] is not only limited to tracing therequirement but also on the alignment of customers goals and
requirements to system/software specification which will help
in achieving the stated objective. Tomoyuki [7] has proposed
traceability concept to maintain and demonstrate the
relationships between requirements and software artifacts such
as design models, source code, and test cases.
Standards like SWEBOK [7], ISO/IEC [1], PDTR 30103[8]
have defined the importance of relationships between different
configuration items but they have not defined any
layout/framework on achieving traceability or alignment
through structured relationship between configuration items.
Few book on requirements engineering [3] focuses on how
high level requirements getting transformed into low-levelrequirements depending upon the relationships between layers
of information. The established Deming‟s principle of process
step and product quality [9] also supports correlation between
process step and product quality. These established concepts
requires to creating a relational structure to check the quality at
each process step. Though these literatures emphasize on some
kind of relationship but they do not exactly focus defining the
structure and related necessary granularity while forming the
structure. They lack a robust mechanism for creating a
structured relationship between the artifacts.
In other disciplines such as manufacturing and systems
engineering we can find models, well established principles
and practices for formulating and tracing the product parts andcomposition. The modern architecture strongly believes in
strong relationship between form or structure [10, 11] of the
product to the purpose or function. Similarly, the concept of
Product Breakdown Structure (PBS) [12, 13, 14] in
manufacturing industry is the breakdown of a product into its
required components. This helps in providing a clear picture of
the product and helps in handling the complex parts with easy
978-1-4673-5765-4/13/$31.00 c 2013 IEEE RE 2013, Rio de Janeiro, Brasil
Industry Track
320
8/19/2019 06636737
http://slidepdf.com/reader/full/06636737 2/6
visuals prints of the components. The relationships and
traceability of product parts is visually transparent.
Various traceability tools presently used in software
industry [3,15] maintain document level linkages focusing on
completeness. Few other tools [15, 16] support some kind of
relation but do not have the adequate granularity and holistic
coverage of the relations.
The approach of the proposed framework is to perform
consistency analysis to correctly and completely track the flow
of requirements with various layers. Here we propose to
leverage the relationships among the requirements
configuration items and propose a model for requirements
alignment and traceability. The research problems taken are
To define guidelines for identification of requirement
elements or configuration items for software systems
To define a framework for requirements tracking through a
requirement configuration structure using the relationships
among requirement configuration items
To conduct consistency analysis on the overall system in
terms of completeness and correctness and identify the
gaps at each phase
To apply this framework for an industry case of softwaresystem and illustrate the applicability of the framework
and the no of inconsistencies in requirements found
through this model.
In the subsequent sections, we describe the proposed
framework - the requirement configuration structure, its
application for requirements traceability of a software
application in a vertical business unit, the result achieved,
followed by the conclusion.
II. FRAMEWORK
Here in this framework, the complete value chain of
software requirements process is considered in the form of key
configuration layers. The complete view of framework with its
encompassing layers, configuration items and their structural
relationships is depicted in Fig 1.
The key components of the framework are:
a multi layered requirement model to ensure the
alignment of requirements right from the business
goals to the software specifications. It comprises of
four key players: business layer, process layer,
requirements layer and specification layer.
a configuration structure to link and track the
requirement items for each layer. This provides a
visual representation of the requirements flow across
various layers. a consistency analysis method to identify the
inconsistencies in the requirements structure. A
configuration inconsistency matrix is populated with
missing linkages of requirements in both forward and
backward directions.
a consistency index computation to indicate the level of
consistency in overall requirements of the software
system.
A.
Layers of the Model
In order to realize the value from the software systems, the
context and business perspective has to be integrated to the
software requirements [17]. Also, to address the leakage issues/
gaps at requirement phase, we initiated the requirement
configuration with the business layer. The key configuration
layers in the framework are FOUR.
1) Business Layer: Here the broad business purpose of the
software system as outlined by the customer is captured in the
form of business goals and objectives. This layer brings the big
picture of the business - the purpose of the software system and
is typically stated by the customer in the RFP (request for
proposal).
2)
Process Layer: The key business processes to realize the
goals and objectives are also identified here. This is a crucial
layer and it is required to identify the end to end chain of
business processes to realize the business goals and objectives.
3)
Requirements Layer: This layer identifies the key
business requirements of the system and is typically the starting
point for many software systems.
Figure 1. Requirement Configuration Structure Framework
321
8/19/2019 06636737
http://slidepdf.com/reader/full/06636737 3/6
4)
Requirement Specification Layer: This layer describes
the analyzed requirements in the form of specifications. The
requirement specification could be in any notation depending
on the life cycle process e.g. SRS for structured model; Use
cases for OO model and User stories for agile model.
Each layer will have a set of defined configuration items. As
a guideline, we have listed the key requirement configuration
items widely followed in software development.
Business Layer: Business Goals and/or Objectives. Process Layer: Business Process (Id + Name) and Sub
Process.
Requirement Layer: Business Requirements
(Requirements group + Requirements Id + Requirements
Description).
Specification Layer: Use Case (Id + Name).
Depending on the context of the client, the life cycle model
and its architecture, the individual configuration items can be
tailored within each layer.
B. Configuration Structure
This composition of requirement configuration items for
each of these layers along with their relationships helps in
creating the requirement configuration structure (RCS). It
provides guidance in identifying and linking configuration/
components item at the business, process and requirement
layers. Fig-1 depicts the layout of the model “Requirement
Configuration Structure”.
The configuration items at one layer are linked to
corresponding items at another layer to create requirement
structure of the software product/application under
development. The framework suggests compartmentalizing
requirements to the processes it serves and similarly to all other
elements of the configuration item.
C. Consistency Analysis Method
The consistency or track of each relationship from theabove created structure is validated on two dimensions. The
relationship consistency analysis involves:
Completeness analysis – Completeness property is “the
degree to which the set of functions covers all the specified
tasks and user objectives” [18]. Completeness analysis of
RCS involves checking whether for each configuration
item in a layer, are there corresponding configuration
item(s) in forward and backward directions and overall
whether the coverage of the configuration items across the
structure is complete. This analysis helps to establish the
end to end track of requirements in RCS.
Correctness analysis – Correctness property is “the degree
to which a system or component is free from faults in itsspecification, design, and implementation” [19].
Correctness analysis of RCS involves checking for faults
or errors in transformation between any two configuration
items in the structure. It validates for each relationship of
RCS, do the next configuration item correctly achieves the
objectives of the previous configuration item or not. This
analysis is to validate the correctness of requirement items
across layers in RCS.
ISO/IEC/IEEE 24765 defines consistency as “the degree
of uniformity, standardization, and freedom from
contradiction among the documents or parts of a system or
component”. In our framework, we extend this definition
and propose a requirement to be “consistent” when both
completeness and correctness properties across its path in
the structure are true and maintained. The requirement is
“inconsistent” if one or more gaps found in RCS either due
to completeness or correctness issues.The inconsistencies found in analysis are populated into a
configuration inconsistency matrix (CIM). The template of
CIM is given in Table-1. X and Y denote the configuration
items among which there is an inconsistency in relationship;
the direction is forward/backward; inconsistency type is
“Missing” for a completeness gap or “Erroneous” for a
correctness gap; analysis describes the gap and count is
summation of the inconsistent configuration items.
TABLE 1. Configuration Inconsistency Matrix
Process Process for which the related inconsistency exist
Config Item XX is a configuration item in the requirement
configuration structure
Config Item YY is a configuration item in the requirement
configuration structure
Direction Relationship: Forward/ Backward
Inconsistency TypeType of relationship inconsistency:
Missing/ Erroneous
Analysis Analysis description
Count No of inconsistencies in the relationship
CIM provides a tool to consolidate all the inconsistencies in
the structure defined above and analyse the data. The numbers
of inconsistencies as found from the CIM are plotted againstthe business process as “inconsistency graphs”. This
information readily gives indication on the completeness and
correctness of requirements realization and gaps in
transformation at each layer. These graphs visually depict the
inconsistency in quality for the requirements.
D.
Requirement Consistency Index
Requirement Consistency Index (RCI) quantifies the
percentage consistency of the requirements. In the consistency
analysis, a requirement is considered “consistent” when both
completeness and correctness across its path in the structure are
maintained and “inconsistent” if one or more gaps found in
analysis. As an initial measurement, RCI considers each
consistency issue with equally weightage resulting in anaverage value for the metric.
Requirement Consistency Index = Percentage consistency
of the requirements.
RCI =A/(B+C)
where,A =Number of Consistent Requirements.
This is computed as ∑Xi where i denote a requirement
varying from 1 to n; Xi is consistency value of the
322
8/19/2019 06636737
http://slidepdf.com/reader/full/06636737 4/6
Figure 2. Requirement configuration structure for the project
requirements.
B = Total number of elicited requirementsC = Number of missing requirements identified
III. CASE STUDY
This framework has been piloted for a critical business
system for a large insurance client of TCS. The framework was
implemented to carry out the due diligence exercise for
requirements, find out the inconsistencies and validate the
alignment and traceability of elicited requirements.The actual implementation of this requirement consistency
analysis was carried by two researchers who were also
involved in designing the framework with the knowledge help
of business analyst. The implementation period was two
months which resulted in the formation of requirement
configuration structure and the consistency index which were
also validated by the SMEs.
The pilot application was an “eRegistration System” which
provides different methods of automatically registering external
customers and maintaining and accessing their accounts. The
existing registration process is enhanced with implementation
of security framework for few processes. The new application
also integrates the Management Information (MI) reportcreation. The following section details the framework
deployment for the case.
A.
Layers of the Framework
The first step is identifying the layers and configuration
items. The framework implementation for the registration
system is illustrated in Fig 2 and described below for each layer.
1)
Business Layer: The key configuration items in this
layer are business objectives. The inputs for this layer have
primarily been taken from the delivery proposal report, process
diagrams/flow charts. Customer has identified three high level
business objectives: customer should be automatically
registered though automatic registration and enhancing the
security framework for the existing registration service and to
enhance the interface of registration service with other
connected application.
2)
Process Layer: The above objectives were transformed
into seven end to end business processes (P1 to P7). Automatic
registration while customer is buying a product; registering the
employee of the organization where the insurance client have
enrolled their insurance products; registering customer through
internal staffs; enhancing registration process with the security
framework and password reset functionality and providing
management information are few of the identified business
processes. Sub-process: Sub-processes are identified by the
decomposition of the main processes.
3) Requirements Layer: Requirements layer has been
developed with the business system analyst group. The above
business processes (P1-P7) were linked to applicationrequirements taken from requirements catalogue which is a
client provided artifact. The requirement is defined with
specific requirement group and ID.
4) Requirement Specification Layer: In this layer, use case
documents were provided by customer. Configuration items
(Use Case) for this layer were properly linked to the
corresponding process and requirements.
323
8/19/2019 06636737
http://slidepdf.com/reader/full/06636737 5/6
Table 2: Configuration Inconsistency Matrix for the project
Process C.Item X C.Item Y Direction Type Analysis Count
P2:Bulk
Registration
Process2-1: Bulk Registration
for product customer
Requirements Forward Missing 1)
Requirements missing regarding security
admin tool main menu for Bulk
Registration as part of tool.2)
Requirements missing regarding bulk
registration file upload.
1
P2:BulkRegistration
Process2-1: Bulk Registrationfor product customer
Requirements Forward Missing 1)
Requirements not adequately definedregarding generation of activation code.
1
P2:BulkRegistration
Requirements:9 Use Case Forward Missing 1)
Use Case missing for confirmation andgeneration of activation code, bulkregister file upload, output generation.
1
P3:Consumer
Self Elected
Registration
Requirements Use Case: Self
Registration
Backward Missing 1)
Password creation and confirnation
requirements missing for self registration
1
P5:Login and
manage account
Process5-1: Consumer Login/
Logout
Requirements Forward Erroneous 1)
Requirements errorneous in nature –
requirements states that customer has to
answer two security question whereascustomer has to asnwer three security
questions.
1
B.
Configuration Structure
The above figure-2, illustrates the requirement
configuration structure (RCS) created for „eRegistration
System” linking all the configuration items, taking an
illustration of “bulk registration” process to fulfill the objective
- “to provide automatic registration facility to the customer”.
The process is divided into two sub-processes “bulk
registration for product customer” and “bulk registration for
workplace customer” All the requirements and use cases are
correctly mapped to these sub-processes as shown in figure-2.
The first row in figure shows the missing use case when going
“forward” from requirements to specification layer. Thus, it
identifies the completeness issue with use case.
Similarly, the sub-process (P2-1) has a requirement on
generation of action code but the listed requirements are
insufficient and do not adequately satisfy the objectives of
activation code generation process. This is a “correctness”related problem with requirements.
C.
Consistency Analysis Method
Based on the RCS structure created, each relationship is
analyzed for consistency in terms of completeness and
correctness as described in the framework. The inconsistencies
found in RCS are logged into configuration inconsistency
matrix (CIM). The table-2 shows a view of the configuration
inconsistency matrix (CIM). The analysis is carried out through
tracking the flow and validating the information. The
inconsistencies listed in CIM have been validated with the
SMEs.
D.
Requirement Consistency Index
Figure-3 depicts the summary of the requirement
inconsistencies as analyzed from configuration inconsistency
matrix (CIM). It also segregates the data based on correctness
and completeness related problems. It can be seen from the
graph that the number of inconsistencies originated from
requirements are very high.
Figure 3. Requirement Inconsistency Graph
The identified seven processes have severe gaps in realizingthe objectives of the eRegistration software system.
Requirements for key processes like bulk registration and auto
registration were completely missing.
Similarly many use cases were missing and are not
specified.
Total no. of requirements for the case is 50 and the No of
consistent requirements 24. Applying the formula, the
requirement consistency index was calculated as 48%.
TABLE 3: Requirement Inconsistency Index for the project
Requirement Consistency Index (RCI) 48%
The low value of index readily gives an indication on the
quality of the overall requirements and the leakage issues
within requirements. It validates the statistics that the
requirements are poorly managed in software industry. The
gaps in requirements are hardest to find but could be unearthed
in this framework through establishment of linkages among the
requirement elements.
324
8/19/2019 06636737
http://slidepdf.com/reader/full/06636737 6/6
The deployment of this framework helped in identifying
many configuration items which were missing and required for
realizing the business needs into software system.
IV. CONCLUSION
The case study results show that significant requirement
inconsistencies can be found through the requirement
configuration structure framework. Requirement consistency
analysis brings out not only the “missing configuration items” from completeness analysis but also “erroneous configuration
items” in the RCS from the correctness analysis. Both forward
and backward traceability for requirements are covered. The
consistency track of complete requirements value chain from
business objectives, business processes and requirements until
the use cases can be maintained, validated and analyzed
through this framework. Computation of Requirement
consistency index for overall requirements provides a
quantifiable measure on the consistency of requirements and
acts as a requirement quality indicator. This method will
benefit the projects to have a structured approach towards
requirements management through the configuration structure
and can go a long way in containing the requirement defects
and improving phase containment.Future work involves considering the weightage of
requirements into metrics computation, further variations in
requirement scenarios such as change scenarios, reverse
engineering scenarios etc. There are plans to customize the
internal tooling platform include computation of these metrics
and implement for multiple projects.
ACKNOWLEDMENT
The authors would like to acknowledge the management of
Tata Consultancy Services for providing an opportunity to
design and validate the framework. We would like to thank Mr
MGPL Narayana, Vice President, TCS for his encouragement
to write this paper, Mrs Lakshmi Suchetha for showing theinterest for the implementation of the framework.
R EFERENCES
[1] “Systems Engineering Guide for ISO/IEC (System Life Cycle
Processes) 15288” , July 2002 .
[2] A. Kannenberg, H. Saiedian, “Why Software Requirements
Traceability Remains a Challenge,” 14 Crosstalk - Journal ofDefense Software Engineering, July/August 2009,www.stsc.hill.af.mil
[3] E. Hull, K. Jackson, J. Dick, “ Requirements Engineering”,2nd
ed., Springer, 2009, ch-9
[4] J. Dick, “Rich Traceability”, Proc. 1st International Workshop
on Requirements Traceability, Edinburgh, Sept 2002
[5] J. Cleland-Huang, O. Gotel, A. Zisman, “Software and Systems
Traceability” 1st ed., Springer, 2012
[6] T. Arao, E. Goto, T. Nagata, “Business Process Oriented
Requirements Engineering Process” Proc. 13th IEEEInternational Conference on Requirements Engineering (RE‟05),IEEE computer society, Dec. 2005,
[7]
IEEE Standard Committee, “Guide to the Software EngineeringBody of Knowledge (SWEBOK)”, ver 3,IEEE ComputerSociety, 2004
[8] ISO/IEC PDTR 30103: Software and systems engineering –
Lifecycle processes – Framework for product qualityachievement
[9] William W Scherkenbach, „Deming‟s Road to continual
improvement‟, SPC Press Inc., Knoxville, Tennessee (615 5845005) 1991, p26-38
[10] Louis H. Sullivan,“The Tall Office Building Artistically
Considered", Lippincott's Magazine, ver 57, March 1896
[11] S. Dionidis, "A Tale of Four Kernels", Proc. 30th International
Conference on Software Engineering (ICSE „08). Leipzig,Germany: Association for Computing Machinery, May 2008,
pp. 381 – 390
[12]
D. Svensson and J. Malmqvist, “Strategies for Product StructureManagement at Manufacturing Firms”. Journal of Computingand Information Science in Engineering, 2002, pp-50-58
[13] Product Breakdown Structure :
http://productbreakdownstructure.com/
[14] Product Breakdown Structure
http://www.professionalpractice.asme.org/ProductMgmt/Systems/Product_Breakdown_Structure.cfm
[15] A. Castor, R. Pinto, C. Silva, and J. Castro,“Towards
Requirement Traceability in TROPOS” , Centro de Informática,Universidade Federal de Pernambuco, Brazil , 2004
[16] P. Zielczynski, “Requirements Management Using IBM
Rational RequisitePro”, IBM press , 2007
[17] Padmalata Nistala, Supriya Kummamuru & MGPL Narayana“An Approach to Understand and Elicit Requirements UsingSystemic Models: Ensuring a Connect From Problem Context to
Requirements” CSER 2013,Conference on Systems Engineering2013, Georgia Institute of Technology, Atlanta, Mar 2013
[18] ISO/IEC 25010:2011 Systems and software engineering--
Systems and software Quality Requirements and Evaluation(SQuaRE)--System and software quality models ,
[19] ISO/IEC/IEEE 24765:2010 Systems and software engineering--
Vocabulary
325
Recommended