boundaryandequivalnce-systematictestdesign-130223120931-phpapp02

8/13/2019 boundaryandequivalnce-systematictestdesign-130223120931-phpapp02

1/28

1

Black Box Test Techniques - Equivalence

Partitioning and Boundary Value Analysis

Author: Ian R McDonald

2010, 2013 Ian McDonald


2/28

2

Overview

In this presentation you will learn about Dynamic Testing Techniques.

This is where data is injected and the outputs monitored for the expected output.

The method is dynamic, since we have to supply data to the system under test.

We can design our test data with no understanding of the internal working of the

systemcalled a Black Box Technique.

Alternatively we may choose to structure our data based on a full knowledge of the

internal workingscalled a White Box Technique. Sometimes also known as ClearBox, Glass Box, Structural or Crystal techniques.

Another approach is to use our experience an judgement to guess what is the best

values to use for datacalled Error Guessing.

In this unit we will learn about 2 specific Black Box techniques to design tests. Theseare:

1. Equivalence Partitioning

2. Boundary Value Analysis


3/28

3

Objectives

By the end of this presentation you should be able to:

1. Understand the terms:

1. Dynamic Testing

2. Black Box

3. Equivalence Partitioning

4. Boundary Value Analysis

2. Understand when it is appropriate to use Black Box techniques

3. Be able to identify equivalence partitions within data.4. Be able to identify boundary values for a system under test.

5. Use material from this unit and reference it to BS7925-1 and BS7925-2


4/28

4

Dynamic Analysis

Dynamic Analysis is the process of evaluating a system or componentbased upon its behaviour during execution BS7925-1 adopted from theIEEE (Institute of Electrical and Electronics Engineers)

We use Dynamic Analysis techniques to define test inputs and matching

expected outputs to exercise a software component or a full system.

There are several Black Box techniques in designing data for testing:

Equivalence Partitioning

Boundary Value Analysis

Cause-effect Graphing (not covered here)

Other techniques e.g. Classification Tree Analysis (not covered here)

Note:The opposite of a Dynamic technique is a Static Technique. StaticTesting is the term applied in reviewing code and documents, where the

programme is not executed (i.e. the programme is in a static state).


5/28

5

Test Techniques

We apply test techniques to ensure:

1. To ensure a systematic approachto testingso that we use theminimum nu

mber of tests to achieve as much testing as possible in acost affective way. In life critical systems we want to ensure that wehave not missed out any specific values.

2. We want to ensure that tests are repeatable. We may have to justify to

our managers, customers an

d even a court that we tested a specific setof values.

3. We want to provide confidencethat we are applying a well structuredtesting approa

ch.

NOTE: We can use tools to help us in defining tests. E.g. theClassification Tree Approach used the Classification Tree Editor (CTE),which is covered in a separate presentation.Black box approaches to testing require no knowledge of the internalworking of an application. In contrast White box approaches do requireknowledge. White box approaches can sometimes be called clear box,transparent or crystal approaches.


6/28

6

Functional Test Case Design

A Test Case is a set of inputs, execution preconditions, and

expected outcomes developed for a particular objective, such as

to exercise a particular programme path or to verify compliance

with a specific requirement. BS7925-1

Functional Test Case Design, selects the test cases that is basedon an analysis of the specification of the component without

reference to the internal workings. BS7925-1

Black Box testing is the use of functional test case design, using

the requirements, component and system specifications and

other documentation describing what is required of the product

under development.


7/28

7

Black Box Testing Exercise

For this exercise, you are a

salesman in a television

shop. A customer comes

into your shop and asks

you to demonstrate that the

TV they intend to purchaseworks.

To see if the TV works, do

you need to know what

goes on inside the set?

What tests do you think you

will want to include?

How could we be

systematic in our test

approach?


8/28

8

Black Box Testing ExerciseSuggested Answers

There are a range of possible answers including:

Check standby light operates from off to standby, standby to onand on to standby.

Check the control panels to verify, volume, brightness, contrast,colour.

Check main terrestrial TV stations. Check Digital TV channels.

Check headphones socket functions.

Check other sockets for formateg can you plug a videorecorder into TV?

Notes: If checking Volume level, you may want a decibel meter and

specify the signal strength at the aerial input.

Would you want to check any ranges for each identified input?


9/28

9

Smoke Test

You may come across the term smoke test.

This was originally applied to civil engineering when smoke was put downa pipe and checks could be quickly made for more obvious leeks.

In hardware engineering, a quick way to check if all the power lines werecorrectly set up, electrolytic capacitors set up the right way, would be toswitch on the circuit and see if any smoke appears.

Smoke tests provide initial confidence, they are however no substitute fordetailed testing. Sometimes a test manager will do a smoke test beforeaccepting a build into detailed testing. This provides confidence that thebuild is at an appropriate state to enter system test.

The TV could be turned on and we can check for a picture of some sortand for sound. This does not mean that the sound matches the picture,or you can change channel.

Would you be happy with a medical system or an aircraft that had onlybeen smoke tested? Probably and most likely no. In the case of the TV,they will have been tested using automation before leaving the factory.


10/28

10

Black Box Testing

Black Box testing

Focuses on WHAT a system does not

HOW it does it.

Focuses on the functional capabilities of

the system

Is also known as functional testing.

Black BoxData input Actual Results

If the actual results are as expected then the test has passed


11/28

11

Where do we use Black Box Testing

Black Box testing is used in the following stages of the software

development life cycle.

Component Testing

Integration of Components (called integration in the small) Systems Testing

Systems Integration (called integration in the large)

User Acceptance Testing (UAT)

Assisting in the business verification and validation of the application

Operational Acceptance Testing (OAT)

In short you can use Black Box analysis at every phase. White box

testing however helps supplement the testing analysis for code

development and integration (in the small).


12/28

12

Equivalence Partitioning #1

It is very difficult, expensive and time

consuming if not at times impossible to test

every single input value combination for a

system. We can break our set of test cases into sub-

sets. We then choose representative values

for each subset and ensure that we test these.

Each subset of tests is thought of as a partition

between neighbouring subsets or domains.


13/28

13


Equivalence Partitioning:

Makes use of the principle that software acts in a

general way (generalises) in the way it deals with

subsets of data, Selects Groups of inputs that will be expected to be

handled in the same way.

Within a group of data, a representative input can

be selected for testing. For many professional testers this is fairly intuitive.

The approach formalises the technique allowing an

intuitive approach to become repeatable.


14/28

14


EP Example:

Consider a requirement for a software system:

The customer is eligible for a life assurancediscount if they are at least 18 and no older

than 56 years of age.

For the exercise only consider integer years.


15/28

15


The customer is eligible for a life assurancediscount if they are at least 18 and no older

than 56 years of age.

18 56

Invalid Partition

Less than or

equal to 18

Invalid Partition

Greater than 56

Valid Partition

Range 19 to 56


16/28

16


What if our developer incorrectly interpreted the requirement as:

The customer is eligible for a life assurance discount if they areover 18 and less than 56 years of age.

People aged exactly 18 or exactly 56 would now not get adiscount.

18 56

Invalid Partition

=< 18

Invalid Partition


17/28

17

Boundary Analysis #1

The customer is eligible for a life assurance discount if they areat least 18 and no older than 56 years of age.

17, 18, 19 55, 56, 57

Invalid Partition

Less than 18

Invalid Partition

Greater than 56

Valid Partition

Range 17 to 56

Boundaries

Test values would be: 17, 18, 19, 55, 56 and 57.

This assumes that we are dealing with integers and so least significant digit

is 1 either side of boundary.


18/28

18


For each boundary we test +/- 1 in theleast significant digit of either side of the

boundary.

Boundary Limit

Boundary Limit + 1Boundary Limit -1

If significant digit was second decimal place, then the

limits above would be +/- 0.01


19/28

19


While the textbooks may limit testing to theboundaries, we are interested in how softwarenormally behaves and how it reacts to handling errorconditions. Therefore it is normal to treat NOT ONLYthe boundaries but also:

A typical mid range value e.g. 37 Zero (since divide by 0 errors can occur).

Negative values

Numbers out of range by a long way e.g. +/-1000

Illegal data entries like nineteen as letters, Fred,banana. Illegal characters such as # $ & @ : ;


20/28

20

Taking EP and BVA Further

Consider the following requirement:

The customers must be at least 18. Customers are eligible for alife assurance discount of 40% if they are at least 18 and no older

than 25 years of age. Customers are entitled to a 30% discount if

they are older than 25 years of age, but under 40. Customers are

entitled to a 10% discount if they are 40 or over, but no older than

56. Over 56 customers are not entitled to a discount.

What are the equivalence partitions?

What are the boundary values to be tested? What other values might you test?


21/28

21

Taking EP and BVA Further -

Answer

The customers must be at least 18. Customers are eligible for a lifeassurance discount of 40% if they are at least 18 and no older than 25years of age. Age is only recorded in integer years. Customers areentitled to a 30% discount if they are older than 25 years of age, but under40. Customers are entitled to a 10% discount if they are 40 or over, but noolder than 56. Over 56 customers are not entitled to a discount.

17, 18, 19 24, 25, 26

40%

discount

38, 39, 40

10%

discount

55, 56, 57

10%

discount

0%

discountinvalid

Might also test: 0, -5, 200, Fred, 0.00000001, some typical mid

range values: 21, 32, 47. Note boundary values tested +/- least

significant recorded digit.


22/28

22

Invalid Partitions and ISEB / ISTQB Examination

Note when taking the ISEB / ISTQB

examinations, they may specifically

require that you identify the valid

partitions and not the invalid partitions.

Be aware of this when studying for the

examination and consult the syllabus for

specific instructions.


23/28

23

Question 1

One of the fields on a form contains a

text box which accepts numeric values in

the range of 18 to 26. Identify the invalid

Equivalence class.

a) 17

b) 19

c) 25d) 21


24/28

24

Solution 1

The text box accepts numeric values in the range 18

to 25 (18 and 25 are also part of the class). So this

class becomes our valid class. But the question is to

identify invalid equivalence class. The classes will be

as follows:Class I: values < 18 => invalid class

Class II: 18 to 25 => valid class

Class III: values > 25 => invalid class

17 fall under invalid class. 19, 25 and 21 fall undervalid class.

So answer is a (17)


25/28

25

Question 2

In an Examination a candidate has to score

minimum of 25 marks in order to pass the

exam. The maximum that he can score is 50

marks. Identify the Valid Equivalence values ifthe student passes the exam.

a) 22,24,27

b) 21,39,40

c) 29,30,31d) 0,15,22


26/28

26

Solution 2

The classes will be as follows:Class I: values < 25 => invalid classClass II: 25 to 50 => valid class

Class III: values > 50 => invalid classWe have to indentify Valid Equivalencevalues. Valid Equivalence values will bethere in Valid Equivalence class.

All the values should be in Class II.

So answer is c (29,30,31)


27/28

27

Ariane 5Lessons Learned

Take a look on YouTube the Ariane 5 Takeoff e.g:http://www.youtube.com/watch?v=c9Hf4qTxdxs

The video shows the takeoff of Arianne 5. The rocket used the

navigation code of Ariane 4. However as the code was from a

working rocket system it was not fully tested in the context of the new

Arianne 5it was assumed there would be no surprises.

Normally we would want to test the interface between new and

existing code and we would want to use boundary value analysis as

part of this.

As the rocket took off one partition set of values was specificallyreached.
http://www.youtube.com/watch?v=c9Hf4qTxdxshttp://www.youtube.com/watch?v=c9Hf4qTxdxs


28/28

28

Testing is about reducing risk

Remember testing is about reducing risk to an acceptable level.Not removing all risk.

Targeting testing using systematic design techniques, such asequivalence partitions and boundaries significantly, traps moreerror conditions for fewer tests.

Boundaries between software interfaces presents a significantrisk. Do not assume that new code will always work with wellestablished older interfaces.

Typically within projects test analyst specialists are brought ontoprojects too late. They need to be present from day one, whenproducing requirements.

Testing can also be underestimated by project managers. In thecase of Arianne 5, this was a project that was delivered to time atthe launch site and was successful. However was it reallysuccessful? A wise project manager mitigates risk early andinvests in testing.

Documents

boundaryandequivalnce-systematictestdesign-130223120931-phpapp02