Upload
varun-gupta
View
481
Download
1
Embed Size (px)
Citation preview
|���������������� �������������������������� ���������������������
A-PDF Merger DEMO : Purchase from www.A-PDF.com to remove the watermark
�
������
�� `�������������� ���`�������������� ����� `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ����� ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
�� �$��#����������� ����$��#����������� ����� �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
£
������
ºº `�������������� ���`�������������� ���ºº `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ����� ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
�� �$��#����������� ����$��#����������� ����� �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
Ñ
����������������������������������������
�� &(�&(�))*��*����+*���+*���,��,��)�)�,��*���,��,��*���,������,��,��
�� `��������#��������������������� ��`��������#��������������������� �������������� ������ ������ ����� ���� � ������������� ������ ������ ����� ���� � ���������
�� ����� ��������������������#�������� ��������������������#��� � ������� �� ������� ��� ������������ �� � ������� �� ������� ��� ������������ ��� ���#�������������� �#������ ���#�������������� �#�����
�� -�-���������� �����(.���� ������,+*����� ��-�-���������� �����(.���� ������,+*����� ��
r
����������������������������������������
�� `��������/�������� �#������#����� � ����`��������/�������� �#������#����� � ������������(���+*�0�� ����#� ����������(���1+*���������(���+*�0�� ����#� ����������(���1+*����������#���(���+*�0�� ���2� �������������#���(���+*�0�� ���2� ������������(�����������(���33+*������ � ������������(���++*������ � ������������(���+
�� 4�������� 5������� ������������� �� ����/�4�������� 5������� ������������� �� ����/�������������������� ����� ���� ��������������������� ����� ���� ������ ������������������ 2��(������+���� �-���� ������������������ 2��(������+���� �-
�� ������������#����������� � ���������������������#����������� � ����������� ��������������� ��#����������� ������ ��������������� ��#����������� ����#������� � �����������#���*����� ���� �� �#������� � �����������#���*����� ���� �� �#������������� ��� ���#���#������������� ��� ���#���
]
����������������������������������������
&(�)*���+
X
������
ºº `�������������� ���`�������������� ���ºº `�������������������`�������������������
ºº ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ����� ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
�� �$��#����������� ����$��#����������� ����� �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
�
3������������������������������3������������������������������
�� `�� �����������#������`�� �����������#�������� &�����#�������&�����#�������6���1�)� � �����6���1�)� � �����
�� `�� ��%�������������*���������� �� ���(�� ��+����`�� ��%�������������*���������� �� ���(�� ��+���� ��#������ ��#�������� ����������#������ *��������#�������������#������ *��������#���
�� -�-�!� "� ����� ���*�����������������-�-�!� "� ����� ���*�����������������
�� ����������#��������� �����������������#��������� ��������� ��� �4�� ������5���������� �4�� ������5�������
�� -�-� �����������-�-� �����������
�� `�����#������`�����#�������� "������� ��"������� ��33����� � �������� � ���
�� �������#�������(������ ����#��������� ��������#�����+�������#�������(������ ����#��������� ��������#�����+�� -�-� 7!����� ������ ���� ���������8���� �-�-� 7!����� ������ ���� ���������8���� �
ù
������
ºº `�������������� ���`�������������� ���ºº `�������������������`�������������������
�� ����������`�����������������������������`�������������������ºº ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ����� ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
�� �$��#����������� ����$��#����������� ����� �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
)9
a����������������������������������a����������������������������������
�� `��������#�����������2��%��2�2���`��������#�����������2��%��2�2��� ���������� ���� � ����������� ������� ���������� ���� � ����������� ��������#����#���m����������m���������� #������� �� �#������� �� ������ ������������� ������������m�����m� $��� � ��$��� � ��
�� -�-�!� "� � � �������������$�� �(� �-�-�!� "� � � �������������$�� �(� �� �������&������*����������������� �������&������*������������������� �*����*��������+��� �*����*��������+
�� -�-�#����������� ����������$��� � ��*�-�-�#����������� ����������$��� � ��*��������� ���� ��*��������*����� �-�������� ���� ��*��������*����� �-
))
������
ºº `�������������� ���`�������������� ����� `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
ºº ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ����� ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
�� �$��#����������� ����$��#����������� ����� �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
)�
¦��������������¦��������������
�� ������ ������������������#���������������� ������������������#����������
�� 7������ ���� �� ����#����������#�����������7������ ���� �� ����#����������#�����������#���������#�����*���2�� ��������2���#���������#�����*���2�� ��������2��� ��� ������������2� ����#������� �� ��� ������������2� ����#������� ����� �� ���� ����������(���1+�������#������ �� ���� ����������(���1+�������#������� ����� ���� ����������(������� ����� ���� ����������(���33++
�� &��������� ����� ���� �� �&��������� ����� ���� �� �
�� -�-������������������ �������������#� ��-�-������������������ �������������#� ��
�� &������ � ���������� �����������&������ � ���������� �����������
)£
¦��������������¦��������������
)Ñ
������
ºº `�������������� ���`�������������� ����� `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ���ºº ���� ������� ������� ������� ����� ������������ ��������������� ����� ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
�� �$��#����������� ����$��#����������� ����� �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
)r
����������������������������
�� ���� ������������/����� �������#����������������� ������������/����� �������#�������������� ���#��� ���#��
�� ������/ ������/�� &�������#��������� � ��� �.���*����1*����*����&�������#��������� � ��� �.���*����1*����*����33*����:*����:
�� ��%� ����� ����������� ���� ���� �� ������ � � �������%� ����� ����������� ���� ���� �� ������ � � �����
�� 8��� �������� �������������#����������8��� �������� �������������#����������
�� ����� ���������� ����� ���������� �� &�����#�������6�r&�����#�������6�r�� � ������(���������� ���1)� � ������ � ������(���������� ���1)� � �������������������#����������+�������������#����������+
�� &��2�� ��������� ������ ��&��2�� ��������� ������ ��
�� ������� � ����������� � ����
�� ����� �2��� ����� � �������������������� �2��� ����� � ���������������
)]
����������������������������
)X
����������������������������
�
�
�
�
�)
������� ����� ������� �����(���� ������� 2�
#������+
)�
������
ºº `�������������� ���`�������������� ����� `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ���ºº ������������ ��������������� ����� ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
�� �$��#����������� ����$��#����������� ����� �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
)ù
���������������������������������
�� ������ �2�������� ���������������� ����������� �2�������� ���������������� ����� � ��� � ���
�� 8�� ��� �� �#*��� ��������*�����������8�� ��� �� �#*��� ��������*������������� ��� � ���������;����������%��2��*���� � ���������;����������%��2��*��������2� �����������������*�4��������������2� �����������������*�4������� � ���5 � ���5
�� 7 ��������� ���� ������ ������ �� � �7 ��������� ���� ������ ������ �� � ��� �#�� ������� �������������0� �#*�� ��� ���#�� ������� �������������0� �#*�� ��� ����� �������� ���� � ������2��������������� �������� ���� � ������2��������������� �#�����#�������� �� ���� �� � �� ����� �#�����#�������� �� ���� �� � �� ������ ����� ��� ���� �������� ����� ��� ���� ����
�9
������
ºº `�������������� ���`�������������� ����� `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ���ºº ������������
ºº �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
�� �$��#����������� ����$��#����������� ����� �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
�)
���������������������� ��������������������������� �����
��� � � � ���� ��'� ��
) )99 )99 ) 7�����
� )99 )99 � 7�����
£ )99 )99 )99 �$��� ��
Ñ )99 )99 )ùù 7�����
r )99 )99 �99 !� ���������
] )99 ) )99 7�����
X )99 � )99 7�����
� )99 )ùù )99 7�����
ù )99 �99 )99 !� ���������
)9 ) )99 )99 7�����
)) � )99 )99 7�����
)� )ùù )99 )99 7�����
)£ �99 )99 )99 !� ���������
`���������������������� �����
����<�)���1�<������<�)99���3 <�)ùù����<��99
��
���������������������� ��������������������������� �����
� � � �
� �
� � � ����
= � ����
> ?? � ����
@ � � ����
A � � ����
B � � 7� �� �
C � � ����
? � ?? � ����
� � � ����
� ����
� � � ����
= 7� �� �
> ?? � ����
@ � � ����
A ?? � ����
B ?? � � ����
C ?? � ����
? ?? ?? 7� �� �
� ?? � � ����
� � � ����
�� � � � ����
�= � � ����
�> � ?? � ����
�@ � � 7� �� �
�A � � ����
�B � � 7� �� �
�C � � ����
�? � ?? � ����
= � � � ����
� � � �
= � � 7� �� �
=� � � � � �
== � � � ����
=> � � ?? � ����
=@ � � � � ����
=A � � ����
=B � � � ����
=C � 7� �� �
=? � ?? � ����
> � � � ����
> � ?? � ����
>� � ?? � � ����
>= � ?? � ����
>> � ?? ?? 7� �� �
>@ � ?? � �
>A � � � ����
>B � � � � ����
>C � � � ����
>? � � ?? �
@ � � � 7� �� �
@ � ����
@� � � ����
@= 7� �� �
@> ?? � ����
@@ � � ����
@A � � ����
@B � � � ����
@C � 7� �� �
@? � ?? � ����
A � � � ����
� �� � � �� � � � ��(A � �� �@+
�£
������
ºº `�������������� ���`�������������� ����� `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ���ºº ������������
�� �� ���������� ��������� ������ ���������� ��������� ����ºº �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
�� �$��#����������� ����$��#����������� ����� �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
�Ñ
���������������� �!��� ��������������������� �!��� �����
`���������������������� �����
��� ��� � ��� ��� ���� ��'� ��
) ] )r )�)� �D���)]*�)�)�
� ] )r )�)£ �D���)]*�)�)£
£ ] )r )ù)� �D���)]*�)ù)�
Ñ ] )r �9)) �D���)]*��9))
r ] )r �9)� �D���)]*��9)�
] ] ) )ù)� �D����*�)ù)�
X ] � )ù)� �D���£*�)ù)�
� ] £9 )ù)� �D���)*�)ù)�
ù ] £) )ù)� �����
)9 ) )r )ù)� �D�������)]*�)ù)�
)) � )r )ù)� �&�������)]*�)ù)�
)� )) )r )ù)� �!�#����)]*�)ù)�
)£ )� )r )ù)� �"�����)]*�)ù)�
��� �����<�)���1�<������<�]���3 <�))����<�)�
�������<�)���1�<������<�)r���3 <�£9����<�£)
�������<�)�)����1�<�)�)£����<�)ù)����3 <��9))����<��9)�
�r
���� ������� ������(]9����)�r+
���������������� �!��� ��������������������� �!��� �����
��� � � �� � �
) ) ) )C)� �D� �� ��*�)C)�
� ) ) )C)= �D� �� ��*�)C)=
= ) ) )?)� �D� �� ��*�)?)�
> ) ) �9)) �D� �� ��*��9))
r ) ) �9)� �D� �� ��*��9)�
] ) � )C)� �D� �� �=*�)C)�
B ) � )C)= �D� �� �=*�)C)=
C ) � )?)� �D� �� �=*�)?)�
? ) � �9)) �D� �� �=*��9))
)9 ) � �9)� �D� �� �=*��9)�
)) ) )r )C)� �D� �� �)]*�)C)�
)� ) )r )C)= �D� �� �)]*�)C)=
)= ) )r )?)� �D� �� �)]*�)?)�
)> ) )r �9)) �D� �� �)]*��9))
)r ) )r �9)� �D� �� �)]*��9)�
)] ) =9 )C)� �D� �� �=)*�)C)�
)B ) =9 )C)= �D� �� �=)*�)C)=
)C ) =9 )?)� �D� �� �=)*�)?)�
)? ) =9 �9)) �D� �� �=)*��9))
�9 ) =9 �9)� �D� �� �=)*��9)�
�) ) =) )C)� � � �� �)*�)C)�
�� ) =) )C)= � � �� �)*�)C)=
�= ) =) )?)� � � �� �)*�)?)�
�> ) =) �9)) � � �� �)*��9))
�r ) =) �9)� � � �� �)*��9)�
�] � ) )C)� � � �� ��*�)C)�
�B � ) )C)= � � �� ��*�)C)=
�C � ) )?)� � � �� ��*�)?)�
�? � ) �9)) � � �� ��*��9))
=9 � ) �9)� � � �� ��*��9)�
��� � � �� � �
=) � � )C)� � � �� �=*�)C)�
=� � � )C)= � � �� �=*�)C)=
== � � )?)� � � �� �=*�)?)�
=> � � �9)) � � �� �=*��9))
=r � � �9)� � � �� �=*��9)�
=] � )r )C)� � � �� �)]*�)C)�
=B � )r )C)= � � �� �)]*�)C)=
=C � )r )?)� � � �� �)]*�)?)�
=? � )r �9)) � � �� �)]*��9))
>9 � )r �9)� � � �� �)]*��9)�
>) � =9 )C)� ����
>� � =9 )C)= ����
>= � =9 )?)� ����
>> � =9 �9)) ����
>r � =9 �9)� ����
>] � =) )C)� ����
>B � =) )C)= ����
>C � =) )?)� ����
>? � =) �9)) ����
r9 � =) �9)� ����
r) ] ) )C)� �D ��*�)C)�
r� ] ) )C)= �D ��*�)C)=
r= ] ) )?)� �D ��*�)?)�
r> ] ) �9)) �D ��*��9))
rr ] ) �9)� �D ��*��9)�
r] ] � )C)� �D �=*�)C)�
rB ] � )C)= �D �=*�)C)=
rC ] � )?)� �D �=*�)?)�
r? ] � �9)) �D �=*��9))
]9 ] � �9)� �D �=*��9)�
�]
������
ºº `�������������� ���`�������������� ����� `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ���ºº ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ����ºº �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
�� �$��#����������� ����$��#����������� ����� �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
�X
�������������������������� ������������������������������� �����
�� )£�`���������������������� �����)£�`���������������������� �����
�� )�r����� ������� �����)�r����� ������� �����
�� `��������#�������� ���� �� �����*��������`��������#�������� ���� �� �����*������������ �� ����������� �����E)999�����E)�99���� �� ����������� �����E)999�����E)�99
�� �� ���� ������������������ ���� �� ������ �� �� ���� ������������������ ���� �� ������ ��� ����� � ��������� �� � � ����������� ��� ����� � ��������� �� � � ����������� ������ ������������� ������ ���9F�������� ������������� ������ ���9F����
�� ��2�� � ����������� �#������������� ����� �� ���2�� � ����������� �#������������� ����� �� �� ���� �����������#���/�E)99*�E)999*�� ���� �����������#���/�E)99*�E)999*�E)�99*�����EX�99E)�99*�����EX�99
��
�������������������������� ������������������������������� �����
��� �%� � �%� ��� � �� � � ���� �
� � �
� � �@ �-@ � � � �
= � = = � � � �
> � >@ >-@ � � � �
@ @ @ @ @ @ � �
A ? ?B@ ?B-@ � �� � �3
B ? ?B ?B � �� � �3
� ? ?@@ ?@-@ � �� � �3
? � �� �
�@ =-B@ � �� � �
= >-@ � �� � �
� >@ A-B@ � �� � �
= > > > > A � �
> � �� �B �BB@ ��A-�@ � �� � �3
�@ �� �B �� �BB ��@-@ � �� � �3
�A �B �� �� �B@@ ��=-�@ � �� � �3
�B �� �� �� ���� ��� ��� �� ��
�� �� �� �? ���@ ��@ ��� �� �� �
�? �� �? �� ��=� ��A ��� �� �� �
�� �? �� �� ��>@ ��? ��� �� �� �
�� >� >� >� >��� ��� �� ��
�� B� �� �? BBB@ �>�@ �� � ���������3
�= B� B? ?� BBB� �>�> �� � ���������3
�> A? �� ?� BB@@ �>�� �� � ���������3
�@ B� �� ?� B��� �>�� �� � ��������
'� � �`��������������������� ������
�ù
�������������������������� ������������������������������� �����
�� �� �����ù���� ��E)999����������� �� �����ù���� ��E)999�����������
�� 7��2� 2�%� ������ �#�������2�� �7��2� 2�%� ������ �#�������2�� �#����0�� ���2�����0�� ����#� ��#����0�� ���2�����0�� ����#� ������������
�� &�������������#��� � ������������&�������������#��� � �������������� ��� ���������� �� ��� ����������
��� ��%� � ��%� ����� ��� ���������� �����
) )9 )) ù )99r )99-Xr ���������� �1
� )� )X )ù )Xùr �)ù-�r ���������� �3
£ )� )ù )X )�9r ��) ���������� �1
'� �� ������������� �����
£9
������
ºº `�������������� ���`�������������� ����� `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ����� ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
ºº �$��#����������� ����$��#����������� ���ºº �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
£)
��������������������������������
�� 8� �#� ����8� �#� ������ G�#�������������� � � ���G�#�������������� � � ����� �#��������������#�������������
�� �$��#�������������������� � ���������� *�2������ � ����������$��#�������������������� � ���������� *�2������ � ��������� ������� ��������� �������0��� ����� ��2������������� ��� ��� ������� ��������� �������0��� ����� ��2������������� ��� ���� �(���� ���*����� �(���� ���*����33���������+���������+
�� ���������� ����� ���� � �������������������� �������������������� ����� ���� � �������������������� ����������$��#��������$��#��������
�� 4 �� �� �����5�4 �� �� �����5�6�4 ��#������ ��������� ������ �56�4 ��#������ ��������� ������ �5�� ���%����� ����������� ��$��#������ ���� �� �� ������ �����%����� ����������� ��$��#������ ���� �� �� ������ ��
������������ 4����������5� ���%������� � ����*����� ���%����� � ������ �����4����������5� ���%������� � ����*����� ���%����� � ������ ������������ ����� �� ���� ������2������� ���� ������� � ���������� ����� �� ���� ������2������� ���� ������� � ���
£�
��������������������������������
�� �������������������� ���/ �������������������� ���/
�� &��� �������#��������*��*��&��� �������#��������*��*��
�� 7��� �������������� ������ ���*�`*������7��� �������������� ������ ���*�`*������
�� ��<��)����������£��<��)����������£
�� `�<�`)����`�����`£����`Ñ`�<�`)����`�����`£����`Ñ
�� ��<��)��������<��)������
�� �)������)*��£�����`£*�����������)������)*��£�����`£*����������
X
Ú
X
X X X
X
Ú Ú
££
������
ºº `�������������� ���`�������������� ����� `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ����� ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
ºº �$��#����������� ����$��#����������� ���ºº �$��#����������$��#���������
ºº ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
£Ñ
�"�����������������������"����������������������
�� �����������������������#����������������������������������#���������������$��#�������������� � ��������$��#�������������� � ����
�� !��������2�%�$��#��������� � �!��������2�%�$��#��������� � ������<������������������� ����� � ���������<������������������� ����� � ����2� �� ������ �������������� �2� �� ������ �������������� �
�� ���� � � �
� �� �� ��
� � �� �� ��
� = �= �= ��
� > �� �> ��
��<�������������=`�<�`�����`�����`=����`>
��<������������������*��=�����`=*����������
X X
X X X
X
Ú Ú Ú
£r
������
ºº `�������������� ���`�������������� ����� `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ����� ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
ºº �$��#����������� ����$��#����������� ���ºº �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ���ºº � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
£]
��������������������������������������������������������
�� `������� ����� �����`������� ����� ����� ����� ���� ����� � ���� ����� ���� ����� � �������� ����� �
�� ����`�����<�£���Ñ�����<��Ñ�����`�����<�£���Ñ�����<��Ñ��� ��� �
�� �$��#������ ����������$��#������ �������������������� ���� �� ������������ ���� �� ��������� ������������������ �������������� ��������� � ����� ��������� � �
�� ���� � � �
�� �� �� ����� �� �� ��
��£ �� �� ��
��Ñ �� �� ��
��@ �� �£ ��
��A �� �£ ��
��B �� �Ñ ��
��C �� �Ñ ��
��? �� �� ��
���� �� �� ������ �� �� ��
���� �� �� ��
���£ �� �£ ��
���Ñ �� �£ ��
���@ �� �Ñ ��
���A �� �Ñ ��
���B �£ �� ��
���C �£ �� ��
���? �£ �� ��
���� �£ �� ������ �£ �£ ��
���� �£ �£ ��
���£ �£ �Ñ ��
���Ñ �£ �Ñ ��
£X
������
ºº `�������������� ���`�������������� ����� `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ����� ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
ºº �$��#����������� ����$��#����������� ���ºº �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ���ºº ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
£�
����������������������������������������������������������������
�� "�����$��#������������� �������#���� �"�����$��#������������� �������#���� �
�� ���������������������������������� ��������� �/�)�,���%�,�X9*�)�,�� ��%�,��9*���������������������������� �/�)�,���%�,�X9*�)�,�� ��%�,��9*�������������������)�,������,�ù9)�,������,�ù9
�� 7�#�������� �/���%�.�)*���%�:�X9*�� ��%�.�)*���������7�#�������� �/���%�.�)*���%�:�X9*�� ��%�.�)*���������� ��%�:��9*������.�)*������:�ù9� ��%�:��9*������.�)*������:�ù9
�� &���#�������� �*�������#������������#����&���#�������� �*�������#������������#���������(�%�2�%�$��#���� � ���+�����(�%�2�%�$��#���� � ���+
�� &�����#�������� �*��� � �����2����#����&�����#�������� �*��� � �����2����#������#����#������� �����������#����2��������#����#������� �����������#����2������#����(����������+#����(����������+
£ù
����������������������������������������������������������������
�� �����/ �����/
�� ������ �*� ���������� ���������� ������������ �*� ���������� ���������� ������2�� � ����� ���� �� ����������#���� � �2�� � ����� ���� �� ����������#���� � �������������6���� ���� ������ ����������������6���� ���� ������ ��������������� ���� �� ������������ ���� �� �
�� � ������ ����������������� � ������ ������ ����������������� � ��������� ���������� ���������#�������� ����� ���������� ���������#�������� �
Ñ9
������
ºº `�������������� ���`�������������� ����� `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ����� ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
ºº �$��#����������� ����$��#����������� ����� �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
ºº �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
Ñ)
��������������������������������������������������������������
������� ������������ �����
�� '� �� �/�!� ���������*�����*�7�����*��$��� ��'� �� �/�!� ���������*�����*�7�����*��$��� ��
�� ������ ����� ������ �� �(����+�$��#��������������� ����� ������ �� �(����+�$��#����������� �)�<�H.�*��*��:�/� �� ������2� ��������*��*����������$��� ��I�)�<�H.�*��*��:�/� �� ������2� ��������*��*����������$��� ��I
�� ���<�H.�*��*��:�/� �� ������2� ��������*��*����������������I���<�H.�*��*��:�/� �� ������2� ��������*��*����������������I
�� �£�<�H.�*��*��:�/� �� ������2� ��������*��*��������������I�£�<�H.�*��*��:�/� �� ������2� ��������*��*��������������I
�� ��<�H.�*��*��:�/�������*��*������������ �������� �����I��<�H.�*��*��:�/�������*��*������������ �������� �����I
�� ���� � � � ���� ��'� ��
'�) @ @ @ �$��� ��
'�� � � £ 7�����
'�£ £ Ñ @ ����
'�Ñ Ñ ) � !� ���������
�
��������������������������������������������������������������
������� ������������ �����
�� 7��� �(������+�$��#���������7��� �(������+�$��#���������
")�<�H.�*��*��:�/���<���<��I"��<�H.�*��*��:�/���<��*���J��I"£�<�H.�*��*��:�/���<��*���J��I"Ñ�<�H.�*��*��:�/���<��*���J��I
"r�<�H.�*��*��:�/���J��*���J��*���J��I;;�"]�<�H.�*��*��:�/���K���1��I"]��<�H.�*��*��:�/���<���1��I"]���<�H.�*��*��:�/���:���1��I;;�"X�<�H.�*��*��:�/���K���1��I"X��<�H.�*��*��:�/���<���1��I"X���<�H.�*��*��:�/���:���1��I;;�"��<�H.�*��*��:�/���K���1��I"���<�H.�*��*��:�/���<���1��I"����<�H.�*��*��:�/���:���1��I
Ñ£
������
ºº `�������������� ���`�������������� ����� `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ����� ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
ºº �$��#����������� ����$��#����������� ����� �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ����ºº �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
ÑÑ
��������������������������������������������������������������
� �!���|�������� �!���|�������
�� 7��� �#������7��� �#�������� )�,���� ��,�)�)�,���� ��,�)��� )�,�����,�£))�,�����,�£)�� )�)��,�����,��9)�)�)��,�����,��9)�
�� ����� ������������������ ��������������� �����$��#��������������$��#���������
�� 8)�<�H���� ��/�)�,���� ��,�)��I8)�<�H���� ��/�)�,���� ��,�)��I�� ")�<�H�����/�)�,�����,�£)�I")�<�H�����/�)�,�����,�£)�I�� L)�<�H�����/�)�)��,�����,��9)��IL)�<�H�����/�)�)��,�����,��9)��I
�� 7�#����$��#���������7�#����$��#����������� 8��<�H���� ��/���� ��.�)�I8��<�H���� ��/���� ��.�)�I�� 8£�<�H���� ��/���� ��:�)��I8£�<�H���� ��/���� ��:�)��I�� "��<�H�����/�����.�)�I"��<�H�����/�����.�)�I�� "£�<�H�����/�����:£)�I"£�<�H�����/�����:£)�I�� L��<�H�����/�����.�)�)��IL��<�H�����/�����.�)�)��I�� L£�<�H����/�����:��9)��IL£�<�H����/�����:��9)��I
����7" 8�� � "�� L�� ���� ��'� �� ��) ] )r )ù)� ];)];)ù)�
��� 3) )r )ù)� ��#��������
��£ )£ )r )ù)� ��#��������
��Ñ ] 3) )ù)� ��#��������
��r ] £� )ù)� ��#�������� ��] ] )r )�)) ��#��������
��X ] )r �9)£ ��#��������
Ñr
��������������������������������������������������������������
� �!���|�������� �!���|�������
�� ����� ����������������������� �������� �4 �� �5� ������� ����������������������� �������� �4 �� �5� ���� ������������� � ��#���;��#����#�� ������������� � ��#���;��#����#
�� "����� ���������/���� ���� ������� ��������� ��� M"����� ���������/���� ���� ������� ��������� ��� M
8)�<�H���� �/���� ������£9������I8��<�H���� �/���� ������£)������I8£�<�H���� �/���� �����&�������I
")�<�H����/�)�,�����,����I"��<�H����/�����<��ù�I"£�<�H����/�����<�£9�I"Ñ�<�H����/�����<�£)�I
L)�<�H����/�����<�)ù99�IL��<�H����/�)�)��,�����,��9)���!"�(����J�)ù99+
�!"�(��������Ñ�<�9+ IL£�<�H����/�)�)��,�����,��9)���!"�(��������Ñ�J�9+�I
!� ������� �� ����$��#���������NNN!� ������� �� ����$��#���������NNN
Ñ]
��������������������������������������������������������������
� �!���|�������� �!���|�������
����7" 8�� � "�� L�� ���� ��'� ��
��) ] )Ñ )ù99 ];)r;)ù99��� X �ù )ù)� X;£9;)ù)�
��£ � £9 )ù)£ ��#��������
��Ñ ] £) )ù99 ��#��������
��%�$��#��������� � �����
����7" 8�� � "�� L�� ���� ��'� ��
��) ] )Ñ )ù99 ];)r;)ù99
��� ] )Ñ )ù)� ];)r;)ù)�
��£ ] )Ñ )ù)£ ];)r;)ù)£
��Ñ ] �ù )ù99 ];£9;)ù99
��r ] �ù )ù)� ];£9;)ù)�
��] ] �ù )ù)£ ];£9;)ù)£
��X ] £9 )ù99 9X;9);)ù99
��C ] £9 )ù)� 9X;9);)ù)�
��ù ] £9 )ù)£ 9X;9);)ù)£
��)9 ] £) )ù99 � '
��)) ] £) )ù)� � '
��)� ] £) )ù)£ � '
��)£ X )Ñ )ù99 X;)r;)ù99
��)Ñ X )Ñ )ù)� X;)r;)ù)�
��)r X )Ñ )ù)£ X;)r;)ù)£
��)] X �ù )ù99 X;£9;)ù99
��)X X �ù )ù)� X;£9;)ù)�
��)C X �ù )ù)£ X;£9;)ù)£
����7" 8�� � "�� L�� ���� ��'� ��
��)ù X £9 )ù99 X;£);)ù99
���9 X £9 )ù)� X;£);)ù)�
���) X £9 )ù)£ X;£);)ù)£
���� X £) )ù99 9C;9);)ù99
���£ X £) )ù)� 9C;9);)ù)�
���Ñ X £) )ù)£ 9C;9);)ù)£
���r � )Ñ )ù99 �;)r;)ù99
���] � )Ñ )ù)� �;)r;)ù)�
���X � )Ñ )ù)£ �;)r;)ù)£
���C � �ù )ù99 � '
���ù � �ù )ù)� 9£;9);)ù)�
��£9 � �ù )ù)£ � '
��£) � £9 )ù99 � '
��£� � £9 )ù)� � '
��££ � £9 )ù)£ � '
��£Ñ � £) )ù99 � '
��£r � £) )ù)� � '
��£] � £) )ù)£ � '
� �����$��#��������� � �����
ÑX
������
ºº `�������������� ���`�������������� ����� `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ����� ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
ºº �$��#����������� ����$��#����������� ����� �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ���ºº �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
�
��������������������������������������������������������������
����������� ���������������� �����
�� 7��� �"�������$��#���������7��� �"�������$��#����������� ���%���%
�� �)�<�H���%/�)�,���%�,�X9�I�)�<�H���%/�)�,���%�,�X9�I�� ���<�H���%/���%�.�)�I���<�H���%/���%�.�)�I�� �£�<�H���%/���%�:�X9�I�£�<�H���%/���%�:�X9�I
�� � ��%� ��%�� �)�<�H�� ��%/�)�,�� ��%�,��9�I�)�<�H�� ��%/�)�,�� ��%�,��9�I�� ���<�H�� ��%/�� ��%�.�)�I���<�H�� ��%/�� ��%�.�)�I�� �£�<�H�� ��%/�� ��%�:��9�I�£�<�H�� ��%/�� ��%�:��9�I
�� `���`����� `)�<�H�����/�)�,������,�ù9�I`)�<�H�����/�)�,������,�ù9�I�� `��<�H�����/������.�)�I`��<�H�����/������.�)�I�� `£�<�H�����/������:�ù9�I`£�<�H�����/������:�ù9�I
Ñù
��������������������������������������������������������������
����������� ���������������� �����
� �����7��� �"�������$��#����������� �����
�� ���� ��%� � ��%� ����� ��� ����������
��� =@ Ñ� Ñ@ =ù�� AÑ�
��� =@ � � � �
��= =@ Ñ� ù� � �
��Ñ =@ � Ñ@ � �
��@ =@ � � � �
��A =@ � ù� � �
��B =@ C� Ñ@ � �
��C =@ C� � � �
��ù =@ C� ù� � �
���� � Ñ� Ñ@ � �
���� � � � � �
���� � Ñ� ù� � �
���= � � Ñ@ � �
���Ñ � � � � �
���@ � � ù� � �
���A � C� Ñ@ � �
���B � C� � � �
���C � C� ù� � �
���ù B� Ñ� Ñ@ � �
���� B� � � � �
���� B� Ñ� ù� � �
���� B� � Ñ@ � �
���= B� � � � �
���Ñ B� � ù� � �
���@ B� C� Ñ@ � �
���A B� C� � � �
���B B� C� ù� � �
�� ���� ��%� � ��%� ����� ��� ����������
��� =@ Ñ� Ñ@ =ù�� AÑ�
��� � � � � �
��= B� C� ù� � �
��%�7��� �"�������$��#����������� �����
r9
��������������������������������������������������������������
����������� ���������������� �����
�� ����<�Ñr�����%��1�£9���� ��%��1��r������������<�Ñr�����%��1�£9���� ��%��1��r��������
�� �)�<�H�.��%*�� ��%*�����:�/�����,�)999�I�)�<�H�.��%*�� ��%*�����:�/�����,�)999�I
�� ���<�H�.��%*�� ��%*�����:�/�)999�.�����,����<�H�.��%*�� ��%*�����:�/�)999�.�����,�)�99�I)�99�I
�� �£�<�H�.��%*�� ��%*�����:�/�����:�)�99�I�£�<�H�.��%*�� ��%*�����:�/�����:�)�99�I
'� �� �������$��#����������� �����
�� ���� ��%� � ��%� ����� ��� ����������
'�) r r r r99 r9
'�� )r )r )r )r99 )Br
'�£ �r �r �r �r99 £A9
r)
������
ºº `�������������� ���`�������������� ����� `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ����� ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
ºº �$��#����������� ����$��#����������� ����� �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ����ºº ������������'���#� ����������������'���#� ����
r�
3������������#����������3������������#����������
)-)- ��� ���� �������������$��#��������� � ���������� ���� �������������$��#��������� � ��������������� ���� �����������2�%�$��#����������������� ���� �����������2�%�$��#��������� � ���*�2�������� ���*������ ���� ����������� ��� ����� � ���*�2�������� ���*������ ���� ����������� ��� �������������$��#��������� � �����������$��#��������� � ���
�-�- ������� ���� ���%������ ����� �� ���� ������������ ���� ���%������ ����� �� ���� �����������������2��� ������� � ���������������� �������������2��� ������� � ���������������� �� ������ ���� ������ ���
£-£- 7������������ ��������������������� �*�2������� ���7������������ ��������������������� �*�2������� ���� �����$��#��������� � ���� ����������#����� �����$��#��������� � ���� ����������#����������(-�-�����������������+������(-�-�����������������+
Ñ-Ñ- �$��#��������� � ���������������� �2������� ��$��#��������� � ���������������� �2������� ��� �������������� ������������������ ���������� ��� �������������� ������������������ ���������� �#���-����������� ����� ������2������ ��#���-����������� ����� ������2������ �������� ��������������������� ������ ��������������������� 33����33��� �#������#������ �#������#���
r£
3������������#����������3������������#����������
r-r- �$��#��������� � �������� ��� �����������������$��#��������� � �������� ��� �������������������������2� �����������#��� � ���-�(������4���5����������2� �����������#��� � ���-�(������4���5� ������ ���������������� ��$��#���������+� ������ ���������������� ��$��#���������+�(-�-�!� "� ����� ���+(-�-�!� "� ����� ���+
]-]- �$��#��������� � ������������� ��2��� ���$��#��������� � ������������� ��2��� �������������� ������������-�7�����������*� �������������� ������������-�7�����������*� �������� ����� ������ �������������� ��������������� ����� ������ �������������� ���������$��#���������*������� ��!� "� ����� ���$��#���������*������� ��!� "� ����� ���
X-X- � �����$��#��������� � ������%���������� ����� �����$��#��������� � ������%���������� ���� �� � ��#����������������� �2��� ����� ����� �� � ��#����������������� �2��� ����� ����� ����� ���� �%�-�7�� ������������������*� ��� ����� ���� �%�-�7�� ������������������*� ���2���� ������ �4����5� � �����*���� ���������� ��2���� ������ �4����5� � �����*���� ���������� ��!� "� ����� ���!� "� ����� ���
rÑ
3������������#����������3������������#����������
�-�- �#��� �������������������4 ���#��� �������������������4 ������ 5�$��#������ ������������#��*����� 5�$��#������ ������������#��*����2���2���� ��!� "� �����-�7�����2���2���� ��!� "� �����-�7��� �������*� ���������4��#����5����� �������*� ���������4��#����5����4�� ���5�$��#������� � ���-��������4�� ���5�$��#������� � ���-������������ *� ���� �� ���� �� ��� ������������ *� ���� �� ���� �� ��� ����������������� �������������������������� ���������������������� � �������� � ���
rr
������
�� `�������������� ���`�������������� ����� `�������������������`�������������������
�� ����������`�����������������������������`��������������������� ���� � ��������`����������������������� � ��������`�������������������
�� ����� ������ �������� ������ ����� ���� ������� ������� ������� ����� ������������ ��������������� ����� ������������
�� �� ���������� ��������� ������ ���������� ��������� ������ �� ���������� ��!� "� � ������ ���������� ��!� "� � ������ �� ���������� ������������� ������ ���������� ������������� ����
�� ������������`��������#����� ���������������`��������#����� ���
�� �$��#����������� ����$��#����������� ����� �$��#����������$��#���������
�� ��%��$��#����������� �����%��$��#����������� ����� � ������$��#����������� ���� ������$��#����������� ����� ����� ������$��#����������� �������� ������$��#����������� ���
�� �$��#����������� ���������� ��������� �����$��#����������� ���������� ��������� ������ �$��#����������� ���������� ��!� "� �&��� ����$��#����������� ���������� ��!� "� �&��� ����� �$��#����������� ���������� ������������� �����$��#����������� ���������� ������������� ������ ������������'���#� ����������������'���#� ����
r]
¦�����¦�����
�� ��� 2����� ���������� ����O�������������� 2����� ���������� ����O����������������� ���*� ����-�D������*����������� ���*� ����-�D������*����� ����(���� ���r�����]+ ����(���� ���r�����]+
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Chapter 6
Equivalence Class Testing
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Equivalence Class Testing
Domain Range
F
Equivalence class testing uses information about the functional mapping itself to identify test cases
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Equivalence Relations
• Given a relation R defined on some set S, R is anequivalence relation if (and only if), for all, x, y, and zelements of S:– R is reflexive, i.e., xRx– R is symmetric, i.e., if xRy, then yRx– R is transitive, i.e., if xRy and yRz, then xRz
• An equivalence relation, R, induces a partition on the set S,where a partition is a set of subsets of S such that:– The intersection of any two subsets is empty, and– The union of all the subsets is the original set S
• Note that the intersection property assures no redundancy,and the union property assures no gaps.
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Equivalence Partitioning
Domain Range
F Define relation R as follows: for x, y in domain, xRy iff F(x ) = F(y). Facts: 1. R is an equivalence relation. 2. An equivalence relation induces a partition on a set. 3. Works best when F is many-to-one 4. (pre-image set)
Domain Range
Test cases are formed by selecting one value from each equivalence class. - reduces redundancy - identifying the classes may be hard
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Forms of Equivalence Class Testing
• Normal: classes of valid values of inputs• Robust: classes of valid and invalid values of
inputs• Weak: (single fault assumption) one from each
class• Strong: (multiple fault assumption) one from
each class in Cartesian Product• We compare these for a function of two
variables, F(x1, x2)• Extension to problems with 3 or more variables
is “obvious”.
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Weak Normal Equivalence Class Testing
• Identify equivalence classes of valid values.• Test cases have all valid values.• Detects faults due to calculations with valid values
of a single variable.• OK for regression testing.
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Equivalence classes (of valid values):{a <= x1 < b}, {b <= x1 < c}, {c <= x1 <= d}{e <= x2 < f}, {f <= x2 <= g}
Weak Normal Equivalence Class Test Cases
a b c d
x2
x1
e
f
g
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Weak Robust Equivalence Class Testing
• Identify equivalence classes of valid and invalidvalues.
• Test cases have all valid values except one invalidvalue.
• Detects faults due to calculations with valid valuesof a single variable.
• Detects faults due to invalid values of a singlevariable.
• OK for regression testing.
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Equivalence classes (of valid and invalid values):{a <= x1 < b}, {b <= x1 < c}, {c <= x1 <= d}, {e <= x2 < f}, {f <= x2 <= g}Invalid classes: {x1 < a}, {x1 > d}, {x2 < e}, {x2 > g}
Weak Robust Equivalence Class Test Cases
a b c d
x2
x1
e
f
g
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
a b c d
x2
x1e
f
g
a b c d
x2
x1e
f
g
Is thispreferable
to this? Why?
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Strong Normal Equivalence Class Testing
• Identify equivalence classes of valid values.• Test cases from Cartesian Product of valid values.• Detects faults due to interactions with valid
values of any number of variables.• OK for regression testing, better for progression
testing.
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Strong Normal Equivalence Class Test Cases
a b c d
x2
x1
e
f
g
Equivalence classes (of valid values):
{a <= x1 < b}, {b <= x1 < c}, {c <= x1 <= d}
{e <= x2 < f}, {f <= x2 <= g}
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Strong Robust Equivalence Class Testing
• Identify equivalence classes of valid and invalid values.• Test cases from Cartesian Product of all classes.• Detects faults due to interactions with any values of any
number of variables.• OK for regression testing, better for progression testing.
(Most rigorous form of Equivalence Class testing, BUT,Jorgensen’s First Law of Software Engineering applies.)
• Jorgensen’s First Law of Software Engineering:– The product of two big numbers is a really big number.– (scaling up can be problematic)
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Strong Robust Equivalence Class Test Cases
a b c d
x2
x1
e
f
g
Equivalence classes (of valid and invalid values):
{a <= x1 < b}, {b <= x1 < c}, {c <= x1 <= d}, {e <= x2 < f}, {f <= x2 <= g}
Invalid classes: {x1 < a}, {x1 > d}, {x2 < e}, {x2 > g}
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Selecting an Equivalence RelationThere is no such thing as THE equivalence relation.
If x and y are days, some possibilities for Nextdate are:
•x R y iff x and y are mapped onto the same year•x R y iff x and y are mapped onto the same month•x R y iff x and y are mapped onto the same date•x R y iff x(day) and y(day) are “treated the same”•x R y iff x(month) and y(month) are “treated the same”•x R y iff x(year) and y(year) are “treated the same”
Best practice is to select an equivalence relation thatreflects the behavior being tested.
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
NextDate Equivalence Classes• Month:
– M1 = { month : month has 30 days}– M2 = { month : month has 31 days}– M3 = { month : month is February}
• Day– D1 = {day : 1 <= day <= 28}– D2 = {day : day = 29 }– D3 = {day : day = 30 }– D4 = {day : day = 31 }
• Year (are these disjoint?)– Y1 = {year : year = 2000}– Y2 = {year : 1812 <= year <= 2012 AND (year ≠ 0 Mod 100)
and (year = 0 Mod 4)– Y3 = {year : (1812 <= year <= 2012 AND (year ≠ 0 Mod 4)
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Not Quite Right
• A better set of equivalence classes for year is– Y1 = {century years divisible by 400} i.e., century leap
years– Y2 = {century years not divisible by 400} i.e., century
common years– Y3 = {non-century years divisible by 4} i.e., ordinary
leap years– Y4 = {non-century years not divisible by 4} i.e., ordinary
common years• All years must be in range: 1812 <= year <= 2012• Note that these equivalence classes are disjoint.
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Weak Normal Equivalence Class Test Cases
Select test cases so that one element from each input domain equivalence class is used as a test input value.
April 1 2000 Jan. 29 1900 Feb. 30 1812 April 31 1901
WN-1 WN-2 WN-3 WN-4
Test Case
Input Domain Equiv. Classes
Input Values Expected Outputs
M1, D1, Y1 M2, D2, Y2 M3, D3, Y3 M1, D4, Y4
April 2 2000 Jan. 30 1900 impossible impossible
Notice that all forms of equivalence class testing presumethat the variables in the input domain are independent;logical dependencies are unrecognized.
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
• With 4 day classes, 3 month classes, and 4 year classes,the Cartesian Product will have 48 equivalence class testcases. (Jorgensen’s First Law of Software Engineeringstrikes again!)
• Note some judgment is required. Would it be better tohave 5 day classes, 4 month classes and only 2 yearclasses? (40 test cases)
• Questions such as this can be resolved by consideringRisk.
Strong Normal Equivalence Class Test Cases
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Revised NextDate Domain Equivalence Classes
The Cartesian Product of these contains 40 elements.
•Month:–M1 = { month : month has 30 days}–M2 = { month : month has 31 days except December}–M3 = { month : month is February}–M4 = {month : month is December}
•Day–D1 = {day : 1 <= day <= 27}–D2 = {day : day = 28 }–D3 = {day : day = 29 }–D4 = {day : day = 30 }–D5 = {day : day = 31 }
•Year (are these disjoint?)–Y1 = {year : year is a leap year}–Y2 = {year : year is a common year}
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
When to Use Equivalence Class Testing
• Variables represent logical (rather than physical)quantities.
• Variables “support” useful equivalence classes.• Try to define equivalence classes for
– The Triangle Problem• 0 < sideA < 200• 0 < sideB < 200• 0 < sideC < 200
– The Commission Problem (exercise)
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Another Equivalence Class Strategy
• “Work backwards” from output classes.
• For the Triangle Problem, could have– {x, y, z such that they form an Equilateral triangle}– {x, y, z such that they form an Isosceles triangle with x = y}– {x, y, z such that they form an Isosceles triangle with x = z}– {x, y, z such that they form an Isosceles triangle with y = z}– {x, y, z such that they form a Scalene triangle}
• How many equivalence classes will be needed forx,y,z Not a triangle?
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
In-Class Exercise
• Apply the “working backwards” approach todevelop equivalence classes for theCommission Problem.
• Hint: use boundaries in the output space.
Software Testing: A Craftsman’s Approach, 3rd Edition Equivalence Class Testing
Assumption Matrix
Robust WorstCase BoundaryValueStrong RobustEquiv. Class
Worst CaseBoundary Value
Strong NormalEquiv. Class
Multiple fault
RobustBoundary Value
Weak RobustEquiv. Class
Boundary Value
Weak NormalEquiv. Class
Single fault
Valid andInvalid Values
Valid Values
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
Chapter 7
Decision Table Based Testing
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
Decision Table Based Testing
• Originally known as Cause and Effect Graphing– Done with a graphical technique that expressed
AND-OR-NOT logic.– Causes and Effects were graphed like circuit
components– Inputs to a circuit “caused” outputs (effects)
• Equivalent to forming a decision table in which:– inputs are conditions– outputs are actions
• Test every (possible) rule in the decision table.• Recommended for logically complex situations.
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
Decision Tables
• Represent complex conditional behavior.• Support extensive analysis
– Consistency– Completeness– Redundancy– Algebraic simplification
• Executable (and compilable)• Two forms: Limited and Extended Entry.• “Don't Care” condition entries require special
attention.• Dependencies usually yield impossible situations
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
Decision Table TerminologyStub Entry
Conditions
Actions
c1 c2 c3
a1 a2 a3 a4
T FTrue False
--
X X
X X
X
X X
X X
X
T F --
True
True
False
False
Rule
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
One Decision Table for the Triangle Problem
XXXa5: Impossible
Xa4: Equilateral
XXXa3: Isosceles
Xa2: Scalene
Xa1: Not a triangle
NYNYNYNY--c4: b = c?
NNYYNNYY--c3: a = c?
NNNNYYYY--c2: a = b?YYYYYYYYNc1: a, b, c are a triangle?
Why are rules 3, 4, and 6 impossible?
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
Decision Table with MutuallyExclusive Conditions
T——c3: February
—T—c2: 31-day month
——Tc1: 30-day month
Rule 3Rule 2Rule 1Conditions
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
conditions R1 R2 R3 c1: month in M1 T -- -- c2: month in M2 -- T -- c3: month in M3 -- -- T Rule count 4 4 4
Rule Counting to Check for Completeness
• A limited Entry decison table with n conditions has 2n rules • A Don't Care entry doubles the count of a rule • What are the possibilities when rule count is not 2n ? • More precise to use F! (must be false) than -- (don't care)
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
A Redundant Decision Table
•! Rule 9 is identical to Rule 4 (T, F, F)
•! Since the action entries for rules 4 and 9 are identical, ! there is no ambiguity, just redundancy.
conditions! 1-4! 5! 6! 7! 8! 9! c1: !! T! F! F! F! F! T
! c2: !! --! T! T! F! F! F
! c3: !! --! T! F! T! F! F
! a1:! ! X! X! X! --! --! X! !
! a2:! ! --! X! X! X! --! --
! a3:! ! X! --! X! X! X! X
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
Decision Table Exercise(revisit the false negative, false positive question)
• Suggested conditions– Expected output is correct– Observed output is correct– Expected and observed outputs agree
• Suggested actions– True pass– True fail– False pass– False fail– Impossible
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
An Inconsistent Decision Table
• Rule 9 is identical to Rule 4 (T, F, F) • Since the action entries for rules 4 and 9 are different there is ambiguity. • This table is inconsistent, and the inconsistency implies non-determinism.
conditions 1-4 5 6 7 8 9 c1: T F F F F T c2: -- T T F F F c3: -- T F T F F a1: X X X -- -- -- a2: -- X X X -- X a3: X -- X X X --
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
Nextdate Limited Entry Decision Table
Conditions c1: month in M1? c2: month in M2? c3: month in M3? c4: day in D1? c5: day in D2? c6: day in D3? c7: day in D4? c8: leap year? Actions a1: impossible a2: next date
This decision table will have 256 rules,many of which will be logically impossible.
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
Decision Table Based Test Cases1. Decision table testing begins with equivalence classes forconditions as in equivalence class testing.
2. The sparseness due to the assumption of independence isaddressed by careful examination of elements in the cross product.
3. For the equivalence classes defined earlier, the cross productcontains 36 elements. The corresponding decision table has 36 rules. <M1, D1, Y1>, <M1, D2, Y1>, <M1, D3, Y1>, <M1, D4, Y1>, <M2, D1, Y1>, <M2, D2, Y1>, <M2, D3, Y1>, <M2, D4, Y1>, <M3, D1, Y1>, <M3, D2, Y1>, <M3, D3, Y1>, <M3, D4, Y1>, <M1, D1, Y2>, <M1, D2, Y2>, <M1, D3, Y2>, <M1, D4, Y2>, <M2, D1, Y2>, <M2, D2, Y2>, <M2, D3, Y2>, <M2, D4, Y2>, <M3, D1, Y2>, <M3, D2, Y2>, <M3, D3, Y2>, <M3, D4, Y2>, <M1, D1, Y3>, <M1, D2, Y3>, <M1, D3, Y3>, <M1, D4, Y3>, <M2, D1, Y3>, <M2, D2, Y3>, <M2, D3, Y3>, <M2, D4, Y3>, <M3, D1, Y3>, <M3, D2, Y3>, <M3, D3, Y3>, <M3, D4, Y3>.
4. Notice that many of these are impossible, e.g., <M1, D4, * >
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
NextDate Extended Entry Decision TableConditions c1: month in M1? M2? M3? c2: day in D1? D2? D3? D4? c3: year in Y1? Y2? Y3? Actions a1: impossible a2: increment day a3: reset day a4: increment month a5: reset month a6: increment year
This decision table will have 36 rules, and corresponds to the cross product. Many of the rules will be logically impossible. Many rules would collapse, except for considerations for December.
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
Revised NextDate Domain Equivalence Classes
The corresponding decision table of these contains 40 elements.
•Month:–M1 = { month : month has 30 days}–M2 = { month : month has 31 days except December}–M3 = { month : month is December}–M4 = {month : month is February }
•Day–D1 = {day : 1 <= day <= 27}–D2 = {day : day = 28 }–D3 = {day : day = 29 }–D4 = {day : day = 30 }–D5 = {day : day = 31 }
•Year (are these disjoint?)–Y1 = {year : year is a leap year}–Y2 = {year : year is a common year}
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
Xa6: increment year
XXXa5: reset month
XXXXa4: incrementmonth
XXXXa3: reset day
XXXXXXXXXXXXXXXa2: increment day
Xa1: impossible
-Y2
Y1
Y2
Y1
-------------c3: year in
D5
D4
D3
D3
D2
D2
D1
D5
D4
D3
D2
D1,
D5
D4
D3
D2
D1
D5
D4
D3
D2
D1
c2: day in
M4M3M2M1c1: month in
22212019181716151413121110987654321
NextDate Extended Entry Decision Table
Notice there are 40 rules in this decision table, corresponding tothe 40 elements in the cross product of the revised equivalenceclasses.
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
Xa6:incrementyear
Xa5: resetmonth
XXXXa4:incrementmonth
XXXXXa3: reset day
XXXXXa2:incrementday
XXXa1:impossible
-Y2Y1Y2Y1--------c3: year in
D4, D5D3D3D2D2D1D5D1,D2,D3,D4D5D1,D2,D3,D4D5D4D1,D2,D3c2: day in
M4M3M2M1c1: month in
21,2220191817161511-14106-9541-3rules
NextDate Extended Entry Decision TableAlgebraically Condensed to 13 rules (test cases)
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
Procedure for Decision Table Based Testing
1. Determine conditions and actions. (Might need to iterate) 2. Develop a (the!) Decision Table, watching for • completeness • don't care entries • redundant and inconsistent rules 3. Each rule defines a test case.
Software Testing: A Craftsman’s Approach, 3rd Edition Decision Table Based Testing
Procedure for Decision Table Based Testing
• Determine conditions and actions. (Might need toiterate)
• Develop a (the!) Decision Table, watching for– Completeness– Don’t care entries– Redundant and/or inconsistent entries– Impossible rules
• Each rule defines a test case.
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Chapter 8
Retrospective on FunctionalTesting
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Retrospective on FunctionalTesting
• Test case development effort
• Test case effectiveness
• Test method selection guidelines
• Case study
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Test Case Development Effort
• As with so many things in life,“You get out of it what you put into it.” --Dad
• Boundary value: almost mechanical
• Equivalence class: effort to identify classes
• Decision table: still more effort
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Numbers of Test Cases
boundary value
decision table
low
high
Number of Test Cases
Sophistication
equivalence class
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Test Case Development Effort
Effort to Identify Test Cases
boundary value
decision table
low
high
Sophistication
equivalence class
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Test Case Effectiveness
• True trade-off between development effort andnumber of test cases.
• Vulnerabilities– Boundary value testing has gaps and redundancies, and
many test cases.– Equivalence class testing eliminates the gaps and
redundancies, but cannot deal with dependenciesamong variables.
– Decision table testing extends equivalence class testingby dealing with dependencies, and supports algebraicreduction of test cases.
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Appropriate Choices of Test Methods
xxa9. decision table
xxa8. strong robust equiv. class
xxa7. strong normal equiv. class
xxa6. weak robust equiv. class
xxa5. weak normal equiv. class
a4. robust worst case BVA
a3. worst case BVA
xa2. robustness BVA
xa1. boundary value analysis (BVA)
–NYNY–NYNYc4. exception handling?
–NNYY–NNYYc3. single fault assumption?
NYYYYNYYYYc2. independent variables?
LLLLLPPPPPc1. variables (P, physical, L, logical)
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Case StudyA hypothetical Insurance Premium Program computes the semi-annual car insurance premium based on two parameters: the policyholder's age and driving record:
Premium = BaseRate*ageMultiplier – safeDrivingReduction
The ageMultiplier is a function of the policy holder's age, and thesafe driving reduction is given when the current points (assigned bytraffic courts for moving violations) on the policy holder's driver'slicense are below an age-related cutoff. Policies are written fordrivers in the age range of 16 to 100. Once a policy holder has 12points, his/her driver's license is suspended (hence there is no needfor insurance). The BaseRate changes from time to time; for thisexample, it is $500 for a semi-annual premium.
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Insurance Premium Program Data
20051.560<= age <= 100
15070.845<= age < 60
10051.035<= age < 45
5031.825<= age < 35
5012.816<= age < 25
Safe DrivingReduction
PointsCutoff
AgeMultiplier
Age Range
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Insurance Premium ProgramCalculations, Test Method Selection
• Premium = BaseRate*ageMultiplier – safeDrivingReduction• ageMultiplier = F1(age) [from table]• safeDrivingReduction = F2(age, points) [from table]• age and safeDrivingReduction are physical variables, with
a dependency in F2.• Boundary values for age: 16, 17, 54, 99, 100• Boundary values for safeDrivingReduction: 0, 1, 6, 11, 12• Robust values for age and safeDrivingReduction are not
allowed by business rules.• Worst case BVA yields 25 test cases, and many gaps,
some redundancy. Need something better.
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Graph of Boundary Value Test Cases
20 100806040age
15
10
5
0
points
Severe gaps!
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Graph of Boundary Value Test Cases(refined boundaries)
20 100806040age
15
10
5
0
points
Severe redundancy!
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Insurance Premium Program TestMethod Selection
• age has ranges that receive similar treatment.equivalence class testing is indicated.– Age ranges per the data table
• The points cutoff is also a range, furtherindication for equivalence class testing.– Points {0, 1}– Points {2, 3}– Points {4, 5}– Points {6, 7}– Points {8, 9, 10, 11, 12}
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Insurance Premium Program StrongNormal Equivalence Class Test Cases
20 100806040age
15
10
5
0
points
Still a lot of redundancy, try decision tables.
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Insurance Premium Program DecisionTable Test Cases
--200--150--100--50--50a2. safeDriving
1.51.50.80.81.81.81.81.82.82.8a1. agemultiplier
5-120-47-120-65-120-43-130-21-120c2. points60-10045-6035-4525-3516-25c1. age is
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Insurance Premium Program DecisionTable Test Cases
20 100806040age
15
10
5
0
points
What about age range endpoints?
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Insurance Premium Program Test Cases(Decision table with boundary values hybrid)
20 100806040age
15
10
5
0
points
Ahhhh, at last!
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Functional Testing
Wrap Up
• The inherent nature of the program beingtested should dictate the test method.– The decision table “expert system” (slide 7)
recommendation is just a start.– Applications are seldom “chemically pure”.
• Hybrid combinations of test methods can bevery useful.
• Good judgment, based on insight, is a signof a craftsperson.
Software Testing: A Craftsman’s Approach, 3rd Edition Path Testing II
Chapter 9
Path Testing–Part 2
Software Testing: A Craftsman’s Approach, 3rd Edition Path Testing II
(McCabe) Basis Path Testing
• in math, a basis "spans" an entire space, such that everything inthe space can be derived from the basis elements.
• the cyclomatic number of a strongly connected directed graph isthe number of linearly independent cycles.
• given a program graph, we can always add an edge from the sinknode to the source node to create a strongly connected graph.(assuming single entry, single exit)
• computing V(G) = e - n + p from the modified program graphyields the number of independent paths that must be tested.
• since all other program execution paths are linear combinationsof the basis path, it is necessary to test the basis paths. (Somesay this is sufficient; but that is problematic.)
• the next few slides follow McCabe's original example.
Software Testing: A Craftsman’s Approach, 3rd Edition Path Testing II
McCabe's ExampleDerived, StronglyConnected Graph
A
B
C
D
E
F
G
1 2
3 45 6
78
9 10
A
B
C
D
E
F
G
V(G) = 10 - 7 + 2(1) = 5
V(G) = 11 - 7 + 1 = 5
McCabe's OriginalGraph
Software Testing: A Craftsman’s Approach, 3rd Edition Path Testing II
McCabe's Baseline Method
To determine a set of basis paths, 1.
Pick a "baseline" path that corresponds to normal execution. (The baseline should have as many decisions as possible.)
2.
To get succeeding basis paths, retrace the baseline until you reach a decision node. "Flip" the decision (take another alternative) and continue as much of the baseline as possible.
3.
Repeat this until all decisions have been flipped. When you reach V(G) basis paths, you're done.
4.
If there aren't enough decisions in the first baseline path, find a second baseline and repeat steps 2 and 3.
Following this algorithm, we get basis paths for McCabe's example.
Software Testing: A Craftsman’s Approach, 3rd Edition Path Testing II
path \ edges traversed 1 2 3 4 5 6 7 8 9 10 p1: A, B, C, G 1 0 0 1 0 0 0 0 1 0 p2: A, B, C, B, C, G 1 0 1 2 0 0 0 0 1 0 p3: A, B, E, F, G 1 0 0 0 1 0 0 1 0 1 p4: A, D, E, F, G 0 1 0 0 0 1 0 1 0 1 p5: A, D, F, G 0 1 0 0 0 0 1 0 0 1 ex1: A, B, C, B, E, F, G 1 0 1 1 1 0 0 1 0 1 ex2: A, B, C, B, C, B, C, G 1 0 2 3 0 0 0 0 1 0 ex1 = p2 + p3 - p1 ex2 = 2p2 - p1
Basis Paths
A
B
C
D
E
F
G
1 2
3 45 6
78
9 10
Software Testing: A Craftsman’s Approach, 3rd Edition Path Testing II
McCabe Basis Paths in the Triangle Program
V(G) = 23 - 20 + 2(1) = 5
Basis Path Set B1p1: 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 16, 18, 19, 20, 22, 23 (mainline)p2: 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 16, 18, 19, 20, 22, 23 (flipped at 9)p3: 4, 5, 6, 7, 8, 9, 11, 12, 13, 21, 22, 23 (flipped at 13)p4: 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 20, 22, 23 (flipped at 14)p5: 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 16, 17, 19, 20, 22, 23 (flipped at 16)
There are 8 topologically possible paths.4 are feasible, and 4 are infeasible.
Exercise: Is every basis path feasible?
13
14
15 16
17 18
19
20
21
22
4 5 6 7 8
10
9
11
12
23
Software Testing: A Craftsman’s Approach, 3rd Edition Path Testing II
Essential Complexity• McCabe’s notion of Essential Complexity deals with the
extent to which a program violates the precepts ofStructured Programming.
• To find Essential Complexity of a program graph,– Identify a group of source statements that corresponds to one of the
basic Structured Programming constructs.– Condense that group of statements into a separate node (with a new
name)– Continue until no more Structured Programming constructs can be
found.– The Essential Complexityof the original program is the cyclomatic
complexity of the resulting program graph.• The essential complexity of a Structured Program is 1.• Violations of the precepts of Structured Programming
increase the essential complexity.
Software Testing: A Craftsman’s Approach, 3rd Edition Path Testing II
Essential Complexity ofSchach’s Program Graph
B
DC E
4
first
A
B
DC E
F
G
last
Thisoriginal
Reducesto this
V(G) = 8 – 5 + 2(1) = 5Essential complexity is 5
Software Testing: A Craftsman’s Approach, 3rd Edition Path Testing II
A
B C
D
E
F G
H I
J K
L
first
last
aa
E
F G
H I
J K
L
first
last
b
a
H I
J KL
first
last
b
c
a
HL
first
last
b
c
da
L
first
last
d
e
first
e
Condensation with Structured Programming Constructs
Software Testing: A Craftsman’s Approach, 3rd Edition Path Testing II
Branching into a loop
Branching out of a loop
Branching into a decision
Branching out of a decision
Violations of Structured Programming Precepts
Software Testing: A Craftsman’s Approach, 3rd Edition Path Testing II
Cons and Pros
• Issues– Linear combinations of execution paths are counter-
intuitive. What does 2p2 – p1 really mean?– How does the baseline method guarantee feasible basis
paths?– Given a set of feasible basis paths, is this a sufficient test?
• Advantages– McCabe's approach does address both gaps and
redundancies.– Essential complexity leads to better programming
practices.– McCabe proved that violations of the structured
programming constructs increase cyclomatic complexity,and violations cannot occur singly.
Software Testing: A Craftsman’s Approach, 3rd Edition Path Testing II
Program NextDate Dim tomorrowDay,tomorrowMonth,tomorrowYear As IntegerDim day,month,year As Integer1. Output ("Enter today's date in the form MM DD YYYY")2. Input (month,day,year)3. Case month Of4. Case 1: month Is 1,3,5,7,8,Or 10: '31 day months (except Dec.)5. ! If day < 316. ! ! Then tomorrowDay = day + 17. ! ! Else8. ! ! ! tomorrowDay = 19. ! ! ! tomorrowMonth = month + 110. ! EndIf11. Case 2: month Is 4,6,9,Or 11 '30 day months12. ! If day < 3013. ! ! Then tomorrowDay = day + 114. ! ! Else15. ! ! ! tomorrowDay = 116. ! ! ! tomorrowMonth = month + 117. ! EndIf18. Case 3: month Is 12: 'December19. ! If day < 31 20. ! ! Then tomorrowDay = day + 121. ! ! Else22. ! ! ! tomorrowDay = 123. ! ! ! tomorrowMonth = 124. ! ! ! If year = 201225. ! ! ! ! Then Output ("2012 is over")26. ! ! ! ! Else tomorrow.year = year + 127. ! ! ! EndIf28. ! EndIf
Software Testing: A Craftsman’s Approach, 3rd Edition Path Testing II
29. Case 4: month is 2: 'February30. ! If day < 2831. ! ! Then tomorrowDay = day + 132. ! ! Else!33. ! ! ! If day = 2834. ! ! ! ! Then!35. ! ! ! ! ! If ((year MOD 4)=0)AND((year MOD 400)"0)36. ! ! ! ! ! ! Then tomorrowDay = 29 'leap year37. ! ! ! ! ! ! Else! ! 'not a leap year38. ! ! ! ! ! ! ! tomorrowDay = 139. ! ! ! ! ! ! ! tomorrowMonth = 340. ! ! ! ! ! EndIf41. ! ! ! ! Else! If day = 2942. ! ! ! ! ! ! ! Then tomorrowDay = 143. ! ! ! ! ! ! ! ! tomorrowMonth = 344. ! ! ! ! ! ! ! Else! Output("Cannot have Feb.", day)45. ! ! ! ! ! ! EndIf46. ! ! ! EndIf47. ! EndIf48. EndCase49. Output ("Tomorrow's date is", tomorrowMonth,! ! ! ! tomorrowDay, tomorrowYear)50. End NextDate
Software Testing: A Craftsman’s Approach, 3rd Edition Path Testing II
11
1
2
3
4
5
6
7
8
9
10
12
13
14
15
16
17
18
19
20
21
22
23
24
25 26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
50
47
48
49
44
45
46
Software Testing: A Craftsman’s Approach, 3rd Edition Path Testing II
15 Output("Locks sold: ", totalLocks)16 Output("Stocks sold: ", totalStocks)17 Output("Barrels sold: ", totalBarrels)18 sales = lockPrice*totalLocks + stockPrice*totalStocks + barrelPrice * totalBarrels19 Output("Total sales: ", sales)20 If (sales > 1800.0) 21 Then22 commission = 0.10 * 1000.023 commission = commission + 0.15 * 800.024 commission = commission + 0.20*(sales-1800.0)25 Else If (sales > 1000.0)26 Then27 commission = 0.10 * 1000.028 commission = commission + 0.15*(sales-1000.0)29 Else commission = 0.10 * sales30 EndIf31 EndIf32 Output("Commission is $", commission)33 End Commission
Program Commission Dim lockPrice, stockPrice, barrelPrice As Real Dim locks, stocks, barrels As Integer Dim totalLocks, totalStocks As Integer Dim totalBarrels As Integer Dim lockSales, stockSales As Real Dim barrelSales As Real Dim sales, commission As Real1 lockPrice = 45.02 stockPrice = 30.03 barrelPrice = 25.04 totalLocks = 05 totalStocks = 06 totalBarrels = 07 Input(locks)8 While NOT(locks = -1)9 Input(stocks, barrels)10 totalLocks = totalLocks + locks11 totalStocks = totalStocks + stocks12 totalBarrels = totalBarrels + barrels13 Input(locks)14 EndWhile
Commission Program Pseudo-Code
Software Testing: A Craftsman’s Approach, 3rd Edition Path Testing II
11
1 2 3 4 5 6 7
8 9 10 12 13
14 15 16 17 18 19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Chapter 10
Data Flow TestingSlice Testing
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Data Flow Testing
• Often confused with "dataflow diagrams“.• Main concern: places in a program where data
values are defined and used.• Static (compile time) and dynamic (execution
time) versions.• Static: Define/Reference Anomalies on a
variable that– is defined but never used (referenced)– is used but never defined– is defined more than once
• Starting point is a program, P, with program graphG(P), and the set V of variables in program P.
• "Interesting" data flows are then tested asmini-functions.
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Definitions• Node n ∈ G(P) is a defining node of the
variable v ∈ V, written as DEF(v, n), iff thevalue of the variable v is defined at thestatement fragment corresponding to node n.
• Node n ∈ G(P) is a usage node of the variablev ∈ V, written as USE(v, n), iff the value of thevariable v is used at the statement fragmentcorresponding to node n.
• A usage node USE(v, n) is a predicate use(denoted as P-use) iff the statement n is apredicate statement; otherwise, USE(v, n) is acomputation use (denoted C-use).
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
More Definitions
• A definition-use path with respect to a variable v(denoted du-path) is a path in PATHS(P) suchthat for some v ∈ V, there are define and usagenodes DEF(v, m) and USE(v, n) such that m and nare the initial and final nodes of the path.
• A definition-clear path with respect to a variablev (denoted dc-path) is a definition-use path inPATHS(P) with initial and final nodes DEF (v, m)and USE (v, n) such that no other node in thepath is a defining node of v.
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Example: first part of the Commission Program1. Program Commission (INPUT,OUTPUT)2. Dim locks, stocks, barrels As Integer3. Dim lockPrice, stockPrice, barrelPrice As Real4. Dim totalLocks, totalStocks, totalBarrels As Integer5. Dim lockSales, stockSales, barrelSales As Real6. Dim sales, commission As Real7. lockPrice = 45.08. stockPrice = 30.09. barrelPrice = 25.010. totalLocks = 011. totalStocks = 012. totalBarrels = 013. Input(locks)14. While NOT(locks = -1)15. Input(stocks, barrels)16. totalLocks = totalLocks + locks17. totalStocks = totalStocks + stocks18. totalBarrels = totalBarrels + barrels19. Input(locks)20. EndWhile21. Output(“Locks sold: “, totalLocks)22. Output(“Stocks sold: “, totalStocks)23. Output(“Barrels sold: “, totalBarrels)
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Rest of Commission Problem23. Output(“Barrels sold: “, totalBarrels)24. lockSales = lockPrice * totalLocks25. stockSales = stockPrice * totalStocks26. barrelSales = barrelPrice * totalBarrels27. sales = lockSales + stockSales + barrelSales28. Output(“Total sales: “, sales)29. If (sales > 1800.0)30. Then31. commission = 0.10 * 1000.032. commission = commission + 0.15 * 800.033. commission = commission + 0.20 *(sales-1800.0)34. Else If (sales > 1000.0)35. Then36. commission = 0.10 * 1000.037. commission = commission + 0.15 *(sales-1000.0)38. Else39. commission = 0.10 * sales40. EndIf41. EndIf42. Output(“Commission is $”, commission)43. End Commission
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Program Graph of Commission Problem107 8 9 11 12 13
14
15 16 2017 18 19
21 22 23 24 25 26 27 28
29
31
32
33
36
41 42
30 34
38
35
37
39
40 43
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Define/Use Test Cases
• Technique: for a particular variable,– find all its definition and usage nodes, then– find the du-paths and dc-paths among these.– for each path, devise a "suitable" (functional?) set of test cases.
• Note: du-paths and dc-paths have both static and dynamicinterpretations– Static: just as seen in the source code– Dynamic: must consider execution-time flow (particularly
for loops)• Definition clear paths are easier to test
– No need to check each definition node, as is necessary fordu-paths
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Define and Use Nodes
18, 23, 26 12, 18totalBarrels
17, 22, 25 11, 17totalStocks
16, 21, 24 10, 16totalLocks
1815barrels
1715stocks
14, 16 13, 19locks
Used at Node Defined at NodeVariable
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Example (continued)13. Input(locks)14. While NOT(locks = -1)15. Input(stocks, barrels)16. totalLocks = totalLocks + locks17. totalStocks = totalStocks + stocks18. totalBarrels = totalBarrels + barrels19. Input(locks)20. EndWhileWe have• DEF (locks, 13), DEF (locks, 19)• USE (locks, 14), a predicate use• USE (locks, 16). A computation use• du-paths for locks are the node sequences <13, 14> (a dc-path),
<13, 14, 15, 16>, <19, 20, 14 >, < 19, 20, 14 , 15, 16>• Is <13, 14, 15, 16> definition clear?• Is < 19, 20, 14, 15, 16> definition clear? What about repetitions?
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Coverage Metrics Based on du-paths• In the following definitions, T is a set of paths in
the program graph G(P) of a program P, withthe set V of variables.
• The set T satisfies the All-Defs criterion for theprogram P iff for every variable v ∈ V, Tcontains definition-clear paths from everydefining node of v to a use of v.
• The set T satisfies the All-Uses criterion for theprogram P iff for every variable v ∈ V, Tcontains definition-clear paths from everydefining node of v to every use of v, and to thesuccessor node of each USE(v, n).
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Coverage Metrics Based on du-paths(continued)
• The set T satisfies the All-P-Uses/Some C-Usescriterion for the program P iff for every variablev ∈ V, T contains definition-clear paths fromevery defining node of v to every predicate useof v; if a definition of v has no P-uses, adefinition-clear path leads to at least onecomputation use.
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Coverage Metrics Based on du-paths(continued)
• The set T satisfies the All-C-Uses/Some P-Usescriterion for the program P iff for every variablev ∈ V, T contains definition-clear paths fromevery defining node of v to every computationuse of v; if a definition of v has no C-uses, adefinition-clear path leads to at least onepredicate use.
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Coverage Metrics Based on du-paths(concluded)
• The set T satisfies the All-du-paths criterion forthe program P iff for every variable v ∈ V, Tcontains definition-clear paths from everydefining node of v to every use of v and to thesuccessor node of each USE(v, n), and thatthese paths are either single-loop traversals orcycle-free.
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Rapps-Weyuker Coverage SubsumptionAll Paths
All DU Paths
All UsesAll C Uses Some P Uses
All P Uses Some C Uses
All Defs All P Uses
All Edges
All Nodes
S. Rapps and E. J. Weyuker "Selecting Software Test Data Using Data Flow Information" IEEE Transactions of Software Engineering , vol 11 no 4 IEEE Computer Society Press, Washington, D. C. , April 1985, pp 367 - 375.
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Exercise: Where does the “Alldefinition-clear paths” coverage metricfit in the Rapps-Weyuker lattice?
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Data Flow Testing Strategies
• Data flow testing is indicated in– Computation-intensive applications– “long” programs– Programs with many variables
• A definition-clear du-path represents a smallfunction that can be tested by itself.
• If a du-path is not definition-clear, it shouldbe tested for each defining node.
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Slice Testing• Often confused with "module execution paths"• Main concern: portions of a program that
"contribute" to the value of a variable at somepoint in the program.
• Nice analogy with history -- a way to separate acomplex system into "disjoint" components thatinteract:– European history– North American history– Orient history
• A dynamic construct.
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Slice Testing Definitions
Starting point is a program, P, with program graphG(P), and the set V of variables in program P.Nodes in the program graph are numbered andcorrespond to statement fragments.
• Definition: The slice on the variable set V atstatement fragment n, written S(V, n), is the setof node numbers of all statement fragments inP prior to n that contribute to the values ofvariables in V at statement fragment n.
• This is actually a “backward slice”.• Exercise: define a “forward slice”.
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Fine Points• "prior to" is the dynamic part of the definition.• "contribute" is best understood by extending the
Define and Use concepts:– P-use: used in a predicate (decision)– C-use: used in computation– O-use: used for output– L-use: used for location (pointers, subscripts)– I-use: iteration (internal counters, loop indices)– I-def: defined by input– A-def: defined by assignment
• usually, the set V of variables consists of just oneelement.
• can choose to define a slice as a compilable set ofstatement fragments -- this extends the meaning of"contribute"
• because slices are sets, we can develop a latticebased on the subset relationship.
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
In the program fragment13. Input(locks)14. While NOT(locks = -1)15. Input(stocks, barrels)16. totalLocks = totalLocks + locks17. totalStocks = totalStocks + stocks18. totalBarrels = totalBarrels + barrels19. Input(locks)20.EndWhile
There are these slices on locks (notice thatstatements 15, 17, and 18 do not appear):
S1: S(locks, 13) = {13}S2: S(locks, 14) = {13, 14, 19, 20}S3: S(locks, 16) = {13, 14, 19, 20}S4: S(locks, 19) = {19}
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Lattice of Slices
• Because a slice is a set of statement fragmentnumbers, we can find slices that are subsets ofother slices.
• This allows us to “work backwards” frompoints in a program, presumably where a faultis suspected.
• The statements leading to the value ofcommission when it is output are an excellentexample of this pattern.
• Some researchers propose that this is the waygood programmers think when they debugcode.
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Example Lattice of Slices
S39
S34
S35
S36
S37
S40
S38
S34: S(commission, 41) = {41}
S35: S(commission, 42) = {41, 42}
S36: S(commission, 43) = {3, 4, 5, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 30, 36, 41, 42, 43}
S37: S(commission, 47) = {47}
S38: S(commission, 48) = {3, 4, 5, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 36, 47, 48}
S39: S(commission, 50) = {3, 4, 5, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 36, 50}
S40: S(commission, 51) = {3, 4, 5, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 36, 41, 42, 43, 47, 48, 50}
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Diagnostic Testing with Slices
• Relative complements of slices yield a "diagnostic"capability. The relative complement of a set B with respectto another set A is the set of all elements of A that are notelements of B. It is denoted as A - B.
• Consider the relative complement set S(commission, 48) -S(sales, 35):– S(commission, 48) = {3, 4, 5,36,18,19, 20, 23, 24, 25, 26, 27, 34, 38,
39, 40, 44,45,47}– S(sales, 35) = {3, 4, 5, 36, 18, 19, 20, 23, 24, 25, 26, 27}– S(commission, 48) - S(sales, 35) = {34, 38, 39, 40, 44,45,47}
• If there is a problem with commission at line 48, we candivide the program into two parts, the computation ofsales at line 34, and the computation of commissionbetween lines 35 and 48. If sales is OK at line 34, theproblem must lie in the relative complement; if not, theproblem may be in either portion.
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Programming with Slices
• One researcher suggests the possibility of“slice splicing”:– Code a slice, compile and test it.– Code another slide, compile and test it, then splice
the two slices.– Continue until the whole program is complete.
• Exercise: in what ways is slice splicingdistinct from agile (bottom up) programming?
Software Testing: A Craftsman’s Approach, 3rd Edition Data Flow Testing
Exercise/Discussion:When should testing stop?
• when you run out of time?• when continued testing causes no new
failures?• when continued testing reveals no new
faults?• when you can't think of any new test cases?• when you reach a point of diminishing
returns?• when mandated coverage has been attained?• when all faults have been removed?
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Structural Testing
Chapter 11
Retrospective on Structural Testing
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Structural Testing
Structural Testing Comparison
• How much testing is enough?
• Effort and size trendlines
• Metrics for test method comparison
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Structural Testing
Exercise/Discussion:When should testing stop?
• when you run out of time?• when continued testing causes no new
failures?• when continued testing reveals no new
faults?• when you can't think of any new test cases?• when you reach a point of diminishing
returns?• when mandated coverage has been attained?• when all faults have been removed?
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Structural Testing
Number of Test Coverage Items
low
high
SophisticationDU-Path
DD-Path
Basis Path
Slice
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Structural Testing
Effort to Identify Test Coverage Items
low
high
Sophistication
DU-PathDD-Path
Basis Path
Slice
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Structural Testing
Number of Coverage Items in theCommission Problem
BasisDD-Path DU-Path Slices048
1216202428323640
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Structural Testing
Metrics for Test Method Comparison• Assume that a functional testing technique M
generates m test cases, and that these testcases are tracked with respect to a structuralmetric S that identifies s elements in the unitunder test. When the m test cases are executed,they traverse n of the s structural elements.
• This framework supports the definition ofmetrics for testing effectiveness.
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Structural Testing
Metrics for Testing Effectiveness
• The coverage of a methodology M withrespect to a metric S is ratio of n to s. Wedenote it as C(M,S).
• The redundancy of a methodology M withrespect to a metric S is ratio of m to s. Wedenote it as R(M,S).
• The net redundancy of a methodology M withrespect to a metric S is ratio of m to n. Wedenote it as NR(M,S).
Software Testing: A Craftsman’s Approach, 3rd Edition Retrospective on Structural Testing
Sample Comparisons
0.630.631.00404025Slices0.760.761.00333325DU-Path2.272.271.00111125DD-Path0.270.271.0011113Decision Table2.272.271.00111125Output BVA
CommissionProgram
11.3611.361.001111125Worst Case BVA2.141.360.6411715BVA
Triangle ProgramNR(M,S)R(M,S)C(M,S)snmMethod
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
Chapter 12
Levels of Testing
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
Levels and Life Cycle Models
• Levels of testing depend primarily on thesoftware life cycle used.
• BUT, most forms of testing levels are derivedfrom the V-Model version of the good, oldWaterfall Model.
• Iterative models introduce the need forregression testing.
• System testing is greatly enhanced when anexecutable specification is used.
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
Software Development Life Cycle Models
• Waterfall (presumes perfect foresight)• Incremental (delayed prototypes)• Rapid Prototyping (elicit user feedback)• Operational Specification (executable)• Transformational Implementation (lab only)• Agile Methods
Increasingly Operationalviews
Ref. Agresti, New Paradigms of Software DevelopmentIEEE Tutorial
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
The Waterfall ModelRequirements Specification
Preliminary Design
Detailed Design
Coding
Unit Testing
Integration Testing
System Testing
Output of a phase is the inputto the next phase
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
Requirements Specification
Preliminary Design
Detailed Design
Coding
Unit Testing
Integration Testing
System Testing
The Waterfall Model (continued)
Feedback Cycles
what
what
what
how
how
how
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
The Waterfall Model (aka the V-Model)
Requirements Specification
Preliminary Design
Detailed Design
Coding
Unit Testing
Integration Testing
System Testing
Levels of Abstraction and Testing
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
The Waterfall Model: Advantages
• Framework fits well with– levels of abstraction– levels of management– programming language packaging
• Phases have clearly defined end products• Convenient for project management• Works well with Functional Decomposition• Massive parallel development
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
The Waterfall Model: Disadvantages
• Long customer feedback cycle resolving faults found duringsystem testing is extremely expensive
• "Exists for the convenience of management"(-- M. Jackson) stifles creativity and unnecessarily constraintsdesigner's thought processes
• Stresses analysis to the exclusion of synthesis• High peak in manpower loading profile• "Requires perfect foresight" -- William Agresti
any errors, omissions in early phases will propagate
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
Requirements Specification
Preliminary Design
Series of Builds
Detailed Design
Coding
Unit Testing
Integration Testing
Regression Testing
Progression Testing
Incremental Software Development
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
Series of Prototypes
Preliminary Design
Detailed Design
Coding
Integration Testing
System Testing
Build Prototype
Define Prototype Objectives
Customer Feedback
Unit Testing
Rapid Prototyping Software Development
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
Rapid Prototyping
Why prototype? 1. To determine feasibility 2. To obtain early customer feedback Keep or dispose? To be rapid, many compromises are made. If a prototype is kept, it will be extremely difficult to modify and maintain. Best practice: dispose once purpose has been served.
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
execute spec
Develop executable
specification
Customer Feedback
Requirements Specification
Preliminary Design
Detailed Design
Coding
Integration Testing
System Testing
Unit Testing
Software Development with an Executable Specification
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
Executable Specifications
Why use an executable specification? 1. To determine behavior 2. To obtain early customer feedback Other uses? 1. Automatic generation of system test cases. 2. Develop order of test case execution 3. Training 4. Early analysis
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
Transformational Implementation
Requirements Specification
System Testing
Formal Requirements Specification
Customer Testing
Working System
Series of Transformations
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
Transformational Specification Pros and Cons
• Advantages:– Intermediate Waterfall phases (design, code, test)
are eliminated.– Customer tests delivered system– All maintenance is done on specification
• Disadvantages:– Specification must be very formal (predicate
calculus)– Series of transformations is not well understood;
tends to be specific to application domains– Very limited success so far.– Doesn't scale up well.– Knowing what to change in the formal specification
is difficult
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
Agile Methods
• Many flavors– eXtreme Programming (XP)– SCRUM– Test-Driven Development
• Customer-Driven• Goals
– Respond to customer– Reduce unnecessary effort– Always have something that works
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
Length of Feedback Cycle(Customer/Developer)
Waterfall
Agile
Where would you putthe other life cyclemodels?
Software Testing: A Craftsman’s Approach, 3rd Edition Levels of Testing
Hybrid Models
• Because they replace the Requirementsspecification phase, rapid prototyping andexecutable specifications can be merged withthe iterative models.
• The Agile Models are highly iterative, but theytypically are not used in combination with rapidprototyping or executable specifications.
Integra(on tes(ng
by Kris(an Sandahl IDA
TDDD04 spring 2012
2
Levels of tes(ng
Requirements
System Design (Architecture,
High-level Design)
Module Design (Program Design, Detailed Design)
Implementation of Units (classes, procedures,
functions)
Unit testing
Module Testing (Integration testing of units)
System Testing (Integration testing of modules)
Acceptance Test
(Release testing)
Validate Requirements, Verify Specification
Verify System Design
Verify Module Design
Verify Implementation
Project Management, Software Quality Assurance (SQA), Supporting Tools, Education
Maintenance
Outline
• Informal example • Theory:
– Decomposi(on-‐based integra(on – Call-‐graph based integra(on – Path-‐based integra(on
Example: Local bus card reader Sell (ckets Registrer travel
Chose (cket
Display
Read RFID
Check balance Check validity
Communicate with server
Deduct money
Power supply
Show balance
User buNons Func(on group, aka Capability, aka Anatom
and
Dependency
Example: Organic integra(on plan Sell (ckets Registrer travel
Chose (cket
Display
Read RFID
Check balance Check validity
Communicate with server
Deduct money
Power supply
Show balance
User buNons
Services
User interface
Server func(ons
Communica(on
Hardware and supply
Big Bang integra(on tes(ng Sell (ckets
Registrer travel
Chose (cket
Display
Read RFID Check balance Check validity
Communicate with server Deduct money
Power supply
Show balance
User buNons Bring everything
together Switch on the current Try to buy a (cket
!
Is Big Bang smart? • Perhaps for small systems • Well-‐defined components and interfaces • No extra soWware needed Problems: • It is very difficult to isolate the defects found, as it is very difficult
to tell whether defect is in component or interface. • Defects present at the interfaces of components are iden(fied at
very late stage. • There is high probability of missing some cri(cal defects which
might surfaced in produc(on. • It is very difficult to make sure that all the cases for integra(on
tes(ng are covered. (www.Tes(ngGeek.com)
BoNom-‐up integra(on 1
Power supply Hardware and supply
Environment crea(on Install components Draw cables Measure – compare with calcula(on
BoNom-‐up integra(on 2
Communicate with server
Power supply
Communica(on
Hardware and supply
Rudimental client Rudimental server
Communica(on possible?
Environment
Drivers
• A pretend module that requires a sub-‐system and passes a test case to it
black-‐box view
Driver
setup driver SUT(x) verifica(on
SUT
driver
SUT
SUT – System Under Test
Is boNom-‐up smart? • If the basic func(ons are complicated, error-‐prone or has development risks
• If boNom-‐up development strategy is used • If there are strict performance or real-‐(me requirements
Problems: • Lower level func(ons are oWen off-‐the shelf or trivial • Complicated User Interface tes(ng is postponed • End-‐user feed-‐back postponed • Effort to write drivers.
Top-‐down integra(on 1 Sell (ckets Registrer travel
Chose (cket
Show balance
Services
Test end-‐users Envrionment: Modules for the services Driving a prototype interface
Is it possible to perform services in a good way?
Top-‐down integra(on 2 Sell (ckets Registrer travel
Chose (cket
Display
Read RFID
Func(on 1 Func(on 2 Func(on 2
Show balance
User buNons
Services
User interface
Rudimentary Server func(ons
Stubs Are all our test scenarios fulfilled?
Stub
• A program or a method that simulates the input-‐output func6onality of a missing sub-‐system by answering to the decomposi(on sequence of the calling sub-‐system and returning back simulated or ”canned” data.
SUT Service(x)
Check x Stub Return y; end
SUT
Stub
Is top-‐down smart? • Test cases are defined for func(onal requirements of the system
• Defects in general design can be found early • Works well with many incremental development methods
• No need for drivers Problems: • Technical details postponed, poten(al show-‐stoppers • Many stubs are required • Stubs with many condi(ons are hard to write
Decomposi(on-‐based integra(on
The func(onal decomposi(on tree: • hierarchical order of processes • return edges excluded • reflects the lexical inclusion of units, in terms of the order in which they need to be compiled
Jorgensen, Paul C "Chapter 13 -‐ Integra(on Tes(ng". SoWware Tes(ng: A CraWsman's Approach, Third Edi(on. Auerbach Publica(ons. © 2008. Books24x7. Go to: hNp://guide.bibl.liu.se/datavetenskap Click: Books24x7, Login Search: ISBN:9780849374753
Example: SATM from Jorgensen
17
Func6onal Decomposi6on of the SATM System
1
A 10 B
D E 11 12 13 14 15
2 3 4 5 6 7 8 9
C
16 17 F 22
18 19 20 21 23 24 25 26 27
Table 1: SATM Units and Abbreviated Names
Unit Level Unit Name 1 1 SATM System A 1.1 Device Sense & Control D 1.1.1 Door Sense & Control 2 1.1.1.1 Get Door Status 3 1.1.1.2 Control Door 4 1.1.1.3 Dispense Cash E 1.1.2 Slot Sense & Control 5 1.1.2.1 WatchCardSlot 6 1.1.2.2 Get Deposit Slot Status 7 1.1.2.3 Control Card Roller 8 1.1.2.3 Control Envelope Roller 9 1.1.2.5 Read Card Strip 10 1.2 Central Bank Comm. 11 1.2.1 Get PIN for PAN 12 1.2.2 Get Account Status 13 1.2.3 Post Daily Transac6ons
Unit Level Unit Nam B 1.3 Terminal Sense & Control 14 1.3.1 Screen Driver 15 1.3.2 Key Sensor C 1.4 Manage Session 16 1.4.1 Validate Card 17 1.4.2 Validate PIN 18 1.4.2.1 GetPIN F 1.4.3 Close Session 19 1.4.3.1 New Transac6on Request 20 1.4.3.2 Print Receipt 21 1.4.3.3 Post Transac6on Local 22 1.4.4 Manage Transac6on 23 1.4.4.1 Get Transac6on Type 24 1.4.4.2 Get Account Type 25 1.4.4.3 Report Balance 26 1.4.4.4 Process Deposit 27 1.4.4.5 Process Withdrawal
Three level func(onal decomposi(on tree
A
C B D
E F H G
Level 1
Level 2
Level 3
Big-‐Bang tes(ng
A
C B D
E F H G
Level 1
Level 2
Level 3
Unit test A
Unit test B
Unit test H
…
System-‐wide test
Environment: A, B, C, D, E, F, G, H
BoNom-‐up tes(ng
A
C B D
E F H G
Level 1
Level 2
Level 3
Environments: Session 1: E, driver(B) S2: F, driver(B) S3: E, F, driver(B) S4: G, driver(D) S5: H, driver(D) S6: G, H, driver(D) S7: E, F, B, driver(A) S8: C, driver(A) S9: G, H, D, driver(A) S10: E, F, B, C, G, H, D, A
Number of drivers: 3 Number of sessions: 10
General formula: Number of drivers: (nodes-‐leaves) Number of sessions: (nodes-‐leaves)+edges
SATM: 10 drivers, 42 sessions
Top-‐down tes(ng
A
C B D
E F H G
Level 1
Level 2
Level 3
Environments: Session 1: A, stub(B), stub(C), stub(D) S2: A, B, stub(C), stub(D) S3: A, stub(B), C, stub(D) S4: A, stub(B), stub(C), D S5: A, B, stub(E), stub(F), C, D, stub(G), stub(H) S6: A, B, E, stub(F), C, D, stub(G), stub(H) S7: A, B, stub(E), F, C, D, stub(G), stub(H) S8: A, B, stub(E), stub(F), C, D, G, stub(H) S9: A, B, stub(E), stub(F), C, D, stub(G), H S10: A, B, E, F, C, D, G, H
Number of stubs: 7 Number of sessions: 10
General formula: Number of stubs: (nodes – 1) Number of sessions: (nodes-‐leaves)+edges
SATM: 32 stubs, 42 sessions
Sandwich tes(ng
A
C B D
E F H G
Level 1
Level 2
Level 3
Environments: Session 1: A, stub(B), stub(C), stub(D) S2: A, B, stub(C), stub(D) S3: A, stub(B), C, stub(D) S4: A, stub(B), stub(C), D S5: E, driver(B) S6: F, driver(B) S7: E, F, driver(B) S8: G, driver(D) S9: H, driver(D) S10: G, H, driver(D) S11: A, B, E, F, C, D, G, H
Number of stubs: 3 Number of drivers: 2 Number of sessions: 11
Fewer stubs and drivers Risk driven Small-‐bang at target level More complicated
Taget level
Poten(al problems
• Ar(ficial: Assumes correct units and interfaces • Test correct structure only • Investment in stubs and drivers • Retes(ng
Call-‐graph integra(on tes(ng
• Use the call-‐graph instead of the decomposi(on tree
• The call graph is directed • Two types of tests:
– Pair-‐wise integra(on tes(ng – Neighborhood integra(on tes(ng
• Matches well with development and builds • Tests behaviour
1
5
7
20
21
9
10
12
11
16
17 18 19
22
23 24
25 26
14 2 3 6 8
4
13
15
27
Pairwise integra(on of SATM One session per edge Real code 40 sessions, but no extra code
1
5
7
20
21
9
10
12
11
16
17 18 19
22
23 24
25 26
14 2 3 6 8
4
13
15
27
Neighborhood integra(on of SATM Integra(ng direct neighbors of nodes Number of sessions: nodes – sinknodes (a sink node has no outgoing calls) SATM: 11 sessions
Poten(al problems
• ”Small-‐bang” problems • Especially fault isola(on in large neighborhoods
• Restes(ng needed if a node is changed • Assumes correct units
Path-‐based integra(on
• Base tes(ng on system level threads • Mo(vated by overall system behaviour, not the structure
• Smooth prepara(on for System level tes(ng
Example: An MM-‐Path module A calls module B, which in turn calls module C
1
2
3 4
5
6
1
2
3
4
1
2 3
4
5
A B C
Defini(ons • Defini6on: A source node in a program is a statement fragment at
which program execu(on begins or resumes.
• Defini6on: A sink node in a program is a statement fragment at which program execu6on halts or terminates.
• Defini6on: A module execu6on path (MEP) is a sequence of statements that begins with a source node and ends with a sink node, with no intervening sink nodes.
• Defini6on: A message is a programming language mechanism by which one unit transfers control to another unit.
Extensions to Defini6ons
More defini(ons
• Defin(on: An MM-‐Path is an interleaved sequence of module execu(on paths (MEP) and messages.
• Defini(on: Given a set of units, their MM-‐Path graph is the directed graph in which nodes are module execu6on paths and edges correspond to messages and returns from one unit to another.
Defini6ons for Integra6on Tes6ng
1
2
3 4
5
6
A
Example (cont.): source nodes and sink nodes
Sink nodes
Source nodes
Example (cont.): source nodes and sink nodes
1
2
3
4
B
Example (cont.): source nodes and sink nodes
Sink nodes
Source nodes
Example (cont.): source nodes and sink nodes
1
2 3
4
5
C
Example (cont.): source nodes and sink nodes
Sink nodes
Source nodes
Example (cont.): source nodes and sink nodes
Example (cont.): source nodes and sink nodes
1
2
3 4
5
6
1
2
3
4
1
2 3
4
5
A B C
Sink nodes
Source nodes
Example (cont.): source nodes and sink nodes
Example (cont.): source nodes and sink nodes
• Module A: – Source nodes: Nodes 1 and 5 – Sink nodes: Nodes 4 and 6
• Module B: – Source nodes: Nodes 1 and 3 – Sink nodes: Nodes 2 and 4
• Module C: – Source nodes: Node 1 – Sink nodes: Node 5
Example (cont.) module execu(on path (MEP)
• MEP (A, I) = < 1, 2, 3, 6 > • MEP (A, II) = < 1, 2, 4 > • MEP (A, III) = < 5, 6 > • MEP (B, I) = < 1, 2 > • MEP (B, II) = < 3, 4 > • MEP (C, I) = < 1, 2, 4, 5 > • MEP (C, II) = < 1, 3, 4, 5 >
Crea(ng a MM-‐path graph Example (cont.) module execu6on path (MEP)
1
2
3 4
5
6
1
2
3
4
1
2 3
4
5
A B C
MEP(A,II)
MEP(A,I)
MEP(B,I) MEP(C,II)
MEP(C,I)
MEP(B,II)
MEP(A,III)
MM-‐path graph example Example (cont.): MM-‐Path graph
MEP (B, I)
MEP (C, I) MEP (A, I)
MEP (A, III)
MEP (B, II)
MEP (C, II)
MEP (A, II)
Messages!
Returns!
Test cases are selected to cover these paths
Problems
• more effort is needed to iden6fy the MM-‐Paths. This effort is probably offset by the elimina(on of stub and driver development.
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Chapter 14
System Testing
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
System Testing
• Threads ( = a system level test cases)• Basis concepts of requirements specification• Identifying threads• Metrics for system testing
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Various Views of Threads• a scenario of normal usage• a Use Case• a system level test case• a stimulus/response pair• behavior that results from a sequence of
system level inputs• an interleaved sequence of port input and
output events• a sequence of transitions in a state machine
description of the system• an interleaved sequence of object messages
and method executions• a sequence of machine instructions• a sequence of source instructions• a sequence of atomic system functions
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Some Observations
• Threads are “dynamic”• Threads occur at execution time• Threads can be identified in (or even better:
derived from) many models– Finite state machines– Decision tables– Statecharts– Petri nets– Use Cases (sufficiently detailed)
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Candidate Threads in the SimpleATM System
• Entry of a digit• Entry of a Personal Identification Number (PIN)• A simple transaction: ATM Card Entry, PIN
entry, select transaction type (deposit,withdraw), present account details (checking orsavings, amount), conduct the operation, andreport the results.
• An ATM session, containing two or moresimple transactions.
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Levels of Threads
• A unit level thread is a path in the programgraph of a unit.
• An integration level thread is an MM-Path.• There are two levels of system level threads:– A single thread– A set of interacting threads
• If necessary, we can deal with threads insystems of systems
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
System Level Threads
• An atomic system function (ASF) is an action that isobservable at the system level in terms of port input andoutput events.
• Given a system defined in terms of atomic systemfunctions, the ASF Graph of the system is the directedgraph in which nodes are atomic system functions andedges represent sequential flow.
• A source ASF is an atomic system function that appearsas a source node in the ASF graph of a system; similarly,a sink ASF is an atomic system function that appears asa sink node in the ASF graph.
• A system thread is a path from a source ASF to a sinkASF in the ASF graph of a system.
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
ASFs and MM-Paths
• MM-Paths “end” at a point of messagequiescence.
• In an event-driven system, ASFs frequently occur“between” points of event quiescence.
• There is no nice connection between ASFs andMM-Paths.
• An ASF corresponds to a “stimulus/responsepair”.– “stimulus/response cluster” is more accurate.– depending on its context, an input event may result in
several distinct output events.
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Classifying the Candidate Threads
• an MM-Path: Entry of a digit• an ASF: Entry of a Personal Identification
Number (PIN)• a thread: A simple transaction: ATM Card
Entry, PIN entry, select transactiontype (deposit, withdraw), presentaccount details (checking or savings, amount), conduct the operation, and report the results.
• a sequence An ATM session, containing twoof threads: or more simple transactions.
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Closer Look at the PIN Entry ASF
• a sequence of system level inputs and outputs:– A screen requesting PIN digits– An interleaved sequence of digit keystrokes and
screen responses– The possibility of cancellation by the customer before
the full PIN is entered– A system disposition (depending on the validity of the
PIN
• Observe:– several stimulus/response pairs– this is the cross-over point between integration and
system testing
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
E/R Model of Basis Concepts
Mainline requirements specification techniques populate some (or all ) portions of this database.
Data
Event
Action
Device Thread
1..n
1..n
1..n
1..n
1..n
1..n
input
output
occur
sequenceOf
1..n
1..n
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Data
Event
Action
Device
Thread
Structural Model
Context Model
Behavior Model
Modeling with the Basis Concepts
Condition
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
1. Card Entry
2.1 First PIN
Try
2.2 Second
PIN Try
2.3 Third PIN
Try
3. Await
Transaction
Choice
DepositBalance Withdrawal
Print Receipt
Incorrect PIN0.10
Incorrect PIN
0.10
Legitimate card 0.95
Correct PIN 0.90
Correct PIN 0.90
Correct PIN 0.90
Incorrect PIN
0.10
Button B1
0.05 Button B2
0.10
Button B3
0.85
1.001.00 Normal 0.85
Low cash 0.10
Low balance 0.05
Bad card 0.05
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Idle
Awaiting First
PIN Try
Await Transaction
Choice
Awaiting Second PIN Try
Wrong Card Display Screen S1,
Eject Card
/ Display Screen S1
Legitimate Card Display Screen S2
Correct PINDisplay Screen S5
Incorrect PIN Display Screen S3
Awaiting Third
PIN Try
Incorrect PIN Display Screen S3
Correct PIN Display Screen S5
Correct PIN Display Screen S5
Incorrect PIN Display Screen S4
Decomposition of Await PIN State(PIN Entry FSM)
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Correct Pin
2.x.10 Digits
Received
2.x.21 Digit
Received
2.x.32 Digits
Received
2.x.43 Digits
Received
2.x.6Cancel
Hit
digit / echo 'X---'
digit / echo 'XX--'
digit / echo 'XXX-'
digit / echo 'XXXX'
cancel
2.x.54 Digits
Received
cancel
cancel
cancel
Cancelled
Incorrect Pin
x1
x2
x3
x4
x5 x6
x7
x8
x9
x10 x11
Port Input Events! digit!! cancel!
Port Output Events! echo 'X---'! echo 'XX--'! echo 'XXX-'! echo 'XXXX'
Logical Output Events! Correct PIN! Incorrect PIN! Canceled
PIN Try finite state machine(PIN Try x)
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Deriving An ASF Test Case
Description:! Correct PIN on First Try
Port Event Sequence in PIN Try FSM
Port Input Event!! ! Port Output Event
! ! ! ! ! ! Screen 2 displayed with '----'1 pressed!! ! ! ! ! ! Screen 2 displayed with 'X---'2 pressed!! ! ! ! ! ! Screen 2 displayed with 'XX--'3 pressed!! ! ! ! ! ! Screen 2 displayed with 'XXX-'4 pressed!! ! ! ! ! ! Screen 2 displayed with 'XXXX'
(Correct PIN)! ! ! Screen 5 displayed
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Deriving An ASF Test Case (cont'd)
Description:! Correct PIN on First Try
Test operator instructions
Initial Conditions: screen 2 being displayed with no digit echoes
Perform the following sequence of steps:1.! Verify:! ! Screen 2 displayed with '----'2.! Cause:! ! 1 pressed!3.! Verify:! ! Screen 2 displayed with 'X---'4.! Cause:! ! 2 pressed!5.! Verify:! ! Screen 2 displayed with 'XX--'6.! Cause:! ! 3 pressed!7.! Verify:! ! Screen 2 displayed with 'XXX-'8.! Cause:! ! 4 pressed!9.! Verify:! ! Screen 2 displayed with 'XXXX'
Post Condition:! Screen 5 displayed
Test Result:!___ Pass! ! ! ___ Fail
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Slight Digression: Architecture of an Automated Test Executor
ATEProcessor Harness
PortBoundary
Cause Digit Keypress
Digit Keypress
Verifyscreen
text
screentext
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Metrics for System Testing
In the PIN Entry ASF, for a given PIN, there are 156 distinct paths from the First PIN Try state to the Await Transaction Choice or Card Entry states in the PIN Entry FSM. Of these, 31 correspond to eventually correct PIN entries (1 on the first try, 5 on the second try, and 25 on the third try); the other 125 paths correspond to those with incorrect digits or with cancel keystrokes.
To control this explosion, we have two possibilities:
•! pseudo-structural coverage metrics•! "true" structural coverage metrics
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Pseudo-structural Coverage Metrics
Behavioral models provide "natural" metrics:
Decision Table Metrics:! •! every condition! ! ! ! ! ! ! ! •! every action! ! ! ! ! ! ! ! •! every rule
FSM Metrics:! ! ! ! •! every state! ! ! ! ! ! ! ! •! every transition
Petri Net Metrics:! ! ! •! every place! ! ! ! ! ! ! ! •! every port event! ! ! ! ! ! ! ! •! every transition! ! ! ! ! ! ! ! •! every marking
These are pseudo-structural because they are just models of the eventual system.
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Pseudo-structural Coverage Metrics for PIN Try
Input Event! ! State Sequence! ! Sequence1,2,3,4! ! ! 2.x.1, 2.x.2, 2.x.3, 2.x.4, 2.x.51,2,3,5! ! ! 2.x.1, 2.x.2, 2.x.3, 2.x.4, 2.x.5C! ! ! ! 2.x.1, 2.x.61,C! ! ! ! 2.x.1, 2.x.2, 2.x.61,2,C! ! ! 2.x.1, 2.x.2, 2.x.3, 2.x.61,2,3,C! ! ! 2.x.1, 2.x.2, 2.x.3, 2.x.4, 2.x.6
Two test cases yield state coverage
Input Event! ! Path of Sequence! ! Transitions
1,2,3,4! ! ! x1, x2, x3, x4, x51,2,3,5! ! ! x1, x2, x3, x4, x6C! ! ! ! x7, x111,C! ! ! ! x1, x8, x111,2,C! ! ! x1, x2, x9, x111,2,3,C! ! ! x1, x2, x3, x10, x11
All six are needed for transition coverage
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Pseudo-structural Coverage Metrics for PIN Entry FSM
Input Event! ! State Sequence! ! Sequence
1,2,3,4! ! !
1,2,3,5,1,2,3,4! ! !
1,2,3,5,C,1,2,3,4
C,C,C! ! ! !
! ! !
How many test cases yield state coverage?
Input Event! ! Path of Sequence! ! Transitions
1,2,3,4! ! !
1,2,3,5,1,2,3,4! ! !
1,2,3,5,C,1,2,3,4
C,C,C! !
How many are needed for transition coverage?
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Consequences of Pseudo-structural Coverage Metrics
1.! Combinatoric explosion is controlled! Selecting test cases from the FSM decomposition reduced
! 156 threads to 10 test cases
2.! Fault isolation is improved! When a "verify" operation fails, use the FSMs to determine:! •! what went wrong! •! where it went wrong! •! when it went wrong
3.! Base information for testing management
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
SATM System Threads
1.! Insertion of an invalid card. (this is probably the "shortest" system thread)
2.! Insertion of a valid card, followed by three failed PIN entry attempts.
3.! Insertion of a valid card, a correct PIN entry attempt, followed by a balance inquiry.
4.! Insertion of a valid card, a correct PIN entry attempt, followed by a deposit.
5.! Insertion of a valid card, a correct PIN entry attempt, followed by a withdrawal.
6.! Insertion of a valid card, a correct PIN entry attempt, followed by an attempt to withdraw more cash than the account balance.
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
SATM System Thread Testing
SATM Test Data
ATM ! PAN! Expected! Checking! SavingsCard! ! ! PIN!! ! Balance!! Balance1! ! 100!! 1234! ! $1000.00! $800.002! ! 200!! 4567! ! $100.00! ! $90.003! ! 300!! 6789! ! $25.00! ! $20.004! ! (invalid)
Port Input Events! ! ! ! Port Output Events
•! Insert ATM Card (n)! ! ! •! Display Screen(n,text)•! Key Press Digit (n)! ! ! •! Open Door(dep, withdraw)•! Key Press Cancel! ! ! •! Close Door(dep, withdraw)•! Key Press Button B(n)! ! •! Dispense Notes (n)•! Insert Deposit Envelope! ! •! Print Receipt (text)! ! ! ! ! ! ! ! •! Eject ATM Card
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Thread 1 Test Procedure
Description: invalid card Test operator instructions Initial Conditions: screen 1 being displayed Perform the following sequence of steps: 1. Cause: Insert ATM Card 4 2. Verify: Eject ATM Card 3. Verify: Display Screen(1, null) Post Condition: Screen 1 displayed Test Result: ___ Pass ___ Fail
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Thread 2 Test ProcedureDescription: valid card, 3 failed PIN attempts
Initial Conditions: screen 1 being displayedPerform the following sequence of steps:
1. Cause: Insert ATM Card 12. Verify: Display Screen(2, '----')1. Cause: Key Press Digit (1)2. Verify: Display Screen(2,'X---')1. Cause: Key Press Cancel2. Verify: Display Screen(2,'----')1. Cause: Key Press Digit (1)2. Verify: Display Screen(2,'X---')1. Cause: Key Press Digit (2)2. Verify: Display Screen(2,'XX--')1. Cause: Key PressCancel2. Verify: Display Screen(2,'----')1. Cause: Key Press Digit (1)2. Verify: Display Screen(2,'X---')1. Cause: Key Press Digit (2)2. Verify: Display Screen(2,'XX--'1. Cause: Key Press Digit (3)2. Verify: Display Screen(2,'XXX-')1. Cause: Key Press Digit (5)2. Verify: Display Screen(2,'XXXX')3. Verify: Display Screen(4, null)3. Verify: Display Screen(1, null)
Post Condition: Screen 1 displayed Test Result: ___ Pass___ Fail
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Thread 3 Test ProcedureDescription: valid card, a correct PIN entry attempt, followed by a balance inquiry of the checking account. Initial Conditions: screen 1 being displayed Perform the following sequence of steps: 1. Cause: Insert ATM Card 1 2. Verify: Display Screen(2, '----') 3. Cause: Key Press Digit (1) 4. Verify: Display Screen(2,'X---') 5. Cause: Key Press Digit (2) 6. Verify: Display Screen(2,'XX--' 7. Cause: Key Press Digit (3) 8. Verify: Display Screen(2,'XXX-') 9. Cause: Key Press Digit (4) 10. Verify: Display Screen(2,'XXXX') 11. Verify: Display Screen(5, null) 12. Cause: Key Press Button B(1) 13. Verify: Display Screen(6, null) 14. Cause: Key Press Button B(1) 15. Verify: Display Screen(14,null) 16. Cause: Key Press Button B(2) 17. Verify: Print Receipt ('$1000.00') 18. Verify: Display Screen(15, null) 19. Verify: Eject ATM Card 20. Verify: Display Screen(1, null) Post Condition: Screen 1 displayed Test Result: ___ Pass ___ Fail
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Thread 4 Test ProcedureDescription: valid card, a correct PIN entry attempt, followed by a $25.00 deposit to the checking account 1. Cause: Insert ATM Card 1 2. Verify: Display Screen(2, '----') 3,5,7,9. Cause: Key Press Digit (1,2,3,4) 4,6,8,10. Verify: Display Screen(2,'XXXX') 11. Verify: Display Screen(5, null) 12. Cause: Key Press Button B(2) 13. Verify: Display Screen(6, null) 14. Cause: Key Press Button B(1) 15. Verify: Display Screen(7, '$----.--') 16. Cause: Key Press Digit (2) 17. Verify: Display Screen(7, '$----.-2') 18. Cause: Key Press Digit (5) 19. Verify: Display Screen(7, '$----.25') 20. Cause: Key Press Digit (0) 21. Verify: Display Screen(7, '$---2.50') 22. Cause: Key Press Digit (0) 23. Verify: Display Screen(7, '$--25.00' 24. Verify: Display Screen(13, null) 25. Verify: Open Door(deposit) 26. Cause: Insert Deposit Envelope 27. Verify: Close Door(deposit) 28. Verify: Display Screen(14, null) 29. Cause: Key Press Button B(2) 30. Verify: Print Receipt ('$1025.00') 31. Verify: Display Screen(15, null) 32. Verify: Eject ATM Card 33. Verify: Display Screen(1, null) Post Condition: Screen 1 displayed Test Result: ___ Pass ___ Fail
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Thread 5 Test Procedure
Description: valid card, a correct PIN entry attempt, followed by an attempt to withdraw more cash than the savings account balance. 1. Cause: Insert ATM Card 2 2. Verify: Display Screen(2, '----') 3,5,7,9. Cause: Key Press Digit (4,5,6,7) 4,6,8,10. Verify: Display Screen(2,'XXXX') 11. Verify: Display Screen(5, null) 12. Cause: Key Press Button B(3) 13. Verify: Display Screen(6, null) 14. Cause: Key Press Button B(2) 15. Verify: Display Screen(7, '$----.--') 16,18,20,22,24 Cause: Key Press Digit (1,1,0,0,0) 17,19,21,23,25 Verify: Display Screen(7, '$-110.00') 24. Verify: Display Screen(8, '----.--') 26. Cause: Key Press Cancel 28. Verify: Display Screen(14, null) 29. Cause: Key Press Button B(2) 30. Verify: Print Receipt ('$90.00') 31. Verify: Eject ATM Card 32. Verify: Display Screen(1, null) Post Condition: Screen 1 displayed Test Result: ___ Pass ___ Fail
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Operational Profiles
Zipf's Law: 80% of the activities occur in 20% of the space
•! productions of a language syntax•! natural language vocabulary•! menu options of a commercial software package•! area of an office desktop•! floating point divide on the Pentium chip
For Threads: a small fraction of all possible threads represents the majority of system execution time.
Therefore: find the occurrence probabilities of threads and use these to order thread testing.
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
1. Card Entry
2.1 First PIN
Try
2.2 Second
PIN Try
2.3 Third PIN
Try
3. Await
Transaction
Choice
DepositBalance Withdrawal
Print Receipt
Incorrect PIN0.10
Incorrect PIN
0.10
Legitimate card 0.95
Correct PIN 0.90
Correct PIN 0.90
Correct PIN 0.90
Incorrect PIN
0.10
Button B1
0.05 Button B2
0.10
Button B3
0.85
1.001.00 Normal 0.85
Low cash 0.10
Low balance 0.05
Bad card 0.05
Common Thread Probabilities Legitimate Card 0.95 PIN ok 1st try 0.90 Withdraw 0.85 Normal 0.85 0.6177375 Rare Thread Probabilities Legitimate Card 0.95 Bad PIN 1st try 0.10 Bad PIN 2nd try 0.10 PIN ok 3rd try 0.90 Withdraw 0.85 Low Cash 0.01 0.00072675
Operational Profiles of SATM
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
SATM Atomic System Functions
ASF1: Examine ATM Card Inputs: PAN from card, List of acceptable cards Outputs: Legitimate Card, Wrong Card
ASF2: Control PIN Entry Inputs: Expected PIN, Offered PIN Outputs: PIN OK, Wrong PIN
ASF3: Get Transaction Type Inputs: Button1, Button2, or Button3 depressed Outputs: call Get Account Type (not externally visible)
ASF4: Get Account Type Inputs: Button1 or Button2 depressed Outputs: call one of Process Withdrawal, Process Deposit, or Display Balance (not externally visible)
ASF5: Process Withdrawal Inputs: Amount Requested, Cash Available, Local Balance Outputs: Process Request (call Dispense Cash) Reject Request (insuficient funds or insuficient balance)
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
SATM Atomic System Functions
ASF6: Process Deposit Inputs: Deposit Amount , Deposit Envelope, Deposit Door Status, Local Balance Outputs: Process Request (call Credit Local Balance) Reject Request
ASF7: Display Balance Inputs: Local Balance Outputs: (call Screen Handler) ASF8: Manage Session Inputs: New Transaction Requested, Done Outputs: (call Get Transaction Type or call Print Receipt)
ASF9: Print Receipt Inputs: Account Number, transaction type and amount, new local balance, time, date Outputs: formatted text for receipt, (call Eject Card)
ASF10: Eject Card Inputs: (invoked) Outputs: (control rollers)
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
"Hidden" Atomic System Functions
ASF11: Dispense $10 Note
ASF12: Screen Handler
ASF13: Button Sensor
ASF14: Digit Keypad Sensor
ASF15: Cancel Sensor
ASF16: Card Roller Controller
ASF17: Control Deposit Door
ASF18: Control Deposit Rollers
ASF19: Control Cash Door
ASF20: Count Cash on Hand
ASF21: Timer
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
SATM Threads
Thread 1: Wrong Card State Sequence: Idle, Idle Event Sequence: Display Screen 1, Wrong Card ASF Sequence:!
Thread 2: Wrong PIN State Sequence: Idle, Await 1st PIN Try, Await 2nd PIN Try, Await 3rd PIN Try, Idle Event Sequence: Display Screen 1, Legitimate Card, Wrong PIN, Wrong PIN, Wrong PIN, Display Screen 4, Display Screen 1 ASF Sequence:!
Thread 3: Balance Inquiry State Sequence: Idle, Await 1st PIN Try, Acquire Transaction Data, Display Balance, Display Balance, Close Session, Idle Event Sequence: Display Screen 1, Legitimate Card, Wrong PIN, Wrong PIN, Wrong PIN, Display Screen 4, Display Screen 1 ASF Sequence:!
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Thread 4: Balance then Deposit then Withdraw State Sequence: Idle, Await 1st PIN Try, Acquire Transaction Data, Display Balance, Close Session, Acquire Transaction Data, Process Deposit, Close Session, Acquire Transaction Data, Process Withdrawal, Close Session, Idle Event Sequence: Display Screen 1, Legitimate Card, Wrong PIN, Wrong PIN, Wrong PIN, Display Screen 4, Display Screen 1 ASF Sequence:!
Thread 5: 3 PIN Tries then Balance then Deposit then Withdraw State Sequence: Idle, Await 1st PIN Try, Await 2nd PIN Try, Await 3rd PIN Try, Acquire Transaction Data, Display Balance, Close Session, Acquire Transaction Data, Process Deposit, Close Session, Acquire Transaction Data, Process Withdrawal, Close Session, Idle Event Sequence: Display Screen 1, Legitimate Card, Wrong PIN, Wrong PIN, Wrong PIN, Display Screen 4, Display Screen 1 ASF Sequence:!
SATM Threads
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
ATM States and Transitions
S1
S2 S3 S4
S5
S6 S7 S8
S9
t1
t2
t3 t4
t5
t6 t7 t8
t9t10 t11
t12t13 t14
t15
t16
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Chapter 15
Interaction Testing
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Interaction: the hot topic of the 1990s
• "Elements of Interaction"Communications of the ACM, (Jan. 93)Robin Milner (1993 Turing Award Lecture)
• "An Investigation of the Therac-25 Accident“, IEEE Computer,(July 93) Nancy G. Leveson and Clark S. Turner.
• "Feature Interactions and Formal Specifications inTelecommunications" IEEE Computer, (Aug. 93) Pamela Zave
• The underlying issue: Non-Determinism• Our Approach:
– A Taxonomy of Interactions– An executable specification– Analysis from graph theory
Software Testing: A Craftsman’s Approach, 3rd Edition System Testing
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Definitions of Interaction
1. "System behavior as a whole does not satisfy the separate specifications of all its [parts]." (Pamela Zave) 2. Relationship between the whole and its parts. 3. Totality of connections among components. 4. Consequences of connections among components. 5. The result of composition. (Robin Milner)
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Feature InteractionFeature: a service provided by a system (activated by a subscriber paying for the service) Telephony Examples (from P. Zave): Notation: d1, d2, d3, and d4 are directory numbers • POTS (Plain Old Telephone Service) • Call Forwarding: calls to d1 are terminated on d2 iff call forwarding is enabled on d1 and activated by defining d2 as the current destination • Calling Party Identification: when active on d2, d2 receives the directory number of all incoming calls • Call Rejection: allows a subscriber to define a list of directory numbers from which calls will not be completed. • Busy Treatment: several possibilities; call override, call rejection, call forward, call forward on busy, do not disturb, automatic re-call
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
(Zaveʼs) Sample Interactions1. Calling Party Identification and Call Rejection calling party identification offers a directory number (data) which is used as an input in the call rejection process. 2. Call Forwarding and Call Rejection d2 rejects calls from d1, d1 forwards calls to d2, d3 calls d1. 3. Call Forward Loop d1 forwards calls to d2 d2 forwards calls to d3 d3 forwards calls to d1 d4 calls d1 4. Voice Mail and Credit Card Calling • In credit card calling, # terminates a call so that a new call can be dialed without re-entering the credit card digits. • In many Voice Mail systems, # is a command (e.g., to hear your recorded message) • d1 makes a credit card call to d2 and gets d2's voice mail. What happens when d1 enters # ?
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
First Clues• Feature interaction is a consequence of adaptive maintenance (i.e., adding new capabilities to an existing system) • Interactions involve connections, and the essence of every modeling technique is to find/establish connections (see Wurmann, Information Anxiety ) • Composition creates connections. What can be connected? • First Approximation:
interaction Should be Should not be
Is
Is not
intended unintended
missing null
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Definitions of Determinism
Let C be some calculation, C (input) = output .
Definition 1:! If the result of C can always be predicted, C is deterministic! ! ! (or pre-determined), otherwise C is non-deterministic.
Definition 2:! If the result of C is always the same, C is deterministic! ! ! (or pre-determined), otherwise C is non-deterministic.
Example: ComputeSalesTax(price, taxrate)
If the tax rate is 4%, sales tax on a $100 item will be $4.00.
If the legislature changes the tax rate to 6%, we have interaction.
If we understand all the points of interaction, the function still is deterministic.
"Concurrency inflicts non-determinism." Robin Milner
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Toward a Taxonomy of InteractionsWhat elements can interact? In what ways can these elements interact? "Basis" system elements (the reality of any system):
Data
Event
Action
Device Thread
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Basis Concepts for Requirements Specification1! DataWhen a system is described in terms of its data, the focus is on the information used and created by the system. Data refers to information that is either initialized, stored, updated, or (possibly)
destroyed.
2! ActionsActions have inputs and outputs, and these can be either data or port events. Some methodology-specific synonyms for actions: transform, data transform, control transform, process, activity, task, method, and service. Actions can be decomposed into lower level actions.
3! DevicesEvery system has devices; these are the sources and destinations of system level inputs and outputs (events that occur at the port boundary). Physical actions (e.g., keystrokes, light emissions from a screen) occur on port devices, and these are translated from physical to logical (or logical to physical)
appearances by actions that execute on other devices (e.g., a CPU executing software).
4! EventsA system level input (or output) that occurs on a port device. Like data, events can be inputs to or outputs of actions. Events can be discrete (such as keystrokes) or they can be continuous (such as temperature, altitude, or pressure). There are situations where the context of present data values changes the logical meaning of physical events. We refer to such situations as "context sensitive port events".
5! ThreadsA thread an instance of execution-time behavior of a system. Two synonyms: a scenario, a use-case. A thread is a sequence of actions, and these in turn have data and events as their inputs and outputs.
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Connections (interactions) Among Basis Concepts
Data
Action
Event
Thread
Data Action Device Event Thread
Square ofOpposition
I/O,usage
contextsensitivity
points ofn-connection
resourcecontention
incidence
timing, raceconditions
timing,concurrency
n-connectivity
points ofn-connection
I/O,usage
contextsensitivity
points ofn-connection
points ofn-connection
incidence
Device
execution
execution
I/O,usage
I/O,usage
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Toward a Taxonomy of Interactions
•! Each element needs another "dimension"; we'll call it Location, where location refers to time and position.
•! Each of these elements can interact with itself.
•! Interactions among pairs of elements are more interesting.
Data
Event
Action
Device Thread
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Some Ground RulesViews of location: • a point in space-time • something that happens in a processor (position) 1. For now, a processor is something that executes threads, or a device where events occur. 2. Since threads execute, they have a strictly positive time duration. 3. In a single processor, two threads cannot execute simultaneously. 4. Events have a strictly positive time duration. 5. Two (or more) input events can occur simultaneously, but an event cannot occur simultaneously in two (or more) processors. 6. In a single processor, two output events cannot begin simultaneously.
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Taxonomy of InteractionsStatic interaction: independent of time Dynamic interaction: time dependent Each of the five "basis elements" can interact in each quadrant:
static dynamic
single processor
multiple processor
We will consider static interactions of data with data, anddynamic interactions of threads with threads.
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Static Interactions (Known to Aristotle)
Interactions in the Square of Opposition. 1. Contradictories: exactly one is true 2. Contraries: cannot both be true 3. Sub-contraries: cannot both be false 4. Subalternation: Truth of superaltern guarantees truth of its subaltern Examples 1. When the pre-condition for a thread is a conjunction of data propositions, contrary or contradictory data values will prevent thread execution. 2. Context sensitive port input events usually involve contradictory data. 3. Case statement clauses are contradictories. 4. Rules in a decision table are contradictories.
All S is P No S is P
Some S is P Some S is not P
contraries
contradictories
sub-contraries
subalternation subalternation
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Static Interactions in Multiple Processors
The "Call Forwarding Loop" in telephony.
Background: when a subscriber defines a call forwarding destination, this becomes call ro¨ting data local to the subscribers telephone office.
Suppose Subscriber A (in Grand Rapids) forwards calls to Subscriber B in Northbrook, and
Subscriber B (in Northbrook) forwards calls to Subscriber C in Phoenix, and
Subscriber C (in Phoenix) forwards calls to Subscriber A inGrand Rapids.
What happens when someone outside this loop calls one of A, B, or C?
Observations:
1.! The call forwarding data is locally correct, but globally, it is contrary.
2.! The global contrary condition is afault, not a failure.
3.! The fault only becomes a failure when a thread creates a dynamic interaction.
4.! (Most telephone systems avoid this by refusing to forward a forwarded call.)
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
n-Connectedness
Linear Graph Theory sheds some light on interaction.
0-connected
ji
1-connected
ji
ji
2-connected
j
i
3-connected
j
i
Two kinds of faults:
Missing n-connectedness occurs when a pair lacks an essential connection, andInappropriate n-connctedness, when a pair has an undesired connection.
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Interpretations of n-Connectedness
0-Connected: true independence
1-Connected: (by an ancestor) resource conflict, context sensitivity (by a descendant) ambiguous cause
2-Connected: define-reference, enable, disable, precedence, prohibit prerequisite
3-Connected: mutual influence, repetition, deadlock
Examples:
Failures under the "single failure" assumption of Reliability Theory
Context Sensitive Port Events
Falkland Island submarine incident
Email loop
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Data with Data
0-Connected! data that are independent
1-Connected! data that are inputs to the same action
2-Connected! data that are sub-alternates,! ! ! ! data that are used in a computation
3-Connected! data that are contraries, contradictories, or sub-contraries,! ! ! ! "deeply" related data, as in iteration or semaphores
Threads with Threads
•! Threads can be n-connected with each other in two ways: via events! and/or via data.•! To explore this, we need a sufficiently expressive notation.
My candidate: Event Driven Petri Nets
Interactions Based on n-Connectedness
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Elements of the Interaction Taxonomy
1.! Basis concept interactions
! data-data! ! ! port-port! data-action! ! ! port-event! data-event! ! ! event-event! data-thread! ! ! event-thread! action-action! ! thread-thread! action-event2.! Refined by appropriate relationship (e.g., n-connectivity, square! of opposition)3.! Further refined by Is/Should modality
Placement of Zave's examples
1.! Calling Party Identification and Call Rejection! Intended thread-thread 2-connectivity2.! Call Forwarding and Call Rejection! Unintended data-data contraries3.! Call Forward Loop! Unintended thread-thread 3-connectivity4.! Voice Mail and Credit Card Calling! Unintended data-event context sensitivity
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
An Event-Driven Petri Net is a tripartite directed graph (P, D, S, In, Out) composed of three sets of nodes, P, D, and S, and two mappings, In and Out, where • P is a set of port events • D is a set of data places • S is a set of transitions • In is a set of ordered pairs from (P∪ D) × S • Out is a set of ordered pairs from S × (P ∪ D) Event-Driven Petri Nets express four of our five basic system constructs; only devices are missing. In an Event-Driven Petri Net , the external inputs are the places with indegree = 0, and the external outputs are the places with outdegree = 0. Graphical conventions for Event-Driven Petri Nets
A Petri Net Model for Interactions
port event data place
transition
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Composing Event-driven Petri Nets
Given two (or more) EDPNs, their composition (unique!) is obtained as follows:
1. If any places or port events have synonymous names, collapse these into unique names. (For example, collapse Lamp ON and Light ON into Lamp ON)
2. Consider all data places and port events to be global (if a place appears in two individual nets, it need only appear once in their composition.)
3. Consider actions to be local. (This preserves the execution sequence of the individual threads.)
4. Adjust all input and output relations (arrows) to preserve the original input and output relations.
5. It may be necessary to add transitions and input/output edges to represent interactions that were not captured in the prior components.
6. Trace through some mainline threads to convince yourself that the composition (especially interactions) is accurate.
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Thread Interaction and CompositionWhen two threads interact (via an event or data), we can always compose their respective EDPNs.
p1 d1
s1
p3
d3
s2
d4
p1 d2
s3
p3 d4
p1
s1
d3
s2
d1 d2
s3
p3
d4
Thread 1 Thread 2 Composition
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
t1
t2
t3
t4
t5
t6
p1 p2
p3
p4
p5
p6
p7
t1
t2
p1 p2
p3
t5
t6
p6
p7
p3
t3
t4
p4
p5
p3
Producer/Consumers Composition
Producer
Consumer 1 Consumer 2
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
The windshield wiper on the Saturn automobile (at least on the 1992 models) is controlled by a lever with a dial. The lever has four positions, OFF, INT (for intermittent), LOW, and HIGH, and the dial has three positions, numbered simply 1, 2, and 3. The dial positions indicate three intermittent speeds, and the dial position is relevant only when the lever is at the INT position. The decision table below shows the windshield wiper speeds (in wipes per minute) for the lever and dial positions.
Saturn Windshield Wiper Controller
Lever! ! OFF!! INT! INT! INT!LOW! HIGHDial!! ! n/a! ! 1! 2! 3! n/a! ! n/a
Wiper! ! 0! ! 4! 6! 12! 30! ! 60
Input Event! Description! ie1! ! lever from OFF to INT! ie2! ! lever from INT to LOW! ie3! ! lever from LOW to HIGH! ie4! ! lever from HIGH to LOW! ie5! ! lever from LOW to INT! ie6! ! lever from INT to OFF! ie7! ! dial from 1 to 2! ie8! ! dial from 2 to 3! ie9! ! dial from 3 to 2! ie10! ! dial from 2 to 1
Output Event! Description! oe1! ! ! 0 w.p.m.! oe2! ! ! 4 w.p.m.! oe3! ! ! 6 w.p.m.! oe4! ! ! 12 w.p.m.! oe5! ! ! 30 w.p.m.! oe6! ! ! 60 w.p.m.
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
OFF
HIGH
LOW
INT
ie1 ?
ie2oe5
ie3oe6
ie4oe5
ie5?
ie6oe1
Lever
1
3
2
ie7?
ie8?
ie10?
ie9?
Dial
Saturn Windshield Wiper FSMs
Input Event! Description! ie1! ! lever from OFF to INT! ie2! ! lever from INT to LOW! ie3! ! lever from LOW to HIGH! ie4! ! lever from HIGH to LOW! ie5! ! lever from LOW to INT! ie6! ! lever from INT to OFF! ie7! ! dial from 1 to 2! ie8! ! dial from 2 to 3! ie9! ! dial from 3 to 2! ie10!! dial from 2 to 1
Output Event! Description! oe1!! ! 0 w.p.m.! oe2!! ! 4 w.p.m.! oe3!! ! 6 w.p.m.! oe4!! ! 12 w.p.m.! oe5!! ! 30 w.p.m.! oe6!! ! 60 w.p.m.
Note that several transition actions are indeterminate because of composition.
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
p1
d3
d2
d1
s1
s2 s5
s6
p6
p2 p5
p15
d4
s3 s4
p3 p4
p16 p15
p11
OFF
HIGH
LOW
INT
ie1 ?
ie2 oe5
ie3 oe6 ie4
oe5
ie5 ?
ie6 oe1
Lever Input Event Description p1 ie1 lever from OFF to INT p2 ie2 lever from INT to LOW p3 ie3 lever from LOW to HIGH p4 ie4 lever from HIGH to LOW p5 ie5 lever from LOW to INT p6 ie6 lever from INT to OFF p7 ie7 dial from 1 to 2 p8 ie8 dial from 2 to 3 p9 ie9 dial from 3 to 2 p10 ie10 dial from 2 to 1 Output Event Description p11 oe1 0 w.p.m. p12 oe2 4 w.p.m. p13 oe3 6 w.p.m. p14 oe4 12 w.p.m. p15 oe5 30 w.p.m. p16 oe6 60 w.p.m. Data Place Description d1 Lever at OFF position d2 Lever at INT position d3 Lever at LOW position d4 Lever at HIGH position d5 Dial at position 1 d6 Dial at position 2 d7 Dial at position 3
Deriving an EDPN from a State Machine
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
p1
s1
d3
s2
d1
p3
d4
d2
s3
Let T1 and T2 be two EDPN threads in which synonym places have been resolved, and with external input and output sets EI1, EI2, EO1, and EO2. Furthermore, let T be the composition of threads T1 and T2, where EI = EI1∩ EI2 and EO = EO1 ∩ EO2 are the external input and output sets of the composed thread T. The threads T1 and T2 are: •0-connected if EI1 ∩ EI2 = ∅, EO1 ∩ EO2 = ∅, EI1 ∩ EO2 = ∅, and EO1 ∩ EI2 = ∅ •1-connected if either EI ≠ ∅ or EO ≠ ∅, •2-connected if either EI1 ∩ EO2 ≠ ∅ or EI2 ∩ EO1 ≠ ∅, •3-connected if both EI1 ∩ EO2 ≠ ∅ and EI2 ∩ EO1 ≠ ∅
n-Connected Thread Interaction
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
p1
d3
d2
d1
s1
s2 s5
s6
p6
p2
p15
d4
s3 s4
p3 p4
p16 p5
p11p7
s7
s8 s9
s10
p10
p8 p9
p12
p5
d6
d5
d7
p13
p14
s11
s12
s13
Full EDPN for the Saturn Windshield Wiper
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
Composition in a Database(all relations are 0..n at each end)
Data
Event
Device
Action Thread
DataInput
DataOutput
EventInput
EventOutput
OccursOn
SequenceOf
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
DataInput Table for d3
LowToInts5Lowd3LowtoHighs3Lowd3ActionNameActionIDDataNameDataID
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
DataInput Table for d2
Intermittentd2
Intermittentd2
Intermittentd2
Intermittentd2
Intermittentd2
ActionNameActionIDDataNameDataID
Software Testing: A Craftsman’s Approach, 3rd Edition Interaction Testing
What Connections Can you Identify Between anEDPN Data Model and Graph Theory?
• Indegree of a place
• Outdegree of a place
• Indegree of a transition
• Outdegree of a transition
• Indegree of an event
• Outdegree of an event
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Chapters 16 - 20
Testing Object-Oriented Software
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Object-Oriented Testing
1. Traditional vs. Object-Oriented Testing 2. Saturn Windshield Wiper Example 3. Testing with O-O Notations
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Traditional vs. Object-Oriented Testing
Data
Event
Action
Device Thread
input
output
Data
Event
Action
Device
Thread
encapsulate
Object
Object-orientation repackages the basis concepts (all relations are 0..n)
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Issues in Object-Oriented Testing
1. Implications of message-based communication With an procedural language, program graphs are "natural". O-O testing must deal with event and message quiescence. 2. Decomposition to composition. Functional decomposition tree as a basis of integration testing is lost. Composition implies unknowable contexts. We can never know all the possible objects with which a given object may be composed. 3. O-O language packaging requires redefinition of testing levels.
What is a unit? Can we continue with integration level
constructs (MM-Path, ASF)? What do O-O threads look like?
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Levels of Object-Oriented Testing
Level Unit Integration System
Item Method of an object? Class? MM-Path Atomic System Function Thread Thread Interaction
Boundary Program graph Message quiescence Event quiescence Source to sink ASF (none)
Notice the cascading levels of interaction: • unit testing covers statement interaction, • MM-Path testing covers method interaction, • ASF testing covers MM-Path interaction, • thread testing covers object interaction, and all of this culminates in • thread interaction.
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Re-usable Testing Techniques
Level Unit Integration System
Item Method of an object Class MM-Path Atomic System Function Thread
Thread Interaction
Technique Traditional functional and/or structural StateChart-based New definition? New definition? New definition? (StateCharts) (as before)
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
The windshield wiper on the Saturn automobiles is controlled by a lever with a dial. The lever has four positions, OFF, INT (for intermittent), LOW, and HIGH, and the dial has three positions, numbered simply 1, 2, and 3. The dial positions indicate three intermittent speeds, and the dial position is relevant only when the lever is at the INT position. The decision table below shows the windshield wiper speeds (in wipes per minute) for the lever and dial positions.
Saturn Windshield Wiper Controller
Lever OFF INT INT INT LOW HIGH Dial n/a 1 2 3 n/a n/a Wiper 0 4 6 12 30 60
Input Event Description ie1 lever from OFF to INT ie2 lever from INT to LOW ie3 lever from LOW to HIGH ie4 lever from HIGH to LOW ie5 lever from LOW to INT ie6 lever from INT to OFF ie7 dial from 1 to 2 ie8 dial from 2 to 3 ie9 dial from 3 to 2 ie10 dial from 2 to 1
Output Event Description oe1 0 w.p.m. oe2 4 w.p.m. oe3 6 w.p.m. oe4 12 w.p.m. oe5 30 w.p.m. oe6 60 w.p.m.
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Wiper
Speed Lever Position Dial Position
Compute Speed
Lever
position
Sense Up Sense Down
Dial
Sense Left Sense Right
position
Saturn Windshield Wiper Objects
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
OFF
HIGH
LOW
INT
ie1 ?
ie2 oe5
ie3 oe6 ie4
oe5
ie5 ?
ie6 oe1
Lever
1
3
2
ie7 ?
ie8 ?
ie10 ?
ie9 ?
Dial
Saturn Windshield Wiper Object FSMs
Input Event Description ie1 lever from OFF to INT ie2 lever from INT to LOW ie3 lever from LOW to HIGH ie4 lever from HIGH to LOW ie5 lever from LOW to INT ie6 lever from INT to OFF ie7 dial from 1 to 2 ie8 dial from 2 to 3 ie9 dial from 3 to 2 ie10 dial from 2 to 1
Output Event Description oe1 0 w.p.m. oe2 4 w.p.m. oe3 6 w.p.m. oe4 12 w.p.m. oe5 30 w.p.m. oe6 60 w.p.m.
Note that several transition actions areindeterminate because of composition.
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
OFF
HIGH
LOW
INT
ie1 m1
ie2 m2
ie3 m3 ie4
m4
ie5 m5
ie6 m6
Lever
1
3
2
ie7 m7
ie8 m8
ie10 m10
ie9 m9
Dial
0 wpm
60 wpm
30 wpm
4 wpm
Wiper
6 wpm 12 wpm
m1m1
m1
m6 m6 m6
m2 m2 m2
m5m5
m5
m3m6
m7 m8
m9m10
Resolving the Indeterminacy with Messages
Messages m1 to m6 inform the Wiper object of the state of the Lever object. Messages m7 to m10 inform the Wiper object of the state of the Dial object.
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
instate(Int)
Instate(1)
60
30
12
6
4
0
instate(Int)instate(Low)
instate(Off)
instate(High)instate(Low)
Instate(2)
Instate(3)
Instate(1)
Instate(2)
Instate(2)
Instate(3)
leverUpleverDown
Off
Int
Low
High
dialUpdialDown
3
2
1
dialUp
dialDown
power on
power off
leverDown
leverDownleverUp
leverUp
Windshield Wiper StateChart
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
• definition in UML • two kinds of o-o software: - data-driven - event-driven • how are these described (for a tester)? • what is an o-o unit: - a class? - a method? • what is the basis for integration testing?
Testing Object-Oriented Software
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
First Example (data-driven)
The o-oCalendar program is an object-oriented implementation of the NextDate function. (NextDate(Mar., 5, 2002)) = Mar., 6, 2002. When this is implemented in procedural code, it is approximately 50 lines long, with a cyclomatic complexity less than 15. (can be 11.) A "pure" (i.e., good practice) object-oriented implementation contains one abstract class and five classes. The next few slides show the UML description of o-oCalendar.
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
CalendarUnit 'abstract class currentPos As Integer CalendarUnit( pCurrentPos) setCurrentPos( pCurrentPos) increment() 'boolean
Day Month m Day( pDay, Month pMonth) setCurrentPos( pCurrentPos) setDay( pDay, Month pMonth) getDay() increment()
Month private Year y private sizeIndex = <31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31> Month(pcur, Year pYear) setCurrentPos(pCurrentPos) setMonth(pcur, Year pYear) getMonth() getMonthSize() increment()
Year Year(int pYear) setCurrentPos( pCurrentPos) getYear() increment() isleap() 'boolean
testIt Date Day d Month m Year y Date( pDay, pMonth,, pYear) increment() printDate()
Classes
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
CalendarUnit
Day Month Year
Date
Day Month Year
Inheritance and Aggregation
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
3
2
1testIt
4
5
6
7
Date.constructor Date.increment19
20
Date.printDate
10
8
9
11
12 15
1613
14
17
18
Program Graphs of Date Methods
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Program Graphs of Day Methods
25
24
23
Day.setDay
26
27
Day.getDay
29
28
30
31 32
33
Day.incrementDay.constructor
23
21
22
a
b
Day.setCurrentPos
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Program Graphs of Month Methods
40
39
39
Month.constructora
b
Month.setCurrentPos
38
37
36
Month.setMonth
40
39
Month.getMonth Month.getMonthSize
48
47
49
50 51
52
Month.increment41
44
42
43
46
45
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Program Graphs of Year Methods
54
53
Year.constructor
b
a
Year.setCurrentPos
55
56
Year.getYear
57
58
59
Year.increment
61
62 63
64
60
Year.isLeap
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Month Date
Year
Day
testIt
1: create 2: increment 3: printDate
1: create 2: increment 3: getYear
1: create 2: increment 3:setMonth 4: getMonth
1: create 2: increment 3: setDay 4: getDay
1:getMonthSize
1:isLeap
Collaboration Diagram
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
testit
Main
Date
Date()
increment()
printDate
Day
setCurrentPos()
Day()
increment()
setDay
getDay
Month
setCurrentPos()
Month()
increment()
setMonth
getMonth
getMonthSize
Year
setCurrentPos()
Year()
increment()
isLeap
getYear
m1
m2
m3
m15
m16
m17
m18
m19
m20
m21
m6
m7
m11m13
m5
m8
m10
m4
m9
m14
All Message Flows
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
testIt
1: printDate()
2:getMonth()
3:getDay()
4:getYear()
Date:testdate Month:m Year:yDay:d
time
Message Sequence Diagram (for an easy test case)
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
testit
Main
Date
Date()
increment()
printDate
Day
setCurrentPos()
Day()
increment()
setDay
getDay
Month
setCurrentPos()
Month()
increment()
setMonth
getMonth
getMonthSize
Year
setCurrentPos()
Year()
increment()
isLeap
getYear
m1
m15
m16
m18
m19
m21
m6
m5
m4
Message Flow for Easy Case
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Currency Converter
U. S. Dollar amount
Equivalent in …
Brazil
Canada
European Community
Japan
Clear
Quit
Compute
Second Example (event-driven)
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Procedural Implementation of Compute Button
procedure Compute ( USDollarAmount, EquivCurrencyAmount)dim brazilRate, canadaRate, euroRate, japanRate, USDollarAmount As SingleIf (optionBrazil) Then EquivCurrencyAmount = brazilRate * USDollarAmount Else If (optionCanada) Then EquivCurrencyAmount = canadaRate * USDollarAmount Else If (optionEuropeanUnion) Then EquivCurrencyAmount = euroRate * USDollarAmount Else If (optionJapan) Then EquivCurrencyAmount = japanRate * USDollarAmount Else Output ("No country selected") EndIF EndIF EndIFEndIF
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
“Object-Oriented”
Implementationof Compute
Button
object optionBrazil () Constant USdollarToBrazilReal= 2.067 private procedure senseClick commandCompute(USdollarToBrazilReal) End senseClickobject optionCanada() Constant USdollarToCanadianDollar = 1.16 private procedure senseClick commandCompute(USdollarToCanadianDollar) End senseClickobject optionEuropeanUnion () Constant USdollarToEuro= 0.752 private procedure senseClick commandCompute(USdollarToEuro) End senseClickobject optionJapan () Constant USdollarToJapanYen = 117.82 private procedure senseClick commandCompute(USdollarToJapanYen) End senseClickprocedure comandCompute ( exchangeRate)dim exchangeRate, USDollarAmount As Single USDollarAmount = Val(txtUSDollarAmount.text) EquivCurrencyAmount = exchangeRate * USDollarAmountEnd procedure Compute
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
“Visual Basic” Style Implementation
Public exchangeRate As Single
Private Sub optBrazil_Click() exchangeRate = 2.067End SubPrivate Sub optCanada_Click() exchangeRate = 1.16End SubPrivate Sub optEuropeanUnion_Click() exchangeRate = 0.752End SubPrivate Sub optJapan_Click() exchangeRate = 117.82End SubPrivate Sub commandCompute () EquivCurrencyAmount = exchangeRate * Val(txtUSDollarAmount.text)End Sub
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Input Events for the Currency Converter
Click on OK in errormessage
ip6Click on EuropeanCommunity
ip2.3
Click on Quit buttonip5Click on Canadaip2.2
Click on Clear buttonip4Click on Brazilip2.1
Click on Compute buttonip3Click on a country buttonip2
Click on Japanip2.4Enter US Dollar amountip1
Input EventsInput Events
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Output Events for the Currency Converter
Indicate Japanop3.4Reset equivalent currency amountop10Indicate European Communityop3.3Reset US Dollar Amountop9Indicate Canadaop3.2
Error Msg: Must select a countryand enter US Dollar amount
op8Indicate Brazilop3.1
Error Msg: Must enter US Dollaramount
op7Indicate selected countryop3Error Msg: Must select a countryop6Display ellipsisop2.5Display foreign currency valueop5Display Japanese Yenop2.4
Reset Japanop4.4Display European CommunityEuros
op2.3Reset European Communityop4.3Display Canadian Dollarsop2.2Reset Canadaop4.2Display Brazilian Realsop2.1Reset Brazilop4.1Display currency nameop2Reset selected countryop4Display US Dollar Amountop1
Output EventsOutput Events
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
ASFs and Data Places for theCurrency Converter
Sense Click on Quit buttons8Sense Click on E. U.s4
Sense Click on Clear buttons7Sense Click on Canadas3
Sense Click on Computebutton
s6Sense Click on Brazils2
Sense Click on Japans5Store US Dollar amounts1
Atomic System FunctionsAtomic SystemFunctions
Country selectedd2US Dollar Amountentered
d1
Data PlacesData Places
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Missing Country
and Dollar
Message
Missing US Dollar Message
US Dollar
Amount Entered
Missing Country Message
Both Inputs Done
Country Selected
Idle
Equiv Amount
Displayed
ip1/op1ip2/op2, op3
ip2/op2, op3ip3/op7ip3/op6
ip6
ip6
ip3/op5
ip6
ip3/op8
ip4 or ip5 op2.5, op4
op9, and op10
ip4 or ip5
ip1/op1
High Level Finite State Machine
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Canada
Euro Comm
Japan
Brazil
Idle
ip2.2 op2.2 op3.2 op4.1
ip2.1 op2.1 op3.1 op4.2
ip2.3 op2.3 op3.3 op4.4
ip2.4 op2.4 op3.4 op4.3
ip2.4 op2.4 op3.4 op4.2
ip2.2 op2.2 op3.2 op4.4
ip2.1 op2.1 op3.1 op4.3
ip2.3 op2.3 op3.3 op4.1
ip2.1 op2.1 op3.1
ip2.2 op2.2 op3.2
ip2.3 op2.3 op3.3
ip2.4 op2.4 op3.4
Details of Option Button FSM
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
StateChart for the Currency ConverterIn Storage
Idle
U .S . Dollar Amount SelectedCountry Selected Missing Country and Dollar Message
Missing U .S . Dollar Message Missing Country Message
Both Inputs Done
Equiv . Amount Displayed
Ip 2/op 2 ,op 3
Ip 1/op 1
Ip 3/op 5
Ip 3 /op 7Ip 3/ op 7
Ip 6Ip 6
Ip 1 /op 1
Ip 2/ op 2 ,op 3
Ip 1 /op 1Ip 2 /op 2 ,op 3
Ip 6
Ip 3 /op 8
Ip 4
GUI action
Executing
Ip 4___________________
op 2 , op 4, op 9 , op 10
ip 5 Quit End Application
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Observations and Conclusions
• Classes/objects are MUCH more complicated than procedures - need to consider inheritance, polymorphism - (the encapsulation part is easy) - o-o design proceeds by composition, not decomposition • Complexity is moved from methods to messaging among objects - hence unit level testing becomes integration level testing - it's like that bump in the rug, you can push down on it, but it pops up somewhere else. • Models to describe integration level testing are inadequate (you can't test what you don't model)
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Extended Definitions
• An MM-Path starts with a method and ends when it reaches a method which does not issue any messages of its own ( message quiescence) • Since MM-Paths are composed of linked method-message pairs in an object network, they interleave and branch off from other MM-Paths.
• An ASF begins with a port input event • This system level input triggers the method-message sequence of an MM-Path which may trigger other MM-Paths • The sequence of MM-Paths ends with a port output event (event quiescence)
An MM-Path in object-oriented software is a sequence ofmethod executions linked by messages.
An atomic system function (ASF ) is a sequence ofstatements that begins with an input port eventand ends with an output port event.
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
Port Input Event
Port Output Event
Data Place
Method Execution Path
Message Send/Return
Constructs for Event- and Message-Driven Petri Nets
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
object A
object B
Message from object A to object B
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
d1
d2
d3d5
d4
d6msg1
msg2
mep1
mep2
mep3
mep4
mep5
mep6
(Second Edition Cover Story)
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
s1
d1
p1
p10
s2
d2
p2
p15
p19
p23
s7
p7
p24
p25
p14
d1 d2
s6
p6
p26
d1 d2
p11
s3
d2
p3
p16
p20
p12
s4
d2
p4
p17
p21
p13
s5
d2
p5
p18
p22
s8
p8
p27
ASFs for the Currency Converter
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
s1
d1
p1
p23
s7
p7
p24
p25
p14
s6
p6
p26
p12
s4
d2
p4
p17
p21
EDPN Composition of four ASFs
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
s1
s3
s7
s6
s4 s5s2
s8
Directed Graph of Intended ASF Sequences
Software Testing: A Craftsman’s Approach, 3rd Edition Object-Oriented Testing
“Forced Navigation” Adjacency Matrix
00000000s810011111s711011111s600101110s500110110s400111010s300111100s200011111s1s8s7s6s5s4s3s2s1