Trend Quality Assurance

Trend Quality Assurance

Edward TsaiEdward Tsai 　　蔡木本蔡木本趨勢科技全球軟體研發部協理

April 22, 2003

RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineContents

Software quality assurance overview Testing skill and debug tools

Testing concepts. Testing methodology. Testing tools.

The importance of QA Automation Software testing in enterprise

environment

RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 line

Overview

What is the quality? What is QA role in Trend? How many different type of testing

strategy? What is the typical software life-cycle? What is the QA challenge and future? What is Trend QA organization?


What is the quality?

2 level of quality: Level 1: product defect rate and reliability. Level 2: product quality, process quality,

customer satisfaction.

Quality from professional views: Conformance to requirement. Fitness for use.

For example: Cadillac and Chevrolet car, Which has good

quality?

RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineWhat is QA role in Trend?

The purpose of QA engineers: To discover all possible errors in

development cycle. To discover every conceivable fault or

weakness in product. To simulate customer’s focus and need. To measure the quality level of product. To delivers the quality report to service

team.

RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineHow many different type

testing strategy? Black-Box testing:

Derived from the functional design spec, without touching the internal program structure.

White-box (or Glass-box) testing: Require knowledge of the internal program

structure and derived from the internal design spec or code.

Trendmicro: Black-box testing and will migrate to Gray-box

testing.


What is the typical software life-cycle?

Concept. Requirement. Design. Implementation or coding. Test. Operation and maintenance.

RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineDOT-U model

1

2

3

4

5

6

7

8

9 10

11

Requirements

specification

Requirements

verification

Functional design

specification

Functional Design

verification

Internal design

specification

Internal design

verification

Products simulation

Usability test

Code and spec

modification

System and

Acceptance

validation

Function validation

Code and spec

modification

Code and spec

modification

Code and spec

modification

Code

Code verification Unit validation Integration validation

1


What is Trendmicro software life-cycle?

Requirement. High-Level design/Planning. Detailed design. Construction. Pre-release. Closure. Service.

RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineWhat is QA

challenge/future?

Professional discipline requiring trained, skilled people.

Advances in technology/tools to create complex software in GUI, Client-Server, Intranet/Internet architecture.

Multi-platform, multi-language. Many/complex compatibility software

testing. Keep on process improvement. Test automation is not mature yet.


Defect Distribution

Requirements56%

Design 27%

Code 7%

Others 10%


The cost of defect

Cost

Time

Alpha Beta Release

RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineTester-to-Developer Ratios

The lesson: The importance of testing is now recognized. Vendors are willing to pay for proper testing.

A dramatic shift: Historically, for Mainframes : 1: 5-10. At Microsoft, 1992 : 2:3. At Microsoft, 1995 : 1:1. At Microsoft, 2000 : 2:1. At Microsoft, 2005 : ? TrendMicro, 2000 : 1 : 2 TrendMicro, 2003 : 1 : 1

RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineWhat is the Trend QA organization?

Project - LeadEngineer Q AEngineer Q AEngineer Q A

Project AM anager - Q A

Project - LeadEngineer - Q AEngineer - Q AEngineer - Q A

C ertifcationM anager-Q A

Project - LeadEngineer - SWEngineer - SWEngineer - SW

Test developm entM anager-Q A

Lab engineerLab engineerLab engineerLab engineer

Lab adm inM anager-Q A

Project - LeadEngineer Q AEngineer Q AEngineer Q A

Project BM anager - Q A

D irector-Q A

G lobal D irector-Q A

RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineTesting skill and debug

tools

Testing methodology.

Test case design.

Testing tools.


Testing methodology

Verification testing: Reference to Dotted-U Model. Verification methods : Inspections, walkthroughs, Technical

reviews, others. The verification tool : Checklists. Include: Requirement, Architecture, Plans, Design Spec,

code, Documentation. Validation testing:

A process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirement.

Include: Unit testing, integration testing, Usability testing, Function testing, Regression testing, System testing, Acceptance testing (Alpha, Beta testing).


Testing case design

Go through all testing synopsis at every checking point. Task-Oriented Functional Test (TOFT) Forced-Error Test (FET) Boundary test Volume Test Stress Test Performance Test Security Test. Configuration Compatibility Test Documentation Test Trend Standards in Product

RUNNING HEADER, 14 PT., ALL CAPS, Line Spacing=1 lineTest Coverage

Test coverage has three components: Requirements coverage. Function coverage. Logic coverage.


Example


Example

Input a Directory name to install a product in NT system. Task-Oriented Functional Test (TOFT)

• Input a legal directory name with length between 1 to 255 character. Forced-Error Test (FET)

• Read privilege only directory, Non-existed drive, more than 255 characters, Not enough disk space.

Boundary test• Input a space or null character, Input one character, Input 255

characters, Input 266 characters. Memory leak after install complete. Volume Test

• Install it with 30 MB free disk space. Stress Test

• Install it with CPU utilization more than 70%, Free visual Memory left less than 1 MB, ...

Performance Test• Memory usage, CPU utilization, Any network traffic generated,...


Example

Security Test.• Does it can be installed from every users or Administrator only.

Configuration Compatibility Test• N/A

Documentation Test• The description are properly and correctly

Trend Standards in Product• Does it install under c:\programs files\TrendMicro\PC-cillin 98?


Testing tools

Testing documents database: Test manager, TCMT.

Bugs tracker system: PVCS Tracker 7.0.

Automation tools: Visual test 4.0, Robot J, ATP. Winrunner.

Source code Coverage: PureCoverage.

System tools: Loadrunner, Load test, Winbench.

Debug tools: Boundchecker, Quantity, Purify, SoftIce.


Q & A

Documents

Trend Quality Assurance