Software Quality Analyst Certification Exam · PDF fileReview of Skill Categories in the Common Body of Knowledge (CBOK) for the Certified Software Quality Analyst: • Quality Principles

Software Quality AnalystCertification Exam Prep

Course Materials & Appendicies

Copyright © 2011 byQAI Global Institute2101 Park Center Drive, Suite 205
Orlando, FL 32835-7614Phone 407-363-1111Fax 407-363-1112Web site: www.qaiworldwide.orgCSQA-BC-3927

Software Quality Analyst Certification Exam Prep Course

Revision and CopyrightTable 1: Revision Table

Copyright

Copyright © Quality Assurance Institute 2011 All Rights Reserved

No part of this publication, or translations of it, may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopy-ing, recording, or any other media embodiments now known or hereafter to become known, without the prior written permission of the Quality Assurance Institute.

DESCRIPTION BY DATE

Page ii © QAI Worldwide ID_CSQA-BC-3927

Table of ContentsPage

CSQA Course Objectives - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -3Introductions - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -4Why Become Certified? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -5The Process - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -6About the Exam - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -7Course Table of Contents - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -8

Skill Category One - - - - - - - - - - - - - - - - - - - - - - - - - - 9Quality Vocabulary - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 10Quality Vocabulary (cont.) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 16Infrastructure for Software Quality Products and Services - - - - - - - - - - - - - - 17The Two Quality Gaps - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 18Quality Attributes for Information Systems - - - - - - - - - - - - - - - - - - - - - - 19Understanding the Quality Challenge - - - - - - - - - - - - - - - - - - - - - - - - 20Questions - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 21Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 22The PDCA Cycle - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 24The PDCA Cycle (cont.) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 27The PDCA Cycle (cont.) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 28The Cost of Quality - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 29The Cost of Quality (cont.) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 30The Cost of Quality (cont.) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 31Key Principles of Quality - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 35The Quality Solution - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 36Best Practices - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 37Six Sigma - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 38What is a Defective Process? - - - - - - - - - - - - - - - - - - - - - - - - - - - - 39Baselining and Benchmarking - - - - - - - - - - - - - - - - - - - - - - - - - - - 40Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 41Quality Assurance (QA) (Process Focus) - - - - - - - - - - - - - - - - - - - - - - 42Quality Control (QC) (Product Focus) - - - - - - - - - - - - - - - - - - - - - - - - 43Quality Control - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 44Just-in-Time (JIT) Techniques - - - - - - - - - - - - - - - - - - - - - - - - - - - 46Sample Questions - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 48Sample Questions (cont.) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 50Total Quality Management - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 59Sample Questions (cont.) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 61

Skill Category 2 - - - - - - - - - - - - - - - - - - - - - - - - - - - - 62What Commitment Means - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 63New Behaviors for Management - - - - - - - - - - - - - - - - - - - - - - - - - - 64Leader vs. Manager - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 66Leadership - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 67Fundamental Mistakes - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 68Quality Knowledge - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 69Awareness Training - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 70Management Committees - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 75Teams, Work Groups - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 76

ID_CSQA-SC-3927 © QAI Worldwide Page iii


Table of Contents (Continued) PageGuidelines for Teams - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 77Team Development Phases - - - - - - - - - - - - - - - - - - - - - - - - - - - - 78Group Compatibility - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 79Written Reports - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 80What is a Brief Description of Management Support? - - - - - - - - - - - - - - - - 82Setting the Proper “Tone” at the Top - - - - - - - - - - - - - - - - - - - - - - - - 83Integrity and Ethical Values - - - - - - - - - - - - - - - - - - - - - - - - - - - - 84Listening Skills - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 863-Step Listening Process - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 87Implementing a Mission, Vision, and Quality Policy - - - - - - - - - - - - - - - - - 91Mission - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 92Vision - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 93Goals - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 94Policy - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 95What is an Effective Work Environment - - - - - - - - - - - - - - - - - - - - - - 96What is Not an Effective Work Environment - - - - - - - - - - - - - - - - - - - - 97The Environment is Essential Because Quality: - - - - - - - - - - - - - - - - - - - 98Sample Questions - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 100

Skill Category 3 - - - - - - - - - - - - - - - - - - - - - - - - - - 101Types of Baselines - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 103Baseline Studies - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 104Steps to Perform a Baseline Study - - - - - - - - - - - - - - - - - - - - - - - - - 105Conducting Objective Baseline Studies - - - - - - - - - - - - - - - - - - - - - - - 106Establishing Baselines - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 108Purpose of a Model - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 109Staged Versus Continuous Models - - - - - - - - - - - - - - - - - - - - - - - - - 110Model Selection Process - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 111Using Models For Assessment and Baselines - - - - - - - - - - - - - - - - - - - 112Software Engineering Institute’s Capability Maturity Model - - - - - - - - - - - - - 113SEI CMMI - Level 1 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 114SEI CMMI - Level 2 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 115SEI CMMI - Level 3 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 116SEI CMMI - Level 4 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 117SEI CMMI - Level 5 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 118Malcolm Baldrige National Quality Award - - - - - - - - - - - - - - - - - - - - - 119National Institute of Standards and Technology - - - - - - - - - - - - - - - - - - - 120MBNQA Awards Criteria - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 121MBNQA Scoring System - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 122ISO 9001: 2000 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 123ISO 9001: 2000 Model Overview - - - - - - - - - - - - - - - - - - - - - - - - - - 124ISO 9001: 2000 Process Model - - - - - - - - - - - - - - - - - - - - - - - - - - 125Model Overview - IS0/IEC 12207 - - - - - - - - - - - - - - - - - - - - - - - - - - 126Model Overview - IS0/IEC 15504 - - - - - - - - - - - - - - - - - - - - - - - - - - 127Sample Questions (cont.) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 130

Skill Category 4 - - - - - - - - - - - - - - - - - - - - - - - - - - 131Establishing a Function to Promote and Manage Quality - - - - - - - - - - - - - - 132Four Variables in a System Development Project - - - - - - - - - - - - - - - - - - 133IT Quality Plan - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 137Quality Tools - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 138

Page iv © QAI Worldwide ID_CSQA-SC-3927


Table of Contents (Continued) PageTool Categories - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 139Management Tools - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 140Statistical Tools - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 141Tools for Presentation - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 142The Deployment Process Phases - - - - - - - - - - - - - - - - - - - - - - - - - 146Critical Success Factors of Deployment - - - - - - - - - - - - - - - - - - - - - - 149Internal Auditing and Quality Assurance - - - - - - - - - - - - - - - - - - - - - - 150Internal Auditing - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 151Quality Assurance - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 152Sample Questions - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 153

Skill Category 5 - - - - - - - - - - - - - - - - - - - - - - - - - - - 155Planning Concepts - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 156The Management Cycle - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 157The PDCA Cycle - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 158The Planning Cycle - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 162Prerequisites to Quality Planning - - - - - - - - - - - - - - - - - - - - - - - - - 164Common Activities in the Planning Process - - - - - - - - - - - - - - - - - - - - 165Common Activities in the Planning Process - - - - - - - - - - - - - - - - - - - - 167Planning to Mature IT Work Processes - - - - - - - - - - - - - - - - - - - - - - 168Maturity Levels by Process Category - - - - - - - - - - - - - - - - - - - - - - - 169Skipping Levels and Reverting Back to Lower Levels - - - - - - - - - - - - - - - 173

Skill Category 6 - - - - - - - - - - - - - - - - - - - - - - - - - - - 174Definition of a Process - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 175Why Processes are Needed - - - - - - - - - - - - - - - - - - - - - - - - - - - 176Process Workbench and Components - - - - - - - - - - - - - - - - - - - - - - 178Process Categories - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 179Process Continuum - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 180How Processes are Managed - - - - - - - - - - - - - - - - - - - - - - - - - - - 181Planning Processes - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 182Planning Processes - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 183Do Processes - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 184Check Processes - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 185Act Processes - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 186Sample Questions - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 187

Skill Category 7 - - - - - - - - - - - - - - - - - - - - - - - - - - - 189Quality Control Basics - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 190Process Workbench and Components - - - - - - - - - - - - - - - - - - - - - - 191Test Stages - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 192Independent Testing - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 193Static vs. Dynamic Testing - - - - - - - - - - - - - - - - - - - - - - - - - - - - 196The “V” Testing Concept - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 197Test Objectives - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 199Reviews and Inspections - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 200Review Formats - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 201Types of Reviews - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 202Testing Methodology - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 203What Type of Project? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 204

ID_CSQA-SC-3927 © QAI Worldwide Page v


Table of Contents (Continued) PageManagement of Verification and Validation - - - - - - - - - - - - - - - - - - - - - 205Verification Techniques - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 207Validation Techniques - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 208Structural and Functional Testing - - - - - - - - - - - - - - - - - - - - - - - - - 210Software Configuration Management - - - - - - - - - - - - - - - - - - - - - - - - 212Change Control Procedures - - - - - - - - - - - - - - - - - - - - - - - - - - - - 213Defect Management Process - - - - - - - - - - - - - - - - - - - - - - - - - - - 214Defect Reporting - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 215Defect Management - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 216Severity vs. Priority - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 217A Sample Defect Tracking Process - - - - - - - - - - - - - - - - - - - - - - - - - 218Using Defects for Process Improvement - - - - - - - - - - - - - - - - - - - - - - 219Sample Questions - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 220

Skill Category 8 - - - - - - - - - - - - - - - - - - - - - - - - - - 222Standard Units of Measure - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 223Metrics - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 224Objective and Subjective Measurement - - - - - - - - - - - - - - - - - - - - - - 225Types of Measurement Data - - - - - - - - - - - - - - - - - - - - - - - - - - - - 226How to Know a Measure Is Good - - - - - - - - - - - - - - - - - - - - - - - - - 227Quantitative Management in an IT Function - - - - - - - - - - - - - - - - - - - - 228Product Measurement - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 229Process Measurement - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 230Common Causes of Variation - - - - - - - - - - - - - - - - - - - - - - - - - - - 231Special Causes of Variation - - - - - - - - - - - - - - - - - - - - - - - - - - - - 232Defining Risk - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 233Characterizing Risk - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 234Managing Risk - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 235Software Risk Management - - - - - - - - - - - - - - - - - - - - - - - - - - - - 236Risk of Integrating New Technology - - - - - - - - - - - - - - - - - - - - - - - - 237Prerequisites to Implementing a Measurement Program - - - - - - - - - - - - - - 238Sample Questions - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 239

Skill Category 9 - - - - - - - - - - - - - - - - - - - - - - - - - - 241COSO Defined - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 242Internal Control Terms - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 243Levels of Organization Control - - - - - - - - - - - - - - - - - - - - - - - - - - - 245Internal Auditor’s Responsibilities - - - - - - - - - - - - - - - - - - - - - - - - - 247Internal Control Models - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 251Building Internal Controls - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 252Building Adequate Security - - - - - - - - - - - - - - - - - - - - - - - - - - - - 253Establishing a Security Baseline - - - - - - - - - - - - - - - - - - - - - - - - - - 255Security Learning Continuum - - - - - - - - - - - - - - - - - - - - - - - - - - - 256Security Control Principles - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 257

Skill Category 10 - - - - - - - - - - - - - - - - - - - - - - - - - - 260Challenges in Selecting Acquired Software - - - - - - - - - - - - - - - - - - - - - 261Quality Professional’s Responsibility - - - - - - - - - - - - - - - - - - - - - - - - 264Contracted Software Selection Process - - - - - - - - - - - - - - - - - - - - - - 274Exercise - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 285

Page vi © QAI Worldwide ID_CSQA-SC-3927


Table of Contents (Continued) PageSample Questions - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 286

Test Taking Tips - - - - - - - - - - - - - - - - - - - - - - - - - - 287On the Day of the Exam - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 288About the Exam - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 289After the Exam - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 290First Thing - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 291Tip #1 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 292The How-to’s - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 293Sample Questions - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 294More How-to’s - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 295Second Round - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 296Check Back - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 297Don’t Change Anything - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 298Change the Chances - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 299Fluff and Stuff - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 300The Distracters - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 301Test Taking Tip Summary - - - - - - - - - - - - - - - - - - - - - - - - - - - - 302The End - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 303

ID_CSQA-SC-3927 © QAI Worldwide Page vii


Table of Contents (Continued) Page

This page intentionally left blank.

Page viii © QAI Worldwide ID_CSQA-SC-3927

Software Quality Analyst Certtification Exam Prep Course

KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALS

NOTES:

ID_CSQA-BC-3927 © QAI Worldwide Page 1



Page 2 © QAI Worldwide ID_CSQA-BC-3927



CSQA Course ObjectivesKEY CONCEPT PRESENTATION GUIDE


• Help the quality professional study and prepare for the CSQA examination.• Reinforce current knowledge• Re-introduce concepts that may not be used everyday• Explain the rationale for use within the IT Industry• Highlight sample questions, both multiple choice and essay type responses.• Provide you with a background of the IT quality fundamentals

- Due to time limitations of this course, it is not intended to cover all the components of the common body of knowledge in great depth.

NOTES:



IntroductionsKEY CONCEPT PRESENTATION GUIDE


• Who are you and your company• Why and when are you taking the exam• Have you taken an exam before – where/when• Time you spend studying now• Study group experience

NOTES:



Why Become Certified?KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALS• The IT and software testing industry is competitive• Recognition by peers• Proof of you profession and skills in the IT field• Increased confidence in personal capabilities • Indicates a professional level of competence • Potentially more rapid career advancement • Greater acceptance in the role as advisor to management

NOTES:



The ProcessKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALS• Satisfy all of the prerequisites required prior to applying for candidacy

- $350.00 Administration fee (includes study guide)• One of the following must be met:

- 4 + 2 = Bachelor’s degree in an accredited college-level institution and 0 years experience or…

- 2 + 4 = Associate’s degree and higher and 2 years experience or…- 0 + 6 = No degree and 6 years experience in the IT service area

• Subscribe to the Code of Ethics • Submit a completed Certification Candidacy Application

NOTES:



About the ExamKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSFour parts to the exam Part Questions Type 1 50 Objective or multiple choice questions.

2 10 Essay or Scenario based questions. 3 50 Objective or multiple choice question. 4 10 Essay or Scenario based questions.

Minimum passing = overall average of 75%

NOTES:



Course Table of ContentsKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSReview of Skill Categories in the Common Body of Knowledge (CBOK) for the Certified Software Quality Analyst:• Quality Principles• Quality Leadership• Quality Baselines• Quality Assurance• Quality Planning• Define, Build, Implement, and Improve Work Processes• Quality Control Practices• Metrics and Measurement• Internal Control and Security• Outsourcing, COTS, and Contracting Quality

NOTES:


Skill Category OneKEY CONCEPT PRESENTATION GUIDEQuality Principles and Concepts

SUPPORTING / EXPLANATORY MATERIALS• Vocabulary• Different Views of Quality• Quality Concepts and Practices• Quality Control and Quality Assurance• Quality Pioneers Approach to Quality

NOTES:



Quality VocabularyKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSUnderstand the technical terms used to describe various testing techniques, tools, principles, concepts, and activities

NOTES:



Quality Vocabulary (cont.)KEY CONCEPT PRESENTATION GUIDEThe Quality Assurance function is charged with implementing the quality policy established by executive management. QAI recommends that the QA director position be a strong leadership position, and emphasizes the strong interpersonal skills necessary to successfully execute activities involved in making improvement occur.

SUPPORTING / EXPLANATORY MATERIALS• Activities that modify the development process to prevent the introduction of flaws

- Staff function• Implements management’s quality policies

- Responsible for continuous improvement of the software development process• Proactive approach focused on defect prevention• Examples:

- Defining change control procedures- Facilitating post-project reviews

• Analyzing metrics to identify opportunities for process improvement

NOTES:




Quality Vocabulary (cont.)KEY CONCEPT PRESENTATION GUIDEThe Quality Assurance function is charged with implementing the quality policy established by executive management. QAI recommends that the QA director position be a strong leadership position, and emphasizes the strong interpersonal skills necessary to successfully execute activities involved in making improvement occur.

SUPPORTING / EXPLANATORY MATERIALS• Activities within the development process to detect the introduction of flaws

- Test planning and execution- Quality control measures a product against the existence of an attribute - Determines whether the product conforms to a standard or procedure (also known as

compliance checking).• Proactive approach focused on defect detection• Examples:

- Writing and executing test cases and scripts- Participating in verification and validation activities- Reporting defects to identify opportunities for process improvement

NOTES:


Quality Vocabulary (cont.)KEY CONCEPT PRESENTATION GUIDEMeeting requirements is a producer’s view of quality. This is the view of the organization responsible for the project and processes, and the products and services acquired, developed, and maintained by those processes. Meeting requirements means that the person building the product does so in accordance with the requirements.

SUPPORTING / EXPLANATORY MATERIALSProducer’s view of Quality:Per Philip Crosby:Conformance to requirements• Doing the right thing• Doing it the right way• Doing it right the first time• Doing it on time without exceeding cost

NOTES:



Quality Vocabulary (cont.)KEY CONCEPT PRESENTATION GUIDEBeing fit for use is the customer’s definition. The customer is the end user of the products or services. Fit for use means that the product or service meets the customer’s needs regardless of the product requirements.

SUPPORTING / EXPLANATORY MATERIALSCustomer’s view of Quality: From Joseph Juran and W. Edwards Deming

Fit for use• Receiving the right product for use• Being satisfied that needs have been met• Expectations have been met• Treated with integrity, courtesy, and respect

NOTES:




Quality Vocabulary (cont.)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSProvider’s view of Quality – This is the perspective of the organization that delivers the products and services to the customer.

NOTES:



Quality Vocabulary (cont.)KEY CONCEPT PRESENTATION GUIDEThis is the perspective of the organization (that may be external to the producer’s company, such as an independent vendor) that provides either the producer and/or the provider with products and services needed to meet the requirements of the customer.

SUPPORTING / EXPLANATORY MATERIALSThe perspective of the organization (which may be external to the producer’s company, such as an independent vendor) that provides either the producer and/or the provider with products and services needed to meet the requirements of the customer.

NOTES:



Infrastructure for Software Quality Products and ServicesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe figure shows the requirements coming from the customer to the producer/provider, who uses them to create the products and services needed by the customer. This process works because of the two-way measurement process established between the involved parties.

NOTES:



The Two Quality GapsKEY CONCEPT PRESENTATION GUIDEClosing these two gaps is the responsibility of the quality function

SUPPORTING / EXPLANATORY MATERIALSThe producer’s gap is the difference between what is specified (the documented requirements and internal standards) versus what is delivered (what is actually built). The customer’s gap is the difference between what the producers actually delivered versus what the customer wanted.

NOTES:



Quality Attributes for Information SystemsKEY CONCEPT PRESENTATION GUIDEQuality is a multifaceted concept driven by customer requirements

SUPPORTING / EXPLANATORY MATERIALSManagement needs to develop quantitative, measurable “standards” for each of these quality criteria for their development projects.

NOTES:

Attributes Definition

Correctness Extent to which a program satisfies its specifications and fulfills the user’s mission objectives.

Reliability Extent to which a program can be expected to perform its intended function with required precision.

Efficiency The amount of computing resources and code required by a program to perform a function.

Integrity Extent to which access to software or data by unauthorized persons can be controlled.

Usability Effort required learning, operating, preparing input, and interpreting output of a program.

Maintainability Effort required locating and fixing an error in an operational program.

Testability Effort required testing a program to ensure that it performs its intended function.

Flexibility Effort required modifying an operational program.

Reusability Extent to which a program can be used in other applications – related to the packaging and scope of the functions that programs perform.

Interoperability Effort required to couple one system with another.


Understanding the Quality ChallengeKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSAQL - Many think that defect-free products and services are not practical or

economical, and thus believe some level of defects is normal and acceptable.

Cost - Quality is frequently associated with cost, meaning that high quality is synonymous with high cost.

Detail - Quality by definition calls for requirements/specifications in enough detail so that the products produced can be quantitatively measured against those specifications.

Standards - Many technical personnel believe that standards inhibit their creativity, and thus do not strive for compliance to standards.

NOTES:



QuestionsKEY CONCEPT PRESENTATION GUIDE


NOTES:



ExerciseKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe extent to which a program satisfies its specifications and fulfills the user’s mission objectives.a. Reliabilityb. Maintainabilityc. Correctnessd. Testabilitye. All of the AboveThe effort required to ensure the program performs its intended function.a. Reliabilityb. Maintainabilityc. Correctnessd. Testabilitye. All of the Above

NOTES:



Exercise (cont.)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSQuality factors have frequently been used as a basis for measuring the quality of an information system. Listed below are two of those quality factors.Define each of the quality factors and provide an effective measure or metric for each of these quality factors? Ease of Use:• Effort required to learn, operate, prepare input and interpret output of the program/system. • Customer surveys targeted to a specific customer control group.• How many clicks it takes the user to find the objectMaintainability:• The effort required to locate and fix errors in the program/system. • The recording of how long it takes to make a change to code that has already been

released. • The recording of how long it takes to retest a change.

NOTES:



The PDCA CycleKEY CONCEPT PRESENTATION GUIDEThe cycle comprises the four steps:6. Plan7. Do8. Check9. Act


NOTES:




The PDCA Cycle (cont.)KEY CONCEPT PRESENTATION GUIDEPlan (P): Devise a plan

SUPPORTING / EXPLANATORY MATERIALSDefine the objective, expressing it numerically, if possible.Clearly describe the goals and policies needed to attain the objective at this stage. Determine the procedures and conditions for the means and methods that will be used to achieve the objective. Plan (P): Devise a plan• Objectives are quantitatively defined• Policies are defined• Practices/processes used to achieve the goals• Objectives are Identified

NOTES:



The PDCA Cycle (cont.)KEY CONCEPT PRESENTATION GUIDEDo (D): Execute the plan

SUPPORTING / EXPLANATORY MATERIALSExecute the plan - Create the conditions and perform the necessary teaching and training to ensure everyone understands the objectives and the plan. Teach workers the procedures and skills they need to fulfill the plan and thoroughly understand the job. Then perform the work according to these procedures.Do (D): Execute the plan• Create the practices (define them)• Provide training• Perform the work according to the defined practice

NOTES:



The PDCA Cycle (cont.)KEY CONCEPT PRESENTATION GUIDECheck (C): Check the results

SUPPORTING / EXPLANATORY MATERIALSCheck the results – As often as possible, check to determine whether work is progressing according to the plan and whether the expected results are obtained. Check for performance of the procedures, changes in conditions, or abnormalities that may appear.Check (C): Check the Results• Determine if work is progressing to plan.• Will anticipated results be realized? • Are goals and objectives satisfied?

NOTES:



The PDCA Cycle (cont.)KEY CONCEPT PRESENTATION GUIDEAct (A): Take the necessary action

SUPPORTING / EXPLANATORY MATERIALSTake the necessary action - If the check reveals that the work is not being performed according to plan, or if results are not what were anticipated, devise measures for appropriate action. Look for the cause of the abnormality to prevent its recurrence. Sometimes workers may need to be retrained and procedures revised. The next plan should reflect these changes and define them in more detail. Act (A): Take the Necessary Action• Devise measures and appropriate actions if work is not progressing to plan.• Look for the cause of abnormalities.• Update the plan.• Update training materials if needed.

NOTES:



The Cost of QualityKEY CONCEPT PRESENTATION GUIDE


The cost-of-quality components are the accounts for which cost information will be collected. In companies, the cost of quality is currently buried in the cost of sales. Within the information services organization, the cost of quality is buried in the cost of operations. This step involves the following two tasks:1. Determine the cost components that management wants to control. This will identify the

cost-of-quality components.2. Determine the cost for these components is collectible. The nature of the business, or the

methodologies used, may prohibit certain types of costs from being collected, or collected with enough consistency and reliability to be of value.

NOTES:



The Cost of Quality (cont.)KEY CONCEPT PRESENTATION GUIDEQuality is an attribute of a product or service. Productivity is an attribute of a process.

SUPPORTING / EXPLANATORY MATERIALSThe cost of quality (COQ) is the money spent beyond what it would cost to build a product right the first time. If every worker could produce defect-free products the first time, the COQ would be zero. Since this situation does not occur, there are costs associated with getting a defect-free product produced.The total production costs associated with the delivery of software applications include both the actual production costs and the cost of quality. The first component - production costs - consists of the costs associated with producing the product “right the first time,” or RTF costs. These costs include labor, materials, and equipment (hardware, software, tools) associated with the actual development of the application. The second component includes the additional costs associated with assuring that the product delivered meets the quality goals established for the product. This cost component is called the Cost of Quality, and includes all costs associated with the prevention, identification, and correction of product defects (includes repair and damage costs.)The cost of quality will vary from one organization to the next. The majority of costs associated with the Cost of Quality are associated with the identification and correction of defects.

NOTES:



The Cost of Quality (cont.)KEY CONCEPT PRESENTATION GUIDEThree cost of quality categories:1. Prevention2. Appraisal3. Failure

SUPPORTING / EXPLANATORY MATERIALSThere are three COQ categories:1. Prevention - Money required preventing errors and to do the job right the first time is

considered prevention cost. This category includes money spent on establishing methods and procedures, training workers and planning for quality. Prevention money is all spent before the product is actually built.

2. Appraisal – Appraisal costs cover money spent to review completed products against requirements. Appraisal includes the cost of inspections, testing and reviews. This money is spent after the product or subcomponents are built but before it is shipped to the user.

3. Failure – Failure costs are all costs associated with defective products. Some failure costs involve repairing products to make them meet requirements. Others are costs generated by failures, such as the cost of operating faulty products, damage incurred by using them and the costs incurred because the product is not available. The user or customer of the organization may also experience failure costs.

NOTES:



The Cost of Quality (cont.)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe information services staff must be as committed to the cost of quality as is management. One of the quality principles is the elimination of fear. This is generally a prerequisite to an effective cost-of-quality program. If individuals fear management reprisals, they will be particularly reluctant to accurately record failure costs. Since this is an extremely important part of the cost-of-quality program, it must be recorded accurately and completely.Studies show that the Cost of Quality is approximately 50% of the cost to build a product or service:

40% = failure costs

7% = appraisal costs

3% = preventive costs

NOTES:



The Cost of Quality (cont)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSMark “P” for Prevention; “A” for Appraisal; “F” for Failure: - (Answers appear in bold)1. Conducting Reviews and Inspections - A2. Implementing Standards - P3. Conducting Rework - F4. Attending Training/Education - P5. Planning for Quality - P6. Code Inspection - A7. Cancelled Projects - F8. Failure & Recovery of Data - F9. Defect Reporting Standards - P10. Quality Improvement - P11. Testing - A12. Conducting a Post Project Review - A13. Lost Benefits - F14. Post Project Maintenance of an component - F (This is maintenance, not enhancements.

Maintenance occurs because something wasn’t right … production bugs, misunderstood requirement, incorrectly implemented requirements.)

NOTES:




SUPPORTING / EXPLANATORY MATERIALSWhich of the following is NOT a category of the Cost of Quality?a. Failure Costb. Appraisal Costc. Testing Costd. Preventive Cost

In defining the cost of quality, appraisal costs are BEST described as:a. Costs incurred to review completed products against requirementsb. Costs which can not be recoupedc. All costs associated with defective productsd. None of the above

NOTES:


Key Principles of QualityKEY CONCEPT PRESENTATION GUIDE


• Management is responsible for quality• Producers must use effective quality control• Quality is a journey, not a destination

NOTES:



The Quality SolutionKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe three actions that must be taken by management:

1. Define Quality2. Control Quality3. Assure Quality

NOTES:



Best PracticesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSA “best practice” is: • A most effective methodology for performing a specific process. • Normally identified by benchmarking or by an independent assessment. • Also identified through winners of quality competitions such as the MBNQA, Deming Prize,

etc.

NOTES:



Six SigmaKEY CONCEPT PRESENTATION GUIDEThe key focus of a Six Sigma program is to develop a good business strategy that balances the cost, quality, features and availability considerations for products.

SUPPORTING / EXPLANATORY MATERIALSWhen considering a project to improve using Six Sigma, the following characteristics are desirable. If one or more of these characteristics is missing, there will likely be barriers to success.• The project should clearly connect to business priorities.• The problem being solved should be very important, such as a 50% process improvement.• The importance of the project should be clear to the organization.• The project should have a limited scope that can be completed in less than six months.• The project should have clear, quantitative measures that define success.Management should support and approve the project to ensure resources are available, barriers are removed, and that the project continues over time.

NOTES:




What is a Defective Process?KEY CONCEPT PRESENTATION GUIDEA defective process is a process that will produce that will produce defects because the process is flawed.

SUPPORTING / EXPLANATORY MATERIALSFor example, in cutting wood, you cannot get a clean cut from a saw that has broken teeth. Likewise, if a programmer is not properly trained in how to save information on a PC under certain circumstances, that information will be lost, and the programmer will have to repeat the steps to recreate the information.Sigma is a statistical term dealing with standard deviation. What is states is, that at certain Sigma levels, in other words, the amount of correct processing under the curve, results in a certain percentage of correct products and a certain percentage of incorrect products.Motorola began using the Sigma system because their workers were conditioned in academia to believe that 99% was close to perfections. However, if only 99 out of 100 airplanes that took off landed correctly, in other words, one crashed, that would not be viewed as a near perfect process. Note that four Sigma is typical of most job shop processes such as physicians writing prescriptions, wait staff creating bills for restaurant customers, and IT professionals building software.A defective process is a process that will produce that will produce defects because the process is flawed. For example, in cutting wood, you cannot get a clean cut from a saw that has broken teeth. Likewise, if a programmer is not properly trained in how to save information on a PC under certain circumstances, that information will be lost, and the programmer will have to repeat the steps to recreate the information.

NOTES:


Baselining and BenchmarkingKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALS• Baseline – Current level of performance• Benchmark - A comparison of one organization’s process to another organization’s process

or a comparison of one organization’s process to an industry model or standard

NOTES:




SUPPORTING / EXPLANATORY MATERIALSThe concept of Six Sigma measures defects per million parts. The number of defects for four Sigma is approximately:a. 3.4b. 233c. 6200d. 66,803The quality concept that focuses on defect rates, as opposed to percent performed correctly is called:a. TQMb. Six Sigmac. Benchmarkingd. Quality Control

NOTES:




Quality Assurance (QA) (Process Focus)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSVery few individuals can differentiate between quality control and quality assurance. Most quality assurance groups, in fact, practice quality control. You will learn the critical difference between control and assurance and how to recognize a control practice from an assurance practice.QA is activities that:• Assists in the development of processes• Fosters the concept of continuous process improvement through defect prevention• Is a staff function

NOTES:



Quality Control (QC) (Product Focus)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSQuality Control (QC) is defined as the processes and methods used to compare product quality to requirements and applicable standards, and the action taken when a nonconformance is detected. QC uses reviews and testing to focus on the detection and correction of defects before shipment of products. QC is the activity that:• Compares products to requirements• Compares work products to applicable standards• Is the responsibility of the organization and individuals who built the product or service• Is a line functionQC is the responsibility of the organization or individuals who builds the product or service

NOTES:



Quality ControlKEY CONCEPT PRESENTATION GUIDE


It is the responsibility of every project team member, not just the test or QA team, responsibility to assure the quality of the products. Developers, managers, and support teams all have a role in assuring quality.Participation in verification techniques, such as walkthroughs and inspections, is critical to the delivery of quality applications. These activities can be supported, but not replaced, by automation.For example, automation can be used to verify programming syntax but can not assure the developer is following the design strategy outlined for the project. Subjective criteria requires expert opinion which only humans can provide.• Techniques for quantifiably assuring that a work product meets its stated objectives:

- Verification: Performed during development on key artifacts• Walkthroughs, reviews, and inspections• Mentor feedback, training, checklists, and standards

- Validation: Performed after a work product is produced• Validating against established criteria • Ensuring product integrates correctly into the environment

• Reactive approach focused on defect detection and removal

NOTES:



Differentiating Between QA and QCKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSQC is an activity that verifies whether or not the product produced meets standards. QA is an activity that establishes and evaluates the processes that produce the products. If there is no process, there is no role for QA The following statements help differentiate QC from QA:• QC relates to a specific product or service.• QC verifies whether particular attributes exist, or do not exist, in a specific product or

service.• QC identifies defects for the primary purpose of correcting defects.• QC is the responsibility of the worker.• QA helps establish processes.• QA sets up measurement programs to evaluate processes.• QA identifies weaknesses in processes and improves them.• QA is a management responsibility, frequently performed by a staff function.• QA evaluates whether or not quality control is working for the primary purpose of

determining whether or not there is a weakness in the process.• QA is concerned with all of the products that will ever be produced by a process.• QA is sometimes called quality control over quality control because it evaluates whether

quality control is working.• QA personnel should never perform quality control unless doing it to validate quality control

is working.

NOTES:


Just-in-Time (JIT) TechniquesKEY CONCEPT PRESENTATION GUIDE


• Developed by Toyota• Ultimate goal is to supply each process with exactly the required items• Must have large amounts of production flexibility and very short lead times• The key is the ability of the production area to quickly switch from job to job

NOTES:



Just-in-Time (JIT) Techniques (cont)KEY CONCEPT PRESENTATION GUIDE


• JIT - The ultimate goal of JIT production is to supply each process with exactly the required items, in exactly the required quantity, at exactly the required time. There are two conditions necessary to reach this situation: large amounts of production flexibility, and very short lead times.

• TQM - Total quality management (TQM) is the term used by many to indicate an organization-wide effort of continuous process improvement. The Federal Quality Institute defines TQM as a strategic, integrated management system for achieving customer satisfaction, which involves all managers and employees and uses quantitative methods to continuously improve an organization's processes.

• Each process gets supplied the required items• Each process gets the exactly required quantity• Each process gets it delivered at exactly the required time• Requires a high level of quality

NOTES:



Sample QuestionsKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe responsibilities of this job include facilitation, process configuration, measurement, and risk analysis.a. Quality Controlb. Quality AssuranceThe responsibilities of this job include conducting inspections, reviews, testing, and focusing on the product.a. Quality Controlb. Quality Assurance

NOTES:




Sample Questions (cont.)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe primary purpose of quality control is to:a. Prevent defects from occurringb. Assign quality responsibilitiesc. Uncover defectsd. Conduct testingThe most effective test approach is to begin testing when:a. A new project beginsb. After requirementsc. After external designd. After coding

NOTES:




SUPPORTING / EXPLANATORY MATERIALSAnswers:QC - Inspection of source code.QC - Verify the jobs you submitted to be run that day have been run.QA - Select and implement a development methodology.QC - Unit testing to validate that the program works.QA - Analysis of defects to determine the stage of origin.QA - Metrics collected to measure the effectiveness of system and unit testing

NOTES:



Quality PioneersKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSDr. Deming defined 14 principles for quality, which formed the basis for the turnaround of the Japanese manufacturing industry. He believed that all 14 principles must be used concurrently to make quality happen. Philip Crosby has developed 14 steps for an organization to follow in building an effective quality program. Dr. Juran believed that managing for quality required the same attention that other functions typically receive. To ensure that adequate attention was given, he developed a trilogy consisting of three interrelated, basic managerial phases/processes: quality planning, quality control and quality improvement. These are known as “The Juran Trilogy” or “The Quality Trilogy.”

NOTES:


Quality Pioneers (cont.)KEY CONCEPT PRESENTATION GUIDE


Dr. W. Edwards Deming’s Quality Principles:1. Create a constancy of purpose2. Adopt a new philosophy3. Cease dependence on mass inspection4. End the practice of awarding business on the basis of price tag alone

NOTES:



KEY CONCEPT PRESENTATION GUIDE


Dr. W. Edwards Deming’s Quality Principles:5. Improve constantly & forever the system of production & service6. Institute training7. Adopt and institute leadership8. Drive out fear9. Break down barriers between staff areas

NOTES:





Dr. W. Edwards Deming’s Quality Principles:10. Eliminate slogans, exhortations, and targets for the work force11. Eliminate numerical quotas for the work force12. Remove barriers that rob people of pride of workmanship13. Encourage education and self-improvement14. Take action to accomplish the transformation

NOTES:



Quality Pioneers (cont)KEY CONCEPT PRESENTATION GUIDE


Philip Crosby has developed 14 steps for an organization to follow in building an effective quality program. 1. Management commitment2. The quality improvement team3. Quality measurement’4. The cost of quality

NOTES:




SUPPORTING / EXPLANATORY MATERIALSPhilip Crosby has developed 14 steps for an organization to follow in building an effective quality program. 5. Quality awareness6. Corrective action7. Zero Defect planning8. Supervisor training9. ZD (Zero Defect) Day

NOTES:




SUPPORTING / EXPLANATORY MATERIALSPhilip Crosby has developed 14 steps for an organization to follow in building an effective quality program. 10. Goal setting11. Error-cause removal12. Recognition13. Quality councils14. Do it over again

NOTES:





SUPPORTING / EXPLANATORY MATERIALSDr. Juran believed that managing for quality required the same attention that other functions typically receive. To ensure that adequate attention was given, he developed a trilogy consisting of three interrelated, basic managerial phases/processes: quality planning, quality control and quality improvement. These are known as “The Juran Trilogy” or “The Quality Trilogy.”Managing for quality requires the same attention as other projectsDeveloped a quality trilogy of basic managerial phases:• Quality Planning• Quality Control• Quality improvement

NOTES:



Total Quality ManagementKEY CONCEPT PRESENTATION GUIDETotal Quality Management = TQMJust-In-Time Technique = JIT

SUPPORTING / EXPLANATORY MATERIALSTQM - Total quality management (TQM) is the term used by many to indicate an

organization-wide effort of continuous process improvement. The Federal Quality Institute defines TQM as a strategic, integrated management system for achieving customer satisfaction, which involves all managers and employees and uses quantitative methods to continuously improve an organization's processes.

JIT - The ultimate goal of JIT production is to supply each process with exactly the required items, in exactly the required quantity, at exactly the required time. There are two conditions necessary to reach this situation: large amounts of production flexibility, and very short lead times.

A managerial philosophy based on the work of the Pioneers:• Cross functional goals• A process of controlled change• Requires a team effort• Communications must be encouraged throughout the organization• Implementation is not a one-size fits all

NOTES:




SUPPORTING / EXPLANATORY MATERIALSWhich of the following is NOT one of Dr. W. Edward Deming’s fourteen quality principles?a. Improve supervisionb. Drive out fearc. Learn the new philosophyd. Zero defectsTotal Quality Management means:a. Managing the quality of entire systemb. Emphasizing the importance of managing qualityc. Implementing company-wide quality attitudes and methodologiesd. Having the Quality Manager included in senior management

NOTES:




SUPPORTING / EXPLANATORY MATERIALSList 3 items that you consider important to having quality control in systems development. Explain each answer:• To reduce risks inherent in computer systems. • To detect variation from specification/expectation. • Establishing confidence that a program does what it is suppose to do. • Prove that the program is no good. • Detecting specification errors and deviations from specifications.

NOTES:



Skill Category 2KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe most important prerequisite for successful implementation of any major quality initiative is commitment from executive management. It is management’s responsibility to establish strategic objectives and build an infrastructure that is strategically aligned to those objectives. This category describes the management processes used to establish the foundation of a quality-managed environmentQuality Leadership• Leadership Concepts• Quality Management Infrastructure• Quality Environment

NOTES:



What Commitment MeansKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSManagement commitment is the single most important requirement for successful implementation of quality management. There is no precedent of successful quality improvement without executive management and the management team leading the effort. Having management commitment does not guarantee quality management success; it only improves the odds for successful implementation. The entire organization must eventually become committed to quality management. • Understand the concepts of quality management• Adopt behaviors required to show commitment• Accept the need to change to a participative management style• Lead in the development of a quality management implementation plan• Lead the formation of the implementation organization• Provide funds for training• Provide time for training and meetings• Publicize and reward results• Monitor and measure progress• Provide personnel and other resources

NOTES:



New Behaviors for ManagementKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSMost managers practice traditional management. They have been taught to control their organization and employees, using an “I’ll tell you what to do, and you’ll do it” mentality. Many managers look at the short-term because their commitment to the organization is short range. The key differences in philosophy between traditional management and quality management environments are illustrated in the table below.

Traditional Management Philosophy• Controls each result• Who made the error?• Correct the error• Employees are the problem• Management accountable to their

manager• Competition between organizations• Motivation from fear of failure• Management of outputs (results) -

focusing on detection of defects• Fire fighting• Accomplishment from meeting quotas,

the monthly or quarterly bottom line

Quality Management Philosophy• Use the process• What allowed the error?• Reduce variation and prevent the error• Refine the process• Management accountable to the customer• Teamwork• Motivation from within (self)• Management of process inputs - methods or

sources of variation that focus on preventing defects

• Continuous process improvement• Accomplishment from long-term impact of

improving processes

NOTES:



New Behaviors for Management (cont.)KEY CONCEPT PRESENTATION GUIDEThe programs needed to change from a traditional to quality management culture must be customized for an organization and its current culture.


NOTES:

Category Traditional Culture Quality Management Culture Mission Maximum return on investment

(ROI), management by objec-tives (MBO)

Ethical behavior and customer satisfaction, climate for continuous improvement, ROI as a measure of performance

Customer Requirements

Incomplete or ambiguous understanding of customer requirements

Uses a systematic approach to seek out, understand, and satisfy both internal and external customer requirements

Suppliers Undirected relationship Partnership

Objectives Orientation to short-term objec-tives and actions with limited long-term perspective

Deliberate balance of long-term goals with successive short-term objectives

Improvement Acceptance of process variability and subsequent corrective action as the norm

Understanding and continually improving the process

Problem-Solving

Unstructured individualistic problem-solving and decision-making

Predominantly participative and interdisciplin-ary problem-solving and decision-making based on substantive data

Jobs and People

Functional, narrow scope, management controlled

Management and employee involvement, work teams, integrated functions

Management Style

Management style with uncertain objectives that instills fear of failure

Open style with clear and consistent objectives, encouraging group-derived continuous improvement

Role of Manager

Plan, organize, assign, control and enforce

Communicate, consult, delegate, coach, mentor, remove barriers, and establish trust

Rewards & Recognition

Pay by job, few team incentives Individual and group recognition and rewards, negotiated criteria

Measurement Orientation toward data gathering for problem identification

Data used to understand and continuously improve processes


Leader vs. ManagerKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSManagement commitment is the single most important requirement for successful implementation of quality management. There is no precedent of successful quality improvement without executive management and the management team leading the effort. Having management commitment does not guarantee quality management success; it only improves the odds for successful implementation. The entire organization must eventually become committed to quality management. • Manager – Works within the system by following accepted practices• Leader – Determines where the organization needs to be; then does what is necessary to

get there

NOTES:



LeadershipKEY CONCEPT PRESENTATION GUIDELeadership and management are two different things. While a manager works within the system following the accepted practices of the system, a leader determines where the organization needs to be, and then does what is necessary to get there.

SUPPORTING / EXPLANATORY MATERIALSIn a business context, leadership is the ability to build the commitment of employees, to endow an organization with a positive perception of itself, and to give employees a positive perception of their role within the business.

NOTES:

Characteristic Behavior Demonstrated Substance Helps others achieve needed substance. Growth Helps others achieve personal and career growth. Opportunities Creates opportunities for others to make uninhibited

contributions to the enterprise. Environment Creates an environment conducive to performance. Empowerment Empowers others. Obstacles Removes obstacles to performance. Support Helps others do what they decide is in their own best interest. Coaching, training, education

Coaches, trains and educates others.

Coordination Helps coordinate the work of others. Market, Outlets Creates a market and outlet for the talents of others. Resources for others Acquires the resources others need. Uniquely Equipped Does what is necessary for success, which others are not

capable of doing. Strategies Creates a vision, communication and trust through positioning

and deployment of self. Persistent Pursues tirelessly the mission of the organization through

linkage with other leaders on strategic issues. Ethical, open, honest Maintains a totally open and honest state with others.



Fundamental MistakesKEY CONCEPT PRESENTATION GUIDE


• Isolation – Lack of maintaining regular contact with people they manage• Inability to reward – Not taking the time to reward the people they manage• Lack of business perspective – Inability to take advantage of opportunities

NOTES:



Quality KnowledgeKEY CONCEPT PRESENTATION GUIDE


• Modeling – Do as I do, not as I say – Manage with language, statistics, and process improvement teams

• Coaching – Instructing, directing, or prompting• Reinforcing – Positive feedback should be come second nature

NOTES:



Awareness TrainingKEY CONCEPT PRESENTATION GUIDE


Prepare for Awareness Training• Select awareness topic• Identify the topics customers• Define objective for awareness training• Define customer benefits• Develop administrative training plan

NOTES:



Awareness Training (cont)KEY CONCEPT PRESENTATION GUIDE


Conduct Awareness Training• Attendees’ needs• Awareness topic/product• Identify objections to product/problem• Overcome objections• Recommend course of Action

NOTES:



Quality Management InfrastructureKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe Bottom-up Approach - This approach sends the message that quality management is something for the employees, but not necessarily for management. This approach is like swimming against the current, and leads to frustration because resources are not readily provided to teams when required. The Middle-out Approach - Starting in the middle of the organization and then progressing simultaneously to the top and bottom of the organization can be successful.The Top-down Approach - Top-down has the highest probability for success, although success is not guaranteed. This model fosters management involvement - the single most important requirement for quality management success.

NOTES:




Quality Management Infrastructure (cont.)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSEach level is linked to the other by the common objective of making people capable of combined performance. The figure above shows the three levels of infrastructure normally needed.

NOTES:



Quality CouncilKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSA Quality Council is composed of the organization’s top executive and his or her direct reports.It may also be referred to as an Executive Council. The Quality Council acts as the steering group to develop the organization’s mission, vision, goals, values, and quality policy. Quality Council:• Initiates and commits to quality management• Incorporates quality management into strategic planning• Allocates resources: budget, people, time• Establishes lower level committees• Defines and deploys policies• Process approval authority• Acts on unresolved items• Provides review and oversight of progress

NOTES:



Management CommitteesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSManagement committees (also called Process Management Committees) are composed of middle managers and/or key staff personnel, and are responsible for deploying quality management practices throughout the organization. One or more committees may be needed depending on the organization’s size and functional diversity. Committees should represent all the skills and functions needed to work on the specific processes or activities.Management Committees:• Gains an understanding of the organization’s mission, goals, and priorities• Commission the development of a process inventory, and process maps• Establish work groups• Monitor progress• Review and approve processes• Quality planning

NOTES:


Teams, Work GroupsKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSTeams are formed under any number of names depending on their purpose. Common names and functions are:• Process Development Teams: They develop processes, standards, etc.• Process Improvement Teams: They improve existing processes, standards, etc.• Work Groups: They perform specific tasks such as JAD, inspections, or testing

- Subject matter experts- Study, define, and improve processes- Pilot newly defined processes- Provide process training- Assist in deployment- Serve as process consultants

NOTES:



Guidelines for TeamsKEY CONCEPT PRESENTATION GUIDE


• The committee selects the teams• Each team has a chairperson• Keep is small (3-5 people)• Develop a work plan• Meet regularly to review the work• Different teams members participate• Team must reach consensus

NOTES:



Team Development PhasesKEY CONCEPT PRESENTATION GUIDE


• Forming• Storming• Norming• Conforming

NOTES:



Group CompatibilityKEY CONCEPT PRESENTATION GUIDE


• Inclusion• Control• Affection

NOTES:



Written ReportsKEY CONCEPT PRESENTATION GUIDE


Good ideas are of little value unless they are accepted and implemented. The QA report is designed to convey information and to change behavior. QA analysts write a report, distribute it, and follow up on the recommendations. The value of the quality function can be rated on whether management accepts the report. • Establish report objectives• Gather factual data and recommendations• Develop a report outline• Draft the report• Review the draft for reasonableness• Have the report reviewed for readability• Review the report with involved parties• Review the report with management• Distribute the report and follow up

NOTES:




The Six Attributes of an Effective Quality EnvironmentKEY CONCEPT PRESENTATION GUIDE


Five major accounting associations (i.e., Financial Executives International, American Institute of Public Accountants, American Accounting Association, The Institute of Internal Auditors, and the Institute of Management Accountants), which is referred to as COSO (Committee of Sponsoring Organizations), was organized to provide guidance on evaluating internal control. They issued this guidance as the COSO Internal Control Framework. The COSO Framework identified the six quality attributes. For each attribute, they listed several control objectives that if implemented would define each of the six attributes.1. Integrity and Ethical Values2. Commitment to Competence3. Management’s Philosophy and Operating Style4. Organizational Structure5. Assignment of Authority and Responsibility6. Human Resource Policies and PracticesImplement Mission, Vision, Goals, Values, and Quality Policy

NOTES:


What is a Brief Description of Management Support?KEY CONCEPT PRESENTATION GUIDE


Management’s “tone” is representative of the environment that management has established that influence the way testers work.

NOTES:



Setting the Proper “Tone” at the TopKEY CONCEPT PRESENTATION GUIDE


• Integrity and ethical values- Incentives- Providing resources so testers can do an effective job

• Commitment to competency• Philosophy and operating style• Organizational structure

NOTES:



Integrity and Ethical ValuesKEY CONCEPT PRESENTATION GUIDE


Key Questions:• Do you do what you say you will?• Do you treat everyone equally?• Are you honest with your suppliers, colleagues and customers?• Are a team member?• Do you work hard every day?

NOTES:



CommunicationsKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSHearing the speaker requires an understanding of the five channels of communication incorporated into speech. Much of listening occurs beyond merely hearing the words. Attending to the speaker is sometimes referred to as being an active listener. Devote your full attention to the speaker to confirm that what you heard is what the speaker intended you to hear. When one has deciphered the information channel (i.e., what the subject is) and related the importance of that subject to the audience, listening must be adjusted to ensure that we get the message we need.• Provide Constructive Criticism• Achieving Effective Listening

- Hearing- Attending- Understanding

• Personal Persuasion• Resolving Customer Complaints• Written Reports

NOTES:



Listening SkillsKEY CONCEPT PRESENTATION GUIDE


Oral communication (which includes listening) is rated as the number-one skill for the tester.Some facts about listening include:• Listening is the first language skill that we develop as children; however, it is rarely taught as

a skill. • Listening is the most frequently used form of communication.• Sales people often lose sales because the believe talking is more important than listening.

NOTES:



3-Step Listening ProcessKEY CONCEPT PRESENTATION GUIDE


The listening process involves three separate steps: 1. Hearing the speaker2. Attending to the speaker3. Understanding the speaker

NOTES:



3-Step Listening Process (cont)KEY CONCEPT PRESENTATION GUIDE


Step 1. Hearing the speaker.Hearing the speaker requires an understanding of the five channels of communication incorporated into speech:• Information Channel - The speaker’s subject.• Verbal Channel - The words used by the speaker.• Vocal Channel - The tone of voice associated with the various words.• Body Channel - The body movements and gestures associated with the information being

conveyed.• Graphic Channel - The pictures, charts, etc. that the speaker uses to emphasize or illustrate

the material being discussed

NOTES:





Step 2. Attending to the Speaker (sometimes referred to as being an active listener).Some suggestions to help in attending to the speaker are:• Concentrate on the speaker.• Maintain eye contact (80%).• Provide feedback.• Restate what you heard.

NOTES:





Step 3. Understanding the SpeakerThere are five types of listening. The type chosen will have an impact on the ability to understand what the speaker is saying.• Type 1: Discriminative Listening• Type 2: Comprehensive Listening• Type 3: Therapeutic Listening• Type 4: Critical Listening• Type 5: Appreciative or Enjoyment Listening

NOTES:



Implementing a Mission, Vision, and Quality Policy KEY CONCEPT PRESENTATION GUIDE


Management may begin by:• Using Industry examples• Understanding of the current culture• Establishing an action plan to develop these statements

NOTES:



MissionKEY CONCEPT PRESENTATION GUIDE


• Explains “WHY” a company, organization, or activity exists and what is done to accomplish it.

• The mission should focus on products and services, and it should be customer-oriented.• “Our mission is to improve continually our products and services to meet our customers’

needs”

NOTES:



VisionKEY CONCEPT PRESENTATION GUIDE


• Establishes where the organization desires to move from its current state.• Senior management should establish the vision, ensuring how it contributes to the business

is clear.• “Our vision is to be a company with a strong customer base, known for reliability, trust and

integrity”

NOTES:



GoalsKEY CONCEPT PRESENTATION GUIDE


• Explains how the vision is to be achieved and are consistent with the vision• “We will build our trust one customer at a time”

NOTES:



PolicyKEY CONCEPT PRESENTATION GUIDE


• Executive management’s commitment to quality should be expressed in writing to all employees in the form of a quality policy.

• “It is our policy to build a quality product to satisfy our customer”

NOTES:



What is an Effective Work EnvironmentKEY CONCEPT PRESENTATION GUIDE


These characteristics of a quality work environment are the same items we emphasized in part 1 of this course on quality principles and concepts. The students might want to use this as a quick checklist to evaluate their quality work environment. If they got 3 yeses, meaning this is how their organization operates, they are well underway to a quality work environment.• Listening to customers to determine their requirements• Identifying costs of quality and focusing on prevention• Doing the right thing right the first time• Continuing process improvement• Taking ownership at all levels of the organization• Demonstrating executive leadership and commitment

NOTES:



What is Not an Effective Work EnvironmentKEY CONCEPT PRESENTATION GUIDE


These items, which represent the lack of a quality work environment, are basically the negative statements regarding what a good work environment is. Again, a student could use this as a quick checklist, and if they answer yes to many or all of these items, they have a lot of work to do to improve their quality environment.• Assuming you know your customer’s requirements• Overlooking the hidden costs of poor quality• Doing it over to make it right• One-time fixes• Assigning responsibility for quality to one department (e.g., QA)• Assigning responsibility for product quality to workers

NOTES:



The Environment is Essential Because Quality:KEY CONCEPT PRESENTATION GUIDE


Emphasize that most of the principles of quality are just plain common sense. For example, your children will be much more likely to do good school work if you motivate them to do good work, describe the vision for them of the opportunities for a good student and reward them on good work, than if you attempt to get good school work from fear and punishment.Emphasize that success, meaning quality from the customer’s view, quality from the producer’s view and alignment to goals and objectives is not a technical problem. It cannot be solved with more hardware and software. It’s a people problem that can only occur through management initiatives. A culture is defined as the ideas, customs, skills, arts, etc. of a given people in a given period. The IT culture is defined by its vision, values, mission, approaches to work, rewards, punishments, excitement one experiences in the work environment, the way in which people view management and themselves as well as management's view of the way they want the IT organization to operate.Explain that if the students had an opportunity to go from like organization to like organization they would easily experience the difference in culture. In their personal lives, they probably experience a different culture in their home, their church, sporting events, clubs and so forth. Most people know the type of culture that they want to be in, but have difficulty defining that culture. We will attempt in the next few minutes to describe the different cultures.

NOTES:




Manage to Facilitate SuccessKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThis course will explain what is meant by vision, value and mission, give examples, and explain the importance. It also will provide guidelines on building a quality culture and a quality infrastructure.Managing to Facilitate Success means doing the following:• Defining the IT vision, value & mission• Building a quality infrastructure• Integrating quality planning into IT planning• Encouraging a quality culture by monitoring and enforcing organizational policies and

procedures

NOTES:




SUPPORTING / EXPLANATORY MATERIALSExplain the major difference between a leader and a manager?Leader:• Forms a Partnership• Seeks out, understands, and satisfies both internal and external customer requirements• Understands and continually improves the processManager:• Undirected relationship• Incomplete or ambiguous understanding of customer requirements• Acceptance of process variability and subsequent corrective action as the norm

NOTES:



Skill Category 3KEY CONCEPT PRESENTATION GUIDEOrganizations need to establish baselines of performance for quality, productivity and customer satisfaction.


These baselines are used to document current performance and document improvements by showing changes from a baseline. In order to establish a baseline, a model and/or goal must be established for use in measuring against, to determine the baseline. • Quality Baseline Concepts• Methods used for Establishing Baselines• Model and Assessment Fundamentals• Industry Quality Models

NOTES:



Baselines DefinedKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSIT organizations have established many different baselines to evaluate current performance and to measure improvement. To help understand the type of baselines that are used in IT, QAI has categorized four most commonly used baselines as listed below. Each one of these will be discussed individually.• Customer surveys• Benchmarking• Management established criteria• Industry models

• A baseline is a current level of performance.• An assessment determines the baseline against the model. (Goals to be accomplished)

NOTES:


Types of BaselinesKEY CONCEPT PRESENTATION GUIDE


Baseline Study is one designed to quantitatively show the current status of an activity• Evaluate the entire population• Sample Survey

NOTES:



Baseline StudiesKEY CONCEPT PRESENTATION GUIDE


Three logical groups to conduct baseline studies:• Quality Assurance Groups• Quality tasks forces• IT Management

NOTES:



Steps to Perform a Baseline StudyKEY CONCEPT PRESENTATION GUIDE


• Identify products/services to be surveyed• Define conformance and nonconformance• Identify survey population• Identify size of population to be surveyed• Develop survey instrument• Conduct survey• Follow up on incomplete surveys• Accumulate and present survey results• Take action and notify participants of that action

NOTES:



Conducting Objective Baseline StudiesKEY CONCEPT PRESENTATION GUIDE


Objective baseline studies are ones which are viewed as factual and non-argumentative.Examples of objective baseline measures include:• Project completed on schedule• Lines of code• Number of programs• Number of people assigned to a project• Number of abnormal terminations

NOTES:



Conducting Objective Baseline Studies (cont)KEY CONCEPT PRESENTATION GUIDE


Subjective baseline studies will be the most commonly conducted studies in measuring quality and productivity.Examples of Subjective baseline measures:• Customer satisfaction• Effectiveness of standards/manuals• Helpfulness of methodologies to solve problems• Areas/activities causing the greatest impediments to quality/productivity• Causes for missed schedules/over-budget conditions• Understandability of training materials• Value of tools• Importance of activities/standards/methods/tools to individual activity

NOTES:



Establishing BaselinesKEY CONCEPT PRESENTATION GUIDE


Four commonly used baselines are:• Customer Surveys• Benchmarking• Management established criteria• Industry Models

NOTES:



Purpose of a ModelKEY CONCEPT PRESENTATION GUIDEA specific model may or may not fully apply to an IT organization, and selecting a model does not have to be an all or nothing decision.


• A model is a goal to be accomplished• Most models define the minimum that has to be accomplished• Models are normally developed under the auspices of a national or international standards

organization• Organizations choose to use models to:

- Satisfy business goals and objectives- Satisfy requirements imposed by customers- Leapfrog the competition- Provide guidance (road map) for continuous improvement

NOTES:




Staged Versus Continuous ModelsKEY CONCEPT PRESENTATION GUIDEA specific model may or may not fully apply to an IT organization, and selecting a model does not have to be an all or nothing decision.

SUPPORTING / EXPLANATORY MATERIALSStaged models:• Composed of a number of distinct levels of maturity• Each level of maturity is decomposed into specific processes.• Each level of maturity serves as the foundation for the next level of maturity.Continuous models:• Processes are improved individually along a maturity scale that is independent of each

other.

NOTES:



Model Selection ProcessKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThere are a number of improvement models:• Software Engineering Institute’s Capability Maturity Model (CMM) and the CMMI• Malcolm Baldrige• ISO 9001: 2000• ISO 12207• ISO 15504 (SPICE)An organization should consider the following when selecting a model:1. Applicability of the model to the organization’s goals and objectives2. Management commitment3. Need for baseline assessments4. Need for measurable goals and objectives

NOTES:



Using Models For Assessment and BaselinesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSUsing Models For Assessment and Baselines• A baseline is a current level of performance.• An assessment determines the baseline against the model. (Goals to be accomplished)

NOTES:



Software Engineering Institute’s Capability Maturity ModelKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSAs shown in the above figure, the CMMI® framework is a method for organizing these steps into five levels of maturity that lay successive foundations to support short and long-term process improvement initiatives.The five maturity levels define an ordinal scale that enables an organization to determine its level of process capability. The framework is also an aid to quality planning as it affords organizations the opportunity to prioritize improvement efforts.Software Engineering Institute’s Capability Maturity Model levels include:

Ad hoc Initial

Disciplined Repeatable

Standard/Consistent Defined

Predictable Managed

Continuous Improvement Optimized

NOTES:



SEI CMMI - Level 1KEY CONCEPT PRESENTATION GUIDEWhen an organization lacks sound management practices, the benefits of good software engineering practices are undermined by reaction-driven commitments.

SUPPORTING / EXPLANATORY MATERIALSThe process capability at Level 1 is considered ad hoc because the software development process constantly changes as the work progresses. Schedules, budgets, functionality, and product quality are generally unpredictable.Level 1 characteristics include:• Unstable work environment• Organization lacks sound management practices• Code and fix methodology• Development processes constantly change• Cannot be repeated from project-to-project

NOTES:



SEI CMMI - Level 2KEY CONCEPT PRESENTATION GUIDEPolicies for managing a software project and procedures to implement those policies are established.

SUPPORTING / EXPLANATORY MATERIALSLevel 2 – Repeatable Key Process Areas• Requirements Management• Software Project Planning• Software Project Tracking and Oversight• Software Subcontract Management• Software Quality Assurance• Software Configuration ManagementLevel 2 characteristics include:• Basic management controls are installed• Planning and managing is based on experience• Realistic commitments• Budget, schedules, and functionality are tracked• Has the ability to repeat past successes

NOTES:



SEI CMMI - Level 3KEY CONCEPT PRESENTATION GUIDEThe standard engineering and management processes for developing and maintaining software across an organization are documented, and these processes are integrated as a whole

SUPPORTING / EXPLANATORY MATERIALSLevel 3 – Defined Key Process Areas• Organization Process Focus• Organization Process Definition• Training• Integrated Software Management• Software Product Engineering• Intergroup Coordination• Peer ReviewsLevel 3 characteristics include:• Organization uses a standard SDLC• Documented and integrated as a whole• A group is assigned ownership of the organization’s process activities• Organization-wide training program• Engineering and project management activities are integrated

NOTES:



SEI CMMI - Level 4KEY CONCEPT PRESENTATION GUIDEThe capability of Level 4 organizations is summarized as predictable because the process is measured and operates within measurable limits.

SUPPORTING / EXPLANATORY MATERIALSLevel 4 – Managed Key Process Areas• Software Quality Management• Quantitative Quality ManagementLevel 4 characteristics include:• Quantitative goals and objectives are set• Productivity and quality are measured• Process control is achieved by narrowing variability• Is predictable because processes are measured and operate within measurable limits

NOTES:



SEI CMMI - Level 5KEY CONCEPT PRESENTATION GUIDEThe organization has the means to identify weaknesses and strengthen the process proactively, with the goal of preventing the occurrence of defects.

SUPPORTING / EXPLANATORY MATERIALSLevel 5 – Optimizing Key Process Areas• Defect Prevention• Technology Change Management• Process Change ManagementLevel 5 characteristics include:• Entire organizational focus is on continuous process improvement• Root cause analysis• Lessons learned

NOTES:



Malcolm Baldrige National Quality Award KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe United States national quality award originated from the Malcolm Baldrige National Quality Improvement Act (Public Law 100-107), signed by President Ronald Reagan on August 20, 1987. That act, named after a former Secretary of Commerce, called for the creation of a national quality award and the development of guidelines and criteria that organizations could use to evaluate their quality improvement efforts. • The United State’s national quality award• Enacted through Public Law 100-107• Named for the then Secretary of Commerce• Awards are given annually• Awards are given in five categories:

- Manufacturing- Service Organizations- Small Business- Education Organizations- Health Care Organizations

NOTES:



National Institute of Standards and TechnologyKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe U.S. Department of Commerce is responsible for the MBNQA and Program. The National Institute of Standards (NIST), an agency of the Department’s Technology Administration manages the Baldrige Program.• Responsible for the MBNQA • Agency of the Department of Technology Administration• Manages the Baldrige Program by governing:

- A board of overseers- A board of examiners- The award recipients

NOTES:



MBNQA Awards CriteriaKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSAwards Criteria:1. Leadership2. Strategic Planning3. Customer and Market Focus4. Information and Analysis5. Human Resources6. Process Management7. Business Results

NOTES:



MBNQA Scoring SystemKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSProcess refers to the methods your organization uses and improves to address the Item requirements in Categories 1-6. The four factors used to evaluate process are Approach, Deployment, Learning, and Integration (A-D-L-I).“Approach” refers to:• The methods used to accomplish the process• The appropriateness of the methods to the Item requirements• The effectiveness of your use of the methods• The degree to which the approach is repeatable and based on reliable data and information

(i.e., systematic)Applicant responses are scored on two evaluation dimensions:• Process

- Approach- Deployment- Learning and Integration

• Results - Current Level of Performance- Rate and Breadth- Benchmarks- Result measures

NOTES:



ISO 9001: 2000KEY CONCEPT PRESENTATION GUIDEThe ISO 9000 family of standards contains 3 standards and many supporting documents.

SUPPORTING / EXPLANATORY MATERIALSAn International quality model developed under the auspices of the International Organization for Standardization. (ISO)The Four primary standards are:• ISO 9000 – Introduction and guide• ISO 9001 – The quality model• ISO 9004 – Guide to help with installation• ISO 10011- Guidelines for auditing

NOTES:



ISO 9001: 2000 Model OverviewKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThe standard is based upon eight quality principles:1. Customer focus2. Leadership3. Involvement of people4. Process approach5. System approach to management6. Continuous improvement7. Factual approach to decision making8. Mutually beneficial supplier relationships

NOTES:


ISO 9001: 2000 Process ModelKEY CONCEPT PRESENTATION GUIDE


NOTES:

CCUUSSTTOOMMEERR

CCUUSSTTOOMMEERR

Management Responsibility

Resource Management

Product Realization

Input OutputProcess

Measurement, Analysis, Improvement

CCUUSSTTOOMMEERR

CCUUSSTTOOMMEERR

Management Responsibility

Resource Management

Product Realization

Input OutputProcess

Measurement, Analysis, Improvement



Model Overview - IS0/IEC 12207KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSISO/IEC 12207, which was published in 1995, is the international standard that covers the software life cycle from concept through retirement. It contains a framework for managing, controlling, and improving the software life cycle activities. For each process, the standard also describes the activities and tasks involved, defining specific responsibilities and identifying outputs of activities and tasks. Since it is a high-level standard, it does not detail how to perform the activities and tasks. • An international standard• Covers the software life cycle from concept through retirement• Describes the major life cycle phases:

- Activities and tasks involved- How phases interface with one another

NOTES:




Model Overview - IS0/IEC 15504KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSSoftware Process Improvement Capability DeterminationAn international standard:• Provides the framework for the assessment of processes• It produces a process profile that represents the maturity of the process rather than a pass/

fail rating

NOTES:



Process Capability DimensionsKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSProcess Capability Dimensions:

5 Optimizing

4 Predictable

3 Established

2 Managed

1 Performed

0 Not Performed

NOTES:




SUPPORTING / EXPLANATORY MATERIALSThe Malcolm Baldrige National Quality Award was created by Public Law 100-107 and is named for Malcolm Baldrige, who served as the Secretary of Defense.a. Trueb. FalseObjective measures are those, which can be accomplished by counting. Which of the following is NOT an objective baseline measurement that can be used to establish baselines?a. Project completed on scheduleb. Number of programsc. Guessing at estimationsd. Number of abnormal terminations

NOTES:




SUPPORTING / EXPLANATORY MATERIALSBriefly explain a major similarity and major difference between the SEI’s CMM model and the ISO’s SPICE model.Similarity:• Maturity LevelsDifferences:• CMM is audited by external company• SPICE is a self-assessment

NOTES:



Skill Category 4KEY CONCEPT PRESENTATION GUIDEQuality Assurance is a professional competency whose focus is directed at critical processes used to build products and services.

SUPPORTING / EXPLANATORY MATERIALSThe profession is charged with the responsibility for tactical process improvement initiatives that are strategically aligned to the goals of the organization. This category describes the management processes used to establish the foundation of a quality-managed environment:Quality Assurance• Establishing a Function to Promote and Manage Quality• Quality Tools• Process Deployment• Internal Auditing and Quality Assurance

NOTES:



Establishing a Function to Promote and Manage QualityKEY CONCEPT PRESENTATION GUIDEA quality function exists when a specific individual/group is assigned the responsibility to assist in improving quality.

SUPPORTING / EXPLANATORY MATERIALSWhile individual workers have responsibility for the quality of their products and services, it is management's responsibility to ensure that the environment is one in which quality can flourish. The ultimate responsibility for quality rests with senior management.Many people argue that because everyone has some quality responsibility, a staff function for quality is unnecessary. That argument is theoretically correct, but in practice unless there is a group charged with responsibility for ensuring quality, the pressures of other priorities such as meeting schedules and budgets frequently takes precedence over quality. A quality function exists when a specific individual/group is assigned the responsibility to assist in improving quality.

NOTES:



Four Variables in a System Development Project KEY CONCEPT PRESENTATION GUIDEThe four project variables are:1. Scope2. Schedule3. Resources4. Quality.


The management challenge in completing the project can be illustrated as a dashboard of four system attribute dials, which get set according to the project criteria as illustrated in the figure above. The four dials are interconnected, so movement of one dial affects one or more of the other dials.

NOTES:



The Challenges of Implementing a Quality FunctionKEY CONCEPT PRESENTATION GUIDETwo basic challenges emerge, and either or both may be present in any particular organization.

SUPPORTING / EXPLANATORY MATERIALSIf an organization is not ready, and also lacks the sales skills to convince the management group, the probability of success is so low that it is best not to bother with implementation at this time. A ready organization with good salesmanship can take care of itself. The other scenarios present different challenges. • Organizations that are not ready, but have the salesmanship, are referred to as “Challenge

1.” • Organizations that are ready, but lack the salesmanship, are referred to as “Challenge 2.”

NOTES:



How the Quality Function Matures Over TimeKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSIT management involved in a corporate quality management program should undertake the following four actions in an effort to rethink and assign their quality responsibilities:1. Determine what's new, changed, and continued quality responsibilities will be assigned to

the IT group as a result of the corporate quality program.2. Evaluate the capability of the IT group to deploy quality initiatives within and outside the

group. Define the skills, commitment, and quality approaches currently available to line management.

3. Determine the value added by utilizing IT QA analysts to support corporate quality management initiatives.

4. Develop a plan to assimilate and leverage the current function into the quality support activity needed to support both the corporate quality management program and IT quality responsibilities.

• Develop a charter• Identify the quality manager• Locate organizationally the quality function• Build support for quality• Staff and train the quality function• Build and deploy the quality tool box• Drive the implementation

NOTES:


Implementing an IT Quality FunctionKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSIT management involved in a corporate quality management program should undertake the following four actions in an effort to rethink and assign their quality responsibilities:• Determine what's new, changed, and continued quality responsibilities will be assigned to

the IT group as a result of the corporate quality program.• Evaluate the capability of the IT group to deploy quality initiatives within and outside the

group. Define the skills, commitment, and quality approaches currently available to line management.

• Determine the value added by utilizing IT QA analysts to support corporate quality management initiatives.

• Develop a plan to assimilate and leverage the current function into the quality support activity needed to support both the corporate quality management program and IT quality responsibilities.

1. Develop a charter2. Identify the quality manager3. Locate organizationally the quality function4. Build support for quality5. Staff and train the quality function6. Build and deploy the quality tool box7. Drive the implementation

NOTES:



IT Quality PlanKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSAn IT Quality Plan Has Two Objectives:• Support the organization’s quality policy• Ensure the quality of the strategic planAn IT Quality Plan should include the following:1. A reference to the organization’s quality policy2. Current baseline data3. Long-term quality goals4. Short-term quality goals5. The means to implement quality objectives6. Resources required

NOTES:



Quality ToolsKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSA quality toolbox is a set of tools, which assists in defining, controlling, and improving quality. The starter toolbox for a QA analyst contains the seven tools listed below. These tools and others are discussed later:• Brainstorming• Flowcharts• Cause-and-effect diagrams• Histograms• Pareto charts• Control charts• Scatter diagrams

NOTES:



Tool CategoriesKEY CONCEPT PRESENTATION GUIDEA quality toolbox is a set of tools, which assists in defining, controlling, and improving quality.

SUPPORTING / EXPLANATORY MATERIALSThe starter toolbox for a QA analyst contains the seven tools listed below. These tools and others are discussed later:• Brainstorming• Flowcharts• Cause-and-effect diagrams• Histograms• Pareto charts• Control charts• Scatter diagramsTool Categories• Management Tools• Statistical Tools• Presentation ToolsEach tool must be:• Selected• Learned• Used in practice

NOTES:



Management ToolsKEY CONCEPT PRESENTATION GUIDE


• Brainstorming• Affinity Diagram• Nominal group technique• Cause and effect diagram• Force field analysis• Flowchart and process map• Benchmarking• Matrix• Quality Function Deployment

NOTES:



Statistical ToolsKEY CONCEPT PRESENTATION GUIDEThe tools covered in this section are used to collect, view, and analyze numbers.


• Check sheet• Histogram• Pareto chart• Run chart• Control chart• Scatter plot

NOTES:




Tools for PresentationKEY CONCEPT PRESENTATION GUIDEPresentations are an integral part of the change process.

SUPPORTING / EXPLANATORY MATERIALSThe involved parties, sometimes called stakeholders, need to be convinced that a proposed change is beneficial, or want to see reports during and after implementation. Stakeholders include management, the individuals that will use the changed process, and the individuals impacted by the changed process. Tools for Presentations include:• Tables• Line Charts• Bar Charts• Pie Charts• Stem and Leaf Charts• Force-Field Analysis

NOTES:



Process DeploymentKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSOne of the most difficult tasks facing any IT function is changing the way that function operates. In most organizations, stability is the norm and change is abnormal. That cycle needs to be reversed, if quality and productivity are to be constantly improved.People resist change because:• It is more difficult to implement change than to develop the approach• People do not like change imposed on them• Personal risk: can be successful with current processes• Making routine mistakes as a result of change• Management may be more committed to meeting schedules and budgets

NOTES:


Getting Buy-In for ChangeKEY CONCEPT PRESENTATION GUIDE


NOTES:

Identify Customer NeedsIdentify Customer Needs

Present SolutionsPresent Solutions

Identify BarriersIdentify Barriers

Address BarriersAddress Barriers

Obtain ApprovalObtain Approval

PlanPlan

DoDo

CheckCheck

ActAct

Identify Customer NeedsIdentify Customer Needs

Present SolutionsPresent Solutions

Identify BarriersIdentify Barriers

Address BarriersAddress Barriers

Obtain ApprovalObtain Approval

PlanPlan

DoDo

CheckCheck

ActAct



Effective KEY CONCEPT PRESENTATION GUIDE


This formula indicates that if neither the individual nor the environment changes, there is not change in behavior

NOTES:

IndividualIndividual ++==BehaviorBehavior EnvironmentEnvironmentIndividualIndividual ++==BehaviorBehavior EnvironmentEnvironment



The Deployment Process PhasesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSThere are three deployment phases - assessment, strategic, and tactical. The assessment and strategic deployment phases represent the Planning component of the PDCA cycle. The tactical deployment phase represents the Do, Check, and Act components of the PDCA cycle.• Phase One: Assessment

- General assessments- Specific assessments

NOTES:



The Deployment Process Phases (cont)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALS• Phase Two: Strategic

- Set the goals- Identify possible solutions- Select and sequence/prioritize change approaches- Develop/acquire/customize the approach

NOTES:



The Deployment Process Phases (cont)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALS• Phase Three: Tactical

- Hear the approach- Accept the approach- Learn the approach- Remember the approach- Do the approach- Check the approach- Act to improve the approach

NOTES:



Critical Success Factors of DeploymentKEY CONCEPT PRESENTATION GUIDEDeployment is much harder than defining an approach.


Approach is an intellectual exercise; deployment is a people-intensive process. There are five intangible attributes called critical success factors that help make deployment work.Five intangible attributes called critical success factors that help make deployment work:1. Deployment is a series of integrated tasks.2. Deployment champions are needed.3. Deployment is a team effort.4. Buy-in is required by the affected parties.5. Deployment / responsibilities must be effectively passed between individuals and between

teams

NOTES:




Internal Auditing and Quality AssuranceKEY CONCEPT PRESENTATION GUIDEBoth internal auditing and QA are professions.

SUPPORTING / EXPLANATORY MATERIALSIt is generally recognized that a profession has the following criteria:• Code of ethics• Common body of knowledge • Statement of responsibilities• Certification program (including continuing education)

NOTES:



Internal AuditingKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSInternal auditing is a management control directed at measuring and evaluating an activity to determine if it is performed in accordance with the policies and procedures of an organization (i.e., meets the intent of management). It is an independent appraisal activity. The specific types of auditing are:

Financial Auditing - Financial auditing is performed in accordance with generally accepted accounting procedures and other applicable laws and regulations to determine that the accounting records are reasonable.

Operational Auditing - Operational auditing is performed to determine that operations are performed in an efficient, effective and economical manner.

Program Auditing - Program auditing is performed to determine that the objectives of specific business activities are being properly fulfilled.

Auditing:• Measures and evaluates activities to determine if they are performed in accordance with

policies and procedures.• Auditing is an independent appraisal activity• An auditor’s work is detached from day-to-day operations• Auditors cannot get involved in developing procedures/standards

NOTES:



Quality AssuranceKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSQA should be a leadership position, emphasizing the strong interpersonal activities involved in making improvement occur. While QA performs many appraisals, it strives to be independent of the activities being appraised. Auditing, by nature, has a negative role; QA, by practice, should have a positive role.Quality Assurance:• A leadership position• Emphasis on strong interpersonal skills• Facilitate the development of solution• Development of policies, standards, and procedures• Measurement and analysis responsibility• Statistical process control overview

NOTES:




SUPPORTING / EXPLANATORY MATERIALSQuality principles dictate that the strategic quality plan should:a. Be written for the quality assurance functionb. Identify individual quality programsc. Be incorporated as a component of the strategic business plan. d. Be developed by the quality assurance functione. Define the size of the quality assurance function Internal auditing is a management control directed at measuring and evaluating an activity. Which statement is NOT considered part of a specific type of auditing activity?a. Financial Auditingb. Operational Auditingc. People Auditingd. Program Auditing

NOTES:




SUPPORTING / EXPLANATORY MATERIALSMany newly appointed quality managers are challenged to implement a quality function. Briefly explain three of the more important factors to be considered that the quality manager must integrate into the operation.• Deployment is a series of integrated tasks.• Deployment champions are needed.• Deployment is a team effort.• Buy-in is required by the affected parties.• Deployment / responsibilities must be effectively passed between individuals and between

teams

NOTES:



Skill Category 5KEY CONCEPT PRESENTATION GUIDEExecutive management establishes the vision and strategic goals.

SUPPORTING / EXPLANATORY MATERIALSPlanning is the process that describes how those strategic goals will be accomplished. Quality planning should be integrated into the IT plan so that they become a single plan. In simplistic terms, the IT plan represents the producer, and the quality plan represents the customer.Quality Planning• Planning Concepts• Integrating Business and Quality Planning• Prerequisites to Quality Planning• The Planning Process• Planning to Mature IT Work Processes

NOTES:


Planning ConceptsKEY CONCEPT PRESENTATION GUIDE


Define the objective, expressing it numerically, if possible.Clearly describe the goals and policies needed to attain the objective at this stage. Determine the procedures and conditions for the means and methods that will be used to achieve the objective. • If you do not know where you are going, all roads will lead you there.• If you fail to plan – plan to fail

NOTES:



The Management CycleKEY CONCEPT PRESENTATION GUIDEThe cycle comprises the four steps of: 1. Plan2. Do3. Check4. Act


NOTES:

Act Plan

Check Do

Act Plan

Check Do



The PDCA CycleKEY CONCEPT PRESENTATION GUIDEThe cycle comprises the four steps of: 1. Plan2. Do3. Check4. Act


Plan (P): Devise a plan• Objectives are quantitatively defined• Policies are defined• Practices/processes used to achieve the goals• Objectives are Identified

NOTES:




The PDCA Cycle (cont.)KEY CONCEPT PRESENTATION GUIDEExecute the plan

SUPPORTING / EXPLANATORY MATERIALSCreate the conditions and perform the necessary teaching and training to ensure everyone understands the objectives and the plan. Teach workers the procedures and skills they need to fulfill the plan and thoroughly understand the job. Then perform the work according to these procedures.Do (D): Execute the plan• Create the practices (define them)• Provide training• Perform the work according to the defined practice

NOTES:



The PDCA Cycle (cont.)KEY CONCEPT PRESENTATION GUIDECheck the results

SUPPORTING / EXPLANATORY MATERIALSAs often as possible, check to determine whether work is progressing according to the plan and whether the expected results are obtained. Check for performance of the procedures, changes in conditions, or abnormalities that may appear.Check (C): Check the Results• Determine if work is progressing to plan.• Will anticipated results be realized? • Are goals and objectives satisfied?

NOTES:


The PDCA Cycle (cont.)KEY CONCEPT PRESENTATION GUIDETake the necessary action

SUPPORTING / EXPLANATORY MATERIALSIf the check reveals that the work is not being performed according to plan or if results are not what were anticipated, devise measures for appropriate action. Look for the cause of the abnormality to prevent its recurrence. Sometimes workers may need to be retrained and procedures revised. The next plan should reflect these changes and define them in more detail. Act (A): Take the Necessary Action• If work is not progressing to plan, devise measures and appropriate actions.• Look for the cause of abnormalities.• Update the plan.• Update training materials, if needed.

NOTES:



The Planning CycleKEY CONCEPT PRESENTATION GUIDE


• Change the schedule• Change the budget• Change the number of resources allocated• Change how one component of software will affect other components• Change in work priorities• Addition or deletion of work activities

NOTES:



The Planning Cycle (cont)KEY CONCEPT PRESENTATION GUIDE


• Establish IT Vision - P• Define Mission - P• Set Goals - P• Strategic Planning - P• Tactical Planning – P• Execution – D• Monitoring – C• Rework - A

NOTES:



Prerequisites to Quality PlanningKEY CONCEPT PRESENTATION GUIDE


• IT vision, mission, and goals documented• Defined planning process• Management support for planning• Planners competent in the planning process• Compliance to the plan• Maintenance of the planning process• Reliable information required.

NOTES:



Common Activities in the Planning ProcessKEY CONCEPT PRESENTATION GUIDE


• Business or Activity Planning• Environment Planning• Capabilities and Opportunities Planning• Assumptions/Potential Planning• Objectives/Goals Planning• Strengths, Weaknesses and Tactics Planning• Priorities and Schedules Planning• Organization and Delegation Planning• Budgets and Resources Planning

NOTES:




Strategic Planning QuestionsKEY CONCEPT PRESENTATION GUIDEThe planning activities described were designed to answers six basic planning questions as listed below.

SUPPORTING / EXPLANATORY MATERIALSThe planning process then documents the answers to these six questions:1. Where are we?2. Where do we want to go?3. How are we going to get there?4. When will it be done?5. Who is responsible for what?6. How much will it cost?

NOTES:


Common Activities in the Planning ProcessKEY CONCEPT PRESENTATION GUIDE


• Policies/Procedures Planning• Strategy/Tactics Planning• Priorities/Schedules Planning• Organization/Delegation Planning• Budget/Resources Planning

• Planning Activities for Outsourced Work

NOTES:



Planning to Mature IT Work ProcessesKEY CONCEPT PRESENTATION GUIDE


NOTES:



Maturity Levels by Process CategoryKEY CONCEPT PRESENTATION GUIDE


NOTES:



Relationships that Can Affect Process MaturityKEY CONCEPT PRESENTATION GUIDE


• People skills and process definitions• Do and check procedures• Individuals' assessment of how they are evaluated to work performed• What management relies on for success• Maturity level to cost to do work• Process maturity to defect rates

NOTES:



Relationships that Can Affect Process MaturityKEY CONCEPT PRESENTATION GUIDE


• Process maturity and cycle time• Process maturity and end user satisfaction• Process maturity and staff job satisfaction• Process maturity to an organization's willingness to embrace change• Tools to process maturity• Control/test process category and quick pay backs

NOTES:



Strategy for Moving to Higher Maturity LevelsKEY CONCEPT PRESENTATION GUIDE


NOTES:



Skipping Levels and Reverting Back to Lower LevelsKEY CONCEPT PRESENTATION GUIDE


Skipping a level is not practical because each level builds the base for moving to a higher level.

Reversion back to lower levels will occur constantly when there is either significant business or technology change. This does not mean that the information organization drops to a lower level, but for specific technology or business change those changes will have to be begun at a lower level and move rapidly up to the organization's current level.

NOTES:



Skill Category 6KEY CONCEPT PRESENTATION GUIDEDefining and continuously improving work processes allows the pace of change to be maintained without negatively impacting the quality of products and services.


Define, Build, Implement, and Improve Work ProcessesProcess management is a term used by many IT organizations to represent the totality of activities involved in defining, building, deploying and maintaining the work processes used to achieve the IT mission. • Process Management Concepts• Process Management Processes

NOTES:




Definition of a ProcessKEY CONCEPT PRESENTATION GUIDEA process is a vehicle of communication, specifying the methods used to produce a product or service.

SUPPORTING / EXPLANATORY MATERIALSIt is the set of activities that represent the way work is to be performed. The level of communication (detail of the process) is normally commensurate with the skill level associated with the job. A process is a:• Communication vehicle or model• Method used to produce a product or service. • Set of work performance activities

NOTES:



Why Processes are NeededKEY CONCEPT PRESENTATION GUIDEProcesses add value to both management and the workers, although the reasons differ.

SUPPORTING / EXPLANATORY MATERIALSFrom a management perspective, processes are needed to:• Explain to workers how to perform work tasks• Transfer knowledge from more experienced to less experienced workers• Assure predictability of work activities so that approximately the same deliverables will be

produced with the same resources each time the process is followed• Establish a basic set of work tasks that can be continuously improved• Provide a means for involving workers in improving quality, productivity, and customer

satisfaction by having workers define and improve their own work processes• Frees management from their activities associated with “expediting work products” to spend

more time on activities such as planning, and customer and vendor interaction

NOTES:



Why Processes are Needed (cont.)KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSFrom a worker’s perspective, work processes are important to:• Increase the probability that the deliverables produced will be the desired deliverables• Put workers in charge of their own destiny because they know the standards by which their

work products will be evaluated• Enable workers to devote their creativity to improving the business instead of having to

develop work processes to build products• Enable workers to better plan their workday because of the predictability resulting from work

processes

NOTES:



Process Workbench and ComponentsKEY CONCEPT PRESENTATION GUIDEThe workbench is a graphic illustration of a process, documenting how a specific activity is to be performed.

SUPPORTING / EXPLANATORY MATERIALSWorkbenches are also called phases, steps, or tasks. A process can be viewed as one or more workbenches. Depending on the maturity of the organization, process workbenches may be defined by process management (or standards) committees, QA analysts, or work teams.

NOTES:

Standard(s)Standard(s)

PolicyPolicy

DO DO ProceduresProcedures

Deliverable(s)Input(s)

No

Yes

Rework

Tool(sTool(s))

CheckCheckProceduresProcedures


PolicyPolicy


Deliverable(s)Input(s)

No

Yes

Rework

Tool(sTool(s))


How

What



Process CategoriesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSApproaches for controlling businesses have evolved over many decades. Many of these business control models include these three general areas of control:1. Management processes2. Work processes3. Check processes

NOTES:


Process ContinuumKEY CONCEPT PRESENTATION GUIDE


NOTES:

Products and Services Continuum

Similar Customized Professional

Work Processes Continuum

ContinuousProcesses

ProfessionalProcesses

Job ShopProcesses

Check Processes Continuum

LiteralControls

IntentControls

Customer Involvement Continuum

Minimal HeavyIterative(Checkpoints)

Products and Services Continuum

Similar Customized Professional

Work Processes Continuum

ContinuousProcesses

ProfessionalProcesses

Job ShopProcesses

Check Processes Continuum

LiteralControls

IntentControls

Customer Involvement Continuum

Minimal HeavyIterative(Checkpoints)



How Processes are ManagedKEY CONCEPT PRESENTATION GUIDEThe infrastructure in a quality management environment should support process management.

SUPPORTING / EXPLANATORY MATERIALSIn a quality management environment: • The infrastructure must supports process management. • Process management is primarily a line responsibility. • All levels of the organization should be involved in both establishing and using work

processes in their daily work.

NOTES:




Planning ProcessesKEY CONCEPT PRESENTATION GUIDEProcess management is a PDCA cycle.

SUPPORTING / EXPLANATORY MATERIALSProcess management processes provide the framework from within which an organization can implement process management on a daily basis. The figure shows how this set of practices can be viewed as a continuous improvement cycle.

NOTES:

Process Improvement

CHECK

Process Mapping

Process Definition

ACT

Enables process improvement

Enables process assessmentProcess Measurement

Enablesprocess definition

PLAN

Process Planning

Process Inventory

DO

Enables process executionProcess Controls

Process Improvement

CHECK

Process Mapping

Process Definition

ACT

Enables process improvement

Enables process assessmentProcess Measurement

Enablesprocess definition

PLAN

Process Planning

Process Inventory

DO

Enables process executionProcess Controls


Planning ProcessesKEY CONCEPT PRESENTATION GUIDE


• Process planning allows priorities to be set for process management projects. • Priorities are set based on the relative importance of the process to accomplishing the

organization’s missions/goals, organizational constraints or readiness, and an assessment of the project’s status.

NOTES:



Do ProcessesKEY CONCEPT PRESENTATION GUIDE


There are two processes within the Do cycle of the process management process:• Process definition• Process control

NOTES:



Check ProcessesKEY CONCEPT PRESENTATION GUIDE


The Check cycle:• Includes process measurement • Determines what strategic and tactical measures and metrics are needed to manage by fact• Incorporates measurement into the appropriate processes.

NOTES:



Act ProcessesKEY CONCEPT PRESENTATION GUIDE


The Act cycle includes the process to reduce the frequency of defects, including process ineffectiveness.

NOTES:




SUPPORTING / EXPLANATORY MATERIALSGood quality practices require that work standards should be developed by:a. IT Managementb. Quality assurancec. Subject matter expertsd. Users of the standarde. Quality controlThree parts of the PDCA cycle are act, plan, and do. The fourth part of the PDCA cycle is:a. Contributeb. Calibratec. Checkd. Create

NOTES:





SUPPORTING / EXPLANATORY MATERIALSApproximately 10 years ago your IT department organized a Standards Committee. The committee determined that standards were needed, wrote those standards, incorporated them into a standards manual and delivered them to the appropriate staff members for implementation. After 10 years, it is generally agreed that the standards program is not working. The IT staff does not follow many of the standards, and management rarely enforces compliance. You, as the Quality Assurance Manager, have been asked to develop a new standards program. Indicate what you believe is the responsibility of the following groups in implementing an effective standards program.

GROUP Role/Responsibility of that Individual

IT Management

Standards Committee

Users of the Standards

Quality Assurance Manager

Quality Control Function

NOTES:



Skill Category 7KEY CONCEPT PRESENTATION GUIDEQuality control practices should occur during product development and throughout product change and operation.

SUPPORTING / EXPLANATORY MATERIALSDuring development, the quality control process is frequently called verification and at the conclusion of development, it is called validation. The quality practitioner should also be familiar with verification and validation techniques, the framework for developing testing tactics, change control and configuration management. • Testing Concepts• Developing Testing Methodologies• Verification and Validation Methods• Software Change Control• Defect Management• Process Management Processes

NOTES:



Quality Control BasicsKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSQuality Control Basics include:• Quality control measures a product against the existence of an attribute • Determines whether the product conforms to a standard or procedure (also known as

compliance checking).• Management is responsible for enforcing compliance to standards and procedures.

NOTES:


Process Workbench and ComponentsKEY CONCEPT PRESENTATION GUIDEThe workbench is a graphic illustration of a process, documenting how a specific activity is to be performed.

SUPPORTING / EXPLANATORY MATERIALSWorkbenches are also called phases, steps, or tasks. A process can be viewed as one or more workbenches. Depending on the maturity of the organization, process workbenches may be defined by process management (or standards) committees, QA analysts, or work teams.

NOTES:


PolicyPolicy


Deliverable(sDeliverable(s))Input(s)Input(s)

NoNo

YesYes

Rework

Tools(sTools(s))



PolicyPolicy


Deliverable(sDeliverable(s))Input(s)Input(s)

NoNo

YesYes

Rework

Tools(sTools(s))




Test StagesKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSTest Stages include:• Unit testing• Integration testing• System testing• User acceptance testing

NOTES:



Independent TestingKEY CONCEPT PRESENTATION GUIDE


Independent test team responsibilities: • System testing• Oversight of user acceptance testing• Provide an unbiased assessment of the quality of an application• Support or participate in other phases of testing including executing special test types such

as performance and load testing

NOTES:



Independent Testing (cont)KEY CONCEPT PRESENTATION GUIDE


Independent test manager responsibilities: • Planning and estimating tests• Designing the test strategy• Ensuring tests are created and executed in a timely and productive manner• Reviewing analysis and design artifacts• Chairing the test readiness review• Managing the test effort• Overseeing acceptance tests

NOTES:



Independent Testing (cont)KEY CONCEPT PRESENTATION GUIDE


Independent tester responsibilities: • Developing test cases and procedures• Planning, capturing, and conditioning test data• Reviewing analysis and design artifacts• Executing tests• Utilizing automated test tools for regression testing• Preparing test documentation• Tracking and reporting defects

NOTES:



Static vs. Dynamic TestingKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSStatic vs. Dynamic Testing:• Static Testing:

- Testing performed without executing code.• Dynamic Testing:

- Testing is being executed on the machine

NOTES:




The “V” Testing ConceptKEY CONCEPT PRESENTATION GUIDEValidation and Verification work together.


Denotes Continuous Testing During the Development Process

• At pre-determined points• Static and/or dynamic test techniques

NOTES:



The “V” Testing Concept (cont.)KEY CONCEPT PRESENTATION GUIDE


NOTES:

Operational orBusinessNeed

VerifyOperational

or Business Need

DefineRequirements

VerifyRequirements

DesignSystem

VerifyDesign

BuildSystem

VerifyConstruction

AcceptanceTest

ValidateOperational

or Business Need

SystemTest

ValidateRequirements

IntegrationTest

ValidateDesign

UnitTest

ValidateConstruction

Static Dynamic

Operational orBusinessNeed

VerifyOperational

or Business Need

DefineRequirements

VerifyRequirements

DesignSystem

VerifyDesign

BuildSystem

VerifyConstruction

AcceptanceTest

ValidateOperational

or Business Need

SystemTest

ValidateRequirements

IntegrationTest

ValidateDesign

UnitTest

ValidateConstruction

Static Dynamic



Test ObjectivesKEY CONCEPT PRESENTATION GUIDE


• A statement of what the test team or tester is expected to accomplish during a specific testing activity

• Are usually defined during requirements analysis, and guide the development of test cases, test scripts, and test data

• Enhance communication by defining the scope of the testing effort, and provide a way to gauge testing progress and success

NOTES:



Reviews and InspectionsKEY CONCEPT PRESENTATION GUIDE


Are Performed To:• Maximize team talent• Identify defects.• Educate a large number of people quicklyRules• Review the product not the producer• Identify defects and issue, don’t resolve them• Every member is responsible for success

NOTES:


Review FormatsKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSProject teams will use multiple review techniques for work products throughout the development lifecycle. The most effective verification and validation (V&V) programs utilize a combination of techniques to maximize the benefit received from reviews while minimizing impact to project timelines. When creating the Quality Plan (or Software Verification and Validation Plan) teams should determine the review technique for each work product based upon the complexity and risk associated with the deliverable, as well as the number of downstream work products that will utilize the artifact under review as input. For example, requirements feed almost every other work product in the software development process, and therefore should be reviewed using more formal techniques.Informal Techniques • Little/no preparation• No formal documentation • Little measurementSemi-Formal Techniques• Author presents material• Wide range of discussion• Metrics should be captured

NOTES:

Formal Techniques• Formal process with well-defined roles• Moderator leads review• Checklists and formal documentation• Metrics captured



Types of ReviewsKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSTypes of Reviews include:• In process Review• Checkpoint Review• Phase-end Review• Software requirements Review• Critical Design Review• Test Readiness Review• Post-implementation Review• Inspections

NOTES:




Testing MethodologyKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSA testing methodology forms the basis for the entire testing effort and should be developed for each project.A testing methodology should be developed for each development, maintenance, purchase, or upgrade project. The testing methodology defines the scope and general direction for testing. It is a high-level plan and should be developed as soon as the project is initiated. A test methodology must answer the following questions: • What type of project?• What is project’s scope?• What type of software?• When will testing occur?• Who will conduct testing?• What are the trade-offs?• What are the Critical Success Factors?

NOTES:


What Type of Project?KEY CONCEPT PRESENTATION GUIDEWe must use different testing approaches for different types of projects, just as we use different development approaches


• Traditional Development System (Waterfall)• Client/Server• Interactive Development/ Prototyping/CASE• Object-Oriented• System Maintenance / Legacy Systems• Purchased / Contracted Software

NOTES:



Management of Verification and ValidationKEY CONCEPT PRESENTATION GUIDE


Management of software development verification and validation activities: • Begins at the start of a project• Performed for all SDLC processes and activities.

NOTES:



Reviews and InspectionsKEY CONCEPT PRESENTATION GUIDE


Are Performed To:• Maximize team talent• Identify defects• Educate a large number of people quicklyReview Formats• Informal (peer review and desk checks) • Semi formal (JAD joint application development)• Formal (Inspection)

NOTES:



Verification TechniquesKEY CONCEPT PRESENTATION GUIDE


Types of Reviews• Feasibility reviews• Requirements reviews• Design reviews• Code walkthroughs• Code Inspections• Requirements tracing

NOTES:



Validation TechniquesKEY CONCEPT PRESENTATION GUIDE


White Box Testing• Structural testing based on knowledge of internal code structure and usually logic driven• Statement coverage• Decision Coverage• Condition Coverage• Decision/Condition Coverage

NOTES:



Validation Techniques (cont)KEY CONCEPT PRESENTATION GUIDE


Black Box Testing• Functional testing based on external specifications without knowledge of how the system is

constructed - usually data or business process driven• Validates that each input produces the appropriate output

- Equivalence partitioning - Boundary analysis - Exploratory Testing (Error Guessing)

NOTES:



Structural and Functional TestingKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSStructural and Functional Testing:Structural testing is considered white box testing because knowledge of the internal logic of the system is used to develop test cases.• Advantages:

- The logic of the software’s structure can be tested- Parts of the software will be tested which might have been forgotten if only functional

testing was performed• Disadvantages

- Its tests do not ensure that user requirements have been met- Its tests may not mimic real-world situations

NOTES:



Structural and Functional Testing (cont)KEY CONCEPT PRESENTATION GUIDE


Functional testing addresses the overall behavior of the program by testing transacting flows, input validation, and functional completeness but no knowledge of the internal logic system is used (black box).• Advantages

- Simulates actual system usage- Makes no system structure assumptions

• Disadvantages- Potential of missing logical errors in software- Possibility of redundant testing

NOTES:



Software Configuration ManagementKEY CONCEPT PRESENTATION GUIDEDynamic nature of business activities requires this to be in place


• Constant program changes• Well-formulated and well-documented procedures• Program manipulation prevention • No unauthorized use. • Primary objective of SCM:

“Get the right changes installed at the right time”

NOTES:



Change Control ProceduresKEY CONCEPT PRESENTATION GUIDE


• All proposed changes should be in writing • Major changes should be approved by the Configuration Control Board• Developers should make and document the program changes, not the operations group• Someone independent of the person who designed and made the change should be

responsible for testing the final revised program• The documentation system should be updated with all change sheets or change registers

and printouts

NOTES:



Defect Management ProcessKEY CONCEPT PRESENTATION GUIDE


The primary goal is to prevent defects• The process should be risk driven.• Defect measurement should be integrated into the development process.• The capture and analysis of the information should be automated.• Defect information should be used for process improvement.

NOTES:



Defect ReportingKEY CONCEPT PRESENTATION GUIDE


Defects are recorded for four major reasons:• Ensure the defect is corrected• Report status of the application• Gather statistics used to develop defect expectations in future applications• Improve the software development process.

NOTES:



Defect ManagementKEY CONCEPT PRESENTATION GUIDE


• Requires a communication mechanism, either manual or automated• Facilitates communication between test and development teams • What to report?

- Defect name and type- Severity and priority- Status- Date and time of detection- Location identified (e.g. component, GUI, etc.)- Detailed description- Component or program where defect was found- Screen prints, etc.- Stage of origination (may be added later)- Person assigned to correct- Correction effort in hours

NOTES:



Severity vs. PriorityKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSSeverity vs. Priority• Based on pre-defined severity descriptions, the test team should assign the severity of a

defect objectively. • In large projects, it may be necessary to assign a priority to the defect which determines the

order in which defects should be fixed.

NOTES:




A Sample Defect Tracking ProcessKEY CONCEPT PRESENTATION GUIDE


• Execute the tests • Log any discrepancies• Determine if the discrepancy is a defect• Assign the defect to a developer• Resolve the defect

NOTES:



Using Defects for Process ImprovementKEY CONCEPT PRESENTATION GUIDEThe primary goal is to prevent defects.


Based on the team’s findings, the process improvement activities should include the following:• Go back to the process that originated the defect to understand the root cause• Go back to the verification and validation processes which should have caught the defect

earlier

NOTES:



SUPPORTING / EXPLANATORY MATERIALSWhich of the following is NOT a verification technique:a. Feasibility Reviewb. Design Reviewc. Requirements Reviewd. Code Inspectione. Unit TestWhich of the following is NOT a validation techniquea. Executing all statements at least onceb. Executing each decision with all possible outcomes at least oncec. Executing test cases based on classes of datad. Executing each decision direction at least once

NOTES:



Sample Questions (cont)KEY CONCEPT PRESENTATION GUIDE


As a test manager for a large IT department, you’ve been asked what information should be recorded about each defect that is uncovered.

Identify and briefly describe what you believe are the four most important characteristics that should be recorded about a defect.

NOTES:




Skill Category 8KEY CONCEPT PRESENTATION GUIDETo effectively measure, one needs to know the basic concepts of measurement. This section provides those basic measurement concepts.

SUPPORTING / EXPLANATORY MATERIALSThere is no clearly defined, commonly accepted method of measuring software products and services. A large number of measures and metrics exist, but only a few have had widespread acceptance.

• Measurement Concepts• Measurement in Software• Variation and Process Capability• Risk Management• Implementing a Measurement Program

NOTES:



Standard Units of MeasureKEY CONCEPT PRESENTATION GUIDEA measure is a single quantitative attribute of an entity.

SUPPORTING / EXPLANATORY MATERIALSIt is the basic building block for a measurement program. Examples of measures are lines of code (LOC), work effort, or number of defects. Measurement cannot be used effectively until the standard units of measure have been defined. For example, talking about lines of code does not make sense until the measure LOC has been defined. • A measure is a single quantitative attribute of an entity. It is the basic building block for a

measurement program.• Since quantitative measures can be compared, measures should be expressed in numerical

terms

NOTES:



MetricsKEY CONCEPT PRESENTATION GUIDEA metric normalizes data so that comparison is possible.

SUPPORTING / EXPLANATORY MATERIALS• A metric is a derived (calculated or composite) unit of measurement that cannot be directly

observed, but is created by combining or relating two or more measures. Since metrics are combinations of measures, they can add more value in understanding or evaluating a process than plain measures. Examples of metrics are mean time to failure and actual effort compared to estimated effort.

• A metric normalizes data so that comparison is possible.

NOTES:



Objective and Subjective MeasurementKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSObjective measurement uses hard data that can be obtained by counting, stacking, weighing, timing, etc. Examples include:• Number of defects• Hours worked• Completed deliverables.An objective measurement should result in identical values for a given measure, when measured by two or more qualified observers.Subjective data is normally observed or perceived. It is a person's perception of a product or activity, and includes personal attitudes, feelings and opinions, such as how easy a system is to use, or the skill level needed to execute the system. Objective measurement uses hard data that can be obtained by counting, stacking, weighing, timing, etc.

NOTES:


Types of Measurement DataKEY CONCEPT PRESENTATION GUIDEIdeally, models should be developed that are capable of predicting process or product parameters, not just describing them.


Four uses of measurement• Manage and control processes• Manage and control products.• Improve processes.• Manage risks.Four types of measured data:• Nominal• Ordinal Data• Interval Data• Ratio Data

NOTES:



How to Know a Measure Is GoodKEY CONCEPT PRESENTATION GUIDEIdeally models should be developed that are capable of predicting process or product parameters, not just describing them.


Measures and resulting metrics should be:Measures and resulting metrics should be:• Reliable• Valid• Easy to Use• Simple• Timely• Easy to Calibrate• Objective• Easily obtainable

NOTES:




Quantitative Management in an IT FunctionKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSAn integral part of an IT function is quantitative management, which contains three aspects:• Measurement Dashboards• Function Points• Statistical Process Control (SPC)Measurement dashboards (also called key indicators) are used to monitor progress and initiate change. Statistical process control is used to ensure that the process behaves in a consistent manner. Line managers use the principles of statistical process control to assess consistency of products and services, and as a basis for continuous process improvement.

NOTES:



Product MeasurementKEY CONCEPT PRESENTATION GUIDEA product can be measured at any stage of its development.

SUPPORTING / EXPLANATORY MATERIALSFor a software product, the requirements, the complexity of the software design, the size of the final program’s source or object code, or the number of pages of documentation produced for the installed systems can be measured.Examples of product measurement:• Size

- Lines of Code- Function Points

• Complexity- Cyclomatic Complexity- Knots

• Quality- Software Quality Factors

NOTES:



Process MeasurementKEY CONCEPT PRESENTATION GUIDEA process can be measured by any of the following:• Attributes of the process, such as

overall development time• Type of methodology used• The average level of experience of the

development staff.

SUPPORTING / EXPLANATORY MATERIALSAccumulating product measures into a metric so that meaningful information about the process can be provided. For example, function points per person-month or LOC per person-month can measure productivity (which is product per resources), the number of failures per month can indicate the effectiveness of computer operations, and the number of help desk calls per LOC can indicate the effectiveness of a system design methodology.A process can be measured by any of the following:• Number of deliverables completed on time• Estimated cost vs. actual costs• Time spent fixing errors• Idle time• Number of modifications

NOTES:



Common Causes of VariationKEY CONCEPT PRESENTATION GUIDEStatistics that took into account common and special causes of variation.


Common causes are those that can be controlled by improving the work processes. All processes contain some inherent variation, or common causes of variation. The amount of variation in a process is quantified with summary statistics.In a computer operation, abnormal terminations cause variation. Typical common causes of abnormal terminations include:• Invalid Data• No Disk Space• Errors in Operating• Job Control Instructions, etc.

NOTES:



Special Causes of VariationKEY CONCEPT PRESENTATION GUIDEStatistics that took into account common and special causes of variation.

SUPPORTING / EXPLANATORY MATERIALSSpecial causes of variation are not present in a process. Special causes are those that must be controlled outside the process; typically they need to be dealt with individually. • Special causes of variation are not present in a process. • They occur because of special or unique circumstances. • In the IT example of abnormal terminations in a computer operation, special causes might

include:- Operator Strikes- Citywide Power Outages- Natural Disasters, such as earthquakes or hurricanes

NOTES:



Defining RiskKEY CONCEPT PRESENTATION GUIDERisk is the possibility that an unfavorable event will occur.

SUPPORTING / EXPLANATORY MATERIALSRisk management involves the activities of defining, measuring, prioritizing, and managing risk in order to eliminate or minimize any potential negative effect associated with risk.• Risk is the possibility that an unfavorable event will occur.• It may be predictable or unpredictable.• Risk has three components, each of which must be considered separately when deterring

how to manage risk:- What can happen?- How likely is it that it will?- What do we do if it does?

NOTES:



Characterizing RiskKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSChanges in a situation can result in new risks. Examples include:• Replacing a team member• Undergoing reorganization• Changing a project's scope.The probability of risk occurring at the beginning of the project is very high (due to the unknowns), whereas at the end of the project the probability is very low. Within a project, many tasks and deliverables are intertwined. The relationship of probability and impact are not linear, and the magnitude of the risk typically makes a difference. Risk has five distinguishing characteristics:• Situational• Time-based• Interdependent• Magnitude dependent• Value-based

NOTES:


Managing RiskKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSChanges in a situation can result in new risks. Examples include, replacing a team member, undergoing reorganization, or changing a project's scope.The probability of risk occurring at the beginning of the project is very high (due to the unknowns), whereas at the end of the project the probability is very low. Within a project, many tasks and deliverables are intertwined. The relationship of probability and impact are not linear, and the magnitude of the risk typically makes a difference. Processes of risk:• Risk Identification• Risk Quantification• Risk Analysis• Risk Prioritization• Risk Response Development• Risk Response Planning• Risk Response Control• Risk Resolution• Risk Monitoring

NOTES:



Software Risk ManagementKEY CONCEPT PRESENTATION GUIDE


Incorporating risk management info the software development life cycle includes planning at the following levels:• Long-Term or High-Level• Medium-Term or Medium Level• Short-Term or Low-Level

NOTES:



Risk of Integrating New TechnologyKEY CONCEPT PRESENTATION GUIDE


The QA Analyst has three roles in integrating new technology:• Determining the Risk• Assuring that the Controls are adequate to reduce the risk• Assuring that existing processes are appropriately modified to incorporate the Use of New

Technology

NOTES:



Prerequisites to Implementing a Measurement ProgramKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSCurrent IT management is often ineffective because the IT function is extremely complex and has few well-defined, reliable processes or product measures to guide and evaluate results. Due to:• Operations complexity• Few well-defined, reliable process or product measures for resultsAs a result:• Accurate and effective estimating, planning, and control are nearly impossible to achieve.Implementing a measurement program requires four prerequisite steps:1. Perceive the need for a measurement program.2. Identify a champion and change agent, and assign organizational responsibility.3. Establish tangible objectives and meaningful measurement program activities.4. Facilitate management buy-in at all levels for the measurement program.

NOTES:





SUPPORTING / EXPLANATORY MATERIALSThe most frequent measure used to quantify efficiency in the use of resources is:a. Qualityb. Customer satisfactionc. Return on Investmentd. Productivitye. PerformanceThese causes of variation are NOT present in a process:a. Commonb. Codingc. Speciald. Sequence

NOTES:




SUPPORTING / EXPLANATORY MATERIALSYour management has defined what they believe are important attributes of a software system. Five attributes are listed below. Develop a metric that will measure each of these five attributes.

Software Attribute Proposed Metric to Measure the Attribute

1. Usability2. Maintainability3. Reliability4. Security5. Service Level

NOTES:



Skill Category 9KEY CONCEPT PRESENTATION GUIDETwo key issues for quality assurance are internal control and security.


Interest in internal control has been highlighted by the passage of the Sarbanes-Oxley Act. Interest in internal control and security has been highlighted by publicized penetrations of security and the increased importance of information systems and the data contained by those systems.• Principles and Concepts of Internal Control• Environmental or General Controls• Transaction Processing Controls• The Quality Professionals Responsibilities• Risk and Internal Control Methods• Building Internal Controls• Building Adequate Security

NOTES:



COSO Defined KEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSFour key terms are understood:Risk – The probability that an undesirable event will occur.Exposure – The amount of loss that might occur if an undesirable event occurs.Threat – A specific event that might cause an undesirable event to occur.Control – Anything that will reduce the impact of risk.A process, effected by an organization’s Board of Directors, management, and other personnel, designed to provide reasonable assurance regarding the achievement of objectives in the following categories:• Effectiveness and efficiency of operations• Reliability of financial reporting• Compliance with applicable laws and regulations

NOTES:


Internal Control TermsKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSInternal Control• The interaction and/or interactivity within an organization that allows the entity to provide for

information accuracy, the safeguard of assets, reliability and efficiency of operations, and adherence to management policies, procedures and processes.

Process• An established methodology that has certain defined components which enable the activity

to be consistently repeated without variability.Testing• The activity performed to reduce the risk and probability of an unfavorable event occurring.

NOTES:



Internal Control Terms (cont)KEY CONCEPT PRESENTATION GUIDE


Risk• The probability that an undesirable event will occur.Exposure• The amount of loss that might occur if an undesirable event occurs.Threat• A specific event that might cause an undesirable event to occur.Control• Any activity, technique, method, or approach that minimizes or eliminates risk.

NOTES:



Levels of Organization ControlKEY CONCEPT PRESENTATION GUIDE

SUPPORTING / EXPLANATORY MATERIALSLevels of Organizational Control:

Control Responsible party to design and monitor

Environmental Controls Executive Management (Management Controls) (Board of Directors)

System Controls Middle Management (Process Controls) (Department Managers)

Transaction Processing Controls Work Staff(Products & Services Controls) (Operations)

NOTES:



Levels of Organization Control (cont)KEY CONCEPT PRESENTATION GUIDE


NOTES:

Responsible party to design and monitor

Executive Management(Board of Directors)

Middle Management (Department Mangers)

Work Staff(Operations)

Rev

enue

Expe

nditu

res

Payr

oll

Prod

uctio

n

Fina

ncia

l R

epo r

ting

Oth

er

Business Systems

Individual Applications Within Systems

Top Mgmt

Day to Day Supervision/ Management

IT Facility

Transaction Within Applications

Routine Transactions -- Nonroutine Transactions

Processing Methods for Transactions

Responsible party to design and monitor

Executive Management(Board of Directors)

Middle Management (Department Mangers)

Work Staff(Operations)

Rev

enue

Expe

nditu

res

Payr

oll

Prod

uctio

n

Fina

ncia

l R

epo r

ting

Oth

er

Business Systems

Individual Applications Within Systems

Top Mgmt

Day to Day Supervision/ Management

IT Facility

Transaction Within Applications

Routine Transactions -- Nonroutine Transactions

Processing Methods for Transactions



Internal Auditor’s ResponsibilitiesKEY CONCEPT PRESENTATION GUIDE


The International Standards for the Professional Practice of Internal Auditing, established by the Institute of Internal Auditors, specify that internal auditors should:• identify and evaluate significant exposures to risk and contribute to the improvement of risk

management and control systems• Monitor and evaluate the effectiveness of the organization’s risk management system• Evaluate risk exposures relating to the organization’s governance, operations, and

information systems • Assist the organization in maintaining effective controls by evaluating their effectiveness and

efficiency and by promoting continuous improvementIt should be recognized that the internal audit function does not have primary responsibility for establishing or maintaining the internal control system.

NOTES:





NOTES:

Environmental controls are the means by which management “manages” the organization.

• Organizational policies

• Organizational structure

• Method of hiring, training, supervising and evaluating

• Processes provided for day-to-day work activities

Environmental controls are the means by which management “manages” the organization.

• Organizational policies

• Organizational structure

• Method of hiring, training, supervising and evaluating

• Processes provided for day-to-day work activities





NOTES:

Transaction Processing Controls

• System that Processes Transactions

• System that Controls the Transaction Processing


• System that Processes Transactions

• System that Controls the Transaction Processing





NOTES:


• Preventive Controls

• Detective Controls

• Corrective Controls


• Preventive Controls

• Detective Controls

• Corrective Controls



Internal Control ModelsKEY CONCEPT PRESENTATION GUIDE


COSO Enterprise Risk Model (ERM)The COSO ERM framework defines risk and enterprise risk management, and provides a foundational definition, conceptualizations, objective categories, components, principles and other elements of a comprehensive risk management framework. ERM consists of eight interrelated categories:1. Internal Environment2. Objective Setting3. Event Identification4. Risk Assessment5. Risk Response6. Control Activities7. Information and Communications8. Monitoring

NOTES:



Building Internal ControlsKEY CONCEPT PRESENTATION GUIDE


The best practices for testing the controls in a software system involve two tasks:• Perform Risk Assessment

- Assess the completeness of the control requirements, which involves evaluating the software system risks.

• Assess Transaction Processing Controls- Follow the flow of transaction processing to assess whether or not controls are needed

so that the transactions are controlled throughout transaction processing.

NOTES:



Building Adequate SecurityKEY CONCEPT PRESENTATION GUIDE


Vulnerabilities in SecurityFunctional Vulnerabilities• Physical Access• IT Operations• Test Processes• Computer Programs• Operating Systems Access and Integrity• Impersonation• Media

NOTES:



Building Adequate Security (cont)KEY CONCEPT PRESENTATION GUIDE


IT Areas Where Security is Penetrated• Data and Report Preparation Facilities• Computer Operations• Non-IT Areas• Online Terminal Systems• Program Terminal Systems• Online Data Preparation and Report Generation• Digital Media Storage Facilities• Online Operations• Central Processors

NOTES:



Establishing a Security BaselineKEY CONCEPT PRESENTATION GUIDE


Creating Baselines• What to collect• From who will the information be collected?• The Precision of the information collected

- Establish a baseline team- Set baseline requirements and objective- Design baseline data collection methods- Train baseline participants- Collect baseline data- Analyze and report computer security status

NOTES:



Security Learning ContinuumKEY CONCEPT PRESENTATION GUIDE


NOTES:



Security Control PrinciplesKEY CONCEPT PRESENTATION GUIDE


Attributes of an Effective Security Control• Simplicity• Fail Safe• Complete Mediation• Open Design• Separation of Privilege• Psychological Acceptability• Layered Defense• Compromise Recording

NOTES:




SUPPORTING / EXPLANATORY MATERIALSMany people believe there is an overlap between internal auditors and quality assurance, however the main role of internal auditing is to:i. Identify and report problemsj. Define and implement solutions to a problemk. Build processes that prevent problems from occurringl. Check all products to make sure they are defect freem. Conduct acceptance testing before usageOne of the primary objectives of process control is to:a. Reduce variabilityb. Increase competitionc. Improve metricsd. Find root causes

NOTES:






Define and give two examples of each of the three categories of control.1. Preventive controls2. Detective controls3. Corrective controls

NOTES:



Skill Category 10KEY CONCEPT PRESENTATION GUIDEQuality of software remains an internal IT responsibility regardless of who builds the software.

SUPPORTING / EXPLANATORY MATERIALSOrganizations can assign software development work responsibilities to outside organizations through purchasing software or contracting services; but they cannot assign the responsibility for quality.The quality professionals need to assure that those quality responsibilities are fulfilled through appropriate processes for acquiring purchased software and contracting for software services.• Quality and Outside Software• Selecting COTS Software• Selecting Software Developed by Outside Organizations• Contracting for Software Developed by Outside Organizations• Operating for Software Developed by Outside Organizations

NOTES:


Challenges in Selecting Acquired SoftwareKEY CONCEPT PRESENTATION GUIDE


• Task or items missing• Software fails to perform• Extra features• Does not meet business needs• Does not meet operational needs• Does not meet people needs

NOTES:





• Quality factors may not be specified• Non-testable requirements and criteria• Customer’s standards may not be met• Missing requirements• Overlooked changes in standards or technology• Training and deployment may be difficult

NOTES:





• Cultural differences• Communication barriers• Loss of employee morale and support• Root cause of the contractor IT organization not addressed

NOTES:



Quality Professional’s ResponsibilityKEY CONCEPT PRESENTATION GUIDE


• Responsibility for the quality COTS or contracted software cannot be contracted• There must be a process to monitor the development and validate the correct functioning of

contracted software• The quality professional needs to look at how the contractor tests in relation to the SEI

CMMI® Capability Maturity Model.

NOTES:



Commercial-Off-The-Shelf (COTS) Selection ChallengesKEY CONCEPT PRESENTATION GUIDE


NOTES:



COTS Selection Process (cont)KEY CONCEPT PRESENTATION GUIDE


• Assure completeness of needs specification• Define critical success factor• Determine compatibility with your computer environment• Assure the software can be integrated into your business system work flow• Demonstrate the software in operation• Evaluate the people fit• Acceptance test the COTS software

NOTES:





Assure completeness of needs specification• Output products and reports• Management decision information

NOTES:





Define critical success factor• Ease of Use• Expendability• Maintainability• Cost-effectiveness• Transferability• Reliability• Security

NOTES:





Determine compatibility with your computer environment• Hardware Compatibility• Operating System Compatibility• Software Compatibility

NOTES:





Assure the software can be integrated into your business system work flow• Current system based on certain assumptions• Existing forms, existing data, existing procedures• COTS based on certain assumptions• COTS uses a predetermined set of forms & proc.• Current system & COTS may be incompatible• COTS is not going to change

NOTES:





Demonstrate the software in operation• Observe during demonstration:• Understandability• Clarity of communications• Ease of use of manuals• Functionality of software• Knowledge to execute• Effectiveness of help routines• Evaluate program compatibility• Data compatibility• Smell Test

NOTES:





Evaluate the people fit• The software can be used as is• Additional training and support is necessary• The software is not usable with the skill sets of the proposed users

NOTES:





Acceptance test the COTS software• Same basic procedure as acceptance testing in-house developed software

NOTES:



Contracted Software Selection Process KEY CONCEPT PRESENTATION GUIDE


• Assure process for contracting software is adequate• Assure requirements & contract criteria are testable• Review adequacy of the contractor’s development process• Review adequacy of the contractor’s test process• Define acceptance testing criteria• Assure the contractor’s report schedule is adequate• Ensure knowledge transfer & intellectual rights protected• Assure ongoing operation and maintenance • Assure the effectiveness of contractual relations

NOTES:



Contracted Software Selection Process (cont)KEY CONCEPT PRESENTATION GUIDE


Assure process for contracting software is adequateContracts should include:• What is done.• Who does it• When it is done• How it is done• Where it is done• Penalties for nonperformance

NOTES:




SUPPORTING / EXPLANATORY MATERIALSAssure process for contracting software is adequateContracts concerns:

NOTES:

• Warranty• Deliverables• Delivery date• Commencement date• Installation• Updates• Contractor support• Costs• Foreign attachments• Penalties• Life of contract• Modification capability• Service discontinuance

• Manual/training discontinuance• Acceptance test criteria• Purchase versus lease• Fairness of contract• Performance of maintenance• Contractor training• Contractor manuals• Supplies• Transportability• Termination• Contractor employee rights• Governing Law• Contractor inspection• Access to developers & documentation





Assure requirements & contract criteria are testable• Common term: performance-based contracting

NOTES:





Review adequacy of the contractor’s development process• The contractor should provide certification that their development process meets an

internationally excepted model (e.g., CMMI, ISO, etc.)

NOTES:





Review adequacy of the contractor’s test process• Evaluate the adequacy and completeness of testing that will be performed.• May perform a comparison against internationally accepted test plan standards (e.g., IEEE)

NOTES:





Define acceptance testing criteriaMinimum the acceptance testing should validate:• The documentation is consistent with execution• Documentation is understandable• User will be adequately trained• It is operable within the operational constraints

NOTES:





Assure the contractor’s report schedule is adequate• You should verify that the contractor will issue reports per your specifications regularly

NOTES:





Ensure knowledge transfer & intellectual rights protected• Training programs for staff• Being advised of defects uncovered by other organizations using software• Ability to contact contractor’s help desk• Use care to protect contractor property rights

NOTES:





Assure ongoing operation and maintenance Major concerns:• Adequacy of control• Adequacy of documentation• Speed of service• Nearness of service• Competency of service• Adequacy of hardware• Skilled personnel• Multi-contractor problem resolution• Cost of services• Cost of operations• Error diagnosis• Error documentation

NOTES:





Assure the effectiveness of contractual relations• Contractor obligations met• Contracting organization obligations met• Needs met• Limits on cost increases• Exercising options (e.g., added features)• Renegotiation• Compensation for error• Returns on termination

NOTES:





Assume that your organization was asked to build an important software project, but your IT organization was currently overcommitted, so it was decided to outsource the development of the software to a contractor. Your contracting officer has asked you as a test manager to write the section of the contract that specifies how the contractor is to test the software. Rather than write that section, list what you think are the five (5) most important contractor test responsibilities that you think should be included in the contract and then describe those responsibilities in enough detail that if the contractor met those testing responsibilities you would be comfortable with the contractor’s ability to effectively test the software.

NOTES:





SUPPORTING / EXPLANATORY MATERIALSSoftware can be developed by an organization within the country of use, or developed in another country. Which of the following is a difference associated with software developed by an organization in another country?a. Missing requirementsb. Customer standards may not be metc. Non-testable requirementsd. Training and deployment may be difficulte. Cultural differencesAt a minimum, the acceptance testing for contracted software should validate:a. The documentation is consistent with the software executionb. The documentation is understandablec. Users will adequately trained in the softwared. It is operable within the operation constraints of the organizatione. All of the above

NOTES:



Test Taking TipsKEY CONCEPT PRESENTATION GUIDE


NOTES:


On the Day of the ExamKEY CONCEPT PRESENTATION GUIDE


• Arrive at the examination location at least 30 minutes before the scheduled start time of the examination

• You must have the following on the day of the exam:• Confirmation Letter from Software Certifications• Photo identification • Two # 2 pencils• Don’t bring personal items into the room• Once exam starts no one will be admitted

NOTES:



About the ExamKEY CONCEPT PRESENTATION GUIDE


Four parts to the examPart Questions Type 1 50 Objective or multiple choice questions – each question has equal point

value 2 10 Essay or Scenario based - each question has equal point value 3 50 Objective or multiple choice questions – each question has equal point

value 4 10 Essay or Scenario based - each question has equal point value

Minimum passing = 75% averaged over all 4 parts of the examination to become certified

NOTES:



After the ExamKEY CONCEPT PRESENTATION GUIDE


• Allow up to 12 weeks for results• Completed results are mailed the same day• Do not discuss the contents of the exam – The code of ethics• Do not call QAI or Software Certifications within those 12 weeks• Results will not be given over the phone.

NOTES:



First ThingKEY CONCEPT PRESENTATION GUIDE


Before you begin – • Remember you have plenty of time• Take a deep breath• Don’t be nervous

NOTES:



Tip #1KEY CONCEPT PRESENTATION GUIDE


• Remember one question may only be worth one point, if you are not able to answer questions later because you have lost time you could lose more points!

• Becoming nervous can make you lose your concentration and that leads to worse results. • Do not insist on completing each question before going to the next.

This is extremely important. If you spend a lot of time on one question that you don't understand there can be two negative outcomes: - Losing time - Becoming nervous

NOTES:



The How-to’sKEY CONCEPT PRESENTATION GUIDE


• For multiple choice parts, read each question’s stem twice, then read ALL the responses• For essay parts, read each of the essay questions thoroughly • As you read through each question (multiple choice and essay) determine whether:

- You absolutely know the answer to this question.- You believe you know the answer to this question. - You are not sure you know the answer, or it would take time to develop an answer.

• For both multiple choice and essay questions, first answer the questions to which you know the answers

NOTES:





The primary purpose of quality control is to:a. Prevent defects from occurringb. Assign quality responsibilitiesc. Uncover defectsd. Conduct testing

The most effective test approach is to begin testing when:a. A new project beginsb. After requirementsc. After external designd. After coding

NOTES:



More How-to’sKEY CONCEPT PRESENTATION GUIDE


If possible, go through the entire test, answering the questions you are sure you know. This results in your being more relaxed and feeling more confident.

NOTES:



Second RoundKEY CONCEPT PRESENTATION GUIDE


Go through the test a second time, working out the answers to more difficult questions. Now you will feel more confident and this will improve your test taking.

Key Tip: Do not waste too much time on any one question.

NOTES:



Check BackKEY CONCEPT PRESENTATION GUIDE


Once you have gone through the test twice, see if any of the questions asked can help you answer those really difficult questions.This is a little used trick. Sometimes questions asked are answered in later questions asking for different things

NOTES:



Don’t Change AnythingKEY CONCEPT PRESENTATION GUIDE


Going back to think about it usually makes you unsure and often causes an error. This is very common so be very careful!

If you have a strong feeling about a question when you first answer it, don't go back and change it later. Usually (but not always) a strong first impulse means we know the answer and we don't really have to think about it too much.

NOTES:



Change the ChancesKEY CONCEPT PRESENTATION GUIDE


Play the odds • If you don't know the answer, write something. • If you are answering a 4 possibility multiple choice question, you will still have a 25% chance

of being correct!

NOTES:



Fluff and StuffKEY CONCEPT PRESENTATION GUIDE


As a test manager for a large IT department, you’ve been asked what information should be recorded about each defect that is uncovered. Identify and explain what you believe are the four most important characteristics that should be recorded about a defect.

NOTES:



The DistractersKEY CONCEPT PRESENTATION GUIDE


• Throw out the ridiculous answers in a multiple choice question • Many tests often include one or two completely ridiculous possibilities in any multiple choice

question. • Find these first and get rid if them. This will usually leave you with a choice between two.

Your odds will have already increased to 50%!

NOTES:



Test Taking Tip SummaryKEY CONCEPT PRESENTATION GUIDE


1. Answer the questions that you believe you know the answer 2. Answer the questions that you think you know the answer3. Answer the questions that you don’t know the answer

NOTES:



The EndKEY CONCEPT PRESENTATION GUIDE


NOTES:






Appendix A - CSQA Skill Self Assessment

he 10 CSQA CBOK Skill Categor ies are common to all quality-related assignments andtherefore, the certification examination focuses equally on all of them. Pleasse note that SkillCategories 9 and 10 are new to the 2006 CSQA Common Body of Knowledge

The 2006 Common Body of Knowledge for the software quality analyst certificate includes these tenskill categories:

Skill Category 1 – Quality Principles and Concepts

Skill Category 2 – Quality Leadership

Skill Category 3 – Quality Baselines

Skill Category 4 – Quality Assurance

Skill Category 5 – Quality Planning

Skill Category 6 – Define, Build, Implement and Improve Work Processes

Skill Category 7 – Quality Control Practices

Skill Category 8 – Metrics and Measurement

Skill Category 9 – Internal Control and Security

Skill Category 10 – Outsourcing, COTS, and Contracting Quality

The 10 CSQA CBOK Skill Categories are common to all quality-relat ed assignments and therefore,the certification examination focuses equally on all of them.

T



Complete the CSQA Skill Assessment Worksheet

To assess your competency of the CSQA CB OK, complete the “CSQA 2006 Skill AssessmentWorksheets” starting on page 308. Follow these guide lines on how to use the worksheet to rate yourcompetency and identify those areas that you need to better understand to successfully pass the CSQAexamination:

1. Assess your competency of ea ch skill listed on the w orksheet. Carefully read each skil lwithin the skill category. Based on your reading of the skill, assess your competency in oneof the following three categories and place a check mark (“ ”) in the appropriate columnon the CSQA 2006 CBOK Competency Rating Table:

Not Competent – “None”

Either you do not understand this skill, or if you do understand it you do not know “what”is required to perform this skill. For exampl e, you may know that an IT quality plan isneeded, but you do not know what is included in an IT quality plan.

Some Competency – “Some”

This assessment means that you know “what” is needed to accomplish a specific skill. Forexample, you may know what is to be included within an IT quality plan, but you havenever actually prepared an IT quality plan. In othe r words, you have book knowledge, butnot how-to knowledge.

Fully Competent – “Full”

This assessment means that you not only know what is required to perform a specific skill,but you have actually used that skill in pe rforming day-to-day work tasks. For example,you have written an IT quality plan.

Note that Skill Category 1 focuses on the voc abulary of IT quality assurance and the basicconcepts on which the quality assurance profession is built. In assessing this category for aquality term such as reliability a “not competent” response means you could not define theterm; a “some competency” response means you could defi ne the term; and a “fullycompetent” response means that you use the term in the pe rformance of your day-to-daywork.



Calculate Your CSQA CBOK Competency Rating

Follow these steps to calculate your competency rating for the CSQA 2006 CBOK. This rating willhelp you determine if you are rea dy to submit your application fo r the CSQA examination or if youneed to study any Skill Categories further in order to pass the examin ation. Use the CBOK SkillCategory Competency Rating Table on page 318 to perform each step below.

1. Total the number of skills you have checked in each of the three co lumns for each skillcategory. Write your number s in the sp ace provided for each skill category on theworksheet. These are your competency rating totals for that skill category.

2. Transfer the three comp etency rating totals for each skill categor y to the correspondingcolumn (“Full,” “Some,” and “None”) in the CSQA Skill Category Competency Ratingstable provided.

3. Tally each column in the table to determine each Ratings Total.

4. Multiply each column by the indicated number to determine the Column Total.

5. Add the Column Totals together to determine the Sum of the Rows Total.

6. Divide the Sum of the Rows Total by 155 (the number of total skills in the CSQA 2006CBOK) to determine your CSQA CBOK Competency Rati ng. This number will bebetween 1 and 3.

Now you are able to determ ine if you are ready to submit your application and take the certificationexamination or if you need further study. Use your CSQA 2006 CBOK Competency Rating from step6 above and the following key to interpret your competency rating:

The closer your score is to “3,” the more competent you are in software quality assurance.

• If your score is a “3,” you are a world-class software quality analyst and ready to sub-mit your application. See the “Introduction” for information on submitting your appli-cation for the CSQA 2006 certification examination.

• If your score is between “2” and “3”, you are a competent quality analyst and ready to submit your application. See the “Introduction” for information on submitting your application for the CSQA 2006 certification examination.

• If your score is between “1” and “2”, you do not have the basic skills necessary to per-form software quality assurance. Study those skills that you rated “None” and then reassess your skills.

• If your score is a “1”, you are not competent in the CBOK. Study those skills that you rated “None” and then reassess your skills.



Skill Category 1 – Quality Principles and Concepts

Before an organization can begin to assess the quality of its pr oducts and services, and identifyopportunities for improvement, it first must have a working knowledge of quality principles and basicconcepts. This category tests th e CSQA candidate’s abil ity to understand and ap ply these principles,which include the quality vocabulary, various ways of defining qualit y, key concepts, distinguishingbetween quality control and quality assurance, and the contributions of quality pioneers.

Competency Rating Totals (total each “ ” in each column): ______ ______ ______

Skill Category 1 – Quality Principles and Concepts Competency RatingSkill # Skill Description Full Some None

1.1Vocabulary of Quality

Understand the vocabulary of quality

1.2The Different Views of Quality

The two quality gaps1.3 Quality attributes for an information system

1.4Quality Concepts and Practices

PDCA Cycle1.5 Cost of quality1.6 Six sigma quality1.7 Baselining and benchmarking1.8 Earned value

1.9Quality Control and Quality Assurance

Understand quality control and quality assurance1.10 Understanding and using the Just-in-Time (JIT) Technique1.11 Differentiating between Quality Control and Quality Assurance

1.12Quality Pioneers Approach to Quality

Includes Dr. W. Edwards Deming, Philip Crosby, and Dr. Joseph Juran

1.13 Total Quality Management



Skill Category 2 – Quality Leadership

The most important prerequisites for suc cessful implementation of any major quality initiative areleadership and commit ment from executive management . Management must create a workenvironment supportive of qua lity initiatives. It is management’s responsibility to establish strategicobjectives and build an infrastructure that is strategically aligned to those obj ectives. This categorycovers the management processes used to establish the foundation of a quality-managed environment,as well as commitment, new behaviors, building the infrastru cture, techniques, approaches andcommunications.


Skill Category 2 – Quality Leadership Competency RatingSkill # Skill Description Full Some None

2.14Leadership Concepts

Executive and middle management commitment2.15 Quality Champion2.16 New Behaviors for Management - traditional management versus

quality management, leadership, the importance of establishing mentoring relationships, and establishing trust

2.17 Empowerment of employees

2.18Quality Management Infrastructure

Quality council2.19 Management committees2.20 Teams and work groups2.21 Process improvement teams

2.22Quality Environment

The six attributes of an effective quality environment2.23 Setting the proper “tone” at the top2.24 Code of ethics and conduct2.25 Open communication2.26 Implementing a mission, a vision, goals, values and a quality

policy2.27 Monitoring compliance to organizational policies and procedures2.28 Enforcement of organizational policies and procedures



Skill Category 3 – Quality Baselines

Organizations need to establis h baselines of performance for quality, productivity and customersatisfaction. These baselines are used to documen t current performance and document improvementsby showing changes from a baseli ne. In order to establish a base line, a model and/ or goal must beestablished for use in measuring against to determine the baseline.

Competency Rating Totals (total each “ ” in each column): ______ ______ _____

Skill Category 3 – Quality Baselines Competency RatingSkill # Skill Description Full Some None

3.29Quality Baseline Concepts

Baselines defined3.30 Types of baselines3.31 Conducting baseline studies

3.32Methods Used for Establishing Baselines

Customer surveys3.33 Benchmarking to establish a baseline goal3.34 Assessments against management established criteria 3.35 Assessments against industry models

3.36Model and Assessment Fundamentals

Purpose of a model3.37 Types of models (staged and continuous)3.38 Model selection process3.39 Using models for assessment and baselines

3.40Industry Quality Models

Software Engineering Institute Capability Maturity Model/CMMI3.41 Malcolm Baldrige National Quality Award3.42 ISO 9001:20003.43 ISO/IEC 122073.44 ISO/IEC TR 155043.45 Post-implementation audits



Skill Category 4 – Quality Assurance

Quality Assurance is a professional competency whose focus is directed at the critical processes used tobuild products and services. The profession is charged with the respon sibility for tact ical processimprovement initiatives that are strategically alig ned to the goals of the organization. This categoryaddresses the understanding and application of quality assurance practices in support of the strategicquality direction of the organizati on. The quality practitioner should understand the importance of aquality function, how to implement a quality function and how it matures over time, as well as how tocreate a quality plan, the use of quality tools, process deployment, and differentiating between internalauditing and quality assurance.


Skill Category 4 – Quality Assurance Competency RatingSkill # Skill Description Full Some None

4.46Establishing a Function to Promote and Manage Quality

The challenges of implementing a quality function4.47 How the quality function matures over time4.48 Support in corporate quality management environment4.49 Implementing an IT quality function

4.50Quality Tools

Management tools 4.51 Statistical tools4.52 Presentation tools

4.53Process Deployment

Getting buy-in for change through marketing4.54 The formula for effective behavior change4.55 The deployment process4.56 Critical success factors for deployment

4.57Internal Auditing and Quality Assurance

Types of internal audits4.58 Differences in responsibilities



Skill Category 5 – Quality Planning

Executive management establishes th e vision and strategic goals of an organization. Planning is theprocess that describes ho w those strategic go als will be acco mplished. Quality planning should beintegrated into the IT plan so that they become a single plan. In simplistic terms, the IT plan representsthe producer and the quality plan represents the customer.


Skill Category 5 – Quality Planning Competency RatingSkill # Skill Description Full Some None

5.59Planning Concepts

The management cycle5.60 The planning cycle

5.61Integrating Business and Quality Planning

The fallacy of having two separate planning processes5.62 Planning should be a single IT activity5.63 Prerequisites to Quality Planning

5.64The Planning Process

Planning process overview5.65 The six basic planning questions5.66 The common activities in the planning process

5.67Planning to Mature IT Work Processes

QAI model and approach to mature IT work processes5.68 How to plan the sequence for implementing process maturity




The world is constantly changi ng. Customers are more knowledgeab le and demandin g, therefore,quality and speed of delivery are now critical need s. Companies must constantly improve their abilityto produce quality products th at add value to thei r customer base. Defi ning and continuouslyimproving work processes allows th e pace of change to be maintained without negatively impactingthe quality of products and services. This category addresses process management concepts, includingthe definition of a process, th e workbench conc ept and components of a process. Additionally, itaddresses the understand ing of definitions and continuous im provement of a pr ocess through theprocess management PDCA cycle.



Competency Rating

Skill # Skill Description Full Some None

6.69Process Management Concepts

Definition of a process6.70 Why processes are needed6.71 Process workbench and components6.72 Process categories6.73 The process maturity continuum6.74 How processes are managed6.75 Process template

6.76

Process Management ProcessesPlanning processes:Process inventory

6.77 Process mapping6.78 Process planning

6.79Do processes:Process definition

6.80Check processes:Identify control points

6.81 Process measurement6.82 Testing

6.83Act processes:Process improvement teams

6.84 Process improvement process



Skill Category 7 – Quality Control Practices

Quality control practices should occur during pr oduct development, pro duct acquisition, productconstruction at the end of de velopment/acquisition and throu ghout product change and operation.During development, the quality control process is frequently called verification and at the conclusionof development, it is called validation. This category addresses the various types of controls and whenthey are best used in the process. The quality practitioner should also be familiar with verification andvalidation techniques, the framework for developing testing tactics, change control and configurationmanagement.


Skill Category 7 – Quality Control Practices Competency RatingSkill # Skill Description Full Some None

7.85Testing Concepts

The tester’s workbench7.86 Test stages7.87 Independent testing7.88 Static versus dynamic testing7.89 Verification versus validation7.90 Stress versus volume versus performance7.91 Test objectives7.92 Reviews and inspections

7.93Developing Testing Methodologies

Acquire and study the test strategy7.94 Determine the type of development project7.95 Determine the type of software system7.96 Determine the project scope7.97 Identify the tactical risks7.98 Determine when testing should occur7.99 Build the system test plan7.100 Build the unit test plans

7.101Verification and Validation Methods

Management of verification and validation7.102 Verification techniques7.103 Validation techniques7.104 Structural and functional testing

7.105Software Change Control

Software configuration management7.106 Change control procedures

7.107Defect Management

Defect management process7.108 Defect reporting 7.109 Severity versus priority7.110 Using defects for process improvement



Skill Category 8 – Metrics and Measurement

A properly established measurement system is used to help achieve mi ssions, visions, goals, andobjectives. Measurement da ta is most reliable wh en it is generated as a by-product of producing aproduct or service. The QA analys t must ensure that quantitative da ta is valued and reliable, andpresented to management in a tim ely and easy-to-use manner. Measurement can be used to gauge thestatus, effectiveness and efficiency of processe s, customer satisfaction, product quality, and as a toolfor management to use in their decision-making processes. This categor y addresses measurementconcepts, the use of measuremen t in a software development e nvironment, variation, processcapability, risk management, the ways measurement can be use d, and how to implement an effectivemeasurement program.


Skill Category 8 – Metrics and Measurement Competency RatingSkill # Skill Description Full Some None

8.111Measurement Concepts

Standard units of measure 8.112 Metrics8.113 Objective and subjective measurement8.114 Types of measurement data8.115 Measures of central tendency8.116 Attributes of good measurement8.117 Using quantitative data to manage an IT function8.118 Key indicators

8.119Measurement in Software

Product measurement8.120 Process measurement

8.121Variation and Process Capability

The measurement program8.122 Installing the measurement program8.123 Common and special causes of variation8.124 Variation and process improvement8.125 Process capability

8.126Risk Management

Defining risk8.127 Characterizing risk8.128 Managing risk8.129 Software risk management8.130 Risks of integrating new technology

8.131Implementing a Measurement Program

The need for measurement8.132 Prerequisites



Skill Category 9 – Internal Control and Security

Privacy laws and increase d accessibility to data have necessi tated increased se curity. Accountingscandals and governmental regu lation such as the Sarbanes-Oxley Act have pl aced increasedimportance on building and mainta ining adequate systems of intern al control. The quality assurancefunction can contribu te to meeting those objec tives by assuring that IT h as adequate processesgoverning internal control and security.


Skill Category 9 – Internal Control and Security Competency RatingSkill # Skill Description Full Some None

9.133Principles and Concepts of Internal Control

Internal control and security vocabulary and concepts9.134 Preventive, detective, and corrective controls

9.135Risk and Internal Control Models

COSO enterprise risk management (ERM) model9.136 COSO internal control framework model9.137 CobiT model

9.138Building Internal Controls

Perform risk assessment

9.139Building Adequate Security

Where vulnerabilities in security occur9.140 Establishing a security baseline9.141 Security awareness training9.142 Security practices



Skill Category 10 – Outsourcing, COTS, and Contracting Quality

Organizations can a ssign software developmen t work responsibilities to outside organizations bypurchasing software or contracting services; but they cannot assign the responsibility for quality.Quality of software remains an internal IT responsibility regardless of w ho builds the software. Thequality professionals need to assure that those quality responsibil ities are fulfilled through appropriateprocesses for acquiring purchased software and contracting for software services.

Competency Rating Totals (total the “ ” in each column): ______ ______ ______

Skill Category 10 – Outsourcing, COTS, and Contracting Quality Competency RatingSkill # Skill Description Full Some None

10.143Quality and Outside Software

Purchased COTS software 10.144 Outsourced software

10.145Selecting COTS Software

Assure completeness of needs requirements10.146 Define critical success factor10.147 Determine compatibility with hardware, operating system, and other

COTS software10.148 Assure the Software can be Integrated into Your Business System

Work Flow 10.149 Demonstrate the Software in Operation10.150 Evaluate People Fit10.151 Acceptance Test the Software Process

10.152Selecting Software Developed by Outside Organizations

Contracting life cycle10.153 Developing selection criteria

10.154Contracting for Software Developed by Outside Organizations

Contract negotiations

10.155Operating for Software Developed by Outside Organizations

Acceptance testing



CSQA 2006 CBOK Competency Rating Table

CBOK Skill Category Competency Ratings

Full Some None

Skill Category 1

Skill Category 2

Skill Category 3

Skill Category 4

Skill Category 5

Skill Category 6

Skill Category 7

Skill Category 8

Skill Category 9

Skill Category 10

Ratings Total

Factor for Multiplication x 3 x 2 x 1

Columns Total

Sum of the Rows Total

Number of CSQA Skills ÷ 155

Your CSQA 2006 CBOK Competency Rating



Instructions for Your Personal Plan of Action

Your personal plan of action should be to improve those skills included in the CSTE Common Body of Knowledge for which you were not fully proficient. You should make two lists of skills by Skill Category. The first list would contain those skills which you rated yourself as "not competent." The second list should be those skills which you rated yourself "some competency." Merge the lists by Skill Category then competency level. The end product would be for each Skill Category you will have a list of skills which you are not com-petent followed by a list of skills for which you have some competency. As you study for the les-sons devoted to those skill categories, continually refer to your list of skills for which you are not competent or have some competency so that you can focus your study effort on the skills in which you need improvement.

My Personal Plan of Action to Improve These Skills

CSQA CBOK Category

Skill # and Description Needing Improvement

Action PlanCompleted Date



Appendix B - Topics for Quality AnalystsThe answers to these essay questions can be found starting on page 321. 1. What is COQ (Cost of Quality) and how it relates to software quality and testing?2. What is the PDCA (Plan, Do, Check, Act) and how it affects test activities.3. What is a risk matrix and be familiar with CSF's (Critical Success Factors)4. What is a test strategy and who participates in the development of it?5. Define the differences between tactical and strategic testing activities.6. Define root cause analysis and also explain the difference between common and special

causes of variation.7. Define a test policy and why it is important.8. Define "structured" approach to testing and explain the V-model concept.9. Define "how much testing is enough".10. Define the difference between verification and validation test activities.11. Name two structural and two functional test activities.12. Define a workbench and explain the four main components.13. How is maintenance testing different from new development testing and how is regression

testing different?14. What is the single most valuable static test technique available to testers?15. How do you differentiate a spec from a requirement?16. What are 6 of the main components of a test script?17. What are the 3 components of a test case? 18. What is the difference between a test case and a test script?19. How are internal controls built into software systems and subsequently how are these con-

trols tested?20. Name at least 3 types of test tools (not vendors).21. Explain the difference between QA and QC.22. Give an example of a test standard.23. Define 5 major components of a system test plan.24. What are the drawbacks of using production data to test?25. What are 5 major components of a good test report?26. What are defects and how are they managed and eliminated?27. What is a Pareto Chart and how is it used in testing?28. What is an Ishakawa diagram and how is it used in testing?29. Give a brief summary of how you would justify the expense of continuing to test versus stop-

ping testing (due to deadlines, for example).



Topics for Quality Analysts Answers

1. COQ – Cost of Quality is all costs OTHER than the actual cost to build the software itself. It includes the cost of prevention, (QA activities), appraisal (testing or QC) and failure costs. It is usually as large or larger than the cost to build the software in the first place. Why it’s important to quality and testing is that most organizations spend far too much money in test-ing and not enough in preventive activates (reviews, walkthroughs, etc.)

2. PDCA is attributed to Dr. Edwards Deming, a quality guru from the 1940’s. First you plan your work, then you do the work, then you check to see that you did the work according to your plan, and then you act on what you found (measure and improve). It is a concept well suited to test planning and execution and finally, the follow-up and reporting.

3. A risk matrix is developed by determining which critical success factors are the most impor-tant to the customers during the project being tested. For example, if usability is critical to the client then usability testing should be the primary concern in the test effort in order to ensure a successful project implementation for that client. These usability tests are then found first on the traceability matrix. Other CSF’s include portability, performance, main-tainability, scalability, ease of use, conformance to standards or regulations, etc.

4. The client, the tester, the developer (or application manager) and project manager should par-ticipate in the development of a test strategy. The basic components include the type of soft-ware being testing, the type of project (mainframe, OO etc.), who is to be involved in the testing, what levels of testing are to be conducted, where the testing will be conducted, the environment in which the testing will occur, the risks, the assumptions, the critical success factors for the project, any tradeoffs that might be negotiable (time, scope, etc.),

5. Strategic activities are the high-level planning (where are we headed) like the master test plan and strategy documents. Tactical testing actives would include test plans at the developer level, system specific test plans, etc. (the how are we going to accomplish this activities.)

6. Root cause analysis involves getting to the first place the defect was injected. For example, many defects found in testing are really attributable to faulty requirements. Common causes of variation in testing are things like forgetting to run a series of test cases. Once identified this type of variation can be eliminated by instituting good standards for the test effort. Spe-cial causes are things that happen unexpectedly and infrequently and are hard to plan for. For example, if the operating system changes and no one tells the testers then the tests may not work. This situation is not one that would usually happen and as such is not usually part of the test plan process.

7. A policy is management’s articulation of behaviors they expect employees to follow. Most companies have policies on vacation and casual day attire but not for things like testing, or requirements gathering so people do whatever they think is correct with no guidance from management on the issue. A policy is high-level and strategic. It doesn’t state exactly WHAT to do, but says WHAT should be done. It does show that management is concerned about the subject enough to write a policy and expect compliance. In testing a simple test policy might be stated like this:

“Depending upon the risk score of each project, specific levels of testing must be con-ducted.” What this assumes is that there is a risk assessment process and that once it has been conducted then there are levels of testing that must be undertaken. When you get to the standards that support this policy, then one of the standards might say:



“If the risk score determination is high, then unit, integration, system, performance, and user acceptance testing must be conducted. If the risk score is medium, then unit, system, and user-acceptance testing must be conducted and if the risk score is low then only system and user- acceptance testing must be completed.” What this does is support the policy with “how” to enforce the policy. The procedures would then go on to define how to write a unit test plan, etc.

8. A structured approach means defining the levels of testing and also the accountabilities for each level. I will send you a copy of the V-model if you send me your fax number.

9. Testing is complete when it has been done according to a documented test plan and no addi-tional defects can be found. In other words, not just when the time is up. It means that the critical success factors have been defined and included in the test plan and once they have been adequately tested and are working then the testing can be assumed to be complete. Also, this statement assumes a level of competency in testing that can “predict” the number of defects that will be found and even where they will be found. Only very mature test organizations are actually able to do this.

10. Verification tests processes; validation tests products. This is easily seen on the V-model.11. Two structural test activities:executing branch coverage analysis, or simply running unit tests

to test logic. Functional test activities: conducting a walkthrough and running functional requirements tests (not logic-based).

12. Workbench is another name for a process. The four main components are: input, output, do and check.

13. Maintenance is every bit as important as new development testing in that there is a produc-tion system that is being used by perhaps thousands of clients so fixing problems is of the utmost concern. Regression testing for maintenance systems requires that any defects found previously are retested each and every time the regression test is run. Sound regression testing requires good tools to determine the paths taken through the code so that at a mini-mum these paths can be covered by test cases in the event that test time is limited.

14. Reviews 15. A requirement comes directly from the customer and is the “what” the client wants in the

project. A specification is the restatement of the functional requirement into “pseudo-code” or language that is used by the programmers to develop or change code. It is much more detailed than are usually a requirements document.

16. Log on, log off, security, the test cases, restart procedure, and how to handle an abnormal occurrence.

17. Condition to be tested, the procedure to run the test, and the expected result.18. A test case is usually very simple and is run manually. A script is done in an automated test

environment and can contain hundreds of test cases.19. Internal controls are developed to address the riskiest areas of a software process. For exam-

ple, input and output criteria are good sources of controls over what goes into a process before the process itself even starts. In very critical systems like financial or human-life sys-tems, management should determine which systems and which processes should have addi-tional controls. The Sarbanes-Oxley Act puts heavy burden now on a company’s system on internal controls.

20. Test case generator, code coverage analyzer, inspection.



21. Quality Assurance means defect prevention; Quality Control is defect detection.22. A test standard might state that “ At least 90% of all test cases must execute correctly before

moving to the next level of test”.23. Functions to be tested, test environment, tools to be used, constraints, how defects will be

handled.24. Production data contains errors. Sometimes we know how production data got “dirty” and

sometimes we don’t. Using it to test can be ok if we understand the extent to which it is dirty and add / delete/ or change data to make the testing most effective.

25. Test report would contain, at a minimum: the results of the testing (which functions work and which do not), what risks to implementation remain, the recommendation of the test team, outstanding defects, and recommendations for improvement as noted by the test team (usually become enhancements).

26. A defect is any nonconformance to requirements. The problem is that requirements are not well defined and therefore just about anything (or nothing) could honestly be considered a defect. A problem is just the word itself. Many organizations won’t use the term “defect”.

27. Pareto is based on the 80/20 rule: 80% of the problems will come from 20% of the work. Using root cause analysis for defect identification, for example, you could show on a Pareto Chart which of the defects occur most frequently and in which are of testing do they show up.

28. An Ishakawa diagram is another name for a fish-bone diagram.29. By conducting a requirements review and identifying an classifying the defects found during

the review you could easily see that finding just one fatal defect in the requirements review would have prevented that same defect from finding its way into the testing where it is most expensive to find (or worse, into production to affect the customers). Put a dollar amount to this fatal defect and it usually gets managements attention since the cost of conducting the review is usually far less than the estimated impact of the defect.



Appendix C - Critical Success FactorsAll systems have processes they perform and data they use and maintain. All systems also have certain attributes they must possess if the system is to be considered a success.Critical success factors are usually a combination of what the system must do (processes on data) and what the system must be (attributes) to be successful. Testing needs to identify and focus on verifying and validating that these critical success factors are met.We can identify general success factors as a basis for our test strategy as well as specific concerns testing should address associated with each factor.Critical Success Factors

Shown here the fifteen success factors you should consider during your test strategy plan-ning:

² Correctness - Assurance that the data entered, processed, and outputted by the application system is accurate and complete. Accuracy and completeness are achieved through controls over transactions and data element, which should commence when a transaction is originated and conclude when the transaction data has been used for its intended purpose.

² File integrity - Assurance that the data entered into the application system will be returned unaltered. The file integrity procedures ensure that the right file is used and that the data on the file and the sequence in which the data is stored and retrieved is correct.

² Authorization - Assurance that data is processed in accordance with the intents of manage-ment. In an application system, there is both general and specific authorization for the pro-cessing of transactions. General authorization governs the authority to conduct different types of business, while specific authorization provides the authority to perform a specific act.

² Audit trail - The capability to substantiate the processing that has occurred. The processing of data can be supported through the retention of sufficient evidential matter to substantiate the accuracy, completeness, timeliness, and authorization of data. The process of saving the supporting evidential matter is frequently called an audit trail.

² Continuity of processing - The ability to sustain processing in the event problems occur. Continuity of processing assures that the necessary procedures and backup information are available to recover operations should integrity be lost due to problems. Continuity of pro-cessing includes the timeliness of recovery operations and the ability to maintain processing periods when the computer is inoperable.

² Service levels - Assurance that the desired results will be available within a time frame acceptable to the user. To achieve the desired service level, it is necessary to match user requirements with available resources. Resources include input/output capabilities, commu-nication facilities, processing, and systems software capabilities.

² Access control - Assurance that the application system resources will be protected against accidental and intentional modification, destruction, misuse, and disclosure. The security procedure is the totality of the steps taken to ensure the integrity of application data and pro-grams from unintentional and unauthorized acts.

² Compliance - Assurance that the system is designed in accordance with organizational strat-egy, policies, procedures, and standards. These requirements need to be identified, imple-mented, and maintained in conjunction with other application requirements.

² Reliability - Assurance that the application will perform its intended function with the required precision over an extended period of time. The correctness of processing deals with the ability of the system to process valid transactions correctly, while reliability relates to the system’s being able to perform correctly over an extended period of time when placed into production.



ts

n

le

it

-an

er-

² Ease of use - The extent of effort required to learn, operate, prepare input for, and interpret output from the system. This test factor deals with the usability of the system to the people interfacing with the application system.

² Maintainability - The effort required to locate and fix an error in an operational system. Error is used in the broad context to mean both a defect in the system and a misinterpretation of user requirements.

² Portability - The effort required to transfer a program from one hardware configuration and/or software system environment to another. The effort includes data conversion, program changes, operating system, and documentation changes.

² Coupling - The effort required to interconnect components within an application system and with all other application systems in their processing environment.

² Performance - The amount of computing resources and code required by a system to per-form its stated functions. Performance includes both the manual and automated segments involved in fulfilling system functions.

² Ease of operation - The amount of effort required to integrate the system into the operating environment and then to operate the application system. The procedures can be both manual and automated.

Test Concerns Matrix The table below identifies specific concerns testing should address associated with each factor.

Critical Suc-cess Fac-tors

Require-ments

Design Program Test Installation Maintain

Correctness Complete, clear, correct, testable, measurable, feasible

Design con-forms to require-ments

Programs conform to design

Functional testing

Proper pro-grams and data are placed into production

Update requiremen

Authoriza-tion

Authorization rules defined

Authoriza-tion method designed to meet rules

Authoriza-tion method imple-mented

Compli-ance test-ing

Prohibit data changes dur-ing installa-tion

Preserve authorizatiorules

File Integrity File integrity requirement defined

File integ-rity controls designed

File integ-rity controls imple-mented

Functional testing

Verify inte-gration of production files

Preserve fiintegrity

Audit Trail Reconstruc-tion require-ments defined

Audit trail designed

Implement audit trail

Functional testing

Record installation audit trail

Update audtrail

Continuity of Process-ing

Impact of fail-ure defined

Contin-gency plan designed

Write con-tingency plan

Recovery testing

Assure integ-rity of previ-ous testing

Update contingency pl

Service Level

Desired ser-vice level defined

Method to achieve ser-vice level designed

Design sys-tem to achieve ser-vice level

Stress test-ing

Implement fail-safe installation plan

Preserve svice level



e ith y

u--

il-

n-

-

r--

Access Control

Access defined

Access pro-cedure designed

Implement security pro-cedures


Control access dur-ing installa-tion

Preserve security

Methodol-ogy

Require-ments com-ply with methodology

Design complies with meth-odology

Programs comply with methodol-ogy

Testing complies with meth-odology

Integration complies with methodology

Maintenanccomplies wmethodolog

Reliability Tolerances established

Data integ-rity controls designed

Data integ-rity controls imple-mented

Manual regression & functional testing

Accuracy and complete-ness of installation verified

Update accracy requirements

Ease of Use Usability specifications determined

Design facil-itates use


Manual support testing

Usability instructions disseminated

Preserve ease of use

Maintain-ability

Maintenance requirements determined

Design is maintain-able

Programs are main-tainable

Inspection Documenta-tion complete

Preserve maintainabity

Portable Portability needs deter-mined

Design is portable


Disaster testing

Documenta-tion complete

Preserve portability

Coupling Systems interface defined

Interface design com-plete


Functional & regres-sion testing

Interface coordinated

Assure proper maitenance

Perfor-mance

Performance criteria estab-lished

Design achieves criteria

Programs achieve cri-teria


Integration performance monitored

Preserve level of performance

Ease of Operation

Operational needs defined

Communi-cate needs to opera-tions

Develop operating procedures

Operations test

Implement operating procedures

Update opeating procedures



Appendix D - Critical Skills QuestionsAnswers to these questions can be found starting on page 338.

COMMUNICATION

Giving/Receiving Information1. List five suggestions for relaying defect information and/or performing criticism effectively.

2. What are three reasons why people don’t listen well?

3. Give one example of an effective audience evaluation technique that can be used to level-set a testing presentation or design a course for test or development staff.

4. How do you know if the information you provided to an audience has been understood and communicated effectively?

5. Give three examples of pertinent information that should always be included with defect reports.

Personal Effectiveness1. You are the quality assurance manager at the APEX company. One of your quality assurance

analysts is engaged in a dispute with a developer over the cause of a defect reported by the analyst. What can/should you do?



PROFESSIONAL DEVELOPMENT

Continuing Education1. List three ways a tester can improve his/her test ability.

2. In what ways, if any, can a tester induce change and/or modification to a development pro-cess?

Leadership and Recognition1. What is the main purpose of a task force?

2. What is the role of a facilitator?

3. What role, in test process definition, should a tester play?

QUALITY PRINCIPLES AND CONCEPTS

Quality Principles/Quality Management1. Define cost of quality.

2. Briefly define management by fact.

3. How many Deming principles are there?



4. How many levels are in the CMM?

Quality Assurance/Quality Control1. Describe the fundamental difference between QA and QC.

2. Give five examples of quality attributes or critical success factors.

METHODS FOR SOFTWARE DEVELOPMENT AND MAINTENANCE

Process Knowledge1. Briefly describe the difference between automated and manual testing tools.

2. Give two examples of each type (Not Vendors).

Roles/Responsibilities1. List three types of “interfaces.”

2. Who should be “accountable” for testing?

TESTING PRINCIPLES AND CONCEPTS

Testing Specifications, Techniques, and Methods.1. What is COTS?



2. List five phases in a typical testing life cycle.

3. List at least four components of a good test strategy.

4. What are three general test strategies during requirements?

5. What are the four verification activities that should occur at the design phase?

Verification Concepts and Reviews/Inspection.1. List three types of reviews.

2. What is an audit?

3. What three rules should be followed for all reviews?

4. What is the main goal of a review?



5. What are the three most common phase-end reviews?

6. Post-implementation reviews are held when:a. The product is certified for releaseb. The product is replacedc. The product has been releasedd. There is a major failure

Test Management1. Draw a test process workbench.

2. List five skills a competent tester should have.

3. Give an example of a system objective and its associated test objective.

Test Standards1. List three industry test standard organizations.



2. Give three examples of internal test standards.

Test Environment1. Differentiate between a test bed and a test script.

2. A test environment includes:

RISK ANALYSIS

Risk Identification1. List three of five dimensions of risk.

2. What are the two risk levels?

3. Define risk as it applies to testing.

Managing Risks1. Give three examples of testing risks.

2. When is risk assessment conducted?



TEST TACTICS

Structural Test Approaches1. List four structural test approaches.

Functional Test Approaches1. Give four functional test approaches.

Test Tools/Environment1. Describe the use of a traceability tool.

2. Discuss why change control is/is not important to a test environment.

3. Change control is sometimes part of:a. Problem report documentsb. Defect analysisc. Software library proceduresd. None of the above

4. Configuration management can be applied to non-test changes.True or False

Pre-Planning Activities1. Give two examples of well-defined user acceptance criteria.

2. What is the difference between a test objective and a test case?



3. Should assumptions be included in a test plan? Why or why not?

4. Define entrance and exit criteria.

Test Plan1. List eight parts of a typical system test plan.

2. A unit level test plan should be input to a system or integration level test plan.True or False

TEST DESIGN

Design Preparation1. Test coverage is defined as:

2. Describe the characteristics of a test bed.

Design Execution1. Give three techniques for creation of test data.

2. A test script includes a procedure for executing the test and what other components? (Name at least 3)



Execute Tests1. Execution of a test in a test plan can only occur during the test “phase” of a project life cycle.

True or False Why?

Compare Actual versus Expected Results1. To compare actual to expected results, the tester first designs test cases, which include at least

these three components:

Test Logs1. What is the purpose of a test log?

2. What type of information should be included on a test log?

Record Discrepancies1. The purpose of recording defects is to make a complete record of all discrepancies found dur-

ing testing. The input to this task would include (name 3 items):

2. A defect can be identified in three ways. Name them.



DEFECT TRACKING AND MANAGEMENT

Testing Defect Corrections1. Who should review defect reports?

2. Regression testing is the easiest way to uncover defects.True or False Why?

QUANTITATIVE MEASUREMENT

Test Completion Criteria/Metrics1. What is the purpose of code coverage tools?

2. Give four examples of test specific metrics.

3. Give one commonly recognized size measurement tool.

TEST REPORTING

Reporting tools and standards1. Give three components included in a system test report.

2. How is statistical analysis used in test reporting?



IMPROVING THE TEST PROCESS

Test Quality Control1. What are two ways to demonstrate the test process has been performed correctly?

Analysis of the Test Process1. How could you analyze the current test process?

Test Continuous Improvement1. How could continuous improvement be applied to a Level One CMM test organization?



Critical Skills Answers

COMMUNICATION

Giving/Receiving Information1. Do it privately.

Have the facts.Be prepared to help the person improve.Follow a process when giving criticism.Be specific on expectations

2. The speaker may not be talking about a topic of interest to them.

They are impatient and have a lot of other stimuli going through their minds.

hey are too busy rehearsing what they will say next.

3. Do an internal self-assessment (for a training course) or ask questions of the audience before and after a presentation

4. You ask questions, and conduct an audience evaluation immediately.

5. What type of defect was found?Where was the defect found?What is the severity level of the defect?

Personal Effectiveness1. You should discuss the defect report with the QA analyst to ensure that the information is

correct and that a root cause analysis was conducted. If it is clearly a case of developer error then have both people discuss and come to an effective resolution. Record the defect and sta-tus.

PROFESSIONAL DEVELOPMENT

Continuing Education1. Prepare for the CSTE exam.

Join a local professional association and actively participate.Read journals and magazines about the testing profession.

2. A tester can induce changes into the development process by providing both qualitative and quantitative data derived from the testing process. With this information and metrics, changes can be identified and introduced into the development process that will ultimately benefit the organization. Information like types and frequency of defects found, where in the develop-ment process they were found, the on-time delivery capability, etc. can assist in improving the overall quality of the delivered products.



Leadership and Recognition1. The main purpose of a task force is to develop options—not to solve a problem.

2. To provide an unbiased focal point to a discussion and keep the discussion “on track.”

3. A tester should play a major role in test process definition since the test process is his/her focus. The tester should review the current test process with an objective of inserting improvements and then should pilot the process and provide feedback and suggestions and/or recommendations. If a new test process is being developed then the tester is the best person to ensure that best testing practices are used.

QUALITY PRINCIPLES AND CONCEPTS

Quality Principles/Quality Management1. COQ is everything other than the true cost to build a product. It includes failure and

appraisal/prevention costs.2. MBF means that the specified results should be qualitative and the processes should provide

adequate quantitative data as a by-product of the process to determine whether or not the specified results are being achieved.

3. 14

4. 5

Quality Assurance/Quality Control1. QA activities are used to prevent the introduction of flaws; activities that find and correct

flaws are QA activities.

2. AccuracyEase of useService levelsPortabilityFile integrity

METHODS FOR SOFTWARE DEVELOPMENT AND MAINTENANCE

Process Knowledge1. Manual tools do not execute program code and do not require executing software; automated

tools do.

2. Manual – Reviews and checklistsAutomated – Code coverage analyzers and file comp



Roles/Responsibilities1. Human to human

System to systemHuman to machine

2. Everyone – ultimately a test manager.

TESTING PRINCIPLES AND CONCEPTS

Testing Specifications, Techniques, and Methods.1. Commercial off-the-shelf software (shrink-wrapped)

2. Requirements phaseDesign phaseProgram phaseImplementation phaseMaintenance phase

3. Critical success factorsRisk analysisAssumptionsMethodology to be followed

4. Conduct a requirements review and identify every requirement.Ensure that every requirement is testable.Determine an overall test strategy.

5. Authorization rules are established.File integrity controls are designed.Design conforms to requirements.Contingency plan is developed.

Verification Concepts and Reviews/Inspection.1. Checkpoint, Inspection, and Walk through

2. An inspection/assessment activity that verifies compliance with plans, policies, and proce-dures

3. Only those who could have developed the product should participate.No personal attacks – review is on product.Set time limit on time for review.

4. To uncover defects.

5. Requirements



DesignTest

6. C



Test Management1.

2. Test process knowledge;Excellent written and oral communication skills;Analytical ability;Knowledge of test tools; andUnderstanding of defect management.

3. System objective – System must be portable to both unit and NT operating systems.Test objective – Parallel test with some results on both platforms.

Test Standards1. IEEE

ISODoD

2. All levels of testing will prepare a test plan.Complete test documentation must be available before implement.All Category 1 and Category 2 defects will be resolved before implementation.

Test Environment1. Test beds are developed for a singly type of test; test scripts are an automated capture/play-

back tool and execute over and over.

2. Tools, plat forms, scripts, hardware, software, resources, and skills needed to support testing.

RISK ANALYSIS

Risk Identification1. Technology integration;

Size and complexity;System environment and stability;Criticality / mission impact; andReliability and integrity.

Proceduresto

DOwork

Proceduresto

CHECKwork

PROCEDURES TO:

DO

CHECK

PRODUCT(S) INPRODUCT(S) OUT

REWORK

TOOLS



2. LU1 criticality / mission impact.LU2 size, system environment, reliability, and integration.

3. Risk analysis will identify high-risk applications in order to at them to more extensive testing and it can help focus testing on the critical components and/or quality dimensions that are most important to the project.

Managing Risks1. Lack of trained testers;

Inadequate change control; andCompressed timeframes.

2. At the earliest possible stage of a development project and then it should be conducted at crit-ical points along the development life cycle.

TEST TACTICS

Structural Test Approaches1. Operations testing

Stress testingRecovery testingSecurity testing

Functional Test Approaches1. Requirements

RegressionParallelError-handling

Test Tools/Environment1. A traceability tool follows the paths taken by a computer program as it processes data.

2. Change control is critical. Testing in an unstable or changing environment does not allow the tester to control the test.

3. C

4. True

Pre-Planning Activities1. Response time must be less than 5 seconds.

At least 20 concurrent users must be allowed into the system.

2. Test objective is the overall test criteria that must be met.Test cases include test specifics,



procedures, inputs, and expected results.

3. Yes, otherwise many test issues fall through the cracks.

4. Entrance criteria – Minimum standards to be met before product will be tested.Exit criteria – Minimum standards to be met before the product is released.

Test Plan1. Software description, milestones, schedule, test materials, test procedures, test cases, test

methods, and test assumptions.

2. True

TEST DESIGN

Design Preparation1. Assuring that the application has been covered by the test process.

2. Test beds are files/tables needed to test a system/unit whose contents have been predefined to meet the conditions defined in designing test cases.

Design Execution1. Scripting

Test bedsProduction data

2. Test caseFunctions to be testedTest performedSet-up proceduresValidation procedures

Execute tests1. False. Execution of a test can occur throughout the project lifecycle if a lifecycle approach to

testing is used.

Compare actual versus expected results.2. The condition, the expected result, and the procedure to run the test.

Test Logs1. As each part of the test plan is executed, control can be maintained by using a document cre-

ated to record test activities. A simple worksheet or spreadsheet can be used.



2. The test identifier, the test activity, who executed the test, the start and stop time of the test, and remarks or comments.

Record Discrepancies1. The test plan, the test cases, and the test logs.

2. Missing, wrong, or extra

DEFECT TRACKING AND MANAGEMENT

Testing Defect Corrections1. QA and developer and/or area which defect was found.

2. True. Regression testing revalidates that changes made to the system do not adversely influ-ence other system functions.

QUANTITATIVE MEASUREMENT

Test Completion Criteria/Metrics1. They are used to show the extent to which the logic in the program was executed during test-

ing.

2. User participation;Number of paths tested;Cost of all testing;Number of defects found; andNumber of requirements tested.

3. LOC Analysis

TEST REPORTING

Reporting tools and standards1. Description of test results and findings (defects);

Summary (environment and references); and Recommendation.

2. Can be used to compare actual number of defects found to the number of defects expected or to indicate the number of functional requirements actually tested.



IMPROVING THE TEST PROCESS

Testing Quality Control1. Perform quality control on the test process by comparing the number of defects in the test

process to previous test process; orUse entrance and exit criteria.

Analysis of the Test Process1. Compare it to best practices for testing or compare current test process to previous test pro-

cesses for evidence of improvement.

Test Continuous Improvement1. It really can’t in the true sense of the concept, since continuous improvement requires a

defined and repeatable process before continuous improvement can be attempted.



Appendix E - Pre-test

1. The producer’s view of quality is:a. Meets requirements b. Fit for usec. Projects completed on-timed. Projects completed within budget

2. Quality assurance is:a. Product focusedb. Meant to prevent defects c. A line positiond. Software testing

3. Quality control is:a. Product focused b. Meant to prevent defectsc. A staff positiond. Software testing

4. Quality assurance is responsible for all of the following except:a. Quality planningb. Measurement and analysisc. Telling technicians how to perform their duties d. Facilitation

5. Quality control techniques may include all the following except:a. System testingb. Integration testingc. Inspectionsd. Defining standards and procedures

6. To calculate the Cost of Quality you must add all the following costs except:a. Cost to build the product b. Failure costsc. Appraisal costsd. Preventive costs

7. Software testing is _____________.a. A preventive cost of qualityb. An appraisal cost of quality c. Is not part of the cost of quality calculationd. The Cost to build the product



8. PDCA stands for:a. Plan, Do, Complete, Activitiesb. Plan, Do, Check, Act c. Perform, Deliver, Check, Actd. Plan, Do, Check, Activities

9. A policy statement answers the question:a. Whatb. Howc. Why d. When

10. We test software for all the following reasons except:a. To reduce riskb. To validate requirements workc. To establish confidence that a program worksd. To verify run times

11. Process improvement is most effective when it is performed by:a. Owners of the processb. Quality assurance personnelc. Independent consultantsd. Internal auditors

12. Functional tests validate:a. System quality characteristicsb. System requirements c. Systems structured. Performance requirements

13. White Box testing is used to validate:a. Business requirementsb. Interfaces to other systemsc. Program structure d. Usability

14. Which of the following would not be considered a critical success factor in acquired soft-ware?a. Ease of useb. Expandabilityc. Reputationd. Reliability

15. Acceptance testing is normally designed and performed by:a. The developerb. The quality control managerc. The product development managerd. The user



16. Black box testing is used to validate:a. Business requirements b. Interfaces to other systemsc. Program structure d. Usability

17. Black box testing is typically performed by:a. An independent test team and business users b. An independent test team and developersc. Developers and 3rd party testersd. Auditors and developers

18. The following are black box test techniques:a. Equivalence portioning and boundary analysis b. Error guessing and statement coveragec. Boundary analysis and path coveraged. Equivalence partitioning and decision point to decision point

19. How much testing is enough:a. When all scripts have been executedb. When the cost to detect defects exceeds the potential damage caused by the defect c. When the scheduled end date is reachedd. When testing detects no more defects

20. If a measure is considered valid it means that:a. It is easy to be obtainedb. It measures what it is intended to measurec. It is objectived. Two or more people can collect the same number

21. If you wanted to evaluate if the software system being acceptance testing fits the needs of the user; which components of fitting the users needs would you include in acceptance testing?a. Datab. Peoplec. Structure d. Rulese. All of the above

22. These model categories (leadership, strategic planning, customer and market focus, informa-tion and analysis, human resources, process management and business results) are part of which of the following quality models:a. Deming Prizeb. Malcolm Baldrige National Quality Awardsc. SEI CMMId. ISO 9000 Standards



23. Unit testing is:a. Performed by the developer and validates system requirementsb. Performed by the test team and validates system requirementsc. Performed by the developer and validates the structure of one program module or com-

ponent d. Performed by the test team and validates the structure of one program module or compo-

nent

24. Which recognized organization has developed the generally accepted model for internal con-trol:a. AICPA b. COSOc. CobiTd. ERME ISO

25. The “V Model” is a life cycle approach that integrates:a. White box and black box testingb. Functional and non-functional test techniquesc. Verification and validation techniques d. Boundary value analysis and equivalence partitioning

26. Reviews are conducted for all the following reasons except:a. Reduce time to marketb. Evaluate individual performance c. To detect defects in the phase where it was introduced into the productd. Knowledge transfer

27. Enforcement of meeting standards is the responsibility of:a. Managementb. Quality assurancec. Quality controld. Auditors

28. Which of the following is considered a subjective measure:a. Lines of codeb. Mean time to failurec. Hours to write a thousand lines of coded. Customer satisfaction



29. A loss associated with a customer leaving your website because it was too difficult to use was estimated to be$125 per occurrence and assume that happens 4 times per work day. If the site will only be active for one 5 day week, what is the maximum amount you could spend on controls if you could totally eliminate the problem of your customers leaving your website?a. $500b. $2,500c. $5,000 d. $13,000

30. Testing software developed by a third party company is best called:a. Microsoft Worm (MSW)b. Commercial-on-the-Shelf (COTS)c. Commercial-off-the-Shelf (COTS)d. Commercially Procured Software (CPS)






Appendix F - Quiz 1

1. Inspection of source code of a program.A. Quality AssuranceB. Quality Control

2. Select and Implementation of a development methodology.A. Quality AssuranceB. Quality Control

3. Unit testing to validate that the program works.A. Quality AssuranceB. Quality Control

4. Metrics collected to measure the effectiveness of unit and system testing.A. Quality AssuranceB. Quality Control

5. Verify that your computer jobs have been run for that day.A. Quality AssuranceB. Quality Control

6. Analysis to determine whether computers jobs are being run on time.A. Quality AssuranceB. Quality Control

7. Metrics collected to determine the effectiveness of system testing.A. Quality AssuranceB. Quality Control

8. Identifying defect in a program for the primary purpose of correcting defects.A. Quality AssuranceB. Quality Control

9. Analysis of defect to determine the start of origin.A. Quality AssuranceB. Quality Control

10. Identifies whether attributes are part of a specific product.A. Quality AssuranceB. Quality Control



Essay One

Mr. Mason is the IT director of your department. Mr. Mason has decided to make quality the number-one objective of the department. To accomplish this objective, Mr. Mason set some very specific quality goals, including improved customer satisfaction and reduced defect rates. To accomplish this objective, Mr. Mason established a documentation librarian to assure that all doc-umentation meets standards.

Mr. Mason requires computer operations to review all new systems and changes prior to turnover to ensure they meet standards. Mr. Mason has also established an independent test group to vali-date that all systems function correctly prior to turning them over to production. Based on what you read, evaluate how good a quality leader you believe Mr. Mason is, and state whether you believe his approach is based on quality principles, and why.



Appendix G - Quiz 21. This view of quality means that the requirement has not been met:

a. Provider viewb. Producer viewc. Customer viewd. Supplier view

2. This view means that the product or service has met the needs regardless of the requirements:a. Provider viewb. Producer viewb. Customer viewc. Supplier view

3. Most IT groups have two quality gaps. They are:a. Producer and providerb. Provider and customerc. Producer and customerd. Supplier and customer

4. Which statement BEST defines the producer gap of quality:a. The difference between what is specified versus what is specified to managementb. The difference between what is specified versus what is actually deliveredc. The difference between what is delivered versus what is specified to managementd. The difference between what is delivered versus what is specified by the customer

5. The extent to which a program satisfies its specifications and fulfills the user’s mission objec-tives is called:a. Correctnessb. Efficiencyc. Usabilityd. Flexibility

6. Effort required to locate and fix an error is a program is called:a. Integrityb. Efficiencyc. Usabilityd. Maintainability



7. This view of quality is based on that of the organization that provides the producer with prod-ucts or services:a. Provider viewb. Producer viewc. Customer viewd. Supplier view

8. Identifying defects in a program for the primary purpose of correcting defects.a. Quality Assuranceb. Quality Control

9. Testing is a:a. Processb. Craft or artc. Specific scienced. Cost-effective initiative

10. This view of quality is based on that of the organization that delivers the product:a. Provider viewb. Producer viewc. Customer viewd. Supplier view



Essay

Quality factors have frequently been used as a basis for measuring the quality of an information system. Listed below are four of those quality factors.

What measure or metric would you recommend as an effective measure or metric for each of these quality factors?

1. Ease of Use

2. Maintainability

3. Reliability

4. Security






Appendix H - Quiz 31. In a cost of quality analysis, the costs required to avoid errors to do the job right the first time is called:

a. Appraisal costsb. Failure costsc. Prevention costsd. Direct costs

2. The P in the PDCA stands for:a. Preventb. Predictc. Pland. Pick

3. The cost of quality includes:a. Appraisal costsb. Testing costsc. Failure costsd. Both A and C

4. Three examples of appraisal activities are:a. Planning, organizing, and leadingb. Software testing, reviews, and inspectionsc. Budgeting, controlling expenses, and testingd. Cost controlling, leading, and coordinating

5. Which of the following is NOT a category of the Cost of Quality?a. Appraisal costsb. Failure costsc. Prevention costsd. Cost to Build

6. In defining the cost of quality, appraisal costs are BEST defined as:a. Costs incurred to review interim products against requirementsb. Costs which can not be recoupedc. All costs associated with defective productsd. None of the above



7. The cost of quality can be reduced by:a. Testing at specific points during developmentb. Testing mostly in the requirements stagec. Testing continuously throughout the SDLCd. Testing only at the User Acceptance Stage

8. Identifying defects in a program for the primary purpose of preventing defects.a. Quality Assuranceb. Quality Control

9. In the PDCA, training is provided during this piece:a. Planb. Doc. Checkd. Act

10. Retesting falls into which category of the Cost of Quality?a. Appraisal costsb. Failure costsc. Prevention costsd. Cost to Build



Essay

The computer operations department in your company established a help desk approximately two years ago. Since that time they have carefully recorded each user inquiry, what problems were reported, and the action taken by the help desk.

List and explain two ways that you could use that information to improve your software development pro-cess.






Appendix I - Quiz 4

1. The leading authority and one who started the quality movement in Japan is:a. Thomas A. Eddisonb. W. Edwards Demingc. Ishia Hiaiakowad. Philip Crosby

2. In Deming’s first principle of “Constancy of Purpose” he is identifying the objective that:a. Once a process is established, don’t change itb. All employees must identify the purpose most meaningful to the companyc. Everyone must work toward a common goald. None of the above

3. Regarding Deming’s “constancy of purpose” principle, who is primarily responsible for setting this?

a. Subject Matter Experts (SME)b. Test Managerc. Executive Managementd. Project Manager

4. The reason Deming suggests a company to reduce the number of suppliers is to:a. Promote partnershipsb. Reduce vendor costsc. Develop long term relationshipsd. All of the abovee. None of the above

5. Quality is often referred to as being free. This is because:a. It’s difficult to quantify consumer satisfactionb. There is a never-ending goal to be achievedc. The payback is greater than its cost over timed. Of continuous improvement

6. Which of the following is NOT one of Dr. W. Edward Deming’s fourteen quality principles?a. Learn a new philosophyb. Drive out fearc. Zero defectsd. Look carefully at work standards


7. This quality guru is best known for his trilogy of quality planning, quality assurance, and qual-ity control.

a. Joseph Juranb. W. Edwards Demingc. Ishia Hiaiakowad. Philip Crosby

8. One of the most difficult aspects in installing an effective TQM program is that:a. It takes too much moneyb. It is difficult to get people to participatec. The company culture most often must changed. Managers do not understand the workers

9. Deming’s four activities of quality management are:a. Requirements, plans, execute, and checkb. Plan, check, do, and feedback loopc. Plan, do, check, and actd. Plan, organize, lead, and control

10. Total Quality Management (TQM) means:a. Managing the quality of entire systemb. Emphasizing the importance of managing qualityc. Implementing company-wide quality attitudes and methodologiesd. Having the Quality Manager included in senior management


Essays

1. Compare and con trast the de finitions of "quality” associ ated with Deming, Juran, andCrosby.



2. Dr. W. Edward Deming stated 14 Quality Principles for organizations to use in becoming aworld class organization. Listed below are two of the principles. Explain what you believeis required to implement each of these two quality principles.

Create consistency of purpose in the company:

Drive out fear:



Appendix J - Quiz 51. What is normally NOT included on a problem identification worksheet that is prepared by

the tester?a. Name of the problem b. Discussion of the problemc. Screen prints of the problem d. Who caused the problem e. Severity of the problem

2. Which of the following is the correct sequence for the five phases of process maturity, as defined in the capability maturity model?a. Initial, repeatable, defined, managed, optimizingb. Initial, defined, repeatable, managed, optimizingc. Initial, managed, defined, repeatable, optimizing d. Initial, defined, optimizing, managed, repeatable

3. Experience has shown that over half of all defects originate in:a. Design b. Coding c. Testing d. Requirements e. Acceptance Testing

4. Some organizations have stated that testers make more defects in performing testing than developers make in building software. The most effective way to reduce tester's defects is to:a. Increase the importance of the testing processb. Apply continuous process improvement in the testing process c. Develop quality control checklists for testers to use d. Hire more competent testers e. Establish an inspection process to inspect test products

5. One way to minimize risk is to:a. Decrease acceptance testingb. Install Controls c. Increase the probability factord. None of the above

6. What type of testing is most closely associated with making system changes:a. Stress testing b. Regression testing c. Liability testing d. Correctness Testing



7. In-process reviews which are used to review the developmental work and identify defects, normally occur:a. Near the conclusion of a life cycle phase b. Preventive costc. At the end of the project d. When directed by test management e. After the project has been placed into production

8. Which of the following is not a component of a tester's workbench? (A workbench is a graph-ical representation of the testing risk.)a. Entrance criteria (inputs)b. Testing policies c. Exit criteria (deliverables) d. Work procedures e. Check procedures

9. Which statement best communicates the advantages of performing quality reviews?a. Reviews are conducted to prevent defects b. Results of the review process can be used to evaluate individual performance c. Reviews are conducted at various milestones to detect defects early in developmentd. Serving on a review team requires no special skills

10. Which is the correct sequence for the quality management infrastructure from lowest to high-est level?a. Teams, Management Committees, Quality Councilb. Quality Council, Management Committees, Teamsc. Management Committees, Teams, Quality Councild. Teams, Quality Council, Management Committeese. Quality Council, Teams, Management Committees

11. Which of the following types of testing is considered dynamic testing?a. Code inspections b. System reviewsc. Checklist d. Unit Testing e. Code reviews

12. Performance testing subjects a system to which of the following types of tests?a. Test scripts b. Large volumes of tests c. How fast the system responds d. Time spent fixing errorse. Error conditions

13. In a cost of quality analysis, the cost to run and maintain a help desk is called:a. Appraisal costb. Preventive costc. Failure cost d. Direct cost



14. Regression testing can be performed in which test stage(s)?a. Unit testingb. String/integration testingc. System testing d. Acceptance testing e. All of the above

15. Three parts of the PDCA cycle are act, plan and do. The fourth part of the PDCA cycle is:a. Contribute b. Calibrate c. Checkd. Create e. Condone

16. Which activity verifies whether a specific attribute(s) is, or is not, in a specific product or ser-vice?a. Quality Control b. Quality Assurance c. Computer Operationsd. Help Deske. Internal Auditor

17. Structural testing and functional testing typically require the same sets of experience, skills, and knowledge.a. Trueb. False

18. White-box testing is primarily:a. Logic driven b. Data drivenc. Bottom-up Drivend. Top Down Drivene. Defect Driven

19. The Quality Assurance is the function responsible for:a. Testing Software b. Writing Processes c. Managing Quality d. Conducting Reviews and Inspectionse. All of the Above

20. When we say that quality assurance must be a staff function, we mean it should:a. Take a direct position in detecting defects b. Report to the lowest executive c. Have direct supervisory responsibility for quality controld. Serve in an advisory capacity for quality, training, planning, measurement, and process

controle. All of the above






Appendix K - Self Written QuizCreate your own multiple choice and essay quiz based on the CBOK Skill Categories discussed.

Question 1.

Answer a.

Answer b.

Answer c.

Answer d.

Correct Answer _____ CBOK Page # _____

Question 2.

Answer a.

Answer b.

Answer c.

Answer d.


Question 3.

Answer a.

Answer b.

Answer c.

Answer d.




Question 4.

Answer a.

Answer b.

Answer c.

Answer d.


Question 5.

Answer a.

Answer b.

Answer c.

Answer d.


Question 6.

Answer a.

Answer b.

Answer c.

Answer d.




Question 7.

Answer a.

Answer b.

Answer c.

Answer d.


Question 8.

Answer a.

Answer b.

Answer c.

Answer d.


Question 9.

Answer a.

Answer b.

Answer c.

Answer d.




Question 10.

Answer a.

Answer b.

Answer c.

Answer d.




Essay 1.

CBOK Page # _____

Answer



Essay 2.

CBOK Page # _____

Answer



Essay 3.

CBOK Page # _____

Answer



Essay 4.

CBOK Page # _____

Answer



Essay 5.

CBOK Page # _____

Answer





Documents

Software Quality Analyst Certification Exam · PDF fileReview of Skill Categories in the Common Body of Knowledge (CBOK) for the Certified Software Quality Analyst: • Quality Principles