Improved Performance through Improved Performance through the use of Defined Metricsthe use of Defined Metrics

by Penny by Penny DennehyDennehy

The Boeing CompanyThe Boeing Company

22

BackgroundBackground

nn Airline metrics exist today (DAL, AAL, UAL, JAL, Airline metrics exist today (DAL, AAL, UAL, JAL, GECAS, CAL, KLM, UPS, FedEx, BAB, etc…..)GECAS, CAL, KLM, UPS, FedEx, BAB, etc…..)

nn Initiatives of Six Sigma, ISO 9000, LEAN, Initiatives of Six Sigma, ISO 9000, LEAN, establish performance metricsestablish performance metrics

nn Challenges Challenges –– internal measurement vs. external internal measurement vs. external measurement (OEM to Airline, OEM to submeasurement (OEM to Airline, OEM to sub--tier tier supplier)supplier)

nn For health of Aerospace industry, metrics are all For health of Aerospace industry, metrics are all part of the supplier/customer relationshippart of the supplier/customer relationship

33

Industry Metric InitiativeIndustry Metric Initiativenn Objective Objective nn To develop a common set of ground rules and To develop a common set of ground rules and

defined parameters for the aerospace defined parameters for the aerospace industry/ trading partners to measure industry/ trading partners to measure performanceperformance

nn Utilize an industry standard Utilize an industry standard –– Spec 2000Spec 2000

nn Ensure metrics are consistent, simple and Ensure metrics are consistent, simple and understandableunderstandable

44

Category A B C D

Quality System Quality Culture Supplier Maturity Matrix N/A N/A

Product Performance Special Action Documents Service Bulletin Quality Number of RejectsQuality Performance Specification Performance Quality Problem Alerts Aircraft Quality Defects per Million

Defects per Million

Communication New A/C Delivery DeliveryService Performance Client Support Delivery (Spares) Maintenance/Mod Issues Lead Time

Delivery Delays/Cancellations

Warranty Issues N/A Number of Claims Number of Claims N/ACycle Time Cycle Time

Service Bulletins N/A S/B - Hours Inaccurate S/B Data Integrity N/A

Initiative Creative SolutionsBest Practices Innovation Responsiveness Strategic Section Tech/Non Tech Support

Airline Metric ExamplesAirline Metric Examples

55

Metrics PhilosophyMetrics Philosophy

Ideal Metrics are:Ideal Metrics are:

nn Important to CustomerImportant to Customernn Traceable Traceable nn QuantifiableQuantifiablenn CredibleCrediblenn Well definedWell definednn Show performance over Show performance over

timetimenn Strategic to the businessStrategic to the business

Exec

1st Line

Employee

Supplier

Sub-Process

Customer

66

Metrics PhilosophyMetrics Philosophy

Ideal metrics for the User are:Ideal metrics for the User are:

nn ActionableActionablenn Timely & AccessibleTimely & Accessiblenn Simple & UnderstandableSimple & Understandablenn Able to drive appropriate Able to drive appropriate

behaviorbehavior

Exec

1st Line

Employee

Supplier

Sub-Process

Customer

77

• To be determined by each user ACCOUNTABILITY:nn Metric Owner (the person who manages the charts)Metric Owner (the person who manages the charts)nn TargetsTargetsnn Sources of the data Sources of the data nn Who collects and manages the database (dataWho collects and manages the database (data--sources)sources)nn The configuration of the chartsThe configuration of the chartsnn Who produces the chartsWho produces the chartsnn Project IntegratorProject Integrator

n Why the metric is valuablenn General statement of why it is importantGeneral statement of why it is importantnn QFD result referenceQFD result reference

n What does the metric depictnn What process does this metric relate to?What process does this metric relate to?nn This is a description of the metricThis is a description of the metricnn What does each plot point depict?What does each plot point depict?

n How is the metric constructednn What are the data elements (ref. where they are defined)What are the data elements (ref. where they are defined)nn What are the criteria to include data elements in the What are the criteria to include data elements in the

calculated plot pointscalculated plot pointsnn What calculations produce the metricWhat calculations produce the metric

Ind

ust

ry M

etri

csM

etri

cs D

icti

on

ary:

A C

hec

klis

t

88

Participants Included…Participants Included…AeroxchangeAeroxchange

Air CanadaAir Canada

Airbus Airbus

American AirlinesAmerican Airlines

Air Transport Association Air Transport Association (ATA) (ATA)

British AirwaysBritish Airways

BoeingBoeing

Cathay PacificCathay Pacific

Crane AerospaceCrane Aerospace

DeltaDelta

Eaton AerospaceEaton Aerospace

EmiratesEmirates

FedExFedEx

GE TransportationGE Transportation

GoodrichGoodrich

Hamilton SundstrandHamilton Sundstrand

HoneywellHoneywell

KLMKLM

MoogMoog

Northwest AirlinesNorthwest Airlines

Parker AerospaceParker Aerospace

Pratt & WhitneyPratt & Whitney

QantasQantas

Rockwell CollinsRockwell Collins

Rolls RoyceRolls Royce

Smiths AerospaceSmiths Aerospace

SouthwestSouthwest

ThalesThales AvionicsAvionics

UPSUPS

99

Industry CommentsIndustry Comments

“I want to share my belief that this is an extremely “I want to share my belief that this is an extremely important initiative for the industry, let alone Air Canada. important initiative for the industry, let alone Air Canada. When one considers that this can and should be When one considers that this can and should be standardized so that we can accurately compare "apples standardized so that we can accurately compare "apples to apples" to benchmark things other than price, the to apples" to benchmark things other than price, the value becomes quite clear. …value becomes quite clear. … Rating systems that are Rating systems that are open and transparent to either side of the relationship open and transparent to either side of the relationship (buyer or seller) are the only ones that will provide (buyer or seller) are the only ones that will provide value. If we continually "spin" numbers to project an value. If we continually "spin" numbers to project an appearance of high or even reasonable appearance of high or even reasonable achievementachievement when another perspective may reveal when another perspective may reveal something quite different, the system is not very useful.”something quite different, the system is not very useful.”

Air Canada, Doug EllisAir Canada, Doug Ellis

1010

ATA Spec 2000ATA Spec 2000

Introducing Chapter 13Introducing Chapter 13

Performance Metric Standards

1111

Six Key Process MetricsSix Key Process MetricsMetric = A common set of ground rules and defined parameters

nn Component Removal/ReliabilityComponent Removal/Reliability

nn Component Repair Component Repair

nn Parts DeliveryParts Delivery

nn Product Support DataProduct Support Data

nn WarrantyWarranty

nn Technical ResolutionTechnical Resolution

1212

Component Removal/Reliability Component Removal/Reliability MetricsMetrics

n Mean Between Removal/Mean Cycles Between Removal (MTBR/MCBR)n A performance figure calculated by dividing the operating time, or total unit flying cycles,

accrued in a period by the number of unit removals (scheduled plus unscheduled) that occurred during the same period. Specified in hours

n Mean Time Between Unscheduled Removal/Mean Cycles Between Unscheduled Removal (MTBUR/MCBUR)

n A performance figure calculated by dividing the total unit flying hours accrued ((Quantity per aircraft) times (Flying Hours or Cycles)) in a period by the number of unscheduled unit removals that occurred during the same period.

n Mean Time Between Failure/Mean Cycle Between Failure (MTBF/MCBF)

n A performance figure calculated by dividing the total unit flying hours accrued ((Quantity per Aircraft) times (Flying hours or cycles)) in a period by the number of confirmed unit failures that occurred during the same period.

n No Fault Found Rate

n A performance figure calculated by dividing the total units returned to the shop with no confirmed defect by the total number of units returned to the shop. This is a percentage of units determined to be no fault found out of the whole population of units removed during the same period

1313

Component Repair MetricsComponent Repair MetricsRepair agency (a more general term for supplier) – includes OEM, 3rd party, etc

nn Repair Agency Processing Time Repair Agency Processing Time n The amount of time a repair agency takes to repair a returned part, as measured from the date

of part receipt to the date part shipped. Measured in calendar days. Including all repair process codes.

nn Adjusted Repair Agency Processing Time Adjusted Repair Agency Processing Time n The amount of time a repair agency takes to repair a returned part, as measured from the date

of part receipt to the date part shipped excluding customer hold time (date of customer notification of hold through date of customer release from hold). Measured in calendar days. Including all repair process codes

nn Performance to Repair Agency Processing Time Commitment Performance to Repair Agency Processing Time Commitment n The percentage of orders that are less than or equal to the Repair Agency Processing Time

commitment. Including all process type codes

1414

Parts Delivery MetricsParts Delivery Metricsnn Performance to Lead time Performance to Lead time

n Supplier performance to Lead Time, or to customer Specified Shipping date for a given reporting period, whichever is later. Canceled orders are excluded. Reported as a percentage

nn Performance to Specified Shipping DatePerformance to Specified Shipping Daten Supplier performance to customer Specified Shipping Data for a given reporting period,

excluding cancelled orders or orders for initial provisioning. Reported as a percentage.

1515

Product Support Data MetricProduct Support Data Metric

nn Revision Activity Revision Activity n Timeliness of product support data revisions: the percentage of changes that have

been incorporated into the product support document. The metric can be calculated to provide supplier performance using all of the supplier’s product support data or by just one type of product data

Data Acceptability is a quality measure and needs to be incorporated into a supplier’s quality control system

1616

Warranty MetricsWarranty Metricsnn Average Warranty claim response timeAverage Warranty claim response time

n Average time warranty claim is processed (accepted, denied or rejected) by warrantor in calendar days over a mutually agreed period)

nn First time claim resolution rateFirst time claim resolution raten To measure the percentage of claims resolved by warrantor in initial response – Plot

point provides the percentage of new warranty claims that are completed on initial submittal

nn Disputed claims response time Disputed claims response time n To measure the processing time by a warrantor of a disputed claim – Plot shows the

number of days required to complete the resolution of disputed claims

nn Warranty claim return rateWarranty claim return raten Measure the percentage of claims that are Returned due to invalid or incomplete data

to identify process issues at claimant – Plot point shows the percentage of warranty claims that are returned without action because of incomplete information or invalid content

1717

Technical Resolution MetricsTechnical Resolution Metrics

nn Overall Cycle Time for Technical Issue Resolution Overall Cycle Time for Technical Issue Resolution n Initial time in days to gather data related to the reported event and to identify reported

event root causes – OEM and Airlines shared responsibility

nn Cycle time for technical solution study Cycle time for technical solution study n Time in days to define the technical solution – OEM responsibility

nn Cycle time for technical solution embodiment Cycle time for technical solution embodiment n Time in days to implement solution – Airline responsibility

1818

Spec 2000 (2005.1)Spec 2000 (2005.1)Chapter 13 Chapter 13 -- Performance Metric StandardsPerformance Metric Standards

Purpose

The assumptions, parameters and definitions for performance metrics outlined in this document will establish a standard for measuring key processes in the aerospace industry. These performance metric standards support contractual requirements, business needs, and ensure simplicity and consistency in the industry.

The processes this chapter will address are:

• Parts Delivery

• Warranty

• Component Repair

• Component Removal/Reliability

• Product Support Data

• Technical Resolution

1919

Next StepsNext Steps

nn Work closely with the Aerospace Work closely with the Aerospace Industry to ensure completion of the Industry to ensure completion of the new chapternew chapter

nn Next meeting first week of FebruaryNext meeting first week of February

nn Work with trading partners to Work with trading partners to incorporate this new standardincorporate this new standard