IWFST2005 1

A Research Framework for Empirical Software Engineering Collaboration and Its Application

in a Software Development Project

Yoshiki Mitani*, Nahomi Kikuchi**, Tomoko Matsumura***, Satoshi Iwamura****, Mike Barker***, Ken-ichi Matsumoto***

*IPA/SEC,NAIST(EASE),**IPA/SEC,***NAIST(EASE),****NTT Software(EASE)

Demonstrate and verify empirical software engineering methods A new research framework In response to government policy Field data sharing between Industry & Academia Concept of a “Macro-measurement tools”

Target project: A Governmental Multi-Vendor Consortium type Software Project

IPA: Information Technology Promotion Agency, JapanSEC: Software Engineering Center

EASE: Empirical Approach to Software Engineering projectNAIST: Nara Institute of Science and Technology

Information-technology Promotion Agency, Japan

SoftwareEngineeringCenter

IWFST2005 2

In-process Project Measurement and Feedback Structure

Bug TrackingSystem

Mailing-listManagement

System

ReviewReport Sheet

Mail

Bug Report

XML Standard Data Format

RDB

EPMBasic

Analyzer

EPMExtended Analyzer

Code CloneAnalyzer

CollaboratingFiltering

tool

DevelopmentEnvironment

ProgramDevelopment

BugManagement

e-Mail

Review

Benchmark Data Report

SEC Benchmark Database(over 1,000projects)

ConfigurationManagementSystem

Source Code

Feedback to in-process Software Development Project

40 Management ItemsOperation Rule, Input Web Template

Operation Rule

20 Management ItemsOperation Rule, Input Template

(CVS)

(GNATS)

OperationRile

400 Items

Q&A Interview to PM

Q&A Check-list

Interview Check-listAnalyzer

400 Management Items

A

B

C

A

B

C

類似度の分布

0102030405060708090100

0.91.0

～

0.80.9

～

0.70.8

～

0.60.7

～

0.50.6

～

0.40.5

～

0.30.4

～

0.20.3

～

0.10.2

～

0.00.1

～

類似度の値域

度数

012345統合 スコープ

タイム

コスト

品質

人的資源

コミュニケーションリスク調達

顧客

技術

組織

基本動作

モチベーション課題管理

012345統合 スコープ

タイム

コスト

品質

人的資源

コミュニケーションリスク調達

顧客

技術

組織

基本動作

モチベーション課題管理

IWFST2005 3

Table1 Measurement Data ItemsProceeding

Project

Data

Data item Review record, Bug report(40 items), Specification Change report, Inter company mail with basic document, Basic Design Document, CVS repository with source code

Tool CVS, GNATS, Mailman, EPM

Data input form & template: Review report form, GNATS web template form

Operation rule CVS, GNATS, Mail, EPM

Collected data are kept safely in SEC secret data room.

Data collecting

Method

Individual IT vender development process: Weekly collect data by media from CVS and GNATS.

Integration test process: Use single GNATS system and collect data from it. CVS data are same as individual development process.

Ended

Project

Data/

Bench mark

Data

Data item SEC defined 400 bench mark data items, IT skill data based on IPA/ITSS

Data collecting

Method

SEC defined bench mark data input form( EXCEL template)

Data collection timing; end of basic design and end of development

Project Context

Monitoring project

Observe whole development process, Join to project meeting.

EPM: Empirical Project MonitorITSS: Information Technology Skill Standard

IWFST2005 4

Table 2 Data Analysis ExampleBasic Analysis Source line of code transition, Check-in opportunity, Check-out

opportunity & frequency,

Number & opportunity of mail, Number & opportunity of bur report,

Pending bug transition, Residual bug transition,

Extended Analysis Review quality analysis from IEEE 982.1(1988):Fault Density, Defect Density,

Measurement from PSP (Personal Software Process)

ODC (Orthogonal Defect Classification): Review quality

File renewal analysis

Bench mark Analysis

Source Code Clone Analysis

Retrieve similar project from 1,000 past project benchmark data .

Skill data Analysis Analyze human factor from IT skill data of development persons

Code Clone Analysis

IWFST2005 5

Fig.1 “Macro Measurement Tools” and its Management & Operation for Software Project Measurement

ASDP:　 Advanced Software Development Project

CVS, GNATS, Mailman

EPM,400itemBenchmarkData,ITSS

EASE project, SEC

MEXT, METI

Prime Minister Leadership

Measurement Target Project

Software Tools

Measurement Platform

Industry & Academia Collaboration Framework

Ministry Office Policy

Government Policy

MacroMeasurementTools

TriggerBudget andPolicy

National Policy

A&I CollaborationManagement & Operation

MeasurementPlatform

Management & Operation

Software ToolManagement & Operation

DevelopmentProject

Management & Operation

EPM: Empirical Project MonitorITSS: Information Technology Skill Standard

EASE: Empirical Approach to Software EngineeringSEC: Software Engineering Center

MEXT: Ministry of Education, Culture, Sports, Science and TechnologyMETI: Ministry of Economy, Trade and Industry

IWFST2005 6

The major factors to realize measurement and data-sharing

• The fundamental I&A collaboration research framework

• Concept and equipment of measurement platform

• Thoughtful measurement plan

• Total framework coordinationAcknowledgment This work is supported by IPA/SEC, METI and MEXT of Japan as part of the Comprehensive Development of e-Society Foundation Software program. We thank the researchers in SEC and the EASE project who kindly support our project.

Information-technology Promotion Agency, Japan

SoftwareEngineeringCenter