デブサミ関西2013【A4】コード品質は曖昧なままか(安竹由起夫氏)

  • View
    1.172

  • Download
    1

Embed Size (px)

DESCRIPTION

DevelopersSummit2013Kansai2013/9/20 ISO/IEC9126

Transcript

Coverity-DevSumiKansai2013.pptx

Summit Developers

Developers Summit 2013 Kansai Action !

A4 #kansumiA4

Summit Developers

Developers Summit 2013 Kansai Action !

2

Summit Developers

Developers Summit 2013 Kansai Action !

COTS

COTS

h(p://scan.coverity.com

Summit Developers

Developers Summit 2013 Kansai Action !

US SEC. 933. IMPROVEMENTS IN ASSURANCE OF COMPUTER SOFTWARE PROCURED BY THE DEPARTMENT OF DEFENSE.

(a) Baseline SoMware Assurance Policy- The Under Secretary of Defense for AcquisiTon, Technology, and LogisTcs, in coordinaTon with the Chief InformaTon Ocer of the Department of Defense, shall develop and implement a baseline soMware assurance policy for the enTre lifecycle of covered systems. Such policy shall be included as part of the strategy for trusted defense systems of the Department of Defense.

(b) Policy Elements- The baseline soMware assurance policy under subsecTon (a) shall--

(1) require use of appropriate automated vulnerability analysis tools in computer so3ware code during the en7re lifecycle of a covered system, including during development, operaTonal tesTng, operaTons and sustainment phases, and reTrement;

(2) require covered systems to idenTfy and prioriTze security vulnerabiliTes and, based on risk, determine appropriate remediaTon strategies for such security vulnerabiliTes;

(3) ensure such remedia7on strategies are translated into contract requirements and evaluated during source selecTon;

NaTonal Defense AuthorizaTon Act 2013

Summit Developers

Developers Summit 2013 Kansai Action !

5

Summit Developers

Developers Summit 2013 Kansai Action !

SAT &

if x=0

... ...

If x != 0

NULL

X!=0 X=0

X!=0 X=0 void foo(int *p) { *p = 42; }

void bar() { foo(p); if(p != 0) { ... } }

int *p = malloc(sizeof(int)); if(p != 0) *p = 42; ... int *p = malloc(sizeof(int)); if(p != 0) *p = 42;

int *p = malloc(sizeof(int));

*p = 42;

htmlEncode()

< > & " '

a b c d < > &

Summit Developers

Developers Summit 2013 Kansai Action !

7

Address

Summit Developers

Developers Summit 2013 Kansai Action !

8

Summit Developers

Developers Summit 2013 Kansai Action !

(source)

Summit Developers

Developers Summit 2013 Kansai Action !

10

Summit Developers

Developers Summit 2013 Kansai Action !

Python

Summit Developers

Developers Summit 2013 Kansai Action !

Samba

Summit Developers

Developers Summit 2013 Kansai Action !

ANTLR

13

Summit Developers

Developers Summit 2013 Kansai Action !

14

Summit Developers

Developers Summit 2013 Kansai Action !

15

1,000

1,000

1,000

5.9 /K 4.85/K 0.69/K 15/

0.05/K 0/K 0.01/K 1/

1.47/K 0.69/K 0.16/K 9/

20 17900 (179M step)

Summit Developers

Developers Summit 2013 Kansai Action !

24

Summit Developers

Developers Summit 2013 Kansai Action !

25

Summit Developers

Developers Summit 2013 Kansai Action !

26

0"

50"

100"

150"

200"

250"

300"

350"

400"

Alameda" Berkeley" Carmel" Davis" Eureka*"

Num

ber'o

f'Def

ects

'

Defects'Addressed'by'Coverity'Quality/Security'Advisor''

High"Impact" Medium"Impact" Low"Impact"

Summit Developers

Developers Summit 2013 Kansai Action !

:

6 3 6

2 2 2 GA

Alameda, Berkeley, Davis, Eureka, Fresno,

:

2

Jira Pivotal Bugzilla

Summit Developers

Developers Summit 2013 Kansai Action !

Front EndCompilation

AnalysisCore Analysis

Coverity Connect (CC)Defect Management

1 Gc_rc gc_pbkdf2_sha1 (const char *P, size_t Plen, 2 const char *S, size_t Slen, 3 unsigned int c, 4 char *DK, size_t dkLen) 5 { 6 char U[20] T[20]; 7 unsigned int hlen = 20, u, l, r, i, k; 8 int rc; char *tmp; size_t tmplen 9 10 if (c == 0) 11 return GC_PKCS5_INVALID_ITERATION_COUNT; 12 r = dkLen - (l - 1) * hLen; 13 14 memcpy (tmp, S, Slen);

:

Tests automated and run during build

Tests automated and run during build

Mostly manuallytested

End-to-end (E2E) testsperformed manually

: Coverity Connect End-to-End

CondenTal: For Coverity and Partner use only. Copyright Coverity, Inc., 2013 28

Summit Developers

Developers Summit 2013 Kansai Action !

Coverity Connect

Confidential: For Coverity and Partner use only. Copyright Coverity, Inc., 2013

29

Summit Developers

Developers Summit 2013 Kansai Action !

30

% C

ode

Teste

d

Effort to develop tests

100%

Diminishing return forincreased test effort1

Not all code is testable - unreachable statements - dead code, ...

2

Not all tested code adds equal value to the test - non-critical code - debug code, legacy code - exception handling, ...

3

- - - -

- - - ...

Summit Developers

Developers Summit 2013 Kansai Action !

31

SCM: Git, CVS, Mercurial, Subversion, Perforce, ClearCase, AccuRev, MS TFS

gcov (C/C++) BullseyeC/C++) IBM PureCovC/C++) Corbertura (Java)

.

Summit Developers

Developers Summit 2013 Kansai Action !

3 : 100% ( ,

, )

0"

2"

4"

6"

8"

10"

12"

14"

16"

18"

20"

0"

5"

10"

15"

20"

25"

30"

35"

28*Ap

r*12"

5*May*12

"

12*M

ay*12

"

19*M

ay*12

"

26*M

ay*12

"

2*Jun

*12"

9*Jun

*12"

16*Ju

n*12"

23*Ju

n*12"

30*Ju

n*12"

7*Jul*

12"

14*Ju

l*12"

21*Ju

l*12"

Num

ber'o

f'Bug

s'Fou

nd'

Num

ber'o

f'Tes

ts'fr

om'T

A'

Date'

Test'Advisor'Applica:on'in'Frontend'Project'

Tests"added"through"TA" Bugs"found"by"TA"tests"

29 19

Keil

32

Summit Developers

Developers Summit 2013 Kansai Action !

20112013 Coverity Connect End-to-End

0"

20"

40"

60"

80"

100"

120"

Alameda" Berkeley" Carmel" Davis" Today"

Pers

on'D

ays'

CC'Manual'Tes0ng'Eort'

0"

4"

8"

12"

16"

20"

Alameda" Berkeley" Carmel" Davis" Today"

Pers

on'D

ays'

E2E'Manual'Tes1ng'Eort'

Alameda' Berkeley' Carmel' Davis' Today'Manual'CC'GUI'Tests' 347' 661' 1006' 931' 1194'Automated'CC'GUI''Tests' 0' 2' 44' 220' 403'

0'

250'

500'

750'

1000'

1250'

1500'

1750'

CC"Test"Automa,on"Progress"

Alameda' Berkeley' Carmel' Davis' Today'Manual'E2E'Tests' 300' 370' 175' 143' 60'Automated'E2E'Tests' 0' 120' 1468' 1761' 3466'

0'

500'

1000'

1500'

2000'

2500'

3000'

3500'

4000'

E2E#Test#Automa-on#Progress#

33

Summit Developers

Developers Summit 2013 Kansai Action !

34

0"

0.2"

0.4"

0.6"

0.8"

1"

1.2"

1.4"

1.6"

Alameda" Berkeley" Carmel" Davis"

Nor

mal

ized

+Num

ber+

of+D

efec

ts+

Customer4found+Defects+

Summit Developers

Developers Summit 2013 Kansai Action !

() Coverity Test Advisor

35

Summit Developers

Developers Summit 2013 Kansai Action !

I suggest your Next AcTon!

Summit Developers

Developers Summit 2013 Kansai Action !

M Y R E C O M M E N D N E X T A C T I O N !

37

!

Summit Developers

Developers Summit 2013 Kansai Action !

CoverityQA/SA

Jenkins-CI

Coverity QA/SA/TA

CoverityPolicy

Manager QAVIP

ALM

Coverity (Coverity Connect)

QA

QNX