Download pdf - デブサミ関西2013【A4】コード品質は曖昧なままか（安竹由起夫氏）

Coverity-DevSumiKansai2013.pptx

Summit Developers

Developers Summit 2013 Kansai Action !

A4 #kansumiA4

Summit Developers


2

Summit Developers


COTS

COTS

h(p://scan.coverity.com

Summit Developers


US SEC. 933. IMPROVEMENTS IN ASSURANCE OF COMPUTER SOFTWARE PROCURED BY THE DEPARTMENT OF DEFENSE.

(a) Baseline SoMware Assurance Policy- The Under Secretary of Defense for AcquisiTon, Technology, and LogisTcs, in coordinaTon with the Chief InformaTon Ocer of the Department of Defense, shall develop and implement a baseline soMware assurance policy for the enTre lifecycle of covered systems. Such policy shall be included as part of the strategy for trusted defense systems of the Department of Defense.

(b) Policy Elements- The baseline soMware assurance policy under subsecTon (a) shall--

(1) require use of appropriate automated vulnerability analysis tools in computer so3ware code during the en7re lifecycle of a covered system, including during development, operaTonal tesTng, operaTons and sustainment phases, and reTrement;

(2) require covered systems to idenTfy and prioriTze security vulnerabiliTes and, based on risk, determine appropriate remediaTon strategies for such security vulnerabiliTes;

(3) ensure such remedia7on strategies are translated into contract requirements and evaluated during source selecTon;

NaTonal Defense AuthorizaTon Act 2013

Summit Developers


5

Summit Developers


SAT &

if x=0

... ...

If x != 0

NULL

X!=0 X=0

X!=0 X=0 void foo(int *p) { *p = 42; }

void bar() { foo(p); if(p != 0) { ... } }

int *p = malloc(sizeof(int)); if(p != 0) *p = 42; ... int *p = malloc(sizeof(int)); if(p != 0) *p = 42;

int *p = malloc(sizeof(int));

*p = 42;

htmlEncode()

< > & " '

a b c d < > &

Summit Developers


7

Address

Summit Developers


8

Summit Developers


(source)

Summit Developers


10

Summit Developers


Python

Summit Developers


Samba

Summit Developers


ANTLR

13

Summit Developers


14

Summit Developers


15

1,000

1,000

1,000

5.9 /K 4.85/K 0.69/K 15/

0.05/K 0/K 0.01/K 1/

1.47/K 0.69/K 0.16/K 9/

20 17900 (179M step)

Summit Developers


24

Summit Developers


25

Summit Developers


26

0"

50"

100"

150"

200"

250"

300"

350"

400"

Alameda" Berkeley" Carmel" Davis" Eureka*"

Num

ber'o

f'Def

ects

'

Defects'Addressed'by'Coverity'Quality/Security'Advisor''

High"Impact" Medium"Impact" Low"Impact"

Summit Developers


:

6 3 6

2 2 2 GA

Alameda, Berkeley, Davis, Eureka, Fresno,

:

2

Jira Pivotal Bugzilla

Summit Developers


Front EndCompilation

AnalysisCore Analysis

Coverity Connect (CC)Defect Management

1 Gc_rc gc_pbkdf2_sha1 (const char *P, size_t Plen, 2 const char *S, size_t Slen, 3 unsigned int c, 4 char *DK, size_t dkLen) 5 { 6 char U[20] T[20]; 7 unsigned int hlen = 20, u, l, r, i, k; 8 int rc; char *tmp; size_t tmplen 9 10 if (c == 0) 11 return GC_PKCS5_INVALID_ITERATION_COUNT; 12 r = dkLen - (l - 1) * hLen; 13 14 memcpy (tmp, S, Slen);

:

Tests automated and run during build

Tests automated and run during build

Mostly manuallytested

End-to-end (E2E) testsperformed manually

: Coverity Connect End-to-End

CondenTal: For Coverity and Partner use only. Copyright Coverity, Inc., 2013 28

Summit Developers


Coverity Connect

Confidential: For Coverity and Partner use only. Copyright Coverity, Inc., 2013

29

Summit Developers


30

% C

ode

Teste

d

Effort to develop tests

100%

Diminishing return forincreased test effort1

Not all code is testable - unreachable statements - dead code, ...

2

Not all tested code adds equal value to the test - non-critical code - debug code, legacy code - exception handling, ...

3

- - - -

- - - ...

Summit Developers


31

SCM: Git, CVS, Mercurial, Subversion, Perforce, ClearCase, AccuRev, MS TFS

gcov (C/C++) BullseyeC/C++) IBM PureCovC/C++) Corbertura (Java)

.

Summit Developers


3 : 100% ( ,

, )

0"

2"

4"

6"

8"

10"

12"

14"

16"

18"

20"

0"

5"

10"

15"

20"

25"

30"

35"

28*Ap

r*12"

5*May*12

"

12*M

ay*12

"

19*M

ay*12

"

26*M

ay*12

"

2*Jun

*12"

9*Jun

*12"

16*Ju

n*12"

23*Ju

n*12"

30*Ju

n*12"

7*Jul*

12"

14*Ju

l*12"

21*Ju

l*12"

Num

ber'o

f'Bug

s'Fou

nd'

Num

ber'o

f'Tes

ts'fr

om'T

A'

Date'

Test'Advisor'Applica:on'in'Frontend'Project'

Tests"added"through"TA" Bugs"found"by"TA"tests"

29 19

Keil

32

Summit Developers


20112013 Coverity Connect End-to-End

0"

20"

40"

60"

80"

100"

120"

Alameda" Berkeley" Carmel" Davis" Today"

Pers

on'D

ays'

CC'Manual'Tes0ng'Eort'

0"

4"

8"

12"

16"

20"

Alameda" Berkeley" Carmel" Davis" Today"

Pers

on'D

ays'

E2E'Manual'Tes1ng'Eort'

Alameda' Berkeley' Carmel' Davis' Today'Manual'CC'GUI'Tests' 347' 661' 1006' 931' 1194'Automated'CC'GUI''Tests' 0' 2' 44' 220' 403'

0'

250'

500'

750'

1000'

1250'

1500'

1750'

CC"Test"Automa,on"Progress"

Alameda' Berkeley' Carmel' Davis' Today'Manual'E2E'Tests' 300' 370' 175' 143' 60'Automated'E2E'Tests' 0' 120' 1468' 1761' 3466'

0'

500'

1000'

1500'

2000'

2500'

3000'

3500'

4000'

E2E#Test#Automa-on#Progress#

33

Summit Developers


34

0"

0.2"

0.4"

0.6"

0.8"

1"

1.2"

1.4"

1.6"

Alameda" Berkeley" Carmel" Davis"

Nor

mal

ized

+Num

ber+

of+D

efec

ts+

Customer4found+Defects+

Summit Developers


() Coverity Test Advisor

35

Summit Developers


I suggest your Next AcTon!

Summit Developers


M Y R E C O M M E N D N E X T A C T I O N !

37

!

Summit Developers


CoverityQA/SA

Jenkins-CI

Coverity QA/SA/TA

CoverityPolicy

Manager QAVIP

ALM

Coverity (Coverity Connect)

QA

QNX