デブサミ2014【13-B-L】テスト自動化を見直そう!自動化への投資が開発チームをクリエイティブにする(安竹由起夫〔コベリティジャパン〕)...
Preview:
DESCRIPTION
コベリティの静的解析は、数多くの開発現場で導入されテスト工程の手戻り作業を削減してきました。クリーンなコードはテストを円滑に実施していくうえで既に必須条件となっています。では、開発マネージャ、開発組織は次に何を目指すべきでしょうか。 本セッションでは「テスト自動化」に焦点をあて、開発チームの利益を拡大するために、いつ、どこを自動化すべきなのか、自動化作業をサポートするツールを含めご紹介します。
Citation preview
- 1. 13-B-L:
- 2. Coverity Connect 2
- 3. 3 Copyright Coverity, Inc., 2014
- 4. 30% IT H21 4
- 5. Linux Coverity SCAN Report 2013 5
- 6. 6
- 7. : 6 3 6 2 2 2 GA Alameda, Berkeley, Davis, Eureka, Fresno, :
2 Jira PivotalTracker Bugzilla Jenkins-CI Next Release Planning
Next Release Execution Release Execution Release Planning Prev.
Release Execution October December November 7 February January
April March June May August July 6 Weeks Hardening September
- 8. Jenkins Hardware'for'STS'Test'Farm' 250# 200# 183# 150# 215#
156# 125# 100# 50# 201# 65# 66# 69# 74# 70# 81# 71# 84# 100# 77#
88# 128# 88# 96# 108# 126# 127# 0# Pre/2011# Q1#2011# Q2#2011#
Q3#2011# Q4#2011# Q1#2012# Q2#2012# Q3#2012# Q4#2012# Q1#2013#
Number#of#Physical#Machines# 8
Total#Number#of#Machines#incl.#Virtual#Machines# Today#
- 9. 9 Copyright Coverity, Inc., 2014
- 10. Coverity - QA QA 10 6 QA 2-3 10
- 11. QA + () 2011 2/3 QA QA 11
- 12. 1 2.5 2011 12 Personnel(Transi,on(from((
Eastern(Europe(to(Calgary( 14" 12" 12 10" 8" 6" 4" 2" 0" Q3"2011"
Q4"2011" Q1"2012" Eastern"Europe" Q2"2012" Calgary"" Q3"2012"
- 13. 13 Copyright Coverity, Inc., 2014
- 14. UI
- 15. % Code Tested 100% 1 Diminishing return for increased test
effort - ... 3 Effort to develop tests 15 2 Not all code is
testable - unreachable statements - - - dead code, ... Not all
tested code adds - equal value to the test - - non-critical code -
- - debug code, legacy code - exception handling, ...
- 16. 16
- 17. 17 Copyright Coverity, Inc., 2014
- 18. 1,372,376 1,149,036 (83.7% ), 223,340 145 851 Untested LOC
250,000 2500 223,340 1979 200,000 2000 1743 150,000 1500 100,000
1000 404 50,000 - 9,636 7,934 145 851 Total Since 5.3 Since 5.4
Since 5.4.1 Untested LOC 223,340 9,636 7,934 851 Untested LOC (%)
100.0% 2.0% 1.6% 0.2% 1979 1743 404 145 # Incomplete Tested Files
500 0 # Incomplete Tested Files Focus tes6ng on latest changes #
Incomplete Tested Files : 18 Untested LOC 0.2% 7%
- 19. 3 : 100% ( , , )
Test'Advisor'Applica:on'in'Frontend'Project' 20" 18" 30" 16" 25"
14" 12" 20" 10" 15" 8" 6" 10" 4" 5" 2" 0" 28 *A pr *1 2" 5* M ay *1
2" 12 *M ay *1 2" 19 *M ay *1 2" 26 *M ay *1 2" 2* Ju n* 12 " 9* Ju
n* 12 " 16 *Ju n* 12 " 23 *Ju n* 12 " 30 *Ju n* 12 " 7* Ju l*1 2"
14 *Ju l*1 2" 21 *Ju l*1 2" 0" Date' 1 9 Tests"added"through"TA"
Bugs"found"by"TA"tests" Number'of'Bugs'Found'
Number'of'Tests'from'TA' 35" 29 19 Keil
- 20. 20 Copyright Coverity, Inc., 2014
- 21. Address 21
- 22. Quality Advisor, Security Advisor
Defects'Addressed'by'Coverity'Quality/Security'Advisor''
Number'of'Defects' 400" 350" 300" 250" 200" 150" 100" 50" 0"
Alameda" Berkeley" High"Impact" Carmel" Medium"Impact" Davis"
Low"Impact" Eureka*" 22
- 23. Customer4found+Defects+ Normalized+Number+ of+Defects+ 1.6"
1.4" 1.2" 1" 0.8" 0.6" 0.4" 0.2" 0" Alameda" Berkeley" Carmel"
Davis" 23
- 24. 01 01001011 0101101011001 01101011000011 010100101101
01011001 25
- 25. 26
- 26. Make every developer a rock star ! 27
- 27. 28
- 28. OSS 29
- 29. 2013 # of Projects 1200 1000 800 600 # of Projects 400 200
0 Jan. 30 Feb. March April May June July Aug. Sept. Oct. Nov.
Dec.
- 30. Apache Hadoop (since Aug. 2013) LOC: 845,000 31
- 31. Coverity Development Testing Platform | | SDLC Policy
Manager Dynamic Analysis IDE Coverity Connect Architecture Analysis
Quality Advisor Security Advisor Test Advisor FindBugs | FxCop /
SCM Coverity SAVE Static Analysis Verication Engine | 32 ALM HP |
IBM
- 32. OWASP Top 10 OWASP Top 10 Coverity 7.0 1: 2: 3: (XSS) 4: 5:
6: 7: 8: (CSRF) 9: 10: Coming soon NA