View
917
Download
1
Category
Preview:
DESCRIPTION
Em primeiro lugar, teste seu projeto. Depois aprenda como NÃO testá-lo. Palestra cheia de exemplos de testes que hoje em dia não escrevemos mais, para nos lembrar dos testes que temos continuar automatizando. Dev Day 2014
Citation preview
Como NÃO testar SW?
uma palestra da
@freire_da_silva
• Director of Product Safety @IndustrialLogic
• Agil desde 2001, lean desde a 1ª série
• Founder: AgilCoop, AgilBits e ).(
• Mestrado na USP: “Reflexões sobre o Ensino de Metodologias Ágeis na Academia, Indústria e Governo”
• Gosto de nadar e construir casas
• Experiência e sucesso com muitos amigos: alex@industriallogic.com
Você TESTA?
TODO SW TEM BUG?
TODO SW TEM BUG!
/1947
1999: !Imperial instead of metric units !cause Mars Climate Orbiter…
to disintegrate!
if((err=SSLFreeBuffer(&hashCtx))!=0) goto fail; if((err=ReadyHash(&SHA1,&hashCtx))!=0) goto fail; if((err=SHA1.update(&hashCtx,&rand))!=0) goto fail; goto fail; if((err=SHA1.update(&hashCtx,&sign))!=0) goto fail; if((err=SHA1.final(&hashCtx,&hsOut))!=0) goto fail; err = sslRawVerify(…);
sslKeyExchange.c
if((err=SSLFreeBuffer(&hashCtx))!=0) goto fail; if((err=ReadyHash(&SHA1,&hashCtx))!=0) goto fail; if((err=SHA1.update(&hashCtx,&rand))!=0) goto fail; goto fail; if((err=SHA1.update(&hashCtx,&sign))!=0) goto fail; if((err=SHA1.final(&hashCtx,&hsOut))!=0) goto fail; err = sslRawVerify(…);
sslKeyExchange.c
/* Read type & payload length first */
hbtype = *p++; n2s(p,payload); p1 = p; !/* Enter response type, length & copy payload */
*bp++ = TLS1_HB_RESPONSE; s2n(payload, bp); memcpy(bp, p1, payload);
/* Read type & payload length first */
hbtype = *p++; n2s(p,payload); if (1+2+payload+16 > s->s3->rrec.length) return 0; /* silently discard per RFC 6520 sec. 4 */ p1 = p; !/* Enter response type, length & copy payload */
*bp++ = TLS1_HB_RESPONSE; s2n(payload, bp); memcpy(bp, p1, payload);
Você AUTOMATIZA SEUS TESTES?
?
#1e5d91
#1e5d91
1describe “test” do 2 it “should do nothing” do 3 fail 4 end 5 end
1describe “test” do 2 it “should do nothing” do 3 fail 4 end 5 end
1describe “test” do 2 it “should do nothing” do 3 true 4 end 5 end
1describe “test” do 2 it “should do nothing” do 3 true 4 end 5 end ?
NON-FUNCTIONAL REQUIREMENTS
SCALABILITY
RELIABILITY
SECURITYUSABILITY
ACCESSIBILITY
MAINTAINABILITY
#1e5d91
por James Reason
RISCOS
FALHAS
Você APAGA seus testes?
?
100% COVERED CODE.
OF CODE YOU COVERLINE
60@Test 61public void setPrice() { 62 Item item = new 63 BasicItem(0.0, true); 64 65 assertEquals(0.0, 66 item.getPrice()); 67 68 //set new price 69 item.setPrice(1.0); 70 assertEquals(1.0, 71 item.getPrice()); 72}
74@Test 75public void isImported_true(){ 76 Item item = new 77 BasicItem(0.0, true); 78 79 assertTrue(item.isImported()); 80}
74@Test 75public void isImported_false(){ 76 Item item = new 77 BasicItem(0.0, false); 78 79 assertFalse(item.isImported()); 80}
28public Double getPrice(){ 25 return price; 26} 27 28public boolean isImported(){ 29 return imported; 30}
WHY DO WE TEST?
1while true do 2 print “vou parar?” 3end
!Murphy’s law
PAIN
FLOW
Cost of Quality Assurance (QA)
Analysis Design Coding Unit Tests Acceptance Tests Production
freqüencia de Uso Das Funcionalidades
RARO 19%
Nunca 45%
Algumas VEZES 16%
Frequente 13%
Sempre 7%
Pair Programming
?
feedback vs
noise
Speed Robustness
non-deterministic
flakycode
Test failed 21 -mes locally & 36 -mes in dev build in the last 6 months.
@google If they fail we simply run flaky tests 3x (and keep statistics). Developer time is much more valuable than server time.
1@Test 2public void errorExerciseUploadTooBig { 3 goToExerciseIntroPage(); 4 goToNextPage(); 5 6 type(“labArchive”, path() + TOO_BIG); 7 pause(1000L); 8 assertText(“error: Upload too big!”); 9}
sleep() demais
How much QUALITY is ENOUGH?
Bugs/1KLOC
0
1
3
4
5
Indústria Nasa
0,004
Cost($/LOC)
0
225
450
675
900
Indústria Nasa
5
850
It will only run once
VS
<- costumer facing
back office ->
@martinfowler you’re doing enough testing if the following is true: ■You rarely get bugs that escape
into production ■You are rarely hesitant to change
some code for fear it will cause production bugs
@joshuakerievsky Test first/after misses the point that TDD is more about emergent design than it is about testing. Do you practice emergent design?
@kentbeck If I don’t typically make a mistake(...), I don’t test for it. !
Wish there were more examples of “what not to test and how not to test”.
2def create_name(fname, lname): 3 if not isInstance(fname, basestring): 4 raise TypeError(“fname must be a String”) 5 if not isInstance(lname, basestring): 6 raise TypeError(“lname must be a String”) 7 name = fname + lname
1 require ‘spec_helper‘ 2 describe Candidate do 3 context ‘associations‘ do 4 it { should have_many(:proposals) } 5 end 6 7 context ‘validations‘ do 8 it { should validate_presence_of :name } 9 10 it { should ensure_lenght_of(:phone). 11 is_at_least(7). 12 is_at_most(14) 13 } 14 15 it { should_not 16 allow_value(‘blah‘).for(:site) } 17 18 it { should 19 allow_value(‘http://www.blah.com‘) 20 .for(:site) } 21 end 22end
1 require ‘valid_url‘ 2 class Candidate < ActiveRecord::Base 3 has_many :proposals 4 5 validates :name, presence: true 6 7 validates :phone, :length => 8 {:in => 8..14}, 9 :allow_blank 10 11 validates :site, :url => true, 12 :allow_nil => true 13 14end
1 require ‘spec_helper‘ 2 describe Candidates do 3 4 let(:candidate) {double ‘candidate‘} 5 6 before :each do 7 Candidate 8 .should_receive(:find) 9 .with(1).and_return(candidate) 10 end 11 12 it ‘should find the candidate‘ do 13 c = Candidate.find(1) 14 c.should eql candidate 15 end 16end
1Feature: Create proposal 2 As a candidate 3 I want to post my proposals 4 So that voters can evaluate them 5 6 Scenario: 7 Given I am logged in 8 And I am posting a proposal 9 When 10 I fill all fields of the proposal 11 Then 12 I should see a success message
1Scenario: Client sees tooltip for plan 2 Given 3 I select the ‘light‘ plan 4 When 5 I mouse over ‘tooltip‘ 6 Then 7 I should see ‘tooltip‘ content 8 And 9 I mouse out ‘tooltip‘ 10 Then 11 I should not see ‘tooltip‘ content
1require ‘spec_helper‘ 2describe ShoppingCart do 3 4 let(:user) {double ‘user‘} 5 let(:product) {double ‘product‘} 6 7 before :each do 8 Authenticator.should_receive(:auth) 9 .and_return(true) 10 end 11 12 it ‘should addProduct & getTotal‘ do 13 Authenticator.auth(user) 14 cart = ShoppingCart.new 15 cart.add_product(product) 16 cart.get_total 17 end 18end
50@Test 51public void changeMarks() { 52 bot.leftClickAt(view, 53 800, 508); 54 addMarkAt(‘drama’, 1); 55 56 bot.leftClickAt(view, 57 900, 508); 58 addMarkAt(‘act’, 3); 59 60 bot.verifyTooltipAt(30, 190); 61}
PIXEL NAZI
1 require ‘spec_helper‘ 2 describe AddressController do 3 4 it ‘should calculate shipping‘ do 5 get :shipping, :zipcode => ‘90210‘ 6 assigns(:shipping).should == ‘8.2‘ 7 end 8 9 end
external dependencies:
DRY
WET
fixtures considered harmful?
JAVASCRIPT?
module('MultiSelectQuizTests',{ setup: function() { var container = document.getElementById("qunit-fixture"); var question = "Which foods are Mexican?"; var answers = [ { 'answerText': 'Tacos', 'value': true }, { 'answerText': 'Sushi', 'value': false } ]; ! this.myQuiz = new MultiSelectQuiz ( container, question, answers ); }, }); !test( "One correct", function() { checkRadio(0, true); checkRadio(1, true); deepEqual(this.myQuiz.grade(), 1, "just 1"); });
CSS?
it(‘centers logo at top of page’, function() { expect(isConTenteCenteredInPage(logo)).toBe(true); expect(elementPicelsFromTopOfPage(logo)).toBe(12); expect(fontSizeOf(logo)).toBe(22); expect(textColorOf(logo)).toBe(WHITE); });
Students(Can’t(Access(Service(
New(Produc5on(Server(
Ok(To(Toggle((Test(Passes(
Immune(System(Fails(
No(Auto?(Rollback(
No(SMS(
No(Policy(
Non?Standard(Tomcat(Runner(
Nginx(Points(To(Down(Service(
Non?Standard(Java(Version( JRE(Crash(
No(Policy(
And$
And$ And$ And$
análise de uma falha
Test journeys
NÃO É BUG É FEATURE
JAVASCRIPT
80% só até amanhã!código de desconto: DEVDAY2014
HTTP://industriallogic.com/shop
PYTHON
Recommended