소프트웨어 QA 및 테스팅 트렌드

© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

소프트웨어 QA 및 테스팅 트렌드Key Findings from World Quality Report 2014-15김기배 차장 ([email protected])Business Development ManagerHP Software Asia Pacific and Japan

mailto:[email protected]

목차

World Quality Report 개요 소프트웨어 QA 및 테스팅 트렌드 Recommendation #1 Recommendation #2 Summary

World Quality Report 개요


World Quality ReportPublished since 2009, the World Quality Report (WQR) is recognized as industry’s largest research study and most comprehensive source of market analysis and commentary on the state of enterprise application quality and testing practices

1,543 respondents(from organizations with over 1,000 employees only)


World Quality Report 2014-15: The Research

Company Size

# of respondents

1,000 – 4,999 4075,000 – 10.000 543

10,000 + 593





소프트웨어 QA 및 테스팅 트렌드


Current Trends in QA and TestingQA and Testing Budgets Continue to Rise, driven by Digital Transformation initiativesBudget

Centralization

SMAC Technologies

Agile & DevOps

Automation


Current Trends in QA and TestingBudget : QA and Testing Budgets Continue to Rise

모든 업계에 걸쳐 QA 예산의 지속적인 증가가 나타나고 있으며 , High-Tech 업계의 경우 IT 예산의 40% 를 QA 에 배정하는 기업이 전체 응답자의 24% 로 조사됨

신규 기술을 요구하는 시장의 압력과 상대적으로 짧은 딜리버리 일정이 QA 예산 증가로 나타남



신규 개발 업무에 대한 QA 및 테스팅 예산 배정이 꾸준히 증가하고 있으며 , 2014 년엔 QA 예산의 약 52% 가 신규 개발 업무에 배정되었음

유지보수 업무에 대한 지속적인 QA 및 테스팅 방안에 대한 대책 마련 필요



업무 복잡도의 증가로 현업을 포함한 이해관계자와 QA 조직의 협업이 요구되어 Onshore 형태의 조직 구성이 높은 효율성을 보임 (Offshore Testing 의 비중이 28% 21% 로 감소 )

신규 기술의 도입으로 인해 특정 기술에 대해 전문성을 가진 QA 및 테스트 인력 구성이 요구됨



QA 및 테스팅 활동에 대한 개선이 필요

전체 응답자의 64%


Current Trends in QA and TestingQA and Testing Budgets Continue to Rise, driven by Digital Transformation initiativesBudgetMany businesses prefer to organize QA & Testing functions with a combination of centralized and decentralized approaches

Centralization

SMAC Technologies

Agile & DevOps

Automation


Current Trends in QA and TestingCentralization : Centralization with Decentralized Approach

중앙 집중 형태의 (Centralized) 테스트 조직으로 구성이 변화되어 왔으며 , 조직의 성숙도가 높아지고 시간이 흐르면서 하이브리드 형태의 조직 구성이 선호되고 있음



Testing Center of Excellence (TCoE) 운영이 지속적으로 증가하고 있음 업계 및 업무에 특화된 전문성 도구 및 프로세스의 표준화 지속적인 경험 축적 및 개선

TCoE 운영을 통해 테스트 조직에 대한 ROI 평가가 수치에 기반하여 이루어짐



모든 조직 혹은 업무가 크라우드 소싱 기반 테스트에 적합하지는 않으나 , 다양한 지역 , 문화권 , 환경에 대한 사용자 경험 측면에서의 피드백을 받을 수 있는 장점을 가지고 조금씩 확산 중



아직도 많은 조직에서 애플리케이션 개발 주기의 후반에 테스팅 및 QA 활동이 이루어지고 있음



CentralizationSecurity, performance and customer experience testing represent increasingly important focus areas for QA & Testing

SMAC Technologies

Agile & DevOps

Automation


Current Trends in QA and TestingSMAC Technologies : Increases in focusing Customer Experiences



모바일 테스팅에 대한 경험이 축적됨에 따라 기존 대비 모바일 테스팅에 어려움을 겪던 부분에 대한 이슈들이 해소되고 있는 추세임



빅데이터 및 분석 기술을 이용한 애플리케이션 테스팅 관련하여 새로운 이슈들이 발생하고 있음 관련 지식 부족 테스팅 프레임워크 / 도구의 부재 수집한 데이터에 대한 검증이 어려움



소셜 미디어의 활성화와 함께 QA 및 테스팅 팀은 애플리케이션에 대한 시장의 피드백을 수집하는 채널로 소셜 미디어를 활용하고 있음



모바일 , 클라우드와 같은 새로운 플랫폼의 도입으로 인해 보안 테스트에 대한 중요도가 높아지고 있으며 기능과 성능에 대한 검증도 지속적으로

이루어지고 있음




SMAC Technologies

Organizations require fast and responsive Testing aligned with agile development methods including focus on DevOps

Agile & DevOps

Automation


Current Trends in QA and TestingAgile & DevOps : Testing aligned with Agile Development Methods

애자일 개발 방법론이 점점 대부분의 조직에서 표준 개발 방식으로 자리잡고 있는 반면 애자일 테스팅 체계는 아직 정립되어 있지 않음



애자일 개발 방식에 맞는 테스팅 접근 법과 전문 인력의 부재로 인해 많은 조직이 애자일에 대한 대응을 어려워 하고 있음






SMAC Technologies


Agile & DevOps

Many organizations’ maturity in the field of test environment, data management and test automation is increasing

Automation


Current Trends in QA and TestingAutomation : Organizations’ Test Automation Maturity is Increasing

많은 조직이 테스트 환경 준비 및 관리에 대한 어려움을 지속적으로 겪고 있음

클라우드의 도입으로 인해 테스트 환경을 클라우드로 전환하는 조직이 늘고 있으며 ( 약 32%), 이를 통해 테스트 비용 절감 효과를 보고 있음


Current Trends in QA and TestingAutomation : Organizations’ Test Automation Maturity is Increasing

변화하는 애플리케이션 환경을 지원하는 다양한 테스트 자동화 도구의 등장과 조직의 테스트 자동화 성숙도 향상으로 인해 테스트 자동화는 지속적으로 고려되고 활용되고 있음




SMAC Technologies


Agile & DevOps

Many organizations’ maturity in the field of test environment, data management and test automation is increasing

Automation

Recommendation #1Shift-Left your Tests


Shift-Left Testing with Virtualization


How Long do you Wait?Wait time Before Service

VirtualizationOn-demand (no wait) 0%

Seconds to minutes 0%30 minutes to 2 hours 0%

4 to 7 hours 2%1 day 1%2 days 1%3 days 9%4 days 3%1 week 8%2 weeks 15%3 weeks 27%1 month 14%2 months 10%3 months 5%4 to 6 months 3%Never (no access ever) 2%Source: voke Market SnapshotTM Report: Service Virtualization – January 2015

81% 84%QA work delayed while

waiting

32 daysAverage wait time

76% Wait at least 2weeks or

more on systems

Development work delayed while waiting


Without virtualization, testing is incomplete

DATA CENTER

Local Virtual Users• Load generators• Local switch

Application under test

Hi-Speed LANHi-Speed WiFiTablets

Smartphones

LOCAL TEST ENVIRONMENT

Branch office Residential

Mobile anywhere

Road warrior

Mobile workforce

SITES

DEVICES

FIREWALL

NETWORK

APPSBusiness systems

Online B2C

Mobile Corporate

Mobile Consume

r

Mobile Consume

r

Reality is rarely local. Why should your test be?


What is Lifecycle Virtualization?

Mainframe

Storage

DB servers

Web services

App servers

Web servers

Load balancers

Mobile components

Major ISP

3rd party cloud

services

Content delivery networks

Mobile carriers &

ISPsBrowsers & devices

Customers

Distributedservices

Employees

User Virtualization

UV

UV

UV

Service Virtualization

SV

SV

SV SV SV

SV

SV

SV

Network Virtualization

NV

NV

NVNV

NV

NV

NV

NV

NV

NV

NV

NV

NV

NV

Data Virtualization

DV

DVDV

DV

DV

DV

DV

DV

DV

DV

Understanding the complexity of composite applications, and challenges


Service Virtualization

Application

Backend

Mainframe

CRM

3rd PartyPayment

RESTSVVirtual Claim Servic

e

SimulatedTransactions

Development/Design Documentation


Performance Insights

Network Virtualization

Application

Backend

Mainframe

CRM

3rd PartyPayment

SVVirtual

MF

VirtualCRM

VirtualPay

Mobile App

Desktop App

Web App

NVNetwo

rkCondit.

WiFi

3G4G

VPN

2.5G

21

3

100K Users

Load Generators

Virtual users

Performance Testing Tool


A Real-Life Scenario

Customer login

Add to cart

Checkout

Shipping detail

Delivery tracking

Shopping app(web or mobile)

Purchase

received

A typical online purchase has predictable landmarks in its process

But… the road from beginning to end is hardly a direct route

Tracking the course of an online purchase




Database activity

Security validation

Inventory query

Database activity

3rd party ship service

Logistical services

Acc’t info

3rd party database

3rd party reporting

Purchase

receivedAdd to cart

Shipping detail

Delivery tracking

Customer login

Security validation

3rd party payment services

Account authorization

Checkout

But… the road from beginning to end is hardly a direct route





Database activity

Security validation

Inventory query

Database activity

3rd party ship service

Logistical services

Acc’t info

3rd party database

3rd party reporting

Purchase

receivedAdd to cart

Shipping detail

Delivery tracking

Customer login

Security validation



Checkout

Network activity and external services are completely integral to this “simple” transaction

N N

N

N

N

N

NN

N

N

N

NN

NN

N

N

N

NS

S S

S

S

S

S

SS

SS



Wait for SAP availibility

Coordinate remote LG for

the test

Pilot release modules 1 and

2Field test

Lifecycle Virtualization delivers velocity

Dev Module 1

Dev Module 2

Test 1 / 2 Perf Test 1 / 2 Release

Wait for Module 1 to be complete

Wait for Module 1 and 2to be complete

A typical application delivery schedule involves a lot of delays…

using Module 1 virtual svcs

with real world conditions

Monitor User Experience

Virtualization enables faster releases and higher quality

Accelerating the Process


Lifecycle Virtualization Results

400+% ROI

50%reduction in application transaction response time

10Ximprovements in mobile transaction times

90

50-90%reduction in wait time

5Xdecreased effort to test

Less than

30%faster application delivery

45%increase in test coverage and cycles in same time

EFFICIENCYPERFORMANCE VALUE

day break even

60%cost savings due tofaster simulation of services

Client Results are Unbeatable

Recommendation #2Measure Real User Experiences


Deliver amazing user experiences

MODERN APPLICATION DEVELOPMENT

Reduce costs

Increase customer attraction/retention

Increase the value of your brand

Get to market faster

The new style of ITA data explosion in 2015…

* Source: “Enterprise Mobile Facts You Need to Know in 2015” by App Data Room.http://appdataroom.com/enterprise-mobile-facts-need-know-2015/

Mobile

Cloud

45%of mobile enterprise app initiatives

will be delayed or go over budget due to infrastructure difficulties

25% IT software budget dedicated to

mobile app development,deployment and management

by 2017

35%of large enterprises will use

mobile app development platforms

1 in 5of these will be specifically

for enterprise use

381 million tabletswill be sold worldwide

Dev Ops

Agile


MODERN APPLICATION DEVELOPMENT

The new style of ITNetworks

Data center moves

ERP

E2E testingAgile

Composite appsSecurity Services

Lost brand reputation

Lost customersLost revenue

Lost competitive advantageReduce costs

Increase customer attraction/retention

Increase the value of your brand

Get to market faster Complicated, unavailable or expensive testing

Deliver amazing user experiencesPoor quality release

Poor performance

Delayed release schedule


Challenges in Testing Mobile


Database activity

Security validation

Inventory query

Database activity

3rd party ship

service

Logistical services

Acc’t info

3rd party database

3rd party reporting

Security validation




Purchase

receivedAdd to cart

Shipping detail

Delivery tracking

Customer login Checkou

t

Welcome to the JungleHow does it look on the

screen?

What’s happening with

CPU / MEM?

OS’s?

Browsers?

What devices?

Do I havethe right

transactions?

Where are my users coming

from?

What network

connection?

What happens offline or

unavailable?

Was the experience

within brand?


Social media for the feedback?


Measure Real User Experiences

Devices and Operating Systems

Application Stability

Crash Analytics

User Flows

Resource Consumption


Measure Focus Improve

The winning strategy for a 5-star app!


What customers are saying about HP AppPulse Mobile

“HP AppPulse Mobile has already given us valuable information in QA environments. The

tool UI was self-explanatory with

necessary graphs and statistics.”

IT specialist, Standard Bank of South AfricaTVID: 369-806-BA5

“Using the FunDex scoring system in the new HP AppPulse Mobile will allow our team to quickly get to problematic areas and improve the adoption of our app.”Executive, Globe TestingTVID: 50D-DOB-434

“86% of surveyed IT organizations would recommend HP AppPulse Mobile because it captures application crash details along with the mobile app user action that caused it.”Survey of 7 users of HP software application performance managementTVID: C2C-C86-A0A

“HP AppPulse Mobile assisted us to identify performance concerns early in development, saving us costs, and is now part of our build process to improve overall quality.”IT manager, large enterprise banking companyTVID: 50D-DOB-434

Summary


SummaryBudget

Centralization

SMAC Technologies

Agile & DevOps

Automation

Shift-Left Testing

Real UX Measurement

Deliver Application Faster without

Compromising Quality

56

Thank You김기배 차장 ([email protected])Business Development ManagerHP Software Asia Pacific and Japan