안드로이드 앱의 성능 테스트 ,자동화가 가능할까 ?
이경민 / LG 전자
About Me
About Me
Android Platform
About Me
Android Platform
SDK
About Me
Android Platform
SDET SDK
About Me
webOSAndroid Platform
SDET SDK
About Me
webOSAndroid Platform
SDET
Test Automation
SDK
About Me
kandroid.org
webOSAndroid Platform
SDET
Test Automation
SDK
About Me
snailee
kandroid.org
webOSAndroid Platform
SDET
Test Automation
SDK
CONTENTS
Why Automated Performance Testing?
How to Automate Performance Testing?
- Historical Analysis w/ kmemtracer
- Measuring Performance
- Controlling UI
Why Automated Performance Testing?
If you can’t mea-sure it,
you can’t manage it. Peter F. Drucker
(1909~2005)
If you can’t mea-sure it,
you can’t manage it. Peter F. Drucker
(1909~2005)
If you can’t automate the measurement of it,
it is difficult to im-prove it.
http://developer.android.com/about/dashboards/index.html
Plat-formVer-sions
Diversity or Fragmentation
http://developer.android.com/about/dashboards/index.html
Plat-formVer-sions
Screen Sizesand Densi-ties
Diversity or Fragmentation
http://www.mactrast.com/2012/05/visualized-the-truth-about-an-droid-fragmentation/
How to Automate Performance Testing?
- Historical Analysis w/ kmemtracer
- Measuring Performance
- Controlling UI
Why Historical Analysis?
Source: Memory and Performance at the 11th Korea Android Conference
Why Historical Analysis?
Source: Memory and Performance at the 11th Korea Android Conference
Why Historical Analysis?
Source: Memory and Performance at the 11th Korea Android Conference
Activity Naviga-tion &
Lifecycle
How to Track Activity Navigation & Lifecycle?
How to Track Activity Navigation & Lifecycle?
Override onCreate, on-Start, …
of your Activity classes
How to Track Activity Navigation & Lifecycle?
Override onCreate, on-Start, …
of your Activity classesModify android.ap-
p.Activity in the framework
How to Track Activity Navigation & Lifecycle?
Override onCreate, on-Start, …
of your Activity classesModify android.ap-
p.Activity in the framework
Is there a way that does not
require the modifica-tion of
an application or framework?
How to Track Activity Navigation & Lifecycle?
Override onCreate, on-Start, …
of your Activity classesModify android.ap-
p.Activity in the framework
Is there a way that does not
require the modifica-tion of
an application or framework?
Instrumentation
Android instrumentation is a set of control methods or "hooks" in the Android system. These hooks control an Android compo-nent independently of its normal lifecycle.They also control how Android loads ap-plications.
(http://developer.android.com/tools/testing/testing_android.html)
Historical Analysis w/ kmemtracer
TestPackage
Application Package
android.test.Instrumenta-
tionTestRunner
Con-trol
Historical Analysis w/ kmemtracerhttps://github.com/snailee/kmem-
tracer-libs
TestPackage
Application Package
android.test.Instrumenta-
tionTestRunner
org.kandroid.memtrac-er.Memory
Instrumentation
Con-trol
Track
TracePackage
android.app.Instrumentation
void callActivityOnCreate(Activity, Bundle)void callActivityOnDestroy(Activity)void callActivityOnNewIntent(Activity, Intent)void callActivityOnPause(Activity)void callActivityOnPostCreate(Activity, Bundle)void callActivityOnRestart(Activity)void callActivityOnRestoreInstanceState(Activity, Bundle)void callActivityOnResume(Activity)void callActivityOnSaveInstanceState(Activity, Bundle)void callActivityOnStart(Activity)void callActivityOnStop(Activity)void callActivityOnUserLeaving(Activity)void callApplicationOnCreate(Application)
android.app.Instrumentation
void callActivityOnCreate(Activity, Bundle)void callActivityOnDestroy(Activity)void callActivityOnNewIntent(Activity, Intent)void callActivityOnPause(Activity)void callActivityOnPostCreate(Activity, Bundle)void callActivityOnRestart(Activity)void callActivityOnRestoreInstanceState(Activity, Bundle)void callActivityOnResume(Activity)void callActivityOnSaveInstanceState(Activity, Bundle)void callActivityOnStart(Activity)void callActivityOnStop(Activity)void callActivityOnUserLeaving(Activity)void callApplicationOnCreate(Application)
Call for Con-trol
android.app.Instrumentation
void callActivityOnCreate(Activity, Bundle)void callActivityOnDestroy(Activity)void callActivityOnNewIntent(Activity, Intent)void callActivityOnPause(Activity)void callActivityOnPostCreate(Activity, Bundle)void callActivityOnRestart(Activity)void callActivityOnRestoreInstanceState(Activity, Bundle)void callActivityOnResume(Activity)void callActivityOnSaveInstanceState(Activity, Bundle)void callActivityOnStart(Activity)void callActivityOnStop(Activity)void callActivityOnUserLeaving(Activity)void callApplicationOnCreate(Application)
Call for Con-trol
Override for Track
org.kandroid.memtracer.MemoryInstrumentation
public class MemoryInstrumentation extends Instrumentation { private static final String ARGUMENT_INSTRUMENT_CLASS = "class"; private MemoryTracer mMemoryTracer; private String mMainActivityName; ... @Override public void onCreate(Bundle arguments) { mMainActivityName = arguments.getString(ARGUMENT_INSTRUMENT_CLASS); mMemoryTracer = createMemoryTracer(); mMemoryTracer.startTracing(getTargetContext().getPackageName()); start(); }
@Override public void callActivityOnCreate(Activity activity, Bundle ici-cle) { String tag = activity.getLocalClassName()+"-OnCreate"; Bundle snapshot = mMemoryTracer.addSnapshot(tag); super.callActivityOnCreate(activity, icicle); }
How to Use kmemtracer?
How to Use kmemtracer?
1. Create an Android Test Project for the trace package.
How to Use kmemtracer?
1. Create an Android Test Project for the trace package.2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory.
How to Use kmemtracer?
1. Create an Android Test Project for the trace package.2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory.3. Edit the <instrumentation> element in the Manifest file.
<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.example.android.apis.test" android:versionCode="1" android:versionName="1.0">
<instrumentation android:name="org.kandroid.memtracer.MemoryInstrumentation" android:targetPackage="com.example.android.apis" /> <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/> <application android:label="@string/app_name" android:icon="@drawable/ic_launcher“> <uses-library android:theme="android.text.runner“> </application></manifest>
How to Use kmemtracer?
4. Install the trace package as well as the applica-tion package.
$ adb install –r ApiDemos.apk$ adb install –r ApiDemosTest.apk
How to Use kmemtracer?
4. Install the trace package as well as the applica-tion package.
5. Start the instrumentation with ‘am instrument’ in the shell.
$ adb install –r ApiDemos.apk$ adb install –r ApiDemosTest.apk
$ adb shell am instrument -e class \> com.example.android.apis.ApiDemos \> com.example.android.apis.tests/\ > org.kandroid.memtracer.MemoryInstrumentation
How to Use kmemtracer?
4. Install the trace package as well as the applica-tion package.
5. Start the instrumentation with ‘am instrument’ in the shell.
6. Do interact with the application.
$ adb install –r ApiDemos.apk$ adb install –r ApiDemosTest.apk
$ adb shell am instrument -e class \> com.example.android.apis.ApiDemos \> com.example.android.apis.tests/\ > org.kandroid.memtracer.MemoryInstrumentation
How to Use kmemtracer?
4. Install the trace package as well as the applica-tion package.
5. Start the instrumentation with ‘am instrument’ in the shell.
6. Do interact with the application.7. Pull the trace file ‘kmemtrace.csv’ from /sdcard/kmemtracer.
$ adb install –r ApiDemos.apk$ adb install –r ApiDemosTest.apk
$ adb shell am instrument -e class \> com.example.android.apis.ApiDemos \> com.example.android.apis.tests/\ > org.kandroid.memtracer.MemoryInstrumentation
$ adb pull /sdcard/kmemtracer/kmemtrace.csv .
How to Use kmemtracer?
8. Open the trace file with Excel and create charts.
How to Automate Performance Testing?
- Historical Analysis w/ kmemtracer
- Measuring Performance
- Controlling UI
What to Measure?
Rendering (or GUI) Performance Frame Rate (FPS), Frame Time , Jankiness, Input Latency, …
Resource Consumption Memory, CPU, I/O (Disk, Network), Battery, …
Network Performance Response time, Throughput, …
…
What to Measure?
Rendering (or GUI) Performance Frame Rate (FPS), Frame Time , Jankiness, Input Latency, …
Resource Consumption Memory, CPU, I/O (Disk, Network), Battery, …
Network Performance Response time, Throughput, …
…
How to Measure Resource Consumption?
How to Measure Resource Consumption?
API cmds
How to Measure Resource Consumption?
API cmds
PublicInter-
nal
How to Measure Resource Consumption?
API cmds
PublicInter-
nalAn-
droidLinux
How to Measure Resource Consumption?
API cmds
PublicInter-
nalAn-
droidLinux
Easy
Flexi-ble
Measuring Memory Usage
Measuring Memory Usage
Public
android.app.ActivityManager.Memo-ryInfoandroid.os.Debugandroid.os.Debug.MemoryInfo
Measuring Memory Usage
Public Internal
android.app.ActivityManager.Memo-ryInfoandroid.os.Debugandroid.os.Debug.MemoryInfo
dalvik.system.VMDebug
Measuring Memory Usage
Public Internal Android
android.app.ActivityManager.Memo-ryInfoandroid.os.Debugandroid.os.Debug.MemoryInfo
adb shell dumpsys meminfo [pid|pname]adb shell procrank
dalvik.system.VMDebug
Measuring Memory Usage
Public Internal Android Linux
android.app.ActivityManager.Memo-ryInfoandroid.os.Debugandroid.os.Debug.MemoryInfo
adb shell dumpsys meminfo [pid|pname]adb shell procrank
adb shell cat /proc/meminfoadb shell cat /proc/<pid>/[s]mapsadb shell cat /proc/vmstatadb shell topadb shell ps
dalvik.system.VMDebug
Measuring CPU Usage
Measuring CPU Usage
Public
android.os.Processandroid.os.Debug.InstructionCount
Measuring CPU Usage
Public Internal
android.os.Processandroid.os.Debug.InstructionCount
com.android.internal.os.Pro-cessStats
Measuring CPU Usage
Public Internal Android
android.os.Processandroid.os.Debug.InstructionCount
adb shell dumpsys cpuinfo
com.android.internal.os.Pro-cessStats
Measuring CPU Usage
Public Internal Android Linux
android.os.Processandroid.os.Debug.InstructionCount
adb shell dumpsys cpuinfo
adb shell cat /proc/statadb shell top
com.android.internal.os.Pro-cessStats
Measuring I/O Usage
Measuring I/O Usage
Public
android.net.TrafficStats
Measuring I/O Usage
Public Internal
android.net.TrafficStats
android.net.NetworkStatsandroid.net.NetworkStatsHistory
Measuring I/O Usage
Public Internal Android
android.net.TrafficStats
adb shell dumpsys netstatsadb shell dumpsys diskstats
android.net.NetworkStatsandroid.net.NetworkStatsHistory
Measuring I/O Usage
Public Internal Android Linux
android.net.TrafficStats
adb shell dumpsys netstatsadb shell dumpsys diskstats
/proc/net/*
android.net.NetworkStatsandroid.net.NetworkStatsHistory
Measuring Battery Usage
Measuring Battery Usage
Public
android.os.BatteryManager
Measuring Battery Usage
Public Internal
android.os.BatteryManager
android.os.BatteryStatscom.android.internal.os.PowerProfile
Measuring Battery Usage
Public Internal Android
android.os.BatteryManager
adb shell dumpsys batteryadb shell dumpsys battery-info
android.os.BatteryStatscom.android.internal.os.PowerProfile
Measuring Battery Usage
Public Internal Android Linux
android.os.BatteryManager
adb shell dumpsys batteryadb shell dumpsys battery-info
None (?)
android.os.BatteryStatscom.android.internal.os.PowerProfile
Tracing Performance Snapshots in kmemtracer
public class MemoryInstrumentation extends Instrumentation {
Tracing Performance Snapshots in kmemtracer
public class MemoryInstrumentation extends Instrumentation { ... private MemoryTracer mMemoryTracer; ... @Override public void onCreate(Bundle arguments) { ... mMemoryTracer = createMemoryTracer(); ... } ...
Create an in-stance of
MemoryTracer
Tracing Performance Snapshots in kmemtracer
public class MemoryInstrumentation extends Instrumentation { ... private MemoryTracer mMemoryTracer; ... @Override public void onCreate(Bundle arguments) { ... mMemoryTracer = createMemoryTracer(); ... } ... @Override public void callActivityOnCreate(Activity activity, Bundle icicle) { String tag = activity.getLocalClassName()+"-OnCreate"; Bundle snapshot = mMemoryTracer.addSnapshot(tag); ... } ...}
Create an in-stance of
MemoryTracer
Add performance snapshot
MemoryTracer$addSnapshot
public Bundle addSnapshot(String label) {
MemoryTracer$addSnapshot
public Bundle addSnapshot(String label) { ... long nativeMax = Debug.getNativeHeapSize() / 1024; long nativeAllocated = Debug.getNativeHeapAllocatedSize() / 1024; long nativeFree = Debug.getNativeHeapFreeSize() / 1024; Debug.MemoryInfo memInfo = new Debug.MemoryInfo(); Debug.getMemoryInfo(memInfo); ...
Get usage data
MemoryTracer$addSnapshot
public Bundle addSnapshot(String label) { ... long nativeMax = Debug.getNativeHeapSize() / 1024; long nativeAllocated = Debug.getNativeHeapAllocatedSize() / 1024; long nativeFree = Debug.getNativeHeapFreeSize() / 1024; Debug.MemoryInfo memInfo = new Debug.MemoryInfo(); Debug.getMemoryInfo(memInfo); ... Bundle snapshot = new Bundle(); snapshot.putString(METRIC_KEY_LABEL, label); ... snapshot.putLong(METRIC_KEY_NATIVE_SIZE, nativeMax); snapshot.putLong(METRIC_KEY_NATIVE_ALLOCATED, nativeAllocated); snapshot.putLong(METRIC_KEY_NATIVE_FREE, nativeFree); ...
Get usage data
Save data to
a bundle
MemoryTracer$addSnapshot
public Bundle addSnapshot(String label) { ... long nativeMax = Debug.getNativeHeapSize() / 1024; long nativeAllocated = Debug.getNativeHeapAllocatedSize() / 1024; long nativeFree = Debug.getNativeHeapFreeSize() / 1024; Debug.MemoryInfo memInfo = new Debug.MemoryInfo(); Debug.getMemoryInfo(memInfo); ... Bundle snapshot = new Bundle(); snapshot.putString(METRIC_KEY_LABEL, label); ... snapshot.putLong(METRIC_KEY_NATIVE_SIZE, nativeMax); snapshot.putLong(METRIC_KEY_NATIVE_ALLOCATED, nativeAllocated); snapshot.putLong(METRIC_KEY_NATIVE_FREE, nativeFree); ... if (mResultsWriter != null) { mResultsWriter.writeTraceSnapshot(snapshot); } return snapshot;}
Get usage data
Write date usingRe-
sultsWriter
Save data to
a bundle
MemoryTracerCsvWriter$writeTraceSnapshotpublic class MemoryTraceCsvWriter implements MemoryTracer.ResultsWriter {
MemoryTracerCsvWriter$writeTraceSnapshotpublic class MemoryTraceCsvWriter implements MemoryTracer.ResultsWriter { ... public MemoryTraceCsvWriter(String[] metricKeys) { mMetricKeys = metricKeys; } ...
Create an in-stance
with metrics to print
MemoryTracerCsvWriter$writeTraceSnapshotpublic class MemoryTraceCsvWriter implements MemoryTracer.ResultsWriter { ... public MemoryTraceCsvWriter(String[] metricKeys) { mMetricKeys = metricKeys; } ... private void openTraceFile(String filename) { ... mTraceOut = new PrintStream(new FileOutputStream(traceFile)); ... } ...
Create an in-stance
with metrics to print
Open a trace file in
the sdcard at init
MemoryTracerCsvWriter$writeTraceSnapshotpublic class MemoryTraceCsvWriter implements MemoryTracer.ResultsWriter { ... public MemoryTraceCsvWriter(String[] metricKeys) { mMetricKeys = metricKeys; } ... private void openTraceFile(String filename) { ... mTraceOut = new PrintStream(new FileOutputStream(traceFile)); ... } ... @Override public void writeTraceSnapshot(Bundle snapshot) { PrintStream out = mTraceOut; for (String key : mMetricKeys) { out.print(snapshot.get(key)); out.print(','); } out.println(); out.flush(); } ...}
Create an in-stance
with metrics to print
Write snap-shot to
a line of the file
Open a trace file in
the sdcard at init
To Do’s
To Do’s
Add more tracers CpuTracer IOTracer BatteryTracer
To Do’s
Add more tracers CpuTracer IOTracer BatteryTracer
Make metrics to gather configurable
To Do’s
Add more tracers CpuTracer IOTracer BatteryTracer
Make metrics to gather configurable
Make trace output destination config-urable Trace out to internal file (not on sdcard) Trace out to an analytics server
To Do’s
Add more tracers CpuTracer IOTracer BatteryTracer
Make metrics to gather configurable
Make trace output destination config-urable Trace out to internal file (not on sdcard) Trace out to an analytics server
Welcome contributionshttps://github.com/snailee/
kmemtracer-libs
How to Automate Performance Testing?
- Historical Analysis w/ kmemtracer
- Measuring Performance
- Controlling UI
Automate Controlling UI
TestPackage
Application Package
android.test.Instrumenta-
tionTestRunner
org.kandroid.memtrac-er.Memory
Instrumentation
Con-trol
Track
TracePackage
Automate Controlling UI
TestPackage
Application Package
android.test.Instrumenta-
tionTestRunner
org.kandroid.memtrac-er.Memory
Instrumentation
Con-trol
Track
TracePackage
Automate Controlling UI
TestPackage
Application Package
android.test.Instrumenta-
tionTestRunner
org.kandroid.memtrac-er.Memory
Instrumentation
Con-trol
Track
TracePackage
Three Alternatives
Three Alternatives
Instrumen-tation
API Level 1 ~ Single activityFramework APIsWhite-box
http://developer.android.com/tools/testing/testing_android.html
Three Alternatives
Instrumen-tation
RobotiumAPI Level 1 ~ Single activityFramework APIsWhite-box
2010/01 ~ Multiple activi-tiesHigh-level APIsWhite-box & black-box
http://developer.android.com/tools/testing/testing_android.htmlhttps://code.google.com/p/robotium/
Three Alternatives
Instrumen-tation
Robotium
uiautomator
API Level 1 ~ Single activityFramework APIsWhite-box
2010/01 ~ Multiple activi-tiesHigh-level APIsWhite-box & black-box
API Level 17 ~Multiple pro-cessesSimple APIsBlack-box
http://developer.android.com/tools/testing/testing_android.htmlhttps://code.google.com/p/robotium/http://developer.android.com/tools/testing/testing_ui.html
Case Study: ApiDemoshttps://github.com/snailee/deview-
2013-samples
Click
Case Study: ApiDemoshttps://github.com/snailee/deview-
2013-samples
Click
Case Study: ApiDemoshttps://github.com/snailee/deview-
2013-samples
Click
Case Study: ApiDemos
Scroll
down
https://github.com/snailee/deview-2013-samples
Click
Case Study: ApiDemos
Scroll
down
https://github.com/snailee/deview-2013-samples
Click
Case Study: ApiDemos
Scroll
down
Click
https://github.com/snailee/deview-2013-samples
Click
Case Study: ApiDemos
Scroll
down
Click
https://github.com/snailee/deview-2013-samples
Click
Case Study: ApiDemos
Scroll
down
Click
Click
https://github.com/snailee/deview-2013-samples
Click
Case Study: ApiDemos
Scroll
down
Click
Click
https://github.com/snailee/deview-2013-samples
Controlling UI Using Instrumentation
Controlling UI Using Instrumentation
1. Create an Android Test Project for the test package.
$ android create test-project --main ../ApiDemos \> –-name ApiDemosInstrumentTest --path ApiDemosInstrumentTest
Controlling UI Using Instrumentation
1. Create an Android Test Project for the test package.
2. Add a test class inheriting ActivityInstrumen-tationTestCase2.
$ android create test-project --main ../ApiDemos \> –-name ApiDemosInstrumentTest --path ApiDemosInstrumentTest
public class ApiDemosInstrumentTest extends ActivityInstrumentationTestCase2<ApiDemos> { ... public ApiDemosInstrumentTest() { super(ApiDemos.class); }
Controlling UI Using Instrumentation
1. Create an Android Test Project for the test package.
2. Add a test class inheriting ActivityInstrumen-tationTestCase2.
3. Add a test method and launch the activity.
$ android create test-project --main ../ApiDemos \> –-name ApiDemosInstrumentTest --path ApiDemosInstrumentTest
public class ApiDemosInstrumentTest extends ActivityInstrumentationTestCase2<ApiDemos> { ... public ApiDemosInstrumentTest() { super(ApiDemos.class); }
public void testNavigate() { final ApiDemos apiDemos = getActivity(); ... }
Controlling UI Using Instrumentation
public void testNavigate() {
4. Find and act on a control.
Controlling UI Using Instrumentation
public void testNavigate() { ... final ListView demoList = apiDemos.getListView(); final View graphicsDemo = getChildViewByText(demoList, "Graph-
ics"); assertNotNull("Graphics demo should exist", graphicsDemo);
Find a view
to con-trol
4. Find and act on a control.
Controlling UI Using Instrumentation
public void testNavigate() { ... final ListView demoList = apiDemos.getListView(); final View graphicsDemo = getChildViewByText(demoList, "Graph-
ics"); assertNotNull("Graphics demo should exist", graphicsDemo);
apiDemos.runOnUiThread(new Runnable() { public void run() { int pos = demoList.getPositionForView(graphicsDemo); demoList.smoothScrollToPosition(pos); demoList.setSelection(pos); demoList.performItemClick(graphicsDemo, pos, graphicsDemo.getId()); } }); ...
Find a view
to con-trol
Act onthe view
4. Find and act on a control.
Controlling UI Using Instrumentation
public void testNavigate() { ... final ListView demoList = apiDemos.getListView(); final View graphicsDemo = getChildViewByText(demoList, "Graph-
ics"); assertNotNull("Graphics demo should exist", graphicsDemo);
apiDemos.runOnUiThread(new Runnable() { public void run() { int pos = demoList.getPositionForView(graphicsDemo); demoList.smoothScrollToPosition(pos); demoList.setSelection(pos); demoList.performItemClick(graphicsDemo, pos, graphicsDemo.getId()); } }); ...
Find a view
to con-trol
Act onthe view
4. Find and act on a control.
Controlling UI Using Instrumentation
private View getChildViewByText(ListView listView, String text) {
4. Find and act on a control: getChildViewByText.
Controlling UI Using Instrumentation
private View getChildViewByText(ListView listView, String text) { View view = null; int count = listView.getCount(); Log.e(TAG, “no. children = " + count); for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); Log.e(TAG, i + "-th text view is " + textView);
Iterate through
child views
4. Find and act on a control: getChildViewByText.
Controlling UI Using Instrumentation
private View getChildViewByText(ListView listView, String text) { View view = null; int count = listView.getCount(); Log.e(TAG, “no. children = " + count); for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); Log.e(TAG, i + "-th text view is " + textView); if (textView != null && textView.getText().equals(text)) { view = textView; break; }
Iterate through
child views
Compare texts
4. Find and act on a control: getChildViewByText.
Controlling UI Using Instrumentation
private View getChildViewByText(ListView listView, String text) { View view = null; int count = listView.getCount(); Log.e(TAG, “no. children = " + count); for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); Log.e(TAG, i + "-th text view is " + textView); if (textView != null && textView.getText().equals(text)) { view = textView; break; } } return view; }
Iterate through
child views
Compare texts
4. Find and act on a control: getChildViewByText.
Controlling UI Using Instrumentation
private View getChildViewByText(ListView listView, String text) { View view = null; int count = listView.getCount(); Log.e(TAG, “no. children = " + count); for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); Log.e(TAG, i + "-th text view is " + textView); if (textView != null && textView.getText().equals(text)) { view = textView; break; } } return view; }
Iterate through
child views
Compare texts
4. Find and act on a control: getChildViewByText.
com.example.android.apis.instrument.test.ApiDemosInstrumentTest:INSTRUMENTATION_RESULT: shortMsg=junit.framework.AssertionFailedErrorINSTRUMENTATION_RESULT: longMsg=junit.framework.AssertionFailedError: OpenGL ES demo should ex-istINSTRUMENTATION_CODE: 0
Controlling UI Using Instrumentation
private View getChildViewByText(final ListView listView, String text) { ... for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); if (textView != null && textView.getText().equals(text)) { view = textView; break; } if (textView == null) { final int lastPos = listView.getLastVisiblePosition(); getInstrumentation().runOnMainSync(new Runnable() { public void run() { listView.smoothScrollToPosition(lastPos + 5, lastPos); } }); getInstrumentation().waitForIdleSync(); i = listView.getFirstVisiblePosition(); } } return view; }
4. Find and act on a control: Revised getChild-ViewByText.
Controlling UI Using Instrumentation
private View getChildViewByText(final ListView listView, String text) { ... for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); if (textView != null && textView.getText().equals(text)) { view = textView; break; } if (textView == null) { final int lastPos = listView.getLastVisiblePosition(); getInstrumentation().runOnMainSync(new Runnable() { public void run() { listView.smoothScrollToPosition(lastPos + 5, lastPos); } }); getInstrumentation().waitForIdleSync(); i = listView.getFirstVisiblePosition(); } } return view; }
Scroll down
4. Find and act on a control: Revised getChild-ViewByText.
Controlling UI Using Instrumentation
5. Monitor a new Activity to launch.
public void testNavigate() { ... Instrumentation.ActivityMonitor graphicsDemoActivityMonitor = getInstrumentation().addMonitor( ApiDemos.class.getName(), null, false); final ApiDemos graphicsDemoActivity = (ApiDemos) graphicsDemoActivityMonitor.waitForActivity(); assertNotNull("Graphics Demo (ApiDemos) activity should not be null", graphicsDemoActivity); ...
Controlling UI Using Instrumentation
5. Monitor a new Activity to launch.
6. Iterate step 4 & 5.
public void testNavigate() { ... Instrumentation.ActivityMonitor graphicsDemoActivityMonitor = getInstrumentation().addMonitor( ApiDemos.class.getName(), null, false); final ApiDemos graphicsDemoActivity = (ApiDemos) graphicsDemoActivityMonitor.waitForActivity(); assertNotNull("Graphics Demo (ApiDemos) activity should not be null", graphicsDemoActivity); ...
Controlling UI Using Instrumentation
7. Build and Install the test package.
$ cd ApiDemosInstrumentTest$ ant debug$ adb install –r bin/ApiDemosInstrumentTest-debug.apk
Controlling UI Using Instrumentation
7. Build and Install the test package.
8. Run the test and see the result.
$ adb shell am instrument -w \> com.example.android.apis.instrument.test/\> android.test.InstrumentationTestRunner
$ cd ApiDemosInstrumentTest$ ant debug$ adb install –r bin/ApiDemosInstrumentTest-debug.apk
Controlling UI Using Instrumentation
7. Build and Install the test package.
8. Run the test and see the result.
$ adb shell am instrument -w \> com.example.android.apis.instrument.test/\> android.test.InstrumentationTestRunner
$ cd ApiDemosInstrumentTest$ ant debug$ adb install –r bin/ApiDemosInstrumentTest-debug.apk
$ ant test
Controlling UI Using Robotium
Controlling UI Using Robotium
1. Create an Android Test Project for the test package.
$ android create test-project --main ../ApiDemos \> –-name ApiDemosRobotiumTest --path ApiDemosRobotiumTest
Controlling UI Using Robotium
1. Create an Android Test Project for the test package.
2. Add ‘robotium-solo-<ver>.jar’ in the ‘libs’ di-rectory.
$ android create test-project --main ../ApiDemos \> –-name ApiDemosRobotiumTest --path ApiDemosRobotiumTest
Controlling UI Using Robotium
1. Create an Android Test Project for the test package.
2. Add ‘robotium-solo-<ver>.jar’ in the ‘libs’ di-rectory.3. Add a test class inheriting ActivityInstrumen-tationTestCase2.
$ android create test-project --main ../ApiDemos \> –-name ApiDemosRobotiumTest --path ApiDemosRobotiumTest
public class ApiDemosRobotiumTest extends ActivityInstrumentationTestCase2<ApiDemos> { ... public ApiDemosRobotiumTest() { super(ApiDemos.class); } ...
Controlling UI Using Robotium
4. Create an instance of Solo in the setUp method and close open activities in the tearDown method. ...
private Solo solo; ...
@Override public void setUp() throws Exception { solo = new Solo(getInstrumentation(), getActivity()); }
@Override public void tearDown() throws Exception { solo.finishOpenedActivities(); } ...
Controlling UI Using Robotium
4. Find and act on a control.public void testNavigate() { solo.assertCurrentActivity("Should be on ApiDemos", ApiDemos.class);
solo.clickOnText("^Graphics$", 1, true); solo.assertCurrentActivity("Should be on a new ApiDemos", ApiDemos.class, true);
solo.clickOnText("^OpenGL ES$", 1, true); solo.assertCurrentActivity("Should be on a new ApiDemos", ApiDemos.class, true);
solo.clickOnText("^Kube$", 1, true); solo.assertCurrentActivity("Should be on a Kube", Kube.class, true);
solo.sleep(3000);}
Controlling UI Using Robotium
6. Build and Install the test package.
$ cd ApiDemosRobotiumTest$ ant debug$ adb install –r bin/ApiDemosRobotiumTest-debug.apk
Controlling UI Using Robotium
6. Build and Install the test package.
7. Run the test and see the result.
$ adb shell am instrument -w \> com.example.android.apis.robotium.test/\> android.test.InstrumentationTestRunner
$ cd ApiDemosRobotiumTest$ ant debug$ adb install –r bin/ApiDemosRobotiumTest-debug.apk
Controlling UI Using Robotium
6. Build and Install the test package.
7. Run the test and see the result.
$ adb shell am instrument -w \> com.example.android.apis.robotium.test/\> android.test.InstrumentationTestRunner
$ cd ApiDemosRobotiumTest$ ant debug$ adb install –r bin/ApiDemosRobotiumTest-debug.apk
$ ant test
Controlling UI Using uiautomator
1. Create an Java Project for the test jar.
Controlling UI Using uiautomator
1. Create an Java Project for the test jar.2. Add ‘android.jar’ and ‘uiautomator.jar’ in the build path.
Controlling UI Using uiautomator
1. Create an Java Project for the test jar.2. Add ‘android.jar’ and ‘uiautomator.jar’ in the build path.3. Add a test class inheriting UiAutomatorTest-Case.
public class ApiDemosUiAutoTest extends UiAutomatorTestCase { ...}
Controlling UI Using uiautomator
4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException {
Controlling UI Using uiautomator
4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome();
Press home button
Controlling UI Using uiautomator
4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome();
UiObject allAppsButton = new UiObject( new UiSelector().description("Apps")); allAppsButton.clickAndWaitForNewWindow();
Click ‘All Apps’ button
Press home button
Controlling UI Using uiautomator
4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome();
UiObject allAppsButton = new UiObject( new UiSelector().description("Apps")); allAppsButton.clickAndWaitForNewWindow();
UiObject appsTab = new UiObject(new UiSelector().text("Apps")); appsTab.click();
Click ‘All Apps’ button
Press home button
Click ‘Apps’ tab
Controlling UI Using uiautomator
4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome();
UiObject allAppsButton = new UiObject( new UiSelector().description("Apps")); allAppsButton.clickAndWaitForNewWindow();
UiObject appsTab = new UiObject(new UiSelector().text("Apps")); appsTab.click();
UiScrollable appList = new UiScrollable( new UiSelector().scrollable(true)); appList.setAsHorizontalList(); UiObject apiDemos = appList.getChildByText( new UiSelector().className(TextView.class.getName()), "API
Demos"); apiDemos.clickAndWaitForNewWindow();
Click ‘All Apps’ button
Press home button
Click ‘Apps’ tab
Click ‘Api-Demos’ icon
Controlling UI Using uiautomator
4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome();
UiObject allAppsButton = new UiObject( new UiSelector().description("Apps")); allAppsButton.clickAndWaitForNewWindow();
UiObject appsTab = new UiObject(new UiSelector().text("Apps")); appsTab.click();
UiScrollable appList = new UiScrollable( new UiSelector().scrollable(true)); appList.setAsHorizontalList(); UiObject apiDemos = appList.getChildByText( new UiSelector().className(TextView.class.getName()), "API
Demos"); apiDemos.clickAndWaitForNewWindow();
UiObject apiDemosPackage = new UiObject( new UiSelector().packageName("com.example.android.apis")); assertTrue(“Should be on ApiDemos", apiDemosPackage.exists()); ...
Click ‘All Apps’ button
Press home button
Click ‘Apps’ tab
Click ‘Api-Demos’ icon
Validate the launch of
‘ApiDemos’
Controlling UI Using uiautomator
5. Find and act on a control.
Controlling UI Using uiautomator
5. Find and act on a control.
... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView"));
Controlling UI Using uiautomator
5. Find and act on a control.
... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView")); UiObject graphicsDemo = demoList.getChildByText( new UiSelector().className(TextView.class.getName()),
“Graphics");
Controlling UI Using uiautomator
5. Find and act on a control.
... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView")); UiObject graphicsDemo = demoList.getChildByText( new UiSelector().className(TextView.class.getName()),
“Graphics"); graphicsDemo.clickAndWaitForNewWindow();
Controlling UI Using uiautomator
5. Find and act on a control.
... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView")); UiObject graphicsDemo = demoList.getChildByText( new UiSelector().className(TextView.class.getName()),
“Graphics"); graphicsDemo.clickAndWaitForNewWindow();
UiObject alphaBitmapDemo = new UiObject( new UiSelector().text(“AlphaBitmap")); assertTrue(“AlphaBitmap should be visible", alpahBitmapDemo.ex-
ists()); ...
Controlling UI Using uiautomator
5. Find and act on a control.
6. Iterate step 5.
... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView")); UiObject graphicsDemo = demoList.getChildByText( new UiSelector().className(TextView.class.getName()),
“Graphics"); graphicsDemo.clickAndWaitForNewWindow();
UiObject alphaBitmapDemo = new UiObject( new UiSelector().text(“AlphaBitmap")); assertTrue(“AlphaBitmap should be visible", alpahBitmapDemo.ex-
ists()); ...
Controlling UI Using uiautomator
7. Compile your test case into a JAR file.
$ android create uitest-project --name ApiDemosUiAutoTest \> --target 1 --path ApiDemosUiAutoTest $ cd ApiDemosUiAutoTest$ ant build
Controlling UI Using uiautomator
7. Compile your test case into a JAR file.
8. Install the jar on your test device.
$ android create uitest-project --name ApiDemosUiAutoTest \> --target 1 --path ApiDemosUiAutoTest $ cd ApiDemosUiAutoTest$ ant build
$ adb push bin/ApiDemosUiAutoTest.jar /data/local/tmp/
Controlling UI Using uiautomator
7. Compile your test case into a JAR file.
8. Install the jar on your test device.
9. Run the test and view the test result.
$ android create uitest-project --name ApiDemosUiAutoTest \> --target 1 --path ApiDemosUiAutoTest $ cd ApiDemosUiAutoTest$ ant build
$ adb push bin/ApiDemosUiAutoTest.jar /data/local/tmp/
$ adb shell uiautomator runtest ApiDemosUiAutoTest.jar \> –c com.example.android.apis.uiauto.test.ApiDemosUiAutoTest
Controlling UI Using uiautomator
7. Compile your test case into a JAR file.
8. Install the jar on your test device.
9. Run the test and view the test result.
$ android create uitest-project --name ApiDemosUiAutoTest \> --target 1 --path ApiDemosUiAutoTest $ cd ApiDemosUiAutoTest$ ant build
$ adb push bin/ApiDemosUiAutoTest.jar /data/local/tmp/
$ adb shell uiautomator runtest ApiDemosUiAutoTest.jar \> –c com.example.android.apis.uiauto.test.ApiDemosUiAutoTest
$ ant test
Controlling UI Using uiautomator
7. Compile your test case into a JAR file.
8. Install the jar on your test device.
9. Run the test and view the test result.
$ android create uitest-project --name ApiDemosUiAutoTest \> --target 1 --path ApiDemosUiAutoTest $ cd ApiDemosUiAutoTest$ ant build
$ adb push bin/ApiDemosUiAutoTest.jar /data/local/tmp/
$ adb shell uiautomator runtest ApiDemosUiAutoTest.jar \> –c com.example.android.apis.uiauto.test.ApiDemosUiAutoTest
$ ant test
Doing both Controlling and Tracking
Doing both Controlling and Tracking
1. Modify org.kandroid.memtracer.MemoryIn-strumentation.public class MemoryInstrumentation extends Instrumentation-TestRunner { ... @Override public void onCreate(Bundle arguments) { ... start(); super.onCreate(arguments); }
@Override public void onStart() { String mainActivityName = getMainActivityName(); if (mainActivityName != null && mainActivityName.length() > 0) { launchMainActivity(getTargetContext().getPackageName(), getMainActivityName()); } else { super.onStart(); } } ...
Doing both Controlling and Tracking
1. Modify org.kandroid.memtracer.MemoryIn-strumentation.public class MemoryInstrumentation extends Instrumentation-TestRunner { ... @Override public void onCreate(Bundle arguments) { ... start(); super.onCreate(arguments); }
@Override public void onStart() { String mainActivityName = getMainActivityName(); if (mainActivityName != null && mainActivityName.length() > 0) { launchMainActivity(getTargetContext().getPackageName(), getMainActivityName()); } else { super.onStart(); } } ...
Doing both Controlling and Tracking
1. Modify org.kandroid.memtracer.MemoryIn-strumentation.public class MemoryInstrumentation extends Instrumentation-TestRunner { ... @Override public void onCreate(Bundle arguments) { ... start(); super.onCreate(arguments); }
@Override public void onStart() { String mainActivityName = getMainActivityName(); if (mainActivityName != null && mainActivityName.length() > 0) { launchMainActivity(getTargetContext().getPackageName(), getMainActivityName()); } else { super.onStart(); } } ...
Doing both Controlling and Tracking
1. Modify org.kandroid.memtracer.MemoryIn-strumentation.public class MemoryInstrumentation extends Instrumentation-TestRunner { ... @Override public void onCreate(Bundle arguments) { ... start(); super.onCreate(arguments); }
@Override public void onStart() { String mainActivityName = getMainActivityName(); if (mainActivityName != null && mainActivityName.length() > 0) { launchMainActivity(getTargetContext().getPackageName(), getMainActivityName()); } else { super.onStart(); } } ...
Doing both Controlling and Tracking
2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory.
Doing both Controlling and Tracking
2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory.3. Edit the <instrumentation> element in the man-ifest file.
Doing both Controlling and Tracking
2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory.3. Edit the <instrumentation> element in the man-ifest file.4. Build and Install the test package.
$ cd ApiDemosInstrumentTest$ ant debug$ adb install –r bin/ApiDemosInstrumentTest-debug.apk
Doing both Controlling and Tracking
2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory.3. Edit the <instrumentation> element in the man-ifest file.4. Build and Install the test package.
5. Run the test and see the result.
$ adb shell am instrument -w \> com.example.android.apis.instrument.test/\> org.kandroid.memtracer.MemoryInstrumentation
$ cd ApiDemosInstrumentTest$ ant debug$ adb install –r bin/ApiDemosInstrumentTest-debug.apk
Doing both Controlling and Tracking
2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory.3. Edit the <instrumentation> element in the man-ifest file.4. Build and Install the test package.
5. Run the test and see the result.
6. Pull the trace file ‘kmemtrace.csv’ from /sdcard/kmemtracer.
$ adb shell am instrument -w \> com.example.android.apis.instrument.test/\> org.kandroid.memtracer.MemoryInstrumentation
$ cd ApiDemosInstrumentTest$ ant debug$ adb install –r bin/ApiDemosInstrumentTest-debug.apk
$ adb pull /sdcard/kmemtracer/kmemtrace.csv .
SUMMARY
SUMMARYWhy Automated Performance Testing?
Measure to improve / Fragmentation
SUMMARYWhy Automated Performance Testing?
Measure to improve / Fragmentation
How to Automate Performance Testing?
- Historical Analysis w/ kmemtracer
Instrumentation for track
SUMMARYWhy Automated Performance Testing?
Measure to improve / Fragmentation
How to Automate Performance Testing?
- Historical Analysis w/ kmemtracer
Instrumentation for track
- Measuring Performance
API (Public / Internal) / cmds (Android / Linux)
SUMMARYWhy Automated Performance Testing?
Measure to improve / Fragmentation
How to Automate Performance Testing?
- Historical Analysis w/ kmemtracer
Instrumentation for track
- Measuring Performance
API (Public / Internal) / cmds (Android / Linux)
- Controlling UI
Instrumentation / Robotium / uiautomator
Q&A
THANK YOU