A particle filter based scheme for indoor tracking on an Android Smartphone

A Particle Filter Based Scheme for Indoor Tracking on an Android

SmartphoneDivye

KapoorB.Tech (IDD) CSI – V

061305Under Guidance of

Dr. Manoj MisraProfessor, E&CE Dept.

IIT Roorkee

Good Afternoon!

Thank you for going through my thesis

• Thesis Stats:– 85 pages of content– 47 figures– 18 tables– 2 algorithms

– Total pages: 100 (excluding the cover page)

Presentation Structure• Follow the structure of the thesis– Chapter Wise

• Emphasis– Understanding the problem– Understanding the current state of

research– Understanding what has been

accomplished in this work.

Presentation Structure…• Introduction• Literature Review• Proposed Work• Implementation Details• Groundwork• Results• Summary

Chapter 1: Introduction

Introduction• 2007 - The rise of the “smartphone”

– Capacitive touch– Fast processors– Sensors

• Culmination of technological advances in many fields– Higher integration– Better fabrication processes– New Technology: capacitive touch, MEMS– Processors

• Advances in hardware have been matched with advances in software

Trends in the Industry• Move GPS navigation from a dedicated device

to your pocket– Location based services, location aware applications

• 2008 onwards

– Route planning and navigation (beta applications)• 2010 and 2011

• Location is paramount for personalization

• How accurately can you locate a user and then track his movements?– The answer determines viability of a number of LBS

A Natural Boundary• Indoor v/s Outdoors:

– The technological landscape is very different.

• Outdoors: A nearly solved problem– Core: High precision GPS– Other coarse methods– Commercially Available solutions

• Indoors: A very fragmented landscape– Multiple, highly specialized approaches– Pros and Cons– Very different accuracy tolerances, site information

requirements, sensor requirements

Motivation• Choose a topic that is:– Current in nature– Broad in scope– Technologically intensive– Has a wide variety of applications

• Applications– Navigational Aids– Shipping Industry, Inventory Systems, Large

Retailers– Augmented Reality and Reality Fusion Gaming

Scope• Develop and characterize a smartphone

based solution for online tracking of pedestrians indoors.

• Fits in well with the overall trend of integrating the virtual and real worlds.

• Addresses some of the identified application areas.

Chapter 2: Literature Review

Literature Review• Progression of Ideas– Initial Wifi based techniques (RADAR [4], SpotOn

[8]) – for positioning and tracking

– Trilateration• Based on existing theory• Link noise-free signal strength to distance• Simple geometry estimates distances

– Location Fingerprinting• Create a discretely sampled version of the signal strength

graph labelled by location• Match measured samples to find the closest location

match

Literature Review…• Progression of Ideas…

– Bluetooth based techniques (Kotanen et al[7])• Extension of [4]

– Analysis of the behaviour of Wifi signals [1] was done to set experimental foundations of the field• Important results: RSS does not decay log-normally.

Orientation of device matters, user introduces systematic variations in the system. Summarized in Table 2.1.

– Independently, the ideas of RADAR [4], Kotanen et al [7] and SpotOn [8] were extended by Ladd et al [9], King et al[10] & others.• Covered in detail in paper [17]• Their systems had the best accuracy results among

contemporaries using similar data

Literature Review…• Ladd et al [9]:– Bayesian Approach to determine a relatively

converged p(x|o0, o1, …, on)

– A HMM was bolted onto the output of the Bayesian model to try and track the user through interpolation

• King et al [10]:– Used digital compasses to provide orientation

information.– Used it for positioning, but not for tracking

applications. – Suggests angular tolerance of 60 degrees for Wifi

samples for maximum positioning accuracy

Literature Review…• Integration of INS with Wifi data– The advent of MEMS made the sensors required

for an INS portable and thus suitable for indoor tracking.

– INS systems for pedestrian navigation using step detection had been explored in an outdoor setting with GPS devices by Ladetto in [18]

– Evennou and Marx [11] and Wang et al [12] explored the application of these techniques for indoor tracking using a particle filters based system that leveraged Wifi site survey information.

– In addition, Wang et al [12] attempted to include basic map information.

Literature Review…• “Due to the large number of

particles, the algorithm is too complex to be implemented on handheld devices. A way to cut down this number of particles must be found.” – Evennou and Marx [11]

• They had attempted to use a trilateration approach with the Wifi signal strengths.

Literature Review…• Wang et al [12] used the Wifi location

fingerprinting method and a weighted particle filter for indoor tracking.

• INS information and map data was subsequently incorporated into the system to improve accuracy.

Literature Review…• Since 2008, a lot of work has focussed

on removing the pain of a Wifi site survey using crowdsourcing techniques. – Redpin [16]

• Smartphone based indoor positioning and tracking systems– Redpin [16] has been a leader in this regard

(but only for positioning)– Other systems based on other devices…

Research Gaps• Systems like Evannou and Marx [11] still

depend on the assumption of a log-normally distributed signal strength which is not true as shown by [1].

• Wifi based systems report very high inaccuracies (2m with large SD) and these errors are not coming down.

• No advanced integration of map information is done for tracking purposes.

• Specific challenges with respect to smartphones have not been determined or addressed.

Chapter 3: Proposed Work

Particle Filters (Background)• Computationally intensive filtering

methods that in an indoor localization context help determine the PDF p(x|o0,o1…,on) through a Monte Carlo simulation of the system.

• They work on a predict-correct cycle and each particle represents a weighted system state.Predicti

onCorrectio

n

Adjust weights

Resampling

Predict the next states No States

Left

Generate new, diverse

states

Proposed Method• Builds upon the work of Evennou and

Marx [11] and Wang et al [12] in a smartphone context.

• Uses sensor information available on a modern, off the shelf, Android smartphone.–Wifi data– Accelerometer Data–Magnetometer Data + (optional)

Gyroscope data– Camera

Proposed Method

Proposed Method• Though shown as a sequential

algorithm, the algorithm will be implemented in an event-driven manner.

• Various constants of the algorithm are sensor and implementation dependant.

• There are many subroutines that are critical for operation of the algorithm

First Fix• Any tracking system must start off

with an estimate of the initial location of the tracked entity.

• The initial location is called the first fix of the algorithm.

• 2 methods are considered:–Manual marking of the starting location– QRCode encoded location information in

the environment

First Fix

• Manual Marking:– Simple principle: ask

the user – “Where are you?”

– Mark it out on the map via the capacitive touchscreen

• QRCodes:– Encode location

information into a machine readable 2D barcode and install it in the environment.

First Fix: QRCodes• What to encode and how to encode

it?

Basic INS• Requires determination of

incremental changes to displacement.

• Required values:– Displacement change– Direction of change.

INS: Step Detection• Simple double integration of accelerometer data

(even by using equations of greater precision such as the Verlet integrals) yields no usable information about displacement. – The integrals diverge in about 3-5 seconds

• Step detection and step size estimation methods similar to the ones used by Wang et al [12] were used.– Counting zero crossings on filtered accelerometer

readings– Counting distinct peaks and valleys in the filtered

accelerometer readings.

Noise Filtering• Many real world sensors like the 3

axis accelerometers on the Android smartphone suffer from Sensor Noise

• Simple Noise rejection procedure

Sample Unfiltered Sensor Readings

Step Detection: Peak and Valley Method

Step Size Estimation• The empirical relationship between

step size and accelerometer readings provided by Analog Devices in [22] is used:

Dynamical Equations of the System

Accounting for Orientation Bias

Accounting for Varying Step Sizes

Ensuring Particle Diversity• The introduction of sampled random

variables into the system is an intentional effort to cause particle diversity.

• However, the choice of the variance of the random variables is critical.– Too small a variance would prevent

achievable states from being evaluated.– Large variance would waste computational

effort on “invalid” particles.

Integrating Map Information (MapSelect)

• This is effectively the “correcting function” for the particle filter.

Map Information Allows Bias Determination

• The bias values are strongly dependent on the degree of map information available.

Particle Insufficiency• Particle insufficiency occurs when no

valid state is found after the “correction” step where transitions are verified to be valid according to map information.

• One common cause of insufficiency is incomplete approximation of the user’s state due to the highly restricted number of particles

Recovery from Particle Insufficiency

Recovery from Particle Insufficiency

Algorithms for Comparison• Comparisons need to be made in the same

environment due to variability of the environment, maps and Wifi site survey information.

• 2 other algorithms are implemented for comparison:– Uncorrected or Dead Reckoning– Wifi Corrected reckoning

• Extension of King et al’s [10] work to tracking using repeated Wifi based positioning with fusion of step information.

Chapter 4: Implementation Details

Typical Structure of the Android OS

Implementation Tools• Eclipse– Java IDE– Tooling support for installation, uninstallation of apps

under development

• Android Debug Bridge– Gives visibility and remote debugging capabilities for

code running on the target system (the smartphone)

• LogCat over ADB– The logging subsystem buffers logs and makes them

visible remotely

Simplified Lifecycle of an App

Interfacing with the Android Sensor API

System Architecture

Classes and their Layers

Notes on Lifecycle Implementation

• We have little to no user interaction once we have started tracking the user path.

• WakeLocks are used to prevent the device from going into an idle state, thus pushing the app into the paused state.

• Centralizing sensor management into a single layer allowed simple handling of the lifecycle events. – The layer does the right thing at lifecycle events– Abstracts complex sensor state management from the

Activity behind a simple façade (pause() and resume())

Computational Complexities at Various Layers

• Absolutely minimal computations upto the step detection layer.

• Slightly greater freedom in implementing algorithms at higher layers– Higher layers are accessed infrequently

Chapter 5: Groundwork

Hardware

• Google Nexus S manufactured by Samsung

• Launched December 2010.

• Sensors of interest: – Accelerometer– Magnetometer– Gyroscope– Orientation Sensor– Wifi NIC– Camera

Characterization of Input Signals

• Required to determine practical limits of accuracy of the sensor data.– Accelerometer–Magnetometer/Orientation Sensor

• Also, to characterize the noise present in the signal itself–Wifi signal strength data

Characterization of the Accelerometer

• 2 Different Scenarios:– The phone is placed on a stable surface– The phone is held in the palm of a user

• Marked increase in sensor noise seen

Characterization of the Accelerometer

Characterization of the Magnetometer

Characterization of the Magnetometer

Magnetometer: Effect of Motion

Magnetometer: Effect of Motion

Test Environment

Test Environment: Suitability• Long corridoors of around 30-32 m in

length

• Large number of Wifi Access Points in the vicinity

• Wifi access points are installed in the corridoors itself– Neighbouring wings and floors also have

APs, thus providing location fingerprint diversity.

Characterization of the Wifi Signal Strength Distribution: Short term

Wifi RSSI Distribution: Long Term (7 days)

Wifi RSSI Distribution:Long Term (7 days)

Wifi RSSI Distribution: Effect of Motion, Position, Orientation

Wifi Site Survey

• Over 1500 samples taken

• 8 different directions

• Painstaking work

• 10+ hours for a dataset– Multiple times

Wifi Positioning Performance

Parameters Determined

Chapter 6: Experiments and Results

First Fix: Manual Method

First Fix: QRCodes

Noise Filtering (Q = 1.3m/s2)

Noise Filtering

Step Detection Performance

Step Detection Performance

Step Variation Analysis

Test Paths

• A-B• A-B-A• A-B zigzag• A-B-C-D• A-B-C-D-C-B-

A

Simple Dead Reckoning

• Significant but stable bias

• Path goes through rooms and walls and is quite unnatural.

• Bias varies with path orientation

Simple Dead Reckoning

Wifi Corrected Reckoning

• Degree of localization is good

• Subjective path quality is poor

Wifi Corrected Reckoning

• Median error is quite good.

• Unfortunately, the standard deviation is quite large.– Expected based on

groundwork

Wifi Corrected Reckoning

Proposed Method

Proposed Method

Proposed Method

Proposed Method: Number of Particles

Proposed Method: Recovery

Comparison with Published Results

Chapter 7: Summary

Points in Summary• Significant improvement in tracking

accuracy compared to published work

• Suitable modifications made to allow running the algorithm on an Android smartphone with its limited resources

Insights• QRCodes are excellent tools for providing first fixes to

tracking algorithms. – Reasonably scalable and extremely easy to use.

• Wifi is a poor choice of signal source to correct for drift errors for step estimate based tracking solutions.

• Having a Wifi survey of an area is a very time consuming matter. However, it helps in recovery.

• Treating all particles as unweighted reduces computational complexity

• Choice of parameters for sensors is very important for good performance

Limitations• A limitation on the number of particles results in a

higher tendency to get lost

• Once lost, recovery is not guaranteed and is usually associated with some tracking error

• A wifi survey is required.– Not very scalable and very time consuming– Can be traded off for lower recovery probabilities

• Lazy KNN type algorithms using the survey data are extremely expensive in time and memory consumption

Limitations• Small, shuffling steps cannot be detected reliably due

to the sensor noise of the accelerometer.– Tradeoff

• Maps with large open spaces provide very little feedback to the system.– But suitable for dense, office like environments

• Moving “backwards” is not handled very well and introduces very significant tracking error.

• Staircases have very different step profiles and cannot be handled– Generic issue for all step based algorithms

Future Enhancements• The MapSelect function can be improved by looking at making

walls near doorways more “porous” to particles.

• Integration of Barometric sensor information (once available via smartphone sensors) needs to be looked at – especially in the context of staircases.

• Methods to change maps once floors are changed need to be developed. Linking floor information in QRCodes needs evaluation.

• SVM needs to be evaluated for improving recovery accuracy using Wifi Survey data.– Data Mining of Wifi Survey dataset

• Additional Benefit: High accuracy tracking along with a Wifi RSSI logger can be used to quickly build a signal strength map of an area. This would be very beneficial to crowdsourced low complexity positioning solutions like Redpin [16]

Thank You. Questions?

Additional Slides