Smart Phone/Pad Sensor Applications Resizing screen/tilt
Environment adjustment of apps for users comfort Adjustment in
cinema, movement prediction Gaming Augmented Reality (AR) AR Gaming
AR Navigation Bar codes Geotagging, recommendations.. Network of
objects, locations and people, 3D social networking? Distributed
sensor systems Noise mapping, traffic information, etc. And
anything you can imagine
Slide 3
Vehicular Sensing People Sensing
Slide 4
I. KAIST Slides based on: Sensor Fusion on Android Devices A
Revolution in Motion Processing, David Sachs, Feb. 2010 Android
Sensors, Stefan Varga, Michal Kostic, Oct. 2010
Slide 5
Contents Elements of mobile sensor networking Programming
Android sensor/location service Setting up sensor/location service
Processing events How do sensors work? Accelerometer, Magnetometer,
Gyroscope, Light Sensor, Proximity Sensor, GPS, Microphone, etc.
How do location services work? GPS, cell-tower location, Wi-Fi
positioning
Slide 6
Elements of Mobile Sensor Networking network infrastructure
Mobile sensors r vehicles, smartphones r run sensing applications r
may be stationary (non- mobile) or mobile
Slide 7
Elements of Mobile Sensor Networking network infrastructure
Wireless nets r mobile to base station r base station: e.g., cell
towers, 802.11 access points; connected to a wired network r (also,
mobile-to-mobile: Bluetooth, Wi-Fi ad hoc) r multiple access
protocol coordinates link access r various data rates, transmission
distance Bluetooth, WiFi Ad Hoc Wi-Fi, 3G/LTE/WiMax
Slide 8
Elements of Mobile Sensor Networking network infrastructure
Computing infra r back-end platform for data processing and web
services r could be part of network infra (or independent) r
centralized or de- centralized infra
Slide 9
Wireless Networking Standards Indoor 10-30m Outdoor 50-200m
Mid-range outdoor 200m 4 Km Long-range outdoor 5Km 20 Km.056.384 1
4 5-11 54 GSM, IS-95, CDMA 2G UMTS/WCDMA, CDMA2000 3G 802.15
802.11b 802.11a,g UMTS/WCDMA-HSPDA, CDMA2000-1xEVDO 3G cellular
enhanced LTE, 802.16 (WiMAX) 200 802.11n Data rate (Mbps) All IP
Wi-Fi Bluetooth 4G
Slide 10
Contents Elements of mobile sensor networking Programming
Android sensor/location service Setting up sensor/location service
Processing events How do sensors work? Accelerometer, Magnetometer,
Gyroscope, Light Sensor, Proximity Sensor, GPS, Microphone, etc.
How do location services work? GPS, cell-tower location, Wi-Fi
positioning
Android API (I) Package: android.hardware Classes:
SensorManager Android sensor service Sensor specific sensor
SensorEventListener SensorEvent specific event of the sensor = data
LocationManager Android location service LocationProvider (e.g.,
WPS, Network, GPS) LocationListener Processing updates for location
(Location) or provider (LocationProvider)
Slide 14
Android API Most sensors interfaced through SensorManager or
LocationManager Obtain pointer to android service using
Context.getSystemService(name) For name, use constant defined by
Context class SENSOR_SERVICE for SensorManager LOCATION_SERVICE for
LocationManager Check for available sensors using List
mgr.getSensorList(int type) Type constants provided in Sensor class
documentation Check for available location providers using List
providers = mgr.getAllProviders();
Slide 15
SensorManager Use getDefaultSensor(int type) to get a pointer
to the default sensor for a particular type Sensor accel =
sensorManager.getDefaultSensor( Sensor.TYPE_ACCELEROMETER);
Register for updates of sensor values using
registerListener(SensorEventListener, Sensor, rate) Rate is an int,
using one of the following 4 constants SENSOR_DELAY_NORMAL (delay:
200ms) SENSOR_DELAY_UI (delay: 60ms) SENSOR_DELAY_GAME (delay:
20ms) SENSOR_DELAY_FASTEST (delay: 0ms) Use the lowest rate
necessary to reduce power usage Registration will power up sensor:
mSensorService.enableSensor(l, name, handle, delay);
Slide 16
SensorManager Unregister for sensor events using
unregisterListener(SensorEventListener, Sensor) or
unregisterListener(SensorEventListener) Undregistering will power
down sensors: mSensorService.enableSensor(l, name, handle,
SENSOR_DISABLE) Perform register in OnResume() and unregister in
OnPause() to prevent using resources while your activity is not
visible SensorListener is deprecated, use SensorEventListener
instead See documentation for Sensor, SensorManager, SensorEvent
and SensorEventListener
Slide 17
SensorEventListener Must implement two methods
onAccuracyChanged(Sensor sensor, int accuracy)
onSensorChanged(SensorEvent event) SensorEvent int accuracy Sensor
sensor long timestamp Time in nanoseconds at which event happened
float[] values Length and content of values depends on sensor
type
Slide 18
API Setup public class MainActivity extends Activity implements
SensorEventListener {.. private SensorManager sm = null; public
void onCreate (Bundle savedInstanceState) {.. sm = (SensorManager)
getSystemService(SENSOR_SERVICE); } protected void onResume () {..
List typedSensors = sm.getSensorList(Sensor.TYPE_LIGHT); // also:
TYPE_ALL if (typedSensors == null || typedSensors.size() Westwood)
Manhattan MobilityReal-track Mobility Time (s) Fraction of actively
harvested meta-data">
Meta-data Harvesting Results Meta-data harvesting: agent
actively harvests meta-data Impact of node density (#nodes), speed,
mobility Higher speed, faster harvesting Higher density, faster
harvesting (more # of meta-data from neighbors) Less restricted
mobility, faster harvesting (Man>Westwood) Manhattan
MobilityReal-track Mobility Time (s) Fraction of actively harvested
meta-data
Slide 65
Trajectory Tracking Agent re-constructs the trajectory of a
node
Slide 66
CarTel: A Distributed Mobile Sensor Computing System Bret Hull,
Vladimir Bychkovsky, Kevin Chen, Michel Goraczko, Allen Miu, Eugene
Shih, Yang Zhang, Hari Balakrishnan, and Samuel Madden Sensys
2006
Slide 67
CarTel System Architecture Open wireless Access Point Users
wireless Access Point Adapters log GPS, OBD, camera data Data sent
via prioritized continuous queries ICEDB Remote Delay-tolerant
relay via WiFi CafNet PortalClients Prioritizes data Answers local
snapshot queries Logs continuous query results ICEDB Server
Slide 68
CarTel S/W Architecture Portal Data Visualization CafNet Stack
CQ Web Server ICEDB Server OBD-II WiFi Monitor Camera Traffic
Speed/ Delay Portal Applications Portal Streaming Sensor Data Cont.
queries + adaptors
Slide 69
Intermittently connected DB (ICEDB) ICEDB server Maintains a
list of continuous queries submitted by applications Queries are
pushed to mobile nodes using CafNets Results from ICEDB clients are
stored in the RDBMS at the portal ICEDB client Process the sensed
data and return the query results using CafNet (carry-and-forward
net) Prioritize the result streams in the order of importance
Slide 70
Intermittently connected DB (ICEDB) Queries in ICEDB are
written in SQL with several extensions for continuous queries and
prioritization EX)SELECT carid, traceid, time, location FROM gps
WHERE gps.time BETWEEN now()-1 mins AND now() RATE 5 mins
Prioritization is required since delivering data in FIFO order is
suboptimal in bandwidth- constrained network Intermittent
connectivity due to high speed mobility and restricted mobility
patterns
Slide 71
Portal Users navigate sensor data in CarTel using a web- based
interface Main components of the portal Portal framework ICEDB
server to retrieve sensor data Data visualization library to
display geo-coded data Trace: all sensor data collected during a
single trip (i.e., between ignition on and off)
Slide 72
The Pothole Patrol: Using a Mobile Sensor Network for Road
Surface Monitoring Jakob Eriksson, Lewis Girod, Bret Hull, Ryan
Newton, Samuel Madden, Hari Balakrishnan MIT Computer Science and
Artificial Intelligence Laboratory Mobisys 2008
Slide 73
Mobile Road Surface Monitoring Potholes: hazardous to drivers
and increasing repair costs due to vehicle damage Determine which
roads need to be fixed Static sensors will not do well requires
mobility! Challenges: Differentiate potholes from other road
anomalies (railroad crossings, expansion joints) Coping with
variations in detecting the same pothole (speed, sensor
orientation)
Slide 74
Pothole Patrol: Architecture Pothole Record Clustering Cab 1
GPS 3 Axis Accelerometer Location Interpolator Pothole Detector Cab
2 GPS 3 Axis Accelerometer Location Interpolator Pothole Detector
Central Server
Detection Algorithm Features of accelerometer data High energy
events are potholes? Not really! Rail road crossings, expansion
joints, door slamming are high energy events Accelerometer data is
processed by embedded computer 256-sample windows Pass through 5
different filters
Slide 77
Detection Algorithm Speed Car is not moving or moving slowly
Rejects door slam and curb ramp events IN Windows of all event
classes SpeedHigh-passz-peak xz-ratio speed vs. z ratio OUT Pothole
Detections
Slide 78
Detection Algorithm High-Pass Removes low-freq components in X-
and Z-axes Filters out events like turning, veering, braking. IN
Windows of all event classes SpeedHigh-passz-peak xz-ratio speed
vs. z ratio OUT Pothole Detections
Slide 79
Detection Algorithm z-peak Prime characteristic for minor
anomalies Rejects all windows with absolute z-acceleration < t z
IN Windows of all event classes SpeedHigh-passz-peak xz-ratio speed
vs. z ratio OUT Pothole Detections tztz
Slide 80
Detection Algorithm xz- ratio Assumes potholes impact only side
of the vehicle Identifies anomalies that span width of the road
(rail crossings, speed bumps) high z-peak + low x-peak vs. high
z-peak + high x-peak (potholes) x peak (within w (=32) samples from
z peak ) < t x * z peak IN Windows of all event classes
SpeedHigh-passz-peak xz-ratio speed vs. z ratio OUT Pothole
Detections tztz txtx
Slide 81
Detection Algorithm speed vs. z ratio At high speeds, small
anomalies cause high peak accelerations Rejects windows where Z
peak < t s X speed IN Windows of all event classes
SpeedHigh-passz-peak xz-ratio speed vs. z ratio OUT Pothole
Detections tztz txtx tsts
Slide 82
Detection Algorithm Filtering parameter threshold tuning via
training Thresholds: z-peak, xz-ratio, and speed vs. z ratio
Perform exhaustive search over a range of values Improving
detection accuracy with clustering Cluster of at least k events
must happen in the same location with small margin of error(d)
Clustering algorithm Place each detection in d X d grid. Compute
pair-wise distances in same or neighboring grid cells Iteratively
merge pairs of distances in order of distance Max intra cluster
distance < t Reported location is the centroid of the
locations
Slide 83
Performance on labeled data Randomly divided into training set
and test set Cf: loosely labeled data (extra information about
manholes, bridges, etc.) Performance ClassHand Labeledw/ Loosely
Labeled Pothole88.9%92.4% Manhole0.3%0.0% Expansion joints2.7%0.3%
Railroad Crossing8.1%7.3%
PIER: Personal Environmental Impact Report, as a Platform for
Participatory Sensing Systems Research Min Mun, Sasank Reddy, Katie
Shilton, Nathan Yau, Jeff Burke, Deborah Estrin, Mark Hansen, Eric
Howard, Ruth West, Peter Boda UCLA, Nokia Mobisys 2009
Slide 86
Participatory Sensing Distributed data collection and analysis
at the personal, urban, and global scale Individuals and
communities make decisions about when and how to Capture Store
Access Analyze Share Participants use mobile phones to gather data
and web services to aggregate and interpret the assembled
information.
Slide 87
Health and Wellness: Personal Environmental Impact Report CO2
emissionsFast food exposure PM2.5
Slide 88
Personal Environmental Impact Report Carbon Impact: a measure
of transportation- related carbon footprint Sensitive Site Impact:
a users transportation related airborne particulate matter
emissions near sites with populations sensitive to it Smog
Exposure: a users transportation-related exposure to particulate
matter emissions Fast Food Exposure: the time integral of proximity
to fast-food eating establishment Ultimately we want people to take
notice of impact and exposure and be able to start conversations.
The absolute numbers are not whats key, but instead, trends over
time. How can one reduce impact and minimize exposure?
Slide 89
PEIR as a Participatory Sensing System "Sensing Pollution
without Pollution-Sensors" Existing Infrastructure Annotation
/Inferences Scientific Models Activity Classification e.g.,
staying, walking, driving GIS Data Annotation e.g. weather, traffic
Impact and Exposure Calculation Data Aggregation Tracklog format
School,hospital,fast food restaurant locations Weather, traffic
data User profile
Slide 90
PEIR as a Participatory Sensing System
Slide 91
Slide 92
SoundSense: Scalable Sound Sensing for People-Centric
Applications on Mobile Phones Hong Lu, Wei Pan, Nicholas D. Lane,
Tanzeem Choudhury, Andrew T. Campbell Dept. of Computer Science,
Dartmouth College Mobisys 2009
Slide 93
SoundSense allows us to sense people and the environment
Slide 94
SoundSense Architecture Admission Control Acoustic Features
Decision Tree Classifier Markov Model Recognizer Ambient Sound
Learning Voice Analysis Music Analysis Sound Waveform Framing &
Admission Control Feature Extraction Coarse Category Classification
Intra-Category Classification
Slide 95
Admission Control Filter out silence and non-informative frames
Criteria: Energy & Spectral Entropy
Slide 96
Discriminative Acoustic Features Frequency Spectrum based
features: Robust to distance/muffling Computationally very cheap
after initial FFT
Slide 97
Discriminative Acoustic Features Spectral Rolloff for Music and
Voice Spectral Rolloff Time
Slide 98
Coarse Category Classification Acoustic Features Decision Tree
Classifier Ambient Sound Voice Music S1S2 S3 S1S2 S3 S1S2 S3 Markov
Model Recognizer Music, Music, Voice, Music, Voice, Music, Music
Music, Music, Music, Music, Music, Music, Music Confusion matrix
for the decision tree classifier with Markov model smoothing
Slide 99
Learning Significant Sound Unique to the individual Not
practical to build supervised classifiers for all the potential
significant sounds Unsupervised learning! Examples: Mystery Sound 1
Car (turning signal) Mystery Sound 2 Vacuum Cleaner Incrementally
build models of encountered sounds Rank them by order of importance
Human-in-the-loop labeling
Slide 100
Evaluation Friday Saturday
Slide 101
: Privacy by Design - Principles of Privacy-Aware Ubiquitous
Systems, Marc Langheinrich, Ubicomp 2001 Location Disclosure to
Social Relations: Why, When, & What People Want to Share,
Consolvo et al., CHI'05 duty-cycling trade-offs (WiFi vs. 3G) :
Decomposing power measurements for mobile devices, Andrew Rice and
Simon Hay, Percom 2010 Coolspots: Reducing the power consumption of
wireless mobile devices with multiple radio interfaces, Pering et
al., Mobisys 2006 Activity classification activity classification :
Activity Recognition from User-Annotated Acceleration Data,
Pervasive 2004
Slide 102
network infra , , Vehicular Sensing People Sensing