Accumulo Summit 2015: Preventing Bugs: How PHEMI Put Accumulo To Work in the Field [Geo]

Preventing BugsPutting Accumulo To Work In The Field

Russ Weeks, PHEMI

Orchard

Precision Agriculture

Sensors

Gateways

Why We ❤ Accumulo

• Continuous Aggregations

• Time-series Data

• Spatiotemporal Data

Farming is Risky!

Adrian Baggaley, FruitForum

Horticultural Sciences Department, University of Florida

USDA

Aggregation Schema

Time-Series Data

• Sensor messages arriving in constant, high-velocity stream

• Common use case: “Show me the last 2 weeks of data for sensor X”

No problem!

Row CF CQ Val

M0 msg timestamp 2014/07/01T19:30

M0 msg sensor_id 1138

M0 msg msg_id 1138:2014/07/01T1930

M0 dev battery 3.3v

M0 env temp 20.3c

M0 env windspd 2.96m/s

Lexicode + Transpose…

Row CF CQ Val

msg:msg_id:1138:2014/07/01T2020 M842 1

msg:msg_id:1138:2014/07/01T2010 M767 1

msg:msg_id:1138:2014/07/01T2000 M684 1

msg:msg_id:1138:2014/07/01T1950 M592 1

msg:msg_id:1138:2014/07/01T1940 M356 1

msg:msg_id:1138:2014/07/01T1930 M0 1

• But… my old database could dereference those message IDs automatically!

• All I had to do was this join, see…

Time-Series Index

Row CF CQ Val(sensor_id:1138,2014/07/01T2020,battery,3.3v) […] M842 1(sensor_id:1138,2014/07/01T2020,windspd,1.83m/s) […] M842 1(sensor_id:1138,2014/07/01T2020,temp,18.6c) […] M842 1(sensor_id:1138,2014/07/01T2010,battery,3.2v) […] M767 1(sensor_id:1138,2014/07/01T2010,windspd,1.51m/s) […] M767 1(sensor_id:1138,2014/07/01T2010,temp,18.8c) […] M767 1

Spatiotemporal

Map Tiles: OpenStreetMap

Pos. 0 Pos. 212-1

Spatiotemporal

OK, But Why Not HBase?

• Iterators are awesome

• Customer is interested in cell-level security

• Future applications may involve controlled data-sharing

Thanks!