The Claremont Reporton Database Research

2009-10-28淡江大學 周清江

2

Background Senior database researchers have gathered every

few years to assess the state of database research and to recommend problems and problem areas deserve additional focus. Laguna Beach, Calif. in 1989 Palo Alto, Calif. (“Lagunita”) in 1990 and 1995 Cambridge, Mass. in 1996 Asilomar, Calif. in 1998 Lowell, Mass. in 2003 Claremont, Calif. in 2008

3

New Focus Areas

New database engine architectures Declarative programming languages Interplay of structured and unstructured data Cloud data services Mobile and virtual worlds

4

A Turning Point in Database Research

Unusually rich opportunities for Technical advances, intellectual achievement,

entrepreneurship, and impact on science and society

Sense of change as a function of several factors Breadth of excitement about Big data Data analysis as a profit center Ubiquity of structured and unstructured data Expanded development demand Architecture shift in computing

5

Research Portfolio Change

Impact and Breadth Evaluated by external measures

Helping new classes of users Powering new computing platforms Making conceptual breakthroughs across computing

6

Two Promising Approaches

Reformation Deconstucting core data-centric ideas and systems Reforming for new applications and architectural realit

ies

Synthesis Leverage good research ideas that have yet to develo

p identifiable, agreed-upon system architectures Data integration, information extraction, data privacy, etc.

7

Research Opportunities

Revisiting Database Engines Declarative Programming for Emerging Platforms The Interplay of Structured and Unstructured

Data Cloud Data Services Mobile Applications and Virtual Worlds

8

Research Opportunities

Main issues cut across the above topics Management of uncertain information data privacy and security e-science and other scholarly applications human centric interactions with data social networks and Web 2.0 personalization and contextualization of query- and

search-related tasks streaming and networked data self-tuning and adaptive systems, and the challenges raised by new hardware technologies and

energy constraints

9

Revisiting Database Engines

Data-intensive tasks for which relational DBs provide poor price/performance Ex: text indexing, serving web pages, media delivery

Room for significant innovation within traditional application domains Analytics for business and science

The cost of software and management relative to hardware is exorbitant

OLTP Need to address data lifecycle issues

Data provenance, schema evolution, and versioning

Good time to try radical ideas

10

Revisiting Database Engines

Two directions of research projects Revolutionary steps in DB system architecture

Broadening the range of applicability Radically improving performance by designing special

purpose DB systems for specific domains

These efforts may be synergistic

11

Revisiting Database Engines

Important research topics in the core DB engine Designing systems for clusters of many-core processors Exploiting remote RAM and Flash as persistent media Treating query optimization and physical data layout as a uni

fied, adaptive, self-tuning task to be carried out continuously Compressing and encrypting data at the storage layer, integr

ated with data layout and query optimization Designing systems for non-relational data models Trading off consistency and availability for better performanc

e and scaleout to thousands of machines Designing power-aware DBMS that limit energy costs withou

t sacrificing scalability

12

Declarative Programming for Emerging Platforms

The urgency of programmer productivity is increasing exponentially as programmers target even more complex environments

No-expert programmers need to be write robust code that scales out across processors in both loosely- and tightly-coupled architectures

13

Declarative Programming for Emerging Platforms

Example: Map-Reduce New declarative languages, based on Datalog, have been d

eveloped for a variety of domain-specific systems Network and distributed systems, computer games, machine le

arning and robotics, compilers, security protocols, and information extraction

Enterprise application programming Ruby on Rails (

http://www.ithome.com.tw/itadm/article.php?c=46863, http://en.wikipedia.org/wiki/Ruby_on_Rails ) LINQ (Language-Integrated Query,

http://www.ithome.com.tw/itadm/article.php?c=44337, http://en.wikipedia.org/wiki/Language_Integrated_Query )

14

Declarative Programming for Emerging Platforms

Research questions Language design

Fairly expressive Attractive syntax, typing and modularity, development tool

s, smooth interactions with the rest of the computing ecosystem

Efficient compilers and runtimes Techniques to optimize code automatically

Across both the horizontal distribution of parallel processors and the vertical distribution of tiers

Should extend techniques behind parallel and distributed DBs

15

The Interplay of Structured and Unstructured Data

Within enterprises, heterogeneous collections of structured data linked with unstructured data

On Web, structured data from Millions of DBs hidden behind forms (deep web) High quality data items in HTML tables on web pa

ges, and mashups providing dynamic views on structured data

Data contributed by Web 2.0 services Photo and video sites Collaborative annotation services On-line structured data repositories

16

The Interplay of Structured and Unstructured Data

Challenges of managing dataspaces Managing a rich collection of structured, semi-stru

ctured, and unstructured data On the web, previous contributions

Techniques for domain-specific search engines Domain-independent tech for crawling through for

ms, and surfacing the resulting HTML pages in a search-engine index

Within enterprises, enterprise search and discovery of relationships between structured and unstructured data

17

The Interplay of Structured and Unstructured Data

Challenge 1 Extract structure and meaning from unstructured

and semi-structured data Applying and managing predictions from large numbers

of independently developed extractors Need algorithms to introspect about the correctness of

extractions Better technology to manage data in context

Discover data sources Discover implicit relationships Determine the weight of an object’s context when

assigning it semantics Maintain data provenance

18

The Interplay of Structured and Unstructured Data

Challenge 2 Develop methods for effectively querying and deriving

insight from the resulting sea of heterogeneous data Analyze keyword query to extract its intended semantics Route the query to relevant sources

Do not assume we have semantic mappings for the data sources Cannot assume that the domain of the query or data sources is

known The system should provide best-effort service and improve over

time Develop index structures to support querying hybrid

data Need new notions of correctness and consistency to provide

metrics and to make cost/quality tradeoffs

19

The Interplay of Structured and Unstructured Data

Challenge 2 Innovation about creating data collections

Web 2.0 Users join ad-hoc communities to create, collaborate, curate,

and discuss data online They rarely agree on schemata ahead of time Schemata need to be inferred from the data and will be highly

dynamic Schemata will be used to guide users to consensus

Need to incorporate visualizations effectively They need to be easy to use

20

Cloud Data Services

Infrastructure change Service-oriented cloud computing

Application services (salesforce.com) Storage services (Amazon S3) Compute services (Google App Engine, Amazon EC2) Data services (Amazon SimpleDB, MS SQL Server Data Services,

Google Datastore) Trade-off between functionality and operational costs Manageability is particularly important

Limited human intervention High-variance workloads: elastic provisioning A variety of shared infrastructures: service tuning depends on ho

w the shared infrastructure is virtualized Urgency of self-managing DB technologies

21

Cloud Data Services

Challenges from scale of cloud computing SQL databases cannot scale to thousands of nodes

Different transactional implementation techniques? Different storage semantics?

More work is needed to synthesize ideas from the literature in cloud computing

Limitations on either the plan space or the search will be required

How programmers will express their programs in the cloud

22

Cloud Data Services

Challenges from scale of cloud computing Data security and privacy

Key to success: target usage scenarios in the cloud New scenarios will emerge with their own challenges

Specialized services pre-loaded with large data-sets “Mash up” data from public and private domains Services reaching out across clouds

Prevalent in scientific data “grids” Federated cloud architectures will enhance the challenges

23

Mobile Applications and Virtual Worlds

This new class of applications need to manage diverse user-created data, synthesize it intelligently, and provide real-time services

Trends in the mobile space Platforms to build mobile applications are mature The emergence of mobile search and social networks suggest a

new set of mobile applications Virtual worlds, like Second Life, increasingly blur the

distinctions with the real world Suggest a more data-rich mixture (co-space)

Applications include rich social networking, massive multi-player games, military training, edutainment and knowledge sharing

24

Mobile Applications and Virtual Worlds

New challenges The need to process heterogeneous data streams to

materialize real-world events The need to balance privacy against the collective

benefit of sharing personal real-time information The need for more intelligent processing to send

interesting events in the co-space to someone in the physical world

25

Moving Forward Survey articles and tutorials are becoming an

increasingly important contribution Risky or speculative papers not championed effectively A need for approachable books on scalable data

management algorithms and techniques Time is ripe for projects to stimulate collaboration and

cross-fertilization of ideas, like information integration Two areas are identified for competitions

System components for cloud computing Large-scale information extraction