11

Knowledge Grid and Knowledge Grid and Cooperative ComputingCooperative Computing

窦 万 春窦 万 春南京大学计算机科学与技术系南京大学计算机科学与技术系

Douwc@nju.edu.cnDouwc@nju.edu.cn

22

ContextContext

Section 1: Section 1:

Concepts,State of art, Tendency, Applications related Concepts,State of art, Tendency, Applications related

to to Grid ComputingGrid Computing

Section 2: Section 2:

Concepts,State of art, Tendency, Applications related Concepts,State of art, Tendency, Applications related

to to Knowledge GridKnowledge Grid

Section 3:Section 3:

Cooperative ComputingCooperative Computing in Knowledge Grid in Knowledge Grid

environmentsenvironments

33

Section 1: Grid ComputingSection 1: Grid Computing

44

What is “Grid Computing”?What is “Grid Computing”?

Grid Computing = Grid + ComputingGrid Computing = Grid + Computing What is Grid?What is Grid? What is ComputingWhat is Computing

55

What is Grid?What is Grid?

66

77

What Defines The Internet What Defines The Internet

““We will probably see the spread of ‘computeWe will probably see the spread of ‘computer utilities, which, like present electric and telepr utilities, which, like present electric and telephone utilities, will service individual homes anhone utilities, will service individual homes and offices across the country”d offices across the country”

-Len Kleinrock. 1969 [1]-Len Kleinrock. 1969 [1]

88

Let begin…Let begin…

IBM 对网格计算的技术理解：

99

Ian Foster:Professor of Computer ScienceThe University of ChicagoDepartment of Computer ScienceUniversity of Chicago

Ryerson Hall, Room 250 1100 E. 58th St. Chicago, Illinois 60637 312-702-3487 (OFFICE)312-702-8487 (FAX)foster@cs.uchicago.edu

1010

What is Grid?What is Grid?

““A computational grid is a hardware and softwaA computational grid is a hardware and software infrastructure that provides dependable, core infrastructure that provides dependable, consistent, pervasive, and inexpensive access to nsistent, pervasive, and inexpensive access to high-end computational capabilities.”high-end computational capabilities.”

- Ian Foster and Carl Kesselman, 1998- Ian Foster and Carl Kesselman, 1998

1111

Do you know…Do you know…

20032003 年年 1010 月中国国家教育部先后与月中国国家教育部先后与 Intel Intel 和和 IBMIBM 公司公司联手共建中国教育科研网格并宣称“是迄今为止有政府联手共建中国教育科研网格并宣称“是迄今为止有政府推出的最宏大的网格工程”推出的最宏大的网格工程”

联想与曙光推出了面向网格的高性能计算机联想与曙光推出了面向网格的高性能计算机 中科院计算所开发了网格系统软件（织女星）中科院计算所开发了网格系统软件（织女星） 《福布斯》杂志预测网格技术将在《福布斯》杂志预测网格技术将在 2004-20052004-2005 年出现高年出现高

峰，到峰，到 20202020 年则将产生一个年产值达年则将产生一个年产值达 2020 万亿美元的大万亿美元的大产业产业

IBMIBM 在 在 20012001 年就 宣布投资年就 宣布投资 4040 亿美元大规模地进入网亿美元大规模地进入网格计算领域，格计算领域， 20032003 年年 1010 月又宣布投资月又宣布投资 100100 亿美元启动亿美元启动面向网格的“按需计算”计划面向网格的“按需计算”计划

…………

1212

ChinaGridChinaGrid

1313

1414

1515

Context of GridContext of Grid

网格是一门技术网格是一门技术 网格是一种标准网格是一种标准 网格是一种方法（实现资源共享的方法）网格是一种方法（实现资源共享的方法） 网格是一种中间件网格是一种中间件 网格是一种支持高级计算的基础设施网格是一种支持高级计算的基础设施 …………

1616

General ideas about the Grid ComputingGeneral ideas about the Grid Computing

网格计算是利用互联网把分散在不同地理位置网格计算是利用互联网把分散在不同地理位置上的多个计算资源，通过逻辑关系组成一台上的多个计算资源，通过逻辑关系组成一台“虚拟的超级计算机”，而这台机器把每一台“虚拟的超级计算机”，而这台机器把每一台参与其中的计算机都作为自己的一个“结点”，参与其中的计算机都作为自己的一个“结点”，成千上万的这样的“结点”并联起来，就组成成千上万的这样的“结点”并联起来，就组成了“一个有超级计算能力的网格”。每一位将了“一个有超级计算能力的网格”。每一位将自己的计算机连接到网格上的用户，也就“拥自己的计算机连接到网格上的用户，也就“拥有了”这台超级计算机，然后就可以随时随地有了”这台超级计算机，然后就可以随时随地地调用其中的计算和信息资源，在获得一体化地调用其中的计算和信息资源，在获得一体化信息服务的同时，最大限度的实现资源共享。信息服务的同时，最大限度的实现资源共享。

1717

Basic ConceptsBasic Concepts

网格网格 (Grid)(Grid) 网格计算网格计算 计算网格计算网格 资源网格 资源网格 数据网格数据网格 信息网格信息网格 知识网格知识网格 网格计算机 网格计算机 …………

xxxxxx 网格网格

网格网格 xxxxxx

E-Science

P2P

Semantic Web

Knowledge Grid

1818

Ian Foster’s Three Point Checklist (2002)Ian Foster’s Three Point Checklist (2002)

A Grid is a system that:A Grid is a system that:

1.1. ““coordinates resources that are not coordinates resources that are not

subject to centralized control”subject to centralized control”

2.2. ““using standard, open, general-purpose using standard, open, general-purpose

protocols and interfaces”protocols and interfaces”

3.3. ““to deliver nontrivial quantities of service”to deliver nontrivial quantities of service”

1919

2 What is Computing2 What is Computing

2020

Paradigms of ComputingParadigms of Computing Distributed ComputingDistributed Computing Cooperative ComputingCooperative Computing Agile ComputingAgile Computing Pervasive/Ubiquitous ComputingPervasive/Ubiquitous Computing Service ComputingService Computing Mobile ComputingMobile Computing Grid ComputingGrid Computing Dependable Computing Dependable Computing Parallel ComputingParallel Computing Autonomic/Autonomous Computing Autonomic/Autonomous Computing ……

xx Computingxx Computing

2121

Further ComprehensionFurther Comprehension

Pervasive Computing:Pervasive Computing:

Context-Aware + Transparent +ConsistencyContext-Aware + Transparent +Consistency Autonomic/Autonomous ComputingAutonomic/Autonomous Computing

Self-*Self-*:: Self-Management, Self-Governing,Self- Self-Management, Self-Governing,Self-Organization, Self-Configuration, Organization, Self-Configuration, self-optimization, self-optimization, self-healing, and self-protectionself-healing, and self-protection

Dependable Computing:Dependable Computing:

Available + Reliable + Safe + SecureAvailable + Reliable + Safe + Secure

2222

Essence of ComputingEssence of Computing

Computing = Computed BehaviorComputing = Computed Behavior

What is a behavior?What is a behavior?

From sociological view of point…From sociological view of point…

Precondition: FormalizationPrecondition: Formalization

Role: The bridge of theory and experimentRole: The bridge of theory and experiment

2323

中文译文中文译文 理论、计算与实验是科学领域中最为基本的研究手理论、计算与实验是科学领域中最为基本的研究手段和应用方式。围绕以“计算”段和应用方式。围绕以“计算” (Computing)(Computing) 为主为主题的理论与应用研究是计算机科学与应用领域中的题的理论与应用研究是计算机科学与应用领域中的核心问题之一。在计算机科学与应用领域中，计算核心问题之一。在计算机科学与应用领域中，计算的概念往往具有非常泛化的含义，所有针对具体问的概念往往具有非常泛化的含义，所有针对具体问题求解的行为和操作都可以看作是计算的具体表现题求解的行为和操作都可以看作是计算的具体表现形式；因此，从本质上而言，计算过程是一种基于形式；因此，从本质上而言，计算过程是一种基于特定理论基础与应用技术的计算机化的运算行为或特定理论基础与应用技术的计算机化的运算行为或任务执行过程。针对特定问题的计算工具、计算环任务执行过程。针对特定问题的计算工具、计算环境都可以为计算主体提供有效的技术支撑。而计算境都可以为计算主体提供有效的技术支撑。而计算机化的问题求解环境，则为计算过程的有序开展提机化的问题求解环境，则为计算过程的有序开展提供了非常有效的技术手段和环境支持。供了非常有效的技术手段和环境支持。

2424

xx xx GridGrid GridGrid xx xx xx xx Computing Computing

Grid Computing

2525

The definition of Grid Computing in our discussionThe definition of Grid Computing in our discussion

The The Grid ComputingGrid Computing is the process of is the process of implementation of a implementation of a computed computed behaviorbehavior, which mainly designed, , which mainly designed, developed, deployed, executed in developed, deployed, executed in GridGrid environments based on certain environments based on certain application scenarios. application scenarios.

2626

Essence of Grid Computing Essence of Grid Computing

Target: Enhance the Computing PowerTarget: Enhance the Computing Power

Infrastructure: Grid Infrastructure: Grid

2727

Grid: An Evolution, not revolutionGrid: An Evolution, not revolution Source: IBM Grid ComputingSource: IBM Grid Computing

Grid can be seen as the latest and most complete evolution of more familiarGrid can be seen as the latest and most complete evolution of more familiar development.development.

Like the Web:Like the Web: Grid keeps complexity hidden: multiple users enjoy a single unified experience.Grid keeps complexity hidden: multiple users enjoy a single unified experience. Unlike the Web:Unlike the Web: enables full collaboration toward real business goal. enables full collaboration toward real business goal. Like Peer-to-PeerLike Peer-to-Peer It allows user to share files.It allows user to share files. Unlike Peer-to-PeerUnlike Peer-to-Peer Not only files, but everything which could be shared .Not only files, but everything which could be shared . Like Clusters and distributed computingLike Clusters and distributed computing It bring computing resource together.It bring computing resource together. Unlike Clusters and distributed ComputingUnlike Clusters and distributed Computing Grid can be geographically distributed and heterogeneous.Grid can be geographically distributed and heterogeneous. Like Virtualization technologiesLike Virtualization technologies enables virtualization of IT resources.enables virtualization of IT resources. Unlike Virtualization technologiesUnlike Virtualization technologies It can enable virtualization of vast and disparate resources.It can enable virtualization of vast and disparate resources.

2828

Originally Targeted ApplicationsOriginally Targeted Applications

What types of applications will grid be What types of applications will grid be used for ?used for ?

Distributed SupercomputingDistributed Supercomputing On-demand ComputingOn-demand Computing NetSolve, large archivesNetSolve, large archives Data-Intensive ComputingData-Intensive Computing

SloanSloan Digital Sky Survey, Weather forecastingDigital Sky Survey, Weather forecasting Collaborative ComputingCollaborative Computing Insors, GriPhyN, SciRUNInsors, GriPhyN, SciRUN

2929

Top 500 Supercomputers (June 2003)Top 500 Supercomputers (June 2003)

Earth Simulator: NEC : Yokohama : 35.86 TFlops

ASCI Q: LANL: Los Alamos: HP Alphaserver SC: 13.88 TFlops

MCR Linux Cluster: LLNL Livermore, 7.634 TFlops

ASCI White: LLNL, Livermore IBM SP Power3, 7.304 TFlops

Seaborg: NERSC/LBNL, Berkeley, IBM SP Power3, 7.303 TFlops

Source : http://www.top500.org

3030

General highlights from Top 500 (June General highlights from Top 500 (June 2003)2003)

157 systems reported to have peak performance above 1 157 systems reported to have peak performance above 1 TFlops.TFlops.

Total accumulated performance is 375 TFlops. ( up from 293 Total accumulated performance is 375 TFlops. ( up from 293 TFlops )TFlops )(floating point operations per second)(floating point operations per second)

Entry level performance is 245.1 GFlops. (Up from 195.8)Entry level performance is 245.1 GFlops. (Up from 195.8) A Total of 119 systems (up from 56) uses Intel processors.A Total of 119 systems (up from 56) uses Intel processors.

149 systems are now labeled as clusters ( up from 53 )149 systems are now labeled as clusters ( up from 53 )

23 of them are self-made ( Up from 14 )23 of them are self-made ( Up from 14 )

Among top 10, 7 from US, 2 from Japan, 1 from France.Among top 10, 7 from US, 2 from Japan, 1 from France.

3131

Economics and ControlEconomics and Control

The infrastructures are very expensive and The infrastructures are very expensive and require years of hard work. require years of hard work.

The shear force of economics will require The shear force of economics will require that these resources are under strict control that these resources are under strict control and are optimally utilized.and are optimally utilized.

Many times freedom is costly and chaotic.Many times freedom is costly and chaotic. This is the starting what we call This is the starting what we call Grid Grid

ComputingComputing

3232

Changing face of Enterprise ComputingChanging face of Enterprise Computing

Most of the recent, enterprise systems are Most of the recent, enterprise systems are collection of heterogeneous resources. collection of heterogeneous resources.

Quality of services traditionally associated with Quality of services traditionally associated with mainframe centric computing are now essential to mainframe centric computing are now essential to the effective conduct of e-business across the effective conduct of e-business across distributed resources, inside as well as outside the distributed resources, inside as well as outside the enterprise.enterprise.

Recently there is upsurge of services providers of Recently there is upsurge of services providers of various types such as web-hosting SP, storage SP, various types such as web-hosting SP, storage SP, application SPapplication SP

All these require standardization.All these require standardization.

3333

网格技术的应用范例网格技术的应用范例 计算网格计算网格 资源网格 资源网格 数据网格数据网格 信息网格信息网格 知识网格知识网格 网格工作流应用网格工作流应用 虚拟组织虚拟组织 …………

3434

小结论小结论 Killer application:Killer application: 高性能技术领域高性能技术领域 ;; ““ 造房不如买房造房不如买房 ,, 买房不如租房买房不如租房”的应用思想”的应用思想 ;; 首先需要一定的基础设施首先需要一定的基础设施 ;; 在特定应用需求情况下的一种计算模式在特定应用需求情况下的一种计算模式 ;; 技术发展到一定阶段的集成模式技术发展到一定阶段的集成模式 (( 应用驱动应用驱动 );); 现有技术的应用挑战现有技术的应用挑战 ;; 泛在协同的应用思想泛在协同的应用思想 ;;

3535

Development PathDevelopment Path

3636

1. 1. 计算机的性能已经提高了计算机的性能已经提高了 8080 亿亿倍倍

3737

2. 2. 但应用的需求增长得更快但应用的需求增长得更快

3838

3. 3. 人们建造越来越大的超级计算机人们建造越来越大的超级计算机

3939

4. 4. 但超级计算机造起来很麻烦但超级计算机造起来很麻烦

4040

5. 5. 集群计算机很容易造集群计算机很容易造

4141

6. 6. 但它的规模也不能无限扩张但它的规模也不能无限扩张

4242

7. P2P7. P2P 计算解决了规模问题计算解决了规模问题

4343

8. 8. 但大多数问题割不成但大多数问题割不成 P2PP2P 喜欢的样喜欢的样子子

4444

9. 9. 网格虚拟出空前的超级计算机网格虚拟出空前的超级计算机

成为下一代 成为下一代 InternetInternet 的发展方向的发展方向

4545

10. 10. 但它是不是最终的必杀武器？但它是不是最终的必杀武器？

…………

4646

Section 2: Knowledge Grid (KG)

4747

What is KGWhat is KG

Fran Berman put it forward that a Fran Berman put it forward that a Knowledge Grid is the convergence of Knowledge Grid is the convergence of a comprehensive computational a comprehensive computational infrastructure along with the scientific infrastructure along with the scientific data collections and applications for data collections and applications for routinely supporting the synthesis of routinely supporting the synthesis of knowledge from that data. knowledge from that data.

4848

Cont.Cont.

Hai Zhuge believed that the Knowledge GHai Zhuge believed that the Knowledge Grid is an intelligent, sustainable Internet arid is an intelligent, sustainable Internet application that enables people or virtual rpplication that enables people or virtual roles (mechanisms that facilitate interoperoles (mechanisms that facilitate interoperation among users, applications, and resoation among users, applications, and resources) to effectively urces) to effectively capturecapture, , publishpublish, , sharsharee, and , and managemanage explicitexplicit knowledge resourc knowledge resources. es.

4949

Five issues related to KGFive issues related to KG

The first is theories, models, methods, and The first is theories, models, methods, and mechanisms for capturing and representing mechanisms for capturing and representing knowledge. knowledge.

The second issue is knowledge visualization and The second issue is knowledge visualization and innovation. innovation.

The third issue is effective propagation and The third issue is effective propagation and management of knowledge in dynamic virtual management of knowledge in dynamic virtual organizations. organizations.

The fourth issue is effective knowledge organization, The fourth issue is effective knowledge organization, evaluation, refinement, and derivation. evaluation, refinement, and derivation.

The fifth is knowledge association and integration. The fifth is knowledge association and integration. Based on those ideas, a Knowledge Grid should Based on those ideas, a Knowledge Grid should

synthesize the integration of the data, computing, synthesize the integration of the data, computing, and the network hardware, the development of the and the network hardware, the development of the software, and the coordination of a large and software, and the coordination of a large and distributed human infrastructure. distributed human infrastructure.

5050

The main function of Knowledge Grid The main function of Knowledge Grid is to synthesize knowledge from data is to synthesize knowledge from data by means of mining and reference, to by means of mining and reference, to enable search engines to make enable search engines to make references, answer questions, and to references, answer questions, and to draw conclusions from masses of data. draw conclusions from masses of data.

5151

Knowledge Grid provides a general, Knowledge Grid provides a general, basic, and public knowledge service basic, and public knowledge service infrastructure for E-Science. It infrastructure for E-Science. It prescribes an ideal application prescribes an ideal application paradigm that would prevail in future paradigm that would prevail in future as advocated by some research [ as advocated by some research [

5252

Feature of Grid: Feature of Grid: Power of ComputingPower of Computing KG is a kind of grid, which focus on KG is a kind of grid, which focus on

knowledge managementknowledge management during during computing.computing.

Knowledge Grid = Grid Computing Knowledge Grid = Grid Computing based on Knowledge Management based on Knowledge Management

Feature of KG: Feature of KG: Power of KM during Power of KM during computing.(computing.(static and dynamicstatic and dynamic))

5353

Taxonomy of KnowledgeTaxonomy of Knowledge

Explicit KnowledgeExplicit Knowledge Tacit KnowledgeTacit Knowledge Problem solving is a reifying process from Problem solving is a reifying process from

explicit knowledge to tacit knowledge explicit knowledge to tacit knowledge OntologyOntology Workflow management to knowledge flow Workflow management to knowledge flow

managementmanagement

5454

Section 3 Cooperative ComputingSection 3 Cooperative Computing

5555

Basic ConceptsBasic Concepts

WKFP:WKFP:

WWeb-based eb-based KKnowledge nowledge FFlow low PPeereer WKFPD:WKFPD:

WWeb-based eb-based KKnowledge nowledge FFlow low PPeer eer DDirectory irectory

5656

5757

A Motivated ExampleA Motivated Example

5858

Organization Scenarios Organization Scenarios

5959

Role DefinitionsRole Definitions

6060

Performance Scenarios Performance Scenarios

6161

6262

Design ParadigmDesign Paradigm

6363

6464

Key IssuesKey Issues

Usage PolicyUsage Policy Role DefinitionRole Definition Context-AwareContext-Aware Engine-DrivenEngine-Driven Role-Based Cooperative ComputationRole-Based Cooperative Computation

6565

Q&AQ&A

Thanks a lot!Thanks a lot!