Molecular Dynamics Simulation - TUM · 2006-10-19 · • modelling aspects of molecular dynamics simulations: – why to leave the classical continuum mechanics point of view? –

Introduction

Examples

Essentials from . . .

Molecular Dynamics – . . .


MD – Approximations . . .

MD – Implement...

MD – Parallelisation


Numerical Methods for . . .

Page 1 of 124

Algorithmen des WissenschaftlichenRechnens II

1. Molecular DynamicsSimulation

Hans-Joachim Bungartz

1) Molecular Dynamics Simulation

Introduction

Examples





MD – Implement...




Page 2 of 124




1.1. Introduction

Remember what Scientific Computing deals with:From phenomena to predictions!

phenomenon, process etc.

mathematical model�

modelling

numerical algorithm� numerical treatment

simulation code�

implementation

results to interpret� visualization

�� embedding

statement tool

�

�

�validation

Introduction

Examples





MD – Implement...




Page 3 of 124




Overview

• modelling aspects of molecular dynamics simulations:

– why to leave the classical continuum mechanics point of view?

– where appropriate?

– which models, i.e. which equations?

• numerical aspects of molecular dynamics simulations?

– how to discretize the resulting modelling equations?

– efficient algorithms?

• implementation aspects of molecular dynamics simulations?

– suitable data structures?

– parallelisation?

Introduction

Examples





MD – Implement...




Page 4 of 124




Hierarchy of Models

Different points of view for simulating human beings:

issue level of resolution model basis (e.g.!)global increasein population

countries, regions population dynamics

local increase inpopulation

villages, individuals population dynamics

man circulations, organs system simulatorblood circulation pump/channels/valves network simulatorheart blood cells continuum mechanicscell macro molecules continuum mechanicsmacromolecules

atoms molecular dynamics

atoms electrons or finer quantum mechanics

Introduction

Examples





MD – Implement...




Page 5 of 124




Scales – an Important Issue

• length scales in simulations:

– from 10−9m (atoms)– to 1023m (galaxy clusters)

• time scales in simulations:

– from 10−15s

– to 1017s

• mass scales in simulations:

– from 10−24g (atoms)– to 1043g (galaxies)

• obviously impossible to take all scales into acount in an explicitand simultaneous way

• first molecular dynamics simulations reported in 1957

Introduction

Examples





MD – Implement...




Page 6 of 124




Applications for Micro and Nano Simulations

Lab-on-a-chip, used in brewing technology (Siemens)

Introduction

Examples





MD – Implement...




Page 7 of 124





Flow through a nanotube (where the assumptions of continuummechanics are no longer valid)

Introduction

Examples





MD – Implement...




Page 8 of 124





Protein simulation: actin, important component of muscles (overlayof macromolecular model with electron density obtained by X-ray

crystallography (brown) and simulation (blue))

Introduction

Examples





MD – Implement...




Page 9 of 124





Protein simulation: human haemoglobin (light blue and purple:alpha chains; red and green: beta chains; yellow, black, and darkblue: docked stabilizers or potential docking positions for oxygen)

Introduction

Examples





MD – Implement...




Page 10 of 124





Material science: hexagonal crystal grid of Bornitrid

Introduction

Examples





MD – Implement...




Page 11 of 124




A Prominent Recent Example:

• Gordon-Bell-Prize 2005 (most important annual supercomput-ing award)

• phenomenon studied: solidification processes in Tantalum andUranium

• method: 3D molecular dynamics, up to 524,000,000 atoms sim-ulated

• machine: IBM Blue Gene/L, 131,072 processors (world’s #1 inNovember 2005)

• performance: more than 101 TeraFlops (almost 30% of thepeak performance)

Introduction

Examples





MD – Implement...




Page 12 of 124




1.2. Essentials from Continuum Mechanics

Fluids

• fluid: notion covering liquids and gases

– liquids: hardly compressible– gases: volume depends on pressure– small resistance to changes of form

• continuum:

– space, continuously filled with mass– homogeneous– subdivision into small fluid voxels with constant physical

properties is possible– idea valid on micro scale upward (where we consider con-

tinuous masses and not discrete particles)

Introduction

Examples





MD – Implement...




Page 13 of 124




Lagrange and Euler Formulation

• Lagrange formulation:

– considers the change of position of material particles– example: velocities of material particles– typical for structural dynamics

• Euler formulation:

– considers a fixed point in space– example: velocity field, describing the velocities of virtual

particles at fixed positions– typical for fluid mechanics

• Arbitrary Lagrangean Eulerian (ALE) formulation:

– refers to an arbitrary configuration– example: a particle, moving with its own system of refer-

ence through a fluid, having a fixed system of reference– widely used in fluid-structure interactions

Introduction

Examples





MD – Implement...




Page 14 of 124




Description of State

• consideration of a control volume V0 (Eulerian perspective)

• description of the fluid’s state via

– the velocity field �v(�x, t) and two thermodynamical quanti-ties, typically

– the pressure p(�x, t) and– the density ρ(�x, t)

• for incompressible fluids, the density ρ is constant (if there areno chemical reactions)

Introduction

Examples





MD – Implement...




Page 15 of 124




Hard Sphere Model

• first, atoms are described in a simplified way

– as “suspended round disks” in 2D space, or– as “suspended round balls” in 3D space

• this is called the hard sphere model

• the fixed (and rather dense) arrangement in a crystal leads to asolid

• for gases, the average distances between particles are verylarge

• atoms and molecules of liquids do not have a fixed arrange-ment, but the density compared with gases is higher

Introduction

Examples





MD – Implement...




Page 16 of 124




Solids in 2D

D

½��3D

D

• the maximum density in 2D comes with a triangu-lar packing

• an ideal crystal without thermal motion provides aperfect 2D grid

• the elementary cell for disks of diameter D formsa rhombus of areaAE = 21

2D

(12

√3D

)= 1

2

√3D2 ≈ 0.866D2

• in an elementary cell, there are parts that can becombined to a complete disk:AP = π

4D2 ≈ 0.785D2

• density Cm = AP

AE=

π4

D2

12

√3D2 = π

6

√3 ≈ 0.9069

Introduction

Examples





MD – Implement...




Page 17 of 124




Solids in 3D

D��D

��D

• fill a box with balls of diameter D

• periodic arrangement is neither formed automati-cally nor unique

• face-centered cubic (fcc) lattice with basis vectorscycl

(±

√2

2D ±

√2

2D 0

)– elementary cell: brick,

VE =(√

2D)3

= 2√

2D3 ≈ 2.828D3

– VP =(61

2+ 81

8

)π6D3 = 2

3πD3 ≈ 2.094D3

– Cm = VP

VE=

23πD3

2√

2D3 = π6

√2 ≈ 0.7405

VE = volume of elementary cell

VP = volume of atoms

Cm = maximum density ratio

Introduction

Examples





MD – Implement...




Page 18 of 124




Fluids

• intermediate states for the transition solid-liquid:

– plasticity: irreversible deformation of the crystal lattice alonggliding planes

– thixotropy: liquid state after destruction of frame-like ag-glomerations of molecules

• for C0 ≈ 0.8 the disks can escape from their positions ⇒ 2Dliquid without a periodic crystal lattice; average distance of twoparticles: O(D)

• gases: smaller densities (C � C0) with an average distance ofO( D√

C)

C = density ratio of material to reference cell volume

Introduction

Examples





MD – Implement...




Page 19 of 124




Scope of Application

• number of molecules to be taken into account:

– Loschmidt number 2, 687 · 1019cm−3: number of moleculesin 1 cm3 of an ideal gas

– Avogadro constant 6.0221415·1023mol−1: number of carbon-12 atoms in 12g of carbon-12, or number of molecules in 1mol (1 mol, under normal conditions, taking a volume of22.4 litres)

– Avogadro number: notion being used in different ways forboth of the above constants, which depend on each other(2, 687 · 1019cm−3 · 22.413996 · 103cm3mol−1 = 6.0221415 ·1023mol−1)

– time steps for numerical simulations are typically in the or-der of femtoseconds (1fs := 10−15s)

• hence: scope of application is limited to nanoscale simulations(at least for the near future)

Introduction

Examples





MD – Implement...




Page 20 of 124




1.3. Molecular Dynamics – the Physical Model

Quantum Mechanics – a “Tour de Force”

• particle dynamics described by the Schrödinger equation

• its solution (state or wave function ψ) only provides probabilitydistributions for the particles’ (i.e. nuclei and electrons) positionand momentum

• Heisenberg’s uncertainty principle: position and momentum cannot be measured with arbitrary accuracy simultaneously

• there are discrete values/units (for the energy of bonded elec-trons, e.g.)

• in general, no analytical solution available

• high dimensional problems: dimensionality corresponds to num-ber of nuclei and electrons

Ψ = Ψ(R1, . . . , RN , r1, . . . , rK , t)

ψ - wave functionR - position of nucleusr - position of electront - time

Introduction

Examples





MD – Implement...




Page 21 of 124




• hence, numerical solution is possible for rather small systemsonly

• therefore, various (simplifying and approximating) approachessuch as density functional method or Hartree-Fock approach(ab-initio Molecular Dynamics, see next slide)

Introduction

Examples





MD – Implement...




Page 22 of 124




Classical Molecular Dynamics

• Quantum mechanics approximation−−−−−−−−→ classical Molecular Dynamics

• classical Molecular Dynamics is based on Newton’s equationsof motion

• molecules are modelled as particles; simplest case: point masses

• there are interactions between molecules

• multibody potential functions describe the potential energy ofthe system; the velocities of the molecules (kinetic energy) area composition of

– the Brownian motion (high velocities, no macroscopic movement),– flow velocity (for fluids)

• ab-initio Molecular Dynamics uses quantum mechanical cal-culations to determine the potential hypersurface, apart fromsemi-empirical potential functions (cf. Car Parrinello MolecularDynamics (CPMD) methods)

• total energy is constant ↔ energy conservation

Introduction

Examples





MD – Implement...




Page 23 of 124




Fundamental Interactions

• Classification of the fundamental in-teractions:

– strong nuclear force– electromagnetic force– weak nuclear force– gravity

O

rk

ri

rj

• interaction → potential energy

• the total potential of N particles is the sum of multibody poten-tials:

– U :=∑

0<i<N U1(ri) +∑

0<i<N

∑i<j<N U2(ri, rj)

+∑

0<i<N

∑i<j<N

∑j<k<N U3(ri, rj, rk) + . . .

– there are ( Nn ) = N !

n!(N−n)!∈ O(Nn) n-body potentials Un,

particulary N one-body and 12N(N − 1) two-body potentials

• force �F = −gradU

Introduction

Examples





MD – Implement...




Page 24 of 124




Van der Waals Attraction

• intermolecular, electrostatic interactions

• electron motion in the atomic hull may result in a temporaryasymmetric charge distribution in the atom (i.e. more electrons(or negative charge, resp.) on one side of the atom than on theopposite one)

• charge displacement ⇒ temporary dipole

• a temporary dipole

– attracts another temporary dipole– induces an opposite dipole moment for a non-dipole atom

and attracts it

• dipole moments are very small and the resulting electric attrac-tion forces (van der Waals or London dispersion forces) are weak and actin a short range only

• atoms have to be very close to attract each other, for a longdistance the two dipole partial charges cancel each other

• high temperature (kinetic energy) breaks van der Waals bonds

Introduction

Examples





MD – Implement...




Page 25 of 124




Well-Known Potentials

i j

rij

• some potentials from mechanics:

– harmonic potential (Hooke’s law): Uharm (rij) = 12k (rij − r0)

2;potential energy of a spring with length r0, stretched/clinchedto a length rij

– gravitational potential: Ugrav (rij) = −gmimj

rij;

potential energy caused by a mass attraction of two bodies(planets, e.g.)

• the resulting force is �Fij = −gradU (rij) = − ∂U∂rij

integration of the force over the displacement results in the energy or a potential

difference

• Newton’s 3rd law (actio=reactio):�Fij = −�Fji

Introduction

Examples





MD – Implement...




Page 26 of 124




Intermolecular Two-Body Potentials

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

0 0.5 1 1.5 2 2.5 3

pote

ntia

l U

distance r

hard sphere potentials

hard sphereSquare−well

Sutherland

σ

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

0 0.5 1 1.5 2 2.5 3

pote

ntia

l U

distance r

soft sphere potentials

soft sphereLennard−Jonesvan der Waals

σ

Introduction

Examples





MD – Implement...




Page 27 of 124




Intermolecular Two-Body Potentials

• hard sphere potential: UHS (rij) =

{∞ ∀ rij ≤ d

0 ∀ rij > dForce: Dirac Funktion

• soft sphere potential: USS (rij) = ε(

σrij

)n

• Square-well potential: USW (rij) =

⎧⎪⎨⎪⎩∞ ∀ rij ≤ d1

−ε ∀ d1 < rij < d2

0 ∀ rij ≥ d2

• Sutherland potential: USu (rij) =

{∞ ∀ rij ≤ d−εr6ij

∀ rij > d

• Lennard Jones potential

• van der Waals potential UW (rij) = −4εσ6(

1rij

)6

• Coulomb potential: UC (rij) = 14πε0

qiqj

rij

ε = energy parameter

σ = length parameter (corresponds to atom diameter, cmp. van der Waals

radius)

Introduction

Examples





MD – Implement...




Page 28 of 124




Lennard Jones Potential

� �

��

O

ri

rj

rij

• Lennard Jones potential: ULJ (rij) = αε((

σrij

)n

−(

σrij

)m)with n > m and α = 1

n−m

“nn

mm

” 1n−m

• continuous and differentiable (C∞), since rij > 0

• LJ 12-6 potential

ULJ (rij) = 4ε

((σrij

)12

−(

σrij

)6)

– m = 6: van der Waals attraction (van der Waals potential)

– n = 12: Pauli repulsion (softsphere potential): heuristic– application: simulation of inert gases (e.g. Argon)

– force between 2 molecules:Fij = −∂U(rij)

∂rij= 24ε

rij

(2(

σrij

)12

−(

σrij

)6)

– very fast fade away ⇒ short range (m = 6 > 3 = d dimen-sion)

Introduction

Examples





MD – Implement...




Page 29 of 124




LJ Atom-Interaction Parameters

� �

atom ε σ[1.38066 · 10−23J ]1 [10−1nm]2

H 8.6 2.81He 10.2 2.28C 51.2 3.35N 37.3 3.31O 61.6 2.95F 52.8 2.83

Ne 47.0 2.72S 183.0 3.52Cl 173.5 3.35Ar 119.8 3.41Br 257.2 3.54Kr 164.0 3.83

ε = energy parameterσ = length parameter (cmp. van der Waals radius)

→ parameter fitting to real world experiments

1 Boltzmann-constant: kB := 1.38066 · 10−23 JK

2 10−1nm = 10−10m = 1 (Ångström)

Introduction

Examples





MD – Implement...




Page 30 of 124




Dimensionsless Formulation

using reference values such as σ, ε, reduced forms of the equationscan be derived and implemented → transformation of the problem

• position, distance�r∗ :=

1σ

�r (1a)

• timet∗ :=

1σ

√ε

mt (1b)

• velocity�v∗ :=

Δt

σ�v (1c)

• potential (atom-interaction parameters are eliminated!): U∗ := Uε

U∗LJ (rij) :=

ULJ (rij)ε

= 4((

r∗ij2)−6

−(r∗ij

2)−3

)(1d)

U∗kin :=

Ukin

ε=

1ε

mv2

2=

v∗2

2Δt∗2 (1e)

• force�F ∗

ij :=�Fijσ

ε= 24

(2

(r∗ij

2)−6

−(r∗ij

2)−3

)�r∗ijr∗ij

2 (1f)

Introduction

Examples





MD – Implement...




Page 31 of 124




Multi-Centered Molecules

CA1 CA2

CA

CB1

CB2

CB

FA1B1

FA1B2FA2B1

FA2B2

FB1A1

FB1A2

FB2A1

FB2A2

FAB

FBA

• molecules can be composed with multipleLJ-centers→ rigid bodies without internal degrees offreedom

• additionally: orientation (quarternions), an-gular velocity

• additionally: moment of inertia (principalaxes transformation)

• calculation of the interactions betweeneach center of one molecule to each centerof the other

• resulting force (sum) acts at the center ofgravity, additional calculation of torque

• MBS (Multi Body System) point of view: instead of moving multi-centered molecules, there is a holonomically constrained mo-tion of atoms (for a constraint to be holonomic it can be expressible as a function f(r, v, t) = 0)

• advantage: better approximation of unsymmetric molecules

• there is not necessarily one LJ center for each atom

Introduction

Examples





MD – Implement...




Page 32 of 124




Mixtures of Fluids

• simulation of various components (molecule types)

• modified Lorentz-Berthelot rules for interaction of molecules ofdifferent types

σAB :=σA + σB

2(2a)

εAB := ξ√

εaεB (2b)

with ξ ≈ 1e.g. N2 + O2 → ξ = 1.007, O2 + CO2 → ξ = 0.979 . . .

A A

B B

� ,�A A

� ,�A A

� ,�AB AB

Introduction

Examples





MD – Implement...




Page 33 of 124




NVT-Ensemble, Thermostat

statistical (thermodynamics) ensemble: set of possible states a sys-tem might be in

• for the simulation of a (canonical) NVT-ensemble, the followingvalues have to be kept constant:

– N : number of molecules– V : volume– T : temperature

• a thermostat regulates and controls the temperature (the kineticenergy), which is fluctuating in a simulation

• the kinetic energy is specified by the velocity of the molecules:Ekin = 1

2

∑i mi�v

2i

• the temperatur is defined by T = 23NkB

Ekin

(N : number of molecules, kB : Boltzmann-constant)

• simple method: the isokinetic (velocity) scaling:

vcorr := βvact mit β =√

Tref

Tact

• further methods e.g. Berendsen-, Nosï¿½Hoover-thermostat

Introduction

Examples





MD – Implement...




Page 34 of 124




Domain

aa

b

b

• Periodic Boundary Conditions (PBC):

– modelling an infinite space, built from identical cells⇒ domain with torus topology

Introduction

Examples





MD – Implement...




Page 35 of 124




Domain

• Minimum Image Convention (MIC):

– with PBC, each molecule and the associated interactionsexist several times

– with MIC, only interactions between the closest represen-tants of a molecule are taken into consideration

Introduction

Examples





MD – Implement...




Page 36 of 124




1.4. Molecular Dynamics – the Mathematical Model

System of ODE

• resulting force acting on a molecule: �Fi =∑

j �=i�Fij

• acceleration of a molecule (Newton’s 2nd law):

� ir =�Fi

mi

=

∑j �=i

�Fij

mi

= −∑

j �=i∂U(�ri,�rj)

∂|rij |mi

(3)

• system of dN coupled ordinary differential equations of 2nd or-der transferable (as compared to Hamilton formalism) to 2dNcoupled ordinary differential equations of 1st order (N : number of

molecules, d: dimension), e.g. independent variables q := r and pwith

�pi := mi�ri (4a)

�pi = �Fi (4b)

Introduction

Examples





MD – Implement...




Page 37 of 124




Boundary Conditions

• Initial Value Problem:position of the molecules and velocities have to be given;initial configuration e.g.:

– molecules in crystal lattice (body-/face-centered cell)– initial velocity

* random direction

* absolute value dependent of the temperature(normal distribution or uniform), e.g.32NkBT = 1

2

∑Ni=1 mv2

i with vi := v0

⇒ v0 :=√

3kBTm

resp. v∗0 :=

√3T ∗Δt∗

• time discretisation: t := t0 + i · Δt→ time integration procedure

Introduction

Examples





MD – Implement...




Page 38 of 124




1.5. MD – Approximations and Discretization

Euler Method

• Taylor series expansion of the positions in time:

�r(t + Δt) = �r(t) + Δt�r(t) +1

2Δt2�r(t) +

Δti

i!�r(i)(t) + . . . (5)

(r, r, r(i): derivatives)

• approximation of (5), neglecting terms of higher order of Δt, aswell as an analogous formulation of �v(t) := �r(t) with �a(t) :=

�v(t) = �r(t) =�F (t)m

leads to the explicit Euler method:

�v(t + Δt).= �v(t) + Δt�a(t) (6a)

�r(t + Δt).= �r(t) + Δt�v(t) (6b)

• implicit Euler method → derivatives at the time step end:

�v(t + Δt).= �v(t) + Δt�a(t + Δt) (7a)

�r(t + Δt).= �r(t) + Δt�v(t + Δt) (7b)

• (6a) in (7b) ⇒ �r(t + Δt).= �r(t) + Δt�v(t) + Δt2�a(t)

Introduction

Examples





MD – Implement...




Page 39 of 124




Störmer Verlet Method

• the Taylor series expansion in (5) can also be performed for−Δt: (Richardson extrapolation for δ = −1)

�r(t − Δt) = �r(t) − Δt�r(t) +1

2Δt2�r(t) +

(−Δt)i

i!�r(i)(t) + . . . (8)

• from (5) and (8) the classical Verlet algorithm can be derived:

�r(t + Δt) = 2�r(t) − �r(t − Δt) + Δt2�r(t) + O(Δt4)

≈ 2�r(t) − �r(t − Δt) + Δt2�a(t)(9)

direct calculation of �r(t + Δt) from �r(t) and �F (t)

• the velocity can be estimated with

�v(t) = �r(t).=

�r(t + Δt) − �r(t − Δt)

2Δt(10)

Introduction

Examples





MD – Implement...




Page 40 of 124




Crank Nicolson Method

• explicit approximation (11a) for half step [t, t + Δt2

] inserted intoimplicit approximation (11b) for half step [t + Δt

2, t + Δt] gives for

v (11c):

�v(t +Δt

2) = �v(t) +

Δt

2�a(t) (11a)

�v(t + Δt) = �v(t +Δt

2) +

Δt

2�a(t + Δt) (11b)

�v(t + Δt) = �v(t) +Δt

2(�a(t) + �a(t + Δt)) (11c)

• alternative conversion to integral equation

�v(t + Δt) − �v(t) =

∫ t+Δt

t

�a(τ) dτ

numerical integration with trapezoidal rule ⇒ (11c)

• generalization with further subdivisions in subintervals leads toRunge Kutta methods. More force-evaluations necessary!

Introduction

Examples





MD – Implement...




Page 41 of 124




Velocity Störmer Verlet Method

The velocity Störmer Verlet method is a composition of a

• Taylor series expansion of 2nd order for the positions (5), and a

• Crank Nicolson method for the velocities (11c)

�r(t + Δt) = �r(t) + Δt�v(t) +Δt2

2�a(t) (12a)

�v(t + Δt) = �v(t) +Δt

2(�a(t) + �a(t + Δt)) (12b)

tt-�t t+�t tt-�t t+�t tt-�t t+�t

r

v

F

tt-�t t+�t

Forward Euler r Force calculation Crank-Nicolson v

memory requirements: (3 + 1) · 3N(calculation of v(t + Δt) requires not only v(t), r(t + Δt) and F (t + Δt), but also F (t) and

therefore 3 + 1 vector fields)

Introduction

Examples





MD – Implement...




Page 42 of 124




Dimensionless Velocity Störmer Verlet Method

• insertion of the equations (1) leads to:(�r := σ�r∗, �v := σ

Δ�v∗, Δt2 := σ2 m

εΔt∗2, �r = 1

m�F := 1

mεσ

�F ∗)

�r∗(t + Δt) = �r∗(t) + �v∗(t) +Δt∗2

2�F ∗(t) (13a)

�v∗(t + Δt) = �v∗(t) +Δt∗2

2�F ∗(t) +

Δt∗2

2�F ∗(t + Δt) (13b)

• procedure:


r

v

F

tt-�t t+�ttt-�t t+�t

Forward Euler r ½ Forward Euler v Force calculation ½ Backward Euler v

1. calculate new positions (13a),partial velocity update: +Δt∗2

2�F ∗(t) in (13b)

2. calculate new forces, accelerations (computationally intensive!)

3. calculate new velocities: +Δt∗2

2�F ∗(t + Δt) in (13b)

• memory requirements: 3 · 3N

Introduction

Examples





MD – Implement...




Page 43 of 124




Leapfrog Method

• for the Leapfrog method, the velocity calculations are shifted fora half time step with respect to the position calculations:

�v(t +Δt

2) = �v(t − Δt

2) + Δt�a(t) (14a)

�r(t + Δt) = �r(t) + Δt�v(t +Δt

2) (14b)


r

v

F

t-�t/2 t+�t/2t-�t/2 t+�t/2 t-�t/2 t+�t/2 t-�t/2 t+�t/2

tt-�t t+�t

• exact arithmetic: Störmer Verlet, Velocity Störmer Verlet andLeapfrog schemes are equivalent

• the latter two are more robust with respect to roundoff errors

Introduction

Examples





MD – Implement...




Page 44 of 124




Leapfrog Method – Thermostat

• for the Leapfrog method, the velocity calculations are shifted fora half time step with respect to the position calculations:

�v(t +Δt

2) = �v(t − Δt

2) + Δt�a(t)

�r(t + Δt) = �r(t) + Δt�v(t +Δt

2)

t-�t t+�t t+�t t-�t t+�t

r

v

F

tt-�t t+�t tt-�t t+�t tt-�t t+�t tt-�t t+�t tt-�t t+�t

Thermostat

• an intermediate step may be introduced for the thermostat �v(t) :=�v(t+Δt

2)+�v(t−Δt

2)

2to synchronize the velocity:

�vact(t) = �v(t − Δt

2) +

Δt

2�a(t) (16a)

�v(t +Δt

2) = (2β − 1)�vact(t) +

Δt

2�a(t) (16b)

Introduction

Examples





MD – Implement...




Page 45 of 124




Multistep, Predictor Corrector Methods

• Multistep methods:

– results are stored for several time steps, which define a(polynomial) interpolant

– use the interpolant (extrapolation) for the integration– initialization with single-step-methods– increased memory requirements caused by storage of data

of previous steps’ data!

• Predictor Corrector methods:

1. use explicit method to determine predictor values for t + Δt

2. implicit method uses predictor values instead of the un-known ones for t + Δt

3. increased computational effort!4. quality of the predictor step caused by the complex chaotic

behaviour is often not very good

Introduction

Examples





MD – Implement...




Page 46 of 124




Multi-Centered Molecules

• for multi-centered molecules, besides position r and velocitiesv, orientations q and angular velocities w have to be also calcu-lated

• candidate: explicit or implicit version of the Fincham Leapfrogrotational algorithm

– r,v,F using classical Leapfrog method– additional orientation q, angular velocity w as well as angu-

lar momentum j

t-�t/2 t+�t/2t+�t/2 t-�t/2 t+�t/2

r

v

F,a

tt-�t t+�ttt-�t t+�t tt-�t t+�t tt-�t t+�t

q

t-�t/2 t+�t/2t+�t/2 t-�t/2 t+�t/2j,w

t

t+�t/2

Introduction

Examples





MD – Implement...




Page 47 of 124




Evaluation of Time Integration Methods

• accuracy (not of great importance)

• stability

• conservation

– of phase space density (symplectic)– of energy– of momentum (especially with PBC (Periodic Boundary Conditions))

• reversibility of time

• use of resources:

– computational effort (number of force evaluations)– maximum time step size– memory usage

Introduction

Examples





MD – Implement...




Page 48 of 124




Reversibility of Time

• time reversal for a closed system means

– a turnaround of the velocities and also momentums; posi-tions at the inversion point stay constant

– traverse of a trajectory back in the direction of the origin

• demand for symmetry for time integration methods

+ e.g. Verlet method- e.g. Euler method, Predictor Corrector methods

• contradiction with

– the H-theorem (increase of entropy, irreversible processes)?(Loschmidt objection)

– the second theorem of thermodynamics?– reversibility in theory only for a very short time

• Lyapunov instability ⇒ Kolmogorov entropy

Introduction

Examples





MD – Implement...




Page 49 of 124




Lyapunov Instability

• Example of a simple system:

– stable case:jumping ball on a plane with slightly disturbed initial hori-zontal velocity ⇒ linear increase of the disturbance

– instable case:jumping ball on a sphere with slightly disturbed initial hor-izontal velocity ⇒ exponential increase of the disturbance(Lyapunov exponent)

• for the instable case, small disturbances result in large changes:chaotic behaviour (butterfly ⇒ hurricane?)

• non-linear differential equations are often dynamically instable

• calculation of the trajectories: badly conditioned problem;a small change of the initial position of a molecule may result ina distance to the comparable original position, after some time,in the magnitude of the whole domain!

• there are also conserved quantities for whichnumerical simulations make sense!

Introduction

Examples





MD – Implement...




Page 50 of 124




Lyapunov Instability: A Numerical Experiment

• setup of 4000 fcc atoms

• for a second setup, the position of a singleatom was changed with a displacement of0.001

• tracing the movement of the atom for bothsetups, the distance increases for eachstep

colours indicate velocity

2.5 3

3.5 4

4.5 5

5.5 7.1 7.2

7.3 7.4

7.5 7.6

7.7

3.5

3.6

3.7

3.8

3.94

tracing a Molecule (with initial displacement)

Molecule 25, run1Molecule 25, run2

4.1

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

2.5 3 3.5 4 4.5 5 5.5

Molecule deviation (with initial displacement)

Introduction

Examples





MD – Implement...




Page 51 of 124




Short-Range Potential

• choosing m = 6 (negative exponent in the LJ-potential) fast decay of po-tential and force

• for each molecule, an influence volume (closed sphere) withcut-off radius rc (Euclidian metrics) can be assumed where ev-ery molecule outside this influence volume is neglected ⇒

U∗LJ,rc

(r∗ij

)=

{4((

r∗ij2)−6 − (

r∗ij2)−3

)for r∗ij ≤ rc

0 for r∗ij > rc

(17a)

�F ∗ij,rc

(�r∗ij

)=

{24

(2(r∗ij

2)−6 − (

r∗ij2)−3

)�r∗ijr∗ij

2 for r∗ij ≤ rc

0 for r∗ij > rc

(17b)

• consider only a subgraph of the interaction-graph

• the anti-symmetric force matrix, related to this graph, is sparse

• the complexity of the calculation can be reducedfrom O (N2) to O(N).

Introduction

Examples





MD – Implement...




Page 52 of 124




Short-Range Interactions

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0 1 2 3 4 5

U*

r*

dim. red. Lennard−Jones 12−6 Potential

−2.5

−2

−1.5

−1

−0.5

0

0.5

0 1 2 3 4 5

F*

r*

dim. red. Lennard−Jones 12−6 Force

1

2

3

4

5

Fij Force matrix/Interaction-graph

- F12 F13 F14 F15

−F12 - F23 F24 F25

−F13 −F23 - F34 F35

−F14 −F24 −F34 - F45

−F15 −F25 −F35 −F45 -

• fast decay of force contributions with increasing distancedense force matrix with O(n2), mostly very small, entries

Introduction

Examples





MD – Implement...




Page 53 of 124




Short-Range Interactions

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0 1 2 3 4 5

U*

r*

dim. red. finites Lennard−Jones 12−6 Potential (rc=2)

−2.5

−2

−1.5

−1

−0.5

0

0.5

0 1 2 3 4 5

F*

r*

dim. red. finite Lennard−Jones 12−6 Force (rc=2)

1

2

3

4

5

Fij Force matrix/Interaction-graph

- F12 F13 F14 0−F12 - 0 F24 F25

−F13 0 - F34 0−F14 −F24 −F34 - F45

0 −F25 0 −F45 -

• cut-off radius leads to a reduction of the computational effortsparse force matrix with O(n) entries

Introduction

Examples





MD – Implement...




Page 54 of 124




Shifted Potentials

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0 1 2 3 4 5

U*

r*

shifted dim. red. finites Lennard−Jones 12−6 Potential (rc=2)

−2.5

−2

−1.5

−1

−0.5

0

0.5

0 1 2 3 4 5

F*

r*

dim. red. finite Lennard−Jones 12−6 Force (rc=2)

U∗LJ,rc,shifted

(r∗ij

)=

{U∗

LJ

(r∗ij

) − U∗LJ (r∗c ) for r∗ij ≤ r∗c

0 for r∗ij > r∗c

�F ∗ij,rc

(�r∗ij

)=

{�F ∗

ij

(r∗ij

)for r∗ij ≤ r∗c


• additionally, constant additive term for the potential⇒ continuous potentialreduced error for the overall potential

Introduction

Examples





MD – Implement...




Page 55 of 124




Shifted Potentials

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0 1 2 3 4 5

U*

r*

shifted dim. red. finites Lennard−Jones 12−6 Potential (rc=2)

−2.5

−2

−1.5

−1

−0.5

0

0.5

0 1 2 3 4 5

F*

r*

shifted dim. red. finite Lennard−Jones 12−6 Force (rc=2)

U∗LJ,rc,shifted

(r∗ij

)=

{U∗

LJ

(r∗ij

) − U∗LJ (r∗c ) − F ∗

LJ (r∗c )(r∗ij − r∗c

)for r∗ij ≤ r∗c


�F ∗ij,rc,shifted

(r∗ij

)=

{�F ∗

ij

(r∗ij

) − F ∗LJ (r∗c ) for r∗ij ≤ r∗c


• additionally, constant additive term for the potential⇒ continuous potential

• additionally, linear additive term for the potential⇒ continuous force

Introduction

Examples





MD – Implement...




Page 56 of 124




Cut-Off Corrections

• due to the cut-off radius, the calculation of

– the potential energy– the pressure

neglects some addends with small absolute values⇒ (small) errors

• cut-off correction tries to correct this error

• constant density and a homogeneus distribution are a prerequi-site

• physical values in the calculated volume can be approximatelyextrapolated

Introduction

Examples





MD – Implement...




Page 57 of 124




1.6. MD – Implementational Aspects

Verlet Neighbour Lists

rc

rmax

• every molecule stores its neigh-bours for a distance rmax > rc

• every nupd time steps (dep. on rmax),the lists are updated

• the "buffer" has to be larger than thecovered distance of a molecule forthat time:

rmax − rc > nupd Δt vm

Introduction

Examples





MD – Implement...




Page 58 of 124




Classical Linked-Cell Algorithm

• molecules are ranged in a lattice of cu-bic cells of side length rc

– hash table with"geometrically motivated" hashfunction

– "Binning" resp. "Bucketing"-techniques from "ComputationalGeometry"

– direct volume representation(voxel) of the influence region

• runtime: O(n)

• only half (point symetry) of the neigh-bour cells are explicitly traversed (New-ton’s 3rd law)

• erase and generate the data structurein each time step

Introduction

Examples





MD – Implement...




Page 59 of 124




Variable Linked-Cell Algorithm

• lattice might be built up from cellsof side length rc

twith t ∈ R

+

• preferable integer numbers are usedfor the divisor t ∈ N

• for t → ∞, the examinatedinfluence volume will converge tothe (optimal) sphere

1

2

3

4

5

6

7

8

9

10

1 2 3 4 5 6 7 8 9 10rc/cellwidth

searchvolume/hemispherevolume

t = 1 t = 2 t = 4 t = 3

Introduction

Examples





MD – Implement...




Page 60 of 124




Linked-Cell Algorithm – Data Structure I

0 1 2 3 4 5 6 7 8 9 10 11

12 23

24 35

108 119

inner zone

boundary zoneHalo

�rc

molecule 1

data

nextincell

molecule 2

data

nextincell

molecule 3

data

nextincell

molecule 4

data

nextincell

molecule 5

data

nextincell

molecule x

data

nextincell

molecule N

data

nextincell

cellseq. 1 cellseq. 2 cellseq. x cellseq. i

• cells are stored as a one-dimensional array (vector)

• intrusive list for the cell molecules

• list to determine the processing sequence

Introduction

Examples





MD – Implement...




Page 61 of 124




Linked-Cell Algorithm – Data Structure II

-50 -49 -48 -47 -46

-39 -38 -37 -36 -35 -34 -33

-28 -27 -26 -25 -24 -23 -22 -21 -20

-16 -15 -14 -13 -12 -11 -10 -9 -8

-4 -3 -2 -1

rc

-25 -24 -23

-19 -18 -17 -16

-14 -13 -12 -11 -10

-8 -7 -6 -5 -4

-2 -1

rc

-20

• offset mask to determine the neighbours

• cache efficiency is influenced by the processing order(temporal locality)

1

2

3

4

5

Introduction

Examples





MD – Implement...




Page 62 of 124




1.7. MD – Parallelisation

Profiling

��

��

��

��

��

��

�� !"#��$%&

��'��( �� !��'��&) �*'�"�+ �� ,�-�� -/�� !�� &�� -/�� !�� &�� -/�� !/�� &�� -/�� !/�� &,��-�� 0�� ( �� !��'��&2 ��5��

• force calculation is dominating

Introduction

Examples





MD – Implement...




Page 63 of 124




Shared Memory Parallelisation

• each process calculates one part (Np

) of the molecules (cells)

• availability of all relevant data (position) because of commonmemory

• “Shared Memory” algorithm: Velocity Störmer Verlet method

1. parallel explicit Euler method r,v (half step) for Np

molecules

2. parallel force calculations for Np

molecules or the respectivecells(force summation critical, respecting Newtons 3rd law: reduction; same with

linked-cell algorithm)

3. parallel implicit Euler method v (half step) for Np

molecules

Introduction

Examples





MD – Implement...




Page 64 of 124




Replicated Data I

• "Atom Decomposition" → Shared Memory parallelisation

• every node has to store all position data

• collective communication for thesynchronization of redundant data

• "Atom Decomposition" algorithm: Velocity Störmer Verlet method

1. explicit Euler method r,v (half step) for Np

molecules

2. distribute (gather-to-all) the Np

position data for each PEto all other PEs

3. force calculation for Np

molecules

4. possible distribution of partial forces to the appropriate PEs5. implicit Euler method for v (half step) for N

pmolecules

Introduction

Examples





MD – Implement...




Page 65 of 124




Replicated Data II

F1

F2

F3

F4

F5

F6

F7

F8

F9

F10

F11

F12

F13

F14

F15

F16

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

�

Force matrix is only “virtual” and not allocated/set up

• costs

– calculation: Np

– communication partners per PE: p − 1

– memory requirements: N positions and Np

forces

Introduction

Examples





MD – Implement...




Page 66 of 124




Replicated Data II

F1

F2

F3

F4

F5

F6

F7

F8

F9

F10

F11

F12

F13

F14

F15

F16

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

�


F1

F2

F3

F4

F5

F6

F7

F8

F9

F10

F11

F12

F13

F14

F15

F16

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

�


• costs

– calculation: N2p

– communication partners per PE: p − 1

– memory requirements: taking advantage of Newtons 3rdlaw needs a vector for N (partial) forces and additional com-munication

Introduction

Examples





MD – Implement...




Page 67 of 124




Force Decomposition I

• each process calculates a part of the molecules and the forcematrix

• on each node: position data of 2 N√p

molecules

• communication: distribution of positions and calculated forces

• "Force Decomposition" algorithm: Velocity Störmer Verlet method

1. explicit Euler method for r,v (half step) for Np

molecules

2. distribution of Np

position data per PE to 2(

N√p− 1

)PEs

3. force calculation of a√

p × N√p

sub-matrix

4. distribution of partial forces to√

p − 1 PEs

5. implicit Euler method for v (half step) for Np

molecules

Introduction

Examples





MD – Implement...




Page 68 of 124




Force Decomposition II

1 2 3 4

5 6 8

9 10 11 12

13141516

7

F1

F2

F3

F4

F5

F6

F7

F8

F9

F10

F11

F12

F13

F14

F15

F16

�

1 5 9 13 2 6 10 14 3 7 11 15 4 8 12 16


• costs

– calculation: Np

– communication partners per PE: 2(√

p − 1)

Introduction

Examples





MD – Implement...




Page 69 of 124




Spatial Decomposition

• domain is decomposed into subdomains

• each processor handles one subdomain

• amount of molecules per processor is variable(molecules are moving!)

• overlapping buffer regions (halo, rc) have to be synchronized

Introduction

Examples





MD – Implement...




Page 70 of 124




Spatial Decomposition

• domain is decomposed into subdomains

• each processor handles one subdomain

• amount of molecules per processor is variable(molecules are moving!)

• overlapping buffer regions (halo, rc) have to be synchronized

• point-to-point communication, dependent of

– decomposition– molecule movement (flow velocity)– communication method: "x-y-z" vs. "direct"

Introduction

Examples





MD – Implement...




Page 71 of 124




Domain Decomposition: Cubes ↔ Slices (1)

• assumption:

– homogeneous molecule distribution– subdomains with N

pmolecules and volume Ld

p: N ↔ Ld

– communication size proportional to halo volume– full halo

• slices:

– 2 neighbour PEs– halo volume: Ld−12 rc = 2Ld rc

L

– bad scaling properties– relatively easy to implement– special case of a cartesian topol-

ogy (→ cube)

• communication

– amount: 2p

– comm./PE: 2 rc

LN

Introduction

Examples





MD – Implement...




Page 72 of 124




Domain Decomposition: Cubes ↔ Slices (2)

• assumption:

– homogeneous molecule distribution– subdomains with N

pmolecules and volume Ld

p: N ↔ Ld

– communication size proportional to halo volume– "complete" halo

• cubes:

– 3d − 1 neighbour PEs

– side length: l = d

√Ld

p= L

d√

p

– halo volume (l + 2rc)d − ld =∑d

i=1

(di

)ld−i(2rc)

i

≈ (d1

)ld−12rc = 2 d ld rc

l=

2 dLd p1d−1 rc

L

• communication

– amount:(3d − 1

)p (direct) or 2d · p (x-y-z)

– comm./PE: 2d rc

LN p

1d−1

Introduction

Examples





MD – Implement...




Page 73 of 124




1.8. Molecular Dynamics – Examples of NanofluidicSimulations

1.8.1. Simulating Diffusion

Fluid

• a fluid is a continuum (a space continuously filled with mass)without a rigid crystal structure:

– liquids: hardly compressible– gases: volume depends on pressure

(looking at isothermal processes)

• small resistance to changes of form

• length scales of a system have to be large compared to themean free path of the molecules→ Knudsen number

– Kn < 0.01: ideal fluid– 0.01 < Kn < 0.1: viscous fluid (Navier Stokes equation)– 0.005 < Kn: kinetic (Boltzmann theory)

Introduction

Examples





MD – Implement...




Page 74 of 124




• the transport of properties in a fluid is caused by

– diffusion– advection

Introduction

Examples





MD – Implement...




Page 75 of 124




Diffusion

• diffusion and thermal conduction are triggered due to the Brow-nian motion

• stochastic models of a microscopic model lead to macroscopicmodels

• the process is driven for instance through a concentration or atemperature gradient, (i.e. a spatial difference)

0 0

ρ

x

Concentration profiles

Dt=0.1Dt=0.5

Dt=1Dt=2

Dt=10

Introduction

Examples





MD – Implement...




Page 76 of 124




MD Diffusion Simulation

t = 1 t = 5 t = 10 t = 15 t = 20

0

0.05

0.1

0.15

0.2

0.25

0 5 10 15 20 25 30 35 40 45 50

Diffusion

t=1t=1t=5

t=10t=15t=20

Introduction

Examples





MD – Implement...




Page 77 of 124




Gradient Diffusion

• the Fick’s equation provides the power density of the materialdiffusion (D: molecular diffusion coefficient [m2s−1]):

ΦM = −Dgradρ (18)

• the thermal flow of a thermal conduction equation (k: thermalconductivity [kg ms−3 K−1]):

ΦW = −k gradT (19)

Gradient: multiplication of the Nabla operator with a scalar function(vector):

grad f = ∇ · f =(

∂∂x

∂∂y

∂∂z

)T · f =(

∂f∂x

∂f∂y

∂f∂z

)T

Introduction

Examples





MD – Implement...




Page 78 of 124




Diffusion Equation

• change in the state variable in the control volume denotes thesum or the integral of all flows over the surface

∂m

∂t=

∂

∂t

∫Ω

ρ dΩ =

∮Γ

D gradρ d�n (20)

• applying Gaussian integration leads to∫

Ω∂ρ∂t

dΩ =∫

Ωdiv (D gradρ)dΩ

and to the diffusion equation (parabolic differential equation):

∂ρ

∂t= D Δρ (21)

divergence: multiplication of the Nabla operator with a vector (scalarvalue):

div �v = ∇ · �v =(

∂∂x

∂∂y

∂∂z

)T · �v =∂vx

∂x+

∂vy

∂y+

∂vz

∂z

Laplace operator: Δ = ∇2, z.B. Δf = div gradf = ∂2f∂x2 + ∂2f

∂y2 + ∂2f∂z2

Gaussian integration:∫

Ωdiv �v dΩ =

∮Γ�v d�n

Introduction

Examples





MD – Implement...




Page 79 of 124




1.8.2. Nucleation

Nucleation process of supersaturated Argon

t∗ = 10000 t∗ = 125000 t∗ = 230000 t∗ = 360000

• nucleation process for an oversaturated Argon vapour at 0.97 Mol/land 80k

• the simulation program automatically detects clusters (droplets)

Introduction

Examples





MD – Implement...




Page 80 of 124




0

2

4

6

8

10

12

35030025020015010050

num

ber

of c

lust

ers

timesteps (103)

Clusters in a supersaturated Argon vapor, 80 K, 0.97 Mol/l

f1(x) = 0.00731 x -2.36f2(x) = 0.00774 x -4.09f3(x) = 0.00749 x -4.698

cluster size > 20cluster size > 30cluster size > 40

f1(x)f2(x)f3(x)

• counting and grouping clusters of certain size ranges, a statisticcan be generated

• the growth of the clusters (slope) is known as nucleation rateand important for macroscopic simulations

Introduction

Examples





MD – Implement...




Page 81 of 124




1.9. Numerical Methods for Long-Range Potentials

1.9.1. Introduction

• so far: focus on short-range potentials such as Lennard-Jones,e.g.

– resulting mutual interactions are restricted to particles insome local neighbourhood

– facilitates numerical treatment and algorithmic organization:no quadratic complexity induced by an "each-with-each"behaviour

• now: tackle long-range potentials, too

– examples: Coulomb or gravitation potential– interactions between remote particles must not be neglected– simple cut-off not possible– nevertheless need for approaches that avoid quadratic com-

plexity

Introduction

Examples





MD – Implement...




Page 82 of 124




• what is long-range?

– intuitively: potential function U(r) does not decrease rapidlywith increasing r

– formally (one possibility): for d > 2, potentials not decreas-ing faster than r−d for increasing r (criterion: integrabilityover IRd)

• typical potentials in applications have both – a short-range part(to be dealt with according to the previous sections) and a long-range part, represented as two additive components:

U(r) := U short + U long

Introduction

Examples





MD – Implement...




Page 83 of 124




Two Main Classes

• grid-based methods:

– decompose U long itself into a smooth but long-range and asingular but short-range part

– for the latter, use the linked-cell approach again– for the first, there exist special so-called grid-based meth-

ods:

* P3M (Particle-Particle–Particle-Mesh) method

* PME (Particle-Mesh-Ewald) method

* SPME (Smooth-Particle-Mesh-Ewald) method)– starting point: PDE-representation of the potential Φ

−ΔΦ(x) =1

ε0

�(x)

* potential as solution of a potential (Poisson) equation

* efficient solution with standard discretisation techniquessuch as Finite Differences or Finite Elements in case ofsmooth solutions

* hence, feasible for the smooth long-range part and forhomogeneous particle distributions

Introduction

Examples





MD – Implement...




Page 84 of 124




• hierarchical or tree-based methods:

– starting point: integral representation of the potential Φ

Φ(x) =1

4πε0

∫�(y)

1

‖y − x‖dy

– advantageous especially for heterogeneous particle distri-butions (frequent in astrophysics, relevant also for molecu-lar dynamics)

– examples:

* panel clustering

* Barnes-Hut method

* (fast) multipole methods

Introduction

Examples





MD – Implement...




Page 85 of 124




1.9.2. Tree-Based Methods

• based on integral representation of the potential

• hierarchical decompositions of the domain of simulation

• adaptive approximation of the particle distribution

• widespread scheme: octrees

• allow for separation of near-field and far-field influences

• log-linear or even linear complexity can be obtained

• high flexibility with respect to more general potentials (as neededfor special applications, such as biomolecular problems)

Introduction

Examples





MD – Implement...




Page 86 of 124




Series Expansion of the Potential

• general (integral) representation of the potential:

Φ(x) =

∫Ω

G(x, y)�(y)dy

(general kernel G, particle density �(y), and domain Ω)

• Taylor expansion of the kernel G (if sufficiently smooth apartfrom the singularity in x = y) in y around y0:

G(x, y) =∑

‖j‖1≤p

1

j!G0,j(x, y0)(y − y0)

j + Rp(x, y)

(multi-index j = (j1, j2, j3), j! = j1!j2!j3!, Gk,j(x, y) mixed (k, j)-th derivative (k-th w.r.t. x, j-th w.r.t. y), remainder Rp(x, y))

• leads to expansion (and approximation) of the potential:

Φ(x) ≈∑

‖j‖1≤p

1

j!Mj(Ω, y0)G0,j(x, y0)

with the so-called moments

Mj(Ω, y0) :=

∫Ω

�(y)(y − y0)jdy

Introduction

Examples





MD – Implement...




Page 87 of 124




Subdivision of the Domain

• separation of near-field and far-field for given x:

Ω = Ωnear ∪ Ωfar , Ωnear ∩ Ωfar = ∅

• decomposition of the far-field into disjoint, convex subdomains:

Ωfar =⋃i

Ωfari

• note: this decomposition depends on x, i.e. it is done for eachparticle position x (efficient derivation possible from one hierar-chical tree structure)

• each Ωfari has an associated point yi

0

• how to choose the subdivision?

diam

‖x − yi0‖

:=supy∈Ωfar

i‖y − yi

0‖‖x − yi

0‖≤ θ

for some suitable constant 0 < θ < 1

Introduction

Examples





MD – Implement...




Page 88 of 124




• resulting approximation for Φ(x):

Φ(x) =

∫Ω

�(y)G(x, y)dy

=

∫Ωnear

�(y)G(x, y)dy +

∫Ωfar

�(y)G(x, y)dy

=

∫Ωnear

�(y)G(x, y)dy +∑

i

∫Ωfar

i

�(y)G(x, y)dy

≈∫

Ωnear

�(y)G(x, y)dy +∑

i

∑‖j‖1≤p

1

j!Mj(Ω

fari , yi

0)G0,j(x, yi0)

Introduction

Examples





MD – Implement...




Page 89 of 124




Error Estimates

• error characteristics for one fix particle position x ∈ Ω:

– local relative approximation error for one Ωfari can be shown

to be of order O(θp+1)

– global relative approximation error (summation over wholefar-field) can be shown to be of order O(θp+1)

• this clarifies the role of θ:

– allows to control the global approximation error in x

– geometric requirement to the far-field subdivision: the closerΩfar

i is located to x, the smaller it has to be to fulfil the θ-condition

• hence: a typical "level of detail"

– the closer, the higher resolved– cf. terrain representation in flight simulators

Introduction

Examples





MD – Implement...




Page 90 of 124




Tree Structures

• central question: how can we construct all these necessaryseparations of near-fields and far-fields and subdivisions of far-fields in an efficient way?

• idea: recursive decomposition of Ω (a square in 2D, a cube in3D – without loss of generality) in cells of different size, termi-nating the subdivision process if a cell is either empty or con-tains just one particle

• concepts:

– kd-tree: alternate subdivision in coordinate direction (x, y,and z), such that the separation produces two subdomainsthat roughly contain the same number of particles each

– quadtree (2D) or octree (3D): subdivision into four congruentsubsquares or eight congruent subcubes, respectively

• the following algorithms (Barnes-Hut etc.) use the octree ap-proach

Introduction

Examples





MD – Implement...




Page 91 of 124




kd-Trees – Example

3

4

5

7

6

11

9

8

10

12

13

1 2 3 4 5

6

7

9

8 10 1312

11

2

1

Introduction

Examples





MD – Implement...




Page 92 of 124




Quadtrees and Octrees – Examples

Introduction

Examples





MD – Implement...




Page 93 of 124




Recursive Computation of the Far-Field Subdivision• starting point: create the octree corresponding to the set of par-

ticles

– each node of the octree represents a subdomain of Ω orone cell

– for each cell i, define some yi0 (the centre point or the centre

of gravity of all particles contained, e.g.) for doing the Taylorexpansion

– for each cell i, let the parameter diam just denote the diam-eter of the smallest surrounding sphere, e.g.

• objective: for each particle position x, use as few cells as pos-sible (i.e. as big cells as possible) for fulfilling the diam-θ rule

• hence: start from root node, checkdiam

‖x − yi0‖

≤ θ ,

stop if fulfilled (no need for further subdivision) and proceed ifnot yet fulfilled

• note that for each x, we typically get a different subdivision

• but note also that all these subdivisions are just subtrees of ourconstructed octree

Introduction

Examples





MD – Implement...




Page 94 of 124




Recursive Computation of the Moments

• now: use this subdivision for the efficient calculation of the localmoments Mj(Ω

fari , yi

0)

• direct (numerical) integration or direct summation are not effi-cient

• therefore: use hierarchical tree structure to calculate all mo-ments for all cells in one run and store them

• crucial property for that:

Mj(Ω1 ∪ Ω2, y0) = Mj(Ω1, y0) + Mj(Ω2, y0) ,

if the point of expansion y0 is the same

• in the (standard) case of different y10 and y2

0, there are simpleconversion formulas:

Mj(Ω1, y0) =∑k≤j

(j

k

)(y0 − y0)

j−kMk(Ω1, y0)

(k ≤ j component-wise, multiplicative binomial coefficients)

Introduction

Examples





MD – Implement...




Page 95 of 124




• this allows for a bottom-up calculation of the moments from theleaves to the root

• in the leaves:

– if no particle present: zero– if one particle of mass m there in x: m(x − yi

0)j

Introduction

Examples





MD – Implement...




Page 96 of 124




Using these Building Blocks

still to be done for a numerical routine:

• how to construct the tree, starting from a given set of particles?

• how to store the tree?

• how to choose cells and expansion points?

• how to determine far-field and near-field?

several algorithmic variants to be discussed in the following

Introduction

Examples





MD – Implement...




Page 97 of 124




1.9.3. The Barnes-Hut Method

• the so-called Barnes-Hut method is the oldest, simplest, andmost widespread hierarchical tree-based approach, dates backto 1986

• original (and still main) target applications: astrophysics (highparticle numbers and a typically very heterogeneous densitydistribution)

• particle-particle interaction via gravitation potential (or modifica-tions):

U(rij) = −GGravmimj

rij

• uses octrees: leaves of the tree represent empty cells or oneparticle, inner nodes represent clusters of particles or pseudoparticles

• main underlying idea: gravitation (or other force) induced bymany particles in one remote cell can be approximated by theinfluence induced by one pseudo particle of the accumulatedmass in the cell’s centre of gravity

Introduction

Examples





MD – Implement...




Page 98 of 124




• three main steps:

– construction of the octree: refinement, until just one or noparticle per cell (top-down recursion)

– calculation of pseudo particles: mass is just sum of thecell’s particles’ masses, associated point is the mass-weightedaverage of the particles’ positions (bottom-up recursion)

– calculation of forces (N incomplete top-down traversals forN particles)

Introduction

Examples





MD – Implement...




Page 99 of 124




Calculation of the Forces• remember: to be done for each particle

• assume a particle with position x ∈ Ω

– start in root node– proceed to son nodes, until the θ − rule: diam

r≤ θ

is fulfilled, where r denotes the distance of the correspond-ing pseudo particle to x

– then, add the resulting interaction influence to the overallresult

• implicit separation into near-field and far-field:

– near-field: all leaves reached during this traversal (single-particle influence)

– far-field: all inner nodes where the process stops, i.e. cellsrepresenting pseudo particles

• several variants concerning the determination of the parameterdiam

• Barnes-Hut method can be interpreted as a special case of theTaylor approximation discussed above for p = 0 (for the calcula-tion of the pseudo particles, we just sum up masses, i.e. zero-thmoments)

Introduction

Examples





MD – Implement...




Page 100 of 124




Example of the Tree Construction and Force Calcula-tion

x i

x i

Introduction

Examples





MD – Implement...




Page 101 of 124




Accuracy and Complexity

• accuracy:

– depends on control parameter θ

– the smaller we choose θ, the larger gets the near-field andthe smaller, hence, gets the error resulting from the corre-sponding far-field approximation

– however, slow O(θ) convergence due to p = 0

• complexity:

– increases for decreasing θ

– for roughly homogeneous particle distributions:

* number of active cells bounded by C log N/θ3 for someconstant C and N particles

* overall cost of order O(θ−3N log N)

– for limit case θ → 0: method degenerates to the quadraticcomplexity of the original “each-with-each” approach with-out far-field approximation

Introduction

Examples





MD – Implement...




Page 102 of 124




Some Remarks on the Implementation

• use a standard data structure for each (pseudo) particle

• standard tree implementation with pointers:

– each node contains one (pseudo) particle at most– subdomain corresponding to a node/cell can be either stored

explicitly or calculated on-the-fly during a tree traversal– four (2D) or eight (3D) pointers from a node to its sons

• linearisation is possible, too (i.e. no need for pointers)

• tree traversal typically by recursion

– pre-order (top-down)– post-order (bottom-up)

• tree construction:

– successive insertion of particles– starting from the empty tree (i.e. the octree consisting of

the root node only)– refine, when necessary (i.e. two particles in one node)

Introduction

Examples





MD – Implement...




Page 103 of 124




Some Remarks on the Implementation (2)

• calculation of pseudo particles:

– bottom-up (post-order) traversal– sum of masses, weighted average of positions as described

before

• calculation of forces:

– outer loop: traversal visiting each leaf (to get the far-fieldapproximation for each particle)

– inner loop: top-down (pre-order) traversal until the θ-rule isfulfilled

– parameter diam on-the-fly

• time integration: essentially as before, now as another treetraversal

• motion:

– either via constructing a new octree– or via modifying the existing octree (generally preferred)

Introduction

Examples





MD – Implement...




Page 104 of 124




1.9.4. Parallelisation on Distributed-Memory MachinesGeneral Remarks

• assumption (in contrast to the earlier discussion): heteroge-neous particle distribution, continuous adaptation of the treestructure

• consequence: uniform decomposition of the domain generallynot sufficient, load imbalance critical

• typical for tree structures:– partitioning into subtrees (possible only significantly far away

from the root)– corresponds to a domain decomposition adapted to the par-

ticle density• peculiarity: computations not only in the leaves, but also in the

inner nodes (pseudo particles!)• hence:

– special treatment for the operations in the upper nodes (thefirst levels below the root) becomes necessary

– this upper part of the tree will be replicated on all proces-sors

• need for efficient re-organisation of the tree (in case of strongmotion)

• keep communication in mind!

Introduction

Examples





MD – Implement...




Page 105 of 124




A Key-Based Approach

• partitioning: need for unique identification/addressing of nodes

– pointers not feasible– therefore: coding with integer keys – unique number for

each (possible) node

• possible key:

– sequence of octal numbers representing the path from theroot to the node

– partitioning: subdivide the range of (potential) keys– drawback: results in a more horizontal (level-wise) subdivi-

sion, which has poor locality properties– hence: additional transformations (shifts, for example) to

ensure a more vertical (subtree-wise) subdivision

• in practice:

– the upper part of the tree is replicated on all processors (in-dicates structure of the domain decomposition, guaranteesconsistency)

– below this layer, subtrees are always on one processor

Introduction

Examples





MD – Implement...




Page 106 of 124




• crucial (but not discussed here in detail):

– organisation of the communication– relevant especially for the force calculation (the situation

when a particle interacts with a pseudo particle residing onanother processor)

Introduction

Examples





MD – Implement...




Page 107 of 124




Dynamic Load Balancing

• imbalance of load is one of the reasons of a poor parallel effi-cieny

• since the situation changes during the simulation process (mo-tion of particles), there is need for a dynamic load balancing, i.e.load balancing during runtime

• hence, running processes must migrate and particles will bere-allocated to other processors

• there are sophisticated load models using load indices to mea-sure load – in our context, we just use the number of particlesas a very simple load indicator

• many strategies, based on very different ideas:

– diffusion model: load is balanced as in diffusion processes– economically motivated models: bidding, broker systems– graph-based models: random matching– space filling curve approach: to be discussed in the follow-

ing

Introduction

Examples





MD – Implement...




Page 108 of 124




• necessary properties of a “good” dynamic load balancing strat-egy:

– effective – it does produce load balance– fast – state of imbalance does not last long– cheap – the balancing needs a small part of the computing

and communication times only

Introduction

Examples





MD – Implement...




Page 109 of 124




Space Filling Curves

• an unconventional load balancing strategy, a bit more in detail

• origin of the idea: analysis and topology (“topological mon-sters”)

• nice example of a construct from pure mathematics that getspractical relevance only decades later

• definition of a space filling curve (SFC), for reasons of simplicityonly in 2 D:

– curve: image of a continuous mapping of the unit interval[0, 1] onto the unit square [0, 1]2

– space filling: curve covers the whole unit square (map-ping is surjective) and, hence, covers an area greater thanzero(!)

f : [0, 1] =: I → Q := [0, 1]2 , f surjective and continuous

• prominent representatives:

– Hilbert’s curve: 1891, the most famous space filling curve– Peano’s curve: 1890, oldest space filling curve– Lebesgue’s curve: quadtree principle, the most important

SFC for computer science

Introduction

Examples





MD – Implement...




Page 110 of 124




Hilbert’s SFC

• the construction follows the geometric conception: if I can bemapped onto Q in the space filling sense, then each of the fourcongruent subintervals of I can be mapped to one of the fourquadrants of Q in the space filling sense, too

• recursive application of this partitioning and allocation processpreserving

– neighbourhood relations: neighbouring subintervals in I aremapped onto neighbouring subsquares of Q

– subset relations (inclusion): from I1 ⊆ I2 follows f(I1) ⊆f(I2)

• limit case: Hilbert’s curve

– from the correspondance of nestings of intervals in I andnestings of squares in Q, we get pairs of points in I and ofcorresponding image points in Q

Introduction

Examples





MD – Implement...




Page 111 of 124




– of course, the iterative steps in this generation process areof practical relevance, not the limit case (the SFC) itself

* start with a generator or Leitmotiv (defines the order inwhich the subsquares are “visited”)

* apply generator in each subsquare (with appropriate sim-ilarity transformations)

* connect the open ends

Introduction

Examples





MD – Implement...




Page 112 of 124




Generation Processes with Hilbert’s Generator

• classical version of Hilbert:

• variant of Moore:

• modulo symmetry, these are the only two possibilities!

Introduction

Examples





MD – Implement...




Page 113 of 124




Some Remarks on the Injectivity

• all iterations, i.e. the longer and longer curves or their generat-ing mappings from I, respectively, are injective, of course

• Hilbert’s curve itself, however, is not injective: there are imagepoints with more than one corresponding original point in I (lookat the defining correlated nesting processes in I and Q)

• this is necessary, since:

– Cantor 1878: between two arbitrary-, but finite-dimensionalsmooth manifolds, there exists a bijective mapping (injec-tive and surjective)

– Netto 1879: if the dimensionalities of two such manifoldsare different, the corresponding bijection can never be con-tinuous (and, hence, defines no SFC)

Introduction

Examples





MD – Implement...




Page 114 of 124




Peano’s SFC

• ancestor of all SFCs

• subdivision of I and Q into nine congruent subdomains

• definition of a leitmotiv, again, defines the order of visit

• now, there are 273 different (modulo symmetry) possibilites torecursively apply the generator preserving neighbourhood andinclusion

serpentine type (left and centre) and meander type (right)

Introduction

Examples





MD – Implement...




Page 115 of 124




Lebesgue’s SFC• many applications in computer science

• compared with the SFCs studied so far, there are several differ-ences:

– both Hilbert’s and Peano’s SFC can be nowhere differen-tiated, whereas Lebesgue’s SFC can be differentiated al-most everywhere

– both Hilbert’s and Peano’s SFC are self-similar (if we applythe mapping to an arbitrary subinterval of I, the result is anSFC of the same type), Lebesgue’s SFC is not self-similar

• continuity and differentiability can be shown

• Lebesgue’s SFC has a generator, too:

• this is just the lexicographic or Morton ordering, well-knownfrom quadtrees and octrees

Introduction

Examples





MD – Implement...




Page 116 of 124




Definition of Lebesgue’s SFC – the Cantor Set

• Cantor set: remove from I the central third, and go on removingthe inner third from the remaining subintervals

• binary and ternary numbers:

– 03.x1x2x3... =∑...

i=1 xi · 3−i, xi ∈ {0, 1, 2}– 02.x1x2x3... =

∑...i=1 xi · 2−i, xi ∈ {0, 1}

• Cantor set is of Lebesgue measure zero and can be formallyrepresented with the help of ternary numbers:

C := {03.(2t1)(2t2)(2t3)... ; ti ∈ {0, 1}}

• definition of the mapping on the Cantor set:

fl(03.(2t1)(2t2)(2t3)...) :=

(02.t1t3t5...02.t2t4t6...

)

• definition of the mapping between the points of the Cantor set:linear interpolation

Introduction

Examples





MD – Implement...




Page 117 of 124




SFC for Load Distribution

• idea:

1. to points in space, assign points on some iteration of anSFC

2. linear order of the respective SFC’s original points on I

3. simple partitioning (assign points to processors) based onthis sequential order

• two techniques for the first step:

– change continuous coordinates (x, y) in Q into binary orquartary codes of length k:(

xy

)�→

(02.x1x2x3...xk

02.y1y2y3...yk

)�→ 04.w1w2w3...wk

provides quadtree leaf addresses of depth k or an orderingon the k-th iteration of Lebesgue’s SFC

– again, start from the binary representation of length k anddetermine recursively the position on the k-th iteration ofHilbert’s SFC

Introduction

Examples





MD – Implement...




Page 118 of 124




SFC for Load Distribution (cont’d)

• in both cases: keys already provide positions on I

• now to be done: (parallel) sorting of the keys (main computa-tional task of the algorithm)

– may be costly at the beginning– however, later: already almost sorted inputs (only small mo-

tions, only a few new grid points created per iterative stepof the PDE solver)

• finally: update the partitioning (weighted or unweighted), stepof migration

Processor 2

Processor 4

Processor 1

Processor 3

125

Domain decomposition, Lebesgue−based (left) vs. Hilbert−based (right)

Mapping of cells to processors (Hilbert)

Introduction

Examples





MD – Implement...




Page 119 of 124




Quality Considerations

• locality:

– continuity guarantees that originals that are close togetherwill be mapped to close image points (for self-similar SFC,we have even some Lipschitz-continuity)

– more important would be a “continuity of the inverse map-ping”; but of course, this is not possible due to the missinginjectivity

– numerous theoretical considerations exist– at least: originals are clustered again (a few clusters only)– Hilbert’s SFC has the best properties

• load distribution:

– excellent parallelization properties, almost perfect balanc-ing

– communication costs are comparably small– good efficieny of load distribution already for small problem

sizes

Introduction

Examples





MD – Implement...




Page 120 of 124




Costs

• communication:

– more complicated (and, hence, longer) subdomain bound-aries due to the partitioning than we get with successivecoordinate bisection in kd-trees (halve the load by a cutin y-direction, then halve load in both parts by a cut in x-direction, and so on)

– communication takes place along subdomain boundaries– overall, a slightly higher communication than with bisection

• load distribution:

– even for large numbers p of processors small costs (just onesorting per step, in contrast to the logp sorting operationswith coordinate bisection)

Introduction

Examples





MD – Implement...




Page 121 of 124




1.9.5. The Fast-Multipole Method

Main Idea

• idea of Barnes-Hut: replace particle-particle interactions by in-teractions between particles and pseudo particles

• effect is an improved complexity: O(N log N) instead of O(N2)for almost homogeneous particle distributions, and still a signif-icant effect otherwise

• next step now:

– avoid calculation of interactions with remote pseudo parti-cles on the “origin” side for each particle on the “effect” sideindividually

– instead, combine particles to pseudo particles on the “ef-fect” side also and calculate interactions of pseudo particleswith remote pseudo particles, only

– cells representing pseudo particles with their contained par-ticles are called clusters

– from such a clucter-cluster interaction, the particle-clusterinteractions can be derived for all of the cluster’s particles

– if done properly, this fast multipole method provides a linearO(N) complexity

Introduction

Examples





MD – Implement...




Page 122 of 124




Fundamentals

• starting point again: Taylor expansion of the kernel G(x, y), butnow in x and y around xl

0 ∈ Ωl and yi0 ∈ Ωi:

G(x, y) =∑

‖k‖1≤p

∑‖j‖1≤p

1

k!j!(x−xl

0)k(y−yi

0)jGk,j(x

l0, y

i0) + Rp(x, y)

• leads to expansion (and approximation) of the potential (here:the part due to the subdomain Ωi):

Φi(x) ≈∑

‖k‖1≤p

1

k!(x − xl

0)k

∑‖j‖1≤p

1

j!Gk,j(x

l0, y

i0)Mj(Ωi, y

i0)

(moments defined as before)

• hence: interaction between Ωi and Ωl is calculated once andcan, afterwards, be used for deriving the particle-cluster inter-actions for all x ∈ Ωl (using suitable transformation rules)

• this principle is now applied hierarchically or recursively:

– first, calculate interactions between large clusters– these are “inherited” to the son level, where now the result-

ing interactions are determined– finally, get particle-X interactions in the leaves

Introduction

Examples





MD – Implement...




Page 123 of 124




Barnes-Hut vs. Fast Multipole – Comparison

x

x

y

yl0

i

i

0

0

Introduction

Examples





MD – Implement...




Page 124 of 124




Subdivision

• instead of a recursive subdivision of Ω, we have to do it for Ω×Ωnow

• same octree structure for origin and effect side

• typically, same “midpoints”: xi0 = yi

0 ∈ Ωi (again, geometricalmidpoint or centre of gravity)

• now, also a “double” θ-criterion:

‖x − xl0‖

‖xl0 − yi

0‖≤ θ and

‖y − yi0‖

‖xl0 − yi

0‖≤ θ ∀x ∈ Ωl, y ∈ Ωi

• hierarchy can be used for an efficient calculation of the interac-tions (for details, we refer to the literature)

• accuracy: as before, an O(θp+1)-characteristics can be shownfor the relative error

• parallelisation: only slightly more complicated than for Barnes-Hut

Documents

Molecular Dynamics Simulation - TUM · 2006-10-19 · • modelling aspects of molecular dynamics simulations: – why to leave the classical continuum mechanics point of view? –