Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite

empowers game creators to shape the future of gaming

SEBASTIEN HILLAIRE - ELECTRONIC ARTS / FROSTBITE

3SIGGRAPH 2016 Course - Physically Based Shading in Theory and Practice

An EA wide collaborationwithin Frostbite

Sky and atmosphere systems

Master’s thesis: Gustav Bodare & Edvard Sandberg (Ghost)

Bioware (Mass Effect Andromeda), DICE (Mirror’s Edge Catalyst), Ghost (Need for Speed)

Volumetric cloud systems

Master’s thesis: Rurik Högfeldt

Bioware: Marc-Andre Loyer (Programmer), Don Arceta (Lead Environment Art) and Soren Hesse (Tech Environment Art)

The rendering research community

Context

Physically based rendering in Frostbite

Transition done in 2014 [Lagarde&deRousiers14]

Huge increase in visual quality

Context

Previous techniques in Frostbite

HDRI captures, color gradients

Atmosphere

Depth, height fog, screen space light shafts

Clouds

Panoramic texture + Visual flow [Guerrette14]

Translated planar cloud layers

Lots of manual work to get components to match

Not dynamic, difficult to achieve time of day

Real-life sky, atmosphere and clouds

With dynamic time of day and evolving weather

Photo by DAVID ILIFF. License: CC-BY-SA 3.0

Scope and motivation

Dynamic time of day and evolving weather at 60 FPS

Physically based rendering

Meaningful material parameters

Decouple material from lighting

Unified interactions

Sky, atmosphere and cloud interact together

Interaction with opaque and transparent objects, global illumination, etc.

Mirror’s Edge Catalyst - DICE[Christin16]

Outline

Sky and atmosphere rendering

Sun rendering

Volumetric cloud rendering

Outline

Sky and atmosphere rendering

Sky and atmosphere

Atmosphere

Rayleigh scattering

Mie scattering

Components

Previous work

Analytical models [CIESky][Hosek12][Preetham99]

Simpler but limited to ground view

Spectral rendering [Elek09][Preetham99]

Sky atmosphere simulation [Bruneton08]

Pre integrated multi-scattering/absorption in 4D LUTs

Scattering occlusion from terrain

Ignore earth shadow 3D LUTs [Elek09]

Improve LUT parameterization [Yusov13][Bruneton08]

[Hosek12]

Sky – algorithm overview

Transmittance LUT

(h, θv)

Single scattering LUT

(h, θv, θl)

Final multi ScatteringLUT(Rayleigh, Mie)

Accumulatescattering

Scattered light integration

N-order scattering[Bruneton08]

[Bruneton08] [Elek09] [Yusov13]

Atmosphere simulation coefficients[Nishita93][Riley04][Bruneton08][Adams74][Kutz12]

Type Scattering (𝒎−𝟏) Extinction (𝒎−𝟏) distribution

Rayleigh(standard air) σ𝒔

𝒓𝒂𝒚= 𝟓. 𝟖𝑒−6, 𝟏. 𝟑𝟓𝑒−5, 𝟑. 𝟑𝟏𝑒−5 σ𝒔

𝒓𝒂𝒚𝒆

−𝒉𝟖.𝟎 𝒌𝒎

Mie(pollution, water, dust, etc.)

σ𝒔𝒎𝒊𝒆 >= 3𝑒−6 a × σ𝒔

𝒎𝒊𝒆 𝒆−𝒉

𝟏.𝟐 𝒌𝒎

Ozone --- σ𝒂𝒐𝒛𝒐 𝒆

−𝒉𝟖.𝟎 𝒌𝒎

a = 1.11

σ𝒂𝒐𝒛𝒐 = 𝟑. 𝟒𝟐𝟔, 𝟖. 𝟐𝟗𝟖, 𝟎. 𝟑𝟓𝟔 𝒎−𝟏 × 𝟔𝒆−𝟓% [Kutz12]

[Bruneton08]

Rayleigh scattering examples

Mie scattering example

Ozone absorption “Essential […] to reproduce the blue of the zenith sky” [Adams74][Kutz12]

Ozone absorption: “Essential […] to reproduce the blue of the zenith sky”[Adams74]

Sky and aerial perspective rendering

Render the sky using ScatteringLUTs(h,θv,θl)

Camera aerial perspective volumes

32x32 x 16 depth slices

Integrate scattering/transmittance for each slices

Easy to apply on transparents (per vertex evaluation)

Scatteredluminance

Transmittance

Time of day - Moving sun

LUTs + moving sun automatically updates:

Aerial perspective

Triggers updates of

Sky environment map

Local reflection volumes

Global illumination

Light probes

Sky environment lighting debug

Sky rendering performance

LUT updates

Needed when changing atmospheric properties

Distribute evaluation over several frames

Each scattering order, sub table parts

While updating, interpolate last results

Complete update frame count

Mean frame GPU cost

1 3.50 ms

19 0.22 ms

Component GPU cost

AP camera volume 32x32x16 0.05 ms

Sky/AP rendering 0.42 ms

LUT updates with 4th order scattering measured on XB1

Sky/Atmosphere rendering cost on XB1 720p

Outline

Sun rendering

Sun disk luminance and ground illuminance

In Frostbite, artists specify illuminance E for sun at zenith for sun solid angle 𝜴 [Lagarde&deRousiers14]

E = 100000 Lux(default sun)

L outer space luminance

current sun position

Atmosphere transmittance on sun luminance = apply transmittance LUT per pixel to outer space luminance

Outline

Previous work

Mesh + HyperTexture [Bouthors08]

High quality but overall expensive

Splatting based rendering [Yusov14]

Particle based, depth aware blending

Specific look did not match our target

Cloud layer [Schneider15]

Completely dynamic solution

Lighting, shadowing, weather

[Yusov14]

[Bouthors08]

[Schneider15]

Similar to [Schneider15]

Cloud layer with weather animation

Dynamic lighting and shadowing

Perlin-Worley noise for realistic cloud details

Beer-Powder

Improvements

Scattering energy conservation

2-lobe phase functions

Time of day and weather interaction with sky and aerial perspective

[Schneider15]

Volumetric cloud lighting components

Transmittance

Ambient

Sun scattering

Cloud self shadow

Phase function

Reality

Scattering integration improvement

Ray marching: improved integration from [Hillaire15]

float3 scattering = float3(0.0, 0.0, 0.0);float transmittance = 1.0;for (s= 0; s< StepCount; ++s){

𝑆 = sample scattered luminance

sampleTransmittance = 𝑒−𝝈𝒕 𝐷 = sample transmittancescattering += 𝑆 * transmittancetransmittance *= sampleTransmittance

Wrong?

Analytically integrate lighting w.r.t. transmittance over depth D (ShaderToy demo)

0𝐷𝑒−𝝈𝒕 𝑥 × 𝑆 𝑑𝑥 =

𝑆−𝑆×𝑒−𝝈𝒕 𝐷

𝝈𝒕

𝝈𝒕 : extinction over range𝑆 : scattered luminance𝐷 : integration distance

scattering += 𝑆 * transmittancetransmittance *= sampleTransmittance

Non energy conservative

transmittance *= sampleTransmittance

scattering += 𝑆 * transmittanceOver attenuation dark media

Analytic integration 𝑆−𝑆×𝑒−𝝈𝒕 𝐷

𝝈𝒕

Energy conservative

Default scattering integration, 21 samples

Default scattering integration, 512 samples

Improved scattering integration, 21 samples

Improved scattering integration, 512 samples

Phase function

Two-lobes HG phase function

𝒑𝒉𝒂𝒔𝒆 𝑔0, 𝑔1, α, 𝜇 = 𝑙𝑒𝑟𝑝(𝒉𝒈𝑷𝒉𝒂𝒔𝒆 𝑔0, 𝜇 , 𝒉𝒈𝑷𝒉𝒂𝒔𝒆 𝑔1, 𝜇 , α)

Tried best fit

Back-scattering still not very visible

Would require multi-scattering…

Two-lobes HG phase + artist control

More freedom to control cloud style

Default: 𝑔0 = 0.8, 𝑔1 = −0.5, α = 0.5

[Bouthors08]

CloudHenyey/GreensteinRayleigh

0.0 0.5 1.0 1.5 2.0 2.5 3.010 4

100 Two-lobe HGfitted to cloud phase

Phase function0.5 1.0 1.5 2.0 2.5 3.0 3.5

0.30.20.1

0.10.20.3

HG phase with g=0.8

Phase function0.5 0.5 1.0 1.5 2.0

2-lobe HG phase with g0=0.8, g1=-0,5 and lerp=0.5

Phase function0.5 1.0 1.5 2.0 2.5 3.0 3.5

0.30.20.1

0.10.20.3

HG phase with g=0.8

Phase function0.5 0.5 1.0 1.5 2.0

2-lobe HG phase with g0=0.8, g1=-0,5 and lerp=0.5

Aerial perspective and clouds Must look consistent under all time of day and weather conditions

Visual mismatch (missing fog on clouds)

Clouds without aerial perspective

Aerial perspective and clouds

Per sample: expensive to evaluate

Instead sampled once on cloud front interface

Compute mean depth on cloud interface weighted by transmittance

Aerial perspective on cloudClouds with aerial perspective

Cloud coverage and scattering influence

Cloud hemisphere sampling aroundcamera

• Integrate cloud luminance• Mean cloud transmittance

New AP scattered luminance =oldAP x transmittance + luminance

Cloud coverage and scattering influenceRealityClouds with aerial perspective interactions

Volumetric clouds performance

720p 900p

Hemisphere sampling: scattering & coverage 0.090 ms 0.090 ms

Main view (with temporal re-projection) 0.720 ms 1.090 ms

Planar reflection (optional) 0.035 ms 0.048 ms

Total 0.835 ms 1.228 ms

Measured on XB1

Cloud main view resolution = 640x360 (720p/2)

Planar reflection resolution scale = 128x72 (720p/10)

Default quality seen in this presentation

Looking toward horizon

Final result

Sun rise with animated clouds

Final result

Mars blue sunset (eye-balled atmosphere)

[NASA]

Conclusion

Physically based sky, atmosphere and cloud rendering solution

Participating media material definition

Physically based and energy conservative scattering formulation

Supports dynamic time of day and evolving weather

Simulation of interactions between sky, sun, aerial perspective and clouds

Default: earth/sun/moon physical values

Simulation parameters can all be changed to simulate extra-terrestrial worlds

Practical use in Mirror’s Edge Catalyst: [Christin16]

Used widely in production by EA Games

Need for Speed, Ghost

Mass Effect Andromeda, BiowareFIFA 2017

Future work

Aerial perspective scattering shadowing by opaque and translucent clouds

Can’t use epipolar sampling min/max optimisation due to translucent clouds

Directional sky ambient by using cloud local gradient as main direction?

Convolution of sky envmap with phase function for cloud ambient lighting

Dynamic weather texture for cloud animation

GI and reflection volumes update

Still a problem when it comes to delay and performance

A complex problem with different priorities for each games

References[Adams74] The influence of Ozone and Aerosol on the Brightness and Color of the twilight Sky, JAS 1974.

[Bouthors08] Interactive Multiple Anisotropic Scattering in Clouds, I3D08.

[Bruneton08] Precomputed Atmospheric scattering, EGSR 2008.

[Christin16] Lighting the City of Glass, GDC 2016.

[CIESky] Spatial distribution of Daylight

[Elek09] Rendering Parametrizable Planetary Atmospheres with Multiple Scattering in Real-time, CESCG 2009.

[Guerrette14] Moving the heavens, GDC 2014.

[Harris02] Real-Time Cloud Rendering for Games, GDC 2002.

[Hestroffer98] Wavelength dependency of the Solar limb darkening, Astronomy and Physics 1998.

[Hillaire15] Physically Based and Unifier Volumetric Rendering in Frostbite, SIGGRAPH 2015.

[Hosek12] An Analytic Model for Full Spectral Sky-Dome Radiance, SIGGRAPH 2012

[Kutz12] Blog post, 2012.

[Lagarde&deRousiers14] Moving Frostbite to PBR, SIGGRAPH 2014.

[NASA] http://www.nasa.gov/multimedia/imagegallery/image_feature_347.html

[Nishita93] Display of the Earth Taking into Account Atmospheric Scattering, SIGGRAPH 1993.

[PBR] Physically Based Rendering book, www.pbrt.org.

[Preetham99] A Practical Analytic Model for Daylight, SIGGRAPH 99.

[Riley04] Efficient Rendering of Atmospheric Phenomena, EGSR 2004.

[Schneider15] The Real-time Volumetric Cloudscapes of Horizon: Zero Dawn, SIGGRAPH 2015.

[Yusov13] Outdoor Light Scattering, GDC 2013.

[Yusov14] High-Performance Rendering of realistic Cumulus Clouds using Pre-Computed Lighting, HPG14.

[Wenzel07] Real time atmospheric effects in game revisited, GDC 2007.

[Wrenninge10] Oz: The Great and Volumetric, SIGGRAPH 2010.

Battlefield, Mass Effect, Need for Speed, FIFA, Plant vs Zombies, and many more.

We are hiring!

Questions?

Thanks!

The Frostbite rendering Team and game teams

Stephen Hill and Stephen McAuley for the course organisation

All the reviewers

Eric Bruneton and Fabrice Neyret

For further discussions, get in touch

sebastien.hillaire@frostbite.com

https://twitter.com/SebHillaire

Additional notes and Mathematica notebook to follow soon…

Bonus slides

Sky – how many scattering orders?

Single Scattering 2nd scattering order 4th scattering order

Scattering texture packing

ScatteringLUTs(h, θv, θl) is not storing (RayScat𝒓𝒈𝒃, MieScat𝒓𝒈𝒃)

But (RayScat𝒓𝒈𝒃, MieScat)

MieScat𝒓𝒈𝒃 = RayScat𝒓𝒈𝒃 ×MieScat

RayScat𝒓×σ𝒔𝒓𝒂𝒚

σ𝒔𝒓𝒂𝒚

(RayScat𝒓𝒈𝒃, MieScat𝒓𝒈𝒃) (RayScat𝒓𝒈𝒃, MieScat)

Height fog

Uses scattered luminance at horizon

Seamless sky/fog transition

Per pixel coverage

Per pixel phase function

Approximations

Assumes low altitude fog

Ignores opaque shadows and self shadowing

HorizonP

ScatteredLightP ≈ ScatteredLightHorizon

Sun disk limb darkening Attenuate luminance near edge of sun disk (Default = sun model [Hestroffer98])

(Very low exposure) No

Night time

Moon and stars rendered as sprite using celestial system

Angular radius ≈ 0,24 to 0,28 degree

Luminance ≈ 2500 cd/m2

Secondary sky/atmosphere scattering source

Luminance? Angular radius?

Night timeSun set

Ambient contribution

Ambient component

Capture the sky environment without sun disk

Integrate luminance to scatter

Approximations

Phase function as uniform

Single environment color ignoring cloud self occlusion

Multi-scattering approximation Use [Wrenninge10]: multi-octaves single scattering / extinction / phase

Without multi-scattering

Multi-scattering approximation Use [Wrenninge10]: multi-octaves single scattering / extinction / phase

Withmulti-scattering

Cloud coverage and scattering influenceThick cloud

+ aerial perspective+ hemisphere coverage/scattering

Thick cloud only

Clouds do not fit

Thick cloud + aerial perspective

Aerial perspective too bright

Sunny 16 validation Sun illuminance at ground level: 100000

Camera: ISO 100, aperture f/16 and shutter speed 1/125

Physically Based Sky, Atmosphere and Cloud Rendering in Frostbite

Presentations & Public Speaking

Effect of Salts on the Thermal Stability of Physically Crosslinked … · 2017-07-21 · Effect of Salts on the Thermal Stability of Physically Crosslinked Gelatin Hydrogels: DSC

Mobilier Furnitures - Atmosphere & Bois

What is the atmosphere? Name the 5 layers of the atmosphere. What is the composition of the atmosphere?

A Physically-Based Motion Retargeting Filter

Atmosphere 2014: RE:SPONSIBILITY - Matt Harasymczuk

The Atmosphere - global warming L2

Physically Based Rendering 기초! 기초!

The Standard Atmosphere - Virginia Techcdhall/courses/aoe2104/Standard...The Standard Atmosphere The Standard AtmosphereThe Standard Atmosphere Some definitionsSome definitions Construction

บรรยากาศ (Atmosphere) |Atm_PartI

Atmosphere & Litosphere (Indonesian)

Bỏng lạnh Frostbite

ANALISIS PENGARUH VARIABEL STORE ATMOSPHERE …

Time scales and spatial patterns of passive ocean-atmosphere decay modes Analysis of simulated coupled ocean-atmosphere decay characteristics – Atmosphere:

atmosphere biosphere

Water in the Atmosphere

Mechanisms of ocean-atmosphere interactions

Atmosphere Workshop

Make the Move: Staying Physically Active at Work

Grey Atmosphere (Mihalas 3)

Agnihotra Method to Purify Atmosphere