Introduction to Bidirectional Path Tracing (BDPT) & Implementation using OpenCL (CEDEC 2015)

INTRODUCTION TO BIDIRECTIONAL PATH TRACING & ITS IMPLEMENTATION USING OPENCL

双方向パストレーシング(BDPT)の基礎からOPENCLによる実装まで

TAKAHIRO HARADA (AMD) SHO IKEDA (RICOH)

SYOYO FUJITA (LTE)

2 | OPENCL BDPT | AUGUST 28, 2015 | HARADA, IKEDA, FUJITA

OVERVIEW OF THIS TALK

y  IntroducJon (Harada, 20min -‐ 25min) ‒  Start with path tracing review ‒ Transforming path tracing to bidirecJonal path tracing

y  BidirecJonal Path Tracing (Ikeda, 20min -‐ 25min) ‒ Classic BDPT ‒  Instant BDPT ‒  Lvc BDPT

y  IntegraJon of BidirecJonal Path Tracing to the Engine (Harada, 5min)

y  Examples (Harada, 5min)


PATH TRACING (MONTE CARLO RAY TRACING)

y  != Raster graphics (game)

y  BeauJful rendering y  Global illuminaJon


PATH TRACING (MONTE CARLO RAY TRACING)

y  ComputaJonally expensive ‒ Take Jme to converge ‒ Noisy at first, gebng cleaner later

y  Make 1 step rendering faster ‒ Total Jme decreases ‒ OpJmizaJon (using more CPU cores, GPUs)

‒ OpenCL (my talk@CEDEC2014)

y  Make 1 step less noisy ‒ Need less steps ‒ Beeer sampling techniques ‒ Clever algorithm

‒  BDPT (this talk)

PATH TRACING IMPORTANCE SAMPLING

MULTIPLE IMPORTANCE SAMPLING


FIRST PATH TRACING


PATH TRACING IMPLEMENTATION #1

for(int i=0; i<nx; i++) for(int j=0; j<ny; j++) !{ ! Ray ray = genPrimaryRay( i, j ); !! float4 coeff = 1.f; ! float4& output = pixel[i,j]; ! for(int depth=0; depth<maxDepth; depth++) ! { ! Hit hit = intersect( ray ); ! if( !hit.hasHit() ) break; !! if( hit.isEmissive() ) ! {// Implicit Connection! output += coeff * getEmission( hit ); ! break; ! } !! ray, f, pdf = randomSample( hit ); ! coeff *= f * dot( hit.m_n, ray.m_dir ) / pdf; ! } !} !


SAMPLING NEXT RAY

y  For non transmission surface, no need to sample the other side

RANDOM SAMPLING



SAMPLING NEXT RAY

y  Random sampling is not always efficient ‒ Diffuse surface => OK ‒ Glossy surface => Hmm L ‒  Specular surface => Bad L

y  We usually know the reflecJon characterisJcs about surface ‒ Why not use them? ‒  “Importance sampling” (BRDF)

RANDOM SAMPLING


IMPORTANCE SAMPLING (BRDF)

y  Specular surface ‒ Only reflect in the mirrored direcJon ‒ Always sample the direcJon (pdf = 1)

y  Glossy surface ‒ Mostly reflect around mirrored direcJon ‒  Sample around the mirrored direcJon

Specular (R) Glossy Mae Specular (T)




for(int i=0; i<nx; i++) for(int j=0; j<ny; j++) !{ ! Ray ray = genPrimaryRay( i, j ); !! float4 coeff = 1.f; ! float4& output = pixel[i,j]; ! for(int depth=0; depth<maxDepth; depth++) ! { ! Hit hit = intersect( ray ); ! if( !hit.hasHit() ) break; !! if( hit.isEmissive() ) ! {// Implicit Connection! output += coeff * getEmission( hit ); ! break; ! } !! ray, f, pdf = brdfSample( hit ); ! coeff *= f * dot( hit.m_n, ray.m_dir ) / pdf; ! } !} !

Random Sampling Brdf Importance Sampling





Random Sampling Brdf Importance Sampling


INEFFICIENT CASE

y  Rough surface + Small light

y  Brdf sampling rarely hits the light

y  SoluJon ‒ We know where the lights are ‒ Why ignoring?? ‒ Generate ray from lights ‒  “Importance sampling” (Light)

Rough

Reference Brdf sampling


INEFFICIENT CASE

y  Rough surface + Small light

y  Brdf sampling rarely hits the light

y  SoluJon ‒ We know where the lights are ‒ Why ignoring?? ‒ Generate ray from lights ‒  “Importance sampling” (Light)

Rough

Reference Brdf sampling Light sampling




for(int i=0; i<nx; i++) for(int j=0; j<ny; j++) !{ ! Ray ray = genPrimaryRay( i, j ); !! float4 coeff = 1.f; ! float4& output = pixel[i,j]; ! for(int depth=0; depth<maxDepth; depth++) ! { ! Hit hit = intersect( ray ); ! if( !hit.hasHit() ) break; ! {// Explicit Connection! ray, pdf = lightSample( hit ); ! Hit hit1 = intersect( ray ); ! if( !hit1.hasHit() ) ! { ! output+=coeff*getEmission( hit1 )/pdf; ! } ! } ! ray, f, pdf = brdfSample( hit ); ! coeff *= f * dot( hit.m_n, ray.m_dir ) / pdf; ! } !} !

Brdf Importance Sampling Light Importance Sampling


IMPLICIT CONNECTION, EXPLICIT CONNECTION

y  Implicit ConnecJon ‒ A ray accidentally hit a light source

‒  Ater primary rays are generated from camera ‒  Ater bounced rays are sampled from BRDF

y  Explicit ConnecJon ‒ A path is intenJonally connected to a light source

‒ Direct illuminaJon (sample a vertex on a light source)

Camera Light

Object

Camera Light

Object



y  We should not accumulaJng implicit hits for this implementaBon

y  Even if a ray hits an emissive surface, we ignore it

y  Sounds like we are wasJng something

NOTE


BRDF SAMPLING + LIGHT SAMPLING

y  Both generate image ‒ Each technique has strong & weak points ‒ Converges to the same image at the end

Brdf sampling (16spp) Light sampling (16spp)

Brdf sampling (inf. spp) Light sampling (inf. spp)

=



y  We should not accumulaJng implicit hits for this implementaBon

y  Even if a ray hits an emissive surface, we ignore it

y  Sounds like we are wasJng something

y  Can we use it somehow?

NOTE



y  Then simply… Take an average? ‒ Works

‒  but not the best

y  We can set any coefficients if they sum up to 1.0 ‒ brdfSampledImage * 0.2 + lightSampledImage * 0.8 ? ‒ brdfSampledImage * 0.8 + lightSampledImage * 0.2 ? ‒ Hmm…

x 0.5 + x 0.5

Brdf sample Light sample

x 0.2 + x 0.8



J L

COMPARING BRDF SAMPLING AND LIGHT SAMPLING

y  Light sampling ‒ Good for rough surface ‒ Bad for sharp surface

y  Brdf sampling ‒ Bad for rough surface ‒ Good for sharp surface

L J



y  Then simply… Take an average? ‒ Works

‒  but not the best

y  We can set any coefficients if they sum up to 1.0 ‒ brdfSampledImage * 0.2 + lightSampledImage * 0.8 ? ‒ brdfSampledImage * 0.8 + lightSampledImage * 0.2 ? ‒ Hmm

y  We do not have to se the same coefficients for all the pixels J ‒  “MulJple importance sampling” ‒ Per pixel weight

x 0.5 + x 0.5


x 0.2 + x 0.8


x + x =

Brdf sample Light sample w_b w_l MIS



y  Uniform weighJng (Average)

y  Per pixel weighJng (MulJple Importance Sampling, MIS)

VISUALIZATION OF MIS

x + x =

Brdf sample Light sample w_b w_l MIS

+ = 1.0 w_b w_l

x + x =

Brdf sample Light sample w_b w_l Average

0.5 0.5

0.5 0.5 + = 1.0 w_b w_l


MULTIPLE IMPORTANCE SAMPLING

y  Light sampling

y  Brdf sampling

0.1 0.9

0.9 0.1

0.5

0.5


MULTIPLE IMPORTANCE SAMPLING WEIGHT

y  Detailed look at brdf & light sampling (direct illuminaJon)

y  Possible to sample the same direcJon using both sampling technique ‒ Only difference is the PDF (probability density funcJon)

y  Rough surface ‒ PDF of Brdf sampling is almost uniform

‒  Low confidence in the sample => Higher noise ‒ PDF of light sampling has a strong spike at light direcJon

‒ High confidence in the sample => Lower noise

Brdf Sampling Light Sampling



y  Detailed look at brdf & light sampling (direct illuminaJon)

y  Possible to sample the same direcJon using both sampling technique ‒ Only difference is the PDF (probability density funcJon)

y  Sharp surface ‒ PDF of Brdf sampling is almost uniform

‒  Very high confidence in the sample => Lower noise ‒ PDF of light sampling has a strong spike at light direcJon

‒  RelaJvely lower confidence in the sample => Higher noise

Brdf Sampling Light Sampling



y  Use sampling PDF to compute weight

y  Weight for light sample (Explicit hit)

y  Weight for brdf sample (Implicit hit)

y  This is very important for an efficient BDPT


PATH TRACING IMPLEMENTATION #4 (FINAL)

for(int i=0; i<nx; i++) for(int j=0; j<ny; j++) !{ ! Ray ray = genPrimaryRay( i, j ); !! float4 coeff = 1.f; ! float4& output = pixel[i,j]; ! float pdfb = 0.f; ! for(int depth=0; depth<maxDepth; depth++) ! { ! Hit hit = intersect( ray ); ! if( !hit.hasHit() ) break; ! if( hit.isEmissive() ) ! {// Implicit Connection! pdfl = lightPdf( hit ); ! w = pdfb / (pdfb + pdfl); ! output += coeff * getEmission( hit ) * w; ! break; ! } ! {// Explicit Connection! ray, pdfl = lightSample( hit ); ! Hit hit1 = intersect( ray ); ! if( !hit1.hasHit() ) ! { ! w = pdfl / (pdfb + pdfl); ! output += coeff * getEmission( hit1 ) * w / pdfl; ! } ! } ! ray, f, pdfb = brdfSample( hit ); ! coeff *= f * dot( hit.m_n, nextRay.m_dir ) / pdfb; ! } !} !

BEYOND PATH TRACING


OUTDOOR


INDOOR/OUTDOOR


INDOOR


PATH TRACING IS NOT PERFECT

y  Path tracing is really good at an out door scene

y  But not for a scene mostly lit by indirect illuminaJon (indoor, day Jme)


PATH TRACING IS NOT PERFECT

64 spp 256 spp 1024 spp x 4 x 4

Indo

or

Indo

or/outdo

or

Outdo

or

Good

Ok

Bad

Path Tracing is..


BAD CONVERGENCE FOR INDOOR SCENE

y  Ray needs to be bounced more than once before connecJng to the light source

y  As we bounce, we get less & less confident about the path ‒  Lower probability => More noise


LIGHT TRACING

y  If traced from the light, it is beeer (more confident)

y  Path Tracing

y  Light Tracing Camera Light

Object

Camera Light

Object

Path Tracing BDPT


PATH TRACING + 1 LIGHT PATH

y  Can we mix path tracing & light tracing ‒ Trace 1 light path ‒ Connect to camera path

Camera Light

Object


PATH TRACING + 1 LIGHT PATH

y  Can we mix path tracing & light tracing ‒ Trace 1 light path ‒ Connect to camera path

y  Improves some situaJons (not all) ‒ Example. A room is lit by bounced light

y  Can we trace more than 1 light path, and combine? ‒ => BidirecJonal Path Tracing

Camera Light

Object

Path Tracing Path Trace + 1 light path

64 spp

BDPT

Bi-Directional Path Tracing (BDPT)

Camera path

Light path

Connection

Camera Light

Object

•  BDPT traces paths from the camera and the light source

•  BDPT combines various path sampling strategies


Camera path

Light path

Connection

Camera Light

Object




Camera path

Light path

Connection

Camera Light

Object




Camera path

Light path

Connection

Camera Light

Object



BDPT

I = w0

w1

+

+

…

wk

·

·

·

x0 xk

•  The contributions of the sampling strategies are weighted

BDPT + Multiple Importance Sampling (MIS)

Xs = x0, · · ·xs + xs+1, · · ·xk

x0 xkxs

xs+1

•  MIS weight computation

ws(X) =ps(X)

Pki=0 pi(X)

x0 xkxs

xs+1

Xs = x0, · · ·xs + xs+1, · · ·xk

ps(X) = �!p 0 ·�!p 1 · · ·�!p s · �p s+1 · · · �p k�1 · �p k

BDPT + Multiple Importance Sampling (MIS) (2

The product of the pdfs of the path sampling

The problem of BDPT on GPU implementation

0 1 2 3Global Memory 0 1 2

•  BDPT stores all the vertices on the camera and light paths

•  The global memory consumption is high













p  Instant BDPT

p  Lvc BDPT

BDPT algorithm with low memory consumption

Instant BDPT

Implicit view ( ) path

Explicit view ( ) path

Implicit light ( ) path

Explicit light ( ) path

VI

VE LE

LI

•  Path sampling strategies are limited to 4 (PT and LT)•  Low memory consumption •  If the camera has no collision detection, 3 sampling strategies

Instant BDPT + MIS

Explicit view ( ) path VE

wVE (X) =pVE (X)

pVE (X) + pVI (X) + pLE (X)

1

wVE (X)= 1 +

pVI (X)

pVE (X)+

pLE (X)

pVE (X)

Instant BDPT + MIS (2

1

wVE (X)= 1 +

pVI (X)

pVE (X)+

pLE (X)

pVE (X)

pVI (X)

pVE (X)=�!p k�1 �p k+1

�!p k�1

�p k+1


1

wVE (X)= 1 +

pVI (X)

pVE (X)+

pLE (X)

pVE (X)

x0xk

s0 =1

P (x�1 ! x0 ! x1)

pLE (X)

pVE (X)= s0s1 · · · sk

x1


1

wVE (X)= 1 +

pVI (X)

pVE (X)+

pLE (X)

pVE (X)

x0xk

s1 =1

P (x0 ! x1 ! x2)G(x0 $ x1)

pLE (X)

pVE (X)= s0s1 · · · sk

x1


1

wVE (X)= 1 +

pVI (X)

pVE (X)+

pLE (X)

pVE (X)

x0xk

pLE (X)

pVE (X)= s0s1 · · · sk sk = P (xk�1 xk xk+1)G(xk�1 $ xk)

x1

Instant BDPT OpenCL implementation

Instant BDPT : Bad case1

x0 xk

x0 xk


x0 xk


x0 xk


x0xk

Specular

Instant BDPT : Bad case

x0xk

Specular


x0xk

Specular


x0xk

Specular


p  Instant BDPT

p  Lvc BDPT

BDPT algorithm with low memory consumption

Lvc BDPT light vertex cache

0 1 2 3Light Vertex Cache

•  Light vertex cache

•  Recursive MIS













•  LvcBDPT doesn’t have the global memory for the camera path

Recursive MIS

x0xk

xs

ws(X) =ps(X)

Pki=0 pi(X)

1

ws(X)=

Pki=0 pi(X)

ps(X) �p s+1 · dEs + 1 +�!p s · dLs+1=

1

ws(X)= �p s+1 · dEs + 1 +�!p s · dLs+1

dE1 =1�!p 0

dEs =1 + �p sdEs�1�!p s�1

dLk =1

�p k+1

Recursive MIS

x0xk

xs

dLs+1 =1 +�!p s+1 · dLs+2

�p s+2

1

ws(X)= �p s+1 · dEs + 1 +�!p s · dLs+1

Lvc BDPT + Recursive MIS

x0xk

xs

dE1 =1�!p 0

1

ws(X)= �p s+1 · dEs + 1 +�!p s · dLs+1

x0xk

xs



1

ws(X)= �p s+1 · dEs + 1 +�!p s · dLs+1

x0xk

xs


dLk =1

�p k+1


1

ws(X)= �p s+1 · dEs + 1 +�!p s · dLs+1

x0xk

xs


dLk�1 =1 +�!p k�1 · dLk �p k


1

ws(X)= �p s+1 · dEs + 1 +�!p s · dLs+1

x0xk

xs


dLs+1 =1 +�!p s+1 · dLs+2

�p s+2


Lvc BDPT OpenCL implementation

InstantBDPT LvcBDPT

InstantBDPT LvcBDPT

InstantBDPT LvcBDPT

INTEGRATION TO THE ENGINE


Back End

World GPU (OCL) -‐  Path Tracing -‐  Light Tracing -‐  -‐ 

World CPU -‐  Path Tracing (OpJmized)

Front End

ENGINE ARCHITECTURE OVERVIEW

Data Loader -‐  Obj -‐  Eson -‐  Fbx

Data Manager -‐  Texture -‐  Material -‐  Mesh

Tahoe API

Texture Loader -‐  Png -‐  Hdr -‐  OpenExr -‐  …

Factory (CPU)

Camera -‐  PerspecJve -‐  Parallel -‐  Bake -‐  VR

Material System -‐  Default -‐  Graph -‐  OSL

Light Sampler -‐  Uniform -‐  Group -‐  StochasJc

Factory (OCL)


Back End

World GPU (OCL) -‐  Path Tracing -‐  Light Tracing -‐  BidirecJonal Path Tracing (LvcBDPT) -‐  BidirecJonal Path Tracing (InstantBDPT)

World CPU -‐  Path Tracing (OpJmized)

Front End

ENGINE ARCHITECTURE OVERVIEW

Data Loader -‐  Obj -‐  Eson -‐  Fbx

Data Manager -‐  Texture -‐  Material -‐  Mesh

Tahoe API

Texture Loader -‐  Png -‐  Hdr -‐  OpenExr -‐  …

Factory (CPU)

Camera -‐  PerspecJve -‐  Parallel -‐  Bake -‐  VR

Material System -‐  Default -‐  Graph -‐  OSL

Light Sampler -‐  Uniform -‐  Group -‐  StochasJc

Factory (OCL)


PATH TRACING CODE

y  Not employing a mega kernel implementaJon

y  There are many small kernels for path tracing ‒ Easier to debug ‒ Performance ‒ Extendibility => Key to add BDPT

y  (See my slides @ CEDEC 2014)


PATH TRACING KERNELS

y  Prepare ‒ Camera ray set up

y  Loop ‒ Ray cast ‒ Implicit hit accumulaJon ‒ Material prepare ‒ Shadow ray set up ‒ Material evaluaJon ‒ Ray cast ‒ Explicit hit accumulaJon ‒ Sample next ray

Path Tracing

Each line is one or more OCL kernels


INSTANT BDPT

y  Prepare ‒  Set up camera ray ‒  Sample light point

y  Loop ‒ Ray cast ‒ Material prepare (sampleBrdf) ‒  Implicit connecJon ‒ Explicit connecJon to light

‒ Make shadow ray ‒  Ray cast ‒  Connect to light

‒  Sample next ray ‒ Update MIS weight term

Trace Light Path Trace Camera Path

Orange: New kernels wrieen for BDPT White : Using path tracing kernels


INSTANT BDPT

y  Prepare ‒  Set up light ray ‒  Sample lens point ‒ Explicit connecJon light to camera

y  Loop ‒ Ray cast ‒ Material prepare (sampleBrdf) ‒ Explicit connecJon to camera

‒ Make shadow ray ‒  Ray cast ‒  Set pixel index ‒  Connect to camera


y  Prepare ‒  Set up camera ray ‒  Sample light point

y  Loop ‒ Ray cast ‒ Material prepare (sampleBrdf) ‒  Implicit connecJon ‒ Explicit connecJon to light

‒ Make shadow ray ‒  Ray cast ‒  Connect to light


Trace Light Path Trace Camera Path

Orange: New kernels wrieen for BDPT White : Using path tracing kernels


LVCBDPT

y  More complicated than Instant BDPT

y  ImplementaJon is not that far from Instant BDPT

EXAMPLES


BDPT PT


BDPT PT


BDPT PT


CLOSING

y  References ‒ AddiJonal note

‒  To be published in CEDEC library ‒ Detailed explanaJon of Instant BDPT, Lvc BDPT

‒ CEDEC 2013 ‒  hep://www.slideshare.net/takahiroharada/introducJon-‐to-‐monte-‐carlo-‐ray-‐tracing-‐cedec2013

‒ CEDEC 2014 ‒  hep://www.slideshare.net/takahiroharada/introducJon-‐to-‐monte-‐carlo-‐ray-‐tracing-‐opencl-‐implementaJon-‐cedec-‐2014

y  Feedback is welcome ‒  For next years presentaJon


NON SYMMETRIC BRDF

Without Fix With Fix


NON SYMMETRIC BRDF

With Fix With Fix

Technology

Introduction to Bidirectional Path Tracing (BDPT) & Implementation using OpenCL (CEDEC 2015)