1. Recovering Inner Slices of Translucent Objects by
Multi-frequency Illumination Kenichiro Tanaka1,2, Yasuhiro
Mukaigawa2, Hiroyuki Kubo2, Yasuyuki Matsushita1, Yasushi Yagi1
1Osaka University, 2Nara Institute of Science and Technology
(NAIST)
2. Goal 2 Scene Near Infrared Photo Inner layer recovery
Result
3. Related work 3 Layered scene Scattering sceneClear scene
Single surface scene Already clear Narasimhan et al. 2005 Nayar et
al. 2006 etc. Szeliski et al. 2000 Li et al. 2014 Our goal
etc.
5. Appearance of layered objects 5 Upper layer Inner layer
Depth dependent PSFs Blurred and superposed observation Complex
appearance Summation of all layers Blurry Deeper layer is more
blurred by scattering
6. Direct reflection Scattering High-frequency illumination 6
[Nayar et al.] Normal observation Direct components
7. What is direct component?
8. HFI separation with PSF model 8 Blurry scene Direct
component Extraction around the center Pattern pitch Brightness
Blurry scene (Spread PSF) Depends on pitch of pattern
9. HFI separation with PSF model 9 Direct componentNon-blurry
scene Blurry scene (Spread PSF) Non-blurry scene (Sharp PSF)
Depends on shape of PSF Brightness Pattern pitch
10. HFI separation for Layered scene 10 Upper layer Inner layer
Normal observation Direct component Pattern pitch Brightness
Different PSFs
11. Multi-frequency illumination Different brightness of direct
components 11 = + Smaller pitch Larger pitch = + Direct
components
12. Multi-frequency illumination Different brightness of direct
components 12 (1, 1) (1, 2) (2, 1) (2, 2) = + = + Smaller pitch
Larger pitch
13. In matrix form 13 = (1, 1) (1, 2) (2, 1) (2, 2) Estimate
via optimization
14. Informative layers exist sparsely along to depth
Optimization Estimate informative layers 14 (1, 1) (1, ) ( , 1) ( ,
) Uninformative layers 1 Uninformative layer goes to zero 2 2 + 1
0arg min subject to From many candidate PSFs Direct components
Brightness Clear layers
15. Experimental Setup 15 Cooled CCD camera Target object Beam
splitter Projector (with NIR light) Pair of same lenses
16. Results 16 Normal NIR photo Inner layer Scene Upper layer
Recovery
17. Results Recovery of the painters signature 17 Upper layer
Inner layerPart of paint
18. Color image extension Mural paint covered by white mold 18
Scene Recovered original paint
19. Applications Arts / History Oil paints Mural paints,
ancient documents Forensic Evidence recovery Medical Skin Oil paint
Ancient document 19 Skin layers Oil paint Disguised secret
22. How many patterns do we use? We use about 15 pitches of
patterns, and each of them, we shift the pattern for one-third of
the pitch, so total projection is about 270 patterns. 22
23. How the real world phenomena relates to this work? Its
important thing. High frequency illumination method separates
diffuse reflection and subsurface scattering. However, diffuse
reflection is a kind of subsurface scattering in a small scale
view. So, how to define them? They are defined by some scale
threshold and in high frequency illumination, this threshold is
determined by the pitch of the pattern. This is our interpretation.
23
24. What type of projector do we use? We use a projector
development kit, named Lightcommander, manufactured by Texas
Instrument. FYI Camera: Aspen CG-6, Apogee Optical components:
OptoSigma, Edmund Optics 24
25. How many layers can be recovered? We recovered 3 layers in
a stacked translucent paper scene. We expect our method can recover
more layers. 25
26. Why the result of oil paint is gray-scale? Because we use
infrared light as a light source. Inner layer cannot be seen using
visible lights, so the observation is not a superposition of
layers. On the other hand, Using near infrared, inner layer can be
slightly seen, so we use near infrared for observing oil paints.
26
27. Did we apply to any real items? No, not yet. However, Now
we are discussing with a museum researcher in Nara, a Japanese
historical city. 27