Investigation of Tracer Particles Realizing 3-Dimensional WaterFlow Measurement for Augmented Swimming Training

Shogo YamashitaThe University of Tokyo

Tokyo, Japan

Shunichi SuwaThe University of Tokyo

Sony Computer Science LaboratoriesTokyo, Japan

Takashi MiyakiThe University of Tokyo

Tokyo, Japan

Jun RekimotoThe University of Tokyo

Sony Computer Science LaboratoriesTokyo, Japan

ABSTRACTPrevious studies have revealed that understanding the 3D move-ment of water contributes to improving propulsion in the waterwhen swimming. Fluid measurements are made by scattering tracerparticles into a liquid, and cameras track the movement of theseparticles to measure the �uid’s �ow. Strong lasers illuminate tracerparticles to make them visible to cameras, but 3D water �ow mea-surements in wide spaces like swimming pools have not yet beensuccessful. This can be owed to the limitations of current opticalsystems impacting the measurable parameters of existing methods,such as the laser capacity and lens size. Moreover, visualized tracerparticles buoyed in swimming pools can a�ect the swimmer’s viewand may be harmful to humans when swallowed. Therefore, wepropose a 3D water �ow tracing technology with tracer particlessuitable for a swimming pool. We use an optical property calledoptical rotation to track the tracer particles. This method wouldbe e�ective in extending the measurable area of water �ow thanprevious methods because it does not require the use of opticalsystems, which are technically di�cult to expand. In this study,we investigated the materials and processing methods for creatingtracer particles for augmented swimming training.

CCS CONCEPTS•Human-centered computing→ Interaction techniques;Vir-tual reality; Mixed / augmented reality;

KEYWORDSUnderwater Spatial Interaction, Underwater Virtual Reality, FluidMeasurement, Swimming, Augmented Sports

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor pro�t or commercial advantage and that copies bear this notice and the full citationon the �rst page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior speci�c permission and/or afee. Request permissions from permissions@acm.org.AH2018, February 7–9, 2018, Seoul, Republic of Korea© 2018 Association for Computing Machinery.ACM ISBN 978-1-4503-5415-8/18/02. . . $15.00https://doi.org/10.1145/3174910.3174918

(A) Above water Surface (B) Under water Surface

Figure 1: Transparent tracer particles for measuring 3-dimensional water �ow. The particle only becomes slightlyvisible in water but is traceable with cameras.

(B) Tracer particles following water flow(A) Bright tracer particles and dark background

Figure 2: Tracing methods for transparent tracer particles.We use polarization-based technology to make the particlesbright against a dark background. This this creates ideal con-ditions for tracking particles using image processing.

ACM Reference Format:Shogo Yamashita, Shunichi Suwa, Takashi Miyaki, and Jun Rekimoto. 2018.Investigation of Tracer Particles Realizing 3-Dimensional Water Flow Mea-surement for Augmented Swimming Training. In AH2018: The 9th Aug-mented Human International Conference, February 7–9, 2018, Seoul, Republicof Korea.ACM, New York, NY, USA, 9 pages. https://doi.org/10.1145/3174910.3174918

1 INTRODUCTIONWe propose a three-dimensional (3D) water �ow tracing technol-ogy meant to augment swimming training. This technology wouldbe helpful for swimmers because it would enable them to viewvisualized water �ows caused by their strokes in a swimming pool.Previous research revealed that an understanding of water’s 3D

AH2018, February 7–9, 2018, Seoul, Republic of Korea S. Yamashita et al.

movement contributes to improved propulsion in water. For ex-ample, a swimsuit designed to create water �ows reduced waterresistance signi�cantly, allowing swimmers to swim faster [6]. Thesuit creates water �ows to reduce the water’s resistance. In the2008 Beijing Olympics, 98 % of all medals were won by swimmerswearing the suit, and those swimmers were responsible for 23 out ofthe 25 world records set during the games. Moreover, some studiesshow that aquatic animals such as �sh and aquatic insects make3D water �ows, creating vortexes of water to swim faster [8, 20].Measurements of 3D water �ows for aquatic animals, as well asobjects such as swimsuits and pipelines, have been achieved byexisting methods. However, it is still not feasible to measure the3D movement of water caused by swimmers’ strokes due to mea-surement limitations and possible adverse e�ects to humans whenusing current methods.

Flow measurements can be assessed using tracer particles thatare scattered in �uids [9]. Cameras track the movement of theseparticles to measure water �ow while strong lasers illuminate theparticles so they are visible to cameras. This method is known asparticle image velocimetry (PIV) [11]. Several researchers haveproposed 3D �ow measurement using multiple cameras, but 3Dwater �ow measurements in a wide space like a swimming poolhave not yet been conducted. One of the reasons for this is thatthe measurable area of existing methods has been limited due tolimitations in optical systems, such as the capacity of the laser andthe size of the lens. Enlarging the optical system is technically di�-cult. In addition, cameras cannot trace the 3D positions of particlesoccluded by other particles, so the number of tracer particles thatcan be seen at once is limited. Moreover, visualized tracer particlesbuoyed in a swimming pool may obstruct swimmers’ views andcould be damaging to humans if swallowed. Human eye contactwith lasers is also inadvisable.

Given these factors, we propose a 3D water �ow tracing tech-nology that uses tracer particles suitable for a swimming pool. Theparticles are highly transparent and do not block a swimmer’s view.Furthermore, we propose a polarization-based tracking techniquefor the transparent tracer particles. This method would extend themeasurable area compared to previous methods because it doesnot require the use of optical systems, whose views are di�cult toenlarge. Therefore, measuring tracer particles in wider 3D spaceswould be more feasible compared to existing methods. The pro-posed tracer particles must also be functional that is, they shouldmove in accordance with water �ows in the measurement �eld andmust be visible by cameras.

Creating tracer particles with these characteristics and proposinga tracing method for them would make a signi�cant contributionto water �ow tracing in swimming pool environments.

2 INVESTIGATION CONTENTIn this study, we investigated the materials and processing methodsfor creating tracer particles for augmented swimming training. Wealso researched tracing methods for �nding tracer particles andrecognized the numbers and positions of tracer particles occludedby other particles.

We created square transparent tracer particles that do not blockswimmers’ lines of sight. The particles have a refractive index close

Light

Polarizing sheet (45°) Polarizing

sheet (-45°) Camera

Light

Polarizing sheet (45°)

Polarizing sheet (-45°) Camera

Tracer particlewith

optical rotation

Light is not delivered to camera Light is delivered to camera

Figure 3: The basic proposed tracer technologymethodology.We use an optical property called optical rotation, which ro-tates the plane of polarized light so the particles only appearbright against a dark background.

Number: 1Color: White

Number: 2Color: Orange

Number: 3Color: Green

Polarization color chart with consideration for dispersion

Figure 4: Recognition of occluded tracer particles by otherparticles. The color or brightness of the particles changesbased on their total thickness. This phenomenon is shownin materials that can rotate the planes of polarization.

to that of water, and as such, the particles cause less refractionon the surface between the particles and water. Accordingly, thetracer particle is only slightly visible to humans in water (Figure 1).Thus, we also propose an optical solution in which polarization-based technology traces the transparent tracer particles’ positions.In this method, we used optical rotation to rotate the plane ofpolarized light, which allows tracer particles to become bright ondark backgrounds. This is ideal for optical tracking using cameras(Figure 2).

Figure 3 outlines the basic methodology of this tracking technol-ogy. We placed a �at white light source in water and covered it witha polarizing sheet (45�) and positioned a camera to face the lightsource. We also placed a polarizing sheet on the lens of the camera(�45�). The light source was dark when viewed from the cameradue to light shielding caused by the polarizing sheets, whose polar-izing angles cross one another. However, with the tracer particlesrotating the angles of the polarizing sheets, the light source can stillreach the camera (Figure 3). Consequently, the particles are onlybright when compared to the background. The color or brightnessof the particles changes in accordance with their thickness

The color or brightness of the particles changes in accordancewith their total thinness (Figure 4). The tracking system can thusrecognize how many particles are occluded by using this informa-tion. If the refractive index of transparent particles is close to thatof water, the particle has less optical distortion on the surface andthe tracking system is able to determine the number of particles

3-Dimensional Water Flow Measurement for Augmented Swimming Training AH2018, February 7–9, 2018, Seoul, Republic of Korea

occluded by others. However, slight optical distortion on the surfacemust be removed by using existing distortion correction techniques.We also investigated optical solutions to that problem in this study.

3 RELATEDWORK3.1 Water Flow Measurement for SwimmingSakakibara et al. investigated the water �ows caused by swimmers’strokes [14, 15]. The study measured and compared the 2D water�ows caused by several amateur and professional swimmers. Theauthors were able to observe speci�c vortexes of water in the �owdata of professional swimmers that were not present in the dataof amateur swimmers. This result implies that understanding howto create water �ows could improve propulsion in the water, and3D water �ow measurements would help in understanding thedetails of water �ow. In the study, swimmers were asked to swimin a swimming machine that created water �ows from the front tosustain the swimmers’ position. Therefore, the machine-propelledwater �ow should generate �ow data. By creating a greater water�ow measurement method for swimming pools, we can remove themachine as a variable and clarify water �ows in an actual swimmingenvironment.

3.2 Flow Measurement Methods3.2.1 2D Flow Measurement. Tracer particles distributed in a

�uid are often used to visualize �ows within that �uid. One of theeasiest ways to visualize the movement of a �uid is to scatter tuftsor small particles within it. This makes the �ows recognizable, butit is necessary to trace the positions of each particle to perform�uid measurements. PIV is widely used for tracing particles in �uid,and the technique employs lasers to illuminate the measurementenvironment in which to visualize the particles. For 2D �ow mea-surements, a �at and widened laser called a laser sheet is used toilluminate a layer of tracer particles buoyed in the �uid. Since onlythe particles in the laser sheets are brighter than the background, acamera can track the movement of the particles in 2D. The systemcan measure the 3Dmovements of particles in a 2D plane if multiplecameras are used; this method is called stereo PIV [21].

3.2.2 3D Flow Measurement. Scanning stereo PIV is a methodof measuring water �ows in a 3D space using stereo PIV in multiplelayers in the measurement �eld [2, 3]. This scanning is achievedby moving the laser sheets from the edge of the measurement �eldto other sites along the time axis. Although scanning stereo PIVcan measure 3D water �ows in 3D space, it cannot measure �owscaused by the strokes of swimmers because it takes considerabletime to completely scanwider 3D spaces. Tomographic PIV achieves3D �ow measurement in a limited 3D space [13]and this methoduses multiple cameras surrounding the measurement �elds witha laser illuminating the 3D space. Using di�erences in the imagesfrom several cameras, the system reconstructs the 3D positions ofthe particles. However, the measurable 3D space is limited due toinsu�cient laser intensity, which must be widened three dimen-sionally. In addition, the cameras cannot track the positions ofparticles occluded by other particles, so the resolution is limiteddue to decreased particle density. Our approach should measurewider 3D spaces compared to previous methods using lasers as a

light source. Our method uses wide white light sources, which arewidely available as room lights and monitor backlights. Since thecamera views the light source directly, the light intensity can belower than when laser sheets are used to illuminate the particlesfrom the side. To extend the measurable area, we can simply addmore light sources and cameras into the measurement �eld. As a re-sult, extending the measurable area in our method is less costly thanin other methods of 3D water �ow measurement. Our approachalso has the potential to track the 3D positions of occluded particlesusing the optical property of tracer particles, where the color or thebrightness change depending on the number of particles hiddenbehind each particle.

Xiong et al. proposed a 3D �ow measurement technology in 3Dspace utilizing a single camera [22]. This method uses chromaticaberration caused by a lens, and the depths of the particles areexpressed as color di�erences in each particle. However, extendingthe measurable area using this technology is costly. This methoduses a parallel light source, and the measurable area is limited tothe 3D space illuminated by the light. The maximum thickness ofthe parallel light is the diameter of the lens or a collimator usinga mirror. Using this method to measure a wider �eld such as aswimming pool is thus technically di�cult, as a large optical systemwould be required to create the wide parallel light.

3.2.3 Without Tracer Particles. There are many methods of de-tecting water �ow speed, such as laser doppler velocimetry, Pitottubes, and hot-wire anemometers. However, they are only availablefor measuring a single direction of water �ow. Additionally, somemethods do not require tracer particles in the �uid to visualize wa-ter �ows; Schlieren photography visualizes di�erences in densityas a range of brightness [5]. This method enables recolonizationof �ows in a transparent gas or �uid. However, this method alsorequires a parallel light to visualize the �ows accurately. There-fore, extending the measurement �eld is also di�cult and costly.In addition, even if the �ow is recognizable from the di�erences inbrightness, reconstructing a 3D �ow is not feasible from the bright-ness information alone. Micro bubbles made by water electrolysiscould visualize water �ows without adding tracer particles in themeasurement area [12]. However, this method has a potential riskof electric shock for humans since the electrodes must be in thewater when measuring. Recognizing the 3D position of the bubblesis also not feasible due to the bubbles’ occlusion of one another.The bubbles created using this method tend to be whitish, and therecognition of the bubbles’ hidden sides is not always possible.

3.3 Augmented Swimming TrainingSome relevant studies have sought to improve training e�ciencyin water sports. Ukai et al. proposed a swimming support systemusing an underwater robot that shows swimmers their swimmingform in real time [18]. The robot swims alongside swimmers sothey can view the robot’s movements at all times. AquaCAVE isa surround-screen swimming pool for underwater virtual reality(VR) and enhanced swimming training, which displays supportiveinformation such as swimming form and previous records [23].Showing swimmers visualized water �ows and instructing them inhow to make proper vortexes of water should enhance the e�ect ofexisting augmented swimming training systems using displays.

AH2018, February 7–9, 2018, Seoul, Republic of Korea S. Yamashita et al.

3.4 Water Flow Measurement forEntertainment Swimming Pool

3.4.1 Surround-Screen Swimming pool. Surround-screen swim-ming pools help swimmers to maintain the motivation to swimby displaying entertainment contents in the pool. These pools areinstalled in several hotels throughout the world. Moreover, somecompanies have created entertainment swimming pools displayinginteractive video contents using projectors or monitors. AquaCAVEis one such surround-screen swimming pool.

3.4.2 Underwater Spatial User Interaction. One of the biggestentertainment challenges in a surround-screen swimming pool iscreating user interaction because the surround screen and waterweaken many position-tracking methods. The main reasons forthis are optical distortion, re�ection, and IR absorption caused bythe water. In addition, position-tracking technology using visiblelights is unstable when complex backgrounds are displayed onthe surround-screen. 3D water �ow measurements should enrichthe entertainment experience by enabling underwater spatial userinteraction by utilizing water �ows in the swimming pool.

When measuring tracer particle movement in �uids, laser sheetsoften illuminate the measurement environment [4]. However, thisillumination is not suitable for surround-screen environments be-cause the light itself and the visualized particles prevent usersfrom viewing the on-screen content. The proposed 3D water �owmethod only requires lighting from screens, and the transparenttracer particles can provide clear views for swimmers.

3.4.3 Transparent Tracer Particles for 2D Water Flow Measure-ment. Yamashita et al. proposed a water �ow visualization tech-nology using �at, transparent tracer particles with birefringence.The system achieves water �ow visualization without blocking theswimmer’s view, but 3D measurements cannot be assessed due tothe particle shape [23]. The study used soft plastic �les as tracerparticles, but 3D position tracking using cameras was not feasiblewith the particles in part because the forms of the particles changeddue to water �ows.

In addition, there are several issues that reduce user immersionin the contents displayed in a swimming pool. For example, theparticles have re�ections on the surface when viewed from certainangles, so some of the particles buoyed in the water are not com-pletely transparent and block users from viewing the displays. In afeasibility test in the study, one of the swimmers commented thatparticles often stuck to swimmers’ bodies due to their �at shape.This physical sensation may reduce the entertainment experiencein the surround-screen swimming pool, so we propose a methodthat involves transparent spherical tracer particles.

4 3D WATER FLOWMEASUREMENT FORAUGMENTED SWIMMING TRAINING

4.1 Technical Problems with Existing MethodExisting 3D water �ow measurement methods are not suitable forswimming pools due to limitations in the maximum measurablearea and adverse e�ects on the human body. Fluorescent micro-sized plastic particles are often used as the tracer particles in current

Other lane

One lane of swimming pool

Partition wall

Multiple waterproof cameras with dome-shaped lenses and polarization sheets (0°)

Wide white light source with polarization sheet (90°) along the wall of the swimming pool

Tracer particles1cm in diameter1/500cm3 intervals

Figure 5: System con�guration in a swimming pool. This �g-ure shows the setup formeasuring 3Dwater�ows in a swim-ming pool lane.

methods. The tracking system shines a strong laser on the measure-ment �eld to visualize the tracer particles, but this combination isnot advisable for �ow measurement in swimming pools becausethe illumination may harm human eyes and the tiny particles mayenter the human body through the mouth, nose, eyes, and/or ears.Furthermore, 3D water �ow measurements in a wide 3D space likea swimming pool lane is not feasible with existing methods, whichrequire a laser or parallel light illumination. Tracer particles in theilluminated areas are thus only traceable using cameras. Expandingthe area of illumination to a wider 3D space requires considerableexpenses due to the sophisticated optical system required.

4.2 Proposed Method4.2.1 System Configuration and Basic Methodology. we inves-

tigated 3D water �ow tracing technology with the potential tomeasure a wider area of swimming pools; this method would havefewer adverse e�ects on the human body.

As the tracer particles, transparent sphere particles about 1cmin diameter were created. The particles were made of a polymerabsorbing a solution of sugar in water. Sugar has an optical propertycalled optical rotation, which rotates the plane of polarization, so weused an aqueous fructose solution in this study. For tracing the po-sitions of the tracer particles distributed in the water, we employeda polarization-based method that provides bright tracer particlesagainst a dark background, which is ideal for optical tracking [19].

The system con�guration in an actual swimming pool is shownin Figure 5. This �gure illustrates the system con�guration whenwe measure 3D water �ows in a lane of a swimming pool. First,we must place partition walls between the measurement �eld andthe other lanes to limit the number of tracer particles moving outof the measurement area. The partition walls were equipped withmultiple waterproof cameras and we placed the cameras insidewaterproof dome-shaped lenses. This con�guration eliminates theoptical distortion caused by water. We use a GoPro 6 Hero cameraand a Telesin 6” dome as the dome-shaped lens in our feasibilitytest. Cameras were placed facing a wide white light source in the

3-Dimensional Water Flow Measurement for Augmented Swimming Training AH2018, February 7–9, 2018, Seoul, Republic of Korea

water; the light source was covered with a polarizing sheet (90�),as were the cameras (0�).

The light source appeared dark when viewed from the camerasdue to light shielding caused by the polarizing sheets on the lightsource and cameras. This happens if the polarizing angles are or-thogonally crossed. Any combination of crossed polarization anglesworks for this tracing method. Tracer particles were distributed inthe water between the light source and the cameras. The transpar-ent tracer particles can rotate the plane of polarized light from thelight source, which causes particles to become brighter than thebackground and therefore traceable by the cameras. Using di�er-ences in the images taken bymultiple cameras, we could reconstructthe 3D positions of the tracer particles. The tracer particles were1cm in diameter, and the number of tracer particles was 1/500cm3.These numbers were determined by our feasibility test to visualizewater �ows caused by swim strokes in a water tank. However, thesize and number of particles should be adjusted according to themeasurement �eld.

4.2.2 Safety and Extension of the Measurable Area. To make thetracer particles trackable by the cameras, it is best to use a widewhite light source such as room light or a monitor back light. Thisform of illumination is safer for humans compared to lasers. Thetracer particles were also less likely to enter the human body sincethey are bigger than facial ori�ces. Even so, for safety reasons,tracer particles should only be made of food, as shown in Figure 6(A). The outlined ingredients are widely used for making arti�cialikura (arti�cial salmon egg) [1]. This way, harmful e�ects fromaccidentally swallowing tracer particles are mitigated. We usedsugar, alginic acid, and calcium in the feasibility test to create thetracer particles (i.e. edible substances). The tracer particles showedoptical rotation as a water-absorbing polymer, and the outer surfacemade of alginic acid and calcium lactate was robust enough to retainits form in the water for over three hours (Figure 6 (B)).

Since the particles are biodegradable, we can create tracer parti-cles that automatically disappear after use by adjusting how longthey remain intact. This feature could bene�t the proposed methodin a swimming pool signi�cantly.

Extending the area of illumination is also more feasible than inprevious methods by using a laser sheet or a parallel light, since thismethod does not require a complex optical system. To extend themeasurable area, we can simply place additional wide light sourcesnext to the original light source and additional cameras to coverthe entire measurement �eld.

4.2.3 Recognition of Occluded Tracer Particles by Other Particles.The camera cannot view a particle when other particles occludeit, but certain optical properties of the tracer particles can solvethis issue. Tracer particles that can rotate their planes of polarizedlight produce di�erent colors or brightness levels when they areoverlapped. This optical property is exhibited by materials withthe optical properties of birefringence or optical rotation. The phe-nomenon allows us to recognize how many particles are occluded.This optical property is demonstrated by materials with the opticalproperties of birefringence or optical rotation. The phenomenonallows us to recognize how many particles are occluded.

If the tracer particles’ refractive index is close to that of water,there is less optical distortion caused by the surface of the particles.

Content: aqueous solution of fructoseOuter surface: robust film made of alginic acid and calcium lactate

Substance name Material Fructose FruitAlginic acid SeaweedCalcium lactate Cheese

(A) Tracer particles made only from foodstuffs (B) Robust outer surface

Figure 6: A tracer particle made only of food. The particleis biodegradable and less harmful if a swimmer accidentallyswallows it. Adjusting biodegradation speedwould allow fortracer particles that automatically disappear after use. Thiswill aid in the clearance of tracer particles signi�cantly.

We can therefore assess the 3D positions of each occluded particleby applying distortion correction. In this paper, we investigatedthe feasibility of this method for measuring the occluded tracerparticles.

4.3 Tracer Particles4.3.1 Necessary Condition. Ideal tracer particles should have

the following characteristics:

• A stable and �xed 3D shape at all times.• Speci�c gravity equal to that of water (1.0).• Ability to rotate the plane of polarization.• Refractive index equal to that of water (1.33).

First, the particles must all have �xed and identical shapes inthe water �ow to measure their 3D positions. For example, if theparticles are always square, we can reconstruct the 3D positionsfrom the images taken bymultiple cameras using epipolar geometry.This method is widely used in existing 3D position tracking systemsand motion capture systems.

Second, the particle must have appropriate �otation to movealong with the water. The ideal �otation is when the speci�c gravityof the particle is equal to that of water. For existing tracer particles,materials with a speci�c gravity close to that of water are oftenused. Some types of tracer particles contain air or water to adjusttheir buoyancy, which makes them �ow with the water smoothly.

Third, the particles must be able to rotate their plane of po-larization, chie�y because they must be bright compared to thebackground for 3D position tracing by image processing. Materialsthat have multiple scattering or �uorescence are also suitable forthis purpose because the optical property cancels the polarizationand the light can thereby be delivered to the camera. However, weemploy rotatory polarization because the above method may blockthe swimmer’s view. The other reason we use optical rotation is tomake occluded tracer particles recognizable. This optical propertyshows di�erent colors or brightness levels when tracer particlesare overlapped.

Fourth, the particle must have a refractive index close to that ofwater for the proposed method; otherwise, the particle will causeoptical distortion on the surface. When the refractive index is veryclose to that of water, the transparent tracer particles become in-visible in water because there is no refraction on the surface. This

AH2018, February 7–9, 2018, Seoul, Republic of Korea S. Yamashita et al.

situation is ideal when the proposed method is used for entertain-ment swimming pools, as the particles do not prevent users fromviewing the contents displayed in the pool. If the optical distor-tion is lower, we can recognize occluded particles using imageprocessing-based distortion correction methods widely used forlens correction.

4.3.2 Materials and Processing Methods. We discovered that ma-terials that can contain large amounts of aqueous solution, such aswater-absorbing polymer balls and spherical agar, have the potentialto ful�ll the necessary tracer particle conditions. By incorporatingan aqueous solution with optical rotation into the material, wecan create tracer particles that can rotate the plane of polarization.When the material absorbs a large amount of water, the speci�cgravity becomes close to that of water and the refraction index willalso become like that of water. The tracer particles then becomeinvisible in the water due to less refraction on the surface, followingthe water �ow more smoothly due to its new speci�c gravity.

Some aqueous solutions of chemical substances have the charac-teristic of optical rotation, such as sugars like sucrose and fructose.Sucrose is common household sugar whereas fructose is knownas fruit sugar and is found in honey and cereals. The safety ofthese substances has been certi�ed and they are widely availableto consumers. Tracer particle brightness depends on the percent-age of sugar in the particles because the rotation angle changes inaccordance with the sugar content and thickness of the particle.

4.3.3 Technical Problems. There are some technical problemscaused by the including an aqueous solution with optical rotationto tracer particles. One of the issues is that the refractive indexincreases due to the inclusion of the aqueous solution. For example,the refractive index of an aqueous solution of sucrose at a concen-tration of 10% is 1.347 while that of water is 1.333 at 20 0 �C, 589.29nm. This di�erence causes optical distortion on the surface of theparticles. The speci�c gravity of sucrose is 1.587�/cm3 while that ofwater is 1.000�/cm3. The other problem is that the speci�c gravityincreases to 1.038�/cm3 when we add 10% more sucrose to thewater at 20 �C [10]. The speci�c gravity of existing tracer particlesvaries from around 0.92�/cm3(polyethylene) to 1.70�/cm3 (SilverCoated Glass Hollow Spheres) [16]. The speci�c gravity must beclose to that of water to make the tracer particles follow the water�ows smoothly, but the inclusion of an aqueous solution with opti-cal rotation increases the speci�c gravity in accordance with theconcentration.

We investigate these e�ects on tracer particles and the feasibilityof the proposed 3D water �ow measurement method.

5 INVESTIGATION RESULTS5.1 Tracing Method

5.1.1 Traceability Test. We investigated whether a tracking sys-tem using image processing can recognize the positions of tracerparticles in a water tank. Figure 7 illustrates the setup for this feasi-bility test. We prepared a water tank with the following dimensions:38cm⇥ 30cm⇥ 28cm. We �lled the tank with 10cm of water (around11.5 L). We placed 23 tracer particles in the water tank (1 tracerparticle/500 cm3). To trace the tracer particles, we used a software li-brary called the ”BlobDetection library” for processing. This library

Polarizing sheet (45°)

Camera

Tracer particles

Polarizing sheet (-45°)

Wide whitelight source

(A) Test setup for tracing tracer particle

Tracer particles

(B) Image taken by camera above water tank

Figure 7: System con�guration for testing the traceability oftracer particles.

allowed us to detect blobs in images with areas of brightness abovea de�ned value. We used water-absorbing polymer balls, which arewidely used for growing plants, around 1cm in diameter as tracerparticles in this test. Water-absorbing polymer balls absorb morethan 200 times their volume in water. The tracer particles absorbedan aqueous solution of lactose at a concentration of 40% in this test.We used a liquid-crystal display (LCD) monitor (Lenovo ThinkVi-sion LT2223p) as the wide white light source, and the screen wascoated with a polarizing sheet (-45�). We recorded video directlyabove the water tank. We placed a polarizing sheet (-45�) on thecamera lens (LUMIX DMC-FZ1000).

The picture in Figure 7 (B) was taken with the following camerasettings: ISO = 1600, focal length = 23.3 mm, exposure = -1 ev,f-number = f/4, and shutter speed = 1/10.

5.1.2 Result and Limitations. The tracking system was able torecognize the positions of the tracer particles in the water tank inthe feasibility test (Figure 8 (A)). However, we found several issueswith the proposed tracing method.

First, the accountancy was higher when we traced the tracerparticles in a dark room than in a room with light (around 3300lm). Tracking particles in a bright room was possible, but the trac-ing system lost the tracer particles occasionally due to the slightdi�erences in brightness between the tracer particles and the back-ground. Thus, darker measurement �elds are preferable to use thetracing method.

Second, the tracer particles appeared longer than their shapein the images due to the persistence of vision. More persistenceof vision and false detections were observed when the water �owwas faster and the measurement �eld was darker (Figure 8 (B)). Assuch, it is necessary to employ technology that can detect the actualposition of tracer particles using images with persistence of visionto accurately measure water �ow.

Third, suspended matter in water may reduce the accurate de-tection of tracer particles, as some scattering changes the opticalproperties of polarized light. For example, multiple scattering can-cels polarization and Rayleigh scattering turns natural light intopolarized light. These can be caused by suspended matter in wa-ter, so water should be as pure as possible when using this �owmeasurement method.

3-Dimensional Water Flow Measurement for Augmented Swimming Training AH2018, February 7–9, 2018, Seoul, Republic of Korea

(A) Result of blob detection

False detection

(B) Examples of false detections

Figure 8: Tracking tracer particles using image processing.

60 % 50% 40% 30% 20% 10% 0%

13.8 ° 9.20° 5.93° 3.90° 2.30° 1.01° 0°

Concentration

Rotation angle

Figure 9: Relationship between concentration of fructose so-lution and angle of rotation.

5.2 Tracer Particles5.2.1 Brightness. The tracer particles aremade ofwater-absorbing

polymer. In this test, the particles absorbed a fructose solution inconcentrations from 10 to 60%. We compared the brightness be-tween them; Figure 9 shows the relationship between concentra-tions of fructose solution and the angle of rotation [17]. The imagein Figure 9 was taken in a room with light and with the followingcamera settings: ISO = 1600, Focal length = 9.12 mm, Exposure =-0.66 ev, F-number = f/3.1, Shutter speed = 1/8.

According to our test, a greater concentration of fructose solutionproduces brighter tracer particles. Brighter particles are ideal forthe proposed tracing method, but the inclusion of fructose solutionmakes more optically distorting and changes speci�c gravity. Agreater concentration of fructose solution can be used only whenthe measurement �eld is shallow or water �ow is not slight. Thisissue can be reduced by using material that rotates the plane ofpolarization more than fructose, such as maltose.

5.2.2 Optical Distortion. Fructose solutions cause optical distor-tion on the surface of tracer particles based on concentration. Weinvestigated the optical distortion in both the air and underwateras shown in Figure 10. We put six tracer particles that had absorbed0% to 60% fructose solution into a �ask and placed it on a gridpattern. According to the test, slight di�erences in refractive indexcan impact invisibility. Therefore, we must reduce the amount ofthe solution when particle invisibility is important, like in the caseof entertainment swimming pools. However, traceability decreasesin this case. As stated in section 5.2.1, materials that rotate theirplanes of polarization signi�cantly can solve this problem. Severalchemicals can rotate polarized light ten times more than fructose,such as helicene. However, we must ascertain the safety and watersolubility of these materials.

In the air

Underwater

60 % 50% 40% 30% 20% 10% 0%

54 14 6. 35 126. 381 401

(% )

( %)

Figure 10: Relationship between concentration of fructosesolution and optical distortion.

0

20

40

60

80

100

120

140

160

0 2 4 6

Luminance

N = 1 N = 2 N = 3 N = 4 N = 5 N = 6

Lum

inan

ce

N

5 17 742396188 546.601

Figure 11: Relationship between the number of overlappedtracer particles and brightness.

5.2.3 Recognition of Occluded Particles. We investigated thechange in brightness and color depending on the number of over-lapping tracer particles. Figure 11 shows the relationship betweenthe brightness and the number of tracer particles. In this test, weused water-absorbing polymer balls containing a fructose solutionwith a concentration of about 20%. as tracer particles in this test.As shown in Figure 11, the brightness changes linearly. We werethus able to estimate the number of particles hidden behind thefront-most particle in our feasibility test. Tracer particles made ofplastic with birefringence as in Figure 4 turned di�erent colors, butthe tracer particles made of fructose solution did not show colors.This di�erence is caused by varying rotation angles depending onthe wavelength of each material.

We also found that position recognition of occluded particleswas not feasible in most cases. This is because optical distortion issigni�cant when we include over 30% fructose solution. Distortioncorrection can only work when the particle has low optical distor-tion. Finding a material with more rotatory polarization should alsoremove technical problems since we can increase the percentage ofwater in the tracer particles by using the content accordingly.

5.3 Other Material and Processing MethodWe employ water-absorptive materials with optical rotation as thetracer particles in this study. We will describe the reason for thisselection and the results of other material investigations in thissection. For producing tracer particles with less optical distortion inthe water, we canmake the particles from �uororesin. The refractiveindex of the material is 1.34, which is very close to that of water.One technical problem is adding the optical property of rotating

AH2018, February 7–9, 2018, Seoul, Republic of Korea S. Yamashita et al.

(B) Stamping �(A) Injection molding � (C) Stereolithography�(B) Stamping (A) Injection molding (C) Stereolithography

Figure 12: Other materials and processing methods. Stan-dard processing methods for 3D plastic products would re-sult in undesirable optical properties for our purpose.

planes of polarization. While we could make balls of �uororesincontain an aqueous solution with optical rotation, the processing of�uororesin is not easy because the material maintains its shape evenat elevated temperatures. Moreover, swallowing plastic is harmfulto human bodies.

The spherical transparent tracer particles must all have the op-tical property of rotating their plane of polarization throughoutthe whole body of the particles. However, standard processingmethods for 3D plastic products would result in undesirable opticalproperties for our purposes. For example, clear plastics made byinjection molding do not have any optical properties (Figure 12 (A)).Therefore, the plastics would not brighten or show any color whenviewed between polarizing sheets. Furthermore, stamping resultsin di�erent refractive indexes depending on the location. For thisreason, for this reason, clear plastics made by stamping show multi-ple colors even when their thicknesses are the same (Figure 12 (B)).According to our tests on the material and the processing methodfor making the proposed tracer particles, a 3D printer using a stere-olithography apparatus can meet the requirements. This printingtechnology creates 3D models in a layer-by-layer fashion usingphotopolymerization [7]. A laser light causes chains of moleculesto link and form polymers. These regularly aligned molecules giveuniform birefringence to the printed model (Figure 12 (C)). Ma-terials with birefringence can rotate their plane of polarization.This produces color variation relative to the thickness of the plas-tics. However, the relationship between color and thickness hasregularity only when the particles are overlapped with the sameorientations in the event of birefringence. Therefore, to accuratelyestimate the number of overlapped particles, we must consistentlytrack the orientation of the particles. In the case of optical rotationdemonstrated by chemical substances such as fructose and sucrose,regularity is maintained even if the orientation of each particle isdi�erent. Therefore, the color or brightness of the particles alwaysrepresents the total thickness of the particles in the images takenby cameras.

6 CONCLUSIONWe proposed 3D water �ow measurement technology that entailsscattering spherical transparent tracer particles in water. The tech-nology has the potential to realize 3D water �ow measurementin a swimming pool, which has not been achieved by previousmethods. In this study, we investigated the feasibility of the tracerparticles and the tracing method. We use light shielding caused bypolarizing sheets and optical rotation caused by tracer particles tomake brighter tracer particles against a dark background in themeasurement �eld. We conducted tests on the traceability of thetracer particles. Identifying bright spots in an image using imageprocessing-based tracing techniques was functional as a tracingsystem. However, the brightness and the clarity of the measure-ment �elds caused several issues; according to our feasibility tests,darker and more transparent measurement �elds are preferablewhen tracing. We also investigated materials and processing meth-ods for tracer particles and found that they require two criteria:1) a stable and �xed 3D shape, and 2) that the chosen material’sre�ective index and speci�c gravity must be close to that of water.An ability to rotate the plane of polarization is also necessary inthe tracing process. According to our research, water-absorbingmaterials containing an aqueous solution exhibiting optical rotationare ideal tracer particles. We also researched the adverse e�ects ofincluding aqueous solutions in this study and con�rmed that theyincrease speci�c gravity and refractive index. Such technical issuesreduce the usability of water �ow measurement technology, but wecan mitigate adverse e�ects by using materials that show strongoptical rotation.

7 ACKNOWLEDGMENTSThis work was supported by ACT-I, Japan Science and TechnologyAgency (JST). The expenses to attend Augmented Human 2018 wassupported by Tateisi Science and Technology Foundation.

This research was partially supported by Hongo Tech Garageat the Division of University Corporate Relations, University ofTokyo. We particularly wish to express our gratitude to Mr. TakaakiUmada and Dr. Tomihisa Kamada for valuable comments on theproject.

3-Dimensional Water Flow Measurement for Augmented Swimming Training AH2018, February 7–9, 2018, Seoul, Republic of Korea

REFERENCES[1] Nicolas Bremond, Enric Santanach-Carreras, Liang-Yin Chu, and Jérôme Bibette.

2010. Formation of liquid-core capsules having a thin hydrogel membrane: liquidpearls. Soft Matter 6, 11 (2010), 2484–2488.

[2] Ch Brücker, D Hess, and J Kitzhofer. 2012. Single-view volumetric PIV via high-resolution scanning, isotropic voxel restructuring and 3D least-squares matching(3D-LSM). Measurement Science and Technology 24, 2 (2012), 024001.

[3] S Burgmann, Ch Brücker, and W Schröder. 2006. Scanning PIV measurements ofa laminar separation bubble. Experiments in Fluids 41, 2 (2006), 319–326.

[4] Gerrit E Elsinga, Fulvio Scarano, Bernhard Wieneke, and Bas W van Oudheusden.2006. Tomographic particle image velocimetry. Experiments in �uids 41, 6 (2006),933–947.

[5] H Fiedler, K Nottmeyer, PP Wegener, and S Raghu. 1985. Schlieren photographyof water �ow. Experiments in �uids 3, 3 (1985), 145–151.

[6] Eisuke Kainuma, Mayumi Watanabe, Chikako Tomiyama-Miyaji, Masashi Inoue,Yuh Kuwano, HongWei Ren, and Toru Abo. 2009. Proposal of alternative mecha-nism responsible for the function of high-speed swimsuits. Biomedical research30, 1 (2009), 69–70.

[7] Po-Ting Lan, Shuo-Yan Chou, Lin-Lin Chen, and Douglas Gemmill. 1997. De-termining fabrication orientations for rapid prototyping with stereolithographyapparatus. Computer-Aided Design 29, 1 (1997), 53–62.

[8] George V Lauder and Eliot G Drucker. 2002. Forces, �shes, and �uids: hydrody-namic mechanisms of aquatic locomotion. Physiology 17, 6 (2002), 235–240.

[9] A Melling. 1997. Tracer particles and seeding for particle image velocimetry.Measurement Science and Technology 8, 12 (1997), 1406.

[10] PS Nikam, HR Ansari, and Mehdi Hasan. 2000. Density and viscosity studies ofglucose and fructose solutions in aqueous and 0.5 mol dm3 aqueous NH 4CL.Journal of Molecular Liquids 87, 1 (2000), 97–105.

[11] Markus Ra�el, Christian E Willert, Jürgen Kompenhans, et al. 2007. Particleimage velocimetry: a practical guide. Springer Science & Business Media.

[12] M Sadatomi, A Kawahara, K Kano, and A Ohtomo. 2005. Performance of anew micro-bubble generator with a spherical body in a �owing water tube.Experimental Thermal and Fluid Science 29, 5 (2005), 615–623.

[13] F Scarano. 2013. Tomographic PIV: principles and practice. Meas. Sci. Technol 24,1 (2013), 012001.

[14] Hideki Takagi, Motomu Nakashima, Yohei Sato, Kazuo Matsuuchi, and Ross HSanders. 2016. Numerical and experimental investigations of human swimmingmotions. Journal of sports sciences 34, 16 (2016), 1564–1580.

[15] Hideki Takagi, Shohei Shimada, Takahiro Miwa, Shigetada Kudo, Ross Sanders,and Kazuo Matsuuchi. 2014. Unsteady hydrodynamic forces acting on a handand its �ow �eld during sculling motion. Human movement science 38 (2014),133–142.

[16] Etoh Takeharu, Takehara Kohei, Yokoyama Yuichi, and Ida Yasuo. 1996. De-velopment Of Supporting Technologies For water �ow Visualization. (1996),87–106 pages.

[17] Y Tsuzuki, J Yamazaki, and K Kagami. 1950. The speci�c rotation of fructose.Journal of the American Chemical Society 72, 3 (1950), 1071–1073.

[18] Yu Ukai and Jun Rekimoto. 2013. Swimoid: a swim support system using anunderwater buddy robot. In Proceedings of the 4th Augmented Human InternationalConference. ACM, 170–177.

[19] Greg Welch, Gary Bishop, Leandra Vicci, Stephen Brumback, Kurtis Keller, andD’nardo Colucci. 2001. High-performance wide-area optical tracking: The hiballtracking system. presence: teleoperators and virtual environments 10, 1 (2001),1–21.

[20] Lalith Niroshana Wickramarathna. 2016. Kinematics and Energetics of SwimmingZooplankton. Ph.D. Dissertation. Universität Koblenz-Landau.

[21] B Wieneke. 2005. Stereo-PIV using self-calibration on particle images. Experi-ments in �uids 39, 2 (2005), 267–280.

[22] Jinhui Xiong, Ramzi Idoughi, Andres A Aguirre-Pablo, Abdulrahman B Aljedaani,Xiong Dun, Qiang Fu, Sigurdur T Thoroddsen, and Wolfgang Heidrich. 2017.Rainbow particle imaging velocimetry for dense 3D �uid velocity imaging. ACMTransactions on Graphics (TOG) 36, 4 (2017), 36.

[23] Shogo Yamashita, Xinlei Zhang, and Jun Rekimoto. 2016. AquaCAVE: AugmentedSwimming Environment with Immersive Surround-Screen Virtual Reality. In Pro-ceedings of the 29th Annual Symposium on User Interface Software and Technology.ACM, 183–184.