Upload
nedaa
View
49
Download
0
Embed Size (px)
DESCRIPTION
Soundpath : Real-Time Portable Control Framework for 3D sound system. Soundpath : 3D 音效系统的 实时便携 控制框架. Outline. Introduction Motivation System Architecture Case Study Conclusion . Introduction. Introduction. - PowerPoint PPT Presentation
Citation preview
无线与传感器网络实验室 Laboratory for Wireless and
Sensor Networks
Soundpath: Real-Time Portable Control Framework for 3D sound
system
Soundpath: 3D音效系统的实时便携控制框架
2
1. Introduction
2. Motivation
3. System Architecture
4. Case Study
5. Conclusion
Outline
无线与传感器网络实验室 Laboratory for Wireless and
Sensor Networks
Introduction
4
Soundpath is a project with the goal of developing real-time portable control framework for 3D sound system such as IOSONO.
The framework uses web service to link various portable devices to 3D sound system.
The portable devices are served as front sensing and control units to generate a 3D sound spectrum.
It eases the creation of installations and oeuvres based on 3D sound system.
An indoor localization system is implemented to link the personal tracking and sound spectrum to enable live performance. It demonstrates the advantages of Soundpath.
Introduction
无线与传感器网络实验室 Laboratory for Wireless and
Sensor Networks
Motivation
6
IOSONO Studio
7
IOSONO system is a world leading audio technology which can provide a 3D Sound effect for interactive designs.To provide such an experience, scenarios are created on another software: the Nuendo Platform. We have to define a source which plays one sound and has one path to create a scenario in Nuendo Platform.Static Active Interaction
Motivation
8
Before we start our project, SCHM has already created a project to do some interaction work using IOSONO.They use an iPad or other IOS device to send standard OSC packets to Nuendo Platform using MAX/MSP and create the sound source.The iPad can provide a rotation matrix. When the user roll or pitch the iPad, the sound will follow its movement.
Motivation
9
We want to develop a framework which can help interactive designers to use various kinds of interactive input such as inertial sensors, smart phones etc to access Nuendo Platform easily.Designers can mainly focus on creating sound effect in art without worring the communication between interactive input and Nuendo Platform.New portable devices can be easily extended and control the IOSONO.Hence, the framework should be scalable, adaptable and easy to extended.
Our goals
无线与传感器网络实验室 Laboratory for Wireless and
Sensor Networks
System Architecture
11
A web application, why not?The aim of developing such a platform is to create an interface capable of being the relay between the interactive input and the 3D sound system.
The Advantages of Web Service□Standardization□Versatility□Cross-platform□Event-driven
The choice of the architecture
12
To meet the constraints, we decided to choose a “three layer” architecture: Controller layer => Handle Sensing data and Request
input, invoke the corresponding Service Service layer => Realize sound effect based on the
sensing data DAO layer => handle the connection with the databaseWe implement an indoor localization system for live performance. Next, I will use it to expain the whole three layer architecture.
A three layer architecture
13
o The smartphone gets two data coming from all the AP in the studio:□ The frequency□ The power of the signal received
o The smartphone sends all the data to the web application via a web GET request
o Each time one of the value changes, the smartphone recalls the request
Controller Layer
14
o Several web-services:□ Trilateration localization□ Centroid localization□ Fingerprinting localization□ Coordinates transformation
o Once receiving requests of webservice, the server will return resultso With the power of signals coming from the AP,
localization algorithm is called => Cardinal coordinates
o Then, transformation of the coordinates => Polar coordinates
Service and DAO Layer
15
o The webservice sents the polar coordinates to the patch Max/MSP
o The patch sends it to the IOSONO systemo One message for one sourceo Sources on IOSONO are directly piloted by the
application
Result
无线与传感器网络实验室 Laboratory for Wireless and
Sensor Networks
Case Study
Indoor localization system with IOSONO for live performance
17
There are 5 APs set up in SiWei studio. The length and width of the room is 6.5m and
5.25m respectively.
Environment
18
A user carrying an Android Smartphone is walking around in the room.
The IOSONO system will generate a sound spectrum following the user’s track.
Expected Result
19
Experiments
Measure the AP signal strength under the condition with/without music
All the sheet of paper you can see represents the position of each data measurements.
20
Experiments Result
Results for one AP (upper left corner) are presented on the two following graphs
the X/Y axes represent the plan of the studio and the Z the signal power.
21
ArchitectureWeb ApplicationAndroid Smartphone
Database
IOSONO IPC
Patch MAX/MSP
With the data sent by the smartphone, the WA calculate the location of the user.Then, the WA gets the sources belonging to the user in the database. It calculates the polar coordinates of each sources and stores it in the database before sending it through the patch MAX/MSP to IOSONO IPC
Power information of each access point is sent to the WA
OSC packet contains the coordinates and ID of the sources
The patch gets the coordinates, ID for one source and sends all this information to the IOSONO IPC
22
http://192.168.0.*:8080/coordinate/real/centroid/-80.0/2452.0/-10.0/2437.0/-100.0/2422.0/-150.0/2412.0/-100.0/2412.0/
Communication format
/iosono/renderer/version1/src 1 0 6.28 -184 1 0 0 0 0 0 0
Android Application will request the Server in the following format
IP address of Server
Page name
Service name
AP1’s signal strength
AP1’s frequency
AP5’s signal strength
AP5’s frequency
OSC packet format for IOSONO system
Polar coordinate
无线与传感器网络实验室 Laboratory for Wireless and
Sensor Networks
Shang Hai J
iaotong
UNIVERCITY
22/10/2
013
Conclusion
24
o We develop a framework for easy connection between IOSONO and interactive inputs.
o The framework uses web service to link various portable devices to 3D sound system.
o Designer can mainly focus on creating sound effect itself and building new web services for new interactive inputs.
o Easy to switch implemented web service o Implement a localization service to link the personal
tracking and sound spectrum for live performance. It demonstrates the advantages of Soundpath.
Conclusion
25
Thank you for your attention
无线与传感器网络实验室 Laboratory for Wireless and
Sensor Networks
Demo
27
Unlike the deployment in SiWei studio, the AP are placed in the Demo Area of this room.
Demo Area is a small scaled region of this room.
While the user is walking around in Demo Area, the sound will follow his track in the same scale of this room.
Demo
Demo Area
28
Weighted Centroid Localization