physically based sound
Post on 13-Jan-2016
66 Views
Preview:
DESCRIPTION
TRANSCRIPT
Physically Based Sound
COMP259 Nikunj Raghuvanshi
Overview
Background
FEM Simulation
Modal Synthesis (FoleyAutomatic)
Comparison/Conclusions
Motivation
Sounds could in-principle be produced automatically, just like graphics: Sound Rendering
Sound Rendering has not received much research effort
Main Goal: Automatic generation of non-music, non-dialogue sound
Sound Production Today
Movies: Foley Artistshttp://www.marblehead.net/foley/index.html
Games: Anyone noticed the huge sound directory in Unreal Tournament?
PBS: Sound Production in Nature
Collisions/Other interactions lead to surface vibrations
Vibrations create pressure waves in airPressure waves sensed by ear
Surface Vibration Pressure Wave Ear
Vibration Propagation Perception
Main Aims of PBS
Physics simulator gives contact/collision information
Assign material properties for sound, Wood, concrete, metal etc.
Sound simulator generates sound using this data (in real time?)
Challenges
Sound must be produced at a minimum of ~44,000 Hz
Extremely High Temporal Resolution (timesteps in the range of 10-6-10-8 s)
Stiffness of underlying systems (eg. Metallic sounds. K/m~=108)
Stability may require even smaller timesteps
Two Approaches
FEM deformable simulationO'Brien, J. F. et. al., “Synthesizing Sounds from Physically Based Motion.” SIGGRAPH 2001.
FoleyAutomatic (Modal Synthesis)Kees van den Doel et. Al., “FoleyAutomatic: Physically-based Sound Effects for Interactive Simulation and Animation.” SIGGRAPH 2001.
Main ideas
Deformable Simulation (arguably) much more “physically based”
Foley Automatic: Additive Synthesis
Component Sinusoids
Sound Signal
Overview
Background
FEM Simulation
Modal Synthesis (FoleyAutomatic)
Comparison/Conclusions
Simulation Requirements
Temporal ResolutionSimulate Vibration as well as PropagationVibration Modeling: Deformable Model for
ObjectsPropagation Modeling: Explicit Surface
RepresentationPhysical/Perceptual Realism
System Structure
Vibration Modelling
FEM with Tetrahedral Elements Linear Basis Functions, green’s strain Explicit Time Integration Typically #nodes = 500, #elements = 1500,
dt = 10-6-10-7 s
Sound Propagation Modelling
Fluid Dynamic FEM simulation of surrounding air? Very expensive. Instead…
Employ Huygen’s Principle: Pressure Wave may be seen as sum of pressure wavelets
ReceiverReceiver
Pressure Wave Pressure
“Wavelets”
n̂ v
ds
nvzp ˆ
msPacz /415 Acoustic Impedance of Air
Surface Vibrations and Sound
Pressure contribution of a patch,
Velocity
Density of Air
Sound Propagation Speed in Air
Unit Normal
Surface Vibrations and Sound
Approximate differential elements with surface triangles
Apply band pass filters: Low pass: windowed sinc filter High pass: DC blocking filter
Result: Pressure known for all surface triangles
Putting it all together
)cos(~
)( rx
apts rx
Pressure/Signal at Receiver
Filtered Average Pressure
Area of Triangle
Visibility Term
Approximation of Beam Pattern
Distance Falloff
n̂
Receiver
r
Vibrationx̂
Propagation Delay
Accumulation Buffer
c
dDelay
Receiver
d1
d2
Source
t=0
t1= d1/c
t2= d2/c
1
2
Receiver Distance from Source
Sound Propagation Speed
Results: Capabilities
General models
Generated sounds are accurate
Stereo Sound
Doppler’s Effect
Demo
Results: Accuracy
Results: Speed
Scene TimeStep(s) Nodes/Elems Time/Audio Time
Bowl 10-6 387/1081 91.3/4.01 mins
Clamped Bar 10-7 125/265 240.4/1.26 mins
Vibraphone 10-7 539/1484 1309.7/5.31 mins
(~1 day)
Timings on a 350MHz SGI Origin MIPS R12K processor
Overview
Background
FEM Simulation
Modal Synthesis (FoleyAutomatic)
Comparison/Conclusions
Features
Modal resonance model of solids Location dependent sounds Impact, slide, roll excitation models Real-time, low latency Easy integration with simulation/animation Practical Do not model propagation of sound from source
to receiver
Synthesis Method
ForceForceVibrationVibrationEmissionEmission
PropagationPropagation ListenerListener SpeakersSpeakers
Sound SamplesSound Samples
User
Vibration
),(),(]1
),([2
2
2txFtxu
tcx
xg iii
i
Surface u(x,t) of body responds to external contact force F(x,t)
u(x,t)F(x,t)
Strain Functional Speed of Sound
Under suitable boundary conditions, the solution to the PDE is a sum of sinusoids
Emission
Sound pressure s(t) linear functional L of surface vibration u(x,t)
)],([)( txuLts i
u(x,t)Ls(t)
nvzp ii ˆ~
Note that propagation is not modeled in above
The Modal Synthesis Model
u(x,t) F(p,t)Ls(t)
Impulse response/modal model
“The response u(x,t) of an arbitrary solid object to an external force can be described as a weighted sum of damped sinusoids”
Since L is linear, it implies at s(t) must be a sum of damped sinusoids too
Example: A 1D string
1st Mode 2nd Mode Frequency = f0
…Higher modes Frequency = f1= 2*f0 Frequency = fk= k*f0
)2sin( 000 tfea td )2sin( 11
1 tfea td )2sin( tfea ktd
kk
Main Idea: Sum contributions of all the modes
The point of impact decides the proportions in which the modes are to be mixed: ak. Therefore, ak is a function of p, the point of impact
The frequencies and damping parameters are a property of the object, and independent of how the object is hit
+ +...+
a0a1 ak
The Modal Synthesis Model
u(x,t) F(p,t)Ls(t)
)2sin()()(1
tfepats ktd
N
kk
k
Impulse response,
modal model
Parameters measured experimentally
Kth mode: Gain Factor Point Damping Vibration of impact Term Frequency
Force Modeling
ImpactSlidingRolling
Wavetable
Stochastic
At runtime: Find gain parameters given the location, strength and kind of force.
Synthesize sound from previous equation.
Impact Forces
•Duration: hardness (T)•Magnitude: energy transfer (w)•Multiple micro-collisions
TtTtwtF 0)),/2cos(1()( Example:
Sliding/Scraping
Micro-collisions lead to noisy audio-force
Sliding/Scraping
Wavetable approach Store force parameters Modulate amplitude with energy transfer Modulate rate with contact speed
Synthesis Approach Fractal noise represents roughness Filter through reson filter Resonance ~ contact speed Width ~ randomness of surface
Rolling
No relative surface motion
Differences with sliding:•Smoother: Use low pass•More damping•Harder to create•Less understood•Essential coupling?
Rolling: Smooth Surfaces
Polyhedral objects do not lead to smooth rolling forces
Instead use smooth surfaces directly
Rolling: Contact Evolution
Evolve the contact in Reduced coordinates
q = (u,v,s,t, )
q q q .. .
c(u,v)
d(s,t)
Rolling: Contact Evolution
Piecewise parametric surfaces, loop subdivision surfaces
Explicit integration, no stabilization Multiple contacts and conforming contacts
are not handled Used only when multiple contacts in close
spatio-temporal proximity
Demo
Dynamic Forces
Contact force
Rolling speed
Slipping speed
Impulses
…and locations
Pebble-in-Wok Demo
Results
0.1% CPU time per mode Graceful degradation of quality The bell demo is interactive Uses a PHANToM for interaction Authors do not report any real timings State that “sound quality” is perception-
based and has no metric as of now
Overview
Background
FEM Simulation
Modal Synthesis (FoleyAutomatic)
Comparison/Conclusions
Discussion
FEM: Physically Rigorous and GeneralToo slow for interactive applicationsDoesn’t scale wellInappropriate to apply a 30fps technique to
44000fps?Maybe too general for the problem
domain?
Discussion
Modal model exploits the vibrational nature
Higher EfficiencyBut, not rigorously physically basedFinding the parameters requires
experimentation and “earballing”No rigorous correlation between physical
and perceptual parameters
Discussion
For Realtime: Need for a technique to cover the middle ground
Extracting modal parameters in general requires solving PDEs
Not possible to do in an automated manner
Approximate modal parameters and then use modal synthesis?
Conclusion
PBS involves orders of magnitude smaller temporal and spatial scales
Research is sparse, problems are denseMain contributions of the two papers
besides vibration modeling: FEM: Efficient modeling of sound propagation FoleyAutomatic: Efficient, Approximate models
to handle surface properties and contact forces
References
O'Brien, J. F., Cook, P. R., Essl G., "Synthesizing Sounds from Physically Based Motion." The proceedings of ACM SIGGRAPH 2001, Los Angeles, California, August 11-17, pp. 529-536.
Kees van den Doel, Paul G. Kry and Dinesh K. Pai, “FoleyAutomatic: Physically-based Sound Effects for Interactive Simulation and Animation” Computer Graphics (ACM SIGGRAPH 01 Conference Proceedings), pp. 537-544, 2001.
Acknowledgements
Some images were taken from the referred papers and the corresponding SIGGRAPH slides
top related