OpenAL Audio effects: reverb, chorus, delay #1708

fallahn · 2020-10-05T13:21:53Z

Description

Implements OpenAL sound effects for reverb, chorus and delay. Can be easily extended to more types supported by OpenAL.
This was discussed some time (10 years!) ago here: https://en.sfml-dev.org/forums/index.php?topic=2245.0 the conclusion being that available OpenAL implementations were not capable. Thankfully things have changed somewhat in the last decade. I've tested this on platforms using openal-soft (windows and ubuntu) and on macOS using Apple's OpenAL library. I was pleasantly surprised to find that the effects are supported by the Apple library, although they are audibly rendered differently - particularly the delay which sounds much softer. However it seems openal-soft is supported on macOS and building SFML against that makes the effects sound consistent across tested platforms. I'm unable to test on mobile platforms.

I'm submitting this mostly because I find the effects useful, and it would be preferable not to have to maintain my own fork of SFML. I'd also like to gauge interest to potentially develop further with features such as low/high pass filters for example.

Tasks

API Overview

sf::SoundSource - Base class of e.g. sf::Music or sf::Sound
- void setEffect(const SoundEffect* effect); - Set or remove (with NULL) effects to anything implementing SoundSource
- const SoundEffect* getEffect() const; - Get the currently set effect or NULL if none is set
- void resetEffect(); - Internal use to reset the effect when the sf::SoundEffect instance is destroyed
sf::SoundEffect - Sound effect base class
- static bool isAvailable(); - Not all platforms support effects (e.g. iOS)
- void setVolume(float volume); - Volume of the effect in the range or 0 to 1
- float getVolume() const; - Retrieve set volume
sf::ChorusEffect
- void setWaveform(Waveform waveform); - Set the LFO waveform shape, either Sine or Triangle
- void setPhase(sf::Int32 angle); - Set the LFO phase
- void setRate(float rate); - Set the LFO modulation rate
- void setDepth(float depth); - Set the amount by which the delay time is modulated
- void setFeedback(float amount); - Set the amount of processed signal that is fed back to the input
- void setDelay(float delay); - Set the average amount of time the sample is delayed
- Waveform getWaveform() const;
- sf::Int32 getPhase() const;
- float getRate() const;
- float getDepth() const;
- float getFeedback() const;
- float getDelay() const;
sf::DelayEffect
- void setDelay(float delay); - Set the delay between the original sound and the first 'tap', or echo instance
- void setLRDelay(float delay); - Set the delay between the first 'tap' and the second 'tap'
- void setDamping(float damping); - Set the amount of high frequency damping applied to each echo
- void setFeedback(float feedback); - Set the amount of feedback the output signal fed back into the input
- void setSpread(float spread); - Set how hard panned the individual echoes are
- float getDelay() const;
- float getLRDelay() const;
- float getDamping() const;
- float getFeedback() const;
- float getSpread() const;
sf::ReverbEffect
- void setDensity(float density); - Set the coloration of the late reverb
- void setDiffusion(float diffusion); - Set the echo density in the reverberation decay
- void setGain(float gain); - Sets the max amount of reflections and reverberation added to the final sound mix
- void setDecayTime(float decay); - Set the reverberation decay time
- void setReflectionGain(float gain); - Set the overall amount of initial reflections relative to the Gain property
- void setReflectionDelay(float delay); - Set the amount of delay between the arrival time of the direct path from the source to the first reflection from the source
- void setLateReverbGain(float delay); - Set the overall amount of later reverberation relative to the Gain property
- void setLateReverbDelay(float delay); - Set the begin time of the late reverberation relative to the time of the initial reflection
- void setRoomRolloff(float rolloff); - Set the attenuation of the reflected sound
- float getDensity() const;
- float getDiffusion() const;
- float getGain() const;
- float getDecayTime() const;
- float getReflectionGain() const;
- float getReflectionDelay() const;
- float getLateReverbGain() const;
- float getLateReverbDelay() const;
- float getRoomRolloff() const;

How to test this PR?

I've updated the 'sound' example in the included examples directory to demonstrate the use of effects. The syntax is similar to that of sf::SoundBuffer to sf::Sound:

sf::ReverbEffect effect;
effect.setDecayTime(2.f);

sf::Music music;
music.loadFromFile("music.wav");
music.setEffect(&effect);
music.play();

Bromeon · 2020-10-05T18:47:55Z

Thanks a lot for your contribution! 🙂

A few remarks regarding the API:

If a sf::SoundEffect derived instance outlives the sf::SoundSource that uses it, it automatically deregisters it through the pointer list. This is nice, but it can be hard to get right, when values change their address.
What would e.g. the following do?
```
ChorusEffect effect, otherEffect;
// setup effects

sound.setEffect(effect);
effect = otherEffect;
```
Are there custom effects that users can implement by deriving sf::SoundEffect?
If setEffect() with a null pointer means to clear the effect, this should be documented.
If resetEffect() is for internal use, it should not be public, but private (if needed with friend).
The necessity of const_cast is often a design error -- you already use mutable for "not visible const-ness", so it should be possible to invoke const methods on the sf::SoundEffect through a const pointer.

There are a few smaller things in the code, but I think they can be discussed at a later stage.

fallahn · 2020-10-05T19:21:26Z

The pointer registration mechanism is copied from sf::SoundBuffer - unless I've missed something then it should be at least as reliable as losing a sf::SoundBuffer while a sf::Sound is still using it. This is the same for resetEffect() - it mimics sf::Sound::resetBuffer(). I considered it should be hidden from the public API when writing it but decided to stick to the existing design as closely as possible. I'd suggest that it probably should be changed, and the sf::SoundBuffer / sf::Sound relationship be updated too.

No, users can't implement their own effects with sf::SoundEffect. I was hoping this would be the case at the outset as it would mean I could create my own effects without having to modify SFML at all. Perhaps I could find some middle ground here and update the API to allow users to create their own custom effect.

I don't recall where I used the const_cast - it may have been a mid-development hack I've forgotten to remove. I'll take another look at that.

eliasdaler

Very nice PR, great job. A lot of work was clearly done here.
I have a few questions/remarks, though, see them in the review.

eliasdaler · 2021-02-27T23:30:42Z

examples/sound/Sound.cpp


-    // Loop while the sound is playing
-    while (sound.getStatus() == sf::Sound::Playing)
+    for (int i = 0; i < 2; ++i)


Maybe change "2" to some constant like "NUM_TIMES_SOUND_PLAYED"?
But I'm also not sure why we need to play it 2 times...

The first loop the audio is played without the effect applied, then the second loop the audio is played with the effect, to demonstrate the difference in output.

I think it would be good to have a visual indicator for this. E.g. show "No effect" for the first play and "Chorus Effect" for the second. Ideally the code would be something like this:

playWithoutEffect(...); playWithEffect(...);

eliasdaler · 2021-02-27T23:31:24Z

examples/sound/Sound.cpp

-
-    // Loop while the music is playing
-    while (music.getStatus() == sf::Music::Playing)
+    for (int i = 0; i < 2; ++i)


Same applies here, consider replacing "2" by a constant or removing this.

include/SFML/Audio.hpp

eliasdaler · 2021-02-27T23:34:34Z

include/SFML/Audio/ChorusEffect.hpp

+    /// magnitudes the sample will repeat and fade out over time. Use this parameter to
+    /// create a "cascading" chorus effect.
+    /// 
+    /// \param amount new feedback amount to set, between -1.f and 1.f. Defaults to 0.25f


This review contains a lot of magic variables like "0.25f" or "0.404f". Can you please explain where they come from? I didn't find why they're significant.
(It's especially not clear why parameter ranges are the way they are: [some_magic_number_1;some_magic_number_2])

These are the default values as laid out in the OpenAL specification. I guess the comments should be at least updated to state that fact.

Yep, reference to OpenAL spec would be enough

include/SFML/Audio/SoundEffect.hpp

eliasdaler · 2021-02-27T23:47:44Z

src/SFML/Audio/SoundSource.cpp

    return Stopped;
 }

+void SoundSource::setEffect(const SoundEffect* effect)


It's still not clear to me why effect can't be a const ref here, same as sf::SoundBuffer is passed by const ref to sf::Sound::setBuffer. To remove effect, we can add removeEffect function, which will be an explicit way of saying "I don't want this effect anymore".

This has been addresed in 1b41eb0 🙂

src/SFML/Audio/SoundEffect.cpp

eXpl0it3r · 2021-04-28T09:14:45Z

Added the public API changes at the top to get a faster overview.
Hid comments that have already been addressed to keep a bit of an overview.
Rebased onto master

If resetEffect() is for internal use, it should not be public, but private (if needed with friend).

The pointer registration mechanism is copied from sf::SoundBuffer - unless I've missed something then it should be at least as reliable as losing a sf::SoundBuffer while a sf::Sound is still using it. This is the same for resetEffect() - it mimics sf::Sound::resetBuffer(). I considered it should be hidden from the public API when writing it but decided to stick to the existing design as closely as possible. I'd suggest that it probably should be changed, and the sf::SoundBuffer / sf::Sound relationship be updated too.

Maybe @LaurentGomila has some insight on why he chose the public design for SoundBuffer over private + friend.

eXpl0it3r · 2021-04-28T09:14:59Z

It's still not clear to me why effect can't be a const ref here, same as sf::SoundBuffer is passed by const ref to sf::Sound::setBuffer. To remove effect, we can add removeEffect function, which will be an explicit way of saying "I don't want this effect anymore".

I think this requires an additional API discussion.
A lot of SFML functions are mostly set-and-never remove, which can be quite annoying at times (e.g. when you want to explicitly close/detach a stream).
With pointers you allow to reset this via the same API that you set it, but now you're dealing with raw pointers and an implicit state the NULL = no/remove effect.
Alternatives include:

Use references, but have a separate function "remove Effect"
NullEffect which can be set by default

Anyone else's opinion on this design?

eXpl0it3r · 2021-04-28T09:16:44Z

setVolume currently deals in the range of 0.f to 1.f, which is different to the sf::SoundSource::setVolume with a range of 0.f to 100.f. Both work since it's just basic math, question is whether we want to have this aligned?

eXpl0it3r · 2021-04-28T09:19:49Z

The ranges are currently silently clamped to the min or max allowed value. That way even if you pass in like -10.f the property will be set to for example 0.f as minimum value.

Question is, should we silently clamp the values or should we assert them instead or assert and then still clamp?

fallahn · 2021-04-28T09:45:33Z

setVolume currently deals in the range of 0.f to 1.f, which is different to the sf::SoundSource::setVolume with a range of 0.f to 100.f. Both work since it's just basic math, question is whether we want to have this aligned?

This is an oversight on my part - I'd expect it to be 0 - 100 the same as the rest of SFML. If I'm honest this has always confused me slightly as OpenAL uses 0 - 1, so I'm not sure why SFML would differ, particularly when it has to be converted, but I'm happy to roll with it.

As for asserting vs clamping - I'd personally prefer assertions because it's more obvious when something is wrong rather than silently hiding an error, although I'd expect OpenAL to print some kind of range error in these cases too. Ultimately though clamping the range has proven to be adequate enough in the existing functions with no adverse side effects that I'm aware of so I'd probably stick with that for consistency.

fallahn · 2021-04-28T18:39:41Z

On further examination setVolume() actually multiplies the output of the associated SoundSource, meaning that 0 - 1 is a more reasonable range than 0 - 100, as well as it not setting the volume to any specific value. Rather it's dependent on the output/current volume of the SoundSource. To that end I've renamed setVolume() to setVolumeMultiplier().

mini code-tidy

update documentation of SoundSource to reflect setting an effect as nullptr removes existing effects add copy/swap assignment operator to SoundEffect class to ensure in-use SoundSources are tidied up

Clarify source of constant parameter values in Effect headers Add missing parameter name in SoundEffect.hpp

… to better describe its effect

OliverGlandberger · 2022-02-06T15:23:44Z

Hi! I just wanted to say that this project looks very promising. Nicely done! :D

If anyone involved has a minute to spare, I am curious about the following things:
• What's the current status of this project?
• Is this code free/public and useable in any type of project (assuming proper credits are provided)?
• I noticed the description mentions high-/lowpass filters, any chance this will actually happen?

I've been working on a game engine on and off for the past 4 years which has SFML as its base. Would be nice if I could incorporate these features when I get around to further developing my SoundManager. (:

fallahn · 2022-02-06T15:38:40Z

Hi! To address your questions:

I have no idea 😄
Of course, it's all under the SFML license - to test it out just clone my branch
This will happen if the existing PR gets accepted. Biggest hold up I believe is the lack of support on iOS.

…const pointer rename SoundSource::resetEffect() to SoundSource::removeEffect() to better express its intended use

fallahn · 2022-02-10T09:34:52Z

It's still not clear to me why effect can't be a const ref here, same as sf::SoundBuffer is passed by const ref to sf::Sound::setBuffer. To remove effect, we can add removeEffect function, which will be an explicit way of saying "I don't want this effect anymore".

I think this requires an additional API discussion. A lot of SFML functions are mostly set-and-never remove, which can be quite annoying at times (e.g. when you want to explicitly close/detach a stream). With pointers you allow to reset this via the same API that you set it, but now you're dealing with raw pointers and an implicit state the NULL = no/remove effect. Alternatives include:
* Use references, but have a separate function "remove Effect"

I've updated SoundSource::setEffect() to take a const reference (for consistency with the existing API) and renamed resetEffect() to removeEffect() since this is what it actually did anyway. It also (albeit unintentionally) addresses the question as to whether or not resetEffect() should be private 😊

… operator

ChrisThrasher · 2024-04-30T03:41:49Z

Closed because #2749 gets rid of OpenAL entirely

eXpl0it3r · 2024-04-30T05:45:11Z

Would've really loved to see this in an earlier version of SFML, but with us moving to miniaudio, it doesn't quite fit in.

Thanks @fallahn for all the work, hopefully it can still be useful to someone. 🙂

eXpl0it3r added feature m:sfml-audio s:undecided labels Oct 5, 2020

This comment has been minimized.

Sign in to view

eliasdaler suggested changes Feb 27, 2021

View reviewed changes

This comment has been minimized.

Sign in to view

eXpl0it3r reviewed Mar 14, 2021

View reviewed changes

src/SFML/Audio/SoundEffect.cpp Outdated Show resolved Hide resolved

eXpl0it3r force-pushed the audio_effects branch from 137bbaa to d9ee97e Compare April 28, 2021 07:59

fallahn and others added 9 commits December 24, 2021 12:45

proof of concept reverb effect using OpenAL effects extensions

8e1eb1b

make openal effects objects shared resources

56e67a7

mini code-tidy

update and test to work on linux

f3e2a92

make compile good on mac

d263755

remove some C++11 specific features

40c7ae6

supply documentation for sf::SoundEffect and sf::ReverbEffect

acd478f

add ChorusEffect

96b2406

add DelayEffect

231211b

add assignment operator to ReverbEffect, ChorusEffect, DelayEffect

df7a65e

fallahn added 10 commits December 24, 2021 12:45

update sound playback example with SoundEffects

b2cbf70

remove C++11 specific syntax

a7ff7de

add missing include for std::min and std::max for visual studio 15

919b084

remove unnecessary const_cast

5137f3d

update documentation of SoundSource to reflect setting an effect as nullptr removes existing effects add copy/swap assignment operator to SoundEffect class to ensure in-use SoundSources are tidied up

better encapsulate AL effect handle in sf::SoundEffect base class.

0235f56

Sort includes in Audio.hpp alphabetically

5f37504

Clarify source of constant parameter values in Effect headers Add missing parameter name in SoundEffect.hpp

add empty implementation for sound effects on IOS

8a96418

update order of includes

564db9c

rename SoundEffect::setVolume() to SoundEffect::setVolumeMultiplier()…

ee0b258

… to better describe its effect

fix typo in iOS stub

934b9d8

eXpl0it3r force-pushed the audio_effects branch from 530f96d to 934b9d8 Compare December 24, 2021 11:50

fallahn added 8 commits February 7, 2022 18:14

cast away unsignedness of effect slot handle to match function parameter

dcba706

cast away unsignedness of effect value to match function parameters

9ff83d5

cast away unsignedness of effect value to match function parameters

439597d

cast away unsignedness of effect ID to match function parameters

ebfbfb0

change reverb parameter names from unsigned to signed

f8c3a56

remove unused parameter names in iOS stubs

c69e849

change SoundSource::setEffect() to take a const reference instead of …

1b41eb0

…const pointer rename SoundSource::resetEffect() to SoundSource::removeEffect() to better express its intended use

update sound example with modified sound effect api

abec956

fix updating any attached sound effects in the SoundSource assignment…

56a85a6

… operator

eXpl0it3r added this to the 3.1 milestone Jun 23, 2022

ChrisThrasher removed the s:undecided label Mar 20, 2023

ChrisThrasher closed this Apr 30, 2024

eXpl0it3r removed this from the 3.1 milestone Apr 30, 2024

binary1248 mentioned this pull request May 4, 2024

Added support for user defined sound effect implementations. #2973

Merged

Uh oh!

OpenAL Audio effects: reverb, chorus, delay #1708

OpenAL Audio effects: reverb, chorus, delay #1708

Uh oh!

Conversation

fallahn commented Oct 5, 2020 • edited by eXpl0it3r Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Tasks

API Overview

How to test this PR?

Uh oh!

Bromeon commented Oct 5, 2020

Uh oh!

fallahn commented Oct 5, 2020

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

eliasdaler left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Uh oh!

eXpl0it3r commented Apr 28, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eXpl0it3r commented Apr 28, 2021

Uh oh!

eXpl0it3r commented Apr 28, 2021

Uh oh!

eXpl0it3r commented Apr 28, 2021

Uh oh!

fallahn commented Apr 28, 2021

Uh oh!

fallahn commented Apr 28, 2021

Uh oh!

OliverGlandberger commented Feb 6, 2022

Uh oh!

fallahn commented Feb 6, 2022

Uh oh!

fallahn commented Feb 10, 2022

Uh oh!

ChrisThrasher commented Apr 30, 2024

Uh oh!

eXpl0it3r commented Apr 30, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

fallahn commented Oct 5, 2020 •

edited by eXpl0it3r

Loading

eXpl0it3r commented Apr 28, 2021 •

edited

Loading