API - Play Sound (loops = infinite?) - Consistency

Mells · December 2012

Hi,

I have a suggestion about the Sound:play(startTime, loops) method.
Can the second argument loops allow infinite loops by setting loops = 0?
Currently :

A value of 0 and 1 will play once

That would be consistent with the way Timer.new(delay, repeatCount) handles infinite loops :

A value of 0 runs the timer infinitely

I believe (need to check again) that this is also the case with GTween (nbLoops = 0 -> infinite loops).

Delay between each loop

Also, if we could set a delay between each loop without using timers or delayedCall, that would be very convenient.
Sound:play(startTime, loops, delay).
In that case delay is not the duration before the sound is played the first time, but the duration between each loop.
Default = 0.

Reference manual

The references in the manual can be found below :

Sound:play(startTime, loops)

Creates a new SoundChannel object to play the sound. By using the retured SoundChannel object, you can stop the sound and monitor the position.

Parameters:
startTime: (number, default = 0) The initial position in milliseconds at which playback should start.
loops: (number, default = 0) Defines the number of times a sound play to the begining before the sound channel stops playback. For example, a value of 0 and 1 will play once, 2 will play twice and so on.
Returns:
A SoundChannel object, which you use to control the sound. This function returns nil if you run out of available sound channels.

Timer.new(delay, repeatCount)

Creates a new Timer object with the specified delay and repeatCount states.

Parameters:
delay: The time interval between timer events in milliseconds.
repeatCount: (default = 0) The number of repetitions. A value of 0 runs the timer infinitely. If nonzero, the timer runs the specified number of times and then stops.

What do you think about it?

plamen · December 2012

Agreed. Vote for consistency and sound:play(0,math.huge()) is a bit ugly to me.

moopf · December 2012

Rather than just something small like this, the sound API needs a desperate overhaul. It's not really fit for purpose at all at present, unfortunately.

Mells · December 2012

@moopf
I don't know where we can check if what you suggest 1) is part of the roadmap and 2) which level of priority was given to it.
I know that you have been asking about it for quite a long time but it's only recently that I have played with the sound API so I don't see the limits yet like you do (probably soon?).

In the meantime, I believe that we can have this little fix (seems quite straightforward) that I suggest and it won't have a big influence on the sound API overhaul.

moopf · December 2012

I guess we come at this from different angles. I don't see the point of building additions on top of something which, at its core, is inadequate and needs a radical overhaul. That's just putting sticking plasters on it which leads to more pain in the future.

But what do I know *shrug*

As for the roadmap, it was listed as near future in September on this thread:

http://www.giderosmobile.com/forum/discussion/comment/12969

But as we now know that roadmap isn't really worth paying any attention to, as Atilim has already teased stuff which contradicts that. So, who knows?

Mells · December 2012

@moopf

That's just putting sticking plasters on it which leads to more pain in the future.

That could be. That's why I just make a suggestion and it's up to Atilim to see if it's worth spending time on it or not.
From my point of view, if it take less than a certain amount of time (minimum required so it's not a huge investment) and if it can make the experience a little better until the API is improved (between now and march) and if enough people find the request would improve the experience, then why not?

As I said, I don't know anything about the amount of time it would take.
If it's just changing a few "if" conditions then maybe it's worth it and will prevent users to make errors because of the lack of consistency.

But I agree (and Atilim would probably agree, as it is part of the roadmap) : the goal is to improve the API.

Mells · December 2012

@atilim
It seems that Sound:play(*startTime*, loops) is not taken into account when the file is in *.wav format (always start from 0).
It works fine with *.mp3 files.

ar2rsawseen · December 2012

Well coming up with the good architecture of classes and API's from the start is a very difficult work. But lucky for us, lua is a very expendable, and there should be no problem for one to write their own API and share with others, if others approve and start using the same API, I see no reason why it will not be implemented in Gideros in the future.

My point is let's try to create something that we think should be in that API and let's see where it goes, it's something I'm trying to achieve in GiderosCodingEasy.
So what do you think, is there a need for separate Sounds and SoundChannel classes?
What other functionalities are needed, like fadinOut, fadeIn, etc.

Of course not all can be implemented, as the bug that @ Mells found and panning can't be implemented from lua too, but at least that would be a start.

If @atilim will start working on new Sounds API, it would be much better to have a list of wishes, requirements, etc. Even better a prototype, which users already use

moopf · December 2012

@ar2rsawseen Whilst I applaud your intentions I'm afraid that here you're very hamstrung by the underlying sound implementation in Gideros and, whilst you can do some things, you can't do what really needs to be done in Lua alone. It exposes so little, and what is exposed seems a little shaky from my experience, that really it needs a rewrite.

I couldn't even get confirmation from Atilim if Gideros uses hardware decoding of an mp3 file where it's available. I've sene him post that playing background music uses 20% CPU on some devices (there's a thread somewhere where he mentions this) which makes me wonder if it does or not.

As for features: Pausing, panning, separation in the handling of background music and sound files, changing to OpenSL. There's a start. I'm happy if we have to handle fading in and out (presuming that issues I encountered whilst doing it are removed, e.g. buffering old track playing for a split second when starting a new track at full volume etc.) Heck, even pausing I can get around (but it's messy).

Out of everything that's provided in Gideros, sound seems to be one of the most neglected features.

atilim · December 2012

Hi,

I've reimplemented the Sound API from ground up for the upcoming release. Currently I'm doing tests on it and I'm about to release a new version.

Here are new additions:
1. Sound objects are reused internally. Therefore a code like:

    Sound.new("ding.wav"):play()
    Sound.new("ding.wav"):play()

won't load ding.wav twice and reuse the first loaded one.

2. Added setPitch/getPitch to SoundChannnel

3. Added pause/resume

4. On iOS, now it's possible to allow iPod music while playing the game. (e.g you can choose Ambient or SoloAmbient as AVAudioSession category)

5. Setting the loop count makes my implementation (really) complicated. I've dropped that implementation and now instead of loop count, you specify as looping or non-looping:

    Sound.new("ding.wav"):play(0)  -- play once
    Sound.new("ding.wav"):play(0, false)  -- play once
    Sound.new("ding.wav"):play(0, true)  -- looping

This modification breaks the backward comparability. Therefore looping parameter also accepts a number and it's <=1 then sound will be played once, otherwise it'll loop<pre escaped='true' lang="lua">
Sound.new("ding.wav"):play(0, 0) -- play once
Sound.new("ding.wav"):play(0, 1) -- play once
Sound.new("ding.wav"):play(0, 2) -- looping
Sound.new("ding.wav"):play(0, 3) -- looping
Sound.new("ding.wav"):play(0, math.huge()) -- looping

@moopf - Gideros uses hardware decoding of an mp3 file where it's available. (on iOS it uses AVAudioPlayer and on Android it uses android.media.MediaPlayer) The %20 CPU is on iPod 2nd gen (although with hardware decoding) and I'm sure it's much more lower with the newer devices.

To support OpenSL, I need to drop Android 2.2 support (%10.3).

moopf · December 2012

Hi @atilim, this sounds good. Still no panning though?

atilim · December 2012

Still I don't know how to implement panning in OpenAL. I read some implementations that mimic panning with 3d audio but that makes audio sources mono.

moopf · December 2012

Interesting @atilim, CocosDenshion (part of cocos2d-iphone) uses OpenAL and that's supported panning ever since I first used it a couple of years ago. Can't remember if it was for mono sounds only, but that would be fine anyway.

atilim · December 2012

I know ObjectAL implements panning with 3d audio. And that library has an option to reduce any stereo data to mono and it says "stereo samples don't support panning or positional audio". I'll also look CocosDenshion.

plamen · December 2012

@atilim:Keeping support of old OS/Hardware is a big pain, like heavy concrete bucket on the feet. You have to decide where Gideros Mobile will live. In the past or in the future. Me, personally, +1 vote for dropping support of Android 2.2 and i cant get it about those 2nd gen Ipods, my Ipod 4th gen/retina is already obsolete.

atilim · December 2012

@moopf - and about CocosDenshion: http://www.cocos2d-iphone.org/wiki/doku.php/cocosdenshion:faq#panning_doesn_t_seem_to_be_working (on the other hand, you can split your stereo source into two mono sources (left and right) and change the volumes of the sources to achieve panning).

@plamen, I was just about to drop Android 2.2 support because I've implemented all sound API from ground up. And after this release, I'll try to drop 2.2 and implement OpenSL as soon as reasonably possible.

moopf · December 2012

Hi @atilim - I must have used mono samples so I guess the question remains, can you support panning on mono samples in Gideros or are you saying I should use the fudge of two mono samples?

atilim · December 2012

@moopf I only can say after looking into XAudio2 http://en.wikipedia.org/wiki/XAudio2 (for windows phone 8) and OpenSL ES. Every platform has different sound API and I need to stay within the intersection of these. But most probably they all support 3d audio therefore panning can be implemented one way or another.

gmarinov · December 2012

@atilim panning makes sense for mono sounds. In stereo (2 channels) that would translate to "balance" (e.g. attenuation of each channel relative to the other) rather than panning each individual channel (I can't think of anything iOS-related that would require it).

Still, the ability to pan a sound is actually quite useful.

Outside of just sound effects I think it might be useful to look into synchronised loops (music layers), endless looping, or the ability to specify not only a start offset + looping, but start offset + loop length.

In the longer run, some basic audio effects such as EQ and echo may be put to good use for effects. As it is now, to simulate a distant sound requires loading a whole new sound file...

Much as I can see reason to support legacy hardware, I think it should be totally fine to drop some of that legacy in favour of extending the APIs and actually looking forward. By the time the products using those APIs are out, more often than not the hardware has moved another cycle forward.

my 2c.

atilim · December 2012

hi @gmarinov - thank you for your great insights.

Here the problem is the available API usually doesn't provide sound effects like reverb, EQ, echo, etc. Although iOS is really strong on sound/audio side, recently with iOS 5 it started supporting these kinds of effects natively. And most of the professional audio libraries (e.g. FMOD, BASS, irrKlang, Miles, Wwise, etc) usually provides licenses per single title/game. Maybe in the future, we can provide Lua bindings for FMOD and allow the users to purchase FMOD licenses by themselves.

For start offset + loop length, I'm planning to introduce a class SoundRegion (similar to TextureRegion) like:

    local sound = Sound.new("music.mp3")
    local soundRegion = SoundRegion.new(sound, 1000, 2000) --> 1000 is starting position and 2000 is length
    soundRegion:play() -- play sound region once
    soundRegion:play(0, true) -- play sound region endlessly

But there isn't any time estimation on this yet.

ar2rsawseen · December 2012

@atilim so now all the sfx will be put in one file, which size will be rounded to x^2 and each sounds will be retrieved using SoundsRegion

Kidding, still it's quite a nice feature, cause you may want to change a melody to more dramatic ones on actions and to more calm parts on easy moments of the game.

moopf · December 2012

@moopf I only can say after looking into XAudio2 http://en.wikipedia.org/wiki/XAudio2 (for windows phone 8) and OpenSL ES. Every platform has different sound API and I need to stay within the intersection of these. But most probably they all support 3d audio therefore panning can be implemented one way or another.

No problem, I completely understand that reasoning.

OZApps · December 2012

Maybe in the future, we can provide Lua bindings for FMOD and allow the users to purchase FMOD licenses by themselves.

@Atilim, while that sounds nice, you will end up with maintaining multiple code bases, one for FMOD, one for the generic sound, etc (unless there will be no basic sound only FMOD), the FMOD could be offered as a plug-in or a commercial plug-in.

However if there would be some way to create/play MIDI type sounds, create notes from octaves and values, that would be very useful. Something like the AY-3-8912 Chip (I guess it was also used in Amstrad machines) which can use ascii strings to play tunes or like Codea does with strings.

gmarinov · December 2012

@OZApps, @atilim, basically some additional hooks and events inside the audio API as it stands could maybe kick-start the process without too much code your end?

For instance: There's an Event.COMPLETE for a sound file that has finished. What is the equivalent for a loop that has just reached it's loop point? That would give us some control over layered or looped music (say add a layer to the already running music, or crossfade to another section, in both cases respecting the internal timing of the music)

With MP3 specifically there's also the issue of fine control over gapless looping soundchannels - http://www.compuphase.com/mp3/mp3loops.htm (scroll to Part 2). From what I can hear Gideros too has the gap at the start/end of a looped mp3 (expected).

Is AAC/M4A out of bounds simply because of Android? (specs on seamless looping AAC natively here http://developer.apple.com/library/mac/#qa/qa1636/_index.html )

As far as I could see, the sound functions operate in milliseconds? That's fairly imprecise compared to samples. Is there a reason for it or is it just with the intention of being easy to understand?

In my version (2012.09), it seems looping an MP3 only respects the start point the first time the sound is played. Also the "complete" event only fires once the number of loops is over (what I spoke about above).

So to recap, better control over playback (start, end, number of loops, possibly seamless looping), an event at the end of a loop.

atilim · December 2012

After reading http://www.compuphase.com/mp3/mp3loops.htm I finally understand what gapless looping means

On Desktop player, I'm using mpg123 and I've built mpg123 library with gapless mode enabled. But I'm not sure if it works as intended or not (it seems it doesn't). And although AAC/M4A is out of options, it's possible to add Ogg Vorbis support (which I understand allows gapless looping)

> the sound functions operate in milliseconds?
Yes. But I can enable floating point miliseconds instead of integer so that you can obtain sample level precision. btw, is sample level precision really needed in real life? isn't miliseconds precise enough?

To have better control on playback, one other option can be adding sync points to the sound channel. And when the playback encounters a sync point, an event is dispatched. But in Gideros nearly all events are queued and dispatched later. Therefore a sync point event may have a delay like 1000/60 ~= 16 miliseconds. I don't know if this limitation makes sync points useless or not.

atilim · December 2012

@OZApps yes you're right. FMOD shouldn't be implemented as an internal feature and should be offered as a (optional) plugin.

gmarinov · December 2012

@atilim, yes sample level precision is a must to avoid clicks and pops when looping, and also when cueing single shot sounds from a sound bank (think footsteps variation). At high sample rate there are 44 samples in each millisecond..

I think sync points might actually be of great benefit, even if events arrive a bit later. I'd move the sync point earlier then see the results that would produce. Again it would be down to the soundchannel/soundregion's ability to handle moving loop points, especially without interrupting the sound if that's still playing..

Ogg is a great option as well, as far as I know a lot of game devs use it. It also sounds better than mp3 at lower frame rates. Its transients are better too. Only thing I'm unsure about is the capability of the device to decode it in hardware... ?

atilim · December 2012

After your post, I've done some research and found that ogg vorbis is not supported by iOS. Therefore now it's out of option. I'm glad you've warned me

Maybe AAC/M4A can be supported with the help of Android's open source AAC library; http://sourceforge.net/mailarchive/message.php?msg_id=29526038

gmarinov · December 2012

I think with some effort on the devs' side we could possibly fool the mp3 decoder into looping at the correct length but at least we need controllable start and loop points. Also that event we discussed.. Maybe an event at the start of a loop might be just as useful.

Practically any move in a similar direction would be of great benefit at this stage...

Howdy, Stranger!

Categories

In this Discussion

Top Posters

API - Play Sound (loops = infinite?) - Consistency

Delay between each loop

Reference manual

Comments

Howdy, Stranger!

Quick Links

Categories

In this Discussion

Top Posters

API - Play Sound (loops = infinite?) - Consistency

Delay between each loop

Reference manual

Comments