Media/audio capture w/ Visual Studio

saeys · March 2023

This may not be Gideros related, but it probably is. (Sorry if it isn't)

Continuous troubles with the microphone plugin.
I suppose UWP support (win32) was added last Gideros update, since it now builds fine in Visual Studio (it didn't before, thanks for the update!). However, the app crashes when calling microphone:start()

VS debug output gives me

Exception thrown at 0x00007FFB3B59537C (KernelBase.dll) in giderosgame.Windows.exe: WinRT originate error - 0xC00D36B3 : 'The stream number provided was invalid.'.

followed by a few

Exception thrown at 0x00007FFB3B59537C (KernelBase.dll) in giderosgame.Windows.exe: WinRT originate error - 0xC00DABE0 : 'No capture devices are available.'.

and a last

Exception thrown at 0x00007FF69E2868A3 in giderosgame.Windows.exe: 0xC0000005: Access violation writing location 0x0000000000000000.

It seems the exceptions are related to media capture because others have had the same issues seemingly using mostly camera capture other platforms with UWP (Unity, OpenCV, Intel Realsense etc).

Using the app's microphone from exe files out of Windows Desktop (QT) or win32 exports works fine.

hgy29 · March 2023

Well, I added microphone plugin for UWP last time, but I couldn't actually test it before release because my web camera died some time between my last video conference call and my attempt at writing a UWP driver for the microphone. I thought I couldn't try with a regular microphone plugged in the MIC input of my PC, but no it seems that UWP media capture framework don't count it as a microphone.
I'll do some more tests on it today, since I got a new web camera!

hgy29 · March 2023

Ok, found a few issues, they'll be fixed for next release. It seems to work now with my webcam. https://github.com/gideros/openal-soft/commit/03a2969020c42d06ab8e8640976ea122d0ade1b0

saeys · March 2023

Awsome, thank you! Looking forward to test it myself.

saeys · April 2023

I can now export UWP, build and play in VS since 2023.4 with no crashes. Yaay!

However, when playing back the recorded sound the pitch is down by about half the frequency. It sounds like the original recording (at 44100 kHz sample rate) is interpreted as a 22050 kHz sample rate sound (then saved as a 44100 kHz). (Recording from the same audio source with an other recording application works fine)
Also, the sound file get trimmed so ~85% of the recorded time are cut.

And movieclips still seems to be played faster than the intended 60 fps.

(This applies to UWP only it seems)

hgy29 · April 2023

I still didn't find the issue with MovieClip (coudln't reproduce), but about recorded audio, did you ask for mono or stereo ? it seems that UWP can only record mono from mics, but if you ask stereo gideros will encode it as stereo. Actually we don't have much control at UWP side about what will be recorded, so you can't even ask for 22050 smp/s. Does it work if you stick to 44100/Mono ?

saeys · April 2023

microphone = Microphone.new(application:getDeviceName(), 44100, 1, 16)

Yep, mono.

This isn't an issue with win desktop or win32 exports, so maybe something happens on the VS side? Then I guess I need to dig deeper into that.

I'll try to reproduce the MovieClip issue with a cleaner code. Later on... :-)

keszegh · July 3

perhaps this is a correct thread to discuss microphone plugin related things.
@hgy29 , so i asked in another thread how to know
in https://wiki.gideros.rocks/index.php/Microphone.new what are the possible deviceNames.
looking at the source code it seems deviceName is not used at all, so perhaps this is not even implemented in the Microphone.new function.
on the other hand if it would be, then probably alcGetString from https://www.openal.org/documentation/OpenAL_Programmers_Guide.pdf
is the function that should be added to gideros to get the list of valid deviceNames.
this would be great, as choosing the input device would be very important (just like it already works for MIDI devices).
thanks

hgy29 · July 3

Yes, this is doable, I'll have a go at implementing it.

hgy29 · July 3

FYI: https://github.com/gideros/gideros/commit/ca5e0e259d855af63e93c15702c7131f33649a1d

Listing available mics:

require "microphone"
for _,d in ipairs(Microphone.getDeviceList()) do
	print("MIC:",d)
end

Encoding an audio file from lua generated data:

  local microphone = Microphone.new(8000, 1, 16)
  microphone:setOutputFile("|D|test.wav")
  microphone:start()
  local b=buffer.create(8000*2)
  buffer.setarrayaccess(b,"i16")
  for i=0,8000-1 do
	  b[i]=math.sin(i/3)*30000
  end
  microphone:writeAudio(b)
  microphone:stop()

Encoding a theora video:

local theora = require "theora.core"
require "microphone"
 
local texw,texh=120*16,68*16
local tex=RenderTarget.new(texw,texh)
local dot=Pixel.new(0xFF0000,10,10) dot:setAnchorPoint(.5,.5)
  local microphone = Microphone.new(8000, 1, 16)
  microphone:setOutputFile("|D|test.ogg")
 
Core.asyncCall(function()
	microphone:start()
	theora.beginVideo(microphone:getStreamId(),"theora",30,texw,texh,3,0.6)
 
	for i=1,360 do
		tex:clear(0x303030)
		tex:draw(dot,math.cos(^<i)*texh*.4+texw/2,math.sin(^<i)*texh*.4+texh/2)
		theora.encodeVideoFrame(microphone:getStreamId(),(i-1),tex,(i==360))
		print(`Encoded frame {i}/360`)
	end
 
	microphone:stop()
	print("Finished")
end)

keszegh · July 3

sounds great, what is 8000 in Microphone.new(8000, 1, 16)?
and how to "[plugin/microphone] Allow creating 'No Mic' recorders"?
are these the same?

keszegh · July 3

also, can you send a compiled plugin so that i can start to implement these features in my app? ty

hgy29 · July 3

the 8000 is the sample rate of the audio data you would supply (if any). 'No Mic' are created just like in this case, without giving a device name (or even nil) as first parameter. The sound encoder still needs to know how the format of the input audio data, eventually passed inside writeAudio() call.

hgy29 · July 3

Also, theora encoding is pure software, so rather slow...

keszegh · July 3

hgy29 said:
Also, theora encoding is pure software, so rather slow...

i noticed that.

keszegh · July 3

previously https://wiki.gideros.rocks/index.php/Microphone.new
deviceName was the first parameter. where is that now then?
also previously nil set it to the default device. now it will set it to a 'no mic' silent device?

keszegh · July 4

@hgy29 it would be great to be able to test the updated mic plugin next week (i wanted to release a new version of my app latest next week for various reasons). also let me know how should i init a mic device with a specific devicename. thanks

hgy29 · July 5

Microphone device name is still thé flirst paramètre of thé constructior, what as changed is that you can now completely omit it to create a no mic device.

keszegh · July 5

thanks, these optional parameters are always strange to me, but it makes sense.

keszegh · July 9

i did not test the new theora options yet, but i did test the microphone device list function using the new plugin version and it works perfect. the documentation will need to be updated when the new version is published.

Howdy, Stranger!

Categories

In this Discussion

Top Posters

Media/audio capture w/ Visual Studio

Comments

Howdy, Stranger!

Quick Links

Categories

In this Discussion

Top Posters

Media/audio capture w/ Visual Studio

Comments