Quick Links: Download Gideros Studio | Gideros Documentation | Gideros community chat | DONATE
Media/audio capture w/ Visual Studio — Gideros Forum

Media/audio capture w/ Visual Studio

This may not be Gideros related, but it probably is. (Sorry if it isn't)

Continuous troubles with the microphone plugin.
I suppose UWP support (win32) was added last Gideros update, since it now builds fine in Visual Studio (it didn't before, thanks for the update!). However, the app crashes when calling microphone:start()

VS debug output gives me
Exception thrown at 0x00007FFB3B59537C (KernelBase.dll) in giderosgame.Windows.exe: WinRT originate error - 0xC00D36B3 : 'The stream number provided was invalid.'.

followed by a few
Exception thrown at 0x00007FFB3B59537C (KernelBase.dll) in giderosgame.Windows.exe: WinRT originate error - 0xC00DABE0 : 'No capture devices are available.'.

and a last
Exception thrown at 0x00007FF69E2868A3 in giderosgame.Windows.exe: 0xC0000005: Access violation writing location 0x0000000000000000.

It seems the exceptions are related to media capture because others have had the same issues seemingly using mostly camera capture other platforms with UWP (Unity, OpenCV, Intel Realsense etc).

Using the app's microphone from exe files out of Windows Desktop (QT) or win32 exports works fine.

Comments

  • hgy29hgy29 Maintainer
    Well, I added microphone plugin for UWP last time, but I couldn't actually test it before release because my web camera died some time between my last video conference call and my attempt at writing a UWP driver for the microphone. I thought I couldn't try with a regular microphone plugged in the MIC input of my PC, but no it seems that UWP media capture framework don't count it as a microphone.
    I'll do some more tests on it today, since I got a new web camera!

    Likes: saeys

    +1 -1 (+1 / -0 )Share on Facebook
  • hgy29hgy29 Maintainer
    Accepted Answer
    Ok, found a few issues, they'll be fixed for next release. It seems to work now with my webcam. https://github.com/gideros/openal-soft/commit/03a2969020c42d06ab8e8640976ea122d0ade1b0

    Likes: pie, saeys

    +1 -1 (+2 / -0 )Share on Facebook
  • saeyssaeys Member
    Awsome, thank you! Looking forward to test it myself.
  • saeyssaeys Member
    I can now export UWP, build and play in VS since 2023.4 with no crashes. Yaay! B)

    However, when playing back the recorded sound the pitch is down by about half the frequency. It sounds like the original recording (at 44100 kHz sample rate) is interpreted as a 22050 kHz sample rate sound (then saved as a 44100 kHz). (Recording from the same audio source with an other recording application works fine)
    Also, the sound file get trimmed so ~85% of the recorded time are cut.

    And movieclips still seems to be played faster than the intended 60 fps.

    (This applies to UWP only it seems)
  • hgy29hgy29 Maintainer
    I still didn't find the issue with MovieClip (coudln't reproduce), but about recorded audio, did you ask for mono or stereo ? it seems that UWP can only record mono from mics, but if you ask stereo gideros will encode it as stereo. Actually we don't have much control at UWP side about what will be recorded, so you can't even ask for 22050 smp/s. Does it work if you stick to 44100/Mono ?
  • saeyssaeys Member
    microphone = Microphone.new(application:getDeviceName(), 44100, 1, 16)
    Yep, mono.

    This isn't an issue with win desktop or win32 exports, so maybe something happens on the VS side? Then I guess I need to dig deeper into that.

    I'll try to reproduce the MovieClip issue with a cleaner code. Later on... :-)
  • keszeghkeszegh Member
    perhaps this is a correct thread to discuss microphone plugin related things.
    @hgy29 , so i asked in another thread how to know
    in https://wiki.gideros.rocks/index.php/Microphone.new what are the possible deviceNames.
    looking at the source code it seems deviceName is not used at all, so perhaps this is not even implemented in the Microphone.new function.
    on the other hand if it would be, then probably alcGetString from https://www.openal.org/documentation/OpenAL_Programmers_Guide.pdf
    is the function that should be added to gideros to get the list of valid deviceNames.
    this would be great, as choosing the input device would be very important (just like it already works for MIDI devices).
    thanks
  • hgy29hgy29 Maintainer
    Yes, this is doable, I'll have a go at implementing it.

    Likes: MoKaLux, keszegh

    +1 -1 (+2 / -0 )Share on Facebook
  • hgy29hgy29 Maintainer
    FYI: https://github.com/gideros/gideros/commit/ca5e0e259d855af63e93c15702c7131f33649a1d

    Listing available mics:
    require "microphone"
    for _,d in ipairs(Microphone.getDeviceList()) do
    	print("MIC:",d)
    end
    Encoding an audio file from lua generated data:
      local microphone = Microphone.new(8000, 1, 16)
      microphone:setOutputFile("|D|test.wav")
      microphone:start()
      local b=buffer.create(8000*2)
      buffer.setarrayaccess(b,"i16")
      for i=0,8000-1 do
    	  b[i]=math.sin(i/3)*30000
      end
      microphone:writeAudio(b)
      microphone:stop()
    Encoding a theora video:
    local theora = require "theora.core"
    require "microphone"
     
    local texw,texh=120*16,68*16
    local tex=RenderTarget.new(texw,texh)
    local dot=Pixel.new(0xFF0000,10,10) dot:setAnchorPoint(.5,.5)
      local microphone = Microphone.new(8000, 1, 16)
      microphone:setOutputFile("|D|test.ogg")
     
    Core.asyncCall(function()
    	microphone:start()
    	theora.beginVideo(microphone:getStreamId(),"theora",30,texw,texh,3,0.6)
     
    	for i=1,360 do
    		tex:clear(0x303030)
    		tex:draw(dot,math.cos(^<i)*texh*.4+texw/2,math.sin(^<i)*texh*.4+texh/2)
    		theora.encodeVideoFrame(microphone:getStreamId(),(i-1),tex,(i==360))
    		print(`Encoded frame {i}/360`)
    	end
     
    	microphone:stop()
    	print("Finished")
    end)

    Likes: MoKaLux, keszegh

    +1 -1 (+2 / -0 )Share on Facebook
  • keszeghkeszegh Member
    sounds great, what is 8000 in Microphone.new(8000, 1, 16)?
    and how to "[plugin/microphone] Allow creating 'No Mic' recorders"?
    are these the same?
  • keszeghkeszegh Member
    also, can you send a compiled plugin so that i can start to implement these features in my app? ty
  • hgy29hgy29 Maintainer
    the 8000 is the sample rate of the audio data you would supply (if any). 'No Mic' are created just like in this case, without giving a device name (or even nil) as first parameter. The sound encoder still needs to know how the format of the input audio data, eventually passed inside writeAudio() call.
  • hgy29hgy29 Maintainer
    Also, theora encoding is pure software, so rather slow...
  • keszeghkeszegh Member
    hgy29 said:

    Also, theora encoding is pure software, so rather slow...

    i noticed that.
  • keszeghkeszegh Member
    previously https://wiki.gideros.rocks/index.php/Microphone.new
    deviceName was the first parameter. where is that now then?
    also previously nil set it to the default device. now it will set it to a 'no mic' silent device?
Sign In or Register to comment.