Quick Links: Download Gideros Studio | Gideros Documentation | Gideros community chat | DONATE
Media/audio capture w/ Visual Studio — Gideros Forum
  • Home
    • ›

    Media/audio capture w/ Visual Studio

    saeyssaeys Member
    in Forum talk
    This may not be Gideros related, but it probably is. (Sorry if it isn't)

    Continuous troubles with the microphone plugin.
    I suppose UWP support (win32) was added last Gideros update, since it now builds fine in Visual Studio (it didn't before, thanks for the update!). However, the app crashes when calling microphone:start()

    VS debug output gives me
    Exception thrown at 0x00007FFB3B59537C (KernelBase.dll) in giderosgame.Windows.exe: WinRT originate error - 0xC00D36B3 : 'The stream number provided was invalid.'.

    followed by a few
    Exception thrown at 0x00007FFB3B59537C (KernelBase.dll) in giderosgame.Windows.exe: WinRT originate error - 0xC00DABE0 : 'No capture devices are available.'.

    and a last
    Exception thrown at 0x00007FF69E2868A3 in giderosgame.Windows.exe: 0xC0000005: Access violation writing location 0x0000000000000000.

    It seems the exceptions are related to media capture because others have had the same issues seemingly using mostly camera capture other platforms with UWP (Unity, OpenCV, Intel Realsense etc).

    Using the app's microphone from exe files out of Windows Desktop (QT) or win32 exports works fine.
    +1 -1Share on Facebook

    Comments

    • hgy29hgy29 Maintainer
      Well, I added microphone plugin for UWP last time, but I couldn't actually test it before release because my web camera died some time between my last video conference call and my attempt at writing a UWP driver for the microphone. I thought I couldn't try with a regular microphone plugged in the MIC input of my PC, but no it seems that UWP media capture framework don't count it as a microphone.
      I'll do some more tests on it today, since I got a new web camera!

      Likes: saeys

      +1 -1 (+1 / -0 )Share on Facebook
    • hgy29hgy29 Maintainer
      Accepted Answer
      Ok, found a few issues, they'll be fixed for next release. It seems to work now with my webcam. https://github.com/gideros/openal-soft/commit/03a2969020c42d06ab8e8640976ea122d0ade1b0

      Likes: pie, saeys

      +1 -1 (+2 / -0 )Share on Facebook
    • saeyssaeys Member
      Awsome, thank you! Looking forward to test it myself.
      +1 -1Share on Facebook
    • saeyssaeys Member
      I can now export UWP, build and play in VS since 2023.4 with no crashes. Yaay! B)

      However, when playing back the recorded sound the pitch is down by about half the frequency. It sounds like the original recording (at 44100 kHz sample rate) is interpreted as a 22050 kHz sample rate sound (then saved as a 44100 kHz). (Recording from the same audio source with an other recording application works fine)
      Also, the sound file get trimmed so ~85% of the recorded time are cut.

      And movieclips still seems to be played faster than the intended 60 fps.

      (This applies to UWP only it seems)
      +1 -1Share on Facebook
    • hgy29hgy29 Maintainer
      I still didn't find the issue with MovieClip (coudln't reproduce), but about recorded audio, did you ask for mono or stereo ? it seems that UWP can only record mono from mics, but if you ask stereo gideros will encode it as stereo. Actually we don't have much control at UWP side about what will be recorded, so you can't even ask for 22050 smp/s. Does it work if you stick to 44100/Mono ?
      +1 -1Share on Facebook
    • saeyssaeys Member
      microphone = Microphone.new(application:getDeviceName(), 44100, 1, 16)
      Yep, mono.

      This isn't an issue with win desktop or win32 exports, so maybe something happens on the VS side? Then I guess I need to dig deeper into that.

      I'll try to reproduce the MovieClip issue with a cleaner code. Later on... :-)
      +1 -1Share on Facebook
    • keszeghkeszegh Member
      perhaps this is a correct thread to discuss microphone plugin related things.
      @hgy29 , so i asked in another thread how to know
      in https://wiki.gideros.rocks/index.php/Microphone.new what are the possible deviceNames.
      looking at the source code it seems deviceName is not used at all, so perhaps this is not even implemented in the Microphone.new function.
      on the other hand if it would be, then probably alcGetString from https://www.openal.org/documentation/OpenAL_Programmers_Guide.pdf
      is the function that should be added to gideros to get the list of valid deviceNames.
      this would be great, as choosing the input device would be very important (just like it already works for MIDI devices).
      thanks
      Longtitle Productions, creator of
      Fragmenter - animated loop machine and IKONOMIKON - the memory game
      +1 -1Share on Facebook
    • hgy29hgy29 Maintainer
      Yes, this is doable, I'll have a go at implementing it.

      Likes: MoKaLux, keszegh

      +1 -1 (+2 / -0 )Share on Facebook
    • hgy29hgy29 Maintainer
      FYI: https://github.com/gideros/gideros/commit/ca5e0e259d855af63e93c15702c7131f33649a1d

      Listing available mics:
      require "microphone"
for _,d in ipairs(Microphone.getDeviceList()) do
	print("MIC:",d)
end
      Encoding an audio file from lua generated data:
        local microphone = Microphone.new(8000, 1, 16)
  microphone:setOutputFile("|D|test.wav")
  microphone:start()
  local b=buffer.create(8000*2)
  buffer.setarrayaccess(b,"i16")
  for i=0,8000-1 do
	  b[i]=math.sin(i/3)*30000
  end
  microphone:writeAudio(b)
  microphone:stop()
      Encoding a theora video:
      local theora = require "theora.core"
require "microphone"
 
local texw,texh=120*16,68*16
local tex=RenderTarget.new(texw,texh)
local dot=Pixel.new(0xFF0000,10,10) dot:setAnchorPoint(.5,.5)
  local microphone = Microphone.new(8000, 1, 16)
  microphone:setOutputFile("|D|test.ogg")
 
Core.asyncCall(function()
	microphone:start()
	theora.beginVideo(microphone:getStreamId(),"theora",30,texw,texh,3,0.6)
 
	for i=1,360 do
		tex:clear(0x303030)
		tex:draw(dot,math.cos(^<i)*texh*.4+texw/2,math.sin(^<i)*texh*.4+texh/2)
		theora.encodeVideoFrame(microphone:getStreamId(),(i-1),tex,(i==360))
		print(`Encoded frame {i}/360`)
	end
 
	microphone:stop()
	print("Finished")
end)

      Likes: MoKaLux, keszegh

      +1 -1 (+2 / -0 )Share on Facebook
    • keszeghkeszegh Member
      sounds great, what is 8000 in Microphone.new(8000, 1, 16)?
      and how to "[plugin/microphone] Allow creating 'No Mic' recorders"?
      are these the same?
      Longtitle Productions, creator of
      Fragmenter - animated loop machine and IKONOMIKON - the memory game
      +1 -1Share on Facebook
    • keszeghkeszegh Member
      also, can you send a compiled plugin so that i can start to implement these features in my app? ty
      Longtitle Productions, creator of
      Fragmenter - animated loop machine and IKONOMIKON - the memory game
      +1 -1Share on Facebook
    • hgy29hgy29 Maintainer
      the 8000 is the sample rate of the audio data you would supply (if any). 'No Mic' are created just like in this case, without giving a device name (or even nil) as first parameter. The sound encoder still needs to know how the format of the input audio data, eventually passed inside writeAudio() call.
      +1 -1Share on Facebook
    • hgy29hgy29 Maintainer
      Also, theora encoding is pure software, so rather slow...
      +1 -1Share on Facebook
    • keszeghkeszegh Member
      hgy29 said:

      Also, theora encoding is pure software, so rather slow...

      i noticed that.
      Longtitle Productions, creator of
      Fragmenter - animated loop machine and IKONOMIKON - the memory game
      +1 -1Share on Facebook
    • keszeghkeszegh Member
      previously https://wiki.gideros.rocks/index.php/Microphone.new
      deviceName was the first parameter. where is that now then?
      also previously nil set it to the default device. now it will set it to a 'no mic' silent device?
      Longtitle Productions, creator of
      Fragmenter - animated loop machine and IKONOMIKON - the memory game
      +1 -1Share on Facebook
    • keszeghkeszegh Member
      @hgy29 it would be great to be able to test the updated mic plugin next week (i wanted to release a new version of my app latest next week for various reasons). also let me know how should i init a mic device with a specific devicename. thanks
      Longtitle Productions, creator of
      Fragmenter - animated loop machine and IKONOMIKON - the memory game
      +1 -1Share on Facebook
    Sign In or Register to comment.