Problem with opening decklink devices via ffmpeg

[signumnova]

Found a bit of an issue with this. The call to avformat_open_input in demuxer.cs is made on the UI thread if the input is a "device". (There is a comment there that this change was made owing to problems with gdigrab?)

Unfortunately the implementation of this for decklink inside ffmpeg has a spin loop in its open code, and if there is nothing connected to the actual decklink input at the time, this causes non-responsiveness on the UI thread for a full 3 seconds, and also means that nothing else can be opened or handled while this is going on. Really bad, and even worse when you try to open more than one input in this way at the same time!

So that's fine - we temporarily stubbed out the call to do it via UIInvoke. But that exposed another problem: the call to init the decklib library gets a com initialisation failure when that call occurs in a thread that is not an STA thread.

The solution we tested was to create an STA thread to make the call:

       /*
                     if (isDevice)
                         Utils.UIInvoke(() => OpenFormat(url, inFmt, fmtOptExtra, out ret));
                     else
         */
         Thread t = new Thread(
         () =>
         {
             OpenFormat(url, inFmt, fmtOptExtra, out ret);
         });
         t.SetApartmentState(ApartmentState.STA);
         t.Start();
         t.Join();

This seems to work now. The root of the problem appears to be that that particular call to decklink inside ffmpeg (the CoInitialize) is using an older single-threaded call - this should really be fixed.

Anyhow for now the hack above needs to be finessed to that it only does it in the case of decklink devices, i.e. where the URL is prefixed with "device://decklink?"

Not sure if this would also address the problem with gdigrab.

Cheers SN

[SuRGeoNix]

FFmpeg needs to resolve those issues, a lot of people having those issues, mpv for example has an open issue with this and it will not work at all. I tried to run gdigrab in STA thread instead of UI and didn't work. So at least gdigrab requires UI thread. Are you sure that decklink will properly work with this solution (might crash after a minute or so?).

I have not tested with other devices, I could try some cameras etc. Maybe I will use this solution for general devices except gdigrab that it will run in UI.

Update 1: Microphone with dshow works fine without using other threads at all - fmt://dshow?audio=Microphone (G433 Gaming Headset)

[signumnova]

Will let you know for sure that this solution works 'reliably' for decklink and ffmpeg. Can't speak for gdigrab though.

Ultimately yes the ffmpeg code should be updated. Am trying to get a build env working for this for Win but this will take a bit of effort and time I think.