This project has moved. For the latest updates, please go here.

Sample Aggregation For WASAPI Loopback

Feb 18, 2013 at 8:54 PM
Edited Feb 18, 2013 at 8:56 PM
I'm currently building a visual application that will capture internal audio using NAudio and the WASAPI Loopback device.

Referring to the code of your other .Net VoiceRecorder sample application, which allows you to display a real-time peak indicator through the Sample Aggregator mechanism. Now this code has been developed around one set WaveFormat (1 channel 16bit PCM @ 44.1KHz ) and seems to do the job nicely.

I'm trying to adapt this approach to be used with uncertain format of WASAPI Loopback to display a peak value in my application, in my system preferences for the playback device (Win 7) my system is set to 2 channel 16bit @ 44.1KHz, but I don't seem to be having much luck getting this working correctly (the silent output wont sit at 0 for example).

In this situation is the WaveFormat returned by WasapiLoopbackCapture actually:

AverageBytesPerSecond 352800
BitsPerSample 32
BlockAlign 8
Channels 2
Extensible
ExtraSize 22
SampleRate 44100

or is this just a generic format output by default?

Do you have any advice/insights that would help in a situation like this?

P.S. Anything I record using WASAPI Loopback to WAV cannot be played by any of the NAudio demos, an acmFormatSuggest exception is thrown.

Thanks,

Ollie
Feb 18, 2013 at 11:24 PM
After a bit of extra perseverance I have figured it out,

And it's as simple as:
 sample32 = BitConverter.ToSingle(buffer, index);
WASAPI Loopback's buffer byte array at DataAvailable contains 32 bit single-precision floats (not 16 bit shorts), so to get a sample value between 1f and -1f you need to use the following code before SampleAggregator:
       void waveIn_DataAvailable(object sender, WaveInEventArgs e)
        {
            byte[] buffer = e.Buffer;
            int bytesRecorded = e.BytesRecorded;
            WriteToFile(buffer, bytesRecorded);

            int bufferIncrement = (int)(this.waveIn.WaveFormat.BlockAlign / this.waveIn.WaveFormat.Channels);
            int bitsPerSample = this.waveIn.WaveFormat.BitsPerSample;

            for (int index = 0; index < e.BytesRecorded; index += bufferIncrement)
            {
                float sample32 = 0;

                if (bitsPerSample <= 16) // Presume 16-bit PCM WAV
                {
                    short sample16 = (short)((buffer[index + 1] << 8) | buffer[index + 0]);
                    sample32 = sample16 / 32768f;
                }
                else if (bitsPerSample <= 32) // Presume 32-bit IEEE Float WAV
                {
                    sample32 = BitConverter.ToSingle(buffer, index);
                }
                else
                {
                    throw new Exception(bitsPerSample + " Bits Per Sample Is Not Supported!");
                }

                // Clip Sample - Prevents Issues Elsewhere
                if (sample32 > 1.0f)
                    sample32 = 1.0f;
                if (sample32 < -1.0f)
                    sample32 = -1.0f;

                sampleAggregator.Add(sample32);
            }
        }
Hope this helps anyone else in future :)

Ollie
Coordinator
Feb 19, 2013 at 1:10 PM
yes, WASAPI normally works with 32 bit float. WAVEFORMATEXTENSIBLE is a bit of a pain to work with, but this seems to be Microsoft's preferred approach going forwards.
Feb 19, 2013 at 11:24 PM
Through my own testing of WASAPI capture, the whole process of writing to file on DataAvailable callback seems slightly unreliable, recording 2-3 minutes worth of audio through the loopback interface and there will be a couple of glitches in the wave file where moments of audio have been missed (<500ms).

I presume this is because the the DataAvailable callback hasn't been executed in time (or at all) for that 100ms worth of data.

I have noticed that WaveIn allows you to supply a window handle for NAudio to post a message back, which might be more reliable. Is there a reason why this hasn't been implemented into the WASAPI classes?
Feb 19, 2013 at 11:35 PM
Do you think if a derivative of WasapiCapture.cs was created where if a WaveFileWriter was specified in advance, the whole write to file process was done immediately at the end of the ReadNextPacket function instead of at the start of the DataAvailable callback, this would workaround the issue?

I'm presuming here though that NAudio is capturing those missing packets in the first place...

Cheers,

Ollie
Feb 19, 2013 at 11:42 PM
Edited Feb 19, 2013 at 11:44 PM
[Double Post]
Coordinator
Feb 20, 2013 at 8:33 AM
there is no windows callback becasue WASAPI does not have window callbacks like WaveOut/WaveIn does. NAudio is just a fairly thin wrapper around the WASAPI APIs. Perhaps audio is being missed because the DataAvailable event isn't being processed quickly enough. You could certainly experiment with your own copy of WasapiIn and see if writing and flushing after ReadNextPacket helps. Let us know how you get on.
Feb 24, 2013 at 12:41 AM
I have been looking into this and it appears that when missed audio occurs, ReadNextPacket is also delayed more than normal (when a standard 50ms 'sleepMilliseconds' value is used), so this isn't an issue with a delay in handling the DataAvailable event, instead it happens much earlier.

During this investigation I logged the Environment.TickCount at key points in the recording chain, as you can see from the below graphs, over a 3.5 minute recording of a song (as a common reference) at various points where the time between calls to ReadNextPacket exceeds 100ms. Beyond this level, noticeable losses can be seen in the written audio at these points, also the 'DataDiscontinuity' flag is raised by AudioCaptureClient.GetBuffer which suggests that the problem lies deeper within the WASAPI loopback capture process.

Image

Any idea what might be holding things up and what might resolve/improve the issue?

Cheers,

Ollie
Coordinator
Feb 25, 2013 at 10:44 AM
well, Sleep isn't an accurate timer in Windows. Thread.Sleep means sleep for at least 50ms. So if it is taking longer than that to return, it indicates the system is busy doing other stuff. You could reduce the length of the sleep. Alternatively, WASAPI does support an event driven mode. I offer this as an option with WASAPI Out. I can't remember if I made it available for WASAPI in, but that might be a good thing to try.
Feb 25, 2013 at 11:17 PM
Is it possible to use the event driven mode with loopback capture?

Working through how you have implemented it in WasapiOut the audioClient can only be initialised with either AudioClientStreamFlags.EventCallback or AudioClientStreamFlags.Loopback, not both?
Coordinator
Feb 26, 2013 at 10:04 AM
they are flags, so you would OR them together
Feb 26, 2013 at 11:18 PM
That's ok, just double checking!

I have tried to implement the event driven mode into WASAPI Capture (plus loopback), see my franken-code below. But for some reason WaitHandle.WaitAny under the DoRecording method never receives a response and always times-out (no matter how big the time-out is).

Any idea why AudioClient isn't responding with events?

https://dl.dropbox.com/u/14822/WASAPI/WasapiCaptureForFile.cs
https://dl.dropbox.com/u/14822/WASAPI/WasapiLoopbackCaptureForFile.cs

Kind regards,

Ollie
Coordinator
Feb 27, 2013 at 6:52 AM
I'd check the Initialize parameters are right (there is a 0 that might need a different value)
http://msdn.microsoft.com/en-gb/library/windows/desktop/dd370875%28v=vs.85%29.aspx
there are some code samples here that might be useful:
http://msdn.microsoft.com/en-gb/library/windows/desktop/dd370844%28v=vs.85%29.aspx

Mark
Feb 27, 2013 at 9:24 PM
Just had a read of that top article, it seems to be initialised correctly (both values are 0), I have found this article about loopback capture:

http://msdn.microsoft.com/en-gb/library/windows/desktop/dd316551(v=vs.85).aspx

Not exactly sure what the implication is to the current method used regarding this statement:
A pull-mode capture client does not receive any events when a stream is initialized with event-driven buffering and is loopback-enabled. To work around this, initialize a render stream in event-driven mode. Each time the client receives an event for the render stream, it must signal the capture client to run the capture thread that reads the next set of samples from the capture endpoint buffer.
Looking elsewhere on the web, others don't seem to think event-driven loopback capture is possible...

http://blogs.msdn.com/b/matthew_van_eerde/archive/2008/12/16/sample-wasapi-loopback-capture-record-what-you-hear.aspx

Ollie
Coordinator
Mar 2, 2013 at 7:02 AM
That's interesting - it seems to suggest that you should play and record at the same time, and use the event from playback to trigger both your reading and writing threads