This project has moved and is read-only. For the latest updates, please go here.

Mixing audio from directshow

Jan 11, 2010 at 2:53 AM

I'm really struggling to wrap my head around NAudio.  I have a series of filter graphs with a SampleBuffer with a callback that receives audio buffers as byte[] arrays filled with PCM encoded samples.  I want to mix the audio from all the currently running graphs together using WaveMixerStream32 and then output the audio via ASIO.   I understand what WaveMixerStream32 requires a WaveOffsetStream stream so data is always present. 

Is there anything built into NAudio that just lets me continuously pass the latest byte[] sample buffers from directshow into some type of WaveStream that I can wrap in a WaveOffsetStream?

- Michael Tanczos

Jan 11, 2010 at 10:01 AM

Hi Michael,

I'm afraid there is nothing built-in yet, although this is a feature I really should get round to adding soon. What is needed is to store the received data temporarily (in some kind of buffer), and return whatever is available in the read method.

Mark

Jan 11, 2010 at 5:06 PM
Edited Jan 12, 2010 at 2:35 AM

So my line of thought is this then and you can tell me if this is correct.   I'll write something up and post it and we can tear it apart then.

Create a class that inherits WaveStream:

  • WaveForm should match the bitrate of the stream, # channels, etc.
  • CanRead should be true, CanSeek false, CanWrite false (?)
  • Flush (no idea what this is for)
  • Create an AddSample(byte[] buffer, int count) method
    • Create a small queue of byte buffers that can be added to for storing incoming sample data.  The queue would include objects that contain a byte[] buffer and a readposition integer
  • Length - Return long.MaxValue
  • Position - Return current read position (make sure it is evenly divisible by BlockAlign)
  • The Read(byte[] buffer, int offset, int count) method should then:
    • Peek at first item in queue and read count bytes from it.  Before the buffer can be removed from the queue you would have to ensure that the whole thing was read
    • If insufficient sample data exists, generate silence
    • Update Position by number of bytes read
  • Override hasData to always return true
  • Anything else?

- Michael Tanczos

Jan 12, 2010 at 3:00 AM
Edited Jan 12, 2010 at 3:57 AM

I wrote up what I stated as follows, but the audio is playing a bit slower than it should and is falling further and further behind the video each second.  The BufferedSampleStream was created as a 48Khz stream to match the input stream.  ugh..  The mixer should technically just about play the queued samples as they are queued I would think, but over time I'm getting a larger and larger buffer queue of samples.  EDIT: Okay, I see that the mixer has a hard coded rate in the constructor of 44.1Khz

I'm setting up the mixer as follows:

        _audioStream = new BufferedSampleStream();
        channelSteam[0] = new WaveChannel32(_audioStream);
        channelSteam[0].Position = 0;

        mixer.AddInputStream(channelSteam[0]);

 

 

 

// BufferedSampleStream code (this does in fact work)

 

    public class SampleBlock
    {
        private byte[] _buffer;
        private int _position;

        public SampleBlock(byte[] buffer)
        {
            _position = 0;
            _buffer = buffer;
        }

        public byte[] Buffer
        {
            get
            {
                return _buffer;
            }
        }

        public int Position
        {
            get
            {
                return _position;
            }
            set
            {
                _position = value;
            }
        }
    }

    public class BufferedSampleStream : WaveStream
    {
        private WaveFormat _waveformat;
        private long _position;
        private Queue _sampQueue;

        public BufferedSampleStream()
        {
            _waveformat = new WaveFormat(44100, 2);
            _position = 0;
            _sampQueue = new Queue();
        }

        public BufferedSampleStream(int rate, int channels)
        {
            _waveformat = new WaveFormat(rate, channels);
            _position = 0;
            _sampQueue = new Queue();
        }

        /// <summary>
        /// Adds a byte[] array of PCM encoded samples to be streamed
        /// </summary>
        public void AddSamples(byte[] buffer, int offset, int count)
        {
            byte[] nbuffer = new byte[count];
            Buffer.BlockCopy(buffer, offset, nbuffer, 0, count);

            lock (_sampQueue)
            {
                _sampQueue.Enqueue(new SampleBlock(nbuffer));
            }
        }

        /// <summary>
        /// We can read from this stream
        /// </summary>
        public override bool CanRead { get { return true; } }

        /// <summary>
        /// We can seek within this stream
        /// </summary>
        public override bool CanSeek { get { return false; } }

        /// <summary>
        /// We can't write to this stream
        /// </summary>
        public override bool CanWrite { get { return false; } }
        
        public override WaveFormat WaveFormat
        {
            get
            {
                return _waveformat;
            }
        }

        /// <summary>
        /// An alternative way of repositioning.
        /// See <see cref="Stream.Seek"/>
        /// </summary>
        public override long Seek(long offset, SeekOrigin origin)
        {
            return Position;
        }

        public override long Length
        {
            get { return long.MaxValue / 32; }
        }

        public override long Position
        {
            get
            {
                return _position;
            }
            set
            {
                _position = value;
            }
        }

        public override bool HasData(int count)
        {
            // This buffered sample stream will always return some type of audio data
            return true;
        }

        /// <summary>
        /// Reads sample data from queued samples, if queue is empty generates silence
        /// </summary>
        public override int Read(byte[] buffer, int offset, int count)
        {
            int read = 0;
            while (read < count)
            {
                int required = count - read;
                if (_sampQueue.Count == 0)
                {
                    // Return a zero filled buffer
                    for (int n = 0; n < required; n++)
                        buffer[offset + n] = 0;

                    read += required;
                }
                else
                {
                    SampleBlock sblock = (SampleBlock)_sampQueue.Peek();

                    int nread = sblock.Buffer.Length - sblock.Position;

                    // If this buffer must be read in it's entirety
                    if (nread <= required)
                    {
                        // Read entire buffer
                        Buffer.BlockCopy(sblock.Buffer, sblock.Position, buffer, offset + read, nread);
                        read += nread;

                        _sampQueue.Dequeue();

                    }
                    else // the number of bytes that can be read is greater than that required
                    {
                        Buffer.BlockCopy(sblock.Buffer, sblock.Position, buffer, offset + read, required);
                        sblock.Position += required;
                        read += required;
                    }
                }

            }

            _position += read;

            return read;
        }
    }

 

 

Jan 12, 2010 at 4:18 AM
Edited Jan 13, 2010 at 5:14 AM

Something like this would present a secondary problem.. is there a way to dynamically resample audio from 48Khz to 44.1Khz (something makes me think this part isn't going to be easy).  I can't be assured that all input streams will be the same bitrate. 

EDIT 1/13: 

To assure that all streams are a uniform bitrate coming out of directshow i'm going to use the ACM Wrapper from Microsoft.. downsampling isn't so great but upsampling everything to 48Khz seems to sound pretty good.  Then I'll just lock the mixer to 48Khz instead of 44.1Khz and no more beating my head against a keyboard to figure out a good way to do this.  I'll have the audio in a format I want to begin with.  As for the resampler, it probably would have just been a lowpass filter with a frequency cutoff of 22.05Khz and then remove every nth sample.

- Michael Tanczos

Apr 15, 2010 at 5:03 PM

you could try the ResamplerDmoStream if the ACM wrapper isn't to your liking.

Mark

 

Apr 15, 2010 at 10:40 PM

Hi

 

I am new here. I wonder if there is any possibility to get an class where I can mix audio streams together. Have you any solution?

Apr 16, 2010 at 2:23 PM

hi goldengel,

try the WaveMixer32Stream class

Mark

Sep 24, 2010 at 1:01 AM

Hi,

I'm trying to get audio streaming over a network using NAudio and stumbled across this but can't seem to get it working. I need to stream different file types so the server side will need to send the stream data as PCM so regardless of the file format it arrives at the client the same.

I have created the class above but can't seem to get it going.

 from the server end (mp3):

NAudio.Wave.Mp3FileReader readerStream = new NAudio.Wave.Mp3FileReader(fileName);
NAudio.Wave.WaveStream pcmStream = NAudio.Wave.WaveFormatConversionStream.CreatePcmStream(readerStream);
NAudio.Wave.WaveStream blockAlignedStream = new NAudio.Wave.BlockAlignReductionStream(pcmStream);
NAudio.Wave.WaveStream inputStream = new NAudio.Wave.WaveChannel32(blockAlignedStream);
I then loop through inputStream and send the data over the network.
from the client end I first init the sound device and create a BufferedSampleStream and point the waveOutDevice to that stream:
waveOutDevice = new NAudio.Wave.DirectSoundOut();

_audioStream = new NAudio.Wave.BufferedSampleStream();
NAudio.Wave.WaveStream inputStream = new NAudio.Wave.WaveChannel32(_audioStream);

waveOutDevice.Init(inputStream);
waveOutDevice.Play();
Then when the network connection on the client receives data, I grab it from the NetworkStream and add it to the BufferedSampleStream:
 
_audioStream.AddSamples(bytReceiveBuffer, 0, (int)bytReceiveBuffer.Length);
 
 
No sound comes out. Where have  I gone wrong?
Sep 24, 2010 at 3:12 AM

OK,

I can now get sound out (there was a problem with my loop for reading the data from the actual audio stream before sending over the network.)

HOWEVER!!!

Now all I get is scrambled noise. I'm guessing I still have a problem with my loop that reads the audio data from the file before sending it (over the network) to the play stream....

Here is some code that essentially does the server/client thing without the actual network bit....

The code below is producing weird noises - and not playing the actual file. It also ends up crashing with an out of memory exception. Playing the file directly works perfectly.

 NAudio.Wave.IWavePlayer waveOutDevice = new DirectSoundOut();
private void button1_Click(object sender, EventArgs e) {
    System.Threading.Thread objThread = new System.Threading.Thread(PlayFile);
    objThread.Start();
}

void PlayFile() {
   string fileName = "c:\\temp\\Music\\1.mp3";
   WaveStream mp3Reader = new Mp3FileReader(fileName);
   WaveStream pcmStream = WaveFormatConversionStream.CreatePcmStream(mp3Reader);
   WaveStream blockAlignedStream = new BlockAlignReductionStream(pcmStream);
   NAudio.Wave.WaveStream audioStream = new WaveChannel32(blockAlignedStream);

   if (true == false) {
      //Play file directly
      //waveOutDevice.Init(audioStream);
   } else {
      NAudio.Wave.BufferedSampleStream bufferedStream = new BufferedSampleStream();
      NAudio.Wave.WaveStream inputStream = new NAudio.Wave.WaveChannel32(bufferedStream);

      waveOutDevice.Init(inputStream);
      waveOutDevice.Play();

      while (true) {
         int intOffset = 0;
         int intSize = 234523;
         byte[] buffer = new byte[intSize];

         int intRead = audioStream.Read(buffer, intOffset, intSize);
         if (intRead == 0) {
            break;
         }

         //SERVER: Code Send data over network


         //CLIENT: Receive data from network and add to buffered stream
         bufferedStream.AddSamples(buffer, intOffset, intSize);
      }
   }
}

Sep 24, 2010 at 6:32 AM

The last code I posted has some major problems within a few seconds, the executable runs out of memory. This is all happening here -> bufferedStream.AddSamples(buffer, intOffset, intSize);

I am completely stuck now. Has anyone got network streaming working with NAudio?

Mar 6, 2013 at 3:53 PM
I know this is several years old now, but I am trying to do the same thing.
I have a new thread: http://naudio.codeplex.com/discussions/435595