This project has moved. For the latest updates, please go here.

Filtered audio playing silently

May 25, 2011 at 1:50 PM

Hi Mark!

So I'm trying to use the Skype Voice Changer as a template for filtering audio.  The sequence I'm doing is pretty simple:

1)  Read in audio samples
2)  Filter audio samples with a lowpass filter
3)  Write the filtered audio back into the wavestream

All of this while the audio is playing.  So I wrap a custom stream around the playing audio and put the logic above in the ProcessData function to be called after data is read in. My issue is that whenever audio gets filtered and written back into the stream I hear only silence coming from stream.  I have verified that the samples being filtered are valid and that they are correctly being written into the stream.  Is this a latency issue or am I doing something wrong?  Code is below.  Thank you for all your help so far Mark, you've been a huge help!  I owe you either a steak or a beer if you're ever in RTP, NC.

        private void ProcessAudio(byte[] buffer, int offset, int count)
        {
            float[] inputSamples = new float[count / 2];
            float[] outputSamples = new float[count / 2];
            int currentSampleIndex = 0;

//  Gather the audio samples

            for (int sample = 0; sample < count / 2; sample++)
            {
                int x = offset + sample * 2;
                short sample16Left = BitConverter.ToInt16(buffer, x);
                short sample16Right = sample16Left;
                if (numberOfChannels == 2)
                {
                    sample16Right = BitConverter.ToInt16(buffer, x + 2);
                    sample++;
                }

                float sample64Left = sample16Left / 32768.0f;
                float sample64Right = sample16Right / 32768.0f;

                inputSamples[currentSampleIndex++] = sample64Left;
                if (numberOfChannels == 2)
                {
                    inputSamples[currentSampleIndex++] = sample64Right;
                }
            }

//  Filter the audio

            bool filteredAudio = FilterUtility.filterAudioSamples(inputSamples, outputSamples, filterChain);
            if (!filteredAudio)
            {
                outputSamples = inputSamples;
            }

//  Write it back into the stream

            for (int sample = 0; sample < count / 2; sample++)
            {
                int x = offset + sample * 2;

                short sample16Left = (short)(outputSamples[sample] * 32768.0f);
                short sample16Right = 0;

                buffer[x] = (byte)(sample16Left & 0xFF);
                buffer[x + 1] = (byte)((sample16Left >> 8) & 0xFF);
                if (numberOfChannels == 2)
                {
                    sample++;
                    sample16Right = (short)(outputSamples[sample] * 32768.0f);
                    buffer[x + 2] = (byte)(sample16Right & 0xFF);
                    buffer[x + 3] = (byte)((sample16Right >> 8) & 0xFF);
                }
                log.Info("l: " + sample16Left + ", r: " + sample16Right);  // logging to verify samples are correct
            }

Coordinator
May 25, 2011 at 2:26 PM

I can't see anything wrong from a quick glance through your code. One good way to test this kind of thing is to create a very simple signal path from a WaveFileReader, through your custom stream and out to a WaveFileWriter to process one second of audio. Then you can examine the output WAV file, and if that is populated correctly, the problem is not with your filter stream. Can you post the code from your Read method?

PS, in the next NAudio there is an ISampleProvider interface, meaning you won't have to do all the bit manipulation yourself.

Mark

May 25, 2011 at 3:05 PM

Thanks for the quick response!  Here is the read method I'm using:

        public override int Read(byte[] buffer, int offset, int count)
        {
            int bytesRead;
            lock (sourceLock)
            {
                bytesRead = SourceStream.Read(buffer, offset, count);
            }
            lock (effectLock)
            {
                ProcessAudio(buffer, offset, bytesRead);
            }

            return bytesRead;
        }

I also took your advice about running the audio from a WaveFileReader through my filtering stream (FilteringStream.cs) and out through a WaveFileWriter.  Here is that code below:

    class Program
    {
        static void Main(string[] args)
        {

            FilterChain filterChain = new FilterChain();

//  Create a lowpass filter

            Lowpass lowpass = new Lowpass();
            lowpass.setCutoffFrequency(2000);
            lowpass.setStopbandAttenuation(120);
            lowpass.setWindowSize(120);
            lowpass.setSampleRate(48000);
            lowpass.setNTaps(512);
            lowpass.setEnabled(true);
            lowpass.synthesize();
            filterChain.addFilterToChain(lowpass);

            GlobalVariables.CURRENT_FILE_NAME = "C:\\MightyRushingWind30sec.wav";
            WaveStream reader = new WaveFileReader(GlobalVariables.CURRENT_FILE_NAME);

//  Wrap the reader with my custom filter stream class

            FilteringStream stream = new FilteringStream(reader, filterChain, reader.WaveFormat.Channels);

//  Write out the data to a test file

            using (WaveFileWriter writer = new WaveFileWriter(@"C:\temp\test.wav", reader.WaveFormat))
            {
                byte[] buf = new byte[4096];
                for (; ; )
                {
                    int cnt = stream.Read(buf, 0, buf.Length);
                    if (cnt == 0) break;
                    writer.WriteData(buf, 0, cnt);
                }
            }

        }
    }

This works great!  The audio is correctly filtered and saved out to the test.wav file.  For the life of me I can't figure out why it won't play correctly when listening to it in real time.  Should I save it to a file and then start playing the audio?

 

 

Coordinator
May 25, 2011 at 3:48 PM

you could, or there may be a simple bug in your audio playback code. Is your app a console app or WinForms/WPF? What output driver are you using?

May 25, 2011 at 6:02 PM

The App is a WPF application.  The output driver I'm using is DirectSound.  Here's the code for the playback:

 

        public void createWaveOutPlayerWithFiltering(bool enableLooping)
        {
            try
            {
                CreateFilteredWaveOut();  //  Create the wave out with DirectSound
            }
            catch (Exception driverCreateException)
            {
                System.Windows.Forms.MessageBox.Show(String.Format("{0}", driverCreateException.Message));
                return;
            }

//  Create the output stream with the filtering stream wrapped around it

            mainOutputStream = CreateFilterInputStream(GlobalVariables.CURRENT_FILE_NAME, enableLooping);
          
            try
            {
                waveOut.Init(mainOutputStream);
            }
            catch (Exception initException)
            {
                System.Windows.Forms.MessageBox.Show(String.Format("{0}", initException.Message), "Error Initializing Output");
                return;
            }

            waveOut.Play();
        }

May 26, 2011 at 1:56 PM

I've slightly altered the example above to narrow the scope a little bit.  This is a console application that uses a flag to determine whether to play the audio or save it to a file.  It saves to the file just fine, but when it tries to play the audio I don't hear anything.  I hope this helps.  And thank you for your help with all of this!

static void Main(string[] args)
{

    FilterChain filterChain = new FilterChain();

    Lowpass lowpass = new Lowpass();
    lowpass.setCutoffFrequency(200);
    lowpass.setStopbandAttenuation(120);
    lowpass.setWindowSize(120);
    lowpass.setSampleRate(48000);
    lowpass.setNTaps(512);
    lowpass.setEnabled(true);
    lowpass.synthesize();
//           filterChain.addFilterToChain(lowpass);

    GlobalVariables.CURRENT_FILE_NAME = "C:\\MightyRushingWind30sec.wav";
    WaveFileReader reader = new WaveFileReader(GlobalVariables.CURRENT_FILE_NAME);

    WaveStream readerStream = new WaveFileReader(GlobalVariables.CURRENT_FILE_NAME);
    FilteringStream stream = new FilteringStream(readerStream, filterChain, readerStream.WaveFormat.Channels);

    bool saveToFile = true;
    int offset = 0;
    if (saveToFile)
    {
    using (WaveFileWriter writer = new WaveFileWriter(@"C:\temp\test.wav", readerStream.WaveFormat))
    {
        byte[] buf = new byte[4096];
        for(;;)
        {
        int cnt = stream.Read(buf, 0, buf.Length);
        if (cnt == 0) break;
        writer.WriteData(buf, 0, cnt);
        offset += cnt;
        }
    }
    }
    else
    {
    Console.WriteLine("Playing Audio....");
    IWavePlayer waveOut = new DirectSoundOut(3000);
    waveOut.Init(stream);
    waveOut.Play();
    }

    Console.Read();
}

Here's the full code for the FilteringStream class:

    public class FilteringStream : WaveStream
    {
        public WaveStream SourceStream { get; private set; }
        private static readonly ILog log = LogManager.GetLogger("QELogger");
        private int numberOfChannels = 0;
        public WaveStream audioStream;
        public FilterChain filterChain = new FilterChain();
        private object sourceLock = new object();
        private object effectLock = new object();
       
        public FilteringStream(WaveStream sourceStream, FilterChain fc, int noc)
        {
            SourceStream = sourceStream;
            filterChain = fc;
            foreach (FixedFIRFilter next in filterChain.getFilterList())
            {
                if (next.isEnabled())
                {
                    Console.WriteLine("Current sample rate: " + GlobalVariables.CURRENT_FILE_SAMPLE_RATE);
                    next.sampleRate = GlobalVariables.CURRENT_FILE_SAMPLE_RATE;
                    next.synthesize();
                }
            }
            numberOfChannels = noc;
            log.Info("Filtering stream filter chain has " + filterChain.getNumberOfActiveFilters() + " active filters");
            log.Info("Wave stream has " + numberOfChannels + " channels");
        }

        public override WaveFormat WaveFormat
        {
            get { return SourceStream.WaveFormat; }
        }

        public override long Length
        {
            get { return SourceStream.Length; }
        }

        public override long Position
        {
            get { return SourceStream.Position; }
            set { SourceStream.Position = value; }
        }

        public override int Read(byte[] buffer, int offset, int count)
        {
            int bytesRead;
            lock (sourceLock)
            {
                bytesRead = SourceStream.Read(buffer, offset, count);
            }
            lock (effectLock)
            {
                ProcessAudio(buffer, offset, bytesRead);
            }

            return bytesRead;
        }

        public void printSamples(String label, float[] samples)
        {
            StringBuilder buffer = new StringBuilder();
            buffer.Append(label);
            for (int i = 0; i < samples.Length; i++)
            {
                if (i != samples.Length - 1) buffer.Append(samples[i] + ", ");
                else buffer.Append(samples[i]);
            }
            log.Info(buffer.ToString());
        }

        private void ProcessAudio(byte[] buffer, int offset, int count)
        {
            float[] inputSamples = new float[count / 2];
            float[] outputSamples = new float[count / 2];
            int currentSampleIndex = 0;
           
            for (int sample = 0; sample < count / 2; sample++)
            {
                int x = offset + sample * 2;
                short sample16Left = BitConverter.ToInt16(buffer, x);
                short sample16Right = sample16Left;
                if (numberOfChannels == 2)
                {
                    sample16Right = BitConverter.ToInt16(buffer, x + 2);
                    sample++;
                }

                float sample64Left = sample16Left / 32768.0f;
                float sample64Right = sample16Right / 32768.0f;

                inputSamples[currentSampleIndex++] = sample64Left;
                if (numberOfChannels == 2)
                {
                    inputSamples[currentSampleIndex++] = sample64Right;
                }
            }

            bool filteredAudio = FilterUtility.filterAudioSamples(inputSamples, outputSamples, filterChain);
            if (!filteredAudio)
            {
                outputSamples = inputSamples;
            }

            for (int sample = 0; sample < count / 2; sample++)
            {
                int x = offset + sample * 2;

                short sample16Left = (short)(outputSamples[sample] * 32768.0f);
                short sample16Right = 0;

                buffer[x] = (byte)(sample16Left & 0xFF);
                buffer[x + 1] = (byte)((sample16Left >> 8) & 0xFF);
                if (numberOfChannels == 2)
                {
                    sample++;
                    sample16Right = (short)(outputSamples[sample] * 32768.0f);
                    buffer[x + 2] = (byte)(sample16Right & 0xFF);
                    buffer[x + 3] = (byte)((sample16Right >> 8) & 0xFF);
                }
            }

                     

        }
    }

 

May 26, 2011 at 2:12 PM

Here's another clue.  Changing the logic for creating the wave out from:

IWavePlayer waveOut = new DirectSoundOut(3000);
waveOut.Init(stream);
waveOut.Play();

to this:

WaveOut waveOut = new WaveOut(WaveCallbackInfo.NewWindow());
waveOut.DesiredLatency = 20000;
waveOut.NumberOfBuffers = 2;
waveOut.DeviceNumber = 0;
waveOut.Init(stream);
waveOut.Play();

Gives me exactly 20 seconds of live audio where as before I was getting 0 seconds.  Does that help any?

Coordinator
May 26, 2011 at 2:52 PM

Well, your waveOut is going out of scope when you leave the else clause, meaning that it could get picked up by the garbage collector while you are still playing. Move the Console.Read up inside the else clause and see if that helps.

Mark