This project has moved and is read-only. For the latest updates, please go here.

Odd byte[] array behaviour in WaveFormat

Jan 27, 2014 at 12:14 PM
Edited Jan 27, 2014 at 12:46 PM
I'm preparing AacWaveFormat based on WaveFormat which has a variable 'Extra Size' at the end. So I added a byte[] array at the end like in WaveFormatExtraData.cs

In AacWaveFormat:
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 40)]
private byte[] pbAudioSpecificConfig = new byte[40];

In WaveFormatExtraData.cs:
// try with 100 bytes for now, increase if necessary
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 100)]
private byte[] extraData = new byte[100];

But Media Foundation doesn't accept it, it says:

System.Runtime.InteropServices.COMException (0xC00D36B4): The data specified for the media type is invalid, inconsistent, or not supported by this object. (Exception from HRESULT: 0xC00D36B4)
at NAudio.MediaFoundation.IMFTransform.SetInputType(Int32 dwInputStreamID, IMFMediaType pType, _MFT_SET_TYPE_FLAGS dwFlags)

The weird thing is that I can add as many normal bytes as I want

public byte pbAudioSpecificConfig1;
public byte pbAudioSpecificConfig2;
public byte pbAudioSpecificConfig7;
public byte pbAudioSpecificConfig9999999;

...which is fine because the ExtraSize is 12 (stock standard AAC Extra Size), anything before pbAudioSpecificConfig1 is not read anyway.

But as soon as I add a "private byte[] pbAudioSpecificConfig = new byte[1];" of any size including 0, I get an exception, even though it is AFTER the 12 standard AAC extra bytes which should be ignored.
   [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Pack = 2)]
    public class AacWaveFormat : WaveFormat
        // *** typedef struct heaacwaveformat_tag {
        // ***  HEAACWAVEINFO wfInfo;
        // ***  BYTE          pbAudioSpecificConfig[1];

        /* HEAACWAVEINFO structure */
        public ushort wPayloadType;
        /// <summary>
        /// wPayloadType
        /// </summary>
        public ushort wAudioProfileLevelIndication;
        /// <summary>
        /// wStructType
        /// </summary>
        public ushort wStructType;
        /// <summary>
        /// wReserved1
        /// </summary>
        public ushort wReserved1;
        /// <summary>
        /// dwReserved2
        /// </summary>
        public UInt32 dwReserved2;   

// All of the above is the 12 standard bytes. Anything after is not read...     

        /* BYTE pbAudioSpecificConfig[1]; */
        /// <summary>
        /// pbAudioSpecificConfig
        /// </summary>
        //public byte[] pbAudioSpecificConfig = new byte[2]; // EXCEPTION
        public byte pbAudioSpecificConfig1; // no problem
        public byte pbAudioSpecificConfig2; // no problem       
        public byte pbAudioSpecificConfig3; // test... no problem
        public byte pbAudioSpecificConfig4; // test
        public byte pbAudioSpecificConfig5; // test
        public byte pbAudioSpecificConfig6; // test
        public byte pbAudioSpecificConfig7; // test .. I can add a million of these, no problem
        // EXCEPTION The data specified is invalid
        [MarshalAs(UnmanagedType.ByValArray, SizeConst = 1)]
        private byte[] pbAudioSpecificConfig = new byte[1]; // byte[2]; // byte[100]; // byte[0]
Tried making it public, no change. Tried removing the MarshalAs, no change. Tried making it an empty array (byte[0]), no change.

Why does MF fail when I introduce a byte[] array AFTER the bytes it is told to read?

Maybe WaveFormatExtraData.cs has the same problem, which I believe it does because I am getting the same exception when loading a WaveFormatExtraData in Media Foundation.

It appears a byte[] array is simply not acceptable in a StructLayout used through a COM interface.


Apparently removing the "= new byte[x]" from the Struct and adding it to the constructor is supposed to work, but no dice.
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 2)]
private byte[] pbAudioSpecificConfig;// = new byte[2];

public AacWaveFormat(......)
    pbAudioSpecificConfig = new byte[2]; // same exception. If I remove this, it works fine. But I can't allocate the bytes obviously, plus it is fixed to 2 bytes since "SizeConst = 2" in the struct.
Perhaps it is because "Pack = 2"?
Maybe byte[] arrays only work with "Pack = 1" since they are an array of sequential data?
Jan 27, 2014 at 2:59 PM
what MF function are you actually calling that throws the exception? Maybe I used the NAudio custom waveformat marshaller for that bit, in which case you'll need to tell it about your AAC one.
Jan 27, 2014 at 3:13 PM
Same as in CreateTransform in MediaFoundationResampler.cs. SetInputType()
Jan 28, 2014 at 12:55 AM
Edited Jan 28, 2014 at 12:55 AM
Here is the code:
                var comObject = CreateAACDecoderComObjectUsingActivator(null);
                var resamplerTransform = (IMFTransform)comObject;
                // Input Media Type
                IMFMediaType inputMediaFormat = MediaFoundationApi.CreateMediaTypeFromWaveFormat(inputWaveFormat);
                resamplerTransform.SetInputType(0, inputMediaFormat, 0); // <EXCEPTION

                    // Output Media Type
                IMFMediaType outputMediaFormat = MediaFoundationApi.CreateMediaTypeFromWaveFormat(outputWaveFormat);
        resamplerTransform.SetOutputType(0, outputMediaFormat, 0);              
Jan 28, 2014 at 8:05 AM
Shame that that approach doesn't work with COM. What I usually fall back to is marshalling an IntPtr and Marshal.StructToIntPtr (or whatever it is called)
Alternatively in your case, do you even need to use CreateMediaTypeFromWaveFormat? Might it be simpler just to create a MediaType and set up its properties (you can find out what properties it should have with the MFT enumeration demo).
Jan 28, 2014 at 8:17 AM
Edited Jan 28, 2014 at 9:54 AM
I'm doing it both ways actually:
public MediaFoundationDecoder(IMFMediaType inputMediaType, IMFMediaType outputMediaType)
public MediaFoundationDecoder(WaveFormat inputFormat, WaveFormat outputFormat)

But for some reason when I pass through the IMFMediaType I get this exception:
System.InvalidCastException: Unable to cast COM object of type 'System.__ComObject' to interface type 'NAudio.MediaFoundation.IMFMediaType'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{44AE0FA8-EA31-4109-8D2E-4CAE4997C555}' failed due to the following error: No such interface supported (Exception from HRESULT: 0x80004002 (E_NOINTERFACE)).

...Anyway I got WaveFormatExtraData working in MediaFoundation by removing the byte[] and adding simple bytes. It seems MediaFoundation checks the whole structure for arrays and refuses to handle them.
   [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Pack = 2)]
    public class WaveFormatExtraDataNEW : WaveFormat
        // try with 100 bytes for now, increase if necessary
        //[MarshalAs(UnmanagedType.ByValArray, SizeConst = 14)]
        //private byte[] extraData = new byte[14];

        /// <summary>
        /// Allows the extra data to be read
        /// </summary>
        //public byte[] ExtraData { get { return extraData; } }
        public byte byte1;
        public byte byte2;
        public byte byte3;
        public byte byte4;
        public byte byte5;
        public byte byte6;
        public byte byte7;
        public byte byte8;
        public byte byte9;
        public byte byte10;
        public byte byte11;
        public byte byte12;
        public byte byte13;
        public byte byte14;
Jan 28, 2014 at 9:52 AM
Edited Jan 28, 2014 at 10:30 AM
This is strange, I cant seem to create an IMFMediaType in one class and send it through the constructor of another class, it seems to break. Even a simple GetUInt32() fails on it. Same exception. I'm creating IMFMediaType the exact same way.

Maybe that is why whoever wrote MediaFoundationTransform.cs ran into the same problem and made CreateTransform() function as an abstract function. I'll just have to do something similar and avoid parsing IMFMediaType through the Constructor.

Anyway, the byte[] problem in WaveFormat is the biggest issue. It doesn't matter if I use CreateMediaTypeFromWaveFormat to create the WaveFormat or use my own wave format, if it has a byte[] in it, even after the ExtraSize cut off, MF fails to load it.

I think we should modify WaveFormatExtraData.cs and set the array this way:

// in struct
IntPtr pbAudioSpecificConfig = Marshal.AllocCoTaskMem(2);

// adding data
Marshal.WriteByte(pbAudioSpecificConfig, 0, 0x12);
Marshal.WriteByte(pbAudioSpecificConfig, 1, 0x10);

It seems to be the proper way of doing it in .NET. I'll test it and see how it goes.
Jan 28, 2014 at 1:28 PM
Edited Jan 28, 2014 at 1:37 PM
Using Marshal.WriteByte to write the extra bytes works fine.

But copying the whole struct in unmanaged code doesn't seem to work. I guess that is where the custom marshal comes in, correct?

Or using Marshal.StructToIntPtr() as you mentioned?

I'm not too good at all this Interop stuff :p
Feb 28, 2014 at 9:54 AM
yes, I don't actually pass WaveFormatExtraData directly into an interop signature, just use Marshal.StructToIntPtr (and vice versa). I thought that was working, as I used it quite a lot with the ACM format enumeration work I was doing.
Mar 2, 2014 at 12:53 AM
I already have this working but with the new WaveFormatExtraData using separate bytes instead of the byte[] array so MF accepts it.
        AacWaveFormat aac = new AacWaveFormat(44100,2,0,0,null);
        IntPtr ptr = Marshal.AllocHGlobal(Marshal.SizeOf(AacWaveFormat));
        WaveFormat waveFormat = MediaFoundationApi.CreateWaveFormatFromMediaType(mt);
        Marshal.StructureToPtr(waveFormat, ptr, true);
        aac = (AacWaveFormat)Marshal.PtrToStructure(ptr, typeof(AacWaveFormat));