Decoding RTP Voice Traffic with Unknown Codec 52

  

I was genuinely surprised when I learned that our VoIP recording software had detected the use of an unknown Codec 52 in our IP network. That simply shouldn't be possible. To the best of my knowledge, only standard codecs are in use here - typically the legacy G.711, or in some cases, the more modern wideband G.722.2 (AMR-WB) for HD voice. But Codec 52? It must be a mistake - such a codec shouldn't even exist.

Check RTP Header Structure 

Let's start by reviewing the standard to understand how VoIP payload types are encoded.

RTP: A Transport Protocol for Real-Time Applications (RFC3550)


   The RTP header has the following format:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |V=2|P|X|  CC   |M|     PT      |       sequence number         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   

The first byte of the RTP header consists of:
  - version (V): 2 bits
  - padding (P): 1 bit
  - extension (X): 1 bit
  - CSRC count (CC): 4 bits

The second byte:
  - marker (M): 1 bit
  - payload type (PT): 7 bits

Now, let's compare the RTP header as defined in the standard with the traffic captured by Wireshark. As shown in the packet capture screenshot below, the payload type of the recorded RTP stream is actually 8.

 

Payload Type Definitions

Encoding type 8 corresponds to the standard PCMA (G.711) codec. The range from 35 to 71 is officially unused.

Payload Types:


 PT   encoding    media type  clock rate   channels
                    name                    (Hz)
 ___________________________________________________
 0    PCMU        A            8,000       1
 1    reserved    A
 2    reserved    A
 3    GSM         A            8,000       1
 4    G723        A            8,000       1
 5    DVI4        A            8,000       1
 6    DVI4        A           16,000       1
 7    LPC         A            8,000       1
 8    PCMA        A            8,000       1
 9    G722        A            8,000       1
               
 ...
               
 35-71   unassigned  ?

However, the recording software reports an unknown codec with the value 52 in decimal (0x34 in hexadecimal) instead of 8.

How is that possible? Where did the software get a byte value of 0x34?

Solution

The issue turned out to be the presence of a 4-byte VLAN tag in the captured traffic. As a result, the recorder misread the RTP payload type, picking the wrong byte - shifted back 4 bytes. Let me examine the byte located 4 bytes earlier.

As you can see, 4 bytes above the misread location lies the UDP header, where the length field has a value of 0xB4 in hexadecimal - that's 180 in decimal or 1011 0100 in binary. When this byte is incorrectly interpreted as the RTP payload type, and only the lower 7 bits are considered (starting from the second bit), we get 011 0100, which is 52 in decimal.

Once we account for the 4-byte shift caused by the VLAN tag, the correct RTP payload type is revealed to be 8, corresponding to the standard PCMA (G.711) codec. In other words, Codec 52 doesn’t actually exist.

The mystery is solved. No miracle here - Codec 52 has not been seen in the wild. Stay safe and healthy.