Port-Independent SSL Detection

By: Ben Williams

Many network-based applications are gaining support for native end-to-end Transport Layer encryption using Secure Sockets Layer (SSL). Secure web connections over HTTPS has been a standard for online merchants, webmail, and financial sites for a number of years. As SSL-based encryption becomes more common for legitimate uses, it has also become more common for malicious purposes. Practically any protocol can be placed in an SSL wrapper to help avoid detection and logging. Some examples of this include IRC and Jabber chat, reverse command shells, and malware beaconing activity.

When monitoring a network or sifting through a large amount of packet data, it is important to identify what network connections were made and what the underlying protocol was for every unique conversation. Many COTS products provide simple protocol detection by keying off the server’s port number and will not provide an indicator that SSL may have been used on a non-standard port.

This entry will focus on the current versions of SSL: SSLv3 and TLSv1. In my experience, these versions account for the majority of SSL traffic across numerous applications. So my usage of ‘SSL’ refers to SSLv3 and TLSv1 throughout this entry.

SSL payloads should always begin with a well-defined five byte header:

Byte 0: Content Type

• 0x14: SSL_Change_Cipher_Spec
• 0x15: SSL_Alert
• 0x16: SSL_Handshake
• 0x17: SSL_Application_Data

Once an SSL session is established, the majority of SSL content type should consist of SSL_Application_Data records. This simply indicates that SSL-encrypted data is being transferred between the authenticated hosts.

Bytes 1-2: SSL Version

• 0x0300: SSLv3
• 0x0301: TLSv1

Independent of SSL content type, the SSL version should always be indicated.

Bytes 3-4: SSL record length

This SSL record length excludes the five byte header.

For this scenario, the SSL record length is not relevant. I am only interested in determining whether SSL data exists within the packet capture, and specifically if it was used on a port other than the standard TCP 443. The search will match against the four valid SSL Content Types at offset 0 of a data payload, as well as the two valid SSL Version types at offset 1 of a data payload. The quickest and most direct way I’ve found to search for SSL data across all network ports is to utilize standard tshark filters.

To begin, start with a tshark filter that will match against the above conditions and print out the default one-line summary of each matching packet:

tshark -r dec20.pcap -R "data.data[0:1] >= 14 && data.data[0:1] <= 17 && (data.data[1:2] == 0300 || data.data[1:2] == 0301)"

An explanation of each option is as follows:

-r         Read packet data from infile
-R        Apply the following display filter before printing or writing the packets

Inspect one byte of each packet’s data payload, at offset 0, to see if the byte falls within the range 0x14 to 0x17 (valid SSL Content Types):

data.data[0:1] >= 14 && data.data[0:1] <= 17

Also inspect two bytes of each packet’s data payload, at offset 1, to see if the bytes are equal to 0x0300 or 0x0301 (valid SSL Versions):

data.data[1:2] == 0300 || data.data[1:2] == 0301

The result of this query against the above packet capture revealed the following:

This is a quick way of identifying specific TCP sessions that require further inspection. To view the full hex and ASCII content of these packets, add the –x flag to the tshark command:

tshark -r dec20.pcap -R "data.data[0:1] >= 14 && data.data[0:1] <= 17 && (data.data[1:2] == 0300 || data.data[1:2] == 0301)" -x

-x         Print a hex and ASCII dump of the packet data

The result of this query against the same packet capture revealed something interesting about the TCP session identified earlier:

If SSL data does exist within the packet capture, most of the content would of course appear to be garbage at this point. But, the above packet shows that 192.168.1.200 has sent what appears to be an SSL server certificate to 192.168.1.168 and provides multiple clues about what’s actually going on between these hosts:

1. 192.168.1.200 is most likely an SSL server for something on TCP 6697
2. 192.168.1.200 is likely running SSL-based IRC services for darknetz.irc
3. 192.168.1.168 is likely running an SSL-enabled IRC client

A quick open-source lookup shows that TCP port 6697 isn’t the official port for IRC SSL, but it is often used as such. The combination of this information provides pretty high confidence that this is SSL-encrypted IRC. Depending on your environment, the next steps of analysis or investigation would greatly vary. If the IRC server’s private key file could be obtained, this traffic could then be properly decoded and decrypted into clear-text IRC traffic.

If used on a large corporate network, the last tshark filter would probably return a high volume of packet data for legitimate encrypted web traffic. To filter out the more common HTTPS traffic and begin the hunt for anomalous or unknown SSL connections, add another simple tshark read filter to the last query:

tcp.port != 443

This technique may lead to the identification of protocols or applications that were previously unknown to a security analysis, forensic, or engineering team. As stated before, it is critical to identify and understand the protocols in a network monitoring or pcap analysis situation. Advanced attackers and threats purposefully hide in ‘network noise’ under the notion that most organizations won’t recognize the single outbound encrypted connection that contains malicious activity.