Solved: IDS/IPS signatures to monitor streaming audio/video applications

zheka_pefti · ‎06-29-2010

Hi folks,
Can someone advise on the names or signatures that could be successfully used to monitor the usage of streaming applications on the network. The plan is to feed them to MARS and then create reports on streaming applications utilization to use it later for creation a security policy preventing bandwidth stealing.

Perhaps any suggestions on how to create a custom signature to monitor audio and video streams would be appreciated.

Eugene

Christopher Dreier · ‎07-05-2010

Hello Eugene,

It is possible to string match the video specified in your capture by examining the Content-Type. Chasing the connection with a TCP-Reset or denying the packet inline will keep the video from playing - which will save the bandwidth that the video would have otherwise used. However, it is important that we establish the IPS appliance's role. The IPS is designed to detect and mitigate attacks by matching known traffic patterns. For TCP, this duty can include as little as dropping one packet to disrupt a flow. The IPS is not fundamentally designed to monitor a flow and provide a byte count for a particular protocol so that protocol usage analysis can be performed.

The signature below will drop packets with the flv-application Content-Type, which will keep the video you tested on break.com from playing. Each video streaming site works differently. A capture from each streaming video site will need to be examined, and another custom signature written, if you wish to block them all. Additionally, keep in mind that several sites offer different options for streaming videos. This may require you to take multiple captures on each site - one for each streaming method.

signatures 60001 0
alert-severity medium
sig-description
sig-name flv-application TCP String
exit
engine string-tcp
event-action produce-alert|reset-tcp-connection
regex-string flv-application
service-ports 80
direction from-service
exit
alert-frequency
summary-mode fire-all
exit
exit
status
enabled true
exit
exit

Thank you,

Blayne Dreier

Cisco TAC IDS Team

**Please check out our Podcast**

TAC Security Show: http://www.cisco.com/go/tacsecuritypodcast

View solution in original post

Christopher Dreier · ‎07-07-2010

Hello Eugene,

I'm not aware of a guide on troubleshooting and creating custom signatures - outside of the IPS configuration guide's "Signature Engines" section (which is actually quite good, link below). TAC does not have an internal guide as such - mostly because creating custom signatures is not a service that TAC provides.

Sniffing the youtube video data itself will reveal a Content-Type video/x-flv. Matching on this Content-Type will keep youtube videos from playing.

You can use the Service HTTP engine to match "youtube" in the Request or Header regex (the packet capture will show that the text appears in both locations), which will cause the entire browser session to be reset. However, this is normally a function assigned to a URL filtering server and not the IPS. It certainly works though, and can be used in testing/proof of concept scenarios.

IPS Configuration Guide - Signature Engines: http://www.cisco.com/en/US/partner/docs/security/ips/7.0/configuration/guide/cli/cli_signature_engines.html

Service HTTP: http://www.cisco.com/en/US/partner/docs/security/ips/7.0/configuration/guide/cli/cli_signature_engines.html#wp1132083

Thank you,
Blayne Dreier
Cisco TAC IDS Team

**Please check out our Podcast**
TAC Security Show: http://www.cisco.com/go/tacsecuritypodcast

View solution in original post

zheka_pefti · ‎07-05-2010

Anyone ???!!!

Is anyone from Cisco advanced security team available? It drives me mad. I just can't fathom that streaming video cannot be matched with a signature.

Here's more input into this problem. I accessed a couple websites with online video and made captures of traffic. I was able to see the response from one of the web server containing "Content-type: application/x-shockwave-flash" string in the HTTP header. Then I created a custom signature using Service HTTP engine and put this regex into the Header Regex field:

[Cc][Oo][Nn][Tt][Ee][Nn][Tt][-][Tt][Yy][Pp][Ee][:][ ][Aa][Pp][Pp][Ll][Ii][Cc][Aa][Tt][Ii][Oo][Nn][/\][Xx][-][Ss][Hh][Oo][Cc][Kk][Ww][Aa][Vv][Ee][-][Ff][Ll][Aa][Ss][Hh]

The signature didn't fire even if I tried using different patterns and occurrences of the above said string. Am I doing something wrong.

The other website with online video didn't give me any specific clues in the HTTP response that might shed some light on the content type/MIME format. The attached print screens show fragments from the captures for HTTP request and the reply. The video gets sent as a regular HTTP data, namely "Content-type: text/html" with "Content-Encoding: gzip"

Am I supposed to create a regex for Content-Encoding which is gzip? Then it would be not feasible at all as the signature might fire for anything other than online video.

If there's no way to do it with IPS/IDS signature than I'd rather say that they are useless items to throw money for. This is just stupid response to Cisco's "fear and intimidation" tactics making everyone pseudo-protected. I'm not trying to build a protection to the customer from video content, on the other hand I want to show the customer the value of Cisco products and provide the management with a visibility of how the bandwidth is used and stolen with accessing video/audio content

Christopher Dreier · ‎07-05-2010

Hello Eugene,

It is possible to string match the video specified in your capture by examining the Content-Type. Chasing the connection with a TCP-Reset or denying the packet inline will keep the video from playing - which will save the bandwidth that the video would have otherwise used. However, it is important that we establish the IPS appliance's role. The IPS is designed to detect and mitigate attacks by matching known traffic patterns. For TCP, this duty can include as little as dropping one packet to disrupt a flow. The IPS is not fundamentally designed to monitor a flow and provide a byte count for a particular protocol so that protocol usage analysis can be performed.

The signature below will drop packets with the flv-application Content-Type, which will keep the video you tested on break.com from playing. Each video streaming site works differently. A capture from each streaming video site will need to be examined, and another custom signature written, if you wish to block them all. Additionally, keep in mind that several sites offer different options for streaming videos. This may require you to take multiple captures on each site - one for each streaming method.

signatures 60001 0
alert-severity medium
sig-description
sig-name flv-application TCP String
exit
engine string-tcp
event-action produce-alert|reset-tcp-connection
regex-string flv-application
service-ports 80
direction from-service
exit
alert-frequency
summary-mode fire-all
exit
exit
status
enabled true
exit
exit

Thank you,

Blayne Dreier

Cisco TAC IDS Team

**Please check out our Podcast**

TAC Security Show: http://www.cisco.com/go/tacsecuritypodcast

zheka_pefti · ‎07-05-2010

Thank you very much, Blayne!!!

Now you got me intrigued. How would you know by looking at the fragments of captures that it should be String TCP engine and not Service HTTP and moreover why "flv-application"? I have never found any good references or guide about it.

I tried it against the same site and bingo the custom signature fired. My plan was first to test it with an alert action and then once we confirm it as a proof of concept we can catch 99% of online video watching by setting the tcp reset action.

Now it seems a never ending saga. I tried this signature against "youtube.com" and unfortunately it didn't fire. Where would I look in the captured file to find the right string? I may attach my pcap file here to this post, it's only 512 KB.

Eugene

Christopher Dreier · ‎07-06-2010

Hello Eugene,

Regarding finding the flv-application Content-Type - your capture showed the website and video you were attempting to load. I started a Wireshark capture, played the video and then matched on the Content-Type in the reassembled packet's header.

String TCP is an unspecialized engine - in that it matches any specified string in a TCP packet. This can offer flexibility for fields that the Service or AIC engines do not successfully match.

Matching for particular sites/players/encoders is definitely a cat-and-mouse game - but that's the fun of custom sigs. The signatures that Cisco provides always target a specific iteration of some exploitation. If we developed them to be wide enough to catch all variations, we would have so many false positives that the solution would become much less useful.

Thank you,
Blayne Dreier
Cisco TAC IDS Team

**Please check out our Podcast**
TAC Security Show: http://www.cisco.com/go/tacsecuritypodcast

zheka_pefti · ‎07-06-2010

Thanks again, Blayne.

Any idea if there are other strings I can match with TCP engine in this cat and mouse game to catch youtube traffic ?

I came across these details about macromedia flash files descriptions - http://www.digitalpreservation.gov/formats/fdd/fdd000130.shtml

They give these magic numbers for file type signatures - Magic numbers Hex: 46 57 53, ASCII: FWS

Can they be utilized in any way to create a custom signature?

What about online radio ?

Eugene

Christopher Dreier · ‎07-06-2010

Youtube uses a similar Content Type. Sniff on port 80 with Wireshark and follow the TCP stream for the video transfer itself. Directly before the video data, you will see the Content Type.

Regarding the magic numbers - anything that comes across the wire can trigger a signature. However, keep in mind that FLV and SWF are often mutually exclusive.

Online radio - It depends on the site/protocol/encoding.

Writing a signature is as simple as monitoring traffic and finding a key that is exclusive to a particular type (or source) of traffic. You can then develop a signature that matches that key.

Thank you,
Blayne Dreier
Cisco TAC IDS Team

**Please check out our Podcast**
TAC Security Show: http://www.cisco.com/go/tacsecuritypodcast

zheka_pefti · ‎07-07-2010

Hi Blayne,

I really appreciate your answers and time you spent. I wish this would be helpful not to me only. I'm still confused by all the intrinsic details of how to make a good custom signature. Is there any good guide? May be TAC has its internal guide on how to troubleshoot and create custom signatures based on regex and content type. I'm looking at the TCP packets of the capture made while watching youtube video and this is what comes from the server:

HTTP/1.1 200 OK
Date: Mon, 05 Jul 2010 23:58:12 GMT
Server: wiseguy/0.6.2
X-Content-Type-Options: nosniff
Content-Encoding: gzip
Set-Cookie: watched_video_id_list=5097f00beb9a2acf9d11293e6452d9adWwMAAABzCwAAAE9UeklpcE45UGg4cwsAAABvOS1VX0l2ME83OHMLAAAAS0V4c0FTRDAtOTg=; path=/; domain=.youtube.com
Expires: Tue, 27 Apr 1971 19:44:06 EST
X-YouTube-MID: pcVY4SnBmeDVtZHpoUkNiVkVOZmpxQzR4SDZFZXMwOWxYeFk3QXk4TVhpWjRKRkNUX2I5U1lB
Cache-Control: no-cache
Content-Type: text/html; charset=utf-8
Content-Length: 17503

q2Lz6;>
}-yXBYycO1`'ky]\P,$E`:wH)U~UZ_kk;o)#zLV19V^&X]~I7T/?L}s^\16o?}H7|2;B77z9%,$(T_%?s'cUd0nTr$l4N~&uHzG@D9kJhaa l,gIs)u2C_%iA+0JII,Q{1'Ih`T1\z7{X+/cy&2z%NvKW4awwIhT
d@,#LBOqz}r+Su8*I86f(6
^odcJ8uaIab0xH|{*JkZD3>,%iU/ux51B>UNhnHyX*4t}!eXfEh!j>mJ|s}p}0f&H6K3#:)1N5bMRvQItU2_64,swb(=P`~Km

I tried to make TCP String based signature and match it against \.[Yy][Oo][Uu][Tt][Uu][Bb][Ee]\.[Cc][Oo][Mm]\. No luck

Then I tried to create HTTP String based signature and by looking at the HTTP portion of the packet which looks like:

GET /watch?v=OTzIipN9Ph8&feature=related HTTP/1.1
Host: www.youtube.com
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-us) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16
Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Referer: http://www.youtube.com/watch?v=o9-U_Iv0O78&playnext_from=TL&videos=PhuEJ6wyeKs&feature=rec-LGOUT-real_rev-rn-3r-7-HM
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Cookie: watched_video_id_list=8c0482051639fa5ffa488173dfe5001aWwIAAABzCwAAAG85LVVfSXYwTzc4cwsAAABLRXhzQVNEMC05OA==; GEO=fb0890c2d1c0f42b3dc126c2e6b9f771cwsAAAAzQ0EYVCBMTDJvAA==; PREF=f1=40000000; VISITOR_INFO1_LIVE=DM3zU9wKOmE; use_hitbox=72c46ff6cbcdb7c5585c36411b6b334edAEAAAAw
Connection: keep-alive

I enabled Header Regex to match against [Hh][Oo][Ss][Tt]\:.\.[Yy][Oo][Uu][Tt][Uu][Bb][Ee]\.[Cc][Oo][Mm]\. and still no luck

I intentionally used Header regex as I assume that HTTP header portion starts after the first CRLN (\r\n) and ends with CRLNCRLN (\r\n\r\n)

Eugene

Christopher Dreier · ‎07-07-2010

Hello Eugene,

I'm not aware of a guide on troubleshooting and creating custom signatures - outside of the IPS configuration guide's "Signature Engines" section (which is actually quite good, link below). TAC does not have an internal guide as such - mostly because creating custom signatures is not a service that TAC provides.

Sniffing the youtube video data itself will reveal a Content-Type video/x-flv. Matching on this Content-Type will keep youtube videos from playing.

You can use the Service HTTP engine to match "youtube" in the Request or Header regex (the packet capture will show that the text appears in both locations), which will cause the entire browser session to be reset. However, this is normally a function assigned to a URL filtering server and not the IPS. It certainly works though, and can be used in testing/proof of concept scenarios.

IPS Configuration Guide - Signature Engines: http://www.cisco.com/en/US/partner/docs/security/ips/7.0/configuration/guide/cli/cli_signature_engines.html

Service HTTP: http://www.cisco.com/en/US/partner/docs/security/ips/7.0/configuration/guide/cli/cli_signature_engines.html#wp1132083

Thank you,
Blayne Dreier
Cisco TAC IDS Team

**Please check out our Podcast**
TAC Security Show: http://www.cisco.com/go/tacsecuritypodcast

zheka_pefti · ‎07-07-2010

Again, thanks to you, Blayne.

I found that I'm an idiot and instead of using "Follow TCP Stream" menu item exactly like you advised I stupidly kept staring at all other TCP packets and got lost and confused in the myriads of information and other rubbish presented by Wireshark.

And it was very easy to find the content type for audio: audio/mpeg.

Viva Wireshark! Viva TAC engineers !