Hunting Exploit Kits with Open Proxy Server

Hey, It has been a while since the last update.

I have been wondering how the researcher get intel on exploit kits. It is easy for large enterprise, you can just setup ids at gateway. Same goes to AV/Security company, they get it from telemetry. How about part-time researcher?

First, what is an exploit kits?
"An exploit kit is basically a utility program or toolkit that can deliver an exploit to its corresponding target program. If the exploit is successful, the kit can then deliver a malicious payload to the compromised computer or mobile device. If you think of a single exploit as being an 'arrow' that can only hit one particular 'sweet spot' on a target, then an exploit kit is the 'bow' that can launch an entire quiverful of arrows at any target that happens to be within range.

In order to get targets to attack, exploit kit operators will typically host their kits on websites, which may be either maliciously crafted websites, or legitimate ones that have been compromised. The kits can then silently probe the computers or mobile devices of any visitors to the site. In some cases, attackers may increase the flow of potential victims to the exploit kit by using some form of web traffic hijacking to redirect more visitors to the poisoned website. For example, websites might be hacked in order to quietly redirect users to the site hosting the exploit kit.

If a visitor's machine is found to be vulnerable to the exploit, the kit then downloads a payload onto the victim (essentially, a drive-by download attack). The payload can be tailored according to the exploit kit operator's wishes, but typically include downloading such malware as ransomware, botnet-related components and banking-trojans.

Most exploit kits can also be updated by their creators or controllers (not always the same party) to add new exploits, allowing them to target any new vulnerabilities found without much fuss. For example, when the Hacking Team data breach occurred in early 2015, exploit code that was detailed in the exposed data was quickly added to various exploit kits."

Flow of Exploit kits:

+-------------------------------+
+      Compromised Website      +
+-------------------------------+
               ||                
               \/                
+-------------------------------+
+           TDS/GATE            +
+-------------------------------+
               ||                
               \/                
+-------------------------------+
+         Exploit Kits          +
+-------------------------------+

What do you need for hunting exploit kits
1. Signature/Rules
2. Access to a lot of HTTP Traffic

Here is my setup for hunting

+-------------------------------+
+        Open Proxy Server      +
+-------------------------------+
               ||                
               \/                
+-------------------------------+
+          ICAP Server          +
+-------------------------------+
               ||                
               \/                
+-------------------------------+
+   Scan URL/Content with Yara  +
+-------------------------------+

ICAP in a nutshell
"The Internet Content Adaptation Protocol (ICAP) is a lightweight HTTP-like protocol specified in RFC 3507 which is used to extend transparent proxy servers, thereby freeing up resources and standardizing the way in which new features are implemented. ICAP is generally used to implement virus scanning and content filters in transparent HTTP proxy caches. Content adaptation refers to performing the particular value added service (content manipulation) for the associated client request/response."

While searching for ICAP server with yara support. I stumbled upon this project, as you may see it is old project with old yara version... and it written in C...
So, I decided to write my own ICAP Server with ability to scan the content/url by using yara. I found this ICAP Server framework, and start working on top of it.

Python ICAP Yara 
- An ICAP Server with yara scanner for URL and content.

This ICAP server designed to handle RESPMOD method. RESPMOD is called when a HTTP response must be modified, in other words, ICAP receive all the request and response header alongside with the response content.

How my ICAP Server works
Example saved information

{
    "content": "hex encoded content",
    "request_header": {
        "accept": [
            "*/*"
        ],
        "host": [
            "blog.rz.my"
        ],
        "user-agent": [
            "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:51.0) Gecko/20100101 Firefox/51.0""
        ]
    },
    "response_header": {
        "content-type": [
            "text/html; charset=UTF-8"
        ],
        "date": [
            "Thu, 09 Feb 2017 03:30:34 GMT"
        ],
        "server": [
            "GSE"
        ],
        "referer": [
            "https://www.google.com/"
        ]
    },
    "rules": [
        "rule1",
        "rule2"
    ]
}

Beside from JSON-formatted data, there's information about triggered rules shown in console also written into separate log file

Example:



References

1. https://www.f-secure.com/en/web/labs_global/exploit-kits
2. https://github.com/Peoplecantfly/icapserver
3. https://github.com/VirusTotal/yara/