Recipe: EVTX, LogParser, Perl
A long time ago...
It has been a long time since last post, I must sadly admit that. I could argue with many good reasons for this silence, but it's better to avoid useless-reasonable thoughts. I'll say just a couple of things: first, I'd like to share my 2cents so it was not a matter of will; secondarily it's not a matter of missing topics. But sharing is tiresome and labored especially when dealing with DFIR and using a different language (that could be easily spotted, couldn't it?). Finally time scheduling for blogging got 0 slots, and this is the result. OK, let's keep in mind these gold thoughts and let's go (a little) further.
EVTX
As everybody knows, the EVTX is the Windows Event Log File format used in Microsoft Windows OSes starting from Vista/2008 up to now. When facing with Windows XP / 2003, the event log file format used was EVT. There exist on the Net enough resources describing in (great?) details these formats. In the DFIR the EVTX files could/should play an important role during analysis. What tools can we use to interface with them?
LogParser
To the best of my knowledge there are few ways to parse and analyze an EVTX file. It must be mentioned the great work by Andreas Schuster, Evtx Parser (Perl) but the gold medal goes to Microsoft LogParser (downloadable here). On the Microsoft site the tool is described as "a powerful, versatile tool that provides universal query access to text-based data such as log files, XML files and CSV files, as well as key data sources on the Windows® operating system such as the Event Log, the Registry, the file system, and Active Directory®". LogParser can do a lot of really useful stuffs, not only on EVTX and it's not limited on parsing but it owns powerful processing functionalities: unfortunately, it runs only in Windows OSes. I personally don't like so much un-portability but with LogParser you can do interesting fast&furious analysis/processing.Ok, EVTX, LogParser and... Perl?
There are many resources on the Net describing LogParser usages, this post will not cover them. Since LogParser cannot do everything, it could be useful to add a layer able to apply some logic. What's better than Perl (Python/Ruby/Java/.Net/etc.)? By using LogParser to hide EVTX format management and to filter desired data, one can process those outputs with a script to, for example, to summarize data, to detect anomalies automatically and so on.Got recipe, what mission?
Recalling Harlan Carvey evtrpt.pl and evtstats.pl Perl scripts that work on EVT Windows Log files, I needed the same processing of EVTX files. By using LogParser and Perl I was able to quickly write two scripts called evtxrpt.pl and evtxcheck.pl.evtxrpt.pl
This script's goal is to provide an overview and a summary of an EVTX file, at least of what I feel to be a good summary. In the next box is shown the script's result on a Security.evtx file.In the next box is shown the script's result on a Application.evtx file.EVenTX RePorT version 20120331
using Microsoft LogParser, summarize EVTX files
copyright 2012 Francesco Picasso
Source Name Event ID Count
----------- -------- -----
Microsoft-Windows-Eventlog 1100 426
Microsoft-Windows-Eventlog 1101 12
Microsoft-Windows-Security-Auditing 4608 467
Microsoft-Windows-Security-Auditing 4616 194
Microsoft-Windows-Security-Auditing 4624 7571
Microsoft-Windows-Security-Auditing 4634 1110
Microsoft-Windows-Security-Auditing 4647 443
Microsoft-Windows-Security-Auditing 4648 2142
Microsoft-Windows-Security-Auditing 4672 6299
Microsoft-Windows-Security-Auditing 4717 6
Microsoft-Windows-Security-Auditing 4720 3
Microsoft-Windows-Security-Auditing 4722 3
Microsoft-Windows-Security-Auditing 4724 200
Microsoft-Windows-Security-Auditing 4726 4
Microsoft-Windows-Security-Auditing 4728 3
Microsoft-Windows-Security-Auditing 4729 4
Microsoft-Windows-Security-Auditing 4731 6
Microsoft-Windows-Security-Auditing 4732 3
Microsoft-Windows-Security-Auditing 4733 4
Microsoft-Windows-Security-Auditing 4734 6
Microsoft-Windows-Security-Auditing 4735 6
Microsoft-Windows-Security-Auditing 4738 200
Microsoft-Windows-Security-Auditing 4902 467
Microsoft-Windows-Security-Auditing 4904 165
Microsoft-Windows-Security-Auditing 4905 165
Microsoft-Windows-Security-Auditing 4907 8761
Microsoft-Windows-Security-Auditing 5024 467
Microsoft-Windows-Security-Auditing 5033 467
Microsoft-Windows-Security-Auditing 5038 2
Microsoft-Windows-Security-Auditing 5056 467
-------------------- Data Range (UTC) ------------------
2010-02-23 10:19:14
to
2012-03-30 21:02:00
--------- Year/Month distribution -------------
Year Month Count
2010
01 0
02 224
03 1837
04 1534
05 1509
06 1305
07 1298
08 2206
09 2311
10 1439
11 137
12 268
2011
01 325
02 137
03 3452
04 5859
05 1015
06 510
07 633
08 380
09 514
10 606
11 265
12 510
2012
01 606
02 409
03 784
04 0
05 0
06 0
07 0
08 0
09 0
10 0
11 0
12 0
EVenTX RePorT version 20120331
using Microsoft LogParser, summarize EVTX files
copyright 2012 Francesco Picasso
Source Name Event ID Count
----------- -------- -----
.NET Runtime Optimization Service 1101 6
.NET Runtime Optimization Service 1130 78
Application Error 1000 164
Application Error 1005 1
Application Hang 1002 51
Brother BrLog 1001 417
Brother BrLog 1002 63
COM+ 781 1
Chkdsk 26212 3
Chkdsk 26213 1
Chkdsk 26214 1
Customer Experience Improvement Program 1005 1
Desktop Window Manager 9002 1
Desktop Window Manager 9003 2
Desktop Window Manager 9007 2
Desktop Window Manager 9009 635
Desktop Window Manager 9010 2
Desktop Window Manager 9013 2
Desktop Window Manager 9016 224
ESENT 102 438
ESENT 103 93
ESENT 210 6
ESENT 213 3
ESENT 215 3
ESENT 220 6
ESENT 221 3
ESENT 222 3
ESENT 223 3
ESENT 224 1
ESENT 225 2
ESENT 300 344
ESENT 301 370
ESENT 302 344
ESENT 411 1
ESENT 440 1
ESENT 488 1
ESENT 609 42
ESENT 612 42
ESENT 619 2
ESENT 626 40
EventSystem 4625 689
HHCTRL 1903 16
HHCTRL 1904 206
HPSrv 105 664
ITSS 1 2
Microsoft Office 12 2000 1
Microsoft-Windows-Backup 753 9
Microsoft-Windows-Backup 754 4
Microsoft-Windows-CAPI2 513 2
Microsoft-Windows-CAPI2 4097 17
Microsoft-Windows-CAPI2 4107 513
Microsoft-Windows-CAPI2 4109 21
Microsoft-Windows-Defrag 258 115
Microsoft-Windows-LoadPerf 1000 1105
Microsoft-Windows-LoadPerf 1001 1100
Microsoft-Windows-LoadPerf 1002 46
Microsoft-Windows-RestartManager 10000 527
Microsoft-Windows-RestartManager 10001 526
Microsoft-Windows-RestartManager 10002 10
Microsoft-Windows-RestartManager 10005 90
Microsoft-Windows-RestartManager 10010 9
Microsoft-Windows-User Profiles Service 1508 4
Microsoft-Windows-User Profiles Service 1530 183
Microsoft-Windows-User Profiles Service 1531 689
Microsoft-Windows-User Profiles Service 1532 623
Microsoft-Windows-User Profiles Service 1534 1
Microsoft-Windows-User Profiles Service 1542 4
Microsoft-Windows-Winsrv 10001 21
Microsoft-Windows-Winsrv 10002 25
MsiInstaller 1001 4
MsiInstaller 1004 4
MsiInstaller 1005 1
MsiInstaller 1015 1
MsiInstaller 1022 207
MsiInstaller 1024 1
MsiInstaller 1025 1
MsiInstaller 1029 21
MsiInstaller 1031 20
MsiInstaller 1033 183
MsiInstaller 1034 53
MsiInstaller 1035 312
MsiInstaller 1036 208
MsiInstaller 1038 24
MsiInstaller 1040 441
MsiInstaller 1042 441
MsiInstaller 10005 1
MsiInstaller 11310 1
MsiInstaller 11701 2
MsiInstaller 11707 178
MsiInstaller 11708 4
MsiInstaller 11719 1
MsiInstaller 11724 50
MsiInstaller 11725 2
MsiInstaller 11728 230
MsiInstaller 11729 4
MsiInstaller 11730 1
MsiInstaller 11923 45
MsiInstaller 11935 2
Outlook 29 22
Outlook 30 2
Outlook 31 1
PerfNet 2004 29
RasClient 20221 819
RasClient 20222 776
RasClient 20223 738
RasClient 20224 738
RasClient 20225 703
RasClient 20226 716
RasClient 20227 116
STacSV 65535 2
SecurityCenter 1 673
SecurityCenter 11 3
ServiceLayer 0 41
SideBySide 33 34
SignInAssistant 0 357
Software Protection Platform Service 900 744
Software Protection Platform Service 902 744
Software Protection Platform Service 903 697
Software Protection Platform Service 1003 753
Software Protection Platform Service 1004 127
Software Protection Platform Service 1007 1
Software Protection Platform Service 1009 1
Software Protection Platform Service 1011 1
Software Protection Platform Service 1013 1
Software Protection Platform Service 1016 2
Software Protection Platform Service 1025 1
Software Protection Platform Service 1033 75
Software Protection Platform Service 1040 2
Software Protection Platform Service 1066 744
Software Protection Platform Service 8200 6
Software Protection Platform Service 8208 6
Software Protection Platform Service 12304 1
Software Protection Platform Service 12305 1
System Restore 8194 213
System Restore 8195 1
System Restore 8196 2
System Restore 8199 1
System Restore 8202 1
System Restore 8212 17
System Restore 8215 1
TomTomHOMEService 10000 7
VMware NAT Service 1000 1393
VSS 13 2
VSS 8193 2
VSS 8212 5
VSS 8219 6
VSS 8224 369
VSS 12293 11
VSS 12305 1
WinMgmt 63 2
WinMgmt 5611 61
WinMgmt 5615 689
WinMgmt 5617 689
Windows Activation Technologies 1 7
Windows Activation Technologies 2 7
Windows Activation Technologies 10 7
Windows Activation Technologies 11 1
Windows Activation Technologies 13 7
Windows Activation Technologies 15 7
Windows Activation Technologies 18 7
Windows Error Reporting 1001 535
Windows Search Service 1003 385
Windows Search Service 1004 1
Windows Search Service 1005 1
Windows Search Service 1008 1
Windows Search Service 1010 1
Windows Search Service 1013 40
Windows Search Service 3036 208
Windows Search Service 4121 1
Winlogon 4004 29
Winlogon 4005 1
Winlogon 4101 689
Wlclntfy 6000 1943
Wlclntfy 6004 2
ufad-ws60 1 8
vmauthd 100 2
vmauthd 1000 296
vmware-converter-agent 1 2
-------------------- Data Range (UTC) ------------------
2009-10-23 15:26:06
to
2012-03-30 21:33:25
--------- Year/Month distribution -------------
Year Month Count
2009
01 0
02 0
03 0
04 0
05 0
06 0
07 0
08 0
09 0
10 1790
11 2301
12 2437
2010
01 2147
02 713
03 1779
04 1568
05 1138
06 1268
07 1164
08 2198
09 2006
10 1303
11 135
12 323
2011
01 226
02 86
03 716
04 823
05 904
06 480
07 546
08 392
09 424
10 493
11 241
12 380
2012
01 767
02 560
03 1098
04 0
05 0
06 0
07 0
08 0
09 0
10 0
11 0
12 0
evtxcheck.pl
This script's goal is to make some basic checks on an EVTX file to quickly point out anomalies (at least of what I need to be spotted). In the next boxes are shown the script's results on the same Security.evtx and Application.evtx files analyzed by the previous script.EVenTX CHECKer version 20120331
using Microsoft LogParser, makes basic checks on EVTX files
copyright 2012 Francesco Picasso
Total Records in 'Security.evtx': 30073
----- Missing Records Detection -----
First Record Number: 9675
Last Record Number: 39747
no missing records detected
----- Back in Time Detection (Tolerance: 60 secs) -----
no back time jumps detected
----- ComputerName(s) -----
ComputerNames(s) used: 1
hpw
----------------------------------
EVenTX CHECKer version 20120331
using Microsoft LogParser, makes basic checks on EVTX files
copyright 2012 Francesco Picasso
Total Records in 'Application.evtx': 30406
----- Missing Records Detection -----
First Record Number: 1
Last Record Number: 30410
no missing records detected
----- Back in Time Detection (Tolerance: 60 secs) -----
no back time jumps detected
----- ComputerName(s) -----
ComputerNames(s) used: 2
37l4247e29-32
hpw
DETECTED 1 changes
- Set to '37l4247e29-32'
at RecordNumber: 1, Event: 4625, Time: 2009-10-23 15:26:07
- Set to 'hpw'
at RecordNumber: 91, Event: 4625, Time: 2009-10-23 15:30:37
----------------------------------
Note: it could be useful to deepen in the SuppressDuplicate mechanism used by Windows to suppress some repeating events. This feature impacts both records sequence (holes) and "back in time detection": I saw the SuppressDuplicate Event message being shown completely de-synched in time (days...).
EvtxCheck is a great idea! Event IDs and their corresponding time stamps are a fantastic way to identify system time tampering. Log Parser is one of my favorite tools. Thank you for writing the script.
ReplyDeleteHi Chad, I really appreciate your feedback, thanks!
DeleteRegarding "evtxcheck" script and system time tampering: there exist some "false positive". For example the "SuppressDuplicate" event could point out that there are holes in Event IDs and time tampering: I *brutally* tried to avoid that by checking the event ID number (see code), but I cannot assure that's the right way.
Moreover I'm facing a case where script (correctly) points out many "back in time points": by manually checking them I saw some late shutdown event and other stuffs I need to understand.
These scripts are great resources for performing analysis. I included a "-s" switch with evtparse.pl (for XP/2003 EVT files) that would list all records, in order by record number, with the corresponding TimeGenerated value...this would allow you to see possible system time changes clearly.
ReplyDeleteThanks to what's being audited/recorded in the new Windows Event Log, there's a great deal more analysis that can be performed.
Hi Harlan, thanks for your feedback too!
DeleteI will download your updated script once replied.
I agree with you that in Windows Event Logs there are a lot of information that should be analyzed. Unfortunately not all events I'm interested in (or, better, not all situations that cause events generation) are straightforward to me and it's quite hard to find references. There exist many testing to be done...
Some of the things you can get from Windows Event Logs on Win7 are discussed in chapter 4 of "Windows Forensic Analysis Toolkit 3/e".
ReplyDeleteI have all your books apart this... That another reason to fill the gap :)
DeleteThank you!