Finding Hidden Processes with Volatility and Scanning with Sysinternals Sigcheck

Happy Memory-Forensics-Thursday!

I get called in to go malware hunting every once in a while.  It's usually after an automatic vulnerability scanner has found something unusual about a particular computer on the network and threw up a red flag.  Once a machine is suspected of being infected, someone needs to go in and validate whether what the vulnerability scanner found is truly a compromise or a false positive, the nature of the infection, and clean it if possible.  I know that the "safest" reaction to the slightest whiff of malware is to immediately disconnect the machine from the network, format it and reinstall the operating system, but in a busy production environment, that extreme approach isn't always feasible or necessary.

We all know that no antivirus product can catch everything, nor is any vulnerability scanner perfect.  But a human with a bit of skill and the right tools can quickly sniff out things that AV has missed.  Malware hunting and forensic analysis really puts one's knowledge of deep Windows internals to the test, possibly more so than anything else, so I find it extremely fun and rewarding.

So today we're going to talk about two tools that will aid you in your journey.  Volatility and Sigcheck.

Volatility is a wondrous framework for analyzing Windows memory dumps.  You can find it here. It's free and open-source.  It's written in Python, but there is also a compiled exe version if you don't have Python installed.  Volatility is a framework that can run any number of plugins, and these plugins perform data analyses on memory dumps, focused on pointing out specific indicators of compromise, such as API hooks, hidden processes, hooked driver IRP functions, interrupt descriptor table hooks, and so much more.  It's not magic though, and it doesn't do much that you could not also do manually (and with much more painstaking effort) with WinDbg, but it does make it a hell of a lot faster and easier.  (We have to wait until 2014 for Win8/8.1 and Server 2012/2012R2 support.)

But first, before you can use Volatility, you must have a memory dump.  (There is a technology preview branch of Volatility that can read directly from the PhysicalMemory device object.)  There are many tools that can dump memory, such as WinPMem, which you can also find on the Volatility downloads page that I linked to earlier.  It can dump in both RAW format and DMP (Windows crash dump) formats.  Make sure that you download a version with signed drivers, as WinPmem loads a driver to do its business, and modern versions of Windows really don't like you trying to install unsigned drivers.  You can also use LiveKd to dump memory using the command .dump -f C:\memory.dmp.

Since Volatility is such a huge and versatile tool, today I'm only going to talk about one little piece of it - finding "hidden" processes.

When a process is created, the Windows kernel assigns it an _EPROCESS data structure.  Each _EPROCESS structure in turn contains a _LIST_ENTRY structure.  That _LIST_ENTRY structure contains a forward link and a backward link, each pointing to the next _EPROCESS structure on either side of it, creating a doubly-linked list that makes a full circle.  So if I wanted to know all of the processes running on the system, I could start with any process and walk through the _EPROCESS list until I got back to where I started.  When I use Task Manager, tasklist.exe or Process Explorer, they all use API functions that in turn rely on this fundamental mechanism.  Behold my awesome Paint.NET skills:


So if we wanted to hide a process from view, all we have to do is overwrite the backward link of the process in front of us and the forward link of the process behind us to point around us.  That will effectively "unlink" our process of doom, causing it to be hidden:


This is what we call DKOM - Direct Kernel Object Manipulation.  A lot of rootkits and trojans use this technique.  And even though modern versions of Windows do not allow user mode access to the \\Device\PhysicalMemory object, which is where the _EPROCESS objects will always be because they're in a non-paged pool, we don't need it, nor do we need to load a kernel mode driver, because we can pull off a DKOM attack entirely from user mode by using the ZwSystemDebugControl API.  But we can root out the rootkits with Volatility.  With the command

C:\> volatility.exe --profile=Win7SP0x86 -f Memory.raw psscan

That command shows a list of running processes, but it does it not by walking the _EPROCESS linked list, but by scanning for pool tags and constrained data items (CDIs) that correspond to processes.  The idea is that you compare that list with a list of processes that you got via traditional means, and processes that show up as alive and well on Volatility's psscan list but not Task Manager's list are hidden processes probably up to no good.

There are other methods of finding hidden processes.  For instance, scanning for DISPATCHER_HEADER objects instead of looking at pool tags.  Even easier, a handle to the hidden process should still exist in the handle table of csrss.exe (Client/Server Runtime Subsystem) even after it's been unlinked from the _EPROCESS list, so don't forget to look there.  (There's a csrss_pslist plugin for Volatility as well.)  Also, use the thrdscan plugin to check for threads that belong to processes that don't appear to exist, which would be another sign of tomfoolery.

Alright, so now you've located an executable file that you suspect is malware, but you're not sure.  Scan that sucker with Sigcheck!  Mark Russinovich recently added VirusTotal integration into Sigcheck, with the ability to automatically upload unsigned binaries and have them scanned by 40+ antivirus engines and give you back reports on whether the file appears to be malicious!  Sigcheck can automatically scan through an entire directory structure, just looking for suspicious binaries, uploading them to VirusTotal, and showing you the results.

Remember that you must accept VirusTotal's terms and conditions before using the service.

Uploading suspicious files to VirusTotal is practically a civic responsibility, as the more malicious signatures that VirusTotal has on file, the more effective the antivirus service is for the whole world.

Comments are closed