SceCli Warning Event 1202, Domain Local Groups, and Alias_Object sAMAccountType, OH MY

I haven't posted in forever, so today it's time to get back to my roots by troubleshooting some good old-fashioned Active Directory problems. I saw this issue in the wild recently, so I thought I'd write about it while it was still fresh on my mind.

An admin came to me asking for help, and explained how one of his customers was experiencing warning events in the Application event log every 5 minutes on their domain controllers, but not their member servers, looking like this:


Warning 1202


Log Name:      Application
Source:        SceCli
Date:          5/1/2014 5:25:56 PM
Event ID:      1202
Task Category: None
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      DC01.CONTOSO.COM
Description:   Security policies were propagated with warning. 0x4b8 : An extended error has occurred.


A couple of things will let you know immediately that this problem has something to do with Group Policy. First, that the event is logged every 5 minutes on each domain controller, which just so happens to be the GPO refresh interval on DCs. Second, that the event source is SceCli, which loosely stands for Security Configuration Client-Side Extension. You'll also notice an accompanying Error ID 7016 in the GroupPolicy event log that gives you even less helpful information:


Error 7016 GroupPolicy


If you were to search for information about this event on the web, you'll no doubt find Microsoft's KB 324383: Troubleshooting SCECLI 1202 Events, and find that the article is almost completely irrelevant to this scenario, except for the little bit at the end that tells you how to enable Winlogon logging. We need to enable that logging on one of the DCs to get a better understanding of what's going on. To enable said logging:

  • Locate and then click the following registry subkey:
    • HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\Winlogon\GPExtensions\{827D319E-6EAC-11D2-A4EA-00C04F79F83A}
  • Add or Edit the following registry value:
    • Value name: ExtensionDebugLevel
    • Data type: DWORD
    • Value data: 2

The log should begin filling up at %SYSTEMROOT%\Security\logs\winlogon.log right away. In that log, I found the underlying cause of the error:

----Configure Group Membership...
                Configure Domain Admins.
                Configure LABS\Domain Admins.
                                object already member of Administrators.
                Configure LABS\IT_Department_Admins.
                Aliases cannot be members of other groups.

                Group Membership configuration was completed with one or more errors.


So something in Group Policy is attempting to configure group membership on this computer (that happens to be a DC) and is encountering an error while doing so. There are only a couple things in Group Policy that are designed to configure group membership. Group Policy Preferences is one... Restricted Groups is another. Let's dig through the GPOs in this domain and see which one is manipulating group memberships:



A GPO linked at the domain level (which means it applies to all domain computers including DCs,) was using Restricted Groups to ensure that the listed security groups were members of the BUILTIN\Administrators group on every computer in the domain. This was working fine on member servers, but causing errors on DCs. There were also other security groups being added to the BUILTIN\Administrators group on member servers and domain controllers successfully. So what was different about the IT_Department_Admins group? Why was it the only group causing us an error? Let's examine the IT_Department_Admins security group:



Spot the difference? The IT_Department_Admins security group is a Domain Local group, while the other groups that were giving us no problems were Global security groups. To cut to the chase instead of droning on about the difference between DL and Global groups, one of the things you'll notice about a Domain Local group object in Active Directory is that it has a sAMAccountType of OBJECT_ALIAS. Page 103 of [MS-SAMR] has this to say about alias accounts:

An alias object refers to a database object whose objectClass attribute is group or derived from group, and whose groupType contains GROUP_TYPE_RESOURCE_GROUP.

Two domains are exposed from a given server: an account domain and a built-in domain; this fact is true for both DC and non-DC configurations. The account domain refers to the object with objectClass domainDNS. The built-in domain refers to the object with the objectClass builtinDomain.

The built-in domain has the characteristic that its objectSid value is invariant (S-1-5-32) through all deployments and only contains aliases. There is exactly one built-in domain for every account domain.

So to fix the issue, we simply converted the Domain Local group to a Global group and called it a day. The issue stems from the fact that domain controllers don't really have a local SAM the same way that standalone Windows machines and Windows domain members do. When you create a DC, the accounts that were part of the "BUILTIN" domain on the computer are removed from the local SAM and put into NTDS.dit and can be found in the Builtin Container using a tool like ADUC.  Accounts in the Builtin container in AD have ... you guessed it: Domain Local scope.

Verifying RPC Network Connectivity Like A Boss

Aloha.  I did some fun Powershelling yesterday and now it's time to share.

If you work in an IT environment that's of any significant size, chances are you have firewalls.  Maybe lots and lots of firewalls. RPC can be a particularly difficult network protocol to work with when it comes to making sure all the ports necessary for its operation are open on your firewalls. I've found that firewall guys sometimes have a hard time allowing the application guy's RPC traffic through their firewalls because of its dynamic nature. Sometimes the application guys don't really know how RPC works, so they don't really know what to ask of the firewall guys.  And to make it even worse, RPC errors can be hard to diagnose.  For instance, the classic RPC error 1722 (0x6BA) - "The RPC server is unavailable" sounds like a network problem at first, but can actually mean access denied, or DNS resolution failure, etc.

MSRPC, or Microsoft Remote Procedure Call, is Microsoft's implementation of DCE (Distributed Computing Environment) RPC. It's been around a long time and is pervasive in an environment containing Windows computers. Tons of Windows applications and components depend on it.

A very brief summary of how the protocol works: There is an "endpoint mapper" that runs on TCP port 135. You can bind to that port on a remote computer anonymously and enumerate all the various RPC services available on that computer.  The services may be using named pipes or TCP/IP.  Named pipes will use port 445.  The services that are using TCP are each dynamically allocated their own TCP ports, which are drawn from a pool of port numbers. This pool of port numbers is by default 1024-5000 on XP/2003 and below, and 49152-65535 on Vista/2008 and above. (The ephemeral port range.) You can customize that port range that RPC will use if you wish, like so:

reg add HKLM\SOFTWARE\Microsoft\Rpc\Internet /v Ports /t REG_MULTI_SZ /f /d 8000-9000
reg add HKLM\SOFTWARE\Microsoft\Rpc\Internet /v PortsInternetAvailable /t REG_SZ /f /d Y
reg add HKLM\SOFTWARE\Microsoft\Rpc\Internet /v UseInternetPorts /t REG_SZ /f /d Y

And/Or

netsh int ipv4 set dynamicport tcp start=8000 num=1001
netsh int ipv4 set dynamicport udp start=8000 num=1001
netsh int ipv6 set dynamicport tcp start=8000 num=1001
netsh int ipv6 set dynamicport udp start=8000 num=1001

This is why we have to query the endpoint mapper first, because we can't just guess exactly which port we need to connect to for a particular service.

So, I wrote a little something in Powershell that will test the network connectivity of a remote machine for RPC, by querying the endpoint mapper, and then querying each port that the endpoint mapper tells me that it's currently using.


#Requires -Version 3
Function Test-RPC
{
    [CmdletBinding(SupportsShouldProcess=$True)]
    Param([Parameter(ValueFromPipeline=$True)][String[]]$ComputerName = 'localhost')
    BEGIN
    {
        Set-StrictMode -Version Latest
        $PInvokeCode = @'
        using System;
        using System.Collections.Generic;
        using System.Runtime.InteropServices;

        public class Rpc
        {
            // I found this crud in RpcDce.h

            [DllImport("Rpcrt4.dll", CharSet = CharSet.Auto)]
            public static extern int RpcBindingFromStringBinding(string StringBinding, out IntPtr Binding);

            [DllImport("Rpcrt4.dll")]
            public static extern int RpcBindingFree(ref IntPtr Binding);

            [DllImport("Rpcrt4.dll", CharSet = CharSet.Auto)]
            public static extern int RpcMgmtEpEltInqBegin(IntPtr EpBinding,
                                                    int InquiryType, // 0x00000000 = RPC_C_EP_ALL_ELTS
                                                    int IfId,
                                                    int VersOption,
                                                    string ObjectUuid,
                                                    out IntPtr InquiryContext);

            [DllImport("Rpcrt4.dll", CharSet = CharSet.Auto)]
            public static extern int RpcMgmtEpEltInqNext(IntPtr InquiryContext,
                                                    out RPC_IF_ID IfId,
                                                    out IntPtr Binding,
                                                    out Guid ObjectUuid,
                                                    out IntPtr Annotation);

            [DllImport("Rpcrt4.dll", CharSet = CharSet.Auto)]
            public static extern int RpcBindingToStringBinding(IntPtr Binding, out IntPtr StringBinding);

            public struct RPC_IF_ID
            {
                public Guid Uuid;
                public ushort VersMajor;
                public ushort VersMinor;
            }

            public static List QueryEPM(string host)
            {
                List ports = new List();
                int retCode = 0; // RPC_S_OK                
                IntPtr bindingHandle = IntPtr.Zero;
                IntPtr inquiryContext = IntPtr.Zero;                
                IntPtr elementBindingHandle = IntPtr.Zero;
                RPC_IF_ID elementIfId;
                Guid elementUuid;
                IntPtr elementAnnotation;

                try
                {                    
                    retCode = RpcBindingFromStringBinding("ncacn_ip_tcp:" + host, out bindingHandle);
                    if (retCode != 0)
                        throw new Exception("RpcBindingFromStringBinding: " + retCode);

                    retCode = RpcMgmtEpEltInqBegin(bindingHandle, 0, 0, 0, string.Empty, out inquiryContext);
                    if (retCode != 0)
                        throw new Exception("RpcMgmtEpEltInqBegin: " + retCode);
                    
                    do
                    {
                        IntPtr bindString = IntPtr.Zero;
                        retCode = RpcMgmtEpEltInqNext (inquiryContext, out elementIfId, out elementBindingHandle, out elementUuid, out elementAnnotation);
                        if (retCode != 0)
                            if (retCode == 1772)
                                break;

                        retCode = RpcBindingToStringBinding(elementBindingHandle, out bindString);
                        if (retCode != 0)
                            throw new Exception("RpcBindingToStringBinding: " + retCode);
                            
                        string s = Marshal.PtrToStringAuto(bindString).Trim().ToLower();
                        if(s.StartsWith("ncacn_ip_tcp:"))                        
                            ports.Add(int.Parse(s.Split('[')[1].Split(']')[0]));
                        
                        RpcBindingFree(ref elementBindingHandle);
                        
                    }
                    while (retCode != 1772); // RPC_X_NO_MORE_ENTRIES

                }
                catch(Exception ex)
                {
                    Console.WriteLine(ex);
                    return ports;
                }
                finally
                {
                    RpcBindingFree(ref bindingHandle);
                }
                
                return ports;
            }
        }
'@
    }
    PROCESS
    {
        ForEach($Computer In $ComputerName)
        {
            If($PSCmdlet.ShouldProcess($Computer))
            {
                [Bool]$EPMOpen = $False
                $Socket = New-Object Net.Sockets.TcpClient
                
                Try
                {                    
                    $Socket.Connect($Computer, 135)
                    If ($Socket.Connected)
                    {
                        $EPMOpen = $True
                    }
                    $Socket.Close()                    
                }
                Catch
                {
                    $Socket.Dispose()
                }
                
                If ($EPMOpen)
                {
                    Add-Type $PInvokeCode
                    $RPCPorts = [Rpc]::QueryEPM($Computer)
                    [Bool]$AllPortsOpen = $True
                    Foreach ($Port In $RPCPorts)
                    {
                        $Socket = New-Object Net.Sockets.TcpClient
                        Try
                        {
                            $Socket.Connect($Computer, $Port)
                            If (!$Socket.Connected)
                            {
                                $AllPortsOpen = $False
                            }
                            $Socket.Close()
                        }
                        Catch
                        {
                            $AllPortsOpen = $False
                            $Socket.Dispose()
                        }
                    }

                    [PSObject]@{'ComputerName' = $Computer; 'EndPointMapperOpen' = $EPMOpen; 'RPCPortsInUse' = $RPCPorts; 'AllRPCPortsOpen' = $AllPortsOpen}
                }
                Else
                {
                    [PSObject]@{'ComputerName' = $Computer; 'EndPointMapperOpen' = $EPMOpen}
                }
            }
        }
    }
    END
    {

    }
}

And the output will look a little something like this:


You can also query the endpoint mapper with PortQry.exe -n server01 -e 135, but I was curious about how it worked at a deeper level, so I ended up writing something myself. There weren't many examples of how to use that particular native API, so it was pretty tough.

Tired of the NSA Seeing What Model of Plug and Play Mouse You're Using?

Not long ago, the story broke that the NSA was capturing internet traffic generated by Windows crash dumps, driver downloads from Windows Updates, Windows Error Reporting, etc.

As per Microsoft's policy, this information, when it contains sensitive or personally identifiable data, is encrypted.

Encryption: All report data that could include personally identifiable information is encrypted (HTTPS) during transmission. The software "parameters" information, which includes such information as the application name and version, module name and version, and exception code, is not encrypted.

While I'm not saying that SSL/TLS poses an impenetrable obstacle for the likes of the NSA, I am saying that Microsoft is not just sending full memory dumps across the internet in clear text every time something crashes on your machine.  But if you were to, for instance, plug in a new Logitech USB mouse, your computer very well could try to download drivers for it from Windows Update automatically, and when that happens, it sends a few details about your PC and the device you just plugged in, in clear text.

Here is where you can read more about that.

So let's say you're an enterprise administrator, and you want to put an end to all this nonsense for all the computers in your organization, such that your computers no longer attempt to contact Microsoft or send data to them when an application crashes or someone installs a new device.  Aside from setting up your own internal corporate Windows Error Reporting server, (who does that?) you can disable the behavior via Group Policy. There are a surprising number of policy settings that should be disabled so that you're not leaking data all over the web:

  • The system will be configured to prevent automatic forwarding of error information.

Configure the policy value for Computer Configuration -> Administrative Templates -> System -> Internet Communication Management -> Internet Communication settings-> “Turn off Windows Error Reporting” to “Enabled”.

  • An Error Report will not be sent when a generic device driver is installed.

Configure the policy value for Computer Configuration -> Administrative Templates -> System -> Device Installation -> "Do not send a Windows error report when a generic driver is installed on a device" to "Enabled".

  • Additional data requests in response to Error Reporting will be declined.

Configure the policy value for Computer Configuration -> Administrative Templates -> Windows Components -> Windows Error Reporting -> "Do not send additional data" to "Enabled".

  • Errors in handwriting recognition on Tablet PCs will not be reported to Microsoft.

Configure the policy value for Computer Configuration -> Administrative Templates -> System -> Internet Communication Management -> Internet Communications settings “Turn off handwriting recognition error reporting” to “Enabled”.

  • Windows Error Reporting to Microsoft will be disabled.

Configure the policy value for Computer Configuration -> Administrative Templates -> Windows Components -> Windows Error Reporting “Disable Windows Error Reporting” to “Enabled”.

  • Windows will be prevented from sending an error report when a device driver requests additional software during installation.

Configure the policy value for Computer Configuration -> Administrative Templates -> System -> Device Installation -> “Prevent Windows from sending an error report when a device driver requests additional software during installation” to “Enabled”.

  • Microsoft Support Diagnostic Tool (MSDT) interactive communication with Microsoft will be prevented.

Configure the policy value for Computer Configuration -> Administrative Templates -> System -> Troubleshooting and Diagnostics -> Microsoft Support Diagnostic Tool -> “Microsoft Support Diagnostic Tool: Turn on MSDT interactive communication with Support Provider” to “Disabled”.

  • Access to Windows Online Troubleshooting Service (WOTS) will be prevented.

Configure the policy value for Computer Configuration -> Administrative Templates -> System -> Troubleshooting and Diagnostics -> Scripted Diagnostics -> “Troubleshooting: Allow users to access online troubleshooting content on Microsoft servers from the Troubleshooting Control Panel (via Windows Online Troubleshooting Service - WOTS)” to “Disabled”.

  • Responsiveness events will be prevented from being aggregated and sent to Microsoft.

Configure the policy value for Computer Configuration -> Administrative Templates -> System -> Troubleshooting and Diagnostics -> Windows Performance PerfTrack -> “Enable/Disable PerfTrack” to “Disabled”.

  • The Application Compatibility Program Inventory will be prevented from collecting data and sending the information to Microsoft.

Configure the policy value for Computer Configuration -> Administrative Templates -> Windows Components -> Application Compatibility -> “Turn off Program Inventory” to “Enabled”.

  • Device driver searches using Windows Update will be prevented.

Configure the policy value for Computer Configuration -> Administrative Templates -> System -> Device Installation -> “Specify Search Order for device driver source locations” to “Enabled: Do not search Windows Update”.

  • Device metadata retrieval from the Internet will be prevented.

Configure the policy value for Computer Configuration -> Administrative Templates -> System -> Device Installation -> “Prevent device metadata retrieval from internet” to “Enabled”.

  • Windows Update will be prevented from searching for point and print drivers.

Configure the policy value for Computer Configuration -> Administrative Templates -> Printers -> “Extend Point and Print connection to search Windows Update” to “Disabled”.

Let's Deploy EMET 4.1!

Howdy!

Let's talk about EMET.  Enhanced Mitigation Experience Toolkit.  Version 4.1 is the latest update, out just last month. It's free. You can find it here. When you download it, make sure you also download the user guide PDF that comes with it, as it's actually pretty good quality documentation for a free tool.

The thing about EMET, is that it is not antivirus. It's not signature-based, the way that traditional AV is. EMET is behavior based.  It monitors the system in real time and watches all running processes for signs of malicious behavior and attempts to prevent them.  It also applies a set of overall system-wide hardening policies that make many types of exploits more difficult or impossible to pull off. The upshot of this approach is that EMET can theoretically thwart 0-days and other malware/exploits that antivirus is oblivious to.  It also allows us to protect legacy applications that may not have been originally written with today's security features in mind.

The world of computer security is all about measure and countermeasure. The attackers come up with a new attack, then the defenders devise a defense against it, then the attackers come up with a way to get around that defense, ad nauseum, forever. But anything you can do to raise the bar for the attackers - to make their jobs harder - should be done.

Here's what the EMET application and system tray icon look like once it's installed:

EMET

From that screenshot, you get an idea of some of the malicious behavior that EMET is trying to guard against.  You can turn on mandatory DEP for all processes, even ones that weren't compiled for it. Data Execution Prevention has been around for a long time, and is basically a mechanism to prevent the execution of code that resides in areas of memory marked as non-executable. (I.e. where only data, not code, should be.)  With DEP on, heaps and stacks will be marked as non-executable and attempts to run code from those areas in memory will fail. Most CPUs these days have the technology baked right into the hardware, so there's no subverting it. (Knock on wood.)

You can turn on mandatory ASLR for all processes on the system, again, even for processes that were not compiled with it.  Address Space Layout Randomization is a technique whereby a process loads modules into "random" memory addresses, whereas in the days before ASLR processes always loaded modules into the same, predictable memory locations. Imagine what an advantage it would be for an attacker to always know exactly where to find modules loaded in any process on any machine.

Then you have your heapspray mitigation. A "heap spray" is an attack technique where the attacker places copies of malicious code in as many locations within the heap as possible, increasing the odds of success that it will be executed once the instruction pointer is manipulated. This is a technique that attackers came up with to aid them against ASLR, since they could no longer rely on predictable memory addresses. By allocating some commonly-used memory pages within processes ahead of time, we can keep the heap sprayer's odds of success low.

Those are only a few of the mitigations that EMET is capable of.  Read that user guide that I mentioned before for much more info.

Oh, and one last thing: Is it possible that EMET could cause certain applications to malfunction? Absolutely! So always test thoroughly before deploying to production. And just like with enterprise-grade antivirus software, EMET also requires a good bit of configuring until you come up with a good policy that suits your environment and gives you the best mix of protection versus application compatibility.

Let's get into how EMET can be deployed across an enterprise and configured via Group Policy. Once you've installed it on one computer, you will notice a Deployment folder in with the program files. In the Deployment folder you will find the Group Policy template files you need to configure EMET across your enterprise via GPO.  First, create your Group Policy Central Store if you haven't already:

Creating Central Store

Copy the EMET.ADMX file into the PolicyDefinitions folder, and EMET.ADML file into the EN-US subfolder.  If all goes well, you will notice a new Administrative Template now when you go to create a new GPO:

EMET GPO

Now you may notice that while I do have the EMET administrative template... all my other admin templates have disappeared! That's because I forgot to copy all of the other admin templates from one of the domain controllers into Sysvol before I took the screen shot. So don't forget to copy over all the other *.admx and *.adml files from one of your DCs before continuing.

Now you can control how EMET is configured in a centralized, consistent, enforceable way on all the computers in your organization.

The next part is deploying the software. The EMET user guide describes using System Center Configuration Manager to deploy the software, and while I agree that SCCM is boss when it comes to deploying software, I don't have it installed here in my lab, so I'm going to just do it via GPO as well.  In fact, I'll do it in the same EMET GPO that defines the application settings too.

Copy the installer to a network share that can be accessed by all the domain members that you intend to deploy the software to:

Copy the MSI

Then create a new GPO with a software package to deploy that MSI from the network location. Make sure the software is assigned to the computer, not the user.  And lastly, you'll likely rip less of your hair out if you turn off asynchronous policy processing like so:

Computer Settings
 + Administrative Templates
    + System
       + Logon
          + Always wait for the network at computer startup and logon: Enabled

And software deployment across an entire organization, that simple. Luckily I didn't even have to apply a transform to that MSI, which is good, because that is something I didn't feel like doing this evening.

Until next time, stay safe, and if you still want to hear more about EMET, watch this cool talk from Neil Sikka about it from Defcon 21!

Finding Hidden Processes with Volatility and Scanning with Sysinternals Sigcheck

Happy Memory-Forensics-Thursday!

I get called in to go malware hunting every once in a while.  It's usually after an automatic vulnerability scanner has found something unusual about a particular computer on the network and threw up a red flag.  Once a machine is suspected of being infected, someone needs to go in and validate whether what the vulnerability scanner found is truly a compromise or a false positive, the nature of the infection, and clean it if possible.  I know that the "safest" reaction to the slightest whiff of malware is to immediately disconnect the machine from the network, format it and reinstall the operating system, but in a busy production environment, that extreme approach isn't always feasible or necessary.

We all know that no antivirus product can catch everything, nor is any vulnerability scanner perfect.  But a human with a bit of skill and the right tools can quickly sniff out things that AV has missed.  Malware hunting and forensic analysis really puts one's knowledge of deep Windows internals to the test, possibly more so than anything else, so I find it extremely fun and rewarding.

So today we're going to talk about two tools that will aid you in your journey.  Volatility and Sigcheck.

Volatility is a wondrous framework for analyzing Windows memory dumps.  You can find it here. It's free and open-source.  It's written in Python, but there is also a compiled exe version if you don't have Python installed.  Volatility is a framework that can run any number of plugins, and these plugins perform data analyses on memory dumps, focused on pointing out specific indicators of compromise, such as API hooks, hidden processes, hooked driver IRP functions, interrupt descriptor table hooks, and so much more.  It's not magic though, and it doesn't do much that you could not also do manually (and with much more painstaking effort) with WinDbg, but it does make it a hell of a lot faster and easier.  (We have to wait until 2014 for Win8/8.1 and Server 2012/2012R2 support.)

But first, before you can use Volatility, you must have a memory dump.  (There is a technology preview branch of Volatility that can read directly from the PhysicalMemory device object.)  There are many tools that can dump memory, such as WinPMem, which you can also find on the Volatility downloads page that I linked to earlier.  It can dump in both RAW format and DMP (Windows crash dump) formats.  Make sure that you download a version with signed drivers, as WinPmem loads a driver to do its business, and modern versions of Windows really don't like you trying to install unsigned drivers.  You can also use LiveKd to dump memory using the command .dump -f C:\memory.dmp.

Since Volatility is such a huge and versatile tool, today I'm only going to talk about one little piece of it - finding "hidden" processes.

When a process is created, the Windows kernel assigns it an _EPROCESS data structure.  Each _EPROCESS structure in turn contains a _LIST_ENTRY structure.  That _LIST_ENTRY structure contains a forward link and a backward link, each pointing to the next _EPROCESS structure on either side of it, creating a doubly-linked list that makes a full circle.  So if I wanted to know all of the processes running on the system, I could start with any process and walk through the _EPROCESS list until I got back to where I started.  When I use Task Manager, tasklist.exe or Process Explorer, they all use API functions that in turn rely on this fundamental mechanism.  Behold my awesome Paint.NET skills:

EPROCESS Good

So if we wanted to hide a process from view, all we have to do is overwrite the backward link of the process in front of us and the forward link of the process behind us to point around us.  That will effectively "unlink" our process of doom, causing it to be hidden:

EPROCESS BAD

This is what we call DKOM - Direct Kernel Object Manipulation.  A lot of rootkits and trojans use this technique.  And even though modern versions of Windows do not allow user mode access to the \\Device\PhysicalMemory object, which is where the _EPROCESS objects will always be because they're in a non-paged pool, we don't need it, nor do we need to load a kernel mode driver, because we can pull off a DKOM attack entirely from user mode by using the ZwSystemDebugControl API.  But we can root out the rootkits with Volatility.  With the command

C:\> volatility.exe --profile=Win7SP0x86 -f Memory.raw psscan

That command shows a list of running processes, but it does it not by walking the _EPROCESS linked list, but by scanning for pool tags and constrained data items (CDIs) that correspond to processes.  The idea is that you compare that list with a list of processes that you got via traditional means, and processes that show up as alive and well on Volatility's psscan list but not Task Manager's list are hidden processes probably up to no good.

There are other methods of finding hidden processes.  For instance, scanning for DISPATCHER_HEADER objects instead of looking at pool tags.  Even easier, a handle to the hidden process should still exist in the handle table of csrss.exe (Client/Server Runtime Subsystem) even after it's been unlinked from the _EPROCESS list, so don't forget to look there.  (There's a csrss_pslist plugin for Volatility as well.)  Also, use the thrdscan plugin to check for threads that belong to processes that don't appear to exist, which would be another sign of tomfoolery.

Alright, so now you've located an executable file that you suspect is malware, but you're not sure.  Scan that sucker with Sigcheck!  Mark Russinovich recently added VirusTotal integration into Sigcheck, with the ability to automatically upload unsigned binaries and have them scanned by 40+ antivirus engines and give you back reports on whether the file appears to be malicious!  Sigcheck can automatically scan through an entire directory structure, just looking for suspicious binaries, uploading them to VirusTotal, and showing you the results.

Remember that you must accept VirusTotal's terms and conditions before using the service.

Uploading suspicious files to VirusTotal is practically a civic responsibility, as the more malicious signatures that VirusTotal has on file, the more effective the antivirus service is for the whole world.