I haven't posted in forever, so today it's time to get back to my roots by troubleshooting some good old-fashioned Active Directory problems. I saw this issue in the wild recently, so I thought I'd write about it while it was still fresh on my mind.
An admin came to me asking for help, and explained how one of his customers was experiencing warning events in the Application event log every 5 minutes on their domain controllers, but not their member servers, looking like this:
Log Name: Application
Date: 5/1/2014 5:25:56 PM
Event ID: 1202
Task Category: None
Description: Security policies were propagated with warning. 0x4b8 : An extended error has occurred.
A couple of things will let you know immediately that this problem has something to do with Group Policy. First, that the event is logged every 5 minutes on each domain controller, which just so happens to be the GPO refresh interval on DCs. Second, that the event source is SceCli, which loosely stands for Security Configuration Client-Side Extension. You'll also notice an accompanying Error ID 7016 in the GroupPolicy event log that gives you even less helpful information:
If you were to search for information about this event on the web, you'll no doubt find Microsoft's KB 324383: Troubleshooting SCECLI 1202 Events, and find that the article is almost completely irrelevant to this scenario, except for the little bit at the end that tells you how to enable Winlogon logging. We need to enable that logging on one of the DCs to get a better understanding of what's going on. To enable said logging:
Locate and then click the following registry subkey:
Add or Edit the following registry value:
Value name: ExtensionDebugLevel
Data type: DWORD
Value data: 2
The log should begin filling up at %SYSTEMROOT%\Security\logs\winlogon.log right away. In that log, I found the underlying cause of the error:
----Configure Group Membership...
Configure Domain Admins.
Configure LABS\Domain Admins.
object already member of Administrators.
Aliases cannot be members of other groups.
Group Membership configuration was completed with one or more errors.
So something in Group Policy is attempting to configure group membership on this computer (that happens to be a DC) and is encountering an error while doing so. There are only a couple things in Group Policy that are designed to configure group membership. Group Policy Preferences is one... Restricted Groups is another. Let's dig through the GPOs in this domain and see which one is manipulating group memberships:
A GPO linked at the domain level (which means it applies to all domain computers including DCs,) was using Restricted Groups to ensure that the listed security groups were members of the BUILTIN\Administrators group on every computer in the domain. This was working fine on member servers, but causing errors on DCs. There were also other security groups being added to the BUILTIN\Administrators group on member servers and domain controllers successfully. So what was different about the IT_Department_Admins group? Why was it the only group causing us an error? Let's examine the IT_Department_Admins security group:
Spot the difference? The IT_Department_Admins security group is a Domain Local group, while the other groups that were giving us no problems were Global security groups. To cut to the chase instead of droning on about the difference between DL and Global groups, one of the things you'll notice about a Domain Local group object in Active Directory is that it has a sAMAccountType of OBJECT_ALIAS. Page 103 of [MS-SAMR] has this to say about alias accounts:
An alias object refers to a database object whose objectClass attribute is group or derived from group, and whose groupType contains GROUP_TYPE_RESOURCE_GROUP.
Two domains are exposed from a given server: an account domain and a built-in domain; this fact is true for both DC and non-DC configurations. The account domain refers to the object with objectClass domainDNS. The built-in domain refers to the object with the objectClass builtinDomain.
The built-in domain has the characteristic that its objectSid value is invariant (S-1-5-32) through all deployments and only contains aliases. There is exactly one built-in domain for every account domain.
So to fix the issue, we simply converted the Domain Local group to a Global group and called it a day. The issue stems from the fact that domain controllers don't really have a local SAM the same way that standalone Windows machines and Windows domain members do. When you create a DC, the accounts that were part of the "BUILTIN" domain on the computer are removed from the local SAM and put into NTDS.dit and can be found in the Builtin Container using a tool like ADUC. Accounts in the Builtin container in AD have ... you guessed it: Domain Local scope.