Probably the Craziest Powershell One-Liner I've Written To Date

Someone at work asked me to identify duplicate computers in two separate AD forests, and remove the one that was no longer needed.  It's assumed as part of business policy that there should not be duplicate server hostnames anywhere in the company - even if they reside in different forests or domains.  But for some reason or another, a computer might get migrated from DomainA to DomainB, but the computer object stays behind in the old domain, etc.  So I decided to just collect all the computers from DomainA and DomainB (in ForestA and ForestB respectively), point out the computer accounts that had the same name in each domain, and list their PwdLastSet attribute next to their name.  If the machine had not updated its password in over 30 days in DomainA, while the machine password was up to date in DomainB, then it was reasonably safe to assume that the machine had been migrated out of DomainA and into DomainB, or vice versa.

I only had Powershell v2 on hand, so I didn't have the relative luxury of automatic foreach, etc.  After collecting the computer objects like $DomainAComputers = Get-ADComputer -Filter * -Properties *, check out this hideous monstrosity I came up with to compare them in a single line:

PS C:\Users\ryan> foreach($C In $(Compare-Object $($DomainAComputers|?{!($_.DistinguishedName.Contains("Disabled Accounts"))}|%{$_.Name}) $($DomainBComputers|?{!($_.DistinguishedName.Contains("Disabled Accounts"))}|%{$_.Name}) -IncludeEqual | ? { $_.SideIndicator -eq "==" })) { $o = $($DomainAComputers|?{$_.Name -eq $C.InputObject}); $n = $($DomainBComputers|?{$_.Name -eq $C.InputObject}); $o.DnsHostName + "`t" + $o.PasswordLastSet + "`t" + $n.DnsHostName + "`t" + $n.PasswordLastSet }

The output looks like this:

computer1.domainA.com    04/17/2013    computer1.domainB.com    01/21/2010
computer2.domainA.com    05/05/2013    computer2.domainB.com    10/11/2011
etc...

You can easily see now that the two computers in DomainA are active, while the computer objects of the same name in DomainB are stale, so I'll delete them.

Now don't get me wrong - this is not elegant or clever. It's thoroughly unreadable and ugly and I'd not brag about it except to say, "Haha, look how much s*#! I can cram on one single line of Powershell!"

A couple things that I thought were interesting:

  • Get-ADComputer gives you a free pseudo-attribute called PasswordLastSet, which is a nicely formatted DateTime. But it's not a "real" attribute of the object in Active Directory. Rather, it's the Powershell cmdlet's courtesy attribute where it automatically converts the real attribute - PwdLastSet - from file time (epoch seconds) to a .NET DateTime object. Many of the Active Directory cmdlets work that way.
  • Compare-Object -ExcludeDifferent didn't seem to work and I'm not sure why.  So I had to just use -IncludeEqual instead and isolate the names that were equal.

My Entry for the Advanced Event #1 of the 2013 Scripting Games

I've been pretty excited about the annual scripting games. This is only my second Games, but they have been a terrific Powershell learning experience for me. This year it's being run by Don Jones and his gang:

http://scriptinggames.org/

http://powershell.org/wp/

Their PHP-based website has already shown to be a little buggy, and I will eat road salt before I enable Java in my browser, so I won't be using their chat room, but you have to cut them some slack as it's a brand new site that has never been used before.

When people comment on the scripts you submit, it can be humbling but is also a good learning experience for being able to tell what people wanted out of your script but didn't get. There's not any error-handling to speak of in this script - I knew that was a risk I was taking by submitting a script with no error handling, but the event description stated that I should "display all errors clearly," which is exactly what the script does with no error handling. Still, I could have still used error handling to make it a little more elegant. Also, I guess I've got to break down and start doing the -WhatIf and -Confirm junk, even though I don't exactly want to, it's going the extra mile.

Without further ado:

#Requires -Version 3
Function Move-OldFiles
{
	<#
		.SYNOPSIS
			Move files that are older than a specified number of days. (Default is 90 days.)
			Use the verbose switch if you want to see output, otherwise the Cmdlet shows only errors.
		.DESCRIPTION
			Move files that are older than a specified number of days (default 90) from the 
			source directory to a destination directory. Directory recursion is on by default,
			but can be disabled with the -NoRecurse switch. The subdirectory structure will be 
			preserved at the destination. By default, all files are moved, but a file pattern
			can be specified with the -Pattern parameter. By default, files that already exist at
			the destination and are readonly are not overwritten, unless the -Force switch is used.
			This cmdlet works with drive letters as well as UNC paths. By default, only errors are shown.
			Use the -Verbose switch if you want to see more output. This function requires Powershell 3.
		.PARAMETER SourceDirectory
			Specifies the source directory from which you want to move files. E.g. C:\Logs or C:\Logs\
			This must be a valid directory. The alias for this parameter is Source.			
		.PARAMETER DestinationDirectory
			Specifies the destination directory to which you want to move files. E.g. E:\Archives or
			E:\Logs\Old\ or \\SERVER02\Share\Logs. This must be a valid directory. The alias for
			this parameter is Destination.
		.PARAMETER OlderThan
			The number of days that a file's age must exceed before it will be moved. This is
			an optional parameter whose default is 90 days. This parameter must be a positive
			integer. The alias for this parameter is Age.
		.PARAMETER Pattern
			This is an optional filename filter. E.g., *.log or *.txt or Report*.html.
			The alias for this parameter is Filter.
		.PARAMETER NoRecurse
			This is a switch that indicates whether the cmdlet will process files in subdirectories
			underneath the specified source directory. By default, recursion is on. Optional.
		.PARAMETER Force
			This is a switch that indicates whether files that already exist at the destination
			and are readonly will be overwritten. By default they are not overwritten. Optional.
		.EXAMPLE
			PS C:\> Move-OldFiles -Source C:\Application\Log -Destination \\NASServer\Archives -OlderThan 90 -Filter *.log
		.EXAMPLE
			PS C:\> Move-OldFiles C:\Logs \\NASServer\Archives 90 *.log
		.EXAMPLE
			PS C:\> Move-OldFiles -SourceDirectory C:\Logs -DestinationDirectory \\NAS\Archives -Age 31 -Pattern *.log -Force
		.EXAMPLE
			PS C:\> Move-OldFiles C:\Logs \\NAS\Archives
	#>
	[CmdletBinding()]
	Param([Parameter(Position = 0, Mandatory = $True, HelpMessage = 'Source directory, e.g. C:\Logs')]
			[ValidateScript({Test-Path $_ -PathType Container})]
			[Alias('Source')]
			[String]$SourceDirectory,
	      [Parameter(Position = 1, Mandatory = $True, HelpMessage = 'Destination directory, e.g. \\NASServer\Archives')]
			[ValidateScript({Test-Path $_ -PathType Container})]
			[Alias('Destination')]
			[String]$DestinationDirectory,
		  [Parameter(Position = 2, Mandatory = $False, HelpMessage = 'The number of days old the file must be in order to be moved.')]
			[ValidateScript({$_ -GT 0})]
			[Alias('Age')]
			[Int]$OlderThan = 90,
		  [Parameter(Position = 3, Mandatory = $False, HelpMessage = 'The file pattern to match, e.g. *.log')]
			[Alias('Filter')]
			[String]$Pattern = "*",
		  [Parameter(Position = 4, Mandatory = $False, HelpMessage = 'Disable directory recursion, i.e. only copy the directory specified.')]
			[Switch]$NoRecurse = $False,
		  [Parameter(Position = 5, Mandatory = $False, HelpMessage = 'Specify to overwrite existing readonly files at the destination.')]
			[Switch]$Force = $False)
	
	$Start = Get-Date
    If(!($SourceDirectory.EndsWith("\")))
    {
	    $SourceDirectory = $SourceDirectory + "\"
    }
    If(!($DestinationDirectory.EndsWith("\")))
    {
        $DestinationDirectory = $DestinationDirectory + "\"
    }
	
	Write-Verbose "Source Directory:       $SourceDirectory"
	Write-Verbose "Destination Directory:  $DestinationDirectory"
	Write-Verbose "Move Files Older Than:  $OlderThan Days"
	Write-Verbose "Filename Filter:        $Pattern"
	Write-Verbose "Exclude Subdirectories: $NoRecurse"
	Write-Verbose "Overwrite if Readonly:  $Force"
	
	If($NoRecurse)
	{
		$SourceFiles = Get-ChildItem -Path $SourceDirectory -Filter $Pattern -File | Where-Object LastWriteTime -LT (Get-Date).AddDays($OlderThan * -1)
		Write-Verbose "$($SourceFiles.Count) files found in $SourceDirectory matching pattern $Pattern older than $OlderThan days."
	}
	Else
	{
		$SourceFiles = Get-ChildItem -Path $SourceDirectory -Filter $Pattern -File -Recurse | Where-Object LastWriteTime -LT (Get-Date).AddDays($OlderThan * -1)
		Write-Verbose "$($SourceFiles.Count) files found in $SourceDirectory matching pattern $Pattern older than $OlderThan days."
	}
	
	[Int]$FilesMoved = 0
	ForEach($File In $SourceFiles)
	{
		Write-Verbose "Moving $($File.FullName)"
		$DestinationFullName = $DestinationDirectory + $($File.FullName).Replace($SourceDirectory, $null)
		$DestinationFileDirectory = $DestinationFullName.Replace($DestinationFullName.Split('\')[-1], $null)
		If($PSBoundParameters['Verbose'])
		{
			Write-Progress -Activity "Move-OldFiles" `
						   -Status "Moving files..." `
						   -CurrentOperation "Transferring $($File.FullName)`..." `
						   -PercentComplete $([Math]::Round($FilesMoved / $SourceFiles.Count * 100, 0))
		}		
		If($Force)
		{
			If(!(Test-Path $DestinationFileDirectory -PathType Container))
			{
				Write-Verbose "Creating directory $DestinationFileDirectory"
				New-Item $DestinationFileDirectory -Type Directory | Out-Null
			}
		    Move-Item -Path $File.FullName -Destination $DestinationFullName -Force | Out-Null
		}
		Else
		{
			If(!(Test-Path $DestinationFileDirectory -PathType Container))
			{
				Write-Verbose "Creating directory $DestinationFileDirectory"
				New-Item $DestinationFileDirectory -Type Directory | Out-Null
			}
		    Move-Item -Path $File.FullName -Destination $DestinationFullName | Out-Null
		}
		$FilesMoved++
	}
	$End = Get-Date
	Write-Verbose "$($SourceFiles.Count) files were moved in $([Math]::Round(((New-TimeSpan $Start $End).TotalSeconds), 1)) seconds."
}

Get-FQDNInfo.ps1

Someone recently asked me if I could write a script for them.  He had a list of several hundred fully qualified domain names (internet URLs essentially) in a file, and he had to get the IP address(es) of each FQDN and then some whois information about those IP addresses.  Running down a list of names and resolving them scriptomatically is a breeze of course, but the whois stuff sounded a little more tricky.  Luckily, ARIN has a sweet REST API ready to go, that made the whole script a snap.

I took the time to return all that data as objects so the output can be pipelined, and there is also an optional "save to CSV" parameter.  I think there are a couple more ways in which the script could be improved, but it works for now.  The output looks like this:

Get-FQDNInfo.ps1

And here's the whole script:

<#
.SYNOPSIS	
	Feed me a bunch of FQDNs, one per line, and I will give you as much info as
	I can about that IP address.
	
.DESCRIPTION
	This script takes an input file. The input file contains a list of FQDNs, one per line.
	With each FQDN, the script will attempt to resolve the name, and then find as much
	info as it can using ARIN REST services.
	
.PARAMETER InFile
	Specify a text file containing the FQDNs you want to scan.
	Each FQDN goes on a separate line. For example:
	
	host.foo.com
	barney.purpledinosaur.com
	et.phonehome.org

.PARAMETER OutFile
	Optional file to write the results to.

.INPUTS
	[System.String]$InFile - The name of the input file to read.
.INPUTS
	[System.String]$OutFile - Optional, the name of the output file to write.

.OUTPUTS
	[System.Object]$FQDNInfoCollection - Contains resolved FQDNInfo objects.

.OUTPUTS
	[System.IO.File]$OutFile - Optional, the output file to write.
	
.EXAMPLE
	PS C:\> .\Get-FQDNInfo.ps1 .\fqdns.txt outfile.txt

.EXAMPLE
	PS C:\> "fqdns.txt" | .\Get-FQDNInfo.ps1
	
.NOTES
	Name  : Get-FQDNInfo.ps1
	Author: Ryan Ries
	Email : ryanries09@gmail.com
	Date  : April 19, 2013

.LINK	
	http://www.myotherpcisacloud.com
#>

Param([Parameter(Mandatory=$True,ValueFromPipeline=$True)][ValidateScript({Test-Path $_ -PathType Leaf})][String]$InFile, [Parameter(Mandatory=$False,ValueFromPipeline=$False)][String]$OutFile)
$FQDNInfoCollection = @()
$EntriesProcessed = 0
Foreach($FQDN In Get-Content $InFile)
{
	$FQDNInfo = New-Object System.Object
	$FQDNInfo | Add-Member -Type NoteProperty -Name "FQDN"        -Value $FQDN
	$FQDNInfo | Add-Member -Type NoteProperty -Name "AddressList" -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "CSVSafeList" -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "NetRange"    -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "CIDR"        -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "NetName"     -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "NetType"     -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "RegDate"     -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "Updated"     -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "Comment"     -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "SOA"         -Value $null
	
	Try	{ $FQDNInfo.AddressList = $([System.Net.Dns]::GetHostEntry($FQDN)).AddressList }
	Catch {	}
	If($FQDNInfo.AddressList -ne $null)
	{
		ForEach($A In $FQDNInfo.AddressList) { $FQDNInfo.CSVSafeList += "$($A)|" }
		$FQDNInfo.CSVSafeList = $FQDNInfo.CSVSafeList.TrimEnd('|')		
		Try
		{
			$ARINData = $(Invoke-WebRequest http://whois.arin.net/rest/ip/$($FQDNInfo.AddressList[0].ToString())`.txt).Content
			$ARINData = $ARINData.Split([Environment]::NewLine)
			Foreach($l In $ARINData)
			{
				If($l.StartsWith("NetRange:"))    { $FQDNInfo.NetRange = $l.SubString(16) }
				Elseif($l.StartsWith("CIDR:"))    { $FQDNInfo.CIDR     = $l.SubString(16) }
				Elseif($l.StartsWith("NetName:")) { $FQDNInfo.NetName  = $l.SubString(16) }
				Elseif($l.StartsWith("NetType:")) { $FQDNInfo.NetType  = $l.SubString(16) }
				Elseif($l.StartsWith("RegDate:")) { $FQDNInfo.RegDate  = $l.SubString(16) }
				Elseif($l.StartsWith("Updated:")) { $FQDNInfo.Updated  = $l.SubString(16) }
				Elseif($l.StartsWith("Comment:")) 
				{ 
					$FQDNInfo.Comment += $l.SubString(16)
					$FQDNInfo.Comment += " "
				}
			}
		}
		Catch { }
		& nslookup -q=soa $FQDN 2>&1> $Env:UserProfile`\temp.txt
		Foreach($_ In Get-Content $Env:UserProfile`\temp.txt)
		{
			If($_.Contains("primary name server =")) { $FQDNInfo.SOA = $_.Split('=')[1].Trim() }
		}		
	}	
	$FQDNInfoCollection += $FQDNInfo
	$EntriesProcessed   += 1
	Write-Host $EntriesProcessed "FQDNs processed."
}

If($OutFile.Length -gt 0)
{
	$FQDNInfoCollection | Export-CSV $OutFile -NoTypeInformation	
}
return $FQDNInfoCollection

ShareDiscreetlyWebServer v1.0.0.3

I wrote a web service.  I call it "ShareDiscreetly".  Creative name, huh?

I wrote the server in C# .NET 4.0.  It runs as a Windows service.

ShareDiscreetlyWebServer serves a single purpose: to allow two people to share little bits of information - secrets - such as passwords, etc., in a secure, discreet manner.  The secrets are protected both in transit and at rest, using the FIPS-approved AES-256 algorithm with asymmetric keys supplied by an X.509 certificate.

Oh, and I made sure that it's thoroughly compatible with Powershell so that the server can be used in a scriptable/automatable way.

You can read a more thorough description of the server as you try it out here.

Please let me know if you find any bugs, exploits, or if you have any feature requests!

Powershell: Get Content Faster with ReadCount!

Do you use Powershell?  Do you use Get-Content in Powershell to read files?  Do you sometimes work with large text files?

If you answered yes to any of the questions above, then read on - this post is for you!

I have a very simple tip that I used today in a script I was writing.  Thought I'd share.

Let's say you have a large text file, such as a packet log from a DNS server that you're debugging.  It might be 300 megabytes and millions of lines.  I was writing a script to parse the file and collect some statistics that I was after.

#
$LogFile = Get-Content $FileName
ForEach($_ In $LogFile)
{
    Do-Stuff
}

When I ran this script against a 52MB text file, the script executed in about 22 seconds.  When I ran the script on a 150MB text file, Powershell proceeded to consume over 3GB of RAM within a few seconds, the script never finished, and after bringing my laptop (Win7 x64, 4GB RAM, 4CPU, PS v3, .NET 4.5) to a crawl for about 5 minutes, Powershell just gave up and returned to the prompt without outputting anything.  I guess it was some sort of memory leak.  But come on... a 150MB file is not even that big...

So I started looking through the help for Get-Content, and it turns out there's an easy workaround:

#
$LogFile = Get-Content $FileName -ReadCount 0
ForEach($_ In $LogFile)
{
    Do-Stuff
}

The -ReadCount parameter specifies how many lines of content are sent through the pipeline at a time. The default is 1. A value of 0 sends all of the content through at one time.

Now when I run the script against the 52MB file, it completes in 2.8 seconds, and when I run it on the 150MB text file, it finishes in 10.2 seconds!