You can find the full documentation for this module in the following blog posts:

http://aka.ms/SIDHistory

 

 

Introduction

This post is the fifth in the "SID Walker, Texas Ranger" series on SID history remediation with PowerShell.  Today we're wrapping up with a handy summary of each post in the series. We will also take the function library we've been using and upgrade it to a PowerShell module. Then we'll walk through the entire SID history remediation process using the provided cmdlets in this module.

The Story So Far

Those of you who follow my blog know that I have been stuck on this theme of SID history for several months now.  Why?  Because I see this quite frequently with customers, and I want to offer some practical guidance on dealing with it.  Here is a summary of the blog series that brought us to today's module:

  1. Using PowerShell to resolve Token Size issues caused by SID history
    Prior to starting the module development this post explained the background of token size issues as related to SID history.  I provided the basic SID history query that we use to produce the report and some great links for more information on token size.
  2. Do Over: SID History One-Liner
    As a follow up to the Token Size post I re-wrote the SID history report query as a one-liner.
  3. PowerShell: SID Walker, Texas Ranger (Part 1)
    This time we looked at Get-ACL and parsing SDDL strings, a warm up for the next post.
  4. PowerShell: SID Walker, Texas Ranger (Part 2)
    Next I wrote a function to swap SID history entries in ACLs/ACEs.  This compensates for a gap in the ADMT, because it cannot migrate SID history for file shares hosted on a NAS.
  5. PowerShell: SID Walker, Texas Ranger (Part 3): Exporting Domain SIDs and Trusts
    Looking at raw SIDs in a report is not very friendly, so I wrote a function that translates domain SIDs into domain names.  This makes the SID history report more meaningful when you can see the name of the domain from whence they came.  Enumerating all forest trusts and their domain SIDs required using some .NET ninja skills.
  6. How To Remove SID History With PowerShell
    To round out the functions I provided Get-SIDHistory and Remove-SIDHistory, emphasizing that this is the LAST step in the process.  I leveraged the previous domain SID function to even give us the ability to remove SID history selectively by old domain name.

I suggest that you go back and read all of the articles linked above. They will give you much more insight into the SID history cleanup process and the nuances of the provided functions. Then skim through the ADMT Guide to get familiar with the big picture.

All of these functions are now wrapped up in the module provided in today's blog post.

Installing the Module

If you've never installed a module there really isn't much to it.  Here's what you do:

  1. Create the module folder (adjust Documents path if necessary):
    New-Item -Type Directory -path "$home\Documents\WindowsPowerShell\Modules\SIDHistory"
  2. Download the attached ZIP file at the bottom of this article.
  3. Unzip the contents into this path:
    C:\Users\<username>\Documents\WindowsPowerShell\Modules\SIDHistory\
  4. Fire up the PowerShell console or ISE.
  5. Import-Module ActiveDirectory (This is a prerequisite.)
  6. Import-Module SIDHistory

Now you can use Get-Command and Get-Help to unwrap the present and see what's inside:

You can use Get-Help -Full for each of the included functions to find syntax and descriptions.

Using the Module

The outline below will guide you through the process of using the functions to help remediate SID history.  Run them in this order.

The functions provided in this module will give you added visibility into the status of your SID history throughout the process and an easy way to target removal in the final phase.

Conclusion

This SID history project has been a lot of fun, and I'm sure there's more we could do with it.  I have a few ideas of my own, but I would like to hear your feedback.  What challenges have you encountered with SID history remediation?  Where do you think PowerShell could help?  Leave a comment below and let me know.

Additional Reading

 

 

 

PowerShell
Edit|Remove
Import-Module ActiveDirectory 
Import-Module SIDHistory 
 


JUNE 15, 2012 UPDATE

More SID History

I get emails frequently from folks who have read my series of articles on Active Directory SID history, so I'm guessing that is a good theme to continue.  Working with a customer recently I was able to enhance the functionality in the Active Directory SID history PowerShell module that I posted last December.  Today's post will publish and discuss those improvements.

Version 1.4
June, 2012

Functions added in this release:
Export-SIDHistoryShare
Merge-CSV

Functions modified in this release:
Convert-SIDHistoryNTFS
Export-SIDMapping
Update-SIDMapping
Get-SIDHistory

Fixes:
Removed Test-Path validation on NewReport parameter of Update-SIDMapping.
Added file validation for DomainFile parameter of Get-SIDHistory.

It's All About The Customer

Imagine if…

Obviously I can't make this stuff up.  Those points describe a customer who recently invited me to help them develop a process to identify and remediate SID history in their forest.  This also gave them a process going forward that would help them do future domain migrations in a way that would minimize the impact of SID history.

Enter PowerShell

I knew the SID history PowerShell module could help them, but I also knew that some improvements would be necessary to help them scale to the size of data collection they would need.  Here's what I did:

These changes added the scalability and flexibility for the customer to begin inventorying hundreds for servers for SID history and manage all the data with a full database they would implement later on their own.  I love the simplicity of CSV in PowerShell!

The New Process

My last post on SID history has the overall steps for installing the module and completing the SID history remediation so I will not repeat that content here.  But I do want to list out the steps for you to automate the NTFS and share SID history data collection:

Automating the Automation

It would be quite time consuming to run the NTFS and share scan commands one at a time against all of your servers.  Instead try these handy PowerShell routines to make the data collection go faster.  As these processes usually take hours to scan large file shares it would be a good idea to let them run over a night or weekend.

# Sample code to scan a group of Windows servers in a text file.
$Servers = Get-Content ".\servers.txt"
ForEach ($ComputerName in $Servers) {
    Export-SIDHistoryShare $ComputerName
    # Grab only file shares that were manually created
    $Shares = Get-WmiObject Win32_Share -ComputerName $ComputerName -Filter "Type=0"
    ForEach ($Share in $Shares) {
        Convert-SIDHistoryNTFS "\\$ComputerName\$($Share.Name)" -WhatIf
    }
}

# Sample code to scan a list of share paths in a text file.
# This can work for Windows or NAS servers.
$Shares = Get-Content ".\shares.txt"
ForEach ($Share in $Shares) {
    Convert-SIDHistoryNTFS $Share -WhatIf
}

Where can I get all this PowerShell goodness?

I have updated this code at the TechNet Script Gallery.  Attached to that entry you'll find the following in a single compressed file:

Extract all of the files into a working folder.  Install the module.  The usual disclaimers apply: this is sample code for use at your own risk.  Enjoy!


JULY 9, 2013 UPDATE

To see all of the articles in this series visit http://aka.ms/SIDHistory.

I would like to thank everyone who has been using the Active Directory SIDHistory PowerShell module and sending me feedback.  Your input helps guide future releases like the one I am publishing today.

I’ve been sitting on some updates for a while, because I prefer to release code that has been field-tested.  I also wanted to time this release with the upcoming PowerShell Deep Dives book where I have a chapter discussing the origins of this module.  The last update was version 1.4 in June of 2012.  This is update 1.5 in July of 2013.

Summary of Changes

I am excited to announce the following key improvements in this release:

Note:  This module version is compatible with PowerShell v2 and any newer versions.  The next release will require PowerShell v3 as a minimum level.

Change Details

This release includes some significant changes and additions, which I have highlighted below. Here is a list of the functions in this release:

PS C:\> Get-Command -Module SIDHistory

CommandType Name                     ModuleName
----------- ----                     ----------
Function    Convert-SIDHistoryNTFS   SIDHistory
Function    Export-DomainSIDs        SIDHistory
Function    Export-SIDHistoryShare   SIDHistory
Function    Export-SIDMapping        SIDHistory
Function    Export-SIDMappingCustom  SIDHistory
Function    Get-ADObjectADSI         SIDHistory
Function    Get-DomainSIDWordCount   SIDHistory
Function    Get-SIDHistory           SIDHistory
Function    Get-SIDHistoryDuplicates SIDHistory
Function    Merge-CSV                SIDHistory
Function    Remove-SIDHistory        SIDHistory
Function    Search-SIDHistoryACL     SIDHistory
Function    Update-SIDMapping        SIDHistory
Function    Watch-Job                SIDHistory

Due to the large number of changes I am not going to include code samples for each function in this article.  Please use Get-Help -Full to see complete details and examples for each module member.

PS C:\> Get-Help Convert-SIDHistoryNTFS -Full

PS C:\> Get-Help Export-SIDMappingCustom -Full

PS C:\> Get-Help Get-DomainSIDWordCount -Full

Etc...

 

 

Functions Added

Export-SIDMappingCustom

Get-SIDHistoryDuplicates

Search-SIDHistoryACL

Watch-Job

Get-DomainSIDWordCount

Get-ADObjectADSI

 

Functions Modified

Export-SIDMapping

Update-SIDMapping

Convert-SIDHistoryNTFS

Export-SIDHistoryShare

 

 

Time-Saving Tip

When using Convert-SIDHistoryNTFS -WhatIf to scan servers and shares it can take hours or days depending on the size of the environment.  Sometimes you may only find a few SIDHistory entries that need to be remediated.  Rather than rescan the entire server or share to fix these ACLs you can target the SID history cleanup this way to save time:

PS C:\> Import-CSV ACL_SID_History.csv |
 Select-Object -Unique -ExpandProperty Folder |
 ForEach-Object {Convert-SIDHistoryNTFS $_}

In this example ACL_SID_History.csv is the NTFS permission scan output from Convert-SIDHistoryNTFS -WhatIf.  We simply get a unique list of folders and feed those into the conversion function.  Obviously this is much faster than rescanning every ACL of every folder in the share subtree.

Please use the comment area below to tell me what features you would like to see improved or added in the module.  I would also appreciate any feedback for how this has helped your projects.


September 2014 Update

Version 1.6.1

Better

 

In large environments I commonly ran into an issue with the “[“ character in file paths. PowerShell version 3.0 resolves this issue by using the –LiteralPath parameter instead of the –Path parameter for cmdlets working in the file system.

 

Stronger

 

Lately I’ve been learning about external help files for PowerShell modules. With this release I have moved the help content from comment-based help inside the module to an external XML help file. I also added an about topic with documentation and release notes. Let me know if you find any mistakes in the updated help.

 

Faster

 

Last year I worked with a customer to improve the speed of the most significant function Convert-SIDHistoryNTFS. This involved two changes:

 

I made a new baseline requirement of PowerShell version 3.0. This gives us the –Directory switch on Get-ChildItem, which saves a huge amount of time when you are scanning a file server with terabytes of files and folders. We only want the folder list anyway. This is a great example of “filter left” in PowerShell.

 

It turns out that using Out-File –Append for logging can be very expensive for performance and consumes a lot of memory. I converted this to use System.IO.FileStream for much more efficient use of resources and speed.

 

Result

 

Now the code is faster, uses less memory, and yields better results. The trade off is that now you must run the code from PowerShell version 3.0 or newer.  I don’t think that is a big deal, because we are almost to version 5.0 as of this writing. Time to level up.

 


But Wait!  There's More…

If you would like me or another Microsoft PFE to visit your company and help you with the ideas discussed on this blog, please contact your Premier Technical Account Manager (TAM).  We would love to come see you.

Follow me on Twitter: @GoateePFE.