Powershell Script to Calculate and compare filehashes for DFSR Pre-Staging

Script transverses the source directory and subdirectorys and selects random # of files. Script then uses Dfsrdiag filehash to gather the file hashes. Comparison is done and output is to console in CSV. Comparison returns True for match and False for nonmatch.

 
 
 
 
 
5 Star
(3)
Add to favorites
Storage
5/20/2011
E-mail Twitter del.icio.us Digg Facebook
  • This seems broke.
    2 Posts | Last post November 01, 2016
    • When I copied this script last week and tested it out it worked.  Now it gets hung after the inputs are provided.  Thought it might be a format issue with the input given so I have tried enclosing the source and destination paths in quotes but then I get an error that the source location does not exist or something.  When I simple enter something like this:  d:\ftr archive  and  \\servername\ftr archive  for the paths everything seems to get hung up when it should be trying to resolve the -recurse on the directory structure of d:\ftr archive.  
      
      $files = $source |dir -recurse | get-random -count $randomfiles 
      
      I am not PS guru or a DFS guru so when this tested good for me last week it highlighted some issue with my dfs pre-seeding that I had previously performed. I have spent the last days trying to figure out what the issue is with this script as I was hoping to see which of my new dfs servers I need to redo.  
      
      What is the correct format to input the source and destination/target directories. 
      Do you have to perform this on the namespace servers?  I would have sworn I did it from my Windows 7 Admin desktop last week, but I might have ran it form one of the dfs servers.  
      
      I need to validate my data so I can resolve my initialization and staging issues. I am dealing with 8 TB of data that I thought I properly pre-seeded.  Need to migrate before the old server completely craps out from heat damage. Old server is 2008SP2 and new are 2012R2. 
    • If you have spaces in your paths, then you need to put quotes around them if you're not actually populating the $source variable with your path:
      
      $files = "d:\ftr archive" |dir -recurse | get-random -count $randomfiles
      
      (Also, spaces in share names = ::sadface:: - I think the quotes should work there too)
  • Better way of matching the hash
    1 Posts | Last post November 01, 2016
    • While I'm sure not many people are using this method any more, I'm intending to test it against Get-DFSRFileHash when you're running it over a million files. It'll be interesting to see if there is any difference in execution time over a reasonable sample (not the entire million).
      
      Instead of splitting, matching, splitting the filehash, it's cleaner to use a regex:
      
      $test = (([regex]'([A-f0-9]{8}-){3}[A-f0-9]{8}$?').Match($sourcehash),'multiline').Value
      
      Assuming it's always 4x8 hex characters separated by three dashes, the above will get the hash value straight out of the dfsrdiag filehash output.
  • Support for paths greater than 260?
    1 Posts | Last post August 15, 2015
    • Hi, been using this script for awhile, and it's been great and goes a long way to keeping my blood pressure down when working with/moving DFS folders :)
      
      This last time I ran into some red text with paths longer than 260. The script still seems to work, I don't know that I've actually 'hit' on a file in the get-random that was longer than 260 though.
      
      I tried using \\?\UNC\server\share\folder in the source and target but it didn't seem to work. I also tried using the Microsoft.experimental.io dll but my PowerShell Fu isn't that strong yet. :)
      
      Have you come across any situations like this, with longer than 260 character filenames/248 character folders and which way do you think would be better to further attempt to handle long paths? the \\?\ or the dll?
      
  • Taking Long Time
    1 Posts | Last post July 07, 2014
    • Hi,
      I Tied your script.. one with local and another one is remote site...
      it's more time to execute the output files.. 
      Can you guide me whats wrong on this..
  • the script also also works using different OS and remotely
    1 Posts | Last post July 13, 2011
    • Hi,
      first - thanks for your script, it makes things a bit more easy.
      
      second, I tried to hash local and remote files via UNC-path - it works without any issues.
      - two w2k8r2 servers, one local and one in a remote office.
      - 1st parameter: local path
      - 2nd parameter: UNC path
      - works fine.
      
      
      third, I tried to hash local and remote files using different OS - this works without any issues too. 
      - one local w2k8r2 server (designated future hub), one remote w2k3r2 server
      - 1st parameter: local path on w2k8r2 server
      - 2nd parameter: UNC path of w2k3r2 server (using hidden share, e.g. F$)
      - works fine.
      
      
      I also made some small modification in your code to make the string comparison work correctly with a server running German language.
      
      line 50, from
        $test = $test -split "operation succeeded",0
      to
        $test = $test -split "Vorgang erfolgreich",0
      
      and
      
      line 61, from
        $test2 = $test2 -split "operation succeeded",0
      to
        $test2 = $test2 -split "Vorgang erfolgreich",0
      
      
      
      
  • This is very poor powershell indeed.
    3 Posts | Last post February 26, 2011
    • Here is a re-write (I don't claim it is perfect)
      param(
          $source =      (read-host "Where is the Source Directory?") ,
          $staged =      (read-host "Where is the Destination Directory?"), 
          $randomfiles = (read-host "How many random sample files do you want to compare?") ,
          $output      = (read-host "Please provide a file name and location for the results. Results are defaulted to CSV format.  (Ex: c:\filehash.csv)") 
      ) 
      # Allow parameters to be passed and only prompt the user if they are not. 
      # Created CSV header
      Set-Content -Path  $output -Value "FileName,Source Hash,Destination Hash,Match Success" 
      DIR -Path $source -recurse | get-random -count $randomfiles  | ForEach-Object {
          $sourcefilename = $_.fullname
          $targetfilename = $_.fullname -replace $source, $staged
          $sourcehash     = dfsrdiag filehash "/filepath:$sourceFileName"
          $sourceHash     = ($sourceHash -replace "File Hash: ","") -replace "operation succeeded","" 
          if (Test-Path -Path $targetFileName) {
              $Targethash = dfsrdiag filehash "/filepath:$TargetFileName"
              $TargetHash = ($TargetHash -replace "File Hash: ","") -replace "operation succeeded","" 
          }
          Else {  $TargetHash = "file Not found" }
          Add-Content -Path -Value "$($_.name),$SourceHash,$targethash,$($sourceHash -eq $TargetHash)" 
      }       
      write-host "Script completed.  File is located at $output."
    • Darn. Fingers going faster than brain. the conversion from source to target doesn't allow for the user including the trailing \ in one but not the other - I decided to keep things simple but then edited out the [regex]::escape which should have stayed: the line should be
      $targetfilename = $_.fullname -replace [regex]::escape($source), $staged
      
      Things I have tried to fix. 
      [1] echo "FileName,Source Hash,Destination Hash,Match Success" | out-file -filepath $output
      and
          echo "$file,$test,$test2,$compare" | out-file -filepath $output -append
      
      Echo is there to help people copy .bat file code.
      Echo <whatever> | out-File should be replaced with set-content 
      Echo <whatever> | out-File -append should be replaced with Add-content 
      
      
      [2] $files = $source |dir -recurse | get-random -count $randomfiles
          ... 
          foreach ($file in $files)
      
      It is better: to write $Source | dir  
      as 
       dir $Source
      
      and to write: $files = dir <etc>   ; foreach ($file in $files) 
      as 
       dir <etc> | foreach
      
      [3] $fullnamereg = "FullName : "
          $regex = [regex]$fullnamereg
          $sourcefilename = $file.fullname -split $regex,2;
      
      $file.fullname will never contain "FullName :" - you only see that if you something like file | fl * all that is needed is
           $sourcefilename = $file.fullname
      
      [4] $test = $sourcehash -split "File Hash: ",2 
          $test = $test -split "operation succeeded",0
          $test =  $test -match "\w"
      
      Can be simplified to replace the leading and training text with nothing and the match "\w" should be redundant 
      
      [5] if ($test -match $test2){$compare = "TRUE"}else {$compare = "False"};
      is better written as 
         $compate = ($test -eq $test2)
      note that -match is for regular expression comparisons, not testing equality. 
      
    • Thank you for your feedback.  I will look at your changes and validate they work.  Some of the code you are talking about I wrote this way because of unexpected outputs based on test cases we found.