Invoke-Async - Allows you to run any cmdlet/function/scriptblock asynchronously

***NOTE Rev 4 10/21/2011***Added a progress bar***NOTE Rev 3 11/30/2012***added write-error to show errors with cmdlet callsadded Measure param to give timing infoadded ability to use alias***NOTE Rev2*** Script has been updated to handle a single item and also closes out the run

Invoke-Async.ps1
 
 
 
 
 
4.2 Star
(16)
10,852 times
Add to favorites
10/21/2013
E-mail Twitter del.icio.us Digg Facebook
  • Parallel File Copy
    1 Posts | Last post August 27, 2019
    • I have the following:
      
      <#
      .SYNOPSIS
      <Brief description>
      For examples type:
      Get-Help .\<filename>.ps1 -examples
      .DESCRIPTION
      Copys files from one path to another
      .PARAMETER FileList
      e.g. C:\path\to\list\of\files\to\copy.txt
      .PARAMETER NumCopyThreads
      default is 8 (but can be 100 if you want to stress the machine to maximum!)
      .PARAMETER LogName
      default is output.csv located in the same path as the Filelist
      .EXAMPLE
      .\CopyFilesToBackup -filelist C:\path\to\list\of\files\to\copy.txt
      .NOTES
      #>
      
      [CmdletBinding()] 
      Param( 
          [String] $FileList = "C:\temp\copytest.csv", 
          [int] $NumCopyThreads = 8,
          [String] $LogName
      ) 
      
      $filesToCopy = New-Object "System.Collections.Generic.List[fileToCopy]"
      $csv = Import-Csv $FileList
      
      foreach($item in $csv)
      {
          $file = New-Object fileToCopy
          $file.SrcFileName = $item.SrcFileName
          $file.DestFileName = $item.DestFileName
          $filesToCopy.add($file)
      }
      
      $sb = [scriptblock] {
          param($file)
          # $FileToCopy = New-Object filetoCopy
          Copy-item -Path $file.SrcFileName -Destination $file.DestFileName
      }
      # $results = Invoke-Async -Cmdlet Copy-Item -Set $filesToCopy -ScriptBlock $sb -Verbose -Measure:$true -ThreadCount 8
      $results = Invoke-Async -Set $filesToCopy -SetParam file -ScriptBlock $sb -Verbose -Measure:$true -ThreadCount 8
      $results | Format-Table
      
      Class fileToCopy {
          [String]$SrcFileName = ""
          [String]$DestFileName = ""
      }
      
      the csv input for which looks like this:
      
      SrcFileName,DestFileName
      C:\Temp\dummy-data\101438\101438-0154723869.zip,\\backupserver\Project Archives\101438\0154723869.zip
      C:\Temp\dummy-data\101438\101438-0165498273.xlsx,\\backupserver\Project Archives\101438\0165498273.xlsx
      
      What am I missing to get this working, because when I run .\CopyFiles.ps1 -FileList C:\Temp\test.csv nothing happens. The files exist in the source path, but the parameters aren't being pulled from the -Set collection
  • Can this function support multiple parameters
    1 Posts | Last post July 04, 2018
    • In some case we need pass multiple parameters for the script block, does this module support this scenario? thanks.
  • Unable to get the result in a variable.
    1 Posts | Last post April 05, 2017
    • I'm unable to get the result in a variable for all the servers. I have set the TreadCount to 25. I have around 2000 servers. But I get the result only for 25 servers. I'm assuming, i get the output for only a single thread. what am i doing wrong?
      
      $servers = 'Srver1', 'Srver2', 'Srver3', 'Srver4', 'Srver5', 'Srver6', 'Srver7', 'Srver8'
      $sb = [scriptblock] {param($system) Invoke-SQLCmd -ServerInstance $system\pos -query 'select @@servername, name from sys.databases'  } 
      $rtn = Invoke-Async -Set $servers -SetParam system  -ScriptBlock $sb -ThreadCount 25 
      $rtn
  • Bug Report: Invoke-Async Invokes n-1 threads
    1 Posts | Last post November 18, 2016
    • Hi, in the "Process" section the line
      for($idx = 0; $idx -lt ($ThreadCount-1) ; $idx++)
      should be
      for($idx = 0; $idx -lt $ThreadCount ; $idx++)
      
      Example: Assume we have 3 objects to Invoke and a ThreadCount=10
      The original code will run 2, then wait for first to complete and finally run the 3rd one. It should Invoke all three in the first iteration.
      
      Like this script!
  • Async call of functions?
    2 Posts | Last post December 21, 2015
    • Hi, is it possible to invoke a function several times aynchronously with different parameters?
    • The cmdlet param can take either a compiled cmdlet or a function. take a look at the example with Test-Connection
  • scriptblock not returning anything
    4 Posts | Last post October 02, 2015
    • Hello, any idea why this is not returning anything? I want it to return folder permissions for all the paths listed in tethpaths.txt. When I type the command directly it works fine. It appears to be running and I'm not getting any errors, it just doesn't output anything.  Thanks in advance!
      
      $sb = [scriptblock] {param($folder) import-module ntfssecurity get-childitem $folder -recurse |where {$_.PsIsContainer} |get-access select Fullname,Account,AccessRights}
      $paths = get-content .\testpaths.txt
      $rtn = invoke-async -set $paths -setparam folder -scriptblock $sb -threadcount 20
      $rtn |sort FullName 
      
    • sorry there should be another pipe character between get-access and select. It won't let me edit what I posted :(
    • hmm, I have a feeling it's related to the ntfssecurity module. 
    • well I fixed it! i just needed a semicolon between import-module ntfssecurity and get-childitem.
      this works:
      $sb = [scriptblock] {param($folder) import-module ntfssecurity; get-childitem $folder -recurse |where {$_.PsIsContainer} |get-access |select Fullname,Account,AccessRights}
      $paths = get-content .\testpaths.txt
      $rtn = invoke-async -set $paths -setparam folder -scriptblock $sb -threadcount 20
      $rtn |sort FullName 
  • Last Thread is serialized
    1 Posts | Last post January 13, 2015
    • I tried to run Invoke-Async, noticed that the block runs for the last server in the array after the rest have finished... 
      Why is that so? 
  • Nested functions
    3 Posts | Last post December 09, 2014
    • Nice code - thanks for putting it together.  I was able to successfully test your function having it call another simple function.  However, in my real-world task I need to complete, I have a fairly complex function that in turn calls other functions.  When I use your function to call my "master" function against several hundred server names, it basically does nothing (I see it spin through the Write-Progress, but nothing gets done).
      
      Is there a trick to use to get it to work with functions that call other functions?  All my functions are in one script, including yours.
      
      Thanks
      NK
    • OK - I figured it out.  I did see Jana's comments but I thought that did not apply to me because I was not using a script block and my functions were already in the same PS1 as your async function...so I was confused.
      
      But I ended up dot sourcing the script file to itself (which is really strange and makes me even more confused) but now it sees all my functions.
      
      Unfortunately, my script is not much faster now using your Async function.  So I think I am going to stick with my original design which used Start-Job calls.  I use a custom homegrown throttling technique that I found on the web and it works pretty good.
    • The thing to keep in mind, and how you solved it, is that each job is a powershell runspace (basically its own instance of PowerShell.exe)
      so when you dot sourced the script it imported all your functions in to those sessions.
      
      Speed wise its only a little faster than start-job for its load time so where it really becomes useful is for simple functions that do a small amount of work that needs to be done over hundreds of systems. In that case the small amount of time you save adds up really quickly
  • Not a question but a tip when using this
    2 Posts | Last post September 23, 2014
    • This is great function. Thank for creating it and sharing.
      
      When I tested, it worked great on simple stuff (like get-process and such) but it would not work on my own functions. It took me a bit to find out that the "context" of execution is different and hence referenced functions have to be "dot sourced" inside the script block (even if they have already been dot-sourced for the session).
      
      I thought I would put this note here for my own future reference and for others who have issues running custom stuff. Just dot-source again inside the script block!
      
      Thanks again for this amazingly compact yet powerful function.
    • Yes Jana, you are right. Because it uses Runspaces to run the jobs it has no knowledge of your current session.
  • Beginner Usage
    2 Posts | Last post March 30, 2013
    • Just learning powershell, how do I set up to start using this script?
    • Just import the file in to your session either by dot sourcing . .\invoke-async.ps1 or by import-module .\invoke-async.ps1
1 - 10 of 11 Items