Merge mailbox folders using EWS (with throttling support)

A PowerShell script that uses EWS (requires the EWS Managed API) to merge folders within a mailbox.  Can be used for multiple folders, and automated against multiple mailboxes.  The script handles throttling and so works against large Office 365 mailboxes.

 
 
 
 
 
4.4 Star
(7)
5,489 times
Add to favorites
Exchange
1/13/2020
E-mail Twitter del.icio.us Digg Facebook
Sign in to ask a question


  • Error When Copying Email - ErrorToFolderNotFound
    2 Posts | Last post Thu 8:32 AM
    • Hello.  Great script hoping to get it working.  I am having an issue when using the -ManageFolderList option and copying email.
      Command:
      .\Merge-MailboxFolder.ps1 -MergeFolderList $merge -SourceMailbox testsource@domain.com -TargetMailbox testtarget@domain.com -Impersonate -Copy -ProcessSubfolders -CreateTargetFolder -Office365 -Credentials (Get-Credential) -LogFile LogFile.txt -TraceFile Tracing.log
      
      Here is my Merge List:
      $merge = @{"\CopiedEMail" = "\Inbox"}
      
      I have tried without the slashes as well, no luck.
      The process finds the emails to copy without issue, but then is unable to copy them because it cannot find the target folder.  It looks like the EWSFolderId may be wrong.  This is the error I receive:
      [MoveItems]Copying from testsource@domain.com:\Top of Information Store\Inbox to testtarget@domain.com:\Top of Information Store\CopiedEmail
      1/10/2020 5:52:35 PM   1 items found; attempting to copy
      1/10/2020 5:52:36 PM   Permanent error ErrorToFolderNotFound reported for item: AAMkADMzMDA2NTkzLWE0MDUtNDc0MC1iYTYwLWUzMzczZGM1MTc1MQBGAAAAAABY6zOT+AZcQorLBJBLeQoyBwAL/8FzoeW6RKudvX9ZYLBzAAAAAAEMAAAL/8FzoeW6RKudvX9ZYLBzAAEKa42cAAA=
      
      This is the error in the trace:
            <m:CopyItemResponse xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types">
              <m:ResponseMessages>
                <m:CopyItemResponseMessage ResponseClass="Error">
                  <m:MessageText>The specified target folder could not be found.</m:MessageText>
                  <m:ResponseCode>ErrorToFolderNotFound</m:ResponseCode>
                  <m:DescriptiveLinkKey>0</m:DescriptiveLinkKey>
      
      Any help is greatly appreciated.
    • Apparently this is a permissions issue:
      
      "When you use impersonation, the account you are using loses all of its own permissions and takes on the permission set of the account normally assigned to the mailbox.  When  you are copying items from one mailbox to another, the source mailbox's regular account NEEDS to have access permissions to the target mailbox, because it is this permission set that will be given to the impersonation account."
      
      I granted the appropriate permissions and the mail copied over.
  • Automatically create foldername based on shared mailbox name where emails are being copied
    1 Posts | Last post December 31, 2019
    • Hi,
      
      I have large task to get rid of our customers shared mailbox archives. To do that I have to copy or move all emails from In-Place Archive to each shared mailbox inbox. However I need to place everything under folder and name the folder based on the shared mailbox name.
      
      For example:
      
      I have shared mailbox named Finance.group@contoso.com. This mailbox has in-place archive in Exchange online and in that Archive there are some folders that has emails. I copy everything under Finance.group@contoso.com primary mailbox inbox to a new folder and named it like this : ARCHIVE-Finance.Group ("ARCHIVE-" first and then shared mailbox display name)
      
      So basically I need a way to automatically name the folder based on each shared mailbox name, where all the emails are being copied to from archive mailbox. Is it possible with this script to take the name of the folder from .txt file? 
      
      I noticed you have created script for larger batches: https://developermessaging.azurewebsites.net/2016/09/23/powershell-batch-movingmerging-mailboxes/ . This worked like a charm when tested with 2 mailboxes :) 
      
      Hopefully I explained myself properly.
      
      BR,
      Juha
      
  • Used for migrating to another tenant?
    2 Posts | Last post November 19, 2019
    • Hi, 
      
      Could this script be used to migrate user from tenantA to tenantB? 
      
      BR, 
       Martin
    • The script uses the same credentials to access all mailboxes, and this is part of the design currently so can't easily be changed.  Given this, I don't think it would be possible to migrate a mailbox from one tenant to another.  I will look into this to see if there is a way to do it, but it may not be possible without rewriting completely.
  • Use existing PSSession to Exchange? Or help?
    5 Posts | Last post October 29, 2019
    • Hello and thanks for this script.  I've known of it since quite a while ago when I found it, but am now finally ready to use it.  I'm having trouble getting authenticated, and I've tried with Impersonation and FullAccess, both fail.  Meanwhile, I'm actually already connected to Exchange Online with the admin account I plan on using against my test user mailbox (and who has the Impersonation role assigned / has FullAccess to the test user mailbox).  Would it be possible to just re-use the existing PSSession?
      
      If the existing PSSession is not possible, could you help me to figure this out?  My error when using -OAuth is:
      
      "New-Object : Cannot find type [Microsoft.IdentityModel.Clients.ActiveDirectory.PlatformParameters]: verify that the assembly containing this type is loaded."
      
      Note**: I do have the DLL Microsoft.IdentityModel.Clients.ActiveDirectory.dll in the same folder as Merge-MailboxFolder.ps1.
      
      My error when specifying -Credential (but not -OAuth) is:
      
      "Error (ThrottledFolderBind): Exception calling "Bind" with "3" argument(s): "The request failed. The remote server returned an error: (401) Unauthorized."                                                                                                        Failed to open source message store (<emailaddress>)"
    • I should add, I am able to get it working if I specify -Credential and use the user's credentials.  So it seems my issue is with using an account that has FullAccess and/or App. Impersonation role assigned.
    • This script doesn't use PowerShell at all.  You could potentially use the same credentials as supplied to a PowerShell session, but they'd need to be assigned to a variable that you could pass to both.
      
      For impersonation to work, your authenticating account must have ApplicationImpersonation rights.  Same for FullAccess.
      
      The ADAL error implies that your version of the dll is outdated - I test with the latest version only, and the most recent tests have been with version 5.2.0.  The version of the loaded dll will be shown in the log.
    • Thanks for pointing out the outdated DLL.  I'm not sure I follow re: "this script doesn't use PowerShell at all".
      
      I did state that the role and FullAccess were both in place, so again, not following.  Although I do retract my need for support - I understand it's not an official tool.  It's just not working as designed.  I don't know how else I could have been more clear, so admittedly this is slightly frustrating.  I can only imagine how frustrated you are and am assuming that is playing into your answer.  Take care, thanks for the script and response.
    • Sorry, I could have been clearer - what I meant is that this script doesn't use Exchange PowerShell at all - meaning it doesn't connect to any remote runspace (which is how Exchange PowerShell works).  Exchange PowerShell is used for administering mailboxes and the organisation, while EWS (which is what this script uses) is purely for accessing mailboxes.
      
      I'll try to provide some more examples for using the script.
  • Failed to resume operations after throttling reset
    2 Posts | Last post October 25, 2019
    • Hi David,
      I have a 3GB mailbox which I am coping to another mailbox. The script went well for initial few folders and thereafter it's started failing for folders which were above approx. 100-150 MB. For all bigger folders the script takes off well, however, after throttle reset it's unable to resume the item copy from a point where it had left and picked up the next folder for processing. I ran the following command:
      
      .\Merge-MailboxFolder.ps1 -SourceMailbox "mailbox1" -TargetMailbox "mailbox2" -Copy -ProcessSubfolders  -CreateTargetFolder -EwsUrl "https://outlook.office365.com/EWS/Exchange.asmx" -Credentials (Get-Credential) -LogFile c:\temp\log.txt
      
      Sharing the logs that I was getting after throttling reset :
      8/14/2019 7:35:20 PM   Item successfully processed: AAMkADk4NTIyMzUyLTc2OTQtNGIwZS04NDcxLTJkZmMyZWRlYzc2NgBGAAAAAACXRqzU2k0QTa4J04dJKZsrBwB5U0YWS5nVQpUBvm7AuSSIAAAAOsMPAAAav5M1pI/BQZSSnJjMckBbAATPSeAfAAA=
      8/14/2019 7:35:20 PM   Sending batch request to copy 50 items (203 remaining)
      8/14/2019 7:35:25 PM   Increased throttling delay to 200ms
      8/14/2019 7:35:25 PM   Throttling detected, server requested back off for 287343 milliseconds
      8/14/2019 7:40:12 PM   Throttling budget should now be reset, resuming operations
      8/14/2019 7:40:12 PM   Restoring throttling delay to 100ms
      8/14/2019 7:40:12 PM   [ThrottledFolderBind]Attempting to bind to folder AAMkADk4NTIyMzUyLTc2OTQtNGIwZS04NDcxLTJkZmMyZWRlYzc2NgAuAAAAAACXRqzU2k0QTa4J04dJKZsrAQB5U0YWS5nVQpUBvm7AuSSIAAAAOsMPAAA=
      8/14/2019 7:40:12 PM   [ThrottledFolderBind]Successfully bound to folder AAMkADk4NTIyMzUyLTc2OTQtNGIwZS04NDcxLTJkZmMyZWRlYzc2NgAuAAAAAACXRqzU2k0QTa4J04dJKZsrAQB5U0YWS5nVQpUBvm7AuSSIAAAAOsMPAAA=
      8/14/2019 7:40:12 PM   [MoveItems]No subfolders found: pamela.mata@domain.com:\Principio del almacén de información\Elementos enviados 
      
    • I'm not sure what the issue could be here, but I have just released an update that fixes the OAuth token issue (when using OAuth, the script could only run until the token expired).  This required quite a few updates to the move logic, and I've tested fairly extensively moving items from a primary mailbox to an archive (the primary mailbox was particularly large in this case so that I could investigate any issues that occur during long runs).
  • MergeFolderList and Combine vs Process subfolders
    3 Posts | Last post October 22, 2019
    • I'm sorry to bombard you with 3 questions all at once.  I have many subfolders under \zzTop\001, so I want to combine all of those subfolders into \zzTop\001.  I simply can't get this to work, nor can I get it to work if I try to use a different target folder.  Here's my log for when the target is different:
      
      Target folder is \zzTop\T001
      Source folder list is \zzTop\001
      [GetFolder]Locating folder: \zzTop\T001
      [ThrottledFolderBind]Attempting to bind to folder AAMkADc<shrank>vAAA=
      [ThrottledFolderBind]Successfully bound to folder AAMkADc<shrank>vAAA=
      [GetFolder]Folder T001 doesn't exist in path \zzTop\T001
      
      Here's my log for when the target is the same:
      
      Target folder is zzTop\001
      Source folder list is \zzTop\001
      [GetFolder]Locating folder: zzTop\001
      [ThrottledFolderBind]Attempting to bind to folder AAMkADc<shrank>vAAA=
      [ThrottledFolderBind]Successfully bound to folder AAMkADc<shrank>vAAA=
      [ThrottledFolderBind]Attempting to bind to folder AAMkADc<shrank>dAAA=
      [ThrottledFolderBind]Successfully bound to folder AAMkADc<shrank>dAAA=
      Source folder located: 001
      [MoveItems]Cannot move or copy from/to the same folder (source folder Id and target folder Id are the same)
      
      I'm unable to figure out when or if CombineSubFolders and ProcessSubfolders need to both be specified or just one or the other.  I want to combine the subfolders (which involves processing them).
      
      Thanks in advance.
    • I'm pretty sure that CombineSubFolder just doesn't work.  I'd have to see it work once to believe that it does.  I'm able to use just -ProcessSubFolders, then all target folder doesn't exist/can't be the same woes go away, but it just moves my subfolders.  If I try to use CombineSubFolders, it just loops through all the subfolders and throws:
      [MoveItems]Target folder is null, cannot move items
      
      If I run into that, then just change the command by dropping -CombineSubfolders, it will work (although all my subfolders don't get collapsed (which is the sole goal)).
    • -CombineSubFolders does work, but the issue you have is that your source and target folders are the same - and the script blocks this (as there is nothing to do - it can't move items from a folder to the same folder).
      
      In this case, you'd be best moving all the items to a new folder, and then moving/renaming that new folder to where you'd like it to be.
  • On-prem mailbox with O365 archive
    1 Posts | Last post September 27, 2019
    • I am using the script and it works perfectly for on-prem mailbox and archive but I am having problems to get it to work when the archive is in O365.
      
      I am using
      
      .\merge-mailboxfolder.ps1 -sourcemailbox xxx@xx.com -targetarchive -impersonate -processsubfolders -createtargetfolder -onlyitemssentreceivedbefore "01/01/2016" -office365 -credentials
      
      What am I missing?
      
      
  • Is this within a nubes capability
    2 Posts | Last post September 04, 2019
    • Hi Folks,
      
      I'm a recent admin to 365 and I have the online archive issue on three inboxes which has brought me here. I'm able to follow the basic PowerShell commands but I get lost with this EWS API and application impersonation. I think I've reached my technical limits. Is this something I can resolve if I persevere? has anyone got links to a basic walkthrough? or should I be contacting a professional and if so what is the safest access profile to give them to help get this issue solved for me. Many thanks!  
    • If you only have an issue on three mailboxes, then it may be easier just to ask the mailbox owners to run the script (or run it for them).  The EWS API simply needs to be downloaded and installed (an older version of the API can be downloaded from https://www.microsoft.com/en-us/download/details.aspx?id=42951).  If the mailbox owner runs the script, then you can use the syntax given in the example for recovering an archive into the main mailbox (assuming that is what you want to do).
      
      Though if you do have any doubts, it would be worth consulting someone who understands Exchange and its functionality.  This script can potentially mess up a mailbox if used incorrectly.
  • Specify target folder
    3 Posts | Last post August 23, 2019
    • Is it possible to specify a target folder in the target mailbox where the source items should be copied to/moved by the script or can it be requested as a new feature in the code?
      
      Say that you´ve SourceMailbox\FolderStructure and instead of placing it in TargetMailbox\FolderStructure as a 1:1 structure, then the target destination could be TargetMailbox\MovedData\FolderStructure where "MovedData" would be specified through a parameter variable and "FolderStructure" representing SourceMailbox hierarchy folder/subfolder structure  
      
    • You can do this simply by specifying the target folder in MergeFolderList.  The target folder is the root where the source folder will be copied to, and it can be anywhere in the mailbox.
      
      e.g.
      $mergeFolderList = @{"\Inbox\Moved" = "\Inbox" }
      
      will copy everything from the Inbox of the source mailbox to the folder \Inbox\Moved of the target mailbox.  Hierarchy, etc. will be preserved so long as the parameters are set (in this case, you'd also need to specify -ProcessSubfolders).  You can also move folders around within the same mailbox this way (though this example wouldn't work in that case as you can't copy a folder to a subfolder of itself).
    • David Barret,
      
      Thank you for your answer; I tried the function MergeFolderList and it worked like a charm!!!
      This script is the best thing since sliced bread... :)
  • Script hangs at # items found; attempting to copy
    2 Posts | Last post August 07, 2019
    • I am using this script but cannot get it to work. It starts and runs but the scripts seems to stall at # items found; attempting to copy. I will run for a long time but nothing seems to happen. 
      
      We have a lot of mailboxes running in Office365 we would like to merge. 
      
      Beneath the command we run
      
      .\Merge-MailboxFolder.ps1 -SourceMailbox "test@domain.com" -targetMailbox "test2@domain.com" -Copy -ProcessSubfolders -CreateTargetFolder -Office365 -Credential $credential -impersonate -logfile C:\Scripts\log.txt -tracefile C:\Scripts\trace.txt
      
      In the log file the same line keeps appearing:
      17/07/2019 15:33:58   Sending batch request to copy 3 items (3 remaining)
      
      Any suggestions?
    • I've improved the retry/error testing logic in the upcoming release, but I suspect that the main issue here is that to copy from one mailbox to another, the source mailbox needs FullAccess rights to the target mailbox.
      
      You are accessing the source mailbox using Impersonation, which means you are acting as that user.  For that user to be able to copy a message to the target mailbox, they need permission to access that mailbox - ApplicationImpersonation on the service account is not enough here, as the copy is initiated from the source mailbox in the context of the source user.  Hopefully that makes sense.
1 - 10 of 29 Items