Compress and Remove Log Files (IIS and others)

Archive Log Files: Manually specify any folder(s) or automatically parse IIS log folders, group by day/month and archive them with 7-Zip. Verify the archives and delete the original log files. Compressed archive will be about 4.5% (or less) of the size of the original log files.

 
 
 
 
 
4.8 Star
(27)
Add to favorites
Servers
1/19/2017
E-mail Twitter del.icio.us Digg Facebook
Sign in to ask a question


  • Check for create instead of modified date
    1 Posts | Last post December 13, 2016
    • Hi Bernie,
      The script is very useful for us but we would like to archive the script of yesterday aswell.
      We only achieved it to archive the log from 2 days ago or archive the log of yesterday and an empty logfile of today within the same zip file.
      
      Is it possible to change this part:
              dir $TargetArchiveFolder | where {  
                  !$_.PSIsContainer -and $_.extension -eq $FileExtension -and $ArchiveGroupingString -f $_.LastWriteTime -le $ArchiveDate  
              } | group {  
                  $ArchiveGroupingString -f $_.LastWriteTime
              } | foreach { 
                  $FilesFound = $true
      
      So that it takes the create date instead of the write time?
      Or if it needs to be changed elsewhere where would that be?
      
      Thanks and kind regards,
      Raoul
  • Download button
    2 Posts | Last post October 17, 2016
    • Hi,
      
      Just a question, for us to use the script, do we copy/paste the script to powershell or is there a download link
      
      Many Thanks
      
      Kind Regards
      
      Anthony
    • Hi Anthony, Copy/Paste should work just fine!
      
      Thanks,
      
      Bernie
  • Changing the archive time period for non-IIS use
    3 Posts | Last post August 15, 2016
    • Bernie,
      
      Thanks for the well thought out script.  It's got great error handling and has kept me from hurting myself quite well :).  I see that it's designed to archive by month or day but I'm trying to use it to archive non-IIS files older than six months in the specified folders every time it's run while leaving the current six months of files in place. 
      
      How should I modify the script to handle that?  
      
      Note: I'm an old batch file guy, my skills in PS are negligible.
      
      I appreciate your consideration.
      Sincerely,
      -felipe
    • Hi Felipe,
      
      Glad you like the script! To answer your question, in the current version of the script there's a section starting at line 246 - Switch($ArchiveGrouping){... Since you want to archive by month, look at the value called "month". One of the lines in that statement calls the AddMonths method. Currently I have it set to -1 so it archives all months except the current one. If you want to keep 6 months worth of logs unarchived, just change that to -6. 
      
      Before
      $ArchiveDate = $CurrentDate.AddMonths(-1).ToString("yyyyMM")
      
      After
      $ArchiveDate = $CurrentDate.AddMonths(-6).ToString("yyyyMM")
      
      Enjoy!
      
      Bernie
    • Bernie,
      
      Thanks for the reply.  I kind of figured that might be it but I preferred the confirmation before proceeding to mess with it.  I am running into an issue where the script doesn't see files to archive in the folders I specified. The following error is logged for the four folders I specified: "20160814: Info: No files found to archive in D:\ServerFolders\AppData\vdfveg\FOLDERNAMES\".
      
      I have the "$UseWebAdministrationSnapIn = $false" and "$FileExtension = ".*" entries in and the following 'ArchiveTargetName' entries:
      
      if (!$UseWebAdministrationSnapIn) { 
          $Targets = @() 
          # The folder(s) (targets) you want to archive 
          # Duplicate the following four lines for each target you want to archive 
          $Properties = @{ArchiveTargetName="ShippedInvoices";  
                          ArchiveTargetFolder="D:\ServerFolders\AppData\vdfveg\shippedinvoices\"} # Don't forget the trailing \  
          $Newobject = New-Object PSObject -Property $Properties 
          $Targets += $Newobject
          $Properties = @{ArchiveTargetName="PackLists";  
                          ArchiveTargetFolder="D:\ServerFolders\AppData\vdfveg\packlists\"} # Don't forget the trailing \  
          $Newobject = New-Object PSObject -Property $Properties 
          $Targets += $Newobject 
          $Properties = @{ArchiveTargetName="Outgoing";  
                          ArchiveTargetFolder="D:\ServerFolders\AppData\vdfveg\outgoing\"} # Don't forget the trailing \  
          $Newobject = New-Object PSObject -Property $Properties 
          $Targets += $Newobject 
          $Properties = @{ArchiveTargetName="Inspections";  
                          ArchiveTargetFolder="D:\ServerFolders\AppData\vdfveg\Inspections\"} # Don't forget the trailing \  
          $Newobject = New-Object PSObject -Property $Properties 
          $Targets += $Newobject 
      } 
      
      
      I definitely have files older that 6 months in there.  It was also failing to see files similarly when it was still set for 1 month.
      
      I appreciate your time and assistance.
      Sincerely,
      -felipe
       
  • Error when running 7-Zip 15.08 Beta
    2 Posts | Last post August 12, 2016
    • Hi Bernie,
      
      The 7zip version check fails with the beta version because it has text in it's fileversion.
      "15.08 Beta" is the string which is captured by: (Get-Item $7z).VersionInfo.FileVersion
      
      "Cannot convert value "15.08 beta" to type "System.Single". Error: "Input string was not in a correct format."
      At C:\Users\jackb.DRIFT\Desktop\Untitled1.ps1:281 char:1
      + [single]$7zVersion = (Get-Item $7z).VersionInfo.FileVersion"
      
      I made a small adjustment to correct this:
      ((Get-Item $7z).VersionInfo.FileVersion).substring(0,2)
      I just take the first 2 digits "15"
      
      There might be a more elegant way, but just throwing this out there in case anyone else stumbles upon this problem. At least it's better than removing the check.
      
      Thanks for your great work :)!
    • Hi Jack,
      
      Thanks for the heads up! Can you shoot me a link to the page where you got that beta so I can test? Looking on SourceForge I can't even find a beta for anything recent, only stuff from the pre-9.20 version...
      
      Thanks!
      
      Bernie
  • Turn the script to universal archiving tool
    2 Posts | Last post June 16, 2016
    • Hi Bernie,
      I wanted to turn your script into an universal archiving tool for our FTP server. I was able to pull out the important variables to parameters, put them to a csv file and use a control script with "for each" logic. What I struggle with is I get the zip files deleted even the variable is $RemoveOldArchives= $false.
      
      I'm a newbie with PS, learned it couple weeks ago...
      Would it be possible to send you the script I'm testing for review please?
      
      Thanks.
      
      Jaro.
      
    • Hi Jaro,
      
      I got your email - I'll follow up with you there. Thanks!
      
      Bernie
  • ftp/ site confusion
    2 Posts | Last post June 16, 2016
    • Your script assume that if a site has ftp binding it is an ftp site. All my sites have both ftp and http bindings and this produce zip files with wrong logs
    • Hi giammin,
      
      You are correct. The WebAdministration module doesn't make it easy to figure out the path if it's an FTP site, so the only way I could think to work around it was to check the site binding. The code that handles that is currently at lines 220-222, seen below.
      
      if ($Site.Bindings.Collection.Protocol -like "*ftp*") 
      	{ $SiteLogfileDirectory = $SiteDirectory+"\FTPSVC"+$Site.ID }
      else { $SiteLogfileDirectory = $SiteDirectory+"\W3SVC"+$Site.ID }
      
      With both bindings on each site you'll have to check the path to your log file folders. If they're all in W3SVC folder structures, you can just replace the three lines above with...
      
      $SiteLogfileDirectory = $SiteDirectory+"\W3SVC"+$Site.ID
      
      Alternatively, you could set $UseWebAdministrationSnapIn = $false and manually specify your log file folders in the section following that variable. 
      
      Let me know your thoughts on that. Thanks!
      
      Bernie
  • Consider using PPMd for compression
    4 Posts | Last post May 26, 2016
    • PPMd was approximately twice as fast (on an AWS t2.medium instance) and resulted in a file size approximately half the size (using 7-zip 16.02).
      
      Using the settings you’re currently using:
      ------------------------------------
      PS C:\Program Files\7-Zip> .\7z.exe a -tzip -mx8 -stl "C:\inetpub\logs\LogFiles\W3SVC2\u_ex160524.DEFLATE.custom.dated.zip" "C:\inetpub\logs\LogFiles\W3SVC2\u_ex160524.log"
      
      7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
      
      Scanning the drive:
      1 file, 34264480 bytes (33 MiB)
      
      Creating archive: C:\inetpub\logs\LogFiles\W3SVC2\u_ex160524.DEFLATE.custom.dated.zip
      
      Items to compress: 1
      
      Files read from disk: 1
      Archive size: 3017076 bytes (2947 KiB)
      Everything is Ok
      ------------------------------------
      
      Using PPMd with some semi-sensible settings (192MB RAM).
      ------------------------------------
      PS C:\Program Files\7-Zip> .\7z.exe a -t7z -stl -ssc -ssw -m0=PPMd:o32:mem192m "C:\inetpub\logs\LogFiles\W3SVC2\u_ex160524.PPMd.custom.dated.7z" "C:\inetpub\logs\LogFiles\W3SVC2\u_ex160524.log"
      
      7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
      
      Scanning the drive:
      1 file, 34264480 bytes (33 MiB)
      
      Creating archive: C:\inetpub\logs\LogFiles\W3SVC2\u_ex160524.PPMd.custom.dated.7z
      
      Items to compress: 1
      
      Files read from disk: 1
      Archive size: 1632519 bytes (1595 KiB)
      Everything is Ok
      ------------------------------------
    • Also, PPMd is single threaded, so you automatically won't use all CPU resources on any server except for the single core varieties (rare).
    • Hi lukescammell,
      
      Great suggestion! I'll look into adding that in as a compression option in case people want to stick with a traditional zip archive. 
      
      Thanks!
      
      Bernie
    • I just published an updated version of the script that includes the PPMd option. Thanks again for the suggestion!
      
      Bernie
  • Great Script
    2 Posts | Last post May 26, 2016
    • Thank you Bernie for this great script.
    • You're welcome, thanks for the feedback!
      
      Bernie
  • Need a Script for IIS log compression !
    2 Posts | Last post May 26, 2016
    • Hi Bernie,
       Hope you are doing well, i have seen your script with SMTP email alert. I need a script for IIS log zip , which need to run in task scheduler on server 2012. Can you please help me out ?
      
      Thank you.
      Regards,
      Selvaraj Manoharan | selvarajmks@gmail.com
    • Hi Selvaraj,
      
      The script can certainly be scheduled with the Task Scheduler. The following article describes how to do so.
      
      https://blogs.technet.microsoft.com/heyscriptingguy/2012/08/11/weekend-scripter-use-the-windows-task-scheduler-to-run-a-windows-powershell-script/
      
      Thanks,
      
      Bernie
  • Trouble with retrieving FTP site logging directory
    2 Posts | Last post March 30, 2016
    • I see in the update log that 20150922 fixed a bug regarding the automatic lookup of FTP logging directories.  I just found and tested your script and I am having the same issue still.  Any advice is appreciated.  The script works great on the IIS web logs.
      
      Windows Server 2008 R2
      IIS 7.5.7600.16385
      
      IIS Site ID: 2
      Bindings: *:21:
      FTP Logging Dir: L:\IISLogs
      
      From log:
      20160129: The requested target archive folder of C:\IISLogs\W3SVC2 does not exist. Please check the requested location and try again.
    • Hi jeevans207,
      
      I'm so sorry for missing your question when you posted it. For whatever reason I didn't get a notification from TechNet like I usually do when someone posts a question.
      
      I'm a little confused because you note that your FTP logs are in L:\IISLogs, but the log file entry you posted references C:\IISLogs\W3SVC2. So it looks like there may be a couple problems going on unless we're missing something. It may be easiest to take this troubleshooting offline - would you mind emailing me so we can follow up directly? Once we find a resolution I can post it back to this site for the benefit of others.
      
      Thanks!
      
      Bernie
1 - 10 of 49 Items