Compress and Remove Log Files (IIS and others)

Archive Log Files: Manually specify any folder(s) or automatically parse IIS log folders, group by day/month and archive them with 7-Zip. Verify the archives and delete the original log files. Compressed archive will be about 4.5% (or less) of the size of the original log files.

 
 
 
 
 
4.8 Star
(28)
Add to favorites
Servers
4/6/2017
E-mail Twitter del.icio.us Digg Facebook
Sign in to ask a question


  • Check for create instead of modified date
    3 Posts | Last post April 28, 2017
    • Hi Bernie,
      The script is very useful for us but we would like to archive the script of yesterday aswell.
      We only achieved it to archive the log from 2 days ago or archive the log of yesterday and an empty logfile of today within the same zip file.
      
      Is it possible to change this part:
              dir $TargetArchiveFolder | where {  
                  !$_.PSIsContainer -and $_.extension -eq $FileExtension -and $ArchiveGroupingString -f $_.LastWriteTime -le $ArchiveDate  
              } | group {  
                  $ArchiveGroupingString -f $_.LastWriteTime
              } | foreach { 
                  $FilesFound = $true
      
      So that it takes the create date instead of the write time?
      Or if it needs to be changed elsewhere where would that be?
      
      Thanks and kind regards,
      Raoul
    • Hi Raoul,
      
      My apologies - I never received an email notification for your post.
      
      The script intentionally ignores 2 days because many log files remain open for writing for a day or two, and admins may need quick access to the most recent day or two for troubleshooting purposes without having to open an archive. 
      
      A quick way to change that 2 day gap is (at line 306 of the current version of the script) in the Switch($ArchiveGrouping) section for "day"
      
      $ArchiveDate = $CurrentDate.AddDays(-2).ToString("yyyyMMdd")
      
      You can change that -2 to -1 or get rid of it entirely.
      
      $ArchiveDate = $CurrentDate.ToString("yyyyMMdd")
      
      To your other point, yes, you could swap out .LastWriteTime with .CreationTime if that produces your desired behavior.
      
      Thanks!
      
      Bernie
    • Hello Bernie
      
      Thank you very much for your answer.
      We use external Software to analyse our Logs so we don't need them in the Log Directory after the new File was created.
      The change of the CurrentDate(-1) still kept yesterdays file and getting rid of it entirely resulted in an Archive with yesterdays File and a small part of todays Log file.
      
      The Script is now working fine this far with the .CreationTime change.
      
      Kind Regards
      Raoul
  • Script not removing *.Log files
    3 Posts | Last post April 14, 2017
    • Hi Bernie, the script is working fine in our environment. i was configured manual method.
      post arching the script is not deleting Source *.Log files which are just archived.
      this need to be removed.
      
      could you kindly help on this.
    • Hi Nizamuddin,
      
      By default the script is set to run in "test mode" so it doesn't delete anything while you're working on getting it configured to your liking. It sounds to me like "test mode" is still enabled. In the current version of the script, at line 67, you should see the following.
      
      $TestMode = $true
      
      Change that to...
      
      $TestMode = $false
      
      If that is set to $false and the log files still aren't deleting, then please double check the permissions on the log file location(s) to make sure that the account that's running the script has permission to delete those files.
      
      Let me know how that goes, thanks!
      
      Bernie
    • Thank you very much for rapid response and effective advice, it's worked... Cheers...
  • Two issues on server 2008 32bit
    3 Posts | Last post April 07, 2017
    • I got the script running great on my server 2012, however, when I tried it on my 2008 32 bit server, I ran into an issue when running the "IIS" option - it fails saying  The requested target archive folder of \W3SVC does not exist.
      
      The folders should be F:\IIS_Logs\W3SVCx
      
      Next I tried the ManualRecurse option on my F:\IIS_Logs folder which has numerous subfolders with logs.   In this case it simply fails saying:  Info: No files found to archive in F:\logs_iis\  -which is true, however, shouldn't be looking to recurse the folders in that location?
    • Sorry - typo - the folder should (in all above instances) be F:\logs_iis\
    • Hi,
      
      It sounds like it could be a permissions issue... Can you try running the script under an elevated PowerShell prompt, as well as double checking the permissions on those folders vs. the user account being used to run the script? 
      
      If all of that checks out fine it'd be helpful if you could provide me a list of your IIS sites, their respective log folder settings, and the paths to those respective sites/logs. If you'd prefer to email me (found in the script header) that info for further troubleshooting instead of posting it publicly please feel free.
      
      Thanks!
      
      Bernie
  • IIS 10
    4 Posts | Last post April 07, 2017
    • Bernie,
      
      Thanks for this script.  A small update to allow it to work properly with IIS 10:
      
      if ($iisVersion.MinorVersion -ge 5 -or $iisVersion.MajorVersion -eq 8) { 
            
      to
      
      if ($iisVersion.MinorVersion -ge 5 -or $iisVersion.MajorVersion -ge 8) { 
      
      Thanks!
      
    • Hi,
      
      Thanks for catching that! I haven't had a chance to test with IIS 10 yet. Have you successfully run the script against IIS 10 after making that change?
      
      Bernie
    • Yes - It works great!
    • Great to hear, thanks for following up!
  • Download button
    2 Posts | Last post October 17, 2016
    • Hi,
      
      Just a question, for us to use the script, do we copy/paste the script to powershell or is there a download link
      
      Many Thanks
      
      Kind Regards
      
      Anthony
    • Hi Anthony, Copy/Paste should work just fine!
      
      Thanks,
      
      Bernie
  • Changing the archive time period for non-IIS use
    3 Posts | Last post August 15, 2016
    • Bernie,
      
      Thanks for the well thought out script.  It's got great error handling and has kept me from hurting myself quite well :).  I see that it's designed to archive by month or day but I'm trying to use it to archive non-IIS files older than six months in the specified folders every time it's run while leaving the current six months of files in place. 
      
      How should I modify the script to handle that?  
      
      Note: I'm an old batch file guy, my skills in PS are negligible.
      
      I appreciate your consideration.
      Sincerely,
      -felipe
    • Hi Felipe,
      
      Glad you like the script! To answer your question, in the current version of the script there's a section starting at line 246 - Switch($ArchiveGrouping){... Since you want to archive by month, look at the value called "month". One of the lines in that statement calls the AddMonths method. Currently I have it set to -1 so it archives all months except the current one. If you want to keep 6 months worth of logs unarchived, just change that to -6. 
      
      Before
      $ArchiveDate = $CurrentDate.AddMonths(-1).ToString("yyyyMM")
      
      After
      $ArchiveDate = $CurrentDate.AddMonths(-6).ToString("yyyyMM")
      
      Enjoy!
      
      Bernie
    • Bernie,
      
      Thanks for the reply.  I kind of figured that might be it but I preferred the confirmation before proceeding to mess with it.  I am running into an issue where the script doesn't see files to archive in the folders I specified. The following error is logged for the four folders I specified: "20160814: Info: No files found to archive in D:\ServerFolders\AppData\vdfveg\FOLDERNAMES\".
      
      I have the "$UseWebAdministrationSnapIn = $false" and "$FileExtension = ".*" entries in and the following 'ArchiveTargetName' entries:
      
      if (!$UseWebAdministrationSnapIn) { 
          $Targets = @() 
          # The folder(s) (targets) you want to archive 
          # Duplicate the following four lines for each target you want to archive 
          $Properties = @{ArchiveTargetName="ShippedInvoices";  
                          ArchiveTargetFolder="D:\ServerFolders\AppData\vdfveg\shippedinvoices\"} # Don't forget the trailing \  
          $Newobject = New-Object PSObject -Property $Properties 
          $Targets += $Newobject
          $Properties = @{ArchiveTargetName="PackLists";  
                          ArchiveTargetFolder="D:\ServerFolders\AppData\vdfveg\packlists\"} # Don't forget the trailing \  
          $Newobject = New-Object PSObject -Property $Properties 
          $Targets += $Newobject 
          $Properties = @{ArchiveTargetName="Outgoing";  
                          ArchiveTargetFolder="D:\ServerFolders\AppData\vdfveg\outgoing\"} # Don't forget the trailing \  
          $Newobject = New-Object PSObject -Property $Properties 
          $Targets += $Newobject 
          $Properties = @{ArchiveTargetName="Inspections";  
                          ArchiveTargetFolder="D:\ServerFolders\AppData\vdfveg\Inspections\"} # Don't forget the trailing \  
          $Newobject = New-Object PSObject -Property $Properties 
          $Targets += $Newobject 
      } 
      
      
      I definitely have files older that 6 months in there.  It was also failing to see files similarly when it was still set for 1 month.
      
      I appreciate your time and assistance.
      Sincerely,
      -felipe
       
  • Error when running 7-Zip 15.08 Beta
    2 Posts | Last post August 12, 2016
    • Hi Bernie,
      
      The 7zip version check fails with the beta version because it has text in it's fileversion.
      "15.08 Beta" is the string which is captured by: (Get-Item $7z).VersionInfo.FileVersion
      
      "Cannot convert value "15.08 beta" to type "System.Single". Error: "Input string was not in a correct format."
      At C:\Users\jackb.DRIFT\Desktop\Untitled1.ps1:281 char:1
      + [single]$7zVersion = (Get-Item $7z).VersionInfo.FileVersion"
      
      I made a small adjustment to correct this:
      ((Get-Item $7z).VersionInfo.FileVersion).substring(0,2)
      I just take the first 2 digits "15"
      
      There might be a more elegant way, but just throwing this out there in case anyone else stumbles upon this problem. At least it's better than removing the check.
      
      Thanks for your great work :)!
    • Hi Jack,
      
      Thanks for the heads up! Can you shoot me a link to the page where you got that beta so I can test? Looking on SourceForge I can't even find a beta for anything recent, only stuff from the pre-9.20 version...
      
      Thanks!
      
      Bernie
  • Turn the script to universal archiving tool
    2 Posts | Last post June 16, 2016
    • Hi Bernie,
      I wanted to turn your script into an universal archiving tool for our FTP server. I was able to pull out the important variables to parameters, put them to a csv file and use a control script with "for each" logic. What I struggle with is I get the zip files deleted even the variable is $RemoveOldArchives= $false.
      
      I'm a newbie with PS, learned it couple weeks ago...
      Would it be possible to send you the script I'm testing for review please?
      
      Thanks.
      
      Jaro.
      
    • Hi Jaro,
      
      I got your email - I'll follow up with you there. Thanks!
      
      Bernie
  • ftp/ site confusion
    2 Posts | Last post June 16, 2016
    • Your script assume that if a site has ftp binding it is an ftp site. All my sites have both ftp and http bindings and this produce zip files with wrong logs
    • Hi giammin,
      
      You are correct. The WebAdministration module doesn't make it easy to figure out the path if it's an FTP site, so the only way I could think to work around it was to check the site binding. The code that handles that is currently at lines 220-222, seen below.
      
      if ($Site.Bindings.Collection.Protocol -like "*ftp*") 
      	{ $SiteLogfileDirectory = $SiteDirectory+"\FTPSVC"+$Site.ID }
      else { $SiteLogfileDirectory = $SiteDirectory+"\W3SVC"+$Site.ID }
      
      With both bindings on each site you'll have to check the path to your log file folders. If they're all in W3SVC folder structures, you can just replace the three lines above with...
      
      $SiteLogfileDirectory = $SiteDirectory+"\W3SVC"+$Site.ID
      
      Alternatively, you could set $UseWebAdministrationSnapIn = $false and manually specify your log file folders in the section following that variable. 
      
      Let me know your thoughts on that. Thanks!
      
      Bernie
  • Consider using PPMd for compression
    4 Posts | Last post May 26, 2016
    • PPMd was approximately twice as fast (on an AWS t2.medium instance) and resulted in a file size approximately half the size (using 7-zip 16.02).
      
      Using the settings you’re currently using:
      ------------------------------------
      PS C:\Program Files\7-Zip> .\7z.exe a -tzip -mx8 -stl "C:\inetpub\logs\LogFiles\W3SVC2\u_ex160524.DEFLATE.custom.dated.zip" "C:\inetpub\logs\LogFiles\W3SVC2\u_ex160524.log"
      
      7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
      
      Scanning the drive:
      1 file, 34264480 bytes (33 MiB)
      
      Creating archive: C:\inetpub\logs\LogFiles\W3SVC2\u_ex160524.DEFLATE.custom.dated.zip
      
      Items to compress: 1
      
      Files read from disk: 1
      Archive size: 3017076 bytes (2947 KiB)
      Everything is Ok
      ------------------------------------
      
      Using PPMd with some semi-sensible settings (192MB RAM).
      ------------------------------------
      PS C:\Program Files\7-Zip> .\7z.exe a -t7z -stl -ssc -ssw -m0=PPMd:o32:mem192m "C:\inetpub\logs\LogFiles\W3SVC2\u_ex160524.PPMd.custom.dated.7z" "C:\inetpub\logs\LogFiles\W3SVC2\u_ex160524.log"
      
      7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
      
      Scanning the drive:
      1 file, 34264480 bytes (33 MiB)
      
      Creating archive: C:\inetpub\logs\LogFiles\W3SVC2\u_ex160524.PPMd.custom.dated.7z
      
      Items to compress: 1
      
      Files read from disk: 1
      Archive size: 1632519 bytes (1595 KiB)
      Everything is Ok
      ------------------------------------
    • Also, PPMd is single threaded, so you automatically won't use all CPU resources on any server except for the single core varieties (rare).
    • Hi lukescammell,
      
      Great suggestion! I'll look into adding that in as a compression option in case people want to stick with a traditional zip archive. 
      
      Thanks!
      
      Bernie
    • I just published an updated version of the script that includes the PPMd option. Thanks again for the suggestion!
      
      Bernie
1 - 10 of 52 Items