More info? Check the SharePoint Dragons blog at http://sharepointdragons.com/ !

14-9 UPDATE

SFWR is best used in combination with http://gallery.technet.microsoft.com/office/Set-IIS-Log-Fields-via-ee9c19b3 This contribution suggests a minimum amount of IIS fields useful for logging and contains a PowerShell script for setting them on a web server.

5-2-2013 UPDATE Released SFWR V1.4

- Did some bug fixes (the slowest reports ordered by alphabet, instead of by number, duh!)

- Clarified up some reports, as there were some questions surrounding the time units in the reports. It should all be clear now.

25-1-2012 UPDATE Released SFWR V1.3

Not everybody has the same IIS logging settings, that's why SFWR uses the DLR to support this. Nevertheless, there were certain reports that threw exceptions and caused processing to halt because they expect certain data to be present (mainly bytes sent and bytes received). v1.3 uses a little Aspect Oriented Programming (AOP) to make sure all report calculation errors are handled.

 

25-5-2012 UPDATE

Released SFWR V1.2

Based on community feedback, the following reports became user name insensitive:

- Requests/User
- Requests/User/Month
- Requests/User/Week
- Unique Visitors

9-5-2012 UPDATE

Released SFWR v1.1

Based on community feedback, two new features were incorporated:

- An overview was added that displays the number of searches executed per user.

- An overview was added of the top slowest requests for a specific user.

-------------------------------------------------------------------------------------------

 

In a previous gallery post we’ve tried to satisfy our interest in the
performance of SharePoint environments, we’ve explored which set of performance
counters you should use for monitoring SharePoint WFEs and database servers: http://gallery.technet.microsoft.com/PowerShell-script-for-59cf3f70 .


This time, we’ll look at another aspect closely tied to this topic: IIS
logs.

The IIS logs are an invaluable way to get to know your web application and
your end users once it’s in production. For us, they are also the first stop
when a web application has performance problems. Therefore, having a tool to
analyze IIS logs is in invaluable asset in our bag of tricks. There are numerous
commercial tools out there that do a reasonable job at analyzing IIS logs and
often providing (more or less) great visual displays while doing it. There are
some things wrong with commercial tools though:

Since we came to the conclusion that having a tool that helps to analyze IIS
log files is essential and that we weren’t fully happy with existing options, we
decided to build such a tool ourselves which primarily gives us ultimate control
when it comes to adding new overviews.

We call the tool the SharePoint Flavored WebLog Reader (sfwr.exe) and it can
be used to analyze any IIS log file or batch of IIS log files. On top of that,
it has specific knowledge about SharePoint, which adds a SharePoint flavor to
the tool in the form of specific overviews that only make sense within a
SharePoint context.

The SharePoint Flavored WebLog Reader has the following advantages:

There are some drawbacks as well:

So, what can it do?

The SharePoint Flavored WebLog Reader provides the following overviews:

Where can I get it?

You can download it here at the TechNet Gallery.

How do I use it?

Open a command prompt, navigate to the folder where sfwr is stored, and
type:

sfwr.exe [items] [log path]

For example:

sfwr.exe 100 c:\temp\logs

If you also want to generate reports targeted towards a specific user, do the following:

sfwr.exe [items] [log path] [complete or partial user name]

For example:

sfwr.exe 100 “c:\temp\logs” margriet

Or

sfwr.exe 100 “c:\temp\logs” loisandclark\margrietbruggeman

What do I need to remember about the tool?

How does it do it?

Since we realize that everyone  could define different IIS logging settings,
we didn’t want to predefine a log structure that we depend on. Instead, we use
the log header that every log file has to determine the structure, and use the
Dynamic Language Runtime (DLR) ExpandoObject to create the structure
dynamically.

Please note: This flexibility came at a cost. It seems that
the extensive use of millions of expando objects causes the limit of (a little
over) 2.3 million log entry items. This doesn’t seem to happen in a version that
uses predefined structures. However, we feel this implementation has superior
flexibility and allows us to generate overviews on a generate-if-possible basis
and is therefore more generically applicable. Besides, 2.3 million items is a
huge amount, so we decided to stick to the DLR approach.

At a high level, we do this:

The code goes like this:

dynamic exp = new ExpandoObject();
for (int i =
0; i < lineEntries.Length; i++)
{
    IDictionary<string, object>
dict = exp;

    // If you were wondering at the strange guard
clause below,

    // VS.NET Code Contracts made us do it!!!
   
if (dict == null) throw new SfwrException("No dictionary");

    if (PropertyNames.Count() !=
lineEntries.Count()) throw new SfwrException("Property names are different from
line entries");
    dict[PropertyNames[i]] = lineEntries[i];
}

W3CImporter.Log.Add(exp);

Later on, we use these Expando objects to generate our overviews. The cool
thing about Expando objects is that you can use the dictionary keys as actual
property names. So, the following is perfectly valid:

dynamic exp = new ExpandoObject();

IDictionary<string, object> dict =
exp;

dict[“Test”] = “Hello Expando!”;

Console.WriteLine(exp.Test);

This is a pretty cool feature that really helps us out in this tool. Now that
we have a way to create a full-blown DTO object with properties on them and everything,
we can use them to create overviews via Linq (or Plinq). Some of these queries
are pretty straightforwards, others are niftier, such as the next one creating
an overview of the number of unique visitors per week:

DateTimeFormatInfo dfi =
DateTimeFormatInfo.CurrentInfo;
Calendar cal = dfi.Calendar;

var result = from log in Logs
              
group log by new { Week = cal.GetWeekOfYear(log.CurrentDateTime,
dfi.CalendarWeekRule, dfi.FirstDayOfWeek) } into grouped
              
select new { Week = grouped.Key.Week, Visitors = grouped.Select(x =>
x.UserName).Distinct().Count() };

Anyways, this is the high level overview of how we do it. With this
infrastructure in place, it’s going to be quite easy to extend the number of
available overviews. We feel like we’ve covered all of the most important ones,
but we need your help to come up with ideas for new overviews and features.

Support?

Most importantly? Use the tool to your benefit and provide feedback! Contact
us at margriet at loisandclark dot eu if you have questions or requests.