12

I've been using Log4Net on a few high traffic websites for a couple of years, and I cannot say that I am a happy customer. So, wanted to see if anybody else has the same concerns:

  1. The CPU overhead with RollingFileAppendor is massive. Some of my websites need to trace 5-10GB per day, and when I enable logging, the CPU utilization more than doubles. I would like to avoid the discussion of why so much tracing is needed. Some mission critical apps have to trace every step of every transaction.

  2. Rolling by date is often unreliable (it logs fine during the day, but then messes up the last day's log file around midnight). This behavior is inconsistent. I've seem more than a few people online that complain on this and nobody seems to have a good solution.

  3. Last but not least, I have not seen any new releases on the Apache website during the last three years. So, this starts to look as an abandoned open source project, and that usually means that it's time to move on to some alternative framework.

So, I am considering giving up Log4Net in favor of the Microsoft Enterprise Library or something else. Is anybody here having the same issues as me?

2
  • 4
    Any website that needs to log 5 - 10GB a day (in addition to the IIS logs) has a design flaw in my view. Log4Net isn't designed to be an auditor Jun 11, 2009 at 14:28
  • Interesting note regarding file rolling at the end of the day. It worked fine on Windows 7 (IIS 7.5), but when we switched to Win 2008 Server (also IIS 7.5) we started seeing a lot of data getting destroyed. Yesterdays 10MB log file was replaced with a few logged lines. Any ideas why?
    – Sire
    Nov 10, 2010 at 14:38

6 Answers 6

3
  1. Yes, it tends to use too much CPU. I had an app where I logged ~15GB/day and CPU usage was kind of high. I cut down logging to ~4GB/day, now the CPU usage is not noticeable at all.
  2. I've never seen this behavior (and I've been using log4net since 1.1.1 (3 years) in high-traffic websites)
  3. Yes, it's kind of quiet, but maybe that's because it's a stable, mature project. And development hasn't totally stopped, you can see in the svn repo that there's been some commits recently. If you're concerned about this, take a look at NLog, it's a younger, more active project.

Here's my appender config for comparison:

<appender name="LogFileAppender" type="log4net.Appender.RollingFileAppender, log4net" >
    <param name="File" value="log" />
    <param name="AppendToFile" value="true" />
    <rollingStyle value="Date" />
    <datePattern value="yyyyMMdd" />
    <maxSizeRollBackups value="7" />
    <layout type="log4net.Layout.PatternLayout, log4net">
        <param name="ConversionPattern" value="%d [%t] %-5p %c [%x] - %m%n" />
    </layout>
</appender>
4
  • mausch, Thanks for your detailed answer! I am very excited to see that Log4Net is working great for you. Would you be able to share how you configure the Log4Net RollingFileAppender and whether you use a Buffering appender? Jun 11, 2009 at 15:04
  • Here is my misbehaving config file: <appender name="WebLog" type="log4net.Appender.RollingFileAppender"> <file value="C:\Logs\Web.log" /> <onlyFixPartialEventData value="false" /> <lockingModel type="log4net.Appender.FileAppender+MinimalLock" /> <rollingStyle value="Composite" /> <datePattern value="yyyyMMdd" /> <staticLogFileName value="true" /> <layout type="log4net.Layout.PatternLayout"> <conversionPattern value="%date %property{NDC}: %message%newline" /> </layout> </appender> Jun 11, 2009 at 15:06
  • I just added my appender config. I'm not a log4net expert, but it looks like the lockingModel is a significant difference... also the rollingStyle. Jun 11, 2009 at 16:16
  • Mausch, Thanks for sharing your configuration! I just tried it on one of my servers with somewhat weird results. It was not creating the log file until I removed the text ", log4net" from the "type" attribute. But even then it was not writing anything to the log file. May I ask which version of Log4Net are you using? Mine is supposed to be the latest full release (February 2006?) Jun 11, 2009 at 22:41
2

You could look at using ASP.NET 2.0's Health Monitoring, and How To: Use Health Monitoring in ASP.NET 2.0

But I think you are going to have similiar problems. You are trying to use a logging tool as an audit tool, not exactly what it was designed for.

"Some mission critical apps have to trace every step of every transaction." - This is information that I would be logging to the database as part of a transaction. How could you guarantee the information is correct if it runs outside of a transaction?

1
  • Mitch, I would agree that some of my needs are around auditing. But we also occasionally have to localize tricky concurrency or data issues that cannot be easily reproduced in our testing environment. So, we are having to temporarily enable debug level tracing on Production and that creates a lot of volume. PS. Talking about Auditing - would you be able to recommend a framework that scales really well? Jun 11, 2009 at 15:31
2

May be it's not your case, but I think that with such volumes of log data you should be using log management system which has zero or minimal effect on your actual application during run time. Rolling around and managing gigabytes of log is rather awkward unless all your application does is create log. Another point - I've heard many complaints from users of entlib logging particularly regarding the performance. I'd check how it would do with your volumes of data before switching to it. But even if you do find it better than log4net, I think you will still be managing huge log files yourself.

2
  • We ended up deploying a software system that schedules and monitors Powershell scripts for archiving application logs in secure fashion. The logs are archived during low traffic hours, so the performance footprint of this operation is not really visible. Can you recommend a log management system that scales better than Log4Net and EngLib? Jun 12, 2009 at 13:18
  • I might be biased for obvious reasons, but please have a look at www.logfaces.com, it's built particularly for problems like that and perhaps would work for you as well.
    – Dima
    Jun 12, 2009 at 13:38
1

So far, I am inclined to blame everything on date-based rolling feature. I've tried swapping it for size-based rolling on a few servers, and I no longer see any data losses. Of course, this is not a pretty workaround because I no longer have one trace file per day. Also, the size-based rolling seems to have a bug that causes the file to roll either way too early or way too late. But this is not as painful as the original problem...

1
  • Thanks for this. I agree there seem to be a bug in the date rolling code (perhaps in connection with IIS threads). For me this is an acceptable workaround for now.
    – Sire
    Nov 9, 2010 at 16:10
0

For the performance issue the great thing about log4net is that you can always profile it to see where your application usage of log4net is bottlenecked and either: 1) Tackle the solution yourself or 2) Find a logging framework that doesn't have that bottleneck.

I can't help too much without knowing your application but from a cursory glance at the log4net source I noticed that the NextCheckDate() function is being called on EVERY void Append(LoggingEvent loggingEvent). I've included a portion of the source for NextCheckDate below and I could definitely picture this causing performance problems in high-volume logging scenarios.

protected DateTime NextCheckDate(DateTime currentDateTime, RollPoint rollPoint){
// Local variable to work on (this does not look very efficient)
DateTime current = currentDateTime;

// Do different things depending on what the type of roll point we are going for is
switch(rollPoint) 
{
    case RollPoint.TopOfMinute:
        current = current.AddMilliseconds(-current.Millisecond);
        current = current.AddSeconds(-current.Second);
        current = current.AddMinutes(1);
        break;

    case RollPoint.TopOfDay:
        current = current.AddMilliseconds(-current.Millisecond);
        current = current.AddSeconds(-current.Second);
        current = current.AddMinutes(-current.Minute);
        current = current.AddHours(-current.Hour);
        current = current.AddDays(1);
        break;

    case RollPoint.TopOfMonth:
        current = current.AddMilliseconds(-current.Millisecond);
        current = current.AddSeconds(-current.Second);
        current = current.AddMinutes(-current.Minute);
        current = current.AddHours(-current.Hour);
        current = current.AddMonths(1);
        break;
}     
return current;}

An optimized version for your application would probably cache the next rollover time in advance and only do a single comparison for each Append

0

So, I am considering giving up Log4Net in favor of the Microsoft Enterprise Library or something else.

For a comparison of alternative logging frameworks you might want to consider, see http://essentialdiagnostics.codeplex.com/wikipage?title=Comparison

It compares .NET Framework System.Diagnostics (built in capabilities), log4net, NLog and Enterprise Library, including a performance comparison.

Each has strengths and weaknesses, but EntLib does particularly bad on the performance comparison and, in terms of features, sometimes has less that the built in .NET Framework System.Diagnostics.

If you are particularly concerned about performance then log4net wins slightly with the .NET Framework System.Diagnostics similar.

NLog has very little overhead when not logging (i.e. just leaving it in the code), faster than either log4net or System.Diagnostics, but as the logging volume increases starts to fall behind.

For high performance logging using System.Diagnostics, with log rotation (including circular), have a look at the EventSchemaTraceListener, which I recently blogged about, but the tool support for viewing the logs (which are in an XML format) isn't very good.

I'd suggest you set up some performance testing if you are concerned. For the comparison listed above the source code of the performance comparison is available in the Essential Diagnostics project, so you can run it yourself, but you might want to tailor to your situation.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.