1

One of our systems is build as a wrapper around a third party customized work flow product that we cannot change.

Recently we got a request to capture some user activity data which is persisted nowhere else than in the application logs. These logs files are quite a few spreading across multiple machines so analyzing those logs to extract the required info would be a relatively complex task.

Creating some aspects to audit those user actions was considered but was quickly ran out giving the amount of third party code you need to decompile and look at. There will also be security restrictions about activating load-time weaving on production application servers.

My initial idea was to redirect the log outputs we are interested in to a different file (actually files) and then analyze those files to process that data. This seems doable but when you start considering things you soon find a lot of issues to deal with when working with files: logs rollover, no transactions, keeping a track of what was processed and what not, etc.

My second thought was to implement a custom pattern layout and configure it to be used for the logs I was interested in. Easy to write, keeping it away from third party code in addition to sending the message to the log file will also save the log entry into a table. Once in the table I can do everything I want starting from there. Code fragment below:

public class CustomPatternLayout extends Layout {
    private final PatternLayout patternLayout;
    private SimpleJdbcTemplate simpleJdbcTemplate;

    public CustomPatternLayout() {
        System.out.println("Creating CustomPatternLayout instance");
        patternLayout = new PatternLayout("%m%n");
        simpleJdbcTemplate = new SimpleJdbcTemplate(...);
    }
    ...
    protected PatternParser createPatternParser(String pattern) {
        System.out.println("CustomPatternLayout: returning pattern parser");
        return new PatternParser(pattern);
    }

    public String format(LoggingEvent event) {
        String log = patternLayout.format(event);
        System.out.println("CustomPatternLayout: about to write log entry to the database: " + log);
        simpleJdbcTemplate.update("INSERT INTO my_login_tracker (log_entry) values (?)", log);
        return log;
    }

Once this done I added it to my log4j.properties

log4j.appender.FILE=org.apache.log4j.RollingFileAppender
log4j.appender.FILE.layout=my.org.utils.logging.CustomPatternLayout

And I can see in the logs my system out entries sign that my custom pattern layout is created and configured. However the actual format method never gets invoked and consequently I get nothing saved in my table. There are still lots of log entries in the logs and it looks like my custom pattern layout is not used.

I used this strategy before to make it possible to output thread id in the logs and it worked perfectly. I admit that was a standalone java application as opposed to this one which is running on a webshere application server.

Any idea about what would cause this and how to address it? Alternative approach that will do the job without involving lots of technologies?

Thank you in advance for your inputs.

3
  • 1
    You have added the layout to a new appender, but I don't see where you added the layout to a logger. Further, why not simply use a JDBCAppender which is specifically designed for (and optimiser for) this task? Finally, if you want to aggregate logs in production, consider using something like Logstash which will have significantly lower overheads than inserting logs into an RMDBS. Sep 30, 2015 at 7:20
  • Thanks Boris. The log4j entries above is only one extract not the full logj.properties file. There is nothing special about my log4j.properties file apart from the fact I replaced the classical log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout with my CustomPatternLayout. Everything else stays the same like in lots of others examples on internet. I will look into the JDBCAppender tomorrow my main idea was to leave the logs as they were and just do the extra data save. It looks like I am missing something though.
    – Julian
    Sep 30, 2015 at 12:30
  • Hi Boris. Thank you very much for your JDBCAppender suggestion. Any day I learn something new is a "worth living day" for me. I was not aware about these extra log4j appenders, actually quite a a rich set, especially in Log4J 2. Once I found a way to make it work with an encrypted the DB password it worked perfectly for me. Please publish your comment as an answer so I can accept it.
    – Julian
    Oct 1, 2015 at 3:08

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.