0

I'm looking for a list of common datetime formats used in logs (e.g. webserver, database, etc).

Even better would be a (java) library that can extract date and time from a given string ( < 10KB).

Does anyone know a good one?

2
  • 1
    How do you plan to interpret '10/11/12': October 11, 2012 or November 10, 2012? Could it be November 12, 2010?
    – Olaf
    Jun 20, 2012 at 21:12
  • 1
    It is part of near-realtime log analysis. The one closest to "now" and not in the future would be the most logical.
    – RobinUS2
    Jun 20, 2012 at 21:22

2 Answers 2

3

this library is likely a good place to start: SimpleDateFormat

The docs contains the an introduction to the standard datetime format strings. But as @Olaf points out, you're going to need to specify what the format is beforehand or there is literally no way differentiate certain dates from one another.

Looks like what you'd want to do is construct a range of date formats that might match, apply all of them to a date string, then see which date is closest to Datetime.now().

4
  • I already wrote 2 formatters for apache httpd access and error logs, however I need to recognize lots and lots of formats. Dynamic is a keyword.
    – RobinUS2
    Jun 20, 2012 at 21:23
  • ahh. I think the docs I linked have the standard format string patterns that are used by just about everyone. But to dynamically determine which pattern to use (a strategy design pattern if I ever heard one) starts getting into NLP territory, unless you have some other signal to switch on, like the particular log type. 05/06/12 has at least three contradictory interpretations, so you have to tell your parser which one to use, right? I suppose how you do that is a function of the larger application context.
    – Ben
    Jun 20, 2012 at 21:30
  • In the context it would be the most logical to be close to "now" but not in the future. That would be a good start I guess.
    – RobinUS2
    Jun 20, 2012 at 21:35
  • 1
    Don't mean to throw another wrench into works, but I remember parsing timesatamps from an Extended Squid log files which were expressed in format: long unix time.milliseconds . I second Ben's suggestion to keep a bunch of patterns, apply them one by one and if timestamp matches, add a sanity check. Next time you stumble upon an unusual format, you add yet another pattern. There is just no way to make it totally automatic. Even if you get the format right, you have to figure out timezone/UTC semantics of the particular log file source.
    – Olaf
    Jun 21, 2012 at 12:41
0

Although this doesn't answer your question directly, but Java includes libraries for working with regular expressions. It would be pretty easy to write a library of your own based on that. I've has a lot of success extracting all sorts of data using regular expression. It would certainly be less than 10kb and would require no external dependencies other than the JDK.

1
  • 1
    The 10KB refered to the maximum string size. Do you have a list of common log formats?
    – RobinUS2
    Jun 20, 2012 at 21:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.