Times out of order

May 17, 2013 at 8:35 PM
First off, I love the concept of log joint, exactly what i've been looking for, haven't found anything remotely similar.

I'm trying to view an xml log file that our application outputs, we have multiple exes writing to the log file each app writes the log message time. Sometimes we can end up with messages out of order based on the time we write for each line.

Logjoint doesn't like messages that are out of order, It just doesn't show those lines. Anything i can do to show these?

May 18, 2013 at 5:03 AM
Looks like LineContainers.cs line: 689 is where the order is evaluated.

if (last != null)
if (msg.Time < last.Last.Time)
    throw new TimeConstraintViolationException();
Most things seem to work without that check. Looks like the BinarySearches might have problems. any other areas you can think of that wouldn't work?

May 18, 2013 at 11:45 AM
By default TimeConstraintViolationException is caught and out-of-order message is ignored. You'll see a little exclamation mark in log source properties window indicating that not all messages are shown.

This is quite often problem though. The solution is to put a line like following to your format's definition file
     <dejitter jitter-buffer-size="48"/>
An example can be found in Oracle-11g-alert-log-xml.format.xml.

This enables automatic messages reordering using specified buffer size. If message is less than jitter-buffer-size positions away from its correct position then the message won't be ignored. Instead it will be shown as it was logged in the correct time order.

<dejitter> element is applicable to regex-based formats as well. <dejitter>used to be hidden. Relatively recently I added a UI element to format editor wizard. However in your case you'll have to edit format definition file manually because there is no wizard for xml-based formats.

Please let me know if that resolves your problem.
May 18, 2013 at 1:34 PM
That works, thanks for the info.

What the story behind logjoint? Sergeys did you write all of this. It's a crap load of code and complicated, makes me feel stupid looking at it. Impressive stuff, keep up the good work.
May 19, 2013 at 4:23 PM
Everything was written my me as a side project of my main work. I changed a couple of employers and a few projects. In each project the logging had its unique specifics: many types of text and xml logs, huge logs, rotated logs, online logs storage, out-of-order messages, cloud logging (azure), ect. I've been adding related features to the tool.

The code is a bit messy and it seems complicated. I have to spend some time on refactoring. Split big files, find better names for some concepts, add comments to main interfaces.