Command line option -T is not restricted to messages from remote hosts

[opty77]

With RFC 3164 (the default) syslogd can use messages' timestamp but it doesn't provide enough information/precision (year, TZ and subseconds). You can't reliably sort messages taken from several files, you would have to log them to a single file too.

RFC 5424 helps a bit but you still have to fake the subsecond part of glibc's syslog(3) messages which may lead to wrong order in sorted list. (Consider the edge of the consecutive seconds.)

But... you can run syslogd -T (undocumented for local messages), let it always generate consistent timestamps (although different) and reliably sort messages from several files. IIRC rsyslog uses this approach by default and I wonder whether syslogd should behave the same. It would fix #92 which led me to these thoughts.

[troglobit]

Interesting observation about the effect of -T on local messages, I was not aware of that! 😮

I've tried real hard to bring sysklogd in-line with the FreeBSD and NetBSD implementations of syslogd, from which old sysklogd is derived from. So I've spent the better part of this morning trying to figure out how they limit the RemoteAddDate (-T) option from just biting on remote messages in their code bases -- doesn't seem they do, and their man-pages (1, 2) document -T similarly to sysklogd.

So at the very least we should update our documentation of -T to match the reality. However, I'm not keen on changing the default behavior of syslogd -- maybe for the next major bump (3.x). By opening up the description of -T instead it would be clearer to an administrator how to control the behavior to suit their needs.

[opty77]

Yeah, it surprised me too.

I definitely agree with documentation update, something like

Always use the local time and date for messages received from the network as well as for local non-kernel ones, instead of the timestamp field supplied in the message. This is useful if some of the originating hosts cannot keep time properly, are unable to generate a correct timestamp or you need more information/precision (year, TZ or subseconds; assuming RFC 5424).

I rather meant changing the default behavior just in RFC 5424 case but that again leads to the last paragraph in #92 (comment).

[troglobit]

I definitely agree with documentation update, something like

Always use the local time and date for messages received from the network as well as for local non-kernel ones, instead of the timestamp field supplied in the message. This is useful if some of the originating hosts cannot keep time properly, are unable to generate a correct timestamp or you need more information/precision (year, TZ or subseconds; assuming RFC 5424).

Mmm, we'll see. Must be succinct and easy to read. Adding this to the next milestone.

I rather meant changing the default behavior just in RFC 5424 case but that again leads to the last paragraph in #92 (comment).

No, the output formats only control the RFC style in the output, not how to interpret the input. It would also be very confusing/inconsistent.

The only thing we should change is the documentation.

[opty77]

Think of a message always containing two types of timestamps: original and syslogd-received. Then you could choose which one to use in the output.

But I guess I should rather finally decide which one to choose instead of trying to make syslogd rsyslog-ish?

[troglobit]

Think of a message always containing two types of timestamps: original and syslogd-received. Then you could choose which one to use in the output.

Sure, but that should not be implicitly changed by the RFCXXXX output format options. For that another action option should be used instead, e.g., ;datetime=replace or similar.

But I guess I should rather finally decide which one to choose instead of trying to make syslogd rsyslog-ish?

The direction of the sysklogd project is quite clear, as I've stated previously, we aim to track our upstream brethren in FreeBSD and NetBSD land and we are not aiming to become an rsyslogd replacement.

[opty77]

I guess that I finally understand.

[opty77]

Worth a mention?

[troglobit]

Worth a mention?

I've held off responding to this, but I really do not understand this comment. The link references a long thread, the link anchor points to a post from the rsyslogd maintainer. What in particular do you, @opty77, think is "worth a mention" here? Or do you mean this particular comment should be taken into consideration for this particular bug report? I honestly cannot guess what people mean, so please try to be more clear in your issue messages.

[opty77]

I consider

If the sender sent no ms, we can not report them. Rather than to pretend "x.000000" they are there, we do not give them.

interesting regarding original dilemma in the very first comment (together with #96) and

Note that starting with the latest v5-devel version AND a recent Linux kernel, we can ask the system for more precise timestamps on messages that come in via the log socket.

which I tried to find in rsyslogd source code but failed and rather saw syslogd -T behavior.

[troglobit]

Great reply, thank you, so much clearer now! 😃