[syslog-ng] Syslog-NG OSE : a more and more difficult choice to make.

[Christophe Brocas]

Hello everybody,

I really enjoy the syntax, the stability, the flexibility and the so
clear and accurate documentation of Syslog-NG OSE. This is why I write
this post, I love the product, my message is definitively not a troll.

Despite above positive aspects, it is more and more difficult to choose
Syslog-NG OSE in corporate environment where you have Linux platforms
and others Unix flavors. Rsyslog comes with security and performance
features (sql driver, disk based bufferring, Solaris port etc) inside
whose can only be acquired through Premium Syslog-NG Edition.

If in the future, Rsyslog provides an AIX port on PPC architecture, I
really think it will be an ended story for Syslog-NG on corporate
environment : it will no more exist a technical reason to stay with an
open source under powered solution like Syslog-NG OSE or to buy a
solution while it exists an opensource solution with same / more features.

I really understand everybody has to earn its life, really. But the
current situation in the open source syslog products area is quite
difficult for Syslog-NG, that's why I wanted to point the above facts in
corporate environment out to you. I don't know how to do : more
appliances, more closed products, more consulting ... but the 2 flavors
(free and paid) of Syslog-NG are imho an each day harder choice to defend.

It is the message from a Syslog-NG user that would like to be able to
promote and use it in its company for a long time.

Thank you for your reading.

Bye
Christophe
--
Christophe Brocas
keyid : 0x237E9DB2
twitter: @cbrocas
web : http://brocas.org/blog/



*****************************************************
"Le contenu de ce courriel et ses eventuelles pièces jointes sont confidentiels. Ils s'adressent exclusivement à la personne destinataire. Si cet envoi ne vous est pas destiné, ou si vous l'avez reçu par erreur, et afin de ne pas violer le secret des correspondances, vous ne devez pas le transmettre à d'autres personnes ni le reproduire. Merci de le renvoyer à l'émetteur et de le détruire.

Attention : L'Organisme de l'émetteur du message ne pourra être tenu responsable de l'altération du présent courriel. Il appartient au destinataire de vérifier que les messages et pièces jointes reçus ne contiennent pas de virus. Les opinions contenues dans ce courriel et ses éventuelles pièces jointes sont celles de l'émetteur. Elles ne reflètent pas la position de l'Organisme sauf s'il en est disposé autrement dans le présent courriel."
*******************************************

[s***@feystorm.net]

Well I dont know what all features rsyslog has, but syslog-ng has all
the ones you mentioned. The sql support and solaris are both available
in the OSE, and the disk based buffering is available in PE. What does
rsyslog have that syslog-ng doesnt? Just curious.

-Patrick

Sent: Thursday, August 12, 2010 9:00:46 AM
From: Christophe Brocas <***@cnamts.fr>
To: syslog-***@lists.balabit.hu
Subject: [syslog-ng] Syslog-NG OSE : a more and more difficult choice
to make.

Post by Christophe Brocas
Hello everybody,
I really enjoy the syntax, the stability, the flexibility and the so
clear and accurate documentation of Syslog-NG OSE. This is why I write
this post, I love the product, my message is definitively not a troll.
Despite above positive aspects, it is more and more difficult to choose
Syslog-NG OSE in corporate environment where you have Linux platforms
and others Unix flavors. Rsyslog comes with security and performance
features (sql driver, disk based bufferring, Solaris port etc) inside
whose can only be acquired through Premium Syslog-NG Edition.
If in the future, Rsyslog provides an AIX port on PPC architecture, I
really think it will be an ended story for Syslog-NG on corporate
environment : it will no more exist a technical reason to stay with an
open source under powered solution like Syslog-NG OSE or to buy a
solution while it exists an opensource solution with same / more features.
I really understand everybody has to earn its life, really. But the
current situation in the open source syslog products area is quite
difficult for Syslog-NG, that's why I wanted to point the above facts in
corporate environment out to you. I don't know how to do : more
appliances, more closed products, more consulting ... but the 2 flavors
(free and paid) of Syslog-NG are imho an each day harder choice to defend.
It is the message from a Syslog-NG user that would like to be able to
promote and use it in its company for a long time.
Thank you for your reading.
Bye
Christophe

[Evan Rempel]

For me, the most compelling differences in favoe of syslog-ng are;

1. Streaming live logs to an application. In our environment we stream the
logs into applications that identify critical events adn then send the events into nagios
for alerting, acknowledgement and reporting. We also send critical events into
out trouble ticket system. Intrusion detection etc.

2. The ability to have the pattern database. it isn't just about collecting logs. Anyone
can do that. Its about mining the logs for the important things, and the
unknown things. The pattern database is critical in this effort.

Evan Rempel

Post by s***@feystorm.net
Well I dont know what all features rsyslog has, but syslog-ng has all
the ones you mentioned. The sql support and solaris are both available
in the OSE, and the disk based buffering is available in PE. What does
rsyslog have that syslog-ng doesnt? Just curious.
-Patrick
Sent: Thursday, August 12, 2010 9:00:46 AM
Subject: [syslog-ng] Syslog-NG OSE : a more and more difficult choice
to make.

Post by Christophe Brocas
Hello everybody,
I really enjoy the syntax, the stability, the flexibility and the so
clear and accurate documentation of Syslog-NG OSE. This is why I write
this post, I love the product, my message is definitively not a troll.
Despite above positive aspects, it is more and more difficult to choose
Syslog-NG OSE in corporate environment where you have Linux platforms
and others Unix flavors. Rsyslog comes with security and performance
features (sql driver, disk based bufferring, Solaris port etc) inside
whose can only be acquired through Premium Syslog-NG Edition.
If in the future, Rsyslog provides an AIX port on PPC architecture, I
really think it will be an ended story for Syslog-NG on corporate
environment : it will no more exist a technical reason to stay with an
open source under powered solution like Syslog-NG OSE or to buy a
solution while it exists an opensource solution with same / more features.
I really understand everybody has to earn its life, really. But the
current situation in the open source syslog products area is quite
difficult for Syslog-NG, that's why I wanted to point the above facts in
corporate environment out to you. I don't know how to do : more
appliances, more closed products, more consulting ... but the 2 flavors
(free and paid) of Syslog-NG are imho an each day harder choice to defend.
It is the message from a Syslog-NG user that would like to be able to
promote and use it in its company for a long time.
Thank you for your reading.
Bye
Christophe

[Balazs Scheidler]

Hello Christophe,

First of all, thanks for your email. I really appreciate honest
opinions, and although not all of your points are accurate, messages
like this actually has an influence on syslog-ng direction.

Post by Christophe Brocas
Hello everybody,
I really enjoy the syntax, the stability, the flexibility and the so
clear and accurate documentation of Syslog-NG OSE. This is why I write
this post, I love the product, my message is definitively not a troll.
Despite above positive aspects, it is more and more difficult to choose
Syslog-NG OSE in corporate environment where you have Linux platforms
and others Unix flavors. Rsyslog comes with security and performance
features (sql driver, disk based bufferring, Solaris port etc) inside
whose can only be acquired through Premium Syslog-NG Edition.

This is not completely true, the platforms supported by syslog-ng are by
no means less than the premium edition. We don't build binaries of the
OSE edition for all of PE's platforms, but the code is the same,
everyone is free to build it on his/her platform of choice. In fact a
number of binary download site do have syslog-ng binaries (sunfreeware
for Solaris, perzl.org for AIX) and we also work together with the
maintainers of these sites on updating OSE packages in these
repositories, just like we worked hard to update the syslog-ng package
in Linux distributions.

syslog-ng OSE had the SQL destination feature since 2.1, first released
in January 2008.

The only missing item in your list is disk based buffering. This is
true, but also quite easy to work around:
* write everything to a local file and
* set up the same file as a source driver

So while it may seem that rsyslog has more hype around it, it isn't
true, that it surpasses syslog-ng OSE in all ways.

Also, I feel important to note that syslog-ng has been refocused in
recent years and now it also cares about the content of the messages. It
is not merely a transport for syslog messages anymore and I think this
certainly is ahead of what rsyslog provides.

This is what those parsers & rewrite rules are about, and also in the
recent 3.2 release it also introduces support for binary but structured
source files (it can read Process Accounting logs). Doing things like
receiving SNMP traps as name-value pairs and polling SQL tables for new
logs are in the pipe.

I'd like to push out an update to the current syslog-ng OSE roadmap at
the webpage, but anyway, here are my plans for the near future:

1) syslog-ng OSE 3.2 is out as an alpha release, but I don't expect too
much problems there, I guess 3.2.0 can be released latest in a month.
syslog-ng was rearchitected to be plugin based and other important
changes were applied (see my last blog posts for more details).

2) syslog-ng OSE 3.3/syslog-ng PE 4.0 is going to be developed in
parallel,
* OSE 3.3 will focus on performance
* PE 4.0 is going to be the last long-term-support release ("stable"
as we call it) based on the current, forked syslog-ng OSE codebase

3) syslog-ng PE and OSE will be merged into PE 4.1, this means that
existing core (e.g. non-plugin) features of the PE will be migrated to
the OSE and core-wise they will become equivalent. This will mean that
the "wildcard log files" and the recent multiline feature will
definitely go to the OSE version. The disk buffer however is still
undecided.

Post by Christophe Brocas
If in the future, Rsyslog provides an AIX port on PPC architecture, I
really think it will be an ended story for Syslog-NG on corporate
environment : it will no more exist a technical reason to stay with an
open source under powered solution like Syslog-NG OSE or to buy a
solution while it exists an opensource solution with same / more features.

I would really question that rsyslog has the same or more features. In
some areas it surpasses syslog-ng, in others it is lacking.

Post by Christophe Brocas
I really understand everybody has to earn its life, really. But the
current situation in the open source syslog products area is quite
difficult for Syslog-NG, that's why I wanted to point the above facts in
corporate environment out to you. I don't know how to do : more
appliances, more closed products, more consulting ... but the 2 flavors
(free and paid) of Syslog-NG are imho an each day harder choice to defend.

Well, don't look at the functionality only. In the PE edition there are:
* binaries for 27 platforms (and growing)
* thorough testing for each release
* long term support

Apart from the few feature differences, PE really makes it easier to
deploy syslog-ng in enterprise environment. If you have 3 different
platforms (Solaris, Linux, AIX), possibly multiple versions of these,
how long does it take to compile syslog-ng on them? And what if there's
a bug/security issue and you need to rebuild?

It is exactly the same set of incentives that for example RedHat uses in
its Enterprise Linux offering. The difference is that we also have some
additional features, because certainly an Operating System is applicable
to more situations, the market is larger and the number of people
willing to pay solely for services is larger.

With syslog-ng, this is not true. But, please read my recent blog post
(also posted to this list).

Post by Christophe Brocas
It is the message from a Syslog-NG user that would like to be able to
promote and use it in its company for a long time.

Hopefully I could at least blur the picture somewhat. It is not black &
white.

[Christophe Brocas]

Post by Balazs Scheidler
Hello Christophe,

Hello Balazs

Post by Balazs Scheidler
First of all, thanks for your email. I really appreciate honest
opinions, and although not all of your points are accurate, messages
like this actually has an influence on syslog-ng direction.

Thank you for understanding the meaning of my message and sorry for my
mistakes.

Post by Balazs Scheidler

Post by Christophe Brocas
Hello everybody,
I really enjoy the syntax, the stability, the flexibility and the so
clear and accurate documentation of Syslog-NG OSE. This is why I write
this post, I love the product, my message is definitively not a troll.
Despite above positive aspects, it is more and more difficult to choose
Syslog-NG OSE in corporate environment where you have Linux platforms
and others Unix flavors. Rsyslog comes with security and performance
features (sql driver, disk based bufferring, Solaris port etc) inside
whose can only be acquired through Premium Syslog-NG Edition.

This is not completely true, the platforms supported by syslog-ng are by
no means less than the premium edition. We don't build binaries of the
OSE edition for all of PE's platforms, but the code is the same,
everyone is free to build it on his/her platform of choice. In fact a
number of binary download site do have syslog-ng binaries (sunfreeware
for Solaris, perzl.org for AIX) and we also work together with the
maintainers of these sites on updating OSE packages in these
repositories, just like we worked hard to update the syslog-ng package
in Linux distributions.
syslog-ng OSE had the SQL destination feature since 2.1, first released
in January 2008.

As Patrick as said before in the thread, totally true. Sorry for my
mistake :(

Post by Balazs Scheidler
The only missing item in your list is disk based buffering. This is
* write everything to a local file and
* set up the same file as a source driver
So while it may seem that rsyslog has more hype around it, it isn't
true, that it surpasses syslog-ng OSE in all ways.

Ok.

Post by Balazs Scheidler
Also, I feel important to note that syslog-ng has been refocused in
recent years and now it also cares about the content of the messages. It
is not merely a transport for syslog messages anymore and I think this
certainly is ahead of what rsyslog provides.

That is right but it depends how each organization uses its syslog
architecture (transport vs messages understanding). I think Syslog-NG
has a rough battle ahead because messages exploitation leads directly to
SIEM solutions.

A quite hard question to answer : where does a log messaging solution
have to stop its development ?

Post by Balazs Scheidler
This is what those parsers & rewrite rules are about, and also in the
recent 3.2 release it also introduces support for binary but structured
source files (it can read Process Accounting logs). Doing things like
receiving SNMP traps as name-value pairs and polling SQL tables for new
logs are in the pipe.
I'd like to push out an update to the current syslog-ng OSE roadmap at
1) syslog-ng OSE 3.2 is out as an alpha release, but I don't expect too
much problems there, I guess 3.2.0 can be released latest in a month.
syslog-ng was rearchitected to be plugin based and other important
changes were applied (see my last blog posts for more details).
2) syslog-ng OSE 3.3/syslog-ng PE 4.0 is going to be developed in
parallel,
* OSE 3.3 will focus on performance
* PE 4.0 is going to be the last long-term-support release ("stable"
as we call it) based on the current, forked syslog-ng OSE codebase
3) syslog-ng PE and OSE will be merged into PE 4.1, this means that
existing core (e.g. non-plugin) features of the PE will be migrated to
the OSE and core-wise they will become equivalent. This will mean that
the "wildcard log files" and the recent multiline feature will
definitely go to the OSE version. The disk buffer however is still
undecided.

Oh, it is a great news !

Of course, it will be great to have disk buffering inside the OSE
because I really think by this way, Syslog-NG would close the story
about syslog transport : Syslog-NG OSE would have all the features
required for log transport : security (authentication, integrity and no
lost of messages), performance and easiness of exploitation (syntax,
wildcard etc).

And then, the debate will go the message exploitation where as you
demonstrated, Syslog-NG is ahead of all others solutions.

One thing :

do you think about switching from OSE and PE editions model to only one
distribution which would be Open Source and selling closed source
plugins which would be usable through this new only open source Syslog-NG ?

IMHO, it would be far more easy to promote in the Open Source community
than open source vs premium editions. But, of course, only you can say
if it is a sufficient model to provide a living for Balabit.

Post by Balazs Scheidler

Post by Christophe Brocas
If in the future, Rsyslog provides an AIX port on PPC architecture, I
really think it will be an ended story for Syslog-NG on corporate
environment : it will no more exist a technical reason to stay with an
open source under powered solution like Syslog-NG OSE or to buy a
solution while it exists an opensource solution with same / more features.

I would really question that rsyslog has the same or more features. In
some areas it surpasses syslog-ng, in others it is lacking.

You are right. The key feature is disk based buffering I think and
that's why I think it would be a major step in Syslog-NG history if you
integrate it inside the Syslog-NG 4.1 OSE.

Post by Balazs Scheidler

Post by Christophe Brocas
I really understand everybody has to earn its life, really. But the
current situation in the open source syslog products area is quite
difficult for Syslog-NG, that's why I wanted to point the above facts in
corporate environment out to you. I don't know how to do : more
appliances, more closed products, more consulting ... but the 2 flavors
(free and paid) of Syslog-NG are imho an each day harder choice to defend.

* binaries for 27 platforms (and growing)
* thorough testing for each release
* long term support
Apart from the few feature differences, PE really makes it easier to
deploy syslog-ng in enterprise environment. If you have 3 different
platforms (Solaris, Linux, AIX), possibly multiple versions of these,
how long does it take to compile syslog-ng on them? And what if there's
a bug/security issue and you need to rebuild?
It is exactly the same set of incentives that for example RedHat uses in
its Enterprise Linux offering. The difference is that we also have some
additional features, because certainly an Operating System is applicable
to more situations, the market is larger and the number of people
willing to pay solely for services is larger.
With syslog-ng, this is not true. But, please read my recent blog post
(also posted to this list).

You have got the point :-)

Post by Balazs Scheidler

Post by Christophe Brocas
It is the message from a Syslog-NG user that would like to be able to
promote and use it in its company for a long time.

Hopefully I could at least blur the picture somewhat. It is not black &
white.

Thank you very much for your answer which is very usefull for users like
us : it gives a good visibility for the future of Syslog-NG.

I really hope that Syslog-NG will be back in the heart of Linux
distributions and users because it deserves it : so clean syntax,
accurate documentation, performance, security and advanced messages parsing.

Bye
Christophe
--
Christophe Brocas
keyid : 0x237E9DB2




*****************************************************
"Le contenu de ce courriel et ses eventuelles pièces jointes sont confidentiels. Ils s'adressent exclusivement à la personne destinataire. Si cet envoi ne vous est pas destiné, ou si vous l'avez reçu par erreur, et afin de ne pas violer le secret des correspondances, vous ne devez pas le transmettre à d'autres personnes ni le reproduire. Merci de le renvoyer à l'émetteur et de le détruire.

Attention : L'Organisme de l'émetteur du message ne pourra être tenu responsable de l'altération du présent courriel. Il appartient au destinataire de vérifier que les messages et pièces jointes reçus ne contiennent pas de virus. Les opinions contenues dans ce courriel et ses éventuelles pièces jointes sont celles de l'émetteur. Elles ne reflètent pas la position de l'Organisme sauf s'il en est disposé autrement dans le présent courriel."
************