Daniel Neubacher
2013-01-02 12:40:55 UTC
Hello there,
I've got a little trouble with the DNS resolving of syslog-ng. Last week I patched my syslog installation with the threaded dns bugfix (https://bugzilla.balabit.com/show_bug.cgi?id=212) and it seems like most of my problems are gone but one is still remaining.
Many times a day messages are sorted into a folder with the DNS name of my syslog-ng server instead of the real host where the log is coming from. The log line still has the right host in the text and most of the time it is working but I could not find any way to reproduce the problem on demand yet. For debugging I've disabled any logging for the server itself but it still happens.
My destinations are configured like this:
destination d_syslog { file("/log/syslog/${R_YEAR}/${R_MONTH}/${R_DAY}/$FULLHOST_FROM/$PROGRAM" template(t_plain)); };
And my dns options:
use_fqdn(yes);
dns_cache(yes);
dns_cache_size(16384);
dns_cache_expire(300);
dns_cache_expire_failed(10);
I've tried disabling the syslog-ng cache,installing a local caching bind and after that a nscd but with no success. With 750 servers sending 30k-40k logs per second the dns querys are too expensive and I need the internal syslog-ng caching. With local bind caching the logs per second are dropping down to 2500.
Does anybody has an idea to fix this?
--
Daniel Neubacher, Network Administrator
***@xing.com<mailto:***@xing.com>
XING AG
Gaensemarkt 43, 20354 Hamburg, Germany
Tel. +49 40 419131-28, Fax +49 40 419131-11
Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 98807
Exec. Board (Vorstand): Dr. Stefan Groß-Selbeck (Vorsitzender), Dr. Thomas Vollmoeller, Ingo Chu, Dr. Helmut Becker, Jens Pape
Chairman of the Supervisory Board (Aufsichtsratsvorsitzender): Dr. Neil Sunderland
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden and may be unlawful.
I've got a little trouble with the DNS resolving of syslog-ng. Last week I patched my syslog installation with the threaded dns bugfix (https://bugzilla.balabit.com/show_bug.cgi?id=212) and it seems like most of my problems are gone but one is still remaining.
Many times a day messages are sorted into a folder with the DNS name of my syslog-ng server instead of the real host where the log is coming from. The log line still has the right host in the text and most of the time it is working but I could not find any way to reproduce the problem on demand yet. For debugging I've disabled any logging for the server itself but it still happens.
My destinations are configured like this:
destination d_syslog { file("/log/syslog/${R_YEAR}/${R_MONTH}/${R_DAY}/$FULLHOST_FROM/$PROGRAM" template(t_plain)); };
And my dns options:
use_fqdn(yes);
dns_cache(yes);
dns_cache_size(16384);
dns_cache_expire(300);
dns_cache_expire_failed(10);
I've tried disabling the syslog-ng cache,installing a local caching bind and after that a nscd but with no success. With 750 servers sending 30k-40k logs per second the dns querys are too expensive and I need the internal syslog-ng caching. With local bind caching the logs per second are dropping down to 2500.
Does anybody has an idea to fix this?
--
Daniel Neubacher, Network Administrator
***@xing.com<mailto:***@xing.com>
XING AG
Gaensemarkt 43, 20354 Hamburg, Germany
Tel. +49 40 419131-28, Fax +49 40 419131-11
Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 98807
Exec. Board (Vorstand): Dr. Stefan Groß-Selbeck (Vorsitzender), Dr. Thomas Vollmoeller, Ingo Chu, Dr. Helmut Becker, Jens Pape
Chairman of the Supervisory Board (Aufsichtsratsvorsitzender): Dr. Neil Sunderland
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden and may be unlawful.