[syslog-ng] Issue with filtering messages and sending them

[Cosmin Neagu]

Hello everyone,

I have an issue with syslog-ng (detailed below) and I tend to incline is
related to syslog-ng, an I hope will find some answers here.

I set up a "solution" so that when someone fails to login to a ssh linux
server, I receive an email with info about that. The idea is like this:
Syslog-ng should look for any messages containing info about login
failures, and when it sees those messages, it should pass them to a
little script that emails them.

Syslog-ng relevant config for that:
filter f_ssh {program (sshd) and match("Failed password"
value("MESSAGE")); };
destination d_sshalert { program("/home/cosmin/sshalert.sh"); };
#this is the script that is sendind the emails
destination d_sshfile { file("/home/cosmin/LOGS/sshdfailed.log"); };
log { source(s_src); filter(f_ssh); destination(d_sshfile);};
#log { source(s_src); filter(f_ssh); destination(d_sshalert);
flags(final); };
log { source(s_src); filter(f_ssh); destination(d_sshalert);};

And the script that is sending the emails:
***@srv:~$ cat sshalert.sh
#!/bin/bash
email="***@gmail.com"
while read event;
do
echo -e "Subject:*** SSH failed attempt on domain.ro ***\nFrom:SSH
Watcher <***@domain.ro>\nTo:***@gmail.com\n\n####### WARNING
#######\n\nA failed SSH attempt has been logged:\n${event}\n\n***
Required actions***\n1. Check ip owner and location:\nEx: #curl
ipinfo.io/X.Y.Z.T\n2. Ban the source IP address in
iptables.rules\n***********************\n\nFor any questions contact:
***@domain.ro\nHave a nice day\n" | /usr/sbin/sendmail -f
***@domain.ro ${email}
done

Everything is working as expected, I receive mails like this when failed
attempts exists:

####### WARNING #######

A failed SSH attempt has been logged:
*Aug 6 15:49:47 srv sshd[18236]: Failed password for someone from
173.XX.220.XX port 59004 ssh2*

*** Required actions***
1. Check ip owner and location:
Ex: #curl ipinfo.io/X.Y.Z.T
2. Ban the source IP address in iptables.rules
***********************

For any questions contact:***@domain.ro
Have a nice day


Please not that the log inserted is the right one: Aug 6 15:49:47 srv
sshd[18236]: Failed password for someone from 173.XX.220.XX port 59004 ssh2
My problem is that form time to time (did not manage to discover a time
pattern), I also receive some strange mails like the following:

####### WARNING #######

A failed SSH attempt has been logged:
*Aug 6 16:09:47 srv -- MARK --*

*** Required actions***
1. Check ip owner and location:
Ex: #curl ipinfo.io/X.Y.Z.T
2. Ban the source IP address in iptables.rules
***********************

For any questions contact:***@domain.ro
Have a nice day


As you can see, the log inserted in the mail is strange: *Aug 6
16:09:47 srv -- MARK --*. I assume that somehow, syslog-ng is the one
that is sending this log to my script which is why I configured another
destination to a local file to test my hypothesis (d_sshfile) but the
message is not inserted in that local file

Now, my question for you is: is there a way to determine if syslog-ng is
the one responsible for sending that strange message and why? Am I doing
something wrong with syslog-ng config (maybe the filter is not right?)
Any help would be appreciated. Thanks

[Fabien Wernli]

Hi,

The "strange" messages you're seeing are MARK signals generated indeed by
syslog-ng.

The fact that you see them in your email means for some reason that your log
path logic fails to do what you want.

Could you please post the full configuration, especially the source
definitions which were missing in your first email. ?

Cheers

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

[Cosmin Neagu]

Wow, that's a quick response :) thanks.

Below is the full initial config, did not change anything on
syslog-ng.conf. The source definition is the "default" one:
source s_src {
system();
internal();
};

I will try to look into some documentations about those MARK signals.



***@srv:/etc/syslog-ng$ cat syslog-ng.conf
@version: 3.5
@include "scl.conf"
@include "`scl-root`/system/tty10.conf"

# Syslog-ng configuration file, compatible with default Debian syslogd
# installation.

# First, set some global options.
options { chain_hostnames(off); flush_lines(0); use_dns(no); use_fqdn(no);
owner("root"); group("adm"); perm(0640); stats_freq(0);
bad_hostname("^gconfd$");
};

########################
# Sources
########################
# This is the default behavior of sysklogd package
# Logs may come from unix stream, but not from another machine.
#
source s_src {
system();
internal();
};

# If you wish to get logs from remote machine you should uncomment
# this and comment the above source line.
#
#source s_net { tcp(ip(127.0.0.1) port(1000)); };

########################
# Destinations
########################
# First some standard logfile
#
destination d_auth { file("/var/log/auth.log"); };
destination d_cron { file("/var/log/cron.log"); };
destination d_daemon { file("/var/log/daemon.log"); };
destination d_kern { file("/var/log/kern.log"); };
destination d_lpr { file("/var/log/lpr.log"); };
destination d_mail { file("/var/log/mail.log"); };
destination d_syslog { file("/var/log/syslog"); };
destination d_user { file("/var/log/user.log"); };
destination d_uucp { file("/var/log/uucp.log"); };

# This files are the log come from the mail subsystem.
#
destination d_mailinfo { file("/var/log/mail.info"); };
destination d_mailwarn { file("/var/log/mail.warn"); };
destination d_mailerr { file("/var/log/mail.err"); };

# Logging for INN news system
#
destination d_newscrit { file("/var/log/news/news.crit"); };
destination d_newserr { file("/var/log/news/news.err"); };
destination d_newsnotice { file("/var/log/news/news.notice"); };

# Some `catch-all' logfiles.
#
destination d_debug { file("/var/log/debug"); };
destination d_error { file("/var/log/error"); };
destination d_messages { file("/var/log/messages"); };

# The root's console.
#
destination d_console { usertty("root"); };

# Virtual console.
#
destination d_console_all { file(`tty10`); };

# The named pipe /dev/xconsole is for the nsole' utility. To use it,
# you must invoke nsole' with the -file' option:
#
# $ xconsole -file /dev/xconsole [...]
#
destination d_xconsole { pipe("/dev/xconsole"); };

# Send the messages to an other host
#
#destination d_net { tcp("127.0.0.1" port(1000) log_fifo_size(1000)); };

# Debian only
destination d_ppp { file("/var/log/ppp.log"); };

########################
# Filters
########################
# Here's come the filter options. With this rules, we can set which
# message go where.

filter f_dbg { level(debug); };
filter f_info { level(info); };
filter f_notice { level(notice); };
filter f_warn { level(warn); };
filter f_err { level(err); };
filter f_crit { level(crit .. emerg); };

filter f_debug { level(debug) and not facility(auth, authpriv, news,
mail); };
filter f_error { level(err .. emerg) ; };
filter f_messages { level(info,notice,warn) and
not facility(auth,authpriv,cron,daemon,mail,news); };

filter f_auth { facility(auth, authpriv) and not filter(f_debug); };
filter f_cron { facility(cron) and not filter(f_debug); };
filter f_daemon { facility(daemon) and not filter(f_debug); };
filter f_kern { facility(kern) and not filter(f_debug); };
filter f_lpr { facility(lpr) and not filter(f_debug); };
filter f_local { facility(local0, local1, local3, local4, local5,
local6, local7) and not filter(f_debug); };
filter f_mail { facility(mail) and not filter(f_debug); };
filter f_news { facility(news) and not filter(f_debug); };
filter f_syslog3 { not facility(auth, authpriv, mail) and not
filter(f_debug); };
filter f_user { facility(user) and not filter(f_debug); };
filter f_uucp { facility(uucp) and not filter(f_debug); };

filter f_cnews { level(notice, err, crit) and facility(news); };
filter f_cother { level(debug, info, notice, warn) or facility(daemon,
mail); };

filter f_ppp { facility(local2) and not filter(f_debug); };
filter f_console { level(warn .. emerg); };

########################
# Log paths
########################
log { source(s_src); filter(f_auth); destination(d_auth); };
log { source(s_src); filter(f_cron); destination(d_cron); };
log { source(s_src); filter(f_daemon); destination(d_daemon); };
log { source(s_src); filter(f_kern); destination(d_kern); };
log { source(s_src); filter(f_lpr); destination(d_lpr); };
log { source(s_src); filter(f_syslog3); destination(d_syslog); };
log { source(s_src); filter(f_user); destination(d_user); };
log { source(s_src); filter(f_uucp); destination(d_uucp); };

log { source(s_src); filter(f_mail); destination(d_mail); };
#log { source(s_src); filter(f_mail); filter(f_info);
destination(d_mailinfo); };
#log { source(s_src); filter(f_mail); filter(f_warn);
destination(d_mailwarn); };
#log { source(s_src); filter(f_mail); filter(f_err);
destination(d_mailerr); };

log { source(s_src); filter(f_news); filter(f_crit);
destination(d_newscrit); };
log { source(s_src); filter(f_news); filter(f_err);
destination(d_newserr); };
log { source(s_src); filter(f_news); filter(f_notice);
destination(d_newsnotice); };
#log { source(s_src); filter(f_cnews); destination(d_console_all); };
#log { source(s_src); filter(f_cother); destination(d_console_all); };

#log { source(s_src); filter(f_ppp); destination(d_ppp); };

log { source(s_src); filter(f_debug); destination(d_debug); };
log { source(s_src); filter(f_error); destination(d_error); };
log { source(s_src); filter(f_messages); destination(d_messages); };

log { source(s_src); filter(f_console); destination(d_console_all);
destination(d_xconsole); };
log { source(s_src); filter(f_crit); destination(d_console); };

# All messages send to a remote site
#
#log { source(s_src); destination(d_net); };

###
# Include all config files in /etc/syslog-ng/conf.d/
###
@include "/etc/syslog-ng/conf.d/*.conf"




Best Regards
Cosmin Neagu

Post by Fabien Wernli
Hi,
The "strange" messages you're seeing are MARK signals generated indeed by
syslog-ng.
The fact that you see them in your email means for some reason that your log
path logic fails to do what you want.
Could you please post the full configuration, especially the source
definitions which were missing in your first email. ?
Cheers
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

[Fabien Wernli]

Hi,

Post by Cosmin Neagu
source s_src {
system();
internal();
};

reading the docs on "mark-mode()" you can see that it's set to "internal"
for "program()" destinations. Also: "When internal mark mode is selected,
internal source should be placed in the log path as this mode does not
generate mark by itself at the destination".

You *did* place "internal()" into the log path through "s_src" which
explains why the mark messages *are* routed through the logpath containing
your email program.

Now you have to find out why the filter isn't working.

What I'd also do if I were you is look up the native "smtp()" destination
which could simplify your config.

Cheers

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

[Scheidler, Balázs]

Also you can set mark-mode() to none which filters out all mark messages.

Post by Fabien Wernli
Hi,

Post by Cosmin Neagu
source s_src {
system();
internal();
};

reading the docs on "mark-mode()" you can see that it's set to "internal"
for "program()" destinations. Also: "When internal mark mode is selected,
internal source should be placed in the log path as this mode does not
generate mark by itself at the destination".
You *did* place "internal()" into the log path through "s_src" which
explains why the mark messages *are* routed through the logpath containing
your email program.
Now you have to find out why the filter isn't working.
What I'd also do if I were you is look up the native "smtp()" destination
which could simplify your config.
Cheers
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

[Cosmin Neagu]

Yeap, actually after reading the doc, this is how I fixed it

Best Regards
Cosmin Neagu

Post by Scheidler, Balázs
Also you can set mark-mode() to none which filters out all mark messages.
Hi,

Post by Cosmin Neagu
source s_src {
system();
internal();
};

reading the docs on "mark-mode()" you can see that it's set to "internal"
for "program()" destinations. Also: "When internal mark mode is selected,
internal source should be placed in the log path as this mode does not
generate mark by itself at the destination".
You *did* place "internal()" into the log path through "s_src" which
explains why the mark messages *are* routed through the logpath containing
your email program.
Now you have to find out why the filter isn't working.
What I'd also do if I were you is look up the native "smtp()" destination
which could simplify your config.
Cheers
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq