- Is it possible to create sockets with syslog-ng similar to how you can do so with syslogd? The reason for this being that I’m running some applications chrooted, and need to open a /dev/log socket that is in that chroot-jail.Of course you can. Just add a source:
source local { unix-stream("/dev/log"); internal(); };
source jail1 { unix-stream("/jail/dns/dev/log"); };
source jail2 { unix-stream("/jail/www/dev/log"); };
You can do this by using a single source:
source local {
unix-stream("/dev/log");
internal();
unix-stream("/jail/dns/dev/log");
};
Note that postfix appears to need a log socket in it’s chroot jail, or it’s logging will stop when you reload syslog-ng:
source postfix { unix-stream("/var/spool/postfix/dev/log" keep-alive(yes)); };
- Directories with names like “Error”, “SCSI”, “”, are showing up in the directory that holds the syslogs for the different hosts that we monitor.Has anyone seen these random directories? Any suggestions on how to deal with them?From the description it’s apparent that logs are being stored in your filesystem with a macro similar to this:
destination std { file( "/var/log/$HOST/$FACILITY"); };
…so that you have directories created with the value of $HOST. This is bad. The host entry in syslog messages is often set to a bad value, especially with messages originating from the UNIX kernel, like SCSI error messages.
The best fix for this is to *never* create files or directories based on unfiltered input from the network (You’d do well to remember that in general). Set the option keep_hostname to (no), and syslog-ng will always replace the hostname field (possibly using DNS, so make sure your local caching DNS is setup correctly).
Here’s the way to keep the hostnames in the log files but ALSO log safely to the filesystem:
options {
long_hostnames(off);
keep_hostname(yes);
use_dns(no);
};
source src {
internal();
unix-stream("/dev/log" keep-alive(yes));
};
# set it up
destination logip {
file("/archive/logs/HOSTS/$HOST_FROM/$FACILITY/$YEAR$MONTH/$FACILITY$YEAR$MONTH$DAY"
owner(syslog-ng) group(syslog-ng) perm(0600) dir_perm(0700) create_dirs(yes)
template("$DATE $FULLHOST $PROGRAM $TAG [$FACILITY.$LEVEL] $MESSAGE\n") );
};
# log it
log {
source(src);
destination(logip);
};
Since you don’t use DNS, your $HOST_FROM directory name will be an IP, but since you keep_hostnames(yes) you’ll still have the hostname AS SENT inside the actual logfile. How’s that for a good setup? I quite like it! 😉If still you really want to use hostnames for directory or file names, read on:
When still using hostnames (from the DNS) for directory names, the author of this FAQ didn’t have garbled $HOST macros go away until he modified all clients to run syslog-ng and transfer over TCP. Both steps might not be required, syslog-ng over UDP might be sufficient, though there’s little reason *not* to use TCP. Modern TCP/IP stacks are tuned to handle lots of web connections, so even a central host for hundreds of machines can use TCP without issues from the use of TCP alone. There will be I/O problems with trying to commit that many hosts’ logs to disk much sooner under most circumstances.
- DNS: I want to use fully qualified domain names in my logs, I have many different hosts named ns1, or www, and don’t want the logs mixed up. Also, I have a question concerning the use_dns option. Is this a global option only, or is there some way to change this per source or destination?First of all, make sure that you have a reliable DNS cache nearby. Nearby may be on the same host or network segment, or even at your upstream provider. Just make sure that you can reach it reliably. syslog-ng blocks on DNS lookups, so you can stop all logging if you start getting DNS timeouts.
Internal syslog-ng DNS caching has recently been worked on, and reportedly works well. This appears to be a good alternative to running a local caching DNS server (‘dns_cache(yes);’).
The use_dns option can be specified on a per-source basis (so can the keep_hostname option).
See also: the section on hostname options directly below.
- What is with all the “hostname” options?When syslog-ng receives a message it tries to rewrite the hostname it contains unless keep_hostname is true. If the hostname is to be rewritten (e.g. keep_hostname is false), it checks whether chain_hostnames (or long_hostname which is an alias for chain_hostnames) is true. If chain_hostnames is true, the name of the host syslog-ng received the message from is appended to the hostname, otherwise it’s replaced.
So if you have a message which has hostname “server”, and which resolves to “server2”, the following happens:
|
keep_hostname(yes) |
keep_hostname(no) |
| chain_hostname(yes) |
server |
server/server2 |
| chain_hostname(no) |
server |
server2 |
I hope this makes things clear.
- I have this config file:
filter f_local0 { facility(local0); }; filter f_local1 { facility(local1); }; destination df_local1 { file("/mnt/log/$R_YEAR-$R_MONTH-$R_DAY/$SOURCEIP/local.log" template("$FULLDATE <> $PROGRAM <> $MSGONLY\n") template_escape(no)); }; log { source(s_tcp); source(s_internal); source(s_udp); source(s_unix); filter(f_local0); filter(f_local1); destination(df_local1); };
When an event arrives at the system by facility local0 or local1, this one is registered in the file never. Can be some bug of syslog-ng or failure in config?If I understand you correctly then the problem is that you’re using two filters which exclude each other however using more filters they are logically AND-ed. If you want to catch messages from local0 and local1 use a filter like this:
filter f_local01 { facility(local0) or facility(local1); };
- I archive my logs like this:
file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/$FACILITY_$HOST_$YEAR_$MONTH_$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes) );
…and over time my archive takes up lots of space. When will syslog-ng implement compression so that I can compress them automatically?You don’t need syslog-ng to compress them, install bzip2 and run a nightly cronjob like this:
/usr/bin/find /var/log/HOSTS ! -name "*bz2" -type f ! -path "*`/bin/date +%Y/%m/%d`*" -exec /usr/bin/bzip2 {} \;
This might need some explaining: find all non-bzipped files that aren’t from today (syslog-ng might still write to them) and compress them with bzip2. This was tested on Debian GNU/Linux with GNU find version 4.1.7.Submitted by Michael King:
We started with the compression script you have, but changed it to use gzip compression. (Less file space efficient, but more time efficient. B2zip takes approximately 20 minutes to decompress vs 2 or 3 minutes for Gzip), added a quick find and delete for empty directories and files modified more than 14 days old.
# Current policy is:
# Find all non-Archived files that aren't from today, and archive them
# Archive Logs are deleted after 14 days
#
#Changes. Change -mtime +14 to the number of days to keep
# Archive old logs
/usr/bin/find /var/log/HOSTS ! -name "*.gz" -type f ! -path "*`/bin/date +%Y/%m/%d`*" -exec /usr/bin/gzip {} \;
# Delete old archives
find /var/log/HOSTS/ -daystart -mtime +14 -type f -exec rm {} \;
# Delete empty directories
find /var/log/HOSTS/ -depth -type d -empty -exec rmdir {} \;
- My syslog-ng.conf has
destination std { file( "/var/log/$HOST/$YEAR$MONTH/$FACILITY" create_dirs(yes)); };
What happens is if /var/log/$HOST/$YEAR$MONTH does not exist, syslog-ng makes that dir, but it’s owner is root:other. I think because daemon’s effective user ID is root. I want to change dir’s owner. Is this possible?
Yes, you can do this using the owner(), group() and perm() options.
For example:
destination d_file { file("/var/log/$HOST/log" create_dirs(yes)
owner("log") group("log") perm(0600)); };
- a) I have snmptrapd running so that any trap that it receives should be logged to local1. I have a filter taking anything received via local1 to a specific file:
filter snmptrap {facility(local1);};
destination snmptraps { file("/var/log/snmptraps";};
Unfortunately a number of traps are getting cut off at a specific point, and the remainder of the trap ends up in syslog and not in the proper destination.syslog defaults to 1024 byte long messages, but this value is tunable in syslog-ng 1.5 where you can set it to a higher value.
options { log_msg_size(8192); };
b) Andreas Schulze points out: “We are running snmptrapd and syslog-ng 1.5.x under Solaris 8 and observed exactly the same problem.
This doesn’t fix the problem for us. It seems that there is a problem in the syslog(3) implementation at least on Solaris. Maybe on Linux, too. This is important, because snmptrapd feeds its messages via syslog(3) to syslog-ng. So syslog-ng never gets the correct message, because its truncated in libc before syslog-ng receive it.
Our solution was, to patch snmptrapd to log its messages via a local Unix DGRAM socket and use this socket as message source for syslog-ng. This fix the problem and works pretty fine and very stable for more than one year in our environment.
Basically you’re screwed on Solaris, but hopefully other implementations aren’t as brain-dead.
- It seems I have syslog clients with unsynchronized clocks and I have files that were created with the time macros, and their date is wrong. What I want is the files to be created with the time/date they are received.It’s an option. use_time_recvd() boolean.
options { use_time_recvd(true); };
- What conf settings can I use for my syslog-ng.conf file so that messages are written to disk the instant they are received?Add sync(0) to your config file.
options { sync(0); };
- I want to run syslog-ng chrooted and as a non-root user.Use syslog-ng 1.5.x’s own -C and -u flags when starting it.
Gheek.net 
Leave a Reply