Typical syslog-ng use-cases
CC-IN2P3
Fabien Wernli
Outline
talk {
infrastructure();
altertative { ... };
channel {
storing(); alerting(); enriching();
};
channel {
misc (); automation(); monitoring();
};
channel {
appendix { ... };
flags(if-time-permits);
};
};
Architecture
Architecture
Storing Logs
Storing Logs
Elasticsearch
destination d_elasticsearch {
elasticsearch2(
client-lib-dir("/usr/share/elasticsearch/plugins/search-guard-5/*.jar:/usr/share/elasticsearch/lib/")
client-mode("https")
concurrent-requests("16")
disk-buffer(
dir("/var/lib/syslog-ng-disq/")
disk-buf-size(53687091200)
mem-buf-size(1073741824)
)
flush-limit('1024')
index("${__es_index:-syslog}-${YEAR}.${MONTH}.${DAY}")
port('9200')
server("node01 node02 node03 node04 node05")
java_keystore_filepath("/etc/syslog-ng/coloss-analyzer-keystore.jks")
java_keystore_password("terces")
java_truststore_filepath("/etc/elasticsearch/coloss/truststore.jks")
java_truststore_password("terces")
http_auth_type("clientcert")
skip-cluster-health-check("yes")
template("$(format-json -s all-nv-pairs --rekey .SDATA.* --shift 7)")
time-zone("UTC")
type("${__es_type:-syslog}")
);
};
Alerting: routing
Routing
Enriching Logs
Enriching Logs
Mailbox source - but why?
Seriously?
- sure, for syslog-unfriendly tools
- and quick'n'dirty solutions
- "You Know, for Search"
- Ex: appliances, electrical equipment, ...
- Ex: yum, (ana)cron, ...
Mailbox source - Examples