Difference between revisions of "Logstash"
From Initech Technical Wiki
| Line 1: | Line 1: | ||
For various reasons, not least of which is timestamp normalisation and/or everything normalisation for that matter i chose to use a standard udp and tcp input method for logstash and skip the in-built syslog one. This meant that i needed to reinvent the wheel somewhat when it came to grok filters however. So here i have documented the grok filters that I built and why. | For various reasons, not least of which is timestamp normalisation and/or everything normalisation for that matter i chose to use a standard udp and tcp input method for logstash and skip the in-built syslog one. This meant that i needed to reinvent the wheel somewhat when it came to grok filters however. So here i have documented the grok filters that I built and why. | ||
| − | |||
| − | |||
== input == | == input == | ||
| Line 29: | Line 27: | ||
<pre> | <pre> | ||
| − | if [message] =~ /Access Server SDK: No log writers/ or | + | if [message] =~ /Access Server SDK: No log writers/ or [message] =~ /Warning: Could not obtain lock on \/mnt\/oracle\/ohs/ or [message] =~ /last message repeated [0-9]+ times/ { |
| − | |||
| − | |||
drop { } | drop { } | ||
} | } | ||
Revision as of 21:03, 18 November 2014
For various reasons, not least of which is timestamp normalisation and/or everything normalisation for that matter i chose to use a standard udp and tcp input method for logstash and skip the in-built syslog one. This meant that i needed to reinvent the wheel somewhat when it came to grok filters however. So here i have documented the grok filters that I built and why.
input
input {
udp {
port => 514
type => syslog
}
tcp {
port => 514
type => syslog
}
}
filter
filter {
if [type] == "syslog" {
Delete useless log messages
if [message] =~ /Access Server SDK: No log writers/ or [message] =~ /Warning: Could not obtain lock on \/mnt\/oracle\/ohs/ or [message] =~ /last message repeated [0-9]+ times/ {
drop { }
}
Match Cisco ACL logs
grok {
match => { "message" => "%{SYSLOG5424PRI}%{NUMBER:sequence}: %{SYSLOGTIMESTAMP:log_timestamp}: \%SEC-6-IPACCESSLOGP: list %{INT:acl_number} %{WORD:action} %{WORD:protocol} %{IP:src_ip}\(%{NUMBER:src_port}\) \-\> %{IP:dst_ip}\(%{NUMBER:dst_port}\), %{INT:packets} packet" }
add_tag => ["grok_match", "cisco_acl_message"]
remove_field => [ "message" ]
}