logging - logstash grok pattern not functioning as expected -

-

i having interesting problem.

after using http://grokdebug.herokuapp.com/ works well. finding following message, though works on above site, , works seemingly identical messages, not working expected.

this pattern:

%{monthday}-%{month}-%{year} %{time} %{prog:program}: %{loglevel:loglevel}: %{user:from} %{ip:ip}\#%{int:port} \(%{greedydata:request}\): %{word:stage}(\s|\s\s)%{greedydata:drop_reason} %{iporhost:to}/%{int:subnet}

this output:

{ "message" => "03-feb-2014 21:33:51.867 queries: info: client 123.123.123.123#57710 (some.dns.server.1.1.1.1.in-addr.arpa): drop  response 231.231.231.0/24", "@version" => "1", "@timestamp" => "2014-02-06t00:51:04.240z", "type" => "ns_query", "host" => "ns2", "path" => "/tmp/named-query.log.29" }

i have seen function expected, odd these in particular not working, if has ideas may doing incorrect please let me know.

below see internals of actual patterns file comparing against:

# parse time stamp whis odd time stamp dns_time_stamp %{monthday}-%{month}-%{year} %{time} # create header, beginning of each line of log dns_header %{dns_time_stamp:log_timestamp} %{prog:program}: %{loglevel:loglevel}: %{user:from} %{ip:ip}\#%{int:port} # different queires dns_query_1 %{dns_header} \(%{iporhost:request}\): %{word:stage}: %{iporhost:request2} %{word:rq_where} %{word:r_type} %{data:flags} \(%{iporhost:serviced_by}\) dns_query_2 %{dns_header} \(%{greedydata:request}\): %{word:stage}: %{greedydata:request2} %{word:rq_where} %{word:r_type} %{data:flags} \(%{greedydata}:serviced_by\) # different drop/slip/etc dns_drop_1 %{dns_header} \(%{iporhost:request}\): %{word:stage} %{greedydata:drop_reason} %{iporhost:to}/%{int:subnet} %{iporhost:requester} %{greedydata:dr_type}  \(%{base16num:request_id}\) dns_drop_2 %{dns_header} \(%{iporhost:request}\): %{word:stage}(\s|\s\s)%{greedydata:drop_reason} %{iporhost:to}/%{int:subnet} %{greedydata:dr_type}  \(%{base16num:request_id}\) dns_drop_3 %{dns_header} \(%{iporhost:request}\): %{word:stage}(\s|\s\s)%{greedydata:drop_reason} %{iporhost:to}/%{int:subnet} dns_drop_4 %{dns_header} \(%{greedydata:request}\): %{word:stage}(\s|\s\s)%{greedydata:drop_reason} %{iporhost:to}/%{int:subnet} dns_drop_5 %{dns_header} \(%{greedydata:request}\): %{word:stage}(\s|\s\s)%{greedydata:drop_reason} %{iporhost:to}/%{int:subnet} %{greedydata:dr_type}  \(%{base16num:request_id}\) # bringing them ease of use in logstash dns_query (%{dns_query_1}|%{dns_query_2}) dns_drop (%{dns_drop_1}|%{dns_drop_2}|%{dns_drop_3}|%{dns_drop_4}|%{dns_drop_5}|) # pattern call in logstash config dns (%{dns_query}|%{dns_drop})


Comments

Post a Comment

Popular posts from this blog

python - Subclassed QStyledItemDelegate ignores Stylesheet -

-
i'm trying subclass qstyleditemdelegate remove focus rectangle in qcombobox . even though i'm calling base implementation of paint function , nothing else, result different. looks if parts of style-sheet influence bounding box of item taken consideration. class pstyleditemdelegate(qstyleditemdelegate): def __init__(self, *args, **kwds): super(pstyleditemdelegate, self).__init__(*args, **kwds) def paint(self, *args, **kwargs): qstyleditemdelegate.paint(*args, **kwargs) what have paint unmodified qstyleditemdelegate ? as suggested tried replacing pyside pyqt4 , works, appears bug. update pyside 1.1.2 1.2.1 result same. unfortunately switch breaks other parts of code, if there no other suggestions i'm going accept answer. edit bug tracked here
Read more

java - HttpClient 3.1 Connection pooling vs HttpClient 4.3.2 -

-
i have used httpclient3.1(java) connection pooling below : import org.apache.commons.httpclient.httpclient; import org.apache.commons.httpclient.multithreadedhttpconnectionmanager; import org.apache.commons.httpclient.params.httpclientparams; import org.apache.commons.httpclient.params.httpmethodparams; import org.apache.log4j.logger; public class httpclientmanager { private static logger logger = logger.getlogger(httpclientmanager.class); private static multithreadedhttpconnectionmanager connectionmanager = new multithreadedhttpconnectionmanager(); private static httpclientparams clientparams = new httpclientparams(); static { try { clientparams.makelenient(); clientparams.setauthenticationpreemptive(false); clientparams.setparameter(httpmethodparams.so_timeout, new integer(30000)); // set so_timeout clientparams.setparameter(httpmethodparams.head_body_check_timeout, new integer(3000)); // head responses
Read more

SQL: Divide the sum of values in one table with the count of rows in another -

-
is possible to, in single query, sum bunch of values in 1 table, , divide count of rows in another? there common id/key in both tables... fyi, win points least info provided in question. answer it's possible...here psuedo code since i'm guessing @ tables , columns select summing / counting whynot (select id, sum(whatever) summing where_ever ) inner join (select id, count(1) counting what_ever)b on a.id = b.id question = 'vague' group a.id
Read more