3

I configured a Fortimail to send its log messages to the Logstash, however, when I view the logs on the kibana Dashboard.

I want to split the message field on several fields such as : time, device_id, from, to ...

see this picture:

enter image description here

I do not know how to proceed in order to customize the fields.

Can give me some ideas to know what files to configure?

5 Answers 5

4

Alcazar's correct, but he gave you next to nothing to go on.

Effectively, in your logstash config, you'll want to set up a filter{} section that describes how to break the log into its individual fields. Note that this is done as logs are ingested, not after they're already in elastic search.

See the documentation for the grok filter for more information.

1

Thanks @Joe. I can split message to several fields (for example, method, duration, controller) by configure logstash.

Here is my realistic example, let’s read these logs from a file:

 input {
  file {
    path => "/var/log/http.log"
  }
}
filter {
  grok {
    match => { "message" => "%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}" }
  }
}
0

You need to do that in your logstash config using something like the kvp filter

0

Check if the latest (June 2019) Elastic 7.2 release addresses your question:

Elastic Logs, version 7.2.0, is available on the Elasticsearch Service, or as part of the default distribution of Elastic Stack

That includes, for the Logs App in Kibana, field pinning, also known as custom columns.

This allows you to specify additional fields to show up in the log viewer, alongside the default fields:

https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt2db52cfca41e7693/5cfff59c6a96af7017dfe507/image1-2.png

0

Take a look at Logstash's grok filter and grok patterns in general. Its like regex with the addition that there are already "built-in" patterns you can use for common data (see https://github.com/elastic/logstash/blob/v1.4.2/patterns/grok-patterns)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.