Log file formats

Log Monitoring can read and analyze:

  • Windows format event logs.
    System, Security, and Application logs are automatically discovered on hosts. Other custom event-log format logs can be added manually on the process group level.

  • Any plain-text log file as long as it contains a time stamp that meets these basic requirements:

    • The file must be encoded as UTF-8 or UTF-16. All other encoded files will be recognized as binary.

    • A time stamp must appear at the beginning of each log entry.

    • The time stamp date can be separated using either of the following:
      (space)
      / (slash)
      - (dash)
      . (period)
      T (Combined date and time in UTC, ISO 8601 format)
      , (comma)
      @ (ampersand)

    • For the time stamp date, either month abbreviations or full names can be used in the date format.
      By default, Log Monitoring recognizes only English month abbreviations and months represented by number (1-12 and jan-dec).

    • Time stamp must include BOTH, the date and time.

    • Time stamp time is in the following format:

  [0-9]{1,2}:[0-9]{2}:[0-9]{2}((\.|,)[0-9]{1,9})?( *AM|PM)? *((GMT|UTC)?[+-][0-9]{2,4})?(GMT|CEST|CET|OTHER_TZ_ABBREVIATION)?
  • A JSON file.
    The time stamp in a JSON file is automatically detected through the time or timestamp tag and it must be in the following time stamp format: ::*_year_*:-:*_month_*:-:*_day_*:T:*_time_*:0::.
    For example: 2018-02-28T16:17:50.000

    Also, the JSON file must meet the following conditions:

    • The date must be in UTC standard.
    • The file cannot contain any headers.
    • In the file, each log entry is represented by one JSON object in one line.
      For example:
      { "log" : "content = 0, t = 1000000000", "time":"2019-10-09T14:45:00.000000Z", "stream" : "stderr" }
      { "log" : "content = 0, t = 1000000000", "time":"2019-10-09T14:46:00.000000Z", "stream" : "stderr" }
      

Examples of valid log file date formats

2018 04 06
2018/04/06
2018-04-06
2018.04.06
2018-04-06T18:46:19Z

Examples of valid log file time formats

12:23:34.123
12:23:34.123GMT+0100
12:23:34.123 GMT+0100
0:00:00 GMT
0:00:00   GMT+0100
00:00:00
12:13:01+0100
12:13:02.123
12:13:03.123123+0100
12:13:02,123
12:13:03,123123+0100
12:13:04GMT
12:13:05GMT+0100
12:13:06GMT+01
12:13:09+0100
12:13:10+01
12:13:12+0200
12:13:13.123pm
12:13:14.123 AM
12:13:15.123PM+01
12:13:16.123 AM+02
12:13:17CEST
12:13:18 CET
00:13:19

Examples of valid log file time stamps

2018-04-06 09:54:04.839 UTC
2018-04-06 11:01:19,625
2018/04/06 11:06:23 UTC
Apr  6 12:23:52
Apr-6 13:35:57.621

Incorrect date formats

Any log file containing an invalid time stamp will generate an Incorrect date format error and will not be analyzed or stored (only the file status will be reported).