How do I troubleshoot a dateparsefailure when using Logstash's date plugin?

Question

I'm getting a date parse failure when trying to parse a log that has the following format:

172.20.21.10 - - [29/Mar/2022:12:41:27 -0400] "some invalid request" 400 226

The event structure looks like this:

{
        "request" => "/um/",
           "auth" => "-",
          "ident" => "-",
      "@metadata" => {
        "path" => "C:/net-logs/m4-httpd-logs/wordpress_access.log-20220403",
        "host" => "M6"
    },
           "verb" => "GET",
        "message" => "172.20.21.10 - - [29/Mar/2022:12:44:36 -0400] \"GET /um/ HTTP/1.1\" 404 17787",
           "path" => "C:/net-logs/m4-httpd-logs/wordpress_access.log-20220403",
          "bytes" => "17787",
       "response" => "404",
       "clientip" => "172.20.21.10",
       "@version" => "1",
           "host" => "M6",
    "httpversion" => "1.1",
      "timestamp" => "29/Mar/2022:12:44:36 -0400"
}

To convert the timestamp field from text to a date I have a filter in the logstash configuration file that looks like this:

filter {
        grok {
              match => {"message" => "%{COMMONAPACHELOG}"}
        }
        mutate {
                remove_field => [ "@timestamp"]
        }
        date {
              match => ["timestamp", "dd/MM/yyyy:HH:mm:ss Z"]
              target => ["timestamp"]
        }
}

I have used the correct syntax for the format according to the documentation for the date filter. I'm not sure whats causing the parse failure since the error msg does not show exactly why the parse failed. It looks like this:

{
        "request" => "/wp-includes/IXR/login",
           "auth" => "-",
          "ident" => "-",
      "@metadata" => {
        "path" => "C:/net-logs/m4-httpd-logs/wordpress_access.log-20220403",
        "host" => "M6"
    },
           "verb" => "GET",
        "message" => "172.20.21.10 - - [29/Mar/2022:12:47:57 -0400] \"GET /wp-includes/IXR/login HTTP/1.1\" 404 17787",
           "tags" => [
        [0] "_dateparsefailure"
    ],
           "path" => "C:/net-logs/m4-httpd-logs/wordpress_access.log-20220403",
       "response" => "404",
          "bytes" => "17787",
       "clientip" => "172.20.21.10",
       "@version" => "1",
           "host" => "M6",
    "httpversion" => "1.1",
      "timestamp" => "29/Mar/2022:12:47:57 -0400"
}

Answer 1

Fixed by creating a mapping for the timestamp field:

{
  "index_patterns": "httpd-*",
  "mappings": {
    "dynamic": true,
    "properties": {
        "request": {
          "type": "text"
        },
        "auth": {
          "type": "text"
        },
        "ident": {
          "type": "text"
        },
        "verb": {
          "type": "text"
        },
        "response": {
          "type": "text"
        },
        "bytes": {
          "type": "integer"
        },
        "clientip": {
          "type": "ip"
        },
        "httpversion": {
          "type": "text"
        },
        "timestamp": {
          "type": "date",
          "format": "dd/MMM/yyyy:HH:mm:ss Z"
        }
    }
  }