Using JSON in Kibana Search

When searching from Kibana, you usually enter the actual query string into the top bar, as we saw in the tutorial. If the query string is not enough for what you need, you also have the option to write JSON on that string.

You can write a JSON object that you would attach to the “query” key when interacting with Elasticsearch in this field, for example:

{ "range": { "numeric": { "gte": 10 } } }

This would be equivalent to writing numeric:>=10 into this field. More often than not, this only makes sense if you need access to options that are only available in the JSON request, but not in the query string.

A warning: if you write JSON query_string into this field (for example, because you want to be able to access lowercase_expanded_terms), Kibana will save the correct JSON for the request, but again show you (after pressing enter) only the “request” part of your JSON. This can be very confusing and of course if you type in the text now and hit enter again, it will also lose the parameters you set via JSON, so this should really be used with care.

Special cases

There are a few other special cases to think about in this section: “I read the entire tutorial, I understood everything, but still my query does not find the data that I expected to receive from it.”

Elasticseach doesn’t find terms in long fields.

This is, in my experience, a fairly common problem and not easy to solve if you don’t know what you are looking for.

Elasticsearch has a parameter ignore_abovethat you can set in the display for each field. This is a numeric value and will cause Elasticsearch to NOT index values for longer than the specified value ignore_abovewhen the document is inserted. The value will still be saved, so when you view the document, you will see it, but you will not be able to find it.

How can I check if this value is set in a field? You need to get the mapping from Elasticsearch by calling <your-elasticsearch-domain>/<your-index-name>/_mapping… In the returned JSON, somewhere there will be a mapping for the desired field, which may look like this:

"fieldName": {
  "type": "string",
  "ignore_above": 15
}

In this case, values over 15 characters are not indexed and cannot be searched.

Example: Using the above mapping, let’s insert two documents into this Elasticsearch:

{ "fieldName": "short string" }
{ "fieldName": "a string longer as ignore_above" }

If you now list all documents (in Kibana or Elasticsearch), you will see that both documents are there and the value of both fields is what you inserted into the string. But if you search now fieldName:longer, you won’t get any results (while fieldName:short will return the first document). Elasticsearch found that the value “string longer than ignore_above” is longer than 15 characters, and therefore it only stores it in the document, but does not index it, so you cannot search for anything in it, since the inverted index for this field will not have the contents of this values.

Search requires a specific field, without which it doesn’t work.

If you can search for example for author:foobut not for foo, then most likely this is a “problem” with your default_field… Elasticsearch precedes the default field before foo. This field can be configured to be different from _all…

Possibly setting the field index.query.default_field was installed to something else and Elasticsearch is not using the field _allwhich could lead to the problem.

It is also possible that the field _all behaves differently than you expected because it was configured in some other way. You can exclude specific fields from a field _all (for example in the above example fieldName could be excluded from indexing in the field _all) or the analysis / indexing options in the field display have been changed _all…

Already, recruitment for the new course stream is open at OTUS “DevOps Practices and Tools”… The translation of this fragment of the article was prepared as part of the recruitment for the course.
We also invite everyone to visit free webinar, where OTUS experts will talk about the situation in the DevOps market and career prospects.
SIGN UP FOR WEBINAR