Multi platform kotlin client for Elasticsearch & Opensearch with easily extendable Kotlin DSLs for queries, mappings, bulk, and more.
| KT Search Manual | Previous: KNN Search | Next: Using Kotlin Scripting |
| Github | © Jilles van Gurp |
This library includes Kotlin DSLs for querying, mapping and other functionality in Elasticsearch. Elasticsearch has a very rich set of features and new ones are being added with every release. Keeping up with that is hard and instead of doing that, we designed the Kotlin DSL to be easily extensible so that users don’t get stuck when e.g. a property they need is missing in the Kotlin DSL or when we have simply not gotten around to supporting a particular feature.
All of the DSLs in this library are based on json-dsl. This is a library that I wrote specifically to create the DSLs for kt-search. But it can of course also be used for other JSON dialects.
The key feature in json-dsl is that it uses a MutableMap for storing property values. This enables you
to define classes with properties that delegate storing their value to this map. For anything that your
classes don’t implement, the user can always write to the map directly using a simple put.
To create your own JsonDSL, all you need to do is extend the JsonDsl class.
For example, consider this bit of Json:
{
"foo": "bar",
"bar": {
"xxx": 1234,
"yyy": true
}
}
To create a DSL for this, you simply create a new class:
// first we create support for representing the bar object
class BarDsl : JsonDsl(
// this is the default
namingConvention = PropertyNamingConvention.AsIs
) {
// you use property delegation to define properties
var xxx by property<Long>()
var yYy by property<Boolean>()
}
class FooDSL : JsonDsl(
) {
var foo by property<String>()
var bar by property<BarDsl>()
// calling this function is nicer than doing
// bar = BarDsl().apply {
// ....
// }
// but both are possible of course
fun bar(block: BarDsl.() -> Unit) {
this["bar"] = BarDsl().apply(block)
}
}
fun foo(block: FooDSL.() -> Unit) = FooDSL().apply(block)
foo {
foo = "Hello World"
bar {
xxx = 123
yYy = false
// you can just improvise things that aren't part of your DSL
this["zzz"] = listOf(1, 2, 3)
this["missingFeature"] = JsonDsl().apply {
this["another"] = " field"
// if you need to you can override naming per field
put(
key = "camelCasing",
value = "may be forced",
namingConvention = PropertyNamingConvention.AsIs
)
// and you can use class properties
// if you want to keep things type safe
put(FooDSL::foo, "bar")
// and of course you can mix this with string literals
// via the RawJson value class
this["raw"] = RawJson("""{"foo":"bar"}""")
}
// you can also use withJsonDsl as a short hand
// for JsonDsl().apply
this["anotherObject"] = withJsonDsl {
this["value"] = "Priceless!"
// you can go completely schemaless if you need to
this["list"] = arrayOf(1,2,3)
this["list2"] = listOf(1,2,3)
// json list elements don't have to be of the
// same type even
this["map"] = mapOf("foo" to listOf(1,"2",3.0,
RawJson("""{"n":42}""")))
}
}
}.let {
println(it.json(pretty = true))
}
This prints:
{
"foo": "Hello World",
"bar": {
"xxx": 123,
"yYy": false,
"zzz": [
1,
2,
3
],
"missingFeature": {
"another": " field",
"camelCasing": "may be forced",
"foo": "bar",
"raw": {"foo":"bar"}
},
"anotherObject": {
"value": "Priceless!",
"list": [
1,
2,
3
],
"list2": [
1,
2,
3
],
"map": {
"foo": [
1,
"2",
3.0,
{"n":42}
]
}
}
}
}
Refer to the documentation of json-dsl for more details on how to use it. Or look at the implementation of the various DSLs in kt-search.
Most of the DSLs in Elasticsearch use snake casing (lower case with underscores). Of course, this goes
against the naming conventions in Kotlin, where using camel case is preferred. You can configure the naming
convention via the namingConvention parameter in JsonDSL. It defaults to snake casing as this is so pervasive
in the Elasticsearch DSLs. If you don’t want this, use the AsIs strategy. Or override the property name of
your properties.
Both the SearchDSL and the IndexSettingsAndMappingsDSL use the same names as the Elasticsearch DSLs
they model where-ever possible. Exceptions to this are Kotlin keywords and functions that are part of the
JsonDsl parent class. For example, size is part of the Map interface it implements and therefore we
can’t use it to e.g. specify the query size attribute.
As an example, we’ll show how the term query implementation is implemented in kt-search.
class TermQueryConfig : JsonDsl() {
var value by property<String>()
var boost by property<Double>()
}
@SearchDSLMarker
class TermQuery(
field: String,
value: String,
termQueryConfig: TermQueryConfig = TermQueryConfig(),
block: (TermQueryConfig.() -> Unit)? = null
) : ESQuery("term") {
init {
put(field, termQueryConfig, PropertyNamingConvention.AsIs)
termQueryConfig.value = value
block?.invoke(termQueryConfig)
}
}
fun QueryClauses.term(
field: KProperty<*>,
value: String,
block: (TermQueryConfig.() -> Unit)? = null
) =
TermQuery(field.name, value, block = block)
fun QueryClauses.term(
field: String,
value: String,
block: (TermQueryConfig.() -> Unit)? = null
) =
TermQuery(field, value, block = block)
The Elasticsearch query dsl has this convention of wrapping various types of queries with a single
field object where the object key is the name of the query. Therefore, TermQuery extends EsQuery, which
takes care of this.
All the query implementations have convenient extension functions on SearchDSL. This ensures that you
can easily find the functions in any place that has a receiver block for SearchDSL and makes for a nice
developer experience when using the DSL.
Term queries always have at least a field name and a value. This is why these are constructor
parameters on TermQueryConfig. Since specifying additional configuration is optional, the block in both
term functions defaults to null.
Since, mostly you will have Kotlin data classes for your document models, there is a variant of the term
function that takes a KProperty. This allows you to use property references.
Here’s an example of how you can use the term query:
SearchDSL().apply {
data class MyDoc(val keyword: String)
query = bool {
should(
term("keyword", "foo") {
boost = 2.0
},
// we can use property references
// instead of string literals
term(MyDoc::keyword, "foo") {
boost = 2.0
},
// the block is optional
term(MyDoc::keyword, "foo")
)
}
}.json(pretty = true)
If you end up writing your own queries, of course please consider making a pull request.
Currently the client uses ktor-client and this is of course fine. However, we use a simple wrapper for this that you can write alternative implementations for. This is probably a case of severe YAGNI (You Aint Gonna Need It), But if you need this for some reason or don’t want to use ktor client, it’s there.
All you need to do for this is implement the RestClient interface.
The search client is of course a bit opinionated in how it is implemented and it picks an httpclient and serialization framework that not everybody might agree with. If this bothers you, you can use just the search DSL and easily build your own client using that.
There are cases where enums must be serialized and the text values are not valid enum values or
they don’t follow Java/Kotlin naming conventions. In these cases implement CustomValue interface
like in this example:
enum class Conflict(override val value: String) : CustomValue<String> {
ABORT("abort"),
PROCEED("proceed");
}
This interface can be of course implemented by any class not only enums.
| KT Search Manual | Previous: KNN Search | Next: Using Kotlin Scripting |
| Github | © Jilles van Gurp |