Commit 20d4e2a0 authored by Lena Heizmann's avatar Lena Heizmann
Browse files

Update readme

parent 172ac836
# Elasticsearch resolver
This streaming app consumes data from a `geoconcordance-request` kafka topic and search in [ElasticSearch](https://www.elastic.co/de/) for possible matches. The result is streamed to the `geoconcordance-response` topic. The app is used for our [api](https://source.dodis.ch/histhub/api) to give possible answers to http requests.
This streaming app consumes data from a `geoconcordance-request` kafka topic and searches in [ElasticSearch](https://www.elastic.co/de/) for possible matches. The result is streamed to the `geoconcordance-response` topic. The app is used for our [api](https://source.dodis.ch/histhub/api) to give possible answers to http requests.
## How it works
The app first requests the resource behind the url from elastic search. The result of this request is used to do a second search to find similar documents. Similarity is defined over the used search parameters. The two-step procedure is an overhead. But at the moment we can't solve this.
The app first requests the resource behind the url from elastic search. The result of this request is used to do a second search to find similar documents. Similarity is defined through the used search parameters. The two-step procedure is an overhead, but at the moment we don't have a better solution.
**Steps**
1. (uri) --> search in ES --> (resource with attributes)
2. (attributes) --> search in ES --> possible matches
## Config
You can configure the app over the `options.fuzziness = 3` and `options.distance = '15000` parameter on creation time. You can override those defaults with the kafka message `"config" { "resolverelasticsearch": {"fuzziness": 2, "distance":"15000"}}`. So we can pass parameters from the request to the app.
You can configure the app through the `options.fuzziness = 3` and `options.distance = '15000` parameter on creation time. You can override those defaults with the kafka message `"config" { "resolverelasticsearch": {"fuzziness": 2, "distance":"15000"}}`. So we can pass parameters from the request to the app.
### Distance
The distance defines a circle from the location of the base-resource to search in. Because of different systematics of the providers it's a good idea to not select a too narrow distance.
......@@ -18,7 +18,7 @@ The distance defines a circle from the location of the base-resource to search i
The [fuzziness](https://www.elastic.co/guide/en/elasticsearch/guide/current/fuzziness.html) is an implementation of [levenstein](https://en.wikipedia.org/wiki/Levenshtein_distance) algorithm to provide some tolerance for different writings of the name of the geo-entity.
## Algorithm
The query to preform the search is simple, but provides good results. It looks for entities with a similar name, not of the same provider in a restricted area.
The query to preform the search is simple, but provides good results. It looks for entities with a similar name, not from the same provider, and in a restricted area.
```json
{
"index": "geolinker",
......@@ -64,7 +64,7 @@ docker push source.dodis.ch:4577/histhub/resolver-elasticsearch
```
## CD
We have a build pipeline in gitlab. So manually building of the image is not longer necessary.
We have a build pipeline in gitlab. So manually building the image is no longer necessary.
## Deploy to k8
This streaming app is part of the [nodejs helm chart](https://source.dodis.ch/histhub/charts/tree/master/charts/data). More detailed information you can find in the [deployment-geolinker](https://source.dodis.ch/histhub/deploy-geolinker/wikis/deployment-manual#resolver)
This streaming app is part of the [nodejs helm chart](https://source.dodis.ch/histhub/charts/tree/master/charts/data). You can find more detailed information in the [deployment-geolinker](https://source.dodis.ch/histhub/deploy-geolinker/wikis/deployment-manual#resolver).
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment