.NET Web API with Elastic Search and Kibana

 ElasticSearch

Elasticsearch is a search engine based on the Lucene library. It provides a distributes, multitenany-capable full-test search engine with an HTTP web interface and schema-free JSON documents.

Kibana

Kibana is a source-available data visualization dashboard software for Elasticsearch.

Adding elastic search and kibana to the web API solution is a very simple tasks and it needs only a library to be installed and few configuration changes.

1. Install Serilog.AspNetCore and Serilog.Sinks.Elasticsearch packages from nuget packkage manager in Visual Studio. Serilog.AspNetCore package helps to manage and show the logs using serilog and Serilog.Sinks.Elasticsearch connects with the elasticsearch instance and loads the logs into the elasticsearch.
2. Next is to change the configurations to add elasticsearch to the API code. In the Program.cs file add the below lines to configure elasticsearch with serilog.

3. Now the configurations for serilog and elastic search has to be added in the app settings file.

        The above elasticseacrh instance is running in the docker with 9200 as the default export port.

4. Next is to run the elasticsearch and kibana instances in docker. To run them let us create a docker compose file with all the elasticsearch and kinbana docker configurations. A basic compose file looks like below

5. After creating the compose file if we run docker compose up command, docker pulls the images defined in the docker compose if they are not present in docker and then runs the container.
6. Once the compose is started, open the kibana instance from docker and add the api log pattern in the kibana

Open the menu and select the Stack Management section under Management

We need to add an index pattern to filter all the logs that matches the pattern

Now open Discover from the menu and in which the logs will be listed

Penned by,

Ramya.