Integrate Audit trail module with Elasticsearch and Kibana
JSON logger is the default logging module built into Sitefinity CMS. It's possible to integrate Sitefinity's Audit trail module with external logging tools as well.To easily integrate with Elasticsearch and Kibana servers, you will need to first install the ElasticSearchAuditTrailLogger module.
There are 2 options for installing the module:
Elasticsearch is an open-source Java-based web server, used by Sitefinity's Audit trail module for storing and querying logged records, instead of using the textual log files.
During setup, you need to run the Elasticsearch server on the web server machine, on a dedicated machine, on premise, in the cloud, or in a cluster of machines.
Kibana is Elasticsearch's web server that uses static HTML pages and client-side AJAX calls for querying and visualizing records from Elasticsearch in the form of advanced reports and custom dashboard screens.
PREREQUISITES:
- Elasticsearch is a Java-based server, which requires Java Development Kit (JDK). Download the JDK from the Oracle Downloads page and install it.
- You set the Path Environment variable. For more information, see the Oracle JDK installation instructions.
RECOMMENDATION: To avoid issues when integrating the Audit trail module, ElasticSearch, and Kibana, we recommend that you work with the following tested version compatibility of the products:
Set up Elasticsearch
Install the ElasticSearch logger module
The first step is to install the
ElasticSearch logger module. To install the module, choose one of the following options:
-
For Sitefinity CMS version 7.2 or later:
The Audit Trail Elasticsearch module is included in Sitefinity CMS version 7.2 or later.
To activate it, in Sitefinity CMS backend, navigate to Administration » Modules and Services. Make sure the Audit Trail and Audit Trail - Elasticsearch modules are installed and active.
- For Sitefinity CMS version 7.1 or earlier:
- Install the Telerik.Sitefinity.Audit.Elasticsearch NuGet package
To install the NuGet package with Visual Studio:
- Open your Sitefinity CMS solution with Visual Studio.
- Make sure you are connected to Sitefinity's NuGet server.
In Visual Studio's menu click Tools » Options...
In the Options window, navigate to NuGet Package Manager » Packages Source.
Verify or add a package source pointing to Sitefinity's NuGet server:
Name: Sitefinity CMS NuGet server
Source: http://nuget.sitefinity.com/nuget
- In Visual Studio's menu, select Options » NuGet Package Manager » Package Manager Console.
- In the Package Manager Console pane, from the Package source dropdown menu select Sitefinity CMS NuGet server. From the Default project dropdown menu select your SitefinityWebApp project.
- Install the NuGet package by running the following command:
install-package Telerik.Sitefinity.Audit.Elasticsearch.
- Build the solution.
- Log into Sitefinity's backend. Navigate to Administration » Modules and Services. Make sure the Audit Trail and Audit Trail - Elasticsearch modules are installed and active.
- Install the ElasticSearch logger module from GitHub
In the Sitefinity CMS GitHub organization, open the ElasticSearchAuditLogger repository.
Follow the instructions in the README.md file to set up the audit logger.
Configure ElasticSearch and Kibana
- Depending on the Kibana version you are working with, download and extract the corresponding ElasticSearch version.
NOTE: You can find the relevant download link in the Recommendation section in the beginning of this article.
NOTE: In the steps to follow, %ElasticsearchPath% refers to the path where your Elasticsearch files reside.
-
If you want the server to be started on a specific port, edit the elasticsearch.yml file, located in the %ElasticsearchPath%/config folder.
From the Set a custom port to listen for HTTP traffic: section, uncomment and change the port to the one you want to use.
NOTE: The default port is 9200.
- To disable forming clusters with other running instances in the same network, edit the %ElasticsearchPath%\config\elasticsearch.yml file and uncomment the following line:
discovery.zen.ping.multicast.enabled: false
For example, you do this when several developers work in the same room.
- Start the Elasticsearch server, either as executable, or as a service:
- To start Elasticsearch as an executable, run %ElasticsearchPath%\bin\elasticsearch.bat.
Ensure that %ElasticsearchPath%\bin\elasticsearch.bat (%ElasticsearchPath%/bin/elasticsearch on Unix) is called on system startup.
- To start Elasticsearch as a service, in %ElasticsearchPath%\bin, in a command prompt, perform the following:
- Install the service by typing service install
- Start the service by typing service start
- In Sitefinity backend, navigate to Administration » Settings » Advanced Settings » ElasticsearchAudit.
- In the ElasticsearchUri field, enter the root URL of your Elasticsearch site.
NOTE: The default value is 9200.
- If you no longer need JSON logging, disable the default JSON logging by deselecting the EnableJsonLogging checkbox.
RECOMMENDATION: As Sitefinity Elasticsearch audit logger invokes Elasticsearch RESTful services synchronously, to improve performance, we recommend that you ensure minimum network latency between your web server and your Elasticsearch server. If this is an issue, you can install a dedicated logging agent (for examaple, Logstash) on the web server machine, to provide out-of-process buffering and persistence of the logged messages.
Set up Kibana
Kibana 4.1.2
- Download and extract Kibana from the Kibana 4.1.2 download page.
NOTE: In the steps to follow, %KibanaPath% refers to the path where your Kibana files reside.
- Open the %KibanaPath%\config\kibana.yml file for editing.
Locate the elasticsearch_url setting and set it to the root URL of your ElasticSearch server.
- Navigate to the %KibanaPath%\bin folder and open the kibana.bat file.
This file hosts Kibana on the port set in the configuration file.
NOTE: The default value is 9200. You can also set which port to be used by Kibana in the - port setting. The default value is 5601.
- Open your hosted Kibana site and configure an index pattern:
- In the Index name or pattern field, type index.
- From the Time-field dropdown menu, select Timestamp.
- Click Create.
Kibana 3.1.0
- Download and extract Kibana from Kibana 3.1.0 download page.
NOTE: In the steps to follow, %KibanaPath% refers to the path where your Kibana files reside.
- Host the extracted folder on a web server that is accessible by the users entitled access to the audit log.
- Open the %KibanaPath%\config.js file for editing.
- Locate the elasticsearch: setting and set it to the root URL of your Elasticsearch server.
NOTE: The default value is 9200.
SECURITY NOTE: As Kibana is a client-side HTML application, which invokes the Elasticsearch REST API directly from the client's browser, the Elasticsearch server must be accessible from the machine of every user who is allowed to view the audit log. To avoid unrestricted access to the audit log. To avoid unrestricted access to the audit log (either through Kibana or just by calling the REST services of the Elasticsearch server), employ some network protection mechanism like a firewall (for example, white listing user IPs) or a dedicated VPN. In addition, make sure that these restrictions do not prevent Sitefinity CMS from connecting to Elasticsearch.