Integrate Audit trail module with Elasticsearch and Kibana

The default logging module of Sitefinity CMS is JSON logger, but you can also integrate Sitefinity CMS Audit trail module with external logging tools.

Elasticsearch is an open-source Java-based web server, used by the Audit trail module for storing and querying logged records, instead of using the textual log files.

Kibana is Elasticsearch web server that uses static HTML pages and client-side AJAX calls for querying and visualizing records from Elasticsearch in the form of advanced reports and custom dashboard screens.

  • PREREQUISITES:
    • Elasticsearch is a Java-based server, which requires Java Development Kit (JDK). Download the JDK from the Oracle Downloads page and install it.
    • You set the Path Environment variable. For more information, see the Oracle JDK installation instructions.
    RECOMMENDATION: To avoid issues when integrating the Audit trail module, ElasticSearch, and Kibana, we recommend checking Sitefinity CMS dependencies in Sitefinity NuGet Repository. You can also check Elasticsearch and Kibana compatibility in Elastic's Support Matrix page. 

    Install Elasticsearch module

    To integrate with Elasticsearch and Kibana servers, you must install the ElasticSearchAuditTrailLogger module. You do this by installing the Telerik.Sitefinity.Audit.Elasticsearch NuGet package from Sitefinity CMS NuGet Repository.

    During setup, you need to run the Elasticsearch server on the web server machine, on a dedicated machine, on premise, in the cloud, or in a cluster of machines.

    Setup Elasticsearch 7.5.1

    1. Download and extract Elasticsearch from the Elasticsearch 7.5.1 download page.
    2. Open the ~\<ElasticsearchPath>\config\elasticsearch.yml file for editing.
    3. Locate the Network section, remove the comment tag and change the port to the one you want to use.

    NOTE: The default value of ElasticSearch server is 9200. For more information, see Elasticsearch Setup Guide.


    Setup Elasticsearch as an audit service

    1. In Sitefinity CMS backend, navigate to Administration » Modules & Services and install the Audit Trail – Elasticsearch module.
    2. Navigate to Administration » Settings » Advanced » ElasticsearchAudit.
    3. In the ElasticsearchUri field, enter the root URL of your Elasticsearch site.
      The default value is http://localhost:9200/.  

    NOTE: If you no longer need JSON logging, navigate to Administration » Settings » Advanced Settings » Audit and disable the default JSON logging by deselecting the EnableJsonLogging checkbox.

    RECOMMENDATION: Because Sitefinity CMS Elasticsearch audit logger invokes Elasticsearch RESTful services synchronously, to improve performance, we recommend that you ensure minimum network latency between your web server and your Elasticsearch server. If this is an issue, you can install a dedicated logging agent - for example, Logstash, on the webserver machine, to provide out-of-process buffering and persistence of the logged messages.

    Setup Kibana 7.5.1

    1. Download and extract Kibana from the Kibana 7.5.1 download page.
    2. Open the ~\<KibanaPath>\config\kibana.yml file for editing.
      Locate the elasticsearch.hosts setting and set it to the root URL of your ElasticSearch server.
    3. NOTE: Follow Kibana Setup Guide for detailed information.

    4. Navigate to the ~\<KibanaPath>\bin folder and open the kibana.bat file.
      This file hosts Kibana on the port set in the configuration file. 

    5. NOTE: The default value of Elasticsearch server is 9200. You can also set another port to be used by Kibana in the server.port setting in the kibana.yml file. The default value is 5601.

    6. Open your hosted Kibana site and configure an index pattern. 
    7. For more information, see the Kibana’s tutorial Define your index patterns.

    NOTE: Because Kibana is a client-side HTML application, which invokes the Elasticsearch REST API directly from the client's browser, the Elasticsearch server must be accessible from the machine of every user who is allowed to view the audit log. To avoid unrestricted access to the audit log (either through Kibana or just by calling the REST services of the Elasticsearch server), employ some network protection mechanism like a firewall (for example, white listing user IPs) or a dedicated VPN. In addition, make sure that these restrictions do not prevent Sitefinity CMS from connecting to Elasticsearch.

    Upgrade single node and related indices

    Elasticsearch can usually be upgraded using a rolling upgrade process so upgrading does not interrupt service.  For more information, see Elastic’s  rolling upgrades.

    Upgrade from  Recommended upgrade path to 7.5.1 
     5.0-5.5
    1. Rolling upgrade to 5.6
    2. Rolling upgrade to 6.8
    3. Rolling upgrade to 7.5.1

    Elasticsearch can read indices created in the previous major version. If you have indices created in 5.x or before, you must reindex or delete them before upgrading to 7.5.1. Elasticsearch nodes will fail to start if incompatible indices are present. Snapshots of 5.x or earlier indices cannot be restored to a 7.x cluster even if they were created by a 6.x cluster. For information about upgrading old indices, see Reindex to upgrade.

    Upgrade to 6.x guide

    This is is a sample guide for the upgrade procedure to version 6.x.

    1. Back up your data.
      For more information, see Snapshot And Restore.
    2. Address any 6.0 breaking changes that affect your applications.
      For more information, see Breaking Changes.
    3. Check the Elasticsearch dependencies log.
    4. For more information, see Configuring Elasticsearch.
    5. Stop sending data to your cluster.
    6. Shut down your cluster and install Elasticsearch 6.x on all nodes. 
      For more information, see Full cluster restart upgrade.
      Note that X-Pack is installed automatically with the default distribution from 6.3 onward.
    7. Restart your Elasticsearch cluster.
    8. Create a new index using:
      PUT /<new_index>
    9. Reindex the old index using the following code:
      POST _reindex
    10. Stop the Elasticsearch 6.x server.
    11. Copy the new_index from <elasticsearch-6.8.0_path>\data\nodes\...\indices to <elasticsearch-7.5.1_path>\data\nodes\...\indices.
    12. Start the Elasticsearch 7.5.1 server.
    13. Reindex the new index using the following code:
      POST _reindex



     

  • Increase your Sitefinity skills by signing up for our free trainings. Get Sitefinity-certified at Progress Education Community to boost your credentials.

    Get started with Integration Hub | Sitefinity Cloud | Sitefinity SaaS

    This free lesson teaches administrators, marketers, and other business professionals how to use the Integration hub service to create automated workflows between Sitefinity and other business systems.

    Web Security for Sitefinity Administrators

    This free lesson teaches administrators the basics about protecting yor Sitefinity instance and its sites from external threats. Configure HTTPS, SSL, allow lists for trusted sites, and cookie security, among others.

    Foundations of Sitefinity ASP.NET Core Development

    The free on-demand video course teaches developers how to use Sitefinity .NET Core and leverage its decoupled architecture and new way of coding against the platform.

    Was this article helpful?