Performance with InMemory vs Distributed output cache

Test results summary

Using distributed cache delivers up to 200 times better startup and warmup performance for a “cold start”, when pages have not yet been compiled, and up to 5 times better startup and warmup when a node is restarted (pages have already been compiled).  This results in improved scalability and high-availability, as well as better browsing experience for users who hit a newly started node. In-memory output cache storage certainly excels at providing the best average page response times once the site is warmed up. Distributed output cache still delivers very good average page response times, and this performance can be affected positively (or negatively) based on the distributed cache storage used.
Following is a detailed analysis of the performance improvements comparing in-memory and distributed output cache storages.

Test setup

Testing the Sitefinity CMS performance with distributed cache storage is executed with the following setup:

Hosting configuration

Setup type

Sitefinity running on 3 web server nodes in load balanced configuration hosted on AWS.

Database server

c5.xlarge - 4CPU, 8GB RAM

Web Server node 1

c5.xlarge - 4CPU, 8GB RAM

Web Server node 2

c5.xlarge - 4CPU, 8GB RAM

Web Server node 3

c5.xlarge - 4CPU, 8GB RAM

Redis cache configuration

Amazon ElastiCache for Redis cache used for NLB messaging and distributed cache storage.

1 node, cache.t2.medium.

Both Redis node and Sitefinity Web server nodes configured in the same AWS Region.

NOTE: When using distributed cache, the performance of your Sitefinity CMS site depends directly on the network latency and distributed cache storage type. For example, running the tests described in this article with ElastiCache for Redis node of type of cache.t2.medium results in 2 times better performance compared to using cache.t2.small. The cache.t2.small and cache.t2.medium are among the most affordable Redis cache storage types and are categorized as Low to Moderate network performance. Using higher-rated distributed cache storage might result in even better performance. 



Sitefinity CMS configuration

License type

Sitefinity CMS Multisite license

Domain configuration

Website domain set to the load balancer URL Note: this configuration is required for the warmup module normal operation if distributed cache is used

Pages

250 MVC pages

Content

Content block widget on each page

Page size (in KB)

600KB

Warm-up module configuration

maxPagesAfterInitializationPerSite="500"

Output cache settings:



Cache duration (in seconds)

2000

varyByHost

true



Test results

The test results list the Sitefinity CMS performance in startup, warmup, and load test to reflect the full scenario of a newly started/restarted site spinning up and warming up, as well as website browsing under load by site visitors.

Startup and warmup

Test scenario description

Metric

In-memory output cache

Distributed output cache

Start any node for the first time (pages have not been compiled yet)

Compilation and warmup duration

1000 seconds

  • 1000 seconds for the first node
  • 5 seconds for all other nodes (all pages already cached in Redis by the 1st node, no compilation or content processing needed)

Restart any node

(pages have already been compiled and exist in the ASP.NET Temporary fields)

Warmup duration

22 seconds

5 seconds

Scale by adding a new node

Compilation and warmup duration

1000 seconds

5 seconds

User load

Test setup

Testing the Sitefinity CMS website user load performance with distributed cache is executed with the following setup:

Hosting configuration

Setup type

Sitefinity running on 3 web server nodes in load balanced configuration hosted on Microsoft Azure.

Database server

Microsoft Azure SQL (Standard S2: 20-100DTUs)

Web Server node 1

7 GB RAM, 2 CPUs

Web Server node 2

7 GB RAM, 2 CPUs

Web Server node 3

7 GB RAM, 2 CPUs

Redis cache configuration

Microsoft Azure Redis cache used for NLB messaging and distributed cache storage.

1 node, C3; 6144 MB cache

Both Redis node and Sitefinity Web server nodes configured in the same Azure Region.

NOTE: When using distributed cache, the performance of your Sitefinity CMS site depends directly on the network latency and distributed cache storage type. Using higher-rated distributed cache storage might result in even better performance. 

Sitefinity CMS configuration

License type

Sitefinity CMS Multisite license

Domain configuration

Website domain set to the load balancer URL Note: this configuration is required for the warmup module normal operation if distributed cache is used

Pages and Content

The Sitefinity Quantum demo website is used for the test. For more information see Sitefinity Quantum GitHub repository.

Output cache settings:

Cache duration (in seconds)

2000

varyByHost

true

The load test has been executed using JMeter v.5.1 load test tool.

Load test configuration

User load

20

Load pattern

Constant pattern (user browse the website all the time)

Test duration

10 min

The load test is executed against a fully warmed up website (all nodes warmed up and initialized the output cache) to measure the differences in website performance when reading from In-memory vs Distributed cache.

Test results

Metric

In-memory output cache

Distributed output cache

Average response time

0.38 seconds

0.56 seconds

Was this article helpful?