Skip to content

Tips on making your cache fast

Jon P Smith edited this page Jul 28, 2023 · 6 revisions

The document isn't about improving the the read (e.g. Get) of the FileStore cache because its very fast (~25 ns.) at all cache sizes tested (see performance read tests in the README file). What this page is about improving the performance of the write (e.g. Set, Remove) and making you aware of maximum size of the FileStore cache file and what happens if you go over the maximum size.

However, if your usage of the FileStore cache only contains small a number of cache entries (say < 500 with key+value less than 50 ASCII chars) then a) the default maximum size of the cache will be fine, and b) any writes to the cache will be fairly fast - see performance write tests in the README file. In that case you don't need to read the rest of this document.

But if you use Unicode characters, byte[] versions, or want to get best write performance its worth skimming the topics in this article.

NOTE: The DistributedFileStoreCacheOptions class contains all the options, each with detailed comments. This class is a good place to learn how you can configure the FileStore cache.

Topics:

Why is there a maximum size?

The FileStore cache creates temporary a byte[] buffer to read in the information. Even though this buffer is released after each read we don't the buffer too big. Also we know that the bigger the cache gets, then the slower the Set / Remove takes - see the Performance figures in the README file.

So the FileStore cache options includes a MaxBytesInJsonCacheFile parameter in the FileStore options, with a default value of 10,000. This sets the maximum size of the json cache file and the internal buffer in the code. Any attempt to add another cache value that makes the file goes over MaxBytesInJsonCacheFile level will be lost!, which is the normal way that cache sizes are applied.

NOTE: I did try using streaming which doesn't need a buffer, but that only works for async calls. That could cause problems between the different sync and async approaches (and streaming wasn't any quicker) so I took it out.

Things that make the cache bigger/slower

If you are using the String version and using ASCII characters, then its already at the smallest versions. But below I describe a number of issues that will make the json bigger and provide suggestions to get round most, but not all issues.

Unicode strings - use json UnsafeRelaxedJsonEscaping

If you use Unicode characters, then by default they will be converted by the System.Text.Json serializer will turn Unicode characters into a number constant, e.g. "\u05D1" which takes 6 ASCII characters. But you can tell the json serializer to keep the Unicode characters as a character which replace "\u05D1" with the Unicode character "ם" (hebrew) which only takes 2 bytes. You can do this by setting the FileStore cache JsonSerializerForCacheFile option to passes through Unicode characters, as shown in the code below

builder.Services.AddDistributedFileStoreCache (options =>
    {
        options.WhichInterface = String;
        options.JsonSerializerForCacheFile = new JsonSerializerOptions 
             { Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping };
    }, builder.Environment);

NOTE: The FileStore cache version Class already has the UnsafeRelaxedJsonEscaping applied as it makes the inner json smaller. See the next section.

Class version - already has UnsafeRelaxedJsonEscaping

The FileStore cache version Class adds extra methods that lets you save complex data in a class. It does this by using the json serializer to turn the class, which is serialized again to go into the cache. By default the characters such as {:, are stored as a number constant, e.g. "\u0022". The result is something like this:

"{\u0022MyClass1\u0022:{\u0022MyInt\u0022:1,\u0022MyString\u0022:\u0022Hello\u0022},\u0022MyInt\u0022:3}"

So, if you choose the Class version those number constants are removed and replaced with the ASCII character (see below), which reduces the size of the json.

"{\"MyClass1\":{\"MyInt\":1,\"MyString\":\"Hello\"},\"MyInt\":3}"

Also, the UnsafeRelaxedJsonEscaping is applied to the serialization of your clases so that any Unicode in the class is also shortened.

Binary data containing non-character values can be BIG

All the data passes thought the Encoding.UTF8.GetBytes and Encoding.UTF8.GetString and the UTF8 encoder turns some values to a character constant, which takes up 6 bytes instead of one byte

A method to help you set correct cache size

You can directly set the FileStore cache option's MaxBytesInJsonCacheFile parameter to a value, there is a method called SetMaxBytesByCalculation that can help you. This is helpful, but you do need to understand each parameters and whether the json UnsafeRelaxedJsonEscaping is applied. Below is the

/// <summary>
/// If you want to set the maximum bytes that the cache can hold then you can use this calculation
/// </summary>
/// <param name="maxEntries">The maximum number of cache entries you want to add to the cache</param>
/// <param name="maxKeyLength">The maximum size of the key string. ASSUMES ASCII characters - increase if Unicode</param>
/// <param name="maxValueLength">The maximum size of the value string</param>
/// <param name="charSize">Optional:
/// If ascii = 1, if unicode and not using UnsafeRelaxedJsonEscaping then 6, 
/// if unicode with UnsafeRelaxedJsonEscaping then 2,
/// if not UTF8 character the 6</param>
/// <param name="percentWithTimeout">Optional: the percent (between 0 and 100) of the entries will have a timeout</param>
public void SetMaxBytesByCalculation(int maxEntries, int maxKeyLength, int maxValueLength, 
     int charSize = 1, int percentWithTimeout = 0)

And you would this method when you register the FileStore cache, as shown below:

builder.Services.AddDistributedFileStoreCache (options =>
    {
        options.WhichInterface = String;
        //set size for up to 100 cache values only using ASCII characters, and not using any timeouts 
        options.SetMaxBytesByCalculation(100, 15, 25, 1);
    }, builder.Environment);

This method doesn't work for the Class version as its difficult to work out how big the classes are. You can always set the MaxBytesInJsonCacheFile property directly in such cases.

NOTE: Don't worry about having the MaxBytesInJsonCacheFile bigger than you need because that doesn't really effect the performance times - its the time to read and write the contents of the json cache file that takes all the time.