Caching

BoxLang ships with an enterprise Caching Engine and Cache Agnostic API Aggregator!

BoxLang Cache Engine

BoxCache Diagram

The BoxLang Cache Engine is a high-performance caching solution designed to optimize data retrieval speeds for applications using the BoxLang programming language. By leveraging advanced caching algorithms and memory management techniques, it minimizes latency, enhances scalability, and improves the overall efficiency of data access. Whether handling frequent read operations or reducing server load, the BoxLang Cache Engine serves as an essential component for deploying robust and responsive applications.

Cache Aggregation

The BoxLang Cache Engine also functions as a powerful cache aggregator. This capability allows it to aggregate various cache providers under a single, unified interface, simplifying the management and operation of caching systems. By providing a centralized API, BoxLang Cache Engine streamlines access to multiple caching solutions, ensuring seamless integration and enhanced flexibility. This feature is particularly beneficial for applications that require a diverse set of caching tactics, as it reduces complexity and boosts overall system coherence.

Features at a Glance

• Cache Aggregator

  • Ability to aggregate different caching engines via our ICacheProvider interface

  • Ability to aggregate different configurations of the same caches

  • Rich aggregation event model

  • Granular logging

• Fast and Simple to use

  • Simple API and configuration parameters

  • Small Footprint

• Solid Core

  • Multi-Threaded

  • Based on Java Concurrency Classes

  • Multiple Eviction Policies: FIFO, LFU, LIFO, LRU, MFU, MRU, Random

  • Memory Management & Memory Sensitive Caching based on Java Soft References

• Extensible & Flexible

  • CachelListeners for event broadcasting

  • Create custom eviction policies

  • Create custom cache providers

  • Create custom cache key filters

  • Create custom object storages

• Highly Configurable

  • JVM Threshold Checks

  • Object Limits

  • Ability to time-expire objects

  • Eternal (singletons) and time-lived objects

  • Fully configurable at runtime via dynamic configurations and hot updates

Configuration

The main BoxLang configuration file configures the BoxCache boxlang.json which can be found in your $BOXLANG_HOME/config/boxlang.json. You can find all the documentation about configuration in the caches area.

Default Caches

Every BoxLang runtime comes pre-configured with the following mandatory caches for operation. You can configure them as you see fit. Please see the caches configuration area.

Cache	Object Store	Hint
default	ConcurrentStore	The default cache in BoxLang is used for queries, templates, and many more internal usages.
bxSessions	ConcurrentStore	If you activate session management in your web or client applications, user session information will be stored here.
bxRegex	ConcurrentSoftReferenceStore	This is where all dynamic regular expressions are compiled and kept.

Providers

Here are the available providers for BoxLang. The table shows the status of completion for each provider and its availability for the open-source version of BoxLang, as well as for +/++ subscribers.

Provider	Description	Status	OS	+/++
BoxLang	The enterprise BoxLang native cache provider can leverage many different object stores.	Done
Redis	A Redis single-node provider	Done
RedisCluster	A Redis cluster cache provider	Done
MongoDB	A MongoDB based Provider	In Progress
Couchbase	A Couchbase based provider	In Progress
ElasticSearch	An Elastic Search provider	In Progress
EhCache	An EhCacheProvider	In Progress

You can also find more providers through third-party providers at www.forgebox.io.

Object Stores

Object stores are the foundational storage layer of the BoxLang cache engine. They provide the actual mechanism for storing, retrieving, and managing cached objects. While cache providers coordinate user interactions and act as a service layer, object stores handle the low-level data persistence and retrieval operations.

Type	Description
BlackHoleStore	Mocking store, just simulates a store, nothing is stored.
ConcurrentSoftReferenceStore	Memory-sensitive storage leveraging Java Soft References.
ConcurrentStore	Leverages concurrent hashmaps for storage.
FileSystemStore	Stores the cache items in a serialized fashion on disk
JDCBStore	Stores caches in JDBC Databases

Each store can have different configuration properties as well.

Eviction Policies

The BoxCache also ships with several different eviction policies that are triggered on eviction conditions like:

• Max objects reached

• Memory threshold

• Garbage collections

• Much more.

Policy	Purpose
LRU (Least Recently Used)	Evicts the objects that haven't been accessed for the longest time. Best for applications with temporal locality where recently accessed items are more likely to be accessed again. Ideal for general-purpose caching scenarios.
MRU (Most Recently Used)	Evicts the most recently accessed objects first. Useful when you want to keep older, established data and remove newly added items. Good for scenarios where recent additions are less valuable than historical data.
LFU (Least Frequently Used)	Evicts objects with the lowest access frequency count. Maintains items that are accessed often, regardless of when they were last accessed. Perfect for caching frequently requested data like popular products or common API responses.
MFU (Most Frequently Used)	Evicts the most frequently accessed objects first. Counterintuitive but useful in scenarios where you want to cycle out "hot" data to make room for less popular items that might become important. Rare use case but valuable for specialized applications.
FIFO (First In, First Out)	Evicts objects in the order they were added to the cache. Simple queue-based eviction that doesn't consider access patterns. Good for time-sensitive data where older entries naturally become less relevant, like news feeds or log entries.
LIFO (Last In, First Out)	Evicts the most recently added objects first, like a stack. Keeps older established data while removing newer additions. Useful when you want to maintain a stable core dataset and only temporarily cache additional items.
Random	Evicts objects randomly without considering access patterns or insertion order. Provides consistent average performance without the overhead of tracking access metadata. Good for scenarios where no clear access pattern exists or when you want to avoid the worst-case behaviors of other policies.

Cache BIFs (Built-in Functions)

BoxLang provides a comprehensive set of Built-in Functions (BIFs) that offer convenient access to the caching infrastructure. These BIFs provide a developer-friendly interface for working with caches, filters, and the cache service, eliminating the need for direct Java interaction.

Overview

Cache BIFs serve as the primary interface for BoxLang developers to interact with the caching system. They provide:

• Simple Access: Easy retrieval of cache instances and services

• Functional Approach: Chainable operations for cache manipulation

• Filter Support: Powerful pattern matching for bulk operations

• Service Discovery: Introspection capabilities for registered caches and providers

• BoxLang Integration: Native BoxLang syntax and conventions

Please note that all get operations from our cache providers return an Attempt. An object that could potentially have your data or not. Fully functional and avoids any null issues.

Available Cache BIFs

Summary Table

BIF	Purpose	Returns
cache()	Get a cache instance by name	ICacheProvider
cacheFilter()	Create filters for cache key operations	ICacheKeyFilter
cacheNames()	List all registered cache names	Array
cacheProviders()	List all registered cache providers	Array
cacheService()	Access the cache service directly	CacheService

cache(cacheName)

The primary BIF for accessing cache instances. Returns a cache provider that implements the ICacheProvider interface.

Syntax:

cache([cacheName="default"])

Parameters:

• cacheName (string, optional): The name of the cache to retrieve. Defaults to "default"

Returns: ICacheProvider - The requested cache instance

Examples:

// Get the default cache
myCache = cache()

// Get a named cache
// Let's assume you have created these caches already in your boxlang.json
userCache = cache( "userSessions" );
apiCache = cache( "apiResponses" 

// Basic cache operations
cache().set("user:123", userData)
// Get operations return a BoxLang attempt
user = cache().get("user:123").getOrDefault( new User() )
cache().clear( "user:123" );

// Chained operations
cache( "userSessions" )
    .set( "session:abc123", sessionData, 1800 ) // 30 minutes
    .set( "session:def456", otherSession, 3600 ) // 1 hour

// Check if cache exists and use it
if ( cacheNames().contains( "productCache") ) {
    product = cache( "productCache" )
        .get( "product:456" )
        .getOrDefault( 0 
}

cacheFilter( filter, useRegex )

Creates filter objects for performing bulk operations on cache keys using pattern matching.

Syntax:

cacheFilter(filter, [useRegex=false])

Parameters:

• filter (string, required): The pattern to match against cache keys

• useRegex (boolean, optional): Use regex pattern instead of wildcard. Defaults to false

Returns: ICacheKeyFilter - A filter object for cache operations

Wildcard Patterns:

• * - Matches any sequence of characters

• ? - Matches any single character

• [abc] - Matches any character in the brackets

• [a-z] - Matches any character in the range

Examples:

// Wildcard filtering (default)
userFilter = cacheFilter( "user:*" )
sessionFilter = cacheFilter( "session:*" )
tempFilter = cacheFilter( "temp_*" )

// Regex filtering
emailFilter = cacheFilter( ".*@domain\.com$", true )
idFilter = cacheFilter( "^id_\d{4,6}$", true )

// Use filters with cache operations
cache().clear( cacheFilter( "temp_*" ) ) // Clear all temp keys
cache().clear( cacheFilter( "user:session_.*", true ) ) // Regex clear

// Get multiple keys with filter, returns struct
userSessionsStruct = cache().get( cacheFilter( "user:*" ) )
// Returns: { "user:123": userData, "user:456": otherUser, ... }

// Bulk operations with filters
cache().clear( cacheFilter( "api:response:*" ) ) // Clear API cache
cache().lookup( cacheFilter( "product:*" ) ) // Check which products are cached

// Complex patterns
cache().clear( cacheFilter( "cache_temp_[0-9]*" ) ) // Clear numbered temp caches
cache().get( cacheFilter( "user_session_????" ) ) // Get 4-char session IDs

Custom Filter Functions:

You can also use closures/lambdas as filters:

// Custom filter using closure
customFilter = key -> {
    return key.getName().startsWith( "special_" ) && 
           key.getName().length() > 10;
};

cache().clear( customFilter )

// One-liner custom filters
cache().clear( key -> key.getName().contains("_expired_") )
cache().get( key -> key.getName().endsWith("_active") )

cacheNames()

Returns an array of all registered cache names in the system.

Syntax:

cacheNames()

Returns: Array - Array of cache names as strings

Examples:

// Get all cache names
allCaches = cacheNames()
println( "Available caches: " & allCaches.toList() )

// Check if specific cache exists
if ( cacheNames().contains( "mySpecialCache" ) ) {
    // Use the cache
    cache( "mySpecialCache" ).set( "key", "value" )
} else {
    // Create the cache or handle missing cache
    println( "Cache 'mySpecialCache' not found!" )
}

// Iterate through all caches
for ( cacheName in cacheNames() ) {
    cacheInstance = cache( cacheName )
    println( "Cache '#cacheName#' has #cacheInstance.getSize()# items" )
}

// Filter cache names
userCaches = cacheNames().filter( name -> name.startsWith( "user" ) )

cacheProviders()

Returns an array of all registered cache provider names.

Syntax:

cacheProviders()

Returns: Array - Array of provider names as strings

Examples:

// Get all available providers
providers = cacheProviders()
println( "Available providers: " & providers.toList() )

// Check provider availability
if ( cacheProviders().contains( "Redis" ) ) {
    // Can create Redis caches
    println( "Redis provider is available" )
} else {
    println( "Redis provider not registered" )
}

// Display provider information
for ( provider in cacheProviders() ) {
    println( "Provider available: #provider#" )
}

cacheService()

Provides direct access to the underlying cache service for advanced operations.

Syntax:

cacheService()

Returns: CacheService - The cache service instance

Examples:

// Get the cache service
service = cacheService()

// Create new caches dynamically
service.createCache( "newCache", "BoxLang", {
    "maxObjects": 5000,
    "defaultTimeout": 1800,
    "evictionPolicy": "LRU"
} )

// Register custom providers
service.registerProvider( 
    "MyProvider", 
    new java:com.mycompany.MyProvider()
)

// Bulk operations
service.clearAllCaches() // Clear all caches
service.reapAllCaches() // Run maintenance on all caches

// Advanced cache management
service.replaceCache( "oldCache", newCacheInstance )
service.shutdownCache( "temporaryCache" )

Practical Usage Patterns

Basic Caching Pattern

// Simple get-or-set pattern using getOrSet
function getUser( userID ) {
    var cacheKey = "user:#userID#"
    
    return cache().getOrSet( 
        cacheKey, 
        () -> userService.loadUser( userID ),
        1800 // Cache for 30 minutes
    )
}

API Response Caching

// API response caching with patterns
function cacheAPIResponse( endpoint, params, response ) {
    var cacheKey = "api:#endpoint#:#hash( serialize( params ) )#"
    cache( "apiCache" ).set( cacheKey, response, 600 ) // 10 minutes
}

function getCachedAPIResponse( endpoint, params ) {
    var cacheKey = "api:#endpoint#:#hash( serialize( params ) )#"
    return cache( "apiCache" ).get( cacheKey )
}

function clearAPICache( endpoint ) {
    if ( isNull( endpoint ) ) {
        // Clear all API cache
        cache( "apiCache" ).clear( cacheFilter( "api:*" ) )
    } else {
        // Clear specific endpoint
        cache( "apiCache" ).clear( cacheFilter( "api:#endpoint#:*" ) )
    }
}

Cache Warming and Preloading

// Cache warming function
function warmupCaches() {
    // Warm user cache with active users
    var activeUsers = userService.getActiveUsers()
    for ( var user in activeUsers ) {
        cache( "users" ).set( "user:#user.id#", user, 7200 ) // 2 hours
    }
    
    // Warm product cache
    var popularProducts = productService.getPopularProducts()
    for ( var product in popularProducts ) {
        cache( "products" ).set( "product:#product.id#", product, 3600 )
    }
    
    println( "Warmed #activeUsers.len()# users and #popularProducts.len()# products" )
}

Cache Statistics and Monitoring

// Cache monitoring function
function getCacheReport() {
    var report = {}
    
    for ( var cacheName in cacheNames() ) {
        var cacheInstance = cache( cacheName )
        var stats = cacheInstance.getStats()
        
        report[ cacheName ] = {
            "size": cacheInstance.getSize(),
            "hits": stats.getHits(),
            "misses": stats.getMisses(),
            "hitRate": stats.getHitRate(),
            "enabled": cacheInstance.isEnabled()
        }
    }
    
    return report
}

// Display cache status
function displayCacheStatus() {
    var report = getCacheReport()
    
    for ( var cacheName in report ) {
        var info = report[ cacheName ]
        
        println( "Cache: #cacheName#" )
        println( " Size: #info.size# items" )
        println( " Hit Rate: #numberFormat( info.hitRate * 100, '0.00' )#%" )
        println( " Hits: #info.hits#, Misses: #info.misses#" )
        println( " Status: #info.enabled ? 'Enabled' : 'Disabled'#" )
        println( "---" )
    }
}

Dynamic Cache Creation

// Create caches based on application needs
function createUserCache( maxUsers ) {
    var service = cacheService()
    
    if ( !service.hasCache( "userCache" ) ) {
        service.createCache( "userCache", "BoxLang", {
            "maxObjects": maxUsers * 2,
            "defaultTimeout": 3600,
            "evictionPolicy": "LRU",
            "objectStore": "ConcurrentHashMap"
        } )
        
        println( "Created user cache for #maxUsers# users" )
    }
    
    return cache( "userCache" )
}

Error Handling and Fallbacks

// Safe cache operations with fallbacks
function safeGet( cacheKey, fallbackFunction ) {
    try {
        return cache().getOrSet( cacheKey, fallbackFunction, 1800 )
    } catch ( any e ) {
        writeLog( "Cache error: #e.message#", "error" )
        // Fallback directly to function if cache fails completely
        return fallbackFunction()
    }
}

Advanced Patterns

Cache Hierarchies

// Hierarchical cache clearing
function clearUserData( userID ) {
    var filters = [
        cacheFilter( "user:#userID#*" ), // User data
        cacheFilter( "session:*user:#userID#*" ), // User sessions
        cacheFilter( "profile:#userID#*" ), // User profiles
        cacheFilter( "pref:#userID#*" ) // User preferences
    ]
    
    for ( var filter in filters ) {
        cache().clear( filter )
    }
}

Multi-Cache Operations

// Operate across multiple caches
function clearAllUserCaches( userID ) {
    var userCaches = cacheNames().filter( name -> name.contains( "user" ) || name.contains( "session" ) )
    
    for ( var cacheName in userCaches ) {
        var userFilter = cacheFilter( "*#userID#*" )
        cache( cacheName ).clear( userFilter )
    }
}

Cache Synchronization

// Synchronize data across multiple caches
function syncUserAcrossCaches( userID, userData ) {
    var caches = [ "primaryUsers", "secondaryUsers", "userProfiles" ]
    
    for ( var cacheName in caches ) {
        if ( cacheNames().contains( cacheName ) ) {
            cache( cacheName ).set( "user:#userID#", userData, 3600 )
        }
    }
}

Best Practices

1. Cache Naming Conventions

// Use descriptive, hierarchical cache keys
cache().set( "user:profile:#userID#", profile )
cache().set( "api:response:v1:#endpoint#:#params#", response )
cache().set( "session:data:#sessionID#", sessionData )
cache().set( "config:app:#version#", configuration )

2. Efficient Filter Usage

// Prefer specific patterns over broad wildcards
cache().clear( cacheFilter( "temp:session:*" ) )        // Good
cache().clear( cacheFilter( "*" ) )                     // Avoid - too broad

// Use regex for complex patterns
cache().clear( cacheFilter( "^user:temp_\d{8}$", true ) ) // Complex pattern

3. Error Handling

// Always handle cache failures gracefully
try {
    return cache().get( key ).orElse( fallbackValue )
} catch ( any e ) {
    writeLog( "Cache error: #e.message#", "error" )
    return fallbackValue
}

4. Performance Monitoring

// Regular cache performance checks
function monitorCachePerformance() {
    for ( var cacheName in cacheNames() ) {
        var cacheInstance = cache( cacheName )
        var hitRate = cacheInstance.getStats().getHitRate()
        
        if ( hitRate < 0.7 ) { // Less than 70% hit rate
            writeLog( "Cache '#cacheName#' has low hit rate: #hitRate#", "warning" )
        }
    }
}

Keep Exploring

BoxLang's Cache BIFs provide a powerful and intuitive interface for working with the caching infrastructure. They enable developers to:

• Access Caches Easily: Simple syntax for retrieving and using cache instances

• Filter Operations: Powerful pattern matching for bulk cache operations

• Service Discovery: Easy introspection of available caches and providers

• Advanced Management: Direct access to cache service capabilities

By leveraging these BIFs, developers can implement sophisticated caching strategies while maintaining clean, readable BoxLang code. The functional approach supports both simple use cases and complex enterprise caching scenarios. Now that we have the basics, we will explore more in-depth how to use the BoxCache engine.