This article describes how to implement a Token Expiration Strategy in Anypoint Studio that handles multiple TTLs (time to live) based on key-value entries using one single Object Store Connector, for example, storing multiple access tokens inside a single Object Store and enforcing the expiration strategy for each token according to their assigned TTL (time to live).
The Object Store Connector is a Mule component that allows the storage of key-value and it can be used for many different purposes. One of the uses of the Object Store is to store access tokens.
The main idea behind access token storage is to promote the reuse of the token to reduce unnecessary roundtrips to the authentication server and avoid overloading the authentication server with consecutive calls to retrieve the token.
Rather than requesting a new token, use the stored token during future calls until it expires.
However, it is essential to implement a Token Expiration/Revocation Strategy to retrieve a new valid token from the authentication server.
This article describes how to implement a Token Expiration Strategy in Anypoint Studio that handles multiple TTLs (time to live) based on key-value entries using one single Object Store Connector, for example, storing multiple access tokens inside a single Object Store and enforcing the expiration strategy for each token according to their assigned TLL (time to live).
Object Store v2 does not offer the option to specify a TTL (time to live) for each key-value entry stored in the Object Store. To resume, each key-value pair stored inside the Object share the same TTL. Consequently, constraining the use of multiple Object Stores to handle multiple TTLs.
It is noteworthy to mention that some operations on the Object Store such as storing a new key-value pair or updating an existing key-value entry, reset the TTL for all key-value pairs inside the Object Store. Due to the reset of the TTL, the Object Store won’t be able to distinguish between expired and valid tokens.
The “Token Expiration Strategy” is a mule flow based on logical and conditional checks to verify for relevant access tokens if their validity time has not been exceeded. This flow is triggered whenever the access tokens are being requested.
The flow in charge of enforcing the Token Expiration Strategy follows the following conditional schema depicted in Figure 1.
Figure 1: Token Expiration Strategy’s Conditional Schema
First of all, it is necessary to check if the token is already stored in the Object Store otherwise a new token must be requested and stored.
In case the token already exists, then its validity time is calculated and compared to the desired TTL. In other words, if it is valid we can reuse the access token straight away otherwise we must request a new token and store it inside the Object Store.
This solution makes use of custom properties file that stores key-value pairs such as <“tokenName”: “custom TTL”>. This properties file is read in the Token Expiration Strategy flow to retrieve the TTL for a specific access token.
This section will describe every step to implement the “Token Expiration Strategy” in Anypoint Studio.
Creation of a properties file (.yaml)
As I already mentioned before, the creation of a properties file (.yaml) is necessary to define key-value pairs that use the access token’s name as the Key and the access token’s TTL as its value. This properties file is read in the Token Expiration Strategy flow to retrieve the TTL for a specific access token. Figure 2 depicts an example of the configuration file.
Figure 2: YAML Configuration file
Note: This solution uses Object Store Connector’s Store operation that will store a given Key and value in the Object Store. This Key will be named according to the access token’s names defined in our properties file.
Creation of an Object Store
When creating the Object Store several values should be set as shown below:
Max Entries: Represents max no. of entries allowed in the object store.
Entry TTL: Represents Entry timeout (after what time an entry times out).
Entry TTL Unit: Entry time to live unit (Seconds/Minutes/Hours/Days).
Expiration Interval: The frequency at the expiration thread should run.
Expiration Interval unit: The expiration interval.
Persistent: When checked the data is stored in the disk, otherwise in-memory (data lost when the app restarts or crashes).
When setting the values several aspects must be taken into account, such as, how to choose the Entry TTL to handle multiple TTLs. The value for this particular field must be equal to or greater than the highest Token’s TTL defined in the previously created properties file.
This requirement must be fulfilled to avoid the scenario where the Object Store Connector reaches the established Entry TTL e.g. “3500 seconds” and marks all tokens as expired whereas some tokens might have a higher TTL, such as “7000”, as defined in the properties file.
Rolling or Static TTL
The Entry TTL can be either rolling or static.
If the Entry TTL is not specified, the default value depends on the Mule version used for the application:
Mule versions 4.2.1 and later: Object stores have a rolling TTL by default. Accessing the data during the last seven days of a 30-day window extends the TTL for another 30 days.
Mule versions earlier than 4.2.1: Object stores have a static TTL of 30 days by default. Updating the data (for example, by overwriting the key) resets the TTL.
Note: This solution provides a static Entry TTL equal to or greater than the highest TTL defined in the properties file.
As mentioned before, some operations on the Object Store such as storing a new key-value pair or updating an existing key-value entry, reset the TTL for all key-value pairs inside the Object Store. Due to the reset of the TTL, the Object Store won’t be able to distinguish between expired and valid tokens.
Let’s take an Object Store with one stored key-value entry and an Entry TTL of 3000 seconds. After 2000 seconds an additional key-value entry is stored in the Object Store. This operation resets the TTL of all existing entries! This means that our first entry’s TTL resets to 3000 seconds but its access token expires in 1000 seconds.
Interface and Input variables
Let’s create a mule flow as shown in the below image, composed of an HTTP Listener, a Logger that prints the start of the flow in the logs, two Set Variable Connectors, a Flow Reference to the “Token Expiration Strategy Flow” and another Logger that prints the end of the flow in the logs.
Figure 4: Simple Interface
The first “Set Variable” component is in charge of storing the token’s name, in our case, it is sent as a query parameter.
The second “Set Variable” component stores the TTL for a specific access token based on the access token name stored previously. The TTL is stored in our custom properties file. Using Dataweave’s lookup function p() it is possible to read values from our custom properties using our previously defined property placeholder.
In this solution, our property placeholders correspond to the concatenation of the root node of our custom property placeholders, namely token_tll, and the access token’s name, as shown below.
ttl: p('token_tll.' ++ vars.tokenName
Object Structure to storeThis section strives to describe the object as well as the required information that is needed to ensure the success of the Token Expiration Strategy.
The object that we want to store and keep inside our Object Store is the token and the object’s creation DateTime which can be created using Dataweave’s now() function. An example of this object is shown below.
Figure 5: Object to Store Containing Token and DateTime
On entering the values for the Key and value that we want to store, our Store Operation will look like Figure 6. Where the Key is specified by the access token name and the value takes our token object.
Figure 6: Object Store’s Store Connector
Generate Token Expiration Strategy Flow
This section shows how to generate the main flow responsible for the Expiration Strategy which must handle the following use cases:
The key does not exist
The key exists but the access token is expired
The key exists and the access token is still valid
Figure 7: Token Expiration Strategy Flow
Figure 9: Object Store’s Retrieve Connector
After retrieving the token, we check using a Choice Router if the token has been retrieved or the default value “”. A new token must be requested and stored if the result of the Retrieve Connector is an empty String “”.
If the token exists in the Object Store then its validity interval must be processed and verified. According to the code snippet below, vars.token_object.date corresponds to the token's insertion time in the Object store after being requested.
(now() - (vars.token_object.date as DateTime)).seconds > ((vars.ttl as Number) - 200)
To verify if the token is still valid we need to subtract the time which is stored with the token inside the Object Store from the current time, namely now() function. The result is then converted in seconds and compared if it is greater than the defined TTL for this token.
Note that during the process we deduct 200 seconds from the initial TTL. The main idea behind this operation is to take into consideration delays that can occur until the actual call, such as the time from requesting a new token and storing it.
In this manner, we make sure that the token is still valid when the request is sent.
It is important to point out that the “Token Expiration Strategy” does not remove any key from the Object Store. Instead, it re-uses the store operation to update/overwrite any pre-existing value associated with a key.