Scripting Redict with Lua #

Redict provides a programming interface that lets you execute custom scripts on the server itself. In Redict, you can use Redict Functions to manage and run your scripts.

Background #

Redict is a “domain-specific language for abstract data types”. The language that Redict speaks consists of its commands. Most the commands specialize at manipulating core data types in different ways. In many cases, these commands provide all the functionality that a developer requires for managing application data in Redict.

The term programmability in Redict means having the ability to execute arbitrary user-defined logic by the server. We refer to such pieces of logic as scripts. In our case, scripts enable processing the data where it lives, a.k.a data locality. Furthermore, the responsible embedding of programmatic workflows in the Redict server can help in reducing network traffic and improving overall performance. Developers can use this capability for implementing robust, application-specific APIs. Such APIs can encapsulate business logic and maintain a data model across multiple keys and different data structures.

User scripts are executed in Redict by an embedded, sandboxed scripting engine. Presently, Redict supports a single scripting engine, the Lua 5.1 interpreter.

Please refer to the Redict Lua API Reference page for complete documentation.

Running scripts #

Redict provides two means for running scripts.

Firstly, the EVAL command enables running server-side scripts. Eval scripts provide a quick and straightforward way to have Redict run your scripts ad-hoc. However, using them means that the scripted logic is a part of your application (not an extension of the Redict server). Every applicative instance that runs a script must have the script’s source code readily available for loading at any time. That is because scripts are only cached by the server and are volatile. As your application grows, this approach can become harder to develop and maintain.

Secondly, Redict Functions are essentially scripts that are first-class database elements. As such, functions decouple scripting from application logic and enable independent development, testing, and deployment of scripts. To use functions, they need to be loaded first, and then they are available for use by all connected clients. In this case, loading a function to the database becomes an administrative deployment task (such as loading a Redict module, for example), which separates the script from the application.

Please refer to the following pages for more information:

When running a script or a function, Redict guarantees its atomic execution. The script’s execution blocks all server activities during its entire time, similarly to the semantics of transactions. These semantics mean that all of the script’s effects either have yet to happen or had already happened. The blocking semantics of an executed script apply to all connected clients at all times.

Note that the potential downside of this blocking approach is that executing slow scripts is not a good idea. It is not hard to create fast scripts because scripting’s overhead is very low. However, if you intend to use a slow script in your application, be aware that all other clients are blocked and can’t execute any command while it is running.

Read-only scripts #

A read-only script is a script that only executes commands that don’t modify any keys within Redict. Read-only scripts can be executed either by adding the no-writes flag to the script or by executing the script with one of the read-only script command variants: EVAL_RO, EVALSHA_RO, or FCALL_RO. They have the following properties:

They can always be executed on replicas.
They can always be killed by the SCRIPT KILL command.
They never fail with OOM error when Redict is over the memory limit.
They are not blocked during write pauses, such as those that occur during coordinated failovers.
They cannot execute any command that may modify the data set.
Currently PUBLISH, SPUBLISH and PFCOUNT are also considered write commands in scripts, because they could attempt to propagate commands to replicas and AOF file.

In addition to the benefits provided by all read-only scripts, the read-only script commands have the following advantages:

They can be used to configure an ACL user to only be able to execute read-only scripts.
Many clients also better support routing the read-only script commands to replicas for applications that want to use replicas for read scaling.

Sandboxed script context #

Redict places the engine that executes user scripts inside a sandbox. The sandbox attempts to prevent accidental misuse and reduce potential threats from the server’s environment.

Scripts should never try to access the Redict server’s underlying host systems, such as the file system, network, or attempt to perform any other system call other than those supported by the API.

Scripts should operate solely on data stored in Redict and data provided as arguments to their execution.

Maximum execution time #

Scripts are subject to a maximum execution time (set by default to five seconds). This default timeout is enormous since a script usually runs in less than a millisecond. The limit is in place to handle accidental infinite loops created during development.

It is possible to modify the maximum time a script can be executed with millisecond precision, either via redict.conf or by using the CONFIG SET command. The configuration parameter affecting max execution time is called busy-reply-threshold.

When a script reaches the timeout threshold, it isn’t terminated by Redict automatically. Doing so would violate the contract between Redict and the scripting engine that ensures that scripts are atomic. Interrupting the execution of a script has the potential of leaving the dataset with half-written changes.

Therefore, when a script executes longer than the configured timeout, the following happens:

Redict logs that a script is running for too long.
It starts accepting commands again from other clients but will reply with a BUSY error to all the clients sending normal commands. The only commands allowed in this state are SCRIPT KILL, FUNCTION KILL, and SHUTDOWN NOSAVE.
It is possible to terminate a script that only executes read-only commands using the SCRIPT KILL and FUNCTION KILL commands. These commands do not violate the scripting semantic as no data was written to the dataset by the script yet.
If the script had already performed even a single write operation, the only command allowed is SHUTDOWN NOSAVE that stops the server without saving the current data set on disk (basically, the server is aborted).