1. Overview
This article is an introduction to Lettuce, a Redis Java client.
Redis is an in-memory key-value store that can be used as a database, cache or message broker. Data is added, queried, modified, and deleted with commands that operate on keys in Redis’ in-memory data structure.
Lettuce supports both synchronous and asynchronous communication use of the complete Redis API, including its data structures, pub/sub messaging, and high-availability server connections.
2. Why Lettuce?
We’ve covered Jedis in one of the previous posts. What makes Lettuce different?
The most significant difference is its asynchronous support via the Java 8’s CompletionStage interface and support for Reactive Streams. As we’ll see below, Lettuce offers a natural interface for making asynchronous requests from the Redis database server and for creating streams.
It also uses Netty for communicating with the server. This makes for a “heavier” API, but also makes it better suited for sharing a connection with more than one thread.
3. Setup
3.1. Dependency
Let’s start by declaring the only dependency we’ll need in the pom.xml:
<dependency>
<groupId>io.lettuce</groupId>
<artifactId>lettuce-core</artifactId>
<version>5.0.1.RELEASE</version>
</dependency>
The latest version of the library can be checked on the Github repository or on Maven Central.
3.2. Redis Installation
We’ll need to install and run at least one instance of Redis, two if we wish to test clustering or sentinel mode (although sentinel mode requires three servers to function correctly.) For this article, we’re using 4.0.x – the latest stable version at this moment.
More information about getting started with Redis can be found here, including downloads for Linux and MacOS.
Redis doesn’t officially support Windows, but there’s a port of the server here. We can also run Redis in Docker which is a better alternative for Windows 10 and a fast way to get up and running.
4. Connections
4.1. Connecting to a Server
Connecting to Redis consists of four steps:
- Creating a Redis URI
- Using the URI to connect to a RedisClient
- Opening a Redis Connection
- Generating a set of RedisCommands
Let’s see the implementation:
RedisClient redisClient = RedisClient
.create("redis://password@localhost:6379/");
StatefulRedisConnection<String, String> connection
= redisClient.connect();
A StatefulRedisConnection is what it sounds like; a thread-safe connection to a Redis server that will maintain its connection to the server and reconnect if needed. Once we have a connection, we can use it to execute Redis commands either synchronously or asynchronously.
RedisClient uses substantial system resources, as it holds Netty resources for communicating with the Redis server. Applications that require multiple connections should use a single RedisClient.
4.2. Redis URIs
We create a RedisClient by passing a URI to the static factory method.
Lettuce leverages a custom syntax for Redis URIs. This is the schema:
redis :// [password@] host [: port] [/ database] [? [timeout=timeout[d|h|m|s|ms|us|ns]] [&_database=database_]]
There are four URI schemes:
- redis – a standalone Redis server
- rediss – a standalone Redis server via an SSL connection
- redis-socket – a standalone Redis server via a Unix domain socket
- redis-sentinel – a Redis Sentinel server
The Redis database instance can be specified as part of the URL path or as an additional parameter. If both are supplied, the parameter has higher precedence.
In the example above, we’re using a String representation. Lettuce also has a RedisURI class for building connections. It offers the Builder pattern:
RedisURI.Builder
.redis("localhost", 6379).auth("password")
.database(1).build();
And a constructor:
new RedisURI("localhost", 6379, 60, TimeUnit.SECONDS);
4.3. Synchronous Commands
Similar to Jedis, Lettuce provides a complete Redis command set in the form of methods.
However, Lettuce implements both synchronous and asynchronous versions. We’ll look at the synchronous version briefly, and then use the asynchronous implementation for the rest of the tutorial.
After we create a connection, we use it to create a command set:
RedisCommands<String, String> syncCommands = connection.sync();
Now we have an intuitive interface for communicating with Redis.
We can set and get String values:
syncCommands.set("key", "Hello, Redis!");
String value = syncommands.get(“key”);
We can work with hashes:
syncCommands.hset("recordName", "FirstName", "John");
syncCommands.hset("recordName", "LastName", "Smith");
Map<String, String> record = syncCommands.hgetall("recordName");
We’ll cover more Redis later in the article.
The Lettuce synchronous API uses the asynchronous API. Blocking is done for us at the command level. This means that more than one client can share a synchronous connection.
4.4. Asynchronous Commands
Let’s take a look at the asynchronous commands:
RedisAsyncCommands<String, String> asyncCommands = connection.async();
We retrieve a set of RedisAsyncCommands from the connection, similar to how we retrieved the synchronous set. These commands return a RedisFuture (which is a CompletableFuture internally):
RedisFuture<String> result = asyncCommands.get("key");
A guide to working with a CompletableFuture can be found here.
4.5. Reactive API
Finally, let’s see how to work with non-blocking reactive API:
RedisStringReactiveCommands<String, String> reactiveCommands = connection.reactive();
These commands return results wrapped in a Mono or a Flux from Project Reactor.
A guide to working with Project Reactor can be found here.
5. Redis Data Structures
We briefly looked at strings and hashes above, let’s look at how Lettuce implements the rest of Redis’ data structures. As we’d expect, each Redis command has a similarly-named method.
5.1. Lists
Lists are lists of Strings with the order of insertion preserved. Values are inserted or retrieved from either end:
asyncCommands.lpush("tasks", "firstTask");
asyncCommands.lpush("tasks", "secondTask");
RedisFuture<String> redisFuture = asyncCommands.rpop("tasks");
String nextTask = redisFuture.get();
In this example, nextTask equals “firstTask“. Lpush pushes values to the head of the list, and then rpop pops values from the end of the list.
We can also pop elements from the other end:
asyncCommands.del("tasks");
asyncCommands.lpush("tasks", "firstTask");
asyncCommands.lpush("tasks", "secondTask");
redisFuture = asyncCommands.lpop("tasks");
String nextTask = redisFuture.get();
We start the second example by removing the list with del. Then we insert the same values again, but we use lpop to pop the values from the head of the list, so the nextTask holds “secondTask” text.
5.2. Sets
Redis Sets are unordered collections of Strings similar to Java Sets; there are no duplicate elements:
asyncCommands.sadd("pets", "dog");
asyncCommands.sadd("pets", "cat");
asyncCommands.sadd("pets", "cat");
RedisFuture<Set<String>> pets = asyncCommands.smembers("nicknames");
RedisFuture<Boolean> exists = asyncCommands.sismember("pets", "dog");
When we retrieve the Redis set as a Set, the size is two, since the duplicate “cat” was ignored. When we query Redis for the existence of “dog” with sismember, the response is true.
5.3. Hashes
We briefly looked at an example of hashes earlier. They are worth a quick explanation.
Redis Hashes are records with String fields and values. Each record also has a key in the primary index:
asyncCommands.hset("recordName", "FirstName", "John");
asyncCommands.hset("recordName", "LastName", "Smith");
RedisFuture<String> lastName
= syncCommands.hget("recordName", "LastName");
RedisFuture<Map<String, String>> record
= syncCommands.hgetall("recordName");
We use hset to add fields to the hash, passing in the name of the hash, the name of the field, and a value.
Then, we retrieve an individual value with hget, the name of the record and the field. Finally, we fetch the entire record as a hash with hgetall.
5.4. Sorted Sets
Sorted Sets contains values and a rank, by which they are sorted. The rank is 64-bit floating point value.
Items are added with a rank, and retrieved in a range:
asyncCommands.zadd("sortedset", 1, "one");
asyncCommands.zadd("sortedset", 4, "zero");
asyncCommands.zadd("sortedset", 2, "two");
RedisFuture<List<String>> valuesForward = asyncCommands.zrange(key, 0, 3);
RedisFuture<List<String>> valuesReverse = asyncCommands.zrevrange(key, 0, 3);
The second argument to zadd is a rank. We retrieve a range by rank with zrange for ascending order and zrevrange for descending.
We added “zero” with a rank of 4, so it will appear at the end of valuesForward and at the beginning of valuesReverse.
6. Transactions
Transactions allow the execution of a set of commands in a single atomic step. These commands are guaranteed to be executed in order and exclusively. Commands from another user won’t be executed until the transaction finishes.
Either all commands are executed, or none of them are. Redis will not perform a rollback if one of them fails. Once exec() is called, all commands are executed in the order specified.
Let’s look at an example:
asyncCommands.multi();
RedisFuture<String> result1 = asyncCommands.set("key1", "value1");
RedisFuture<String> result2 = asyncCommands.set("key2", "value2");
RedisFuture<String> result3 = asyncCommands.set("key3", "value3");
RedisFuture<TransactionResult> execResult = asyncCommands.exec();
TransactionResult transactionResult = execResult.get();
String firstResult = transactionResult.get(0);
String secondResult = transactionResult.get(0);
String thirdResult = transactionResult.get(0);
The call to multi starts the transaction. When a transaction is started, the subsequent commands are not executed until exec() is called.
In synchronous mode, the commands return null. In asynchronous mode, the commands return RedisFuture . Exec returns a TransactionResult which contains a list of responses.
Since the RedisFutures also receive their results, asynchronous API clients receive the transaction result in two places.
7. Batching
Under normal conditions, Lettuce executes commands as soon as they are called by an API client.
This is what most normal applications want, especially if they rely on receiving command results serially.
However, this behavior isn’t efficient if applications don’t need results immediately or if large amounts of data are being uploaded in bulk.
Asynchronous applications can override this behavior:
commands.setAutoFlushCommands(false);
List<RedisFuture<?>> futures = new ArrayList<>();
for (int i = 0; i < iterations; i++) {
futures.add(commands.set("key-" + i, "value-" + i);
}
commands.flushCommands();
boolean result = LettuceFutures.awaitAll(5, TimeUnit.SECONDS,
futures.toArray(new RedisFuture[0]));
With setAutoFlushCommands set to false, the application must call flushCommands manually. In this example, we queued multiple set command and then flushed the channel. AwaitAll waits for all of the RedisFutures to complete.
This state is set on a per connection basis and effects all threads that use the connection. This feature isn’t applicable to synchronous commands.
8. Publish/Subscribe
Redis offers a simple publish/subscribe messaging system. Subscribers consume messages from channels with the subscribe command. Messages aren’t persisted; they are only delivered to users when they are subscribed to a channel.
Redis uses the pub/sub system for notifications about the Redis dataset, giving clients the ability to receive events about keys being set, deleted, expired, etc.
See the documentation here for more details.
8.1. Subscriber
A RedisPubSubListener receives pub/sub messages. This interface defines several methods, but we’ll just show the method for receiving messages here:
public class Listener implements RedisPubSubListener<String, String> {
@Override
public void message(String channel, String message) {
log.debug("Got {} on channel {}", message, channel);
message = new String(s2);
}
}
We use the RedisClient to connect a pub/sub channel and install the listener:
StatefulRedisPubSubConnection<String, String> connection
= client.connectPubSub();
connection.addListener(new Listener())
RedisPubSubAsyncCommands<String, String> async
= connection.async();
async.subscribe("channel");
With a listener installed, we retrieve a set of RedisPubSubAsyncCommands and subscribe to a channel.
8.2. Publisher
Publishing is just a matter of connecting a Pub/Sub channel and retrieving the commands:
StatefulRedisPubSubConnection<String, String> connection
= client.connectPubSub();
RedisPubSubAsyncCommands<String, String> async
= connection.async();
async.publish("channel", "Hello, Redis!");
Publishing requires a channel and a message.
8.3. Reactive Subscriptions
Lettuce also offers a reactive interface for subscribing to pub/sub messages:
StatefulRedisPubSubConnection<String, String> connection = client
.connectPubSub();
RedisPubSubAsyncCommands<String, String> reactive = connection
.reactive();
reactive.observeChannels().subscribe(message -> {
log.debug("Got {} on channel {}", message, channel);
message = new String(s2);
});
reactive.subscribe("channel").subscribe();
The Flux returned by observeChannels receives messages for all channels, but since this is a stream, filtering is easy to do.
9. High Availability
Redis offers several options for high availability and scalability. Complete understanding requires knowledge of Redis server configurations, but we’ll go over a brief overview of how Lettuce supports them.
9.1. Master/Slave
Redis servers replicate themselves in a master/slave configuration. The master server sends the slave a stream of commands that replicate the master cache to the slave. Redis doesn’t support bi-directional replication, so slaves are read-only.
Lettuce can connect to Master/Slave systems, query them for the topology, and then select slaves for reading operations, which can improve throughput:
RedisClient redisClient = RedisClient.create();
StatefulRedisMasterSlaveConnection<String, String> connection
= MasterSlave.connect(redisClient,
new Utf8StringCodec(), RedisURI.create("redis://localhost"));
connection.setReadFrom(ReadFrom.SLAVE);
9.2. Sentinel
Redis Sentinel monitors master and slave instances and orchestrates failovers to slaves in the event of a master failover.
Lettuce can connect to the Sentinel, use it to discover the address of the current master, and then return a connection to it.
To do this, we build a different RedisURI and connect our RedisClient with it:
RedisURI redisUri = RedisURI.Builder
.sentinel("sentinelhost1", "clustername")
.withSentinel("sentinelhost2").build();
RedisClient client = new RedisClient(redisUri);
RedisConnection<String, String> connection = client.connect();
We built the URI with the hostname (or address) of the first Sentinel and a cluster name, followed by a second sentinel address. When we connect to the Sentinel, Lettuce queries it about the topology and returns a connection to the current master server for us.
The complete documentation is available here.
9.3. Clusters
Redis Cluster uses a distributed configuration to provide high-availability and high-throughput.
Clusters shard keys across up to 1000 nodes, therefore transactions are not available in a cluster:
RedisURI redisUri = RedisURI.Builder.redis("localhost")
.withPassword("authentication").build();
RedisClusterClient clusterClient = RedisClusterClient
.create(rediUri);
StatefulRedisClusterConnection<String, String> connection
= clusterClient.connect();
RedisAdvancedClusterCommands<String, String> syncCommands = connection
.sync();
RedisAdvancedClusterCommands holds the set of Redis commands supported by the cluster, routing them to the instance that holds the key.
A complete specification is available here.
10. Conclusion
In this tutorial, we looked at how to use Lettuce to connect and query a Redis server from within our application.
Lettuce supports the complete set of Redis features, with the bonus of a completely thread-safe asynchronous interface. It also makes extensive use of Java 8’s CompletionStage interface to give applications fine-grained control over how they receive data.
Code samples, as always, can be found over on GitHub.