Redis: A terrible way to migrate your cache
Ilhan Ates

Ilhan Ates / January 31, 2023

3 min read––– views

Due to terrible business decisions, you stored persistent data on redis in an unorganized fashion with no backups, and now you want to move it to another instance.

An oddly specific, weird and (excuse my incapability to find another interesting adjective) bad situation, and I unfortunately know what you can do.

Before starting, just noting you have alternatives that might work much better depending on your sitation: like this.

The terrible command

redis-cli keys '*' | xargs -I '{}' redis-cli migrate <target-host> <target-port> "" <target-db-number> 5000 KEYS '{}'

  • redis-cli keys '*' This first part gets ALL of the keys (in sync, by the way, so your database will freeze). You might not want to do this if your redis db is somehow enormous.
  • xargs -I '{}' This defines a replace-string with the token . In other words, we can now use in the rest of the command and it will run for every line of output from the first part, where is replaced by the output.
  • redis-cli migrate This is a command to migrate one key from one host to another. You can check out the migrate command here. <target-db-number> should probably be 0 if you're not using multiple databases in one redis instance.

Authentication

You might be using authentication on your databases! In that case you can use this:

redis-cli -a <source-auth> keys '*' | xargs -I '{}' redis-cli -a <source-auth> migrate <target-host> <target-port> "" <target-db> 5000 KEYS '{}' AUTH <target-auth>

Database Number

You might also want to specify a database number, which I had to (with great shame). In that case you can use the following:

redis-cli -a <source-auth> -n <source-db> keys '*' | xargs -I '{}' redis-cli -a <source-auth> -n <source-db> migrate <target-host> <target-port> "" <target-db> 5000 KEYS '{}' AUTH <target-auth>

Warning!

This will DELETE the key on the source after creating the exact key:value on the target. The deletion is done for every single key one by one right after moving, not at the end of this entire process.

I once wiped half of my redis database accidentally because this command was taking a long time to run, and I flushed the target db after killing the process, thinking it got stuck and this would fix it. Apparently it did move a good 70% of the keys already and I just flushed those away.

The Verdict

Although it's pretty terrible practice, I wanted to document this. The world is big and there might be other people using redis in a terrible way like I did.