Installing Redis 2.4 (or latest) on dotCloud
UPDATE: Redis 2.4 is now supported as a dotCloud service, but this article applies to any custom version of Redis on dotCloud
I’ve been using dotCloud for about 6 months now and I think it’s a terrific service. Unfortunately, their services usually lag a version or 2 behind (no judgement, I’m sure it takes incredible manpower to keep so many services functioning, much less up-to-date). Someone over there did have the foresight to allow custom build scripts for almost any service you can think of though. It’s one of these custom scripts we’ll be creating today.
Version 0.8.0 of Nohm, an ORM for node.js and Redis, adds a lot of really nice features like built-in sorting (actually added in 0.7.0 I think) and Pub/Sub at the model-level. I’m currently building the user dashboard for Netmarks, and both of these features were a perfectly timed godsend. However, Nohm 0.8.0 requires Redis 2.4, and dotCloud’s Redis implementation is only version 2.2 at the time of this post.
This brings us to today, where I’ll go through how to install Redis 2.4 (or whatever the latest version is) on dotCloud and migrate your existing data. (the completed files are on Github if you don’t want to read the rest). First, we’ll create the dotcloud.yml file necessary to create any service on dotCloud.
redis: type: custom buildscript: build systempackages: - gcc ports: redis: tcp process: ~/redis-server ~/redis.conf
Simple, right? If you aren’t familiar with the dotCloud build file you should read this first, otherwise, to the breakdown!
Looks easy enough, but what the heck is going on here? Thanks so much for asking. I’ll tell you.
This tells the dotCloud builder to use the custom script we’ll write in a minute to install and initialize the service, not a pre-packaged dotCloud service.
The name of the BASH script we’ll use to install Redis on the dotCloud instance. This is a relative url, so be sure to provide the full path if it doesn’t reside in the same directory.
systempackages: - gcc
Redis requires GCC be installed. We’re just making sure the server we’re pushing to installs it if necessary.
ports: redis: tcp
This part actually took me a bit to figure out. What we’re asking dotCloud to do here is open a TCP port named PORT_REDIS in the environment files and assign the port number to an environment variable with the same name. As you’ll see later though, this is not the port on which we connect to Redis. It looks like dotCloud proxies requests from a public port (named “DOTCLOUD_DB_REDIS_PORT” if you copy my configuration) to PORT_REDIS. In the
postinstall script below you’ll see we send that info as output to make it easier on the user.
process: ~/redis-server ~/redis.conf
Start Redis. The redis.conf referenced there is the next thing we’ll create. The
build script moves all the relevant files to the home directory so they’re easily referenced and executed.
daemonize yes save 900 1 save 300 10 save 60 10000 requirepass change_me
* notice the extra line breaks at the bottom, they’re intentional
I’m not going to go into too much detail here. In the Redis Quickstart Guide there’s a section titled “Installing Redis More Properly” which talks about managing processes and such. Since dotCloud handles that for us with
supervisord the only thing we take away is adding
daemonize yes to the config.
The next set of lines determine how often Redis writes the content in memory to the disk based on the number of transactions that occur. The Redis website explains all of this better than I ever could.
Lastly, we add a password for our database. It’s one line in the config and takes 2 seconds, please add a password. Especially if there’s the slightest possibility you’ll ever use it in production. Trust me, it’s a good idea.
Like I said, the Redis website and installation have documentation on the default config. If you want to get the most out of Redis, it’s seriously worth your time to read through it. Now, let me show you where the magic happens
The build file
#!/bin/bash wget http://download.redis.io/redis-stable.tar.gz tar -xvf redis-stable.tar.gz cd redis-stable make cd .. cp -a . ~/ cd redis-stable/src cp redis-server ~/ cp redis-cli ~/
If you’ve never installed Redis before and you skipped the Quickstart Guide above, now is the time to read it.
We’re creating a bash script here. Safe bet that this will work on any Linux installation.
wget http://download.redis.io/redis-stable.tar.gz tar -xvf redis-stable.tar.gz cd redis-stable make
Download, unzip, and
make Redis. Pretty standard stuff here.
cd .. cp -a . ~/
Navigate up to the temp directory that we’re installing from and copy all the files to the dotcloud user’s home directory. Not only are some of these files going to be necessary in the following steps, but it also helps to have access to these files if something goes wrong. dotCloud doesn’t like to let you out of your sandbox to poke around.
cd redis-stable/src cp redis-server ~/ cp redis-cli ~/
Jump back into the installation directory and copy our executables into the home directory. The
process: ~/redis-server ~/redis.conf in dotcloud.yml relies on these files being in the home directory.
#!/bin/bash #assign the port in the ENV vars to the redis.conf echo port $PORT_REDIS >> ~/redis.conf #output the info to the CLI user echo '********************' echo 'Use the following url to access this service:' echo $DOTCLOUD_DB_REDIS_URL echo '********************'
This is just a little script that runs after Redis has finished installing and performs 1 very important function: adding the port number assigned by dotCloud to the redis.conf.
echo port $PORT_REDIS >> ~/redis.conf
This ensures that Redis is listening on the correct port when the requests are proxied from the public port which we find in
The output at the bottom of
postinstall prints the host and port to the console so you, the user, can easily update your app without having to SSH into the dotCloud service and look it up. You’re welcome
I was going to go into migrating your data from an existing Redis installation on dotCloud, but this post is starting to get really long. If you’re interested in migrating your data then feel free to shoot me an email or look into the Redis
SAVE command to get you started. Feel free to leave a comment below if you’re having trouble, and I’ll try to help out as much as I can.
UPDATE: I found this article, which is the best way I’ve seen to migrate your data from one Redis server to another.
If you enjoyed this post, please follow me on Twitter.