I have a bad habit acquired from my years as a Dot Com CTO. When the time comes to pick a server for a new project, I always overbuy. I’d rather pay a hundred dollars more per month then have a server that can’t take the load. One of the driving forces behind this decision is the time it takes to migrate to a more powerful server, if I discover that it is needed. So I have a collection of dedicated servers that I lease from various webhosts that collectively waste about $500 a month. That’s a small percentage when considered as a cost of doing business across all my clients, but it is still wasteful.
I’ve thought about moving to Amazon’s AWS system, but every time I look at the docs I get turned off. I’m an application builder, not a professional sysadmin. I have no problem managing a Linux based server, but when it comes to configuring and optimizing Apache and MySQL, I turn to professionals. The AWS docs make it clear that this system is built by people who LOVE tweaking servers. They also seem to love really detailed command driven operations spanning several lines, with very complex parameter names, with very odd capitalization. I couldn’t care less about that. If I could just say, “Create a new server instance, and make it this big.” I’d be thrilled.
That is what I have now found with Rackspace.com. Their cloud servers let me clone multiple server instances, and upsize or downsize them with a menu. I’m testing this for a client who wants to collect tweets that have a lot of flow. His search terms for the Twitter streaming API retrieve about 60,000 tweets an hour. If I had to lease a dedicated server for this, I would have spent at least $150 to $200 a month to be sure I could handle the load. Instead I had my sysadmin create the cheapest server instance at Rackspace, at $11 per month. The entire pricing structure is here.
Once the basic server configuration with all of my code was set up, I made a server image with Rackspace’s control panel that could be used to create a new server instance in minutes without having to pay my sysadmin again. I ran the tweet collection for a few hours, and found that this server size was too small. The server load went up above 3.0, and queries were completely stalled. All I had to do was ask for the next size server ($22 a month) using the menu, and 10 minutes later I was up again with the new configuration. This ran much better for inserting tweets, but queries were still too slow. So after a few hours of watching the server, I decided to bump it up again to the next size at $44. This configuration looks like it will work. Server load is about 0.5, and the queries we need to run complete in a few seconds.
Overall this has been a great experience. I love the idea of being able to size up gradually until I see the server handling a real world load, and then downsizing if that load drops. I’m going to start moving some of my existing sites across to Rackspace next with the hope of saving at least $300 to $400 a month. Then I’l be experimenting with clusters of different sized servers to handle more complex site requirements.
