We've recognized it time and time again.
Just this past week, the GHOST issue (glibc gethostbyname buffer overflow) popped up. Many people all over the world were scrambling to patch their servers. Large companies and small companies alike, they all had to deal with it one way or another.
As usual with anything like this, we looked to Engine Yard for an update. Our app servers and database servers are hosted on Amazon EC2 and Engine Yard is a PaaS that sits on top of it. Engine Yard provided their customers with a timely update about the issue and how they planned to solve it.
They've gained our trust over time, so we simply stopped worrying about it and waited until they released their patch. Once they released their patch, we followed their detailed instructions and applied the patch on our own schedule. As with something as far reaching as GHOST, it wasn't super easy, but we made it through and we're now the proud owners of patched servers and relaxed minds.
With our servers hosted on vanilla EC2 and without a talented, dedicated sysops guy, this situation would have been a nightmare for us. Instead of shipping new improvements to our product, we would've been at a full stop investigating, experimenting, testing, etc. etc. We don't consider ourselves experts when it comes to managing servers. While most of our engineers have plenty of experience with Linux, we don't want to mess around with sysops.
We want to focus on what we do best - releasing software.
Do we always want to be afraid of Chef recipes, server updates, automated provisioning and the like? Certainly not. However, at this point, we don't want to rely on it as a core competency.
By not hosting your product on a PaaS, you're designating sysops as a core competency of your engineering team. If you don't, you'll pay the price.
One day, you might pay the price in security. You won't have anyone watching out for security issues and your servers will go unpatched, open to abuse.
Next time, you might lose a week or two on scaling. You won't have a proper, load-balanced cluster. Your servers will be on fire. Your engineering team will be firefighting for days. They'll have to learn Chef, haproxy, and countless other things they haven't been spending time with.
You'll most definitely be paying the price during your day-to-day operations. Your deploys will cause downtime. Deploys will be frowned upon. Your engineering won't be happy. They'll have to stay late to deploy something that should've been deployed as soon as it was ready.
Companies that manage everything from deployments to automated scaling are saying, "Hey, sysops is important to us. It's a core competency of our business. We get huge value from it." If you're trying to imitate them and you don't have anyone focused on sysops, you're not ready. Companies like Facebook get huge value from their data centers. They need them. If you're not ready to hire someone with a specific focus on the subject, you don't.
The argument for small businesses and PaaS is a numbers game - in a good way. You might think it's more expensive at first, but it's not. Here's an easy list of things we receive from Engine Yard:
- Someone to talk to when a weird problem pops up
- Someone to be on the lookout for security patches
- Someone to ask about scaling
- Someone to alert us about server maintenance on EC2
- Someone to improve our server configuration
That's just the short list. When you start to think about the amount of value you get from a PaaS, it's enormous. It's far cheaper than what you would be paying someone in annual salary to be your sysops guy.