Project IdealismI've been hearing it a lot lately. Maybe you've been hearing it, too. "Engineering is the easy part," they say.
Try telling that to Twitter in early 2007 when they faced massive growth and intense scaling issues. I don't think you'd find many engineers at Twitter telling you how easy it is to manage 600 requests per second and 180 Rails instances backed by 1 database server.
The point is: engineering is hard. Everything is hard.
Ok, maybe I'm exaggerating a bit here. There is one case where "engineering is the easy part."
Let's say you have an idea. You want to build that idea into a weekend project. You've probably heard of this happening quite a bit. How many "I built this in a weekend while watching every episode of House of Cards" blog posts have you read over the years? Building a project in a weekend is easy. You shouldn't be focused on perfection. Most everything works the way it should, but it definitely isn't refined. If you overcome your lizard brain, you can ship it and see what people think.
Let's say a few people like what you've built. You have users! They start using the app as any good user should do. Guess what? They find bugs. Now, you've got to fix them (if you want your users to continue liking what you've built). Better yet, it's not just one bug. This bug, that bug. You have to prioritize bugs. You can't fix them all at once - even if each user cares passionately about the bug they reported.
Still easy?
BUT! Remember, you took some shortcuts so you can finish your project in a weekend? You have a choice to make. You can put a band-aid on it and simply patch it up, or you can do some refactoring for the future to make sure it doesn't happen again. You might even want to write a test for it - you know, because TDD 4 lyfe.
Now you've fixed those bugs and your users are happy again. Slowly, they're telling their friends and you start to get some SEO juice with Google. But, guess what? You're not done yet. These new users want new features. Surprise, surprise - your weekend project can't do everything a user might want. Better yet, you're in the same situation as the bugs. Just like all the bugs, every user has a feature they need.
You have another engineering decision to make. How does this new feature integrate into the code you've already written? You'll need to plan for the future. There will always be new features. You don't want to screw yourself over in six months. Granted, some features are pretty easy and you've done them before, but there's always a slight variation. There might even be a library out there to help you, but once again, you'll likely have to alter its behavior - even if it's just a little bit.
Still easy?
Your weekend project is starting to turn into an actual product. You have a faithful group of early adopters. They're always looking out for you and sharing what they're thinking. You've been able to keep them happy through bug fixes and new features. You still have bugs and you have a giant backlog of new features; however, you've been able to navigate successfully until this point. Your app is still fast, but not quite as fast as when you first started. Your app works - there are some bugs, but for the most part people can use your app with success. For the most part, people are happy.
You're starting to grow faster now. People are paying attention. Remember Twitter? You might start feeling their pain. Not many apps have massive scaling pains like Twitter, but most apps go through periods where you have to adjust. Your weekend project was built and tested with one user, you. Now, you might have hundreds, maybe thousands, of people using your app on a daily basis. This is entirely different than one user. New problems will popup because your app is serving more than just one user.
Source: http://i.imgur.com/te0y7.jpg
These are great problems to have, but they definitely aren't easy.
Engineering is the easy part only applies to companies that stop at Version 1.0. How many companies do you know that stop there? None. The engineering team gets you to Version 20.0. Through bug fixes, new features, scaling, migrations, architecture, data centers, etc., your engineering team is there every step of the way.
It took Twitter months of hard work to straighten out their scaling issues. It most certainly wasn't easy. They failed - quite a bit, but they didn't give up. Through months of pain, they eventually came up with solution after to solution to fix the fail whale. Their solutions were most definitely not easy. Keep in mind - they did all of this while people were still using Twitter. Downtime is just unacceptable in this day and age.
On the flip side, I had the opportunity to observe what happens when you have an engineering team that runs into major problems when upgrading a product. Now, I'm positive this team had the best of intentions and I'm positive they were trying as hard as they could to avoid these issues. I'm also positive they'll learn from these mistakes. However, they failed massively, and they know it.
Engineers are responsible for many different things. If you don't do one of those things right, it can snowball into other problems.
Communication is key. As an engineer, writing code is a small part of the job. In practice, most engineers write code for a fraction of their day. Instead of writing code, they're communicating with project managers to estimate time to production. They're talking with the designer about ways to work a new design into the existing product. They're thinking about what's next and how they'll manage the next database update required for that upcoming feature in three months. They're worrying about how their code base isn't as stable and well-tested as they had hoped for.
Most importantly, all of this isn't easy - not by a long shot. This is hard stuff. Don't buy into the link-bait. Engineering is not easy. Just like customer support, project management, and product ownership - engineering is a key part of your organization. If you think you can just throw bodies at an engineering problem, you're wrong.
Those bodies might be able to solve the problem at hand. They might get you somewhere and help you raise funding. However, those bodies certainly aren't worried too much about the future. They aren't setting you up for long-term success. You'll pay dearly for it later on.
Engineering isn't easy if you're doing it right. Engineering should be invested for the long-term. When you finally get there, you'll have an engineering team fully invested in your organization. Your engineering team will be a valuable asset full of talented people that know every tool, piece of software, and server your company has in its possession.
Don't say engineering is the easy part. It's not.
Thanks to @andrewwicklandr and @alexlaprade for reviewing this and providing some ideas for making it better.