I like to analyze my past work through the prism of evaluating my mistakes. (I’m sure my therapist would have a field day with that one.) I launched my first publicly available engineering project in early December, and as George W. Bush once said, “Mistakes were made.”
I studied CS at Carnegie Mellon for undergrad, and as part of my classes, I built a number of projects. But it’s one thing to build something for a class where maybe one or two people will use it for a brief period, and another altogether to ship something for the general public.
Upon graduation, I worked at Google for three years, where, as you can imagine, they make it pretty much impossible for junior engineers to royally screw anything up (though I still found a way to, but that’s a story for another day).
After about a year of operating as a traditional recruiting agency (where we also made plenty of mistakes), my co-founder (and twin) had learned enough to launch a free software product to help engineers find new jobs in a non-annoying and non-spammy way. For about three months, I was heads-down building our tool that would index open engineering jobs across the internet, then hand-match an engineer to five job openings each month that matched their preferences and qualifications.
While I was maniacally coding, my brother was building our beta audience of early users. By the time we were ready to launch, we had 142 users completely signed up and ready to receive our job recommendations. (We’ll talk about the mistakes we made in building that audience in our next post.) After launching, it took me all of a minute to realize I made a big mistake. Talk about a letdown.
All in all, our beta launch has been a success. We’ve gotten really positive feedback from our early users, we know exactly what features we need to work on next, and we have a better sense of what kind of engineers find the most utility out of our tool. But my goal here is to highlight the engineering mistakes I made to help you avoid the same ones when you launch your next project. I also want to have this as a reference, so I never repeat these errors.
1. Didn’t Test Properly (Horrible, No Good, Very Bad Start)
In order to receive job recommendations from us, you have to complete a basic form, so we know what kind of engineering roles you’re looking for and where. Now, before we had that form built, we threw up a landing page and collected emails to see if people would be interested in what we were building.
When the signup form was ready, we emailed our mailing list, prompting them to complete that form. This was a really big moment for us. Anyone who completed the form would officially receive our job recommendations! My brother wrote the email, nervously hit send, and less than two minutes later, we had four emails letting us know the form didn’t work. Womp womp.
This was one of the lowest moments in our company’s history. We never wanted to be a recruiting agency. We did that for a year to learn what software product we could build to help engineers learn about new jobs. When we were in the drudges of recruiting, the launch of a software product was the light at the end of the tunnel. So now, in our first big moment for that product, something was wrong.
We tested that form. We tested it robustly with different inputs. We really, really tried to break it. So what happened? When I built the signup form, I had to change the way our backend was structured. When we were just collecting email addresses, we didn’t need much of a schema. But now we had users and attributes about those users, so we needed a real database structure.
When the new schema was built, we migrated all of the people who signed up with their emails to this new schema. When these people then tried to complete our signup form, they got an error saying, “this user already exists.” When we had tested the form, we only tested it with new email addresses, not in our system. Luckily, we were able to quickly fix the error (after five minutes of pure panic) with the help of these very generous early adopters.
This seems like such an egregious error in hindsight. It’s funny, I’m a solid developer, but when working on this project, I was juggling so many balls in the air that I made some basic mistakes. At Google, I was bowling with bumpers. When those bumpers got removed, it was like I had to re-learn some of the basics. Some things you only really learn by messing them up, and I can assure you, I won’t make this mistake again.
2. DevOps Will Be the Death of Me
Let me preface this section by saying: I don’t know anything about DevOps. As in, I almost never deployed code to my own server. So when building this project, I was looking for a service where I could push code to production and have our site work with little tuning or configuration involved on my end.
I worked on the Google Cloud team during my Google tenure, so I figured that’s what I should use when launching this project. This was a mistake (I’ll expound on the reasons why next). The overall lesson here is that I should have avoided the sunk cost fallacy and ditched GCP when there were early signs that it wasn’t what I was looking for.
My experience with GCP and AppEngine was not at all what I wanted. I assumed that AppEngine would be flexible enough to easily cater to our stack (Django, Angular, and at the time, MySQL). I also assumed that AppEngine and any other Google Cloud Product like CloudSQL would work nicely together with little configuration.
I just got too many errors I didn’t know how to handle, and there was little documentation or support online when you Googled (or DuckDuckGo-ed or Bing-ed or AltaVista-ed) the error. The straw that broke the camel’s back and forced us to switch off GCP was the inability to deploy code to production for two weeks.
I have a feeling if you know what you’re doing, the user experience of using AWS vs Azure vs GCP is largely the same (though I could be wrong about this). What’s frustrating is that even though I’m directly in the demographic of people these platforms want to attract, they don’t focus much attention on me.
We’ve since switched to Heroku, and I no longer worry about making sure our site stays up and running. It’s a bit more expensive (about $125 a month for us vs $50 for Google), but I place the value of my time and mental well-being at greater than $75 a month, so we’re okay with this trade-off.
3. Got a Case of “Shiny Objective Syndrome“
If you frequent forums or podcasts about startups, I’m sure the phrase “Minimum Viable Product” (MVP) has lost all meaning to you. It’s used so commonly to describe vastly different products and product stages, that it has just become another space on the board of “startup bingo.” To me, the fundamental idea of an MVP is to build the most basic version of your product to test if people will use it and to learn how to improve that product for the next version.
If there is a prevailing hypothesis in this blog post, it is the gap between reading and doing is the size of the Grand Canyon. Sure, I knew that version one of what I was building should be pared down as much as possible. If you look at the original list of features we thought needed to be in this version, I would say we got 90 percent of the way there.
Still, I let a cool engineering/math challenge inflate the work I should have done. In our first version of the product, my tool would whittle down thousands of engineering jobs for a given person to a list of about 30 jobs based on qualifying factors like location, job title, years of experience, etc. Then from that list, my co-founder and I would manually select five jobs each month that we thought were the best fit for that person.
You know what we didn’t need for this version? A convoluted way to score and sort the remaining list of 40-50 jobs to save us time in choosing the top five. I spent a lot of time coming up with a clever way to weigh factors that I thought mattered to engineers in determining if a given job was what they were looking for. As it turns out, the data we were automatically collecting on all of these jobs was not yet clean or standardized enough to make that score meaningful.
We quickly scrapped this ranking system because there was no correlation between the score and the likelihood we would choose a job for a given engineer. All in all, this took me about two days to develop, which would have been way better spent getting our product in front of users earlier to start getting feedback.
BeamJobs Blunders
Want to see how else we screw up in the future? Get a summary of our weekly posts.
Mistakes Were Made—A Reflection
I hope that reading this will help save you time and angry-typing into your computer when you launch your next project. I know I’ll continue to mess up—it’s just the nature of trying to build something new. What I want to avoid at all costs is messing up in the same way over and over.
So, I’ll write these posts on a semi-regular basis. I’ll also try to write about some of my successes because, despite the premise of this article, things are going pretty well for us right now! We have users who really like our product, and we’re solving a problem in a space we think is completely dysfunctional for job seekers. Until my next post, happy building.
