A few weeks ago we posted a challenge, inviting all applicants for our Web Engineer position to hack into our backend. As Jack explained, posting jobs on a number of the standard behemoth career sites is futile; providing a deluge of unqualified applicants that just aren't the right fit. Knowing that the kind of person we're looking for isn't the type to browse around Career Builder and Monster.com, we decided to build this puzzle, to attract intelligent and creative minds.
The response was overwhelming.
Immediately after the post went live, it was submitted to Hacker News where it quickly jumped to the front page. Later it was submitted to reddit and somehow appeared on Spanish Slashdot clone Barrapunto. This approach to hiring seemed to strike a nerve with a lot of folks. One commenter summed up our intentions by saying "This is an excellent way to screen applicants, and to find candidates that truly 'live and breathe' PHP and MySQL." A hacker with nothing better to do even spammed our inbox with thousands of "YO MOMMA" emails. Said hacker aptly used the comment section of the application form to inquire "can haz moar diskspace?".
A number of people initially balked at the test's difficulty, one saying "Wow, that took me 2 seconds," not realizing they had only completed one step of the three step solution. But among those who actually completed the puzzle, there was a lot of discussion (both internally within SeatGeek and externally on Twitter/Reddit/HN/etc.) about the test's difficulty. Some highly capable web developers solved it in as little as 7-8 minutes and complained that it wasn't hard enough. But our goal here wasn't to create a puzzle so difficult that all who solved it would be decidedly hireable. We just wanted a reliable initial screen--something that filtered out the career "CSS experts" and those who were otherwise incompetent. Many spent an entire afternoon working on this thing without success. I hope they liked the challenge, but I'm also glad we didn't have to use our internal resources reviewing their resume.
We thought you'd be interested in seeing the results of the challenge, and pulled the traffic and submission numbers from October 25th through today:
- Our blog post received 10,647 pageviews from 9,088 unique visitors.
- The Hacker News post had 37 comments and sent 2,360 visitors.
- Spanish Slashdot clone Barrapunto sent 2,008 visitors.
- The reddit post had 50 comments and sent 1,942 visitors.
- Facebook and Twitter sent 844 visitors.
- 366 succesful submissions to the challenge.
- ~100 serious applicants. It's worth noting that a majority of "applicants" weren't interested in the job; they just wanted a challenge.
- 28 people made it to a phone screen.
- 12 people made it to an in person interview.
The location of visitors spanned far and wide across the world, the headline for this post pulled from a blogger in Argentina who went out of his way to write up a solution (translated) to the challenge. We saw retweets in several languages, and pulled the geographic data from our analytics:
Unfortunately our logs refresh every 10 days, but we took a sample of submissions to determine the time it took each applicant to complete the challenge:
We're pleased with the results of this experiment--we learned a lot and will soon hire an applicant who applied through this screen. What do you think of the way we approached the application process? How would you change or improve it to increase the quality of applicants? Let us know in the comments or on Twitter @SeatGeek.