Inside the Minds of Bug Bounty Researchers


Without a doubt, bug bounties continue to grow in popularity. Sponsors and attendees at this year’s Blackhat USA 2015 conference in Las Vegas seemed to share this same sentiment.

One of the biggest names in software didn’t hold back, making their plans for the future well known. Jason Shirk, security architect at Microsoft responsible for their bug bounty program, announced in a blog post that there are a number of changes to the software giant’s bug bounty program. With a heavy focus on the newly released Windows 10 operating system, the most notable change can be found in the the reward for a qualifying Mitigation Bypass submission being doubled from $50,000 to $100,000, which Shirk said will "bring defense up on par with offense." To further promote their program, Microsoft held a party in honor of their big bounty researchers, feeding and hydrating the attendees while flashing the names and underground researcher handles of the top 100 on the screens above the audience.

Also, right around this same time, Facebook announced a move to invest more in their security efforts by hiring former Yahoo CISO, Alex Stamos. During Stamos’ tenure at Yahoo, he and his team of “Yahoo Paranoids” established a formal bug bounty program which has paid out more than $1 million in rewards to its community of researchers. One can only expect Stamos to continue the tradition of bug bounties at Facebook – something the company is familiar with but could certainly benefit from a boost of top-notch security-minded experience.

Bugcrowd, providers of a crowd-sourced bug bounty platform, took advantage of the researchers (aka hackers) gathered at the world’s top hacker event to release their inaugural State of Bug Bounty report. The 21-page report, which shows that incentivized, invitation-only programs are the best way to Improve the signal-to-noise ratio for a company’s security assessment program, appeared to garner a lot of attention during the Blackhat conference.

A few highlights from the report include:

  • During the reported period, 17,994 unique researchers from 147 countries collectively submitted 37,227 bugs, of which 3,621 were rewarded for a total payout of $724,839. In this same period, we saw a top reward of $10,000 for a single valid submission.
  • The number of invitation-only programs running recently surpassed public programs with increased value realized in invitation-only programs that offer better signal-to-noise ratio results: 36% invitation-only submissions are marked valid compared to half as many (18%) from public programs.
  • Companies running bug bounty programs typically include large tech companies, though Bugcrowd’s platform data shows that a growing number of organizations outside the high-tech industry are joining the bug bounty party.

Understanding these and other State of Bug Bounty metrics can provide a customer a very valuable security playbook, such as the type and severity of vulnerabilities found in real-world production sites–not to ignore the interesting details of who the researchers are and what motivates them.

Honoring their community of researchers while at the same time bringing together some of the best researchers on the planet, the Bugcrowd team also hosted a get-together during the Blackhat conference. Unlike Microsoft’s massive free-for-all, Bugcrowd’s gathering was more intimate, enough so that I actually had a chance to get a good chat going with a few of the researchers. I also took this opportunity to chat with Bugcrowd’s Director of Technical Operations, Jason Haddix, whose team effectively functions as the conduit and support between the researchers and the companies that are running the bounties on Bugcrowd’s platform.

During my conversation with Haddix, he pointed out that there are a few types of researchers – those interested in payouts, others focused ranking points, and the remainder fine with a combination of the two (money and points). Some of the researchers I spoke with (they asked to remain anonymous) were definitely all about the money - points and notoriety didn't matter to them in the slightest. Other researchers I spoke with saw the value in the points, using them as a means to gain a competitive edge over other researchers – this notoriety comes in handy when the invites get sent out for the invitation-only programs. The researchers interested in the points were younger, less established researchers and needed the recognition.

In addition to points, Bugcrowd often provides other avenues for lesser known researchers to get their name out in the security community: guest blogs, interviews, and podcasts are all popular brand-building vehicles for researchers.

One of the most successful, and arguably one of the most established, researchers at the moment is Bitquark. While an exact figure wasn't mentioned when speaking with him, it's a safe bet to say Bitquark has earned tens of thousands of dollars participating in Bugcrowd and other bug bounty programs, including Facebook and Google. He said he hasn’t engaged much in Microsoft bug bounty programs as he had plenty of other things on his plate already.

Of the bug bounty platform options available, Bitquark said he really likes Bugcrowd for a few reasons:

  1. The platform is ready to use
  2. There are a good variety of programs to participate in
  3. The Bugcrowd team is great in how they support the researchers and help the companies get the most out of their programs

Bitquark, whose real name is actually Jon (sorry, there’s no last name printed on this business card…), now does security research for Tesla - who also runs a bug bounty program via Bugcrowd under Bitquark’s watch. Speaking of Tesla, they partnered with Bugcrowd to pay the two researchers who publicized security research for Tesla’s Model S to present at DEFCON and Blackhat. Kudos to these two companies for investing in the research community in this way.

With his role at Tesla keeping him very busy, Bitquark isn't participating in as many bug bounty programs as he has in the past; though he still enjoys applying good skills to interesting projects when he can find the time.

Bitquark attributes his success to focusing on the areas he believes will have the highest impact and produce the best results for the company when the issues he identifies are fixed. These are often logic bugs.

Bitquark will look at the app as if he were looking at it from inside the code. He will identify the steps involved in a process, and mess with them; skip a step, repeat a step, even break or end the process before it is supposed to finish. He will also look at validation bugs, as another example; he finds great success there as well.

Whatever Bitquark focuses on, he tends to follow a pattern. “It isn't something I purposefully sit down and say ‘OK, now I will do 1, 2, 3’,” said Bitquark. “It's more second nature to me than that.”

When he hits a roadblock in his research, Bitquark takes a step back and presses forward again, using his knowledge and experience for how he has previously busted through those roadblocks. “I definitely keep in touch with other researchers to stay on top of the latest in research techniques and findings,” he said when asked if he collaborated with other researchers. “In some cases, I even discuss a roadblock with another researcher as a means to break through it. This isn’t the case very often though.”

Regardless of the motivation behind the testing, all around, there was consensus amongst the researchers that they want to do the right thing. Researchers want to help companies find and fix vulnerabilities in their software and their hardware – and they enjoy the challenge in doing so.

“Every bug found is important,” said Haddix. “It's easy as a researcher to get frustrated with the developers when you find a no-brainer bug – they should know better. It's equally easy to get frustrated with the companies paying (sometimes big money) to find a bug…a bug that should not be left for a big bounty researcher to find,” he added.

“If the money is good, we (the researchers) don't mind,” added one of the community researchers, in response to finding simple stupid bugs. “If it's for points, and if we don’t care about the points, then spending time finding a no-brainer bug can be viewed as a waste of time,” the anonymous researcher added. In Bugcrowd’s case, they use these low-hanging-fruit bugs as an opportunity to educate the customer.

Let us not get too complacent… for every role involved in a bug bounty, education remains a key element.