Crowdsourcing application security closes the automated assessment gap

You could say Kris Lahiri, VP operations and chief security officer at enterprise file sharing platform provider Egnyte, was a bit of a skeptic when he first considered adding crowd-sourced penetration testing to the firm’s application security regimen. Indeed, the idea of giving permission for a bunch of unknown eyes to scour over their systems to see what they uncover is enough to make many security professionals hesitant.

Over the years since its founding in 2007, Egnyte’s approach to ensuring it was releasing software that didn’t place customers at-risk went through the evolution one would expect. Initially, the company identified and mitigated web application flaws that slipped through development with manual web application tests, explains Lahiri, but hiring outsiders to conduct software code assessments proved to be more time-consuming for their pace of updates. “We realized that the entire process takes about two to three weeks, and we could never move rapidly. Being a software-as-a-service company, we are innovating fast,” Lahiri says. Lahiri explains that, typically, Egnyte publishes new software updates, features and enhancements every two weeks. “It became clear that deep-dive manual application security assessments every six months, while valuable, is too slow,” he says.

So that their application security assessments kept pace with the frequency of their software updates, Lahiri and his team turned to automated web application security assessment services. “While these platforms do check apps for potential flaws, and are quite effective, they do require considerable training to learn how an application works to be optimally effective,” he says. Lahiri says he wasn’t comfortable with the lag time between when an update is published or a new application is released and when a web application assessment tool became adequately trained.

Also, even when fully trained it is possible for web application assessment tools to miss software flaws. This is especially true for web applications, which tend to be more dynamic than most other types of applications. “Web application assessment software also lags behind development trends and toolsets. Development tools change so often that web application security assessors need to stay very focused just to keep up,” Lahiri says.

“While we realized that we had to pay more attention to training automated software assessment tools, we also realized that there were many types of risks, such as missing some input, or social engineering type attacks, or someone trying to escalate privileges that are not readily, or even possible, to detect in purely automated way,” Lahiri says.

The decision to crowdsource application security

Lahiri began to consider adding crowd-based software security testing provided by application security startup Cobalt Labs to Egnyte’s processes. The idea would be to find any security related flaws that made it past internal software security tests during development, automated application security tests, and periodic manual web application pen tests. But he remained skeptical. “My first doubt was because we are a startup and weren’t interested in running a public bounty program as a Facebook or Google would. Also, I wasn’t sure about the type or quality of researchers we’d get. Finally, I worried that a flaw uncovered could become public and tarnish the company brand,” he says. “I hesitantly went ahead, and we tried a crowd-sourced application security program,” he says.

Most understand the benefits of software security code reviews or bug bounty programs. A crowdsourced penetration test combines some elements of both: crowdsourced code review with the structure of traditional pen tests — only a crowdsourced application pen test is limited to security researchers who are established with a third-party. Think of it as a private, but third-party curated, software assessment.

Lahiri and his team decided they’d scope a crowdsourced penetration test. “We asked them to conduct a deep dive into the platform, and scoped it out so we could learn if researchers could perform functions that they shouldn’t have permissions for,” Lahiri explains. “We found very quickly that we were going to get value from these assessments,” he says. While the Cobalt assessment didn’t locate any urgent vulnerabilities, which is a testament to the internal testing the Egnyte team conducts, they did locate several low and medium vulnerabilities that would require remediation. “I knew at that point no matter what automated tools are available on the market, this is the type of service that we would always need to leverage as we grow,” he says.

With those results in hand, Lahiri sought to apply crowdsourced penetration tests to their mobile development. And as Egnyte started developing more mobile apps, they realized there was a limited number of effective mobile application security testing tools on the market. “We moved mobile testing to Cobalt and crowd-sourced assessments,” he says.

When it comes to software security, Lahiri is reasonably confident in Egnyte’s internal release criteria, which includes quality assurance and regression tests, automated security checks, as well as regular periodic software security assessment scans on their public-facing and production applications. But they’re never going to find everything. With the crowdsourced pen testing, Lahiri says that they have found and fixed things that needed attention. Most would certainly agree that makes the extra effort worth it.