Passbrains gives enterprise clients the ability to tap into external and internal crowds

Crowdsourcing and software testing are often mentioned as a good fit, and this is for a good reason — a company releasing an app needs to ensure that the app runs glitch-free on an ever-increasing list of devices and operating systems, and that it seamlessly operates across geographies.

For an internal team, testing and validating the app using all the relevant quality variables and key performance indicators (KPIs) are extremely difficult tasks. However, innovations in crowdsourcing offer elegant solutions to these seemingly intractable challenges, as substantiated by Eric Raymond’s point of view on Linus Torvalds’s work on the Linux operating system, “given enough eyeballs, all bugs are shallow.”

It is essential that an enterprise’s apps function well and are bug-free, and the enterprise’s image can be easily damaged when released apps don’t work as expected due to inadequate testing and quality assurance activities. That’s reason enough for enterprises to further strengthen their QA efforts by applying innovative testing methods such as crowdsourced software testing to ensure top-notch quality of their software products. Crowdtesting helps organizations achieve better coverage under real world conditions and collect user feedback early on during the product development process. As a side perk, enterprises also gain access to a global on-demand talent pool that was traditionally unavailable in-house.

PASS Group, a software testing company headquartered in Switzerland, helps global enterprises to achieve superior software quality by engaging scalable numbers of professional software testers from their passbrains community on-demand. passbrains was founded in 2012 to specifically serve enterprise customers getting their applications tested under real world conditions, covering a wide range of devices, operating systems, mobile operators and languages in the desired geographies. As Mayank Mittal, the company’s director, told us recently, “we have been continually expanding the functionality of the passbrains crowdtesting platform, and shaping our crowdtesting processes to cater to the specific requirements of large enterprise customers”.

passbrains is a one-of-its-kind tester community with over 30,000 members, a large proportion of which are professional testers with niche skills and experiences much sought by enterprise clients. Though passbrains is a young business offering compared to PASS Group’s rich, more than 20 years history in traditional software and system quality assurance, the platform continues to attract exceptional talents from over 100 countries, and the company is further refining, shaping and expanding the range of its services portfolio. This is an impressive development, given the due diligence and scrutiny the testers have to endure before they are allowed to participate in passbrains test projects. Hence, the influx of professional testers with specific skills and experiences, and the growth rate of the passbrains community deserve due respect. “Anyone can join the community as a tester, though once users join, they will need to pass our initial assessment through participating in sandbox projects, allowing us to determine their capabilities,” Mittal said.

As users participate in these sandbox projects they earn Career Credit Points; once they reach a certain number of points, they are allowed to start working on real client projects. Testers are then invited for participation in projects based on their profile, test equipment, track record, community standing, and clients’ demographic selection criteria.

In general, there are three ways clients can engage with passbrains. The first is a managed service model, in which a customer works with passbrains to determine the test criteria and implement the crowd solution. “

Customers either ask us to define the testing strategy, or they give us clear requirements: an application that’s to be deployed in specific geographic regions, and needs to be tested on specific kinds of devices or operating systems, browsers, whatever it is,” Mittal explained. “Once we have received the requirements, we analyze them and select the right testers from our community to match our customer’s needs.”

In this model, an in-house project manager is assigned to the project, who continuously manages the test coverage, oversees testers, monitors the progress, and reviews the reported issues before client delivery. The core value propositions of this approach are speed of delivery, single point of contact for test management, professional reporting of key issues, and streamlined delivery. Such a model is the most suited for enterprise clients seeking a turnkey crowdtesting service.

The second is a self-service model, wherein customers use the passbrains community for test execution, but like to own and manage the community and test governance activities through their in-house project managers, who direct and oversee the test management activities. “

In this case we are providing support to our clients’ project managers in any phase of their projects” Mittal said. “Clients can use our experience to refine their crowdtesting process, improve their overall efficiency or solve specific challenges”.

This model suits the needs of enterprise clients that wish to pursue a more active role in project governance. Clients requiring specific know-how and direct communication with testers and eventually tight coupling with internal test teams have seen significant quality improvements of their products through the self service model. The flexibility that stems from this model also perfectly matches the unique needs of startups and smaller clients with transactional quality assurance requirements, organizations with customer-centric products seeking a community consensus and feedback, and companies that seek to quickly ramp up their testing team with an external service provider, but aren’t necessarily seeking to outsource QA activities or insource resources from testing vendors.

The third is a product offering — passbrains Enterprise White Label Platform. Using this model, clients could purchase a PaaS (Platform as a Service) product license or set up a private platform instance within in their own in-house infrastructure or their own cloud. Once set up, the custom-branded crowdtesting platform is available to the organization for onboarding its own employees, testing professionals, partners, and other stakeholders, and also for having on-demand access to the passbrains community, when needed, depending on the requirements of an internal testing project. “

Clients can improve the utilization of their workforce by onboarding their staff on the passbrains platform, and at the same time, if they are in urgent need of additional resources, they can engage passbrains’ ‘public’ testers as freelancers,” Mittal said.

This model is highly sought by clients with privacy concerns and/or excess internal testing capacity. Clients contemplating a shared services software testing business unit without significant organizational restructuring also immensely benefit from this model as the platform has the necessary governance structures in place to facilitate coupling wherever required. Companies that wish to use confidential, proprietary data in software testing, but are reticent to sharing data with a 3rd party also benefit from the platform hosted internally.

Though clients aren’t necessarily required to start with the managed service model before moving to the more independent models, it may be challenging for organizations with little crowdsourcing experience to jump right in and deploy a white label platform to achieve their crowdtesting objectives.

Despite the scalability of crowdtesting initiatives, the Law of Decreasing Returns is also applicable in community-driven models. Beyond a certain inflection point, too big a community could be inefficient and detrimental, as community dynamics could prove counter-productive unless the proper structures, processes and methodologies are implemented. Developing these capabilities is expensive and this experience is best cultivated over several engagements. Companies could alternatively work with a trusted service provider like passbrains in identifying the best practices and architectures that potentially maximize the outcomes. “

We try to diversify the coverage,” Mittal explained. “We make sure that the required locations, functionality and device configurations are covered optimally. Typically we engage 30-50 testers in a test cycle. Larger numbers of testers are only invited in specific project types for example to test the load or stress endurance of an application; otherwise, it will be overkill.”

Not all clients are a good fit for the enterprise solution, Mittal suggested. An ideal client would need to run a significant number of projects, so that the platform is used actively, all stakeholders are comfortable with the tools, and to ensure that community dynamics weather well. The client should also have robust internal QA organization structures and processes in place to act upon the feedback and numerous issue reports delivered by their internal and/or external crowd. “

To ensure that a partnership makes sense, passbrains takes the clients through a rigorous training and pilot program,” Mittal said. Throughout this initialization process, which can take up to three months, passbrains and the client determine whether any customizations to the passbrains platform or processes are required to help the client company reach its goals quickly and effectively. After successful completion of several projects, a client company is ready to go on its own, using passbrains’ advisory only when needed. “

Slowly, we take a back seat so the client can start running things on their own. We still help with planning and strategy, but at a certain point, when they start to feel comfortable, they can begin operating their projects autonomously,” Mittal said. passbrains continues to stay involved with the client by providing valuable platform support and software updates as well as by sharing best practices and process improvements that benefit the customer.

The passbrains enterprise platform encompasses a comprehensive toolkit, which allows end-to-end delivery management of crowdtesting projects. “We continuously invest in developing and integrating new platform features for our customers, testers and project managers”, Mittal boasted.

Among these are: crowd selection filters that allow customers to choose the right testers; management tools for testing tasks and test cases, an issue tracking system which can dynamically exchange data with industry standard tools; a survey management tool; and a social knowledge sharing and networking solution for ideation and problem solving with community members, just to name a few.

The ultimate goal for passbrains is to enable their enterprise clients to build their next-gen, dynamic QA organization that makes use of not only an external crowd, but also leverages internal available skills and resources and taps into the hidden wisdom of their own employees. “That’s a lofty goal, but one that passbrains is already delivering on,” Mittal said. “We are seeing strong efficiency and effectiveness improvements after implementation of our process model. Enterprise crowdtesting leads to paradigm changes in our customers’ organizations.”

This article was originally published on

Write new comment

Comments (0)

No comments found!