How to leverage crowdtesting for functional and usability tests of intermediate releases

Continuous integration, first named and proposed by Grady Booch in 1991, has been widely adopted across the software industry to prevent integration problems which usually would occur after longer periods of coding. Continuous integration came along with continuous testing and strongly relied on test automation to provide fast and continuous feedback regarding the level of business risk in each build

In today’s digital world, compatibility and usability aspects are playing an extremely important role for the success of mobile and web apps, which is why crowdsourced testing has become a crucial element in the overall quality assurance process for ‘real-world’ validation purposes. As we know, crowdsourced testing helps to ensure compatibility with devices, OS, networks, languages across different geographies, and to validate usability by exposing release candidates to selected user panels. In the early years, crowdtesting was mainly applied during UAT for major releases only, but recently the market has shifted left also within the crowdtesting space, adopting agile on-demand models for getting intermediate releases crowdtested as part of continuous testing activities. And this happened for very good reason, as assigning internal teams to test intermediate release candidates becomes very challenging, when testing has to happen on-demand and around the clock. Also such repetitive testing activities with static test teams can easily lead to tester fatigue  and result in sincere defect glitches. For these reasons, crowdsourced continuous testing with dedicated tester pools, aka ‘continuous crowdtesting’, is nowadays recognised as a more flexible and effective solution.

And this is how continuous crowdtesting works: 

Dedicated pools of suitable test participants are recruited from managed crowds by selecting crowd members based on their domain background, skills, devices, and eventually by their geo location and personas. Pool members get briefed on the apps they shall test, and remain standby for getting engaged in specific test cycles with functional and usability tests for new release candidates. Briefings usually cover topics like new user journeys, technical specifics, application functionalities with higher complexity, test scopes, tools to be applied, and requirements on test reporting. 

Tester pools need to hold 3-5 times the number of participants needed for each test cycle. Thanks to this approach, it can always be ensured that the required number of testers will be available for participation at any time, and that testers can be rotated periodically to avoid fatigue. Very often, customers need their tests to be conducted during the night to ensure having all bugs fixed and qualified by the next day. Having access to a global crowdtesters community with testers from suitable time-zones can be of great help here, and also serve for getting different language versions and locality-specifics tested at the same time.

All of this requires the use of a crowdtesting platform which supports the selection and qualification of the best suitable testers, availability checking of participants for individual test cycles, and providing new builds to participants who signed up for particular tests. Flexible workflow management for issue reporting, reviewing and bugfix qualifications, as well as online analytics for progress monitoring and results checking have also proven to be a crucial requirement for continuous crowdtesting.

The really interesting aspect of continuous crowdtesting is that it allows to validate usability at earlier development stages and more frequently, which avoids bad surprises during user acceptance testing shortly before major releases. 

Ask for our case studies or a free consultation if you want to learn more about the possibilities and benefits of continuous crowdtesting and how to get it implemented in your organisation.

Write new comment

Comments (0)

No comments found!