Nobody owns the Crowd. And did you ever think about Crowd Activity Metrics?

This is part 2 of our series on Crowd Metrics & Crowd Quality

Crowdsourcing has the potential to disrupt traditional work models in many different sectors, yet Crowd is a very liquid, volatile matter. People join crowdsourcing platforms for many different reasons and, if they start to take up work assignments they get offered, the degree of their stickiness is hardly predictable. All crowdsourcing platforms see people come and go on a daily basis and in most cases, they are subscribed to multiple different and often competitive platforms. So it is obvious that a crowd can never be owned by a vendor. 

In the first post of this series I explained why Crowd Size is a vanity metric. But what are the really meaningful and actionable metrics you should look for? Here's about the first pretty important metric:


Crowd Activity

A crowd lives and dies with utilisation. Crowd members who did not receive any project offer within the first 3-6 months after signup, will most probably lose interest and simply stall their membership, mostly even without unsubscribing. For this reason each crowd holds a membership graveyard of unknown size, except if the crowdsourcing vendor would run some life check procedures every now and then to be able to remove dead member accounts.

As mentioned in our previous post, you may assume that only a small percentage of crowd members are actively delivering work on a platform. Thumb rules such as the 1-9-90 rule or the 1% rule provide some general idea, but of course the level of activity will vary significantly when you look at different segments of a crowd, because crowd utilisation is driven by demand and demand is never evenly distributed across geographies, demographics, skills etc.  

The capacity of individual crowd members which they can provide for any kind of crowd working is a major factor which determines the overall Crowd Activity. Asking crowd members to specify their average capacity (e.g. in average hours per day, week or month) will definitely be useful but not really tell much about their availability at a given time and duration for a project assignment or even just a small task, because there can be various reasons for not accepting project offers, such as lack of interest in a specific project or personal reasons such as travel, illness etc. This is a factor crowdsourcing vendors can hardly estimate nor influence but they can continuously measure Crowd Activity metrics to develop some understanding of average acceptance rates for project calls or offers they can anticipate.

Interesting metrics about Crowd Activity can be derived from analytics such as the average number of project participations per member over a given time period (e.g. during the last 12 months) and the average number of accepted vs. rejected project participations per member over a given time period. Of course these metrics should be drilled down to filter the part of the crowd which is of specific interest (such as e.g. certain geographies) and it could also be interesting to assess the trend of these metrics over the last few years to see how the vendor’s crowd develops over time.

Understanding all of this, you might agree that Crowd Activity can be very different when you compare crowdsourcing vendors’s crowds and therefore is a critical metric to investigate instead of Crowd Size only. Crowd Activity can be assessed by reviewing vendors’ reports on such metrics (if they accept to provide such information) or by running a pilot project to capture such metrics right on the spot and based on your specific crowd requirements. 

In the next post of this series on Crowd Metrics and Crowd Quality I will talk about Crowd Performance as another important Metric, so stay tuned and please do not hesitate to share your comments or questions!


Authored by:

Dieter Speidel, CEO


Write new comment

Comments (0)

No comments found!