Success criteria for the Wikimedia Space prototype

@jcrben The value of the success criteria (apart from defining success criteria) :wink: is to identify the areas of social and technical development that we want to prioritize. For instance, we are setting a high bar in terms of user engagement and multilingualism, at levels that (say) wikimedia-l doesn’t even attempt to reach (or that would be counterproductive if it reached; imagine a single mailing list with “40 daily engaged users”… “discussing in languages other than English (at least 2 non-Latin scripts, one of them right-to-left)”. And this is because we are not attempting to replicate wikimedia-l (or any other existing Wikimedia channel). We are attempting to create and consolidate the Space that the Wikimedia movement needs for its continuous growth.

These success criteria show that this task is not trivial, and that we need a lot of people involved to achieve them. A team of Foundation staff plus some contractor budget will not cut it alone. Hundreds of volunteers participating regularly at different levels are needed to consider this initiative a success and move to the next stage.

And yes @jcrben, of course you can help! :slight_smile: As an event organizer, posting your events here and promoting Space at these events is a priceless contribution that nobody is better qualified than you to make.

PS: I have copied your request for a new feature to Space Calendar: limit events by geography? in order to keep this topic here focused on success criteria. Thank you very much for submitting your ideas to improve this Space!

2 Likes

One criterium I miss, and that seems to touch on a main weakness of this system, is a minimum amount of true engagement (at least X posts per month) from non-WMF volunteer community. From what I spot in the summaries I get (although those may be biased?), engagement is mostly among WMF staff and a small number of very active volunteers.

1 Like

Good point. Let’s define success criteria.

In terms of what can be measured reasonably, I have started here:

Thoughts?

We already have several success criteria about number of active participants, so I guess the sauce here is behind the percentage of non-WMF / WMF participants. What ratio should define success?

One additional thought is that while we all want to have Space participation clearly driven by volunteers, we don’t want to hamper the participation and engagement of Foundation teams either. In the past days I have heard from a couple of Foundation colleagues that they are posting less or stopping to post altogether because Foundation staff is already “too predominant”. While this is understandable (and we keep working on the promotion of Space among volunteers) we need to be careful defining success criteria that propitiate undesired anti-patterns.

2 Likes

See my comment on a similar proposal: Success criteria for the Wikimedia Space prototype
Success criteria which become less likely to be met when someone participates more are unhealthy.

1 Like

Also WMF in the username is not really reliable, for example this is technically my staff account. (Not that differentiating between staff and volunteer account is particularly meaningful for me as my staff role has nothing to do with Space.)

1 Like

This is true. Not everyone who signed up with a Foundation email has “wmf” in their username (although the vast majority do). But the inverse is also true—users from Wikimedia France, for example, have WMFr in their usernames which were probably counted here and shouldn’t have been. Once usernames are enforced on the 28th, this will be somewhat clearer, and we can get more precise about these metrics.

(I never received a ping of the responses - so only noticed this when i signed in again just now)
I agree with the nuances made about how to measure WMF exactly (given the number of people, we can even do this manually currently - but if successful it’d be nice to scale up beyond that some day). I’m a little hesitant to ask for a minimum percentage of non-WMF, because that could discourage WMF engagement, which is not what I’d want. But whatever numbers you come up with for total engagement, maybe add an additional constraint at 80% of that number for non-WMF engagement? For example, if your goal is to have 200 participants post at least 5 times, the sister-goal would be 160 non-WMF participants post at least 5 times. (I’m grabbing the 80% out of a hat, was more thinking about 90% for a mature platform)
I can imagine the concerns with WMF dominance too, though: When I see mostly WMF posts, that definitely does not encourage me. Not because I dislike WMF staff members, but because the conversation is on unequal footing the same way I get discouraged by some volunteers on wikimedia-l over the years. But that should not necessarily be included in the success criteria as a platform, but rather be a conversation about what is desirable as a healthy community.

2 Likes

I think we can agree that the four data points listed above are useful. We can review the numbers manually to avoid flagrant mistakes. We can accept that some grey area exists and expect it to be small enough not to affect the numbers significantly.

About the success criteria for this prototype phase, two possible references are the Wikimedia Summit 2019 and Wikimania 2019, key movement events that are getting high satisfaction rates from their participants. For this current phase, early adopters and movement organizers are the primary targets, and these audiences are likely to be part of these events in a similar proportion. I guess we can get the numbers of Foundation - non-Foundation participants in both events, and define criteria accordingly.

Once we move from prototype to production, the ratio of non-Foundation staff is likely to grow with the increase of participation beyond early adopters, and with the corresponding arrival of other topics/areas where the Foundation’s involvement is little to none.

2 Likes

The Wikimedia Summit 2019 had about 190 non-Foundation participants. Wikimania 2019 had about 770.

Based on this, I suggest the following changes in the success criteria:

  • 770 registered users (excluding Foundation staff).
  • 190 users liking or posting monthly (excluding Foundation staff).
  • 40 users posting at least 5 times in the past month (excluding Foundation staff).

40 is roughly 20% of 190. 20% is the proportion between daily visits and daily engaged users in the current success criteria.

Users with “WMF” in their username are counted as Foundation staff.