Alexa Champions is “a recognition program designed to honor the most engaged developers and contributors in the community”. In a previous post, we revealed that there were some very well rated posted hidden several hundred pages deep in the Alexa marketplace. Eric Olson, co-founder of 3PO-Labs and Alexa Champion, pointed out that all 3 of these skills were by Alexa Champions. It seems odd for skills from champion developers to be rendered essentially undiscoverable by the very company that features them. This raises the question of whether Alexa Champions have any objective advantage over regular developers. I took a deep dive in the performance of these developers. Today, we will look at Alexa Champion skills ratings.
Below are results for 13 Alexa Champions. Our analysis did not include Invoked Apps because their wildly successful skills make them an extreme outlier. We also excluded VoiceXP because their 36 skills makes it hard to match the developer with an industry average (there is only one other developer with 36 skills). We also excluded 9 developers for only having 1 or 2 skills. We could not find skills on the US marketplace for the other Alexa Champions.
In the graphs, you will see the name of the Alexa Champion developer next to “n=# average”. This represents the average of all developers with “#” number of skills (same number that the Alexa Champion published). The thin black bars represent the variability of the average. If the bar is small, that means there was less variability and most data points were close to the average. A larger bar means that the data points were far from the average. This variability would be more or less than the average.
Alexa Champions are people who are deeply engaged with voice first development. We would expect their skills to be generally good and well reviewed. Here, we see that Alexa Champions have higher average skill ratings. In other words, their skills are better liked than the average. There is a lot of variability though, meaning that while on average, their skills are more highly rated, not all of their skills are above average. There are exceptions to this though, Opearlo has consistently made good skills, which probably explains why they were acquired. I was a bit strict with my analysis and calculated a value of “0” for skills that have no ratings. This is why Pretzel Labs, whose skills are all very well reviewed, has such a low average skill rating.
In conclusion, Alexa Champions make great skills! On average, their skills receive better ratings than regular skill developers.
3 thoughts on “Deep dive into Alexa Champions: Ratings (November 2019)”