The goal of this document is to flesh out potential composite rankings for apps on app.co.
Current Ranking(Tweets)
As of June 6th, 2018, our only metric for ranking is the number of tweets that either mention the app’s handle or use the domain of the app. We collect the total number of these tweets over the past 7 days.
Although this gets us some kind of ranking, the overall feedback is that people don’t like this metric. It’s not very well correlated to most people’s subjective judgement of the quality of an app. This metric also lends itself to give higher rank to apps that are more likely to be tweeted. For example, the app‘ForkDelta’ has been in the top 5 consistently. The reason it gets so many tweets is because anyone can list their own coin on ForkDelta, and then they tweet out a link to start trading their coin. It’s almost 100% schilling when you look at the search results.
Near Future(SimilarWeb traffic data)
We are very close to finalizing our contract with SimilarWeb to get API access from them. We get a few key metrics:
Total web traffic
Pages per visit
Time on site
Bounce rate
As soon as we get API access, we’ll start collecting this data. The easiest update is to allow users to change the sorting of apps by a different metric. This wouldn’t be any composite ranking, just allowing users to change the raw field they’re ranking by, such as‘total visits’.
We’ll be able to get this data aggregated by day, week, month, and year. We’ll have to decide which metric we want to use for rankings.
Duration for aggregating ranking data
As stated above, we currently get tweets over a 7-day period, although we update this metric every day. I think it might make more sense to only collect tweets in the last 24 hours. This would let make the rankings change more often, and give people more reason to come back to the site to see how ratings change.
Composite Metrics
Above, I discussed simple metric-based rankings. It’s more interesting, I think, to come up with a composite ranking, so that we have a better‘global’ way of ranking apps.
In the end, it’s probably best to use each one of these metrics, and display different rankings in different places, where appropriate.
This option only uses raw metrics, with no human-based criteria.
First, we’d calculate the percentile for each app by each metric. An app’s metric in the 90th percentile means“out of all apps on app.co, it has more {metric} that 90% of other apps, and worse than 10% of other apps”.
For example, we might find these percentiles for CryptoKitties(I’m making these number up, I haven’t ran the numbers yet):
97% total visits
90% bounce rate
70% tweets
99% pages per visit
Then, we’d make a composite score with some human-decided weight for each metric. For example, we could choose:
0.5 for visits
0.1 for bounce rate
0.2 for tweets
0.2 for pages per visit
Then, our hypothetical CryptoKitties would get a score of(97 * .5 + 90 * .1 + 70 * .2 + 99 * .2 = 91.3). The highest score any app could get is 100.
The benefit of this ranking is that it doesn’t rely on a single metric, and instead uses a bundle of metrics. The downside is that it still only measures popularity, and nothing to do with how decentralized it is.
Current Ranking (Tweets)
Near Future (SimilarWeb traffic data)
Duration for aggregating ranking data
Composite Metrics
Option 1: Purely metric-based composite ranking (aka ‘Hotness / Popularity’)