back

by jader201·12y ago·view on hn ↗
Get rid of ratings altogether. Period.

I want to see usage statistics:

- How many users are continually coming back to the app?

- How many times a day/week/month, on average, is the app opened?

- How long is the average "session"?

- How many days is the app used after it's initial use?

- How long does an app stay on the device before it's deleted?

These stats are more objective and less manipulable -- and therefore, more valuable -- than user reviews/ratings.

And I realize that the value of these will vary app to app -- i.e. the stats of a productivity app can't be compared to the stats of a casual game -- but it gives me a much better idea of how an app is being used, and therefore how I might use it vs. reviews/ratings.

7 comments
This doesn't account for apps that are unusually efficient. My bus app allows shortcuts on the home screen to arrival times for a stop, so each day I spend under a minute with it open, yet it's a better app than a clumsy interface that would waste two minutes.
I think this would still address that (and I mention this in another response below).

As someone deciding between the two, I will see that you keep coming back to the app, even if you don't use it for a very long time.

Versus the other app that you averaged a longer session, but stopped using after a shorter period.

We'd have to be careful of apps that game the system by installing some super-small/innocuous service that would hit Google Play once/day just to keep your usage statistics falsely high -- even when you don't open the app yourself.

I don't know enough about the guts of Android to be sure you could do this, but it seems likely.

Ideally, usage would be handled by the OS, and it could actually distinguish between running in the background vs. being active. Background usage would be discarded, ideally.
Not necessarily. What if your app comes with a widget (or tile for Windows Phone apps) that tells you all the information you need. I'd have that visible at all times and would never need to open the app itself.
I have an iPhone, so I'm not sure how they're generally used.

But I would assume that if you have a widget docked, then you are likely getting some use of it, and if not, you will undock it (and maybe also delete it).

So there is still a level of usage you could measure from widgets, though maybe not quite as meaningful as that you get from regular apps.

There are some apps that I almost never open, because I get _all_ of the utility of the app from the push notifications. Two examples are:

- Pushover

- CN Air Quality

The stats can include how many average push notifications they send over a time. If they send a lot, but you keep the app installed for a long time then you like the push notifications and that would show in the stats.
I think that's something that could certainly be accommodated by an algorithm developed to give an aggregate 'score' for apps. Certainly the fact that the app is used every single day, regardless of how long, would be a very big plus for its rating.
Ignore session length and look at repeat usage / lack of uninstall?
On Android, widgets would be an issue.
See jader201's comment: https://news.ycombinator.com/item?id=7192705

That said there is no easy answer for widgets.

Uber would do poorly by this metric because I rarely need to take a black car anywhere, so even though the app is great I rarely open it.

The app that shows me when my bus is coming is pretty crappy, but I use it multiple times a day.

The deleting part is interesting, but I'm not sure how many people bother deleting apps they don't use.

OP's idea is a good one.

People who use lots of apps will delete apps they do not use. That stat would also confound "rent-a-user" schemes to pump up usage stats.

> Uber would do poorly by this metric because I rarely need to take a black car anywhere, so even though the app is great I rarely open it.

Would prospective purchasers realize this, though, by the nature of the app? In other words, would I also realize that I would rarely use it (on a daily basis), but see that you do continue to come back to the app and haven't deleted it over a long period of time?

> The app that shows me when my bus is coming is pretty crappy, but I use it multiple times a day.

If you use it multiple times a day, then you apparently are finding it useful enough to continue coming back to it. And in my mind, that's all I care about.

I know this isn't perfect, but again compared to user reviews, this would be way better in a large majority of the cases.

Especially in a genre where it's getting abused, like games.

> Would prospective purchasers realize this, though, by the nature of the app?

So what you're saying is, we should burden the users with a ton of metrics, then let them figure out which of those metrics are useful for each particular app, and whether you want a higher or lower score on each of those metrics on a case-by-case basis?

There are many ways usage stats can be used to provide more value than ratings. Provide a ton of metrics to those that like them and know how to interpret them, or aggregate them somehow for those that want an "X out of 10" ranking.

Or anywhere in between.

Well, what do the stars actually mean? Is it a measure of whether or not the app is useful? Or some subjective measure of how "good" it is?

I shudder when I think of the ways you could game a system like this. If longer time-in-app is a positive signal, I could easily make my app slower.

> Well, what do the stars actually mean? Is it a measure of whether or not the app is useful? Or some subjective measure of how "good" it is?

Right now, they're pretty meaningless, because often, the majority those that take time to review them are counterfeit. 90% of the people that use an app never take time to review it. So I never know what most people think.

> I shudder when I think of the ways you could game a system like this.

As it stands, I can't think of how there would be more gameability to a system like this vs. ratings. Ratings are so easy to game, anybody can do it, and it doesn't cost them anything.

To game something like this, I would have to "game" it over long periods of time. Not to mention, I would have to purchase a separate device, or sacrifice my own time with my device.

It's a much more painful process to game something like this.

> If longer time-in-app is a positive signal, I could easily make my app slower.

If you make your app slower, you are sacrificing user experience, which will eventually hurt your usage. How could you game your own app without it hurting the usability of it?

It works for porn, sort by "views" seems to pretty consistently be better content than "rating".... i think, i heard once from someone...
Unfortunately, getting a lot of those stats requires the the app logs everyone's activity. I guess this is fine for a game client that needs to connect to a server to work anyway, but for an app whose main functions don't require internet access, most people probably wouldn't want to share the above stats.
Then I'll just buy 100 $30 android devices and leave my apps open around the clock. Hell maybe I'll buy 1000 of them and rent out time on them to unscrupulous app developers.
Maybe so, but this would cost you a lot more vs. purchasing reviews, and would probably still be outweighed by the usage statistics of "real" users.

If going by usage statistics, every user -- and more importantly, real users -- opt in to "rating" the app. Now, you're depending on people actually taking time and being willing to review the app, which is mostly only done by false users.

Add geo-diversity to the algorithm; now you have to physically locate your devices across a wider area, which becomes really awkward. True, the notion of 'hire for stats' is still a problematic one.
Can't you spoof using an emulator also?
on several VMs
Active installs (as in people who have the app installed on device that is being used) would cover most of these while still being relevant across different app types.

Of course some people leave their apps installed even if they don't use them, but my guess would be it's not that many (any data on this?).

> Of course some people leave their apps installed even if they don't use them, but my guess would be it's not that many (any data on this?).

True, but this should still average out in the end. That is, those that leave bad apps installed are also leaving good apps installed. But those that like to keep their devices tidy will give real meaning behind the numbers.

In other words, don't look at the numbers absolutely, but relatively.

The idea is good, however i think it could be abused quite easily. I.e. spoof a couple thousand Android devices and send fake statistics.
Two reasons I don't think is true:

First, how much harder is it to spoof actual devices, and take the time to spoof usage on these apps, vs. what it takes to spoof a single review?

Second, add to that the fact that spoofing 2000 app usages compared to the several thousand (or hundred thousand?) actual real users where real usage is being collected from. If you spoof 2000 reviews -- which are super easy to do, can be done in a matter of minutes with no long-term overhead -- you are aggregated with a much smaller number of real reviews actually being submitted.