Item — HN Reader

> Subsampling is often done because you can get the level of prediction desired at a much lower cost, both in terms of hardware and employee hours.

While that's valid, I think the author's concern is that authors frequently do not demonstrate that the subsampling is representative. So the conclusions from the sampled data my not be as accurate as is claimed.

(edit: minor phrasing change)

2 comments

If only there was an entire field that exists to characterize when subsampling a population is valid and sound.

If only my non-statistical peers would recognize that sampled=fast and fast=more checks and explorations. It's like they recognize that fast compile times are a great thing (I've had to wait in line with punchcards, and it sucks), but they are completely oblivious to the exact same argument when it comes to data.

Why spend years learning those results when you can ignore them at no cost?

And have a happy client that finally gets the results he wanted when the other analysts said it was impossible.

Elsewhere he talks about how subsamples are chosen because they take but seconds to run instead of minutes. If a sample is small enough that you're saving that much time, and analysis is that cheap, you can still do five-fold or ten-fold cross validation in less time than the full data set analysis and get a very good idea on if your subsample is representative of the data or not.

That's what I took from it. Forbes is aimed at the people buying solutions so from the perspective of a CIO or CTO (or even the people they're supporting with their analysis systems), the article is telling them that they may not be getting what they think they paid for.

This is about industry trends, and you CAN get a representative sample in reasonable time, for some definition of reasonable. The takeaway from this article is that what someone in the data space may think is reasonable isn't what someone who just paid for an army of data scientists and a data lake solution because those things are sexy thinks is reasonable.