It's a lagging indicator - based on past manufacturing, past purchases, past support practices. This is unescapable, but when I buy cars (and stocks) the past is not a reliable indicator of future performance.

It's flawed/biased - the rankings are based on "member" surveys, which makes the sample biased (likely not representative of all car buyers, CR has never attempted to prove it is as far as I know), which is a flaw if you're using it to make (large financial) decisions. Using surveys rather than evidence is perhaps a more fundamental problem. Essentially this "ranking" is of the perception of quality among a subset of people rather than a serious, quantitative ranking.