▲ 149 points
back
102 comments
Many years ago, I had an argument kind of like this with my mom about the difference between decaf Diet Coke and plain Diet Coke. Since she was the Mathemetician in the family, I let her design the double-blind taste test.
When I could conclusively tell the difference every time, she was flabbergasted. But she did finally believe me that there actually was a taste difference between them, at least there was for me. So, she stopped trying to get me to drink decaf.
I performed a reasonably rigorous experiment for my AP statistics class in high school - I tested a ton of different sodas and their diet equivalents across my classmates, and of the 8 sodas tested almost all had a statistically significant bias in favor of the non-diet version. For coca cola, I also tested the non-caffeinated versions - the non caffeinated regular Coke was the top performer.
The only exception to this was diet SunKist - it handily beat regular SunKist, by a large margin.
This shouldn't be too surprising, as caffeine itself has a flavor. It is somewhat bitter, which is one reason why it's particularly common in root beer.
(pedantic mode on)
That reminds me of my mom, always calling it decaf. There is a difference between decaffinated and caffeine free or non-caffeinated. Coke can be non-caffeinated, meaning that it can be made without adding caffeine. Decaffinated implies that there was caffeine present at some time, but was taken out (like coffee).
That's only half the test. The other half is to have them decide which is which.
The human taste is incredibly good at discerning two different flavors. Not so good at identifying them.
Decaf diet coke is the nost useless thing anyway. You get neither coffein, it is just water with some taste.
That people exist who drink decaf diet sodas boggles the mind.
Why not drink carbonated water at that point? Or just water, you know.
Or drink real soda in moderation.
Maybe I didn't do a good job scanning the comments, but I'm surprised nobody (not even the article) mentions the NSF certification body.
Taste is important but very subjective. I don't buy a water filter unless it shows up as certified by NSF[1].
A few years ago I almost fell for a "water purification system" on Amazon, with stellar reviews that was supposed to filter out everything. With a baby on the way I figured it was worth the price tag. Good thing I did my research. The water filtration system was not certified or verified by anybody.
edit:
Same thing applies to humidifiers, air cleaners, air conditioners, etc. Don't buy unless it has been certified[2].
Well, maybe it's a bit more complicated than that.
It's certainly true that if you're using a water filter to remove something dangerous rather than remove "hardness" or "chlorination", then you're going to want a certification (and regularly test to see if the RO filter is still working). I'd put ship, cistern, and catchment on that list of critical filters.
However, if your decision is primarily based on flavor (as the article's is) or mineral build-up, then it's not clear that it's so important since you're drinking the water anyway!
I'd note that one brand you may be referring to (although it's true for others) has both "residential" and "professional" series. The latter are WQA Certified (NSF/ANSI 58 & 53)... although you won't find them listed on the NSF website, they are both ANSI and SCC (Canada) certified.
http://yourwaterfilterguide.com/difference-apec-water-reside...
Thanks!!! Safety above taste seems obvious for a water filter.
Or you could get a conductometer and measure the purity yourself.
What is the equivalent certification called for air cleaners?
Apparently the author is married to a water filter, but at least she's stylish.
> This experiment was inspired by an argument with my wife, a stylish but atrocious water filter, and the explosion of start-ups attempting to turn everything you purchase into a subscription service.
It's got the Oxford comma. What more do you want?
That would explain this:
> How I Used R To Win An Argument With My Wife
Aren't we all just complicated water filters given a particular perspective?
If only English had precedence rules!
This; right here; is why I replace all of my punctuation with only semicolons;
Hahaha. But I think the author is just listing the things that inspired the argument.
Very nice setup, but he didn't show he couldn’t tell the difference between tap and the Soma water.
A biased subject who can pick out both untreated water and Soma-filtered water from the other variants could lie about that, and claim that untreated water and Soma water taste the same.
As a thought experiment, assume the variants are plain water, orange juice, and beer. A biased subject could easily claim orange juice and water taste the same and look the same, and are easily distinguished from beer.
Also, but minor: this is about taste. Such experiments typically are done in the dark, to rule out an effect of how the water looks on taste judgments.
I agree about the risk of bias. If he cheated, even subconsiously, on the tap vs Soma test, the results are consistent with bottled and Soma being indistinguishable. Which is the opposite conclusion.
The suspicious inability to distinguish bottled water from Soma agrees with this. He wouldn't be able to cheat on that test if he actually couldn't tell the difference, but could cheat on tap vs Soma like you said about water and orange juice.
If he didn't even know which two types he was comparing, that would protect from that kind of bias. However, from the setup, it looks like he does know.
If you think he's lying he could have made up the entire blog post too... :P
But more seriously, how would you suggest designing an experiment that worked even with liars determined to thwart it?
Why was Soma so hard to distinguish from Bottled? Article seems to gloss over this point?
This bothered me a lot too.
If he can tell the difference between 'tap v bottled' but not 'tap vs soma'... then 'soma vs bottled' would intuitively have a different result if his conclusion is 100% correct.
I suspect there's more to it, but the author was focusing on winning an argument so... I guess that killed the curiosity required to dig further for a more thorough conclusion.
It wasn't and I think that's the point. Essentially soma was hard to distinguish from tap water, which OP had said tasted chlorinated, given that soma was supposed to filter and improve tap water it was failing to meet its stated objective.
I recently did a similar blind taste-test for mineral waters vs my tap water: https://www.gwern.net/Tea#water-experiment
I used a lazy susan to implement randomization/blinding, and an interesting Bayesian variant on best-arm finding to try to optimize sample selection. I found my tap water is weakly in the middle and perhaps 1 or 2 mineral waters were better, but not strongly so.
The results are not quite comparable to OP because I designed it in terms of deciding which in a pair tasted better, not whether I could guess which was which correctly.
Eh, I get the author is being sarcastic, but for someone who buys water filters for their looks, his Soma jug looks just like a slightly plump version of Brita. (What's wrong with Brita, anyway?)
in stereotypical gender role parlance, it does not pass the "wife test".
also filters are overly expensive and useless. issues that the fancy filter only aggravates, so...
It may be worth noting that if your water is only chlorinated with chlorine (and not chloramine, which is quite common now as well), leaving it in the fridge overnight can allow some of the chlorine to evaporate, making your water taste better even without a filter.
In this case he allowed both his tap water and filtered water to cool overnight, which would tend to understate the taste difference between freshly pulled tap water and freshly filtered water (if that filter effectively removes chlorine).
Fun little article, but I find the overall premise a bit weird. I always thought of water filters as something that you use to remove harmful things in the water, not improve the taste. Filtered water may taste different, possibly even better, but that's not the goal when I use one.
I have lived in cities where the tap water, to me, was undrinkably gross tasting. It was probably just fine to drink, but I found myself drinking less water than I should because of the taste. A Brita filter helped a lot.
Edit: and not all cities I've lived in are that way. In my current city the tap water is just fine.
My parents have used a Brita filter for years.
For them it started out because they were fed up of descaling the kettle, as they live in a very hard water area (lots of dissolved minerals) and the kettle would scale up easily. But they also always use the filtered water for drinking, because they prefer the taste now.
My sister doesn't, she drinks it straight from the tap when she's visiting.
But it's not at all about water safety. It's just about flavour and the longevity of the kettle (and the iron).
Not in the US. The vast majority of US tap water is perfectly safe to drink. The only reason to filter it is taste.
It's definitely my goal -- I assume my water's perfectly healthy, but removing the chlorine makes a world of difference in terms of taste.
But I live in a city. If you're in the country and drink well water, your priorities might be the complete opposite.
The part I find interesting is the 100% success rate between bottled and tap, which is something bottled water drinkers insist is true (being able to taste the difference, that is), but are often told is just in their heads.
I have a Soma water filter... their filters used to work well and quickly, then they "improved" them and it takes 10 minutes to fill the pitcher. Yeah.. i'm probably gonna toss the top soon and just enjoy the glass pitcher
"Initially I thought just to buy a Brita, but Brita filters always seemed like something you’d shove in your dorm room mini-fridge and not display on your kitchen counter."
This is a pretty abrupt context switch after coming from the article on stereotypes, trailer homes and rural folk that's on the front page right now.
> this point I just decided to throw the pitcher into the fridge (cold water masks poor taste) and use it for a few weeks to see if there were any changes.
if chlorine was the actual cause of the bad taste, wouldn't the water "aired out" the chlorine after a hour or so?
> First, I filled all the pitchers the previous night
Chlorine evaporates. If he's worried about chlorine taste this step - allowing the jug to sit overnight - will fix it.
I have a Soma pitcher and PUR pitcher at home (I drink enough water to need both). They taste about the same to me.
When does Soma sue this guy? This seems like a risky article to me. Whether or not they have a case, doesn't matter. They can make life miserable.
You aren't going to be married long.
This was an All in the Family episode. For whatever reason, Gloria can;t tell the difference in taste between Coke, Pepsi, and RC but Meathead can.
"HOW I USED R TO WIN AN ARGUMENT WITH MY WIFE"
I prefer a Berkey myself.
"...a practice pioneered in the 90s by “8 CDs For A Penny!” Columbia House"
LOL. Try 1950s. I love how Millennials think everything was "pioneered" in the 90s and 2000s.
I wonder how much efficiency and productivity hours are lost to stupid preconceptions like "Brita isnt something you put on a kitchen counter".
Soma sent out a really self-congratulatory marketing email on Black Friday a few years back, telling people, "Don't buy a Soma today." gushing about living in the moment and enjoying time with friends and family instead of shopping.
I wrote back and asked if they were so serious about that message, why didn't they take the Buy button off their website?
Shockingly, no response.