▲ 255 points
back
122 comments
The City of Chicago's lawyers went the opposite direction in response to @tpacek's affidavit that the release of table/column names would have "marginal value" to an attacker. The city latched onto that to get a trial that eventually went to the IL Supreme Court and lost.
[I]n my affidavit, I wrote that SQL schemas would provide “only marginal value” to an attacker. Big mistake. Chicago jumped on those words and said “see, you yourself agree that a schema is of some value to an attacker.” Of course, I don’t really believe that; “only marginal value” is just self-important message-board hedging. I also claimed on the stand that “only an incompetently built application” could be attacked with nothing but it’s schema. Even I don’t know what I meant by that.
>The City of Chicago's lawyers went the opposite direction
Not really.
>I wrote that SQL schemas would provide “only marginal value” to an attacker. Big mistake. Chicago jumped on those words and said “see, you yourself agree that a schema is of some value to an attacker.”
The City of Chicago's argument was that something of ANY value, no matter how insignificant, would help an attacker exploit their system, and was therefore possible to keep secret under the FOIA law.
It’s not just the UK who has standardized on these language; the U.S. intelligence community also has a list of required terminology to use for different confidence levels and different likelihoods — and distinguishing between them. It’s all laid out in ICD-203, publicly available at https://www.dni.gov/files/documents/ICD/ICD-203.pdf
I’ve found it very helpful in the same vein as RFC 2119 terminology (MUST, SHOULD, MAY, etc.); when you need your meanings to be understood by a counterparty and can agree on a common language to use.
Interesting. This terminology really makes no sense without more shared context, in my view. For example, I would not describe something that happens to me every month as a "remote possibility". Yet for a 3% chance event, repeated every day, monthly occurrences are what we expect. Similarly, someone who describes events as "nearly certain" would surely be embarrassed when one of the first 20 fails to happen, no?
I was so frustrated when I tried to get doctors to quantify their assessment of risk for a surgery my sister was about to undergo. They simply wouldn't give me a number, not even "better or worse than even odds". Finally an anesthesiologist privately told me she thought my sister had maybe a one-third chance of dying on the table and that was enough for me. I'm not sure how much fear of liability had to do with this reluctance, or if it was just a general aversion to discussing risk in quantitative terms (which isn't that hard, gamblers do it all the time!).
Doctor here
1. It’s generally difficult to quantify such risks in any meaningful manner
2. Provision of any number adds liability, and puts you in a damned-if-does, damned-if-it-doesn’t-work-out situation
3. The operating surgeon is not the best to quantify these risks - the surgeon owns the operation, and the anaesthesiologist owns the patient / theatre
4. Gamblers quantify risk because they make money from accurate assessment of risk. Doctors are in no way incentivised to do so
5. The returned chance of 1/3 probably had an error margin of +/-33% itself
I think a lot of people don't understand statistics, which may make it hard for doctors to choose how to communicate things, even if they do have important knowledge that could be helpful.
I once asked a doctor how long a relative might have to stay in intensive care:
A: Oh, I couldn't possibly say.
Q: Do you think he might be home in 3 or 4 days?
A: Oh, no, not that soon.
Q: So it might even be as long at 3 weeks?
A: I highly doubt it would be that long.
Q: So a reasonable estimate might be 1-2 weeks?
A: Oh, I couldn't possibly say.
I started the conversation having no idea whatsoever how long it would be, but I ended up with a good feel for a time estimate along with error bars.
> she thought my sister had maybe a one-third chance of dying on the table and that was enough for me
But what was the alternative? I understand that you didn't get an answer, but the alternative of not operating could have been worst
Gamblers are a poor example. Their decisions hardly effect anyone else or institutions or nations.
Increase the cost of the fallout of a decision (your relationships, your bosses job, your orgs existence, economy, national security etc etc) and the real fun starts.
People no matter what they say about other people's risk avoidance, all start behaving the same way as the cost increases.
This is why we end up with Trump like characters up the hierarchy, every where you look, cause no one capable of appreciating the odds, wants to be sitting in those chairs and being held responsible for all kinds of things outside their control.
Its also the reason why we get elaborate Signalling (costumes/rituals/pageantry/ribbons and medals/imposing buildings/PR/Marketing etc) to shift focus away from quantifying anything. See Theory of the Leisure Class. Society hasn't found better ways to keep Groups together while handling complexity the group is incapable of handling. Even small groups will unravel if there is too much focus on low odds of a solution.
> Since then, some governments have tried to clean up the language of probability. After the Iraq War—which was influenced by misinterpreted intelligence
While I laud the gracious application of Hanlon's Razor here, I also think that, for at least some actors, the imprecision was the feature they needed, rather than the bug they mistakenly implemented.
Anyone else find the standard "probability yardstick" very misleading on the "unlikely" side? I know the whole point of the article is that English speakers can interpret these phrases differently, but calling a 1-in-3 chance "unlikely" seems off. I would shift that whole side down—30% as "a possibility", 10% as "unlikely", 5% as "highly unlikely".
You're right, whatever they pick will be wrong, but that's "missing the forest for the tree"
The goal is to remove uncertainty in the language when documenting/discussing situations for the state.
It doesn't matter that it's wrong colloquially or "feels wrong". It's that when you're reading or talking about a subject with the government, you need to use a specific definition (and thusly change your mental model because everyone is doing as such) so that no one gets misunderstood.
Would it be better to always use raw numbers? Honestly I don't know.
That was a good read (and short, with a cool graph—I want to know who tagged "Almost No Chance" as 95% likely; a would-be Pratchett fan, perhaps). In biz, that's part of why I like to separate out goals ("we'll focus on growing traffic") and concrete objectives ("25% audience growth between now and June 1st").
My feeling is it's a measure of the number of people who read the question wrong.
But is it EXACTLY a million to one chance?
I've had the same sort of difficulty with phrases like "most" or "almost all" or "hardly any"—I crave for these to map to unambiguous numbers like the probability yardstick referenced in this article.
I spun up a quick survey[1] that I sent out to friends and family to try to get some numbers on these sorts of phrases. Results so far are inconclusive.
"Almost all" is an interesting one, because it has family of mathematical definitions in addition to any informal definitions. If X is a set, "almost all elements of X" means "all elements of X except those in a negligible subset of X", where "negligible" depends on context but is well-defined.
If there's a finite subset of an infinite set, almost all members of the infinite set are not in the finite set. E.g. Almost all integers are not 5: the set of integers equal to five is finite and the set of integers not equal to five is countably infinite.
Likewise for two infinite sets of different size: Almost all real numbers are not integers.
Etc.
The more precisely they are defined, the less frequently will you see them used correctly.
"Almost all" in math can mean "except at every integer or fraction" :)
"Rare" versus "common" is an interesting one. They sound like antonyms, but I don't think the typical probabilities are really symmetrical. Maybe something like 0%-10% for rare (although some sources say 5%) and something like 40%-100% for common.
"common" has such a large spread because meaning behind it is sort of "at least one in each sample", where that sample can be anything (graspable)
if you're a teacher and one student per class does the same thing - it's common. Even though it's only 1/25 or 1/30 of all students
Maybe it's my amount of video games played in childhood that influenced that, but common and rare are just two points on a spectrum (with at least "uncommon" in between)
Text of NIE 29-51 "Probability of an Invasion of Yugoslavia in 1951"
Partial HTML: https://history.state.gov/historicaldocuments/frus1951v04p2/...
Full text PDF scan: https://www.cia.gov/readingroom/docs/CIA-RDP79R01012A0007000...
Who are the people that have a small bump of believing "better than even" is 10-20%? Why?
You also see the opposite bump for most of the negative assessments. My assumption is that they're likely reading the question backwards. ie. "how unlikely" vs "how likely" or similar.
Why not just actually list the number you have in mind so everyone's on the same page "we consider it a serious possibility - about 60% - that bla bla bla"
Almost no one making these statements has an actual number in mind, or they would just say it. Probably not even in intelligence, definitely not in popular usage.
Because that's not how the mind works. We don't have conscious access to our internal credence of some event in probabilities (unclear if we even evaluate probabilities internally at all).
Because then it doesn't happen and (dumb) people will say "see you were wrong".
It's strange to map language to probability ranges. The guidance should be to just say the probability range. No ambiguity. Clear. Actionable and also measurable.
It's still a subjective estimate, but Samosvety (predictor group) does seem to work that way, and HPMOR suggested something similar. Basically assign probabiltiies to less complex unknowns using numbers pulled out of your butt if that's all you can do. Then you can compute conditional probabilities of various more complicated events using those priors. At least then, a consistent set of numbers has carried through the calculation, even if those numbers were wrong at the outset. It's suppose to help your mental clarity. I guess you can also perturb the initial numbers to guess something like a hyperdistribution at the other end.
I haven't tried this myself and haven't run across a situation to apply it to lately, but I thought it was interesting.
That's the other way around -- there was no probability range to begin with.
How would you possibly measure the “Probability of an Invasion of Yugoslavia in 1951”, in March 1951?
Or use a histogram.
I have a habit of saying "almost definitely" which I have tried to break but I still fall back to it occasionally. And I know several people who will say something is "definitely possible" or "certainly a possibility" or something along those lines. It's all insecure language we use to avoid making a statement that might turn out to be wrong.
I often say "definitely possible" when I am not sure what the chance of something happening is but I ought to acknowledge that it is possible. It is definitely possible that I should choose better language to communicate this.
"The odds are more like a million to one!"
"So...you're telling me there is a chance!"
Interesting. Two things that jumped out to me were 1) why do the regions of the standardization line not overlap or at least meet? And 2) What's up with the small but clear minority of people who took all the 'unlikely' phrasings to mean somewhere in the realm of 90 to 100%? My guess would be they're misreading the question and that is their estimate of unlikelihood?
Because many people cannot or will not accept ambiguity. Charitably, I suppose this comes from a desire to logically deduce risk by multiply the severity of the consequences by the chance that something will happen. Uncharitably, it gives decisionmakers a scapegoat should they need one.
The actual problem is its not possible to assign a percentage probability - or even a vague 'highly likely'- to a once-off event.
As long as the prediction is not 0% or 100%, its impossible to be wrong.
So, without hesitation, I predict with 99,999999% certainty that an asteroid will hit the Eiffel tower before the hour.
The bonkers thing is that all of the visualizations show basically all of the terms pretty close to the middle on a log odds scale. If "highly unlikely" means 10-20%, how do you express 1 in 10,000??
Realistically? That's within the margin of error of "completely impossible"
For uncommon values you could still use exact language, i.e. just say "1 in 10,000".
I don't understand the point of standardizing language around specific numerical ranges when they could just use numbers.
Gotta love how "almost no chance" includes probabilities less than zero.
Yugoslav communists were pawns of Moscow from the beginning and all the way until the end of WW2. After WW2, they grew some balls and put the interests of degenerate Yugoslavian state before the interests of the communist international, that's what almost cost them their state. And also they were traitors of their peoples because during WW2 they didn't support national movements but wanted cosmopolitan socialistic Yugoslavia with the help of Soviet Union.
How was this not on lesswrong.com, they are all about ]0..1[
This problem crops up everywhere, especially when it's a consequential claim. Eg when the US Department of Energy says with 'low confidence' that the Sars-COV2 outbreak and pandemic was 'most likely' was the result of a laboratory leak, what number does that translate to on the certainty scale?
Also, what likelihood can we assign to claims that the virus was deliberately modified at the furin cleavage site as part of a gain-of-function research program aimed at assessing the risks of species-jumping behavior in bat coronaviruses? This is a separate question from the lab escape issue, which dould have involved either a collected wild-type virus or one that had been experimentally modified.
Perhaps experts in the field 'misinterpreted the evidence' back in the early months of the pandemic, much as happened with the CIA and its 'intelligence on Iraq'?
https://interestingengineering.com/health/us-doe-says-covid-...