back
101 comments
I speak German and I have just read the text, it is vastly complicated court and lawyer speak.

But it is clear that this legislature is pushing for crime prevention based on aggregated data from computer systems(where from, exactly?)

The arguments are that a manual extraction and collection would be too slow and that this should be limited to only a handful of severe potential crimes. As a prevention rather than mitigation technique.

This is quite bad.

Now the objection is not purely because the data is coming from computers, but because it will reveal data on people tangentially connected to the targets.

This is so wrong in so many ways and the people who are pushing from this have learned nothing from the stasi and nazi regimes, or perhaps they are nostalgic.

It is very clear that these things are pushed by rather smart minds, matter experts and this makes it clear they do not care about the innocent peoples privacy, as they could not possibly be so ignorant to overlook this matter.

Justice even the conventianal way spawns injustices, now the lazy government wants to automate and scale their procedures. W.t.h.

> innocent peoples privacy

It isn't just about their privacy but also their very freedom. Being in some police database might very well land you in the crosshairs of some investigation and related niceness like searching your home, taking you in for questioning and keeping you there because you didn't say something...

> It is very clear that these things are pushed by rather smart minds, matter experts and this makes it clear they do not care about the innocent peoples privacy

I was working that this behaviour by FAANG will normalise it, and will lead to our governments doing the same.

Same 'smart people' work for government and fang, they are gonna behave the same way.

Super interesting ruling in my book.

Skipping ahead to the fact that automated data analysis is not prohibited per se, just needs to be reasonably targeted, the two most interesting paragraphs are the last ones for me.

Basically the court mandates that any future laws passed for legalizing automated data analysis must explicitly

1. For any given crime to be automatically analysed, define the data model and define appropriate analysis techniques or methods. Those laws should be interesting reads.

2. For any one analysis happening, document and later publish to the public domain the reason for initiating an analysis, the methods chosen, the criteria for labeling someone as "guilty/innocent" - basically the entire process so that the analysis is auditable end to end.

Again, this is not a suggestion in the ruling, but a requirement.

This is really good news for personal liberties in my book.

Stark contrast to US/ China data mining on citizens.

2. reminds me of the 2016 French Act for a Digital Republic :

(scroll down to 3.1.(g) Public sector)

https://www.mondaq.com/france/technology/1059760/artificial-...

> whenever an individual decision is taken on the basis (even partially) of an algorithm, the administration must [...] explain [using easy to understand language]†, at the request of the individual, how the relevant algorithm works [...], by providing the following information:

    the degree and mode of contribution of the algorithm to the decision making;
    the data processed and its sources;
    the processing settings and their weighting applied to the situation of the data subject; and
    the operations carried out by the processing.
† a very important detail that this translation for some reason omits ??

Now, something that has been bugging me since then, especially in the «AI» context that was also being discussed more or less at the same time by the legislators, is that so far this is impossible to do for neural network based programs, since the final program is not based on any algorithm (as commonly understood), or at least not one that even the programmers themselves could understand, much less distill into a commonly intelligible form !

An earlier poster wrote: > This is so wrong in so many ways and the people who are pushing from this have learned nothing from the stasi and nazi regimes, or perhaps they are nostalgic.

Whereas you say: >Skipping ahead to the fact that automated data analysis is not prohibited per se, just needs to be reasonably targeted, the two most interesting paragraphs are the last ones for me.

I think this reads like the court curtailed reasonably broad data grabbing behaviors such as those Snowdon has revealed happened illegally and are still happening in the US. It still permits effective, focused/targeted data mining.

So at least the court learned from the Nazi/Stasi times but also from the times of left-wing terrorism ("Rasterfahndung" during the Bader-Meinhoff terrorist gang) - i.e., that these techniques are effective yet must not be applied to broadly. If this reading is correct (I'm not a legal expert), then the main danger is that lawmakers are incapable (or unwilling) to implement requirements (1.) and (2).

This is such a German judgement. So the police are using big data to predict who will commit crimes, which is sort of dystopian. But what the court finds unconstitutional about it is that they're using computers.

"Automated data analysis or interpretation requires justification under constitutional law."

I'm sure something is lost in translation. But even assuming a lot of context, I find it hilarious. Germany's highest court seems to have stumbled into avoiding a dystopia not because they don't want some biased ML model bullying minorities, but because they plain and simple don't like all this computer nonsense. (Grumble grumble.)

Requiring humans to sift through the data makes it easier to limit its use.

You either need a lot more budget (= oversight) to hire all the people to do the work, or do less of it.

Sure computers also cost money but you can keep using them as long as you pay the (already budgeted) power bill and (also already budgeted) maintenance/upkeep.

That's not correct. In the same paragraph it states that in general a digital analysis of information is necessary for increasing police effectiveness and that this is suitable under constitutional law.

The issue that the constitutional court is pointing out, is that there is no appropriate threshold for using such analysis software. In other words they that that you can't use is for anything, just when it is really necessary and proportional.

This is such a German judgement. So the police are using big data to predict who will commit crimes, which is sort of dystopian.

Police in the United States are close to this already. Chicago's police department used to brag about its data-crunching ability to predict where crime would happen. At the time, it was just by area, but it was stated that with enough data, it could be narrowed down to the block. It's not that big a leap to extrapolate that to the individual.

My guess, though, is that it didn't work, and the city was just parroting the promises made by the salespeople of whatever system they bought for this task. If it worked, then police would preemptively flood certain areas where they know crime is about to happen. But, as is seen on television most nights, and in the complaints of the aldermen, the police mostly seem to react to crime, rather than prevent it. The use of social media by crime mobs has only made it worse.

As a German native I think you're a bit off both on translation and based on understanding of the foundations of the German constitution.

No constitutional law expert myself, trying to contextualize in the hope it helps:

Core pillars of the German constitutional system are informational self-determination (i.e. personal data has a very high degree of protection under the constitution, see Germany's history) and proportionality (in using its powers, the state must choose methods appropriate to specific situations, obv. to hinder misuse of power - see Germany's history again).

So the ruling in essence, as I understand it, says among other things:

1. Automated data analysis needs to respect informational self-determination of anyone (i.e. no data mining per se on people who are not suspects for a given type of crime being investigated)

2. Automated data analysis needs to be proportional, meaning that the purpose of running any analysis "for the benefit of the public" must be reasonably well established and the analysis must have a strict goal.

I.e. if John stole something, you have to analyse the data such that you can credibly prove that identification for theft was the purpose.

Starting with looking for thieves but ending up identifying X movie pirates, Y weed consumers etc. is not proportional and crosses the right to informational self-determination.

So in the above example, the movie piraters and weed smokers might have not had an official investigation into their offenses on record, so by law the presumption of innocence must hold - you cannot do blanket investigations without reasonable cause under German law.

In the final paragraphs, the text actually states under which conditions automated data analysis would fullfil constitutional requirements:

1. The legislative needs to codify which data points exactly can be used for a given offense where offenders are to be identified with data analysis. Laws also need to explicitly codify limits of usable data.

2. If automated data analysis is performed by authorities, they must document the reason for initiating an analysis, the methods chosen, the criteria for labeling the outputs as "guilty/innocent", document the entire procedure and be able to release all of it to the public domain so that public data protection officers - or anyone else - can audit the results.

All of this sounds super reasonable to me.

To be clear, automatic data analysis for the purpose of extreme threats to national or public security are exempt if I understand correctly. These would be cases where it is proportional to perform such analyses for the greater good.

But you cannot do mass number crunching on ordinary citizens.

Yay in my book.

I no lawyer but a friend of who is said that there are differences in how countries practices law. The nordic countries in Europe are concerned with the process. The assumption being that if the process is right the judgment will be just. In southern Europe one is more concerned with the outcome, as long as the judgment is just, then there is wiggle room in the process.

This seems like a judgment that focuses on process and not outcome.

I have probably butchered what my friend said.

The one argument I rarely see against automated analysis is that any analysis has a false positive rate. If you hire 1,000 humans to do that analysis there's simply a limit to how many errors they can make in total.

If you introduce automation doing the same thing at a scale corresponding to 10,000× the human capacity, you will have 10,000× as many errors. This will violate Blackstone's ratio and erode trust in the system.

TIL: Blackstone's ratio
Well, every model has its errors, and intrusive criminal "prevention" is self-fulfilling.

Just imagine, everyday 6AM an officer come at your door to check your apartment "to prevent" your crimes, suddenly every random guy in the street looks suspicious and spying on you, then some officers just randomly walks by your workplace, police stop you everytime you travel... This would eventually lead to an insult to the mafia guy and then boom you have already commited contumacy and the model was right! You are a criminal and you should be punished!

It’s a little like a machine controlled ‘stop and frisk’.
you just described the minority experience in many parts of US
It is unconstitutional because the law was written too broadly. The court said two things. It's fine to use this for highly legally protected goods, e.g., protection of body, life, or freedom. However, the same high standards need to apply like for hidden surveillance measures currently used.
Editorialized title unfortunately. The legislation of two German states to allow automated data analysis ... is unconstitutional. They'll need to narrow the scope and specify thresholds when to apply the data (and will probably do well to limit application and storage to not run the risk to have it struck down again).

It's not a broad decision that automated data analysis for crime-prevention is unconstitutional.

"Dieser Eingriffsanlass bleibt angesichts der besonders daten- und methodenoffen formulierten Befugnisse weit hinter der wegen des konkreten Eingriffsgewichts verfassungsrechtlich gebotenen Schwelle einer konkretisierten Gefahr zurück."

This roughly translates into "there has to be a sufficiently concrete danger to warrant such a ambiguously formulated infringement."

I truly appreciate this sentiment. Any collection or analysis of data "just because there might be something in there" has to be weighed against the right of everyone that is being data-mined.

This freaked me out about Minority Report when I first watched it: those precogs are essentially hacking into the future memories of everyone involved in _and by happenstance surrounding_ the lives of future criminals and their crimes.

Mass-scale datamining is effectively the real-world equivalent of telepathy. And I want neither precogs nor the government in my mind.

I think it's crazy that people are considering "linking previously unconnected automated databases and data sources in analysis platforms and permitting systematic access of data across sources through searches." as something like Minority Report.

The truth is that Law Enforcement investigations are currently taking information from dozens of silos/databases, manually copy&pasting them into Excel and Word and then finally making a pdf or powerpoint to share the results. This is obviously a really inefficient way to do things and wastes a lot of taxpayer money.

A technical solution that would allow LE to crosslink data they already have is a no-brainer.

P.S: Very biased view, as I'm building/selling exactly this tool :)

> A technical solution that would allow LE to crosslink data they already have is a no-brainer.

A technical solution that is Open Source, that falls under federally mandated criteria for how it can be legally used, that automatically documents any and all access and makes the entire audit log part of any output result so that all happenings are easily auditable by 3rd parties... might be a no brainer.

Anything else I'm highly skeptical.

If going after syndicated and or international crime, fine.

First feature I'd like to see is linking Finanzamt with LE and go after money laundering - Germany being an absolute paradise to the detriment of everyone else.

If it takes manual copy paste to protect ordinary citizens from overreaching police, I welcome that technical limitation.

Very biased, as I am an ordinary citizen.

How do you sleep at night?
Every human needs to understand that having a computer look at data representing you is a search.
So how would fraud prevention be interpreted?

For eg: denying service from a German retailer depending on the ML prediction result of the transaction being classified as fraud probable(Germany has pay by invoice), would also be constitutional or not?

Or credit score/risk rating using ML, trained on a feature extracted from parameters like Zip code, ethnicity could also classified as automated data analysis for prevention of criminal acts, right?

Or this is only applicable to police?

In general a retailer can refuse any customer for any reason (unless you're discriminating against a protected class)

The reason they do this isn't to fight crime, but to save the business from losing money. So I don't see how a ruling about preventing crime would apply.

Only applicable to police use.
This is straight up some Minority Report stuff
The only problem with Minority Report (aside from torturing the precogs, I mean) was treating the prevented murder as a crime and punishing it. If they had just used it as an intervention tool ... well, there'd be no movie.
OMFD, just watched it yesterday! 8(

I think it's more like borderline presumption of guilt.

In the US - this should also be the case. It should fall under the 4th amendment as unreasonable search and seizure. But the NSA, and probably other agencies, has been doing it for years anyway.
For small crimes. For bigger crimes (limit TBC) they're fine.

And I think that is ok...

This concept was explored in a stephen spielberg movie. Minority report. Good movie. Watch it. Stars Tom Cruise and has mission impossible vibes.

In the end the government cancelled the program because of free will. A person always has the choice to change his actions because his actions aren't predetermined.

(In Germany)
Also it's not unconstitutional in principle, but the range where it would be allowable is fairly narrow. If I understood it correctly, it would be permissible in similar circumstances as those that allow wiretapping
I think it’s important to mention Palantir in this context.
time to make amendments!

just like once upon a time it was constitutional to own slaves and now it isn't

just think about the monetary cost-savings! (nevermind the human misery)

What do they mean with "first alternative"?
If it prevents even one misgendering it will all be worth it.
There has to be a poe's law equivalent for gender issues at this point, if the article is anyway political someone's gonna mention it because uhh they gotta be mad at someone
In the medium-term, the cultural shear between the US and Germany will continue to be highlighted by decisions like this.

Germany leans heavily to more and more privacy because they have the experience in recent memory of what a government can do with vast information collection and centralized authority.

America leans lightly towards more and more centralization of data because they have the experience in recent memory of what individuals who "slip through the cracks" can do. So in Germany we see decisions like this and in the US we see police using privately-curated DNA databases to defrost the cold case of a serial killer, and they're lauded as heroes.

(It's worth noting, of course, that this is all relative. Relative to most of Europe, the US is starting from a place of massive decentralization and minimal individual-tracking. Some people still find social security numbers controversial).

Can't find the study now, but there was a sociological survey of software engineers in the US and Europe, one really interesting thing was that the top social concern when implementing solutions for Americans was fairness, for Germans it was privacy. The other concern didn't even make the top 5 for either.
If china ever gets democratic, a new mighty privacy player will appear. Chinas citizens have the same nasty experience as east germans had with the stasi and the poles, chechs with the cheka. They have been spied upon, reported upon by there fellow citizen and have been victims of "zersetzung". Aka social engineering to destroy a citizen.

https://www.stasi-unterlagen-archiv.de/mfs-lexikon/detail/ze...

Link is german: Shows how denunciation aka controlled leaking of private life like affairs was used, to destroy the lifes of resisting citizens.

PS: FB and Google could in theory use similar instruments against citizens that put pressure on them. Nobody could ever catch them at it.

If the ccp ever gets swept away and replaced by a new better china, the resulting parties will outdo one another to promise freedom and privacy.

I don't understand that last bit : in my understanding, the way that social security numbers are used in the US would be illegal in most (?) countries in Europe ?

For instance, in France specifically since the 1974 SAFARI scandal that resulted in the creation of the French data protection authority and assorted laws in 1978, and happened when the government in secret attempted to (and had to stop due to the outrage) link various (then paper) administrative files into a centralized computer database through a single social security number.

> Germany leans heavily to more and more privacy because they have the experience in recent memory of what a government can do with vast information collection and centralized authority.

I theorize that in a hundred years or so, North Americans will have learned this lesson the hard way and will have atrocities in recent history, too.

I'm sad I won't see the day when Bluffdale is repurposed as a memorial to state surveillance, much like the Stasi headquarters are now in Berlin.

As if they give a shit. This was the case since decades they doing similar practices.
Why? IMO we should use facial recognition, digital ID, a cashless society and everything at our disposal to stamp out crime.

It's the working class who pay the price at the moment. We get burgled, mugged and killed, meanwhile the wealthy lawyers and politicians can pontificate over constitutionality from their gated communities and private vehicles.

It's insane that the EU is trying to ban automatic facial recognition with the terrible AI act - https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A... - nevermind the sheer amount of bureaucracy it introduces for startups (a government register of "possibly harmful" AI - wtf?).