▲ 1450 points
back
522 comments
You'd be surprised / scared / outraged if you knew how common this is. Any time you've been in a public place for the past few years, you've likely been watched, analysed and optimised for. Advertising in the physical world is just as scummy as it's online equivalent.
Check out the video here http://sightcorp.com/ for an ultra creepy overview. You can even try their live demo: https://face-api.sightcorp.com/demo_basic/.
Woah, so this got a bit of interest. To be honest I'm a little surprised this seems to be news to the HN crowd.
I feel a little bad about calling out one API provider specifically, so here's a bunch more: https://www.kairos.com/ https://skybiometry.com/ https://azure.microsoft.com/en-us/services/cognitive-service... http://www.affectiva.com/ http://www.crowdemotion.co.uk/ http://emovu.com/e/ https://www.faceplusplus.com/
Face tracking, emotional analytics and vision based demographics analysis is a pretty huge industry. There's a entire spectrum of uses for this tech, from the altruistic (psychology labs, humans factors research), to the well, not.
Indeed. This isn't new. It's everywhere.
The cameras retailers use with their surveillance systems are coming with facial recognition built in now. [1]
And lots of retailers, banks, etc, are using systems that track people's visits across multiple locations. [2]
You'll see a lot of these systems being sold as fraud/loss prevention solutions. The reason for this is that it's a relatively easy sell this way - customers can count how many thieves they've caught this way to easily determine the ROI they're getting on the system. Once the systems are in place, it's relatively easy to start using them for marketing related purposes.
Not all uses of systems like these are necessarily unethical. Consider a case where you want to set up a rule like 'if the average lineup length at the checkouts exceeds 5 people, call backup cashiers'. The problem is that once you have something like this in place, it's very tempting for company execs to want to use the data for legal but less than ethical purposes.
[1] https://www.axis.com/ca/en/solutions-by-application/facial-r... [2] https://www.facefirst.com/solutions/face-recognition-predict...
I'm 65 and it says I am 39.
This is a wonderful app. I will use it every day!
It also picked up the colors in my aloha shirt perfectly. (Anyone who knows me knows that I am to aloha shirts as Steve Jobs was to black turtlenecks.)
When I want to feel young and go shopping for shirts, now I know what to do!
I tried the demo. A little sad, that it sees me as 12 years older than I am, and that I apparently always look angry and disgusted! I'm going to blame it on my glasses and bushy beard, and try to look at the bright side - apparently face scanning systems aren't quite good enough to get a read on me yet. (And try not to be too sad about looking like a grumpy old man)
(Anyone else with a glasses, a beard, or other non-typical facial features want to comment? I'm curious now how well their system handles these?)
I was very unhappy to discover that shoes now often have RFIDs built into soles. This + anti-theft RFIDs readers that are already deployed by the entry of most stores can allow to easily assign unique ids to shoppers.
By grinning like an idiot I was classified as a very happy 33 year old female. I'm a 25 year old male.
I tried variations of the standard expressions and pulled off sad, disgust, anger quite easily.
I knew binge watching Lie To Me before my psychology mid would come handy at some point!
I was blown away by the accuracy of Microsoft's https://how-old.net/ This just kicked it up a notch.
"Emotion Recognition
Understand how your customers feel. Detect and measure facial expressions like happiness, surprise, sadness, disgust, anger and fear."
Creepy, indeed.
>You can even try their live demo
I'd rather not give them my facial image so they can optimize for me.
In Japan at least, before automated facial recognition, cashiers recorded buyer demographics by hand. I would think other places do it too.
Edit: Here is what the buttons look like. Gender and age. https://image.slidesharecdn.com/hvc-c-android-prototype20141...
During 2010-2012, I was part of a startup called Clownfish Media. We basically created something very similar to this and got scary accurate results then. Given how accessible computer vision has become, the image in the tweet comes at no surprise to me.
Best part - we got a first gen raspberry pi to crunch all the data locally at 2-5fps. Gender, age group (child, youth, teen, young adult, middle age, senior), and approximate ethnicity were all recorded and logged. Everyone had a unique profile and could track people between cameras and days (underlying facial features do not change).
Next time you look at digital signage, just be aware that it is probably looking back at you.
(Supposedly) Lee Gambles comment on Reddit -
"Hi. I am the original taker of the photo. There is a screen that normally shows peppes pizza advertisements in front of peppes pizza in Oslo S. The advertisements had crashed revealing what was running underneath the ads. As I approached the screen to take a picture, the screen began scrolling with my generic information - That I am young male (sorry my profile picture was misleading, not a woman), wearing glasses, where I was looking, and if I was smiling and how much I was smiling. The intention behind my original post on facebook was merely to point out that people may not know that these sort of demographics are being collected about them merely by approaching and looking at an advertisement. the camera was not, at a glance, evident. It was merely meant as informational, maybe to point out what we all know or suspect anyway, but just to put it out in the open. I believe the only intent behind the data collected is engagement and demographic statistics for better targeted advertisements."
Source: https://www.reddit.com/r/norge/comments/67jox4/denne_kr%C3%A...
To be honest, I am fairly surprised at the reaction here on HN. It's not really surprising to see such system, it would be more surprising if such system did not exist because offline ads is a huge business and the technology is here. This goes together with conversion tracking at physical shops, etc.
I am equally surprised by the comments about how come engineers implement such systems, how they find it ethical, etc. I'm sorry, but it sounds just a bit out of touch with the real world, or just outside of HN bubble. Given the things that money motivates people to do, it's probably one of the least unethical things that has been done.
I am not judging that this is right or wrong, I am simply stating the fact that nothing about this should be surprising. Yes, this is slightly sad, but that's simply the reality of technological advancement. It's not really possible to expect the rest of the world to use the technology only for things considered 'right', etc.
As someone working on a similar project (specifically, emotion recognition) I'm highly interested to hear how such a product should look like to be not considered unethical. So far from the comments I see that:
- it should be made clear that you are being analyzed e.g. by big yellow sticker near the camera
- no raw data should be stored
- it should be used to collect statistics, not identify individuals (?)
Is it sufficient to consider such a software as a fair use? What else would you add to the list to make it reasonable?
Uh, I got sidetracked and brain hammered by the devolving discussion on that Twitter thread, thus couldn't find the context for this pizza shop kiosk - Is it a customer service portal that attempts to identify the person in front of it to try and match up with an order, or a plain advertising display that is trying to capture the demographics of the people who happen to stop in front of it and look at it?
It saddens me that there are people in our profession for whom implementing such a thing presents no moral or ethical dilemma.
Here is the software the sign is using.
Though not at the level depicted in the movie, I am nonetheless reminded of Minority Report.
I've sifted through 80% of the comments here, and I couldn't find a mention about the unintended consequences of this technology.
Ethical vs. Unethical, Pro-Privacy vs. Against Privacy are the two common discussion points. I, however, think the bigger problem here is that there's a very non-zero probability that this technology may cause unintended consequences simply by relying on false/inaccurate data.
For one, I work in analytics (loaded catch-all occupation) and I work with people who would marry their "data skills" if they could. In my industry, false positives of 80% is acceptable, and openly admitted errors in "machine-learning" logic (quoted to highlight my company's buzz-word usage, but practically non-existent) are made daily. People create algorithms, and people make errors.
Let's let our imagination run wild here for a second: It's 2030, and this technology becomes ubiquitous to the point where no one objects. Businesses take all the data from sentiments, gender, age...etc. to optimize for their target demographic, and price accordingly. In other words, let's assume this tech is used for perfect price discrimination. Economic theory dictates this is a win-win for everyone since everyone starts paying their willingness to pay. But, let's assume there's a catastrophe and medicine is in dire need. Price discrimination works fine assuming perfect competition, and is a useful framework, but it breaks down empirically where we live in a society that doesn't behave so rationally. Who survives? Those willing to pay the most, and the algorithm worked flawlessly here. But it was not intended to dictate who survives.
What I'm trying to say is that we should be cognizant of the fact that we don't live in a perfect bubble, and technology like this should be scrutinized for it's effects exhaustively- including any unintended consequences. We live in a society (duh), and as a society, it is up to us, with the help of policy makers, to determine the fate of this technology.
Okay, time to start wearing ski masks and Santa Claus costumes in public at all times.
There should be one of these in front to the ad to give passers by more privacy:
http://hackaday.com/2010/10/15/window-curtain-moves-to-scree...
I'm a little surprised about the HN reaction on this one. You guys didn't seem to care about collection of passive biometrics a year ago: https://news.ycombinator.com/item?id=11172652 . What's changed?
This is in itself not scary compared to what a random website does when I visit.
That is - given that what we see in this log is actually all it does. What's scary is what we don't see (does it store this? does it cross reference anything? does it target ads based on it)?
I don't really think that's the case (here, yet) but I do think it's scary that it's so easy to do that its not just done as a proof of concept but actually used in production in a low tech industry.
Gathering demographic or sentiment without storing, cross referencing (has this person been here before etc) or otherwise using the data for anything such as targeting ads - is kind of acceptable. I mean it wouldn't be hard to do that manually via a camera if you wanted to test the engagement of an ad. I'm sort of hoping this is just some tech project from a university or something, and not an actual product you can buy and hook into some adtech service.
Edit: as someone else pointed out - it's not a proof of concept it's an adtech off the shelf product. Because of course :( http://www.adflownetworks.com/audience-detection/
And just today I got an ad (in the paper mail) from an electronics distributor notifying me of new parts they stock. Among them was an embedded face and expression recognition engine that would emit pretty much this data, in a convenient text output you can read into any little microcontroller and act on (omron B5T-007001-010 if anyone is interested). This is no longer exciting cutting edge technology, it's off the shelf. And terrifying.
For consideration :
How to ZAP a Camera: Using Lasers to Temporarily Neutralize Camera Sensors
For an even more disturbing idea, Titan Park in Beijing has machines in its toilets which scan peoples' faces before dispensing toilet paper.
https://nakedsecurity.sophos.com/2017/03/21/park-uses-facial...
Oh look, Ghostery's product pitch comes in handy now
https://www.youtube.com/watch?v=EKzyifAvC_U
With this much personal care to really know their customers by face I'm sure they put just as much personal care into the quality and craft of the product /s
Despicable. Any authors of this work should be publicly shamed and punished. And don't get me started on what should befall the owners of capital that drove this.
A norwegian article on the subject: http://www.dinside.no/okonomi/reklameskilt-ser-hvem-du-er/67...
Google translate is readable, if not super-mega-accurate: https://translate.google.com/translate?sl=auto&tl=en&js=y&pr...
I saw a pitch for this tech 5 years ago. Not sure the name of the company. The idea is they can measure engagement (how long you looked), approximate age and sex.
Five years ago it didn't seem so sinister. A lot has happened since then, I guess.
Funny that this has been posted by ambient/experimental music producer Lee Gamble https://soundcloud.com/leegamble
This has been happening for at least a decade. I had to do some updates on a system in 2008 that had this same functionality built in, and they were far from the first company to do it.
I don't see any evidence that this is a "facial recognition system."
It's likely hard to legislate against software that attempts to detect if there is a person, what their expression is, and guesses at their gender.
You could imagine that job being done by a person (just noting how many people stopped at the advertisement, and what their expression was). I don't think there's really a way to make that illegal.
I suppose I think it's something that people should be aware of, though.
I work in a big retail company here in Brazil.
If you enter most of our stores with a phone in your pocket, you're being tracked. They track where you went, in front of what shelves you stopped and for how long, if you went to the cashiers of just left...
And if we track people here in the third world, you can be sure you are being much more tracked in first world stores.
Original Reddit post with background:
https://www.reddit.com/r/norge/comments/67jox4/denne_kr%C3%A...
"...peppes pizza in Oslo S."
There are many solutions that do this, both proprietary and Open Source. Accuracy is influenced by lots of factors, some to do with the setup (camera angle, lighting) the hardware (camera quality, computation speed) and the subjects (race, facial hair, glasses). We used this in research projects involving elderly mood assessment and in television viewer's emphatic responses. The package we used was marketed by Noldus ( http://noldus.com ) and developed bu Vicarvision ( http://vicarvision.nl ), but most of these packages perform at about the same level.
This is nothing new; Ad Companies are actively marketing this features. See i.e. http://livedooh.com
Quote from their website: "Audience Measurement included
The information and statistics needed in order to realize audience targeting in DOOH is gathered through livedooh’s integrated anonymous video analysis, which collects information about gender, age and length of view. Audience metrics are used by the ad server’s decision engine to optimize advertisement delivery and increase performance."
It just makes too much sense to show ads based on the demographics. They now have robots in malls too. They are just recording everything, processing, logging, extracting, selling and up selling. There is no privacy. The problem is not only does it make economic sense just to have these robots, the added intelligence from the data mining makes it even more attractive.
This is illegal in both Sweden and Norway though.
The Island Airport(Billy Bishop) in Toronto is littered with adverts that are camera connected. Other than, maybe power management, looking at what is possible in OpenCV gives a good indication of what can, and probable is, being done. From tracking where you look to matching faces....
The problem is that this is an agency(of the government) owned facility.
Imagine you can automatically track customer loyalty and offer them discounts. Here is an integration example using arduino also https://www.hackster.io/dasdata/dascognitiveservices-c2d991
India is forcing this Orwelling nightmare upon all its citizens (under very shady circumstances).
http://www.rediff.com/news/column/the-aaadhar-effect-say-bye...
God the comments on that tweet are enough to make me lose all faith in humanity, honestly what is wrong with people.
I wonder if someone wore a shirt with this pattern while walking in front of it
https://thenextweb.com/tech/2017/01/04/anti-facial-recogniti...
how does it detect which gender you identify as? mind reading?
I wonder how accurate these measurements are in practice. They could just be placeholder implementations, right?
male young adult
I think they zeroed in on their demographic, good job!