back

by wsxiaoys·9y ago·view on hn ↗
For those who think it's just another lame DL based instagram filter...

The method proposed in the paper(https://arxiv.org/abs/1707.03491) is mimicing a photographer's work: From taking the picture(image composition) to post-processing(traditional filter like HDR, Saturation. But also GAN powered local brightness editing).In the end it also picks the best photos(Aesthetic ranking)

Selected comments from professional photographers at the end of paper is very informative. There's also a showcase of model created photos in http://google.github.io/creatism

[Disclaimer: I'm the second author of the paper]

12 comments
My ex-wife is a professional photographer. She does weddings, boudoir, and portraits mostly.

Here's the thing. If she had say, 2 shoots in a week, maybe a total of say 8 hours of shooting time, the selection, editing, and styling of those photos could take 3x or 4x as long as the time spent shooting. She would have to be editing constantly.

I've always thought a perfect application for this sort of technology would be a model trained on that particular photographer's style, which goes through a batch of photos and selects the best candidates and then presents the user with a few choices for each photo it selected and styled.

I think the ultimate would be a NAS with this capability embedded. The photographers I have met through her over the years seem savvy about photo tech, but don't generally use online storage solutions or know much more than how to admin a WP site at best. SOHO solution would be ideal imo.

Great work! I'm excited to see where this goes.

I don't think the NAS will come at all: the economics just don't work out.

It's more likely this will hit consumers hands via an automatically applied 'enhancement' in Google Photos first.

An on-prem storage solution specifically targeting these features at professional photographers would sell.

Storage is always a problem with every photographer I have met. They keep stacks of external USB drives or burn archives to DVD. They don't trust or want to use online storage.

A package that provided these models, trainable on that photographer's particular style, with autocropping, watermarking, etc. would sell very well among this group.

I bet you could make millions. Or even tens of millions. Great for a start-up. But Google doesn't bother with such small fry---especially if it comes attached to a complicated hardware business model.

At these small numbers, they are more likely to just give away access to the software for free.

Oh totally, this isn't something I would see Google doing, and I agree that Photos is definitely a more sensible place for this to start showing up for their consumers en masse. This would definitely a targeted startup type product.

If you had a system to:

- batch ingest new photos

- stylize them to your own style

- categorize them

- autoselect candidate photos for final tweaking

- back up final album photos to local storage and S3

- archive to Glacier the rest of the photos from the shoot that will likely never be touched again

You would have a product that I think would be very attractive to professional and semi-professional photographers.

I thought of a cool application . You could use this to generate desktop photos for desktop environments that are either localized to a place you are near or the user could type in a location that they want to see .
Or an actual photography class app where a user could tell google exactly where they are standing on streetview and have the app tell them the best direction/angle/settings to take their pic and walk them through WHY it is as such
It can't do the latter, except in a very basic way of pointing to proximity to the training set. ML is great at imitating, but good look getting it (or the people programming it) to be able to articulate any of the underlying principles. This is a perennial problem in the arts, which might be why the examples on this page are pretty ho-hum from an aesthetic point of view.
I bet this is something we will get eventually. As I think about it, my mind is flooding with ideas on how this would effect other areas of life.
Cool- Are you going to apply it to all images on Google Maps? How about an 'Auto Enhance" button on Google photos?
They already have that button. But it's doing rather boring things at the moment---but sure to improve in the future.
Yeah, it's more akin to Auto Tone/Auto Contrast/Auto Color in Photoshop: often a quick and easy way to tweak levels/curves on a photo that's already decent, but much more varied results on anything with mixed or more atypical colors and levels of light.
Did you have anyone say "Professional photographers don't apply HDR filters to single photos"?
Why would they say that? You can't have HDR with a single exposure.

I think you're trying to say professional photographers don't apply tone mapping to single exposures, but that is false. There are times when it's not 'technically proper' to apply tone mapping to a single exposure, but it makes for a better picture.

Really it sounds like you're trying to make a snarky comment referencing how some professional photographers complain about novices over using HDR on flickr. But who cares. You can have a professional quality photograph that uses what some pros think is incorrect tone mapping. Just like how you can have a novice hack together a working professional-level product that many professional engineers think is trash. If it works and meets the criteria, who cares.

> You can't have HDR with a single exposure

14 stops of dynamic range in a single shot (RAW on a modern DSLR) counts in my book.

HDR worthy of the name seems to require multiple levels of light gathering, to me. Either exposure time or aperture needs to vary. If you don't, you either blow out on bright light, or you have too much noise in dim areas. You might not notice if the scene is well and reasonably evenly lit though.
You never vary aperture when taking bracketed exposures to merge to HDR. The depth of field would vary from exposure to exposure, making things inconsistent when you merge them. It's shutter speed you vary.

My point is that with 2005-era DSLRs, I had to use HDR techniques (or strobes) to, say, correctly expose a room and the environment out a window. Shooting into the sun, I'd have to use HDR, polarizers, and/or ND grads to expose both the sky and the foreground. With my D750? I've got 4 stops more dynamic range and the situations where I need HDR have almost all gone away. I can get the same thing from a single RAW exposure.

These days, HDR only seems necessary if you want to have ridiculous tone mapping where you pull in your global highlights to be within a stop or two of your global shadows and still keep the noise levels down.

I still find a need to bracket occasionally.

But to your point... it seems that "HDR" means two different things these days.

1. A tool to expand the dynamic range.

2. A style of tone mapping.

These two things frequently coincide, but there's no reason for #1 to have crazy surreal tones, and #2 frequently comes from a single exposure.

Sure they do, what do you think the huge dynamic range capability of modern professional level cameras is used for?
Will this ever get implemented in Google Photos automatic editing?
Related - if not editing, use a panorama image taken from the phone to create stunning professional photographs.
and in a few years, we find a whole bunch of professional photographers out of a job...
By that time I think everyone will be out of a job. At least photographers will still have a hobby ;-)
You say that like it's a bad thing.
Apologies for being that guy...

Was there any control for, e.g., the dynamic lighting filter being the factor that carried most of the water, rather than the other manipulations?

FWIW, I did think your tool did a good job finding good compositions within mundane scenes. Looking at the originals, I have to admit that in many or even most of the cases, I wouldn't have spotted the opportunity - which points to an area I can improve my craft.

Still lame, just cropping and increasing saturation/contrast. Some of the pictures in the blog post look worse than the originals, they are artificial.
The images look really good.

I'm curious: Is this creatism link what the algorithm considers the best images, or did you manually filter out some of the less good ones?

Thanks! Interesting work, indeed. I found the showcase photos appealing and nice. I briefly skimmed through the paper with hope to find some interpretation of the “aesthetics”, but couldn't find it. What is meant under “all aesthetic aspects” and “aesthetic quality”? Did you perform a critical analysis of the network? How do you know that the network learned “aesthetics”?
I tried a style transfer network on one of your images . Here is what I got https://prisma-ai.com/p/3a91d082-feee-443e-8062-4e85d2720ea0
Is the code available?
on the pro side: you could use this to determine the most photogenic scenes in the world.

On the con side: only an AI photo-bot can find and capture the most photogenic scenes in the world.