back
111 comments
I use the flowerchecker app http://www.flowerchecker.com/ where you have to pay a small fee per plant identification. It actually uses real human biologists to identify the plant.
I've been using it a lot this spring to identify all the wild things growing in my yard. Love it.
You mean college students?
I did this for common plant diseases using Tensorflow on Android when learning how to use TF last year. Found the accuracy to be not that great with real world images that are not in the dataset (with a dataset of 50,000 images from PlantVillage). I think there are just too many visual similarities between various plants (especially when taken from various angles) that the network doesn't focus on the right things. Did a short writeup here:

http://helminen.co/plant-disease

It could work for a subset of plants, or potentially with a much larger training set - but I think NIR spectral/hyperspectral imaging would be the way forward here with more differentiating data points.

>I think there are just too many visual similarities between various plants

I get the impression that you tried to train a single classifier to diagnose disease in any species in the PlantVillage database.

You might get better results by training a separate classifier for each plant species (or starting with just one species, such as tomato, for which PlantVillage has 10 disease categories). A farmer knows what crop they're growing, so can select the correct classifier when they submit a photo.

Did you use inputs other than just image for identification? I would have thought that adding location and time of year alone could significantly improve results.
Not an expert -- would this be a result of training data doing a much better job isolating the subject vs a real world photo that may have other plants in the scene? Not that this is a solution in all cases, but is it possible your results would have improved with better real world pictures?
50k images is too few for that many classes
Nice. Will check it out.

Just knowing what questions to ask, what a bight could be would be very helpful to us plebes.

This product already exists. It's called Google Image Search. For instance, I was actually able to identify the second flower he takes a picture of in the video as Camellia Japonica simply from a screenshot of the video. I'd be blown away if they were able to assemble a better training set than Google. Kind of disappointing, as from the headline I was expecting something based on NIR spectroscopy, like the SciO molecular sensor (https://www.consumerphysics.com/)
There are two problems with this focus. First, lots and lots of photos on internet are incorrectly identified. I had seen web pages telling you how to include some edible spice in your dishes, with a stock photo of a similar species (very poisonous) incorrectly tagged as yummy.

Second, is probably a Camellia japonica... but could be also a Camellia x williamsii. And you need to know that there is a Camellia sazanqua also. A trained human can spot the "too big leaves for sazanqua" in miliseconds (or too much shiny, or suspiciously blue, or photoshopsly faked, etc) but this is not so easy for a program. Search image will not spot the differences and just let you with the most common option.

In fact. There is a better product, Google Goggles - https://play.google.com/store/apps/details?id=com.google.and...
Slightly off-topic: am I the only one who has started uninstalling apps and using the mobile browser for most things?
A great app, collaborative tools and community around plant identification is the french pl@ntnet http://identify.plantnet-project.org/ from Agropolis Fondation, Tela Botanica, INRIA, CIRAD, CNRS, INRA, IRD and Montpellier University
> Fast forward to today - the tech is available and the team is assembled to make PlantSnap a reality. With a beta version completed, and 250,000 images in the database

250,000 images doesn't sound like a large enough training set to be effective on anything but the most common plants.

To help others understand why, the "Deep Learning" book, which summarizes current state-of-the-art DL, advises having at least 5000 samples (images) per class for OK performance (equal to non-DL approaches) and 50k to 100k for state-of-the-art. A class here would be a plant species.

So even with 5k samples, the 250k image corpus would only have 50 species, using this rule of thumb. A good engineer could pick up DL and build a system that performs to this standard, because the tricks are all written down in the literature.

If they do better, they either exploit unpublished methods, or researched those methods themselves, with their researchers.

We did this http://www.luontoportti.com/suomi/en/ a couple of years ago. It is based on an elimination process – you describe what you see and the application narrows down on the options. Works very well in practice, and can be applied to all sorts of indentification purposes in the wild, such as fish, birds and butterflies.

We also gave the image regocnition path a thought but it seemed to be quite a tall order. Hopefully they come up with a novel approach on this!

I used something called "dichotomic keys" in middle school, which are basically search trees ("does the leaf have lobes? If yes go to page 20")

I've looked for an app like that for years, seems yours is, thanks!

When can we get this for identifying plants/animals/rocks/fungus/etc in the wild? Would love this on a phone when hiking.
A similar app was released a while ago, I'd love to compare the two. For reference: http://leafsnap.com/
I use this android app for recognizing plants by picture: http://identify.plantnet-project.org/

It's made by french researchers I think. It doesn't work perfectly but I did identify a lot of plants I don't know with it.

You can snap multiple pictures of the same plant, for example, 2 pictures of the leaves, 1 of a flower and 1 of the bark, and then use the combination to search. You can also submit your observations to have them identified by experts.

Only slightly related, but I've always wondered about an app that told you what crop was growing in the area you were driving through. Often they're recognisable, but sometimes they're not. No business case for it, just to satisfy curiosity.
This is very cool. I've been thinking about something similar, but only about mushrooms. Some people in my country pick wild mushrooms and sometimes it's very difficult to distinguish between the edible and the poisonous types. Such application, if existed could save many lives. The same concept could be used for other plants that people gather, like herbs and wild berries.
Finally, I can identify mushrooms that will kill me and eat the ones that won't.
I've had good luck with plants.usda.gov identifying plants by geographical location. Just use the advanced search, choose the location you found the plant, e.g. state:county, and as much information as you can deduce, flower color, growth form, etc. It'll usually provide pictures and interesting things like whether Native Americans used it medicinally, etc.
http://www.gardenanswers.com app has the ability to automatically identify plants. If you can't find your plant, because it isn't blooming or is immature, or just isn't distinctive enough for image search, you can ask a horticulturist for a small fee.
Definitely an interesting concept. There's a bit more information on the indiegogo website [1].

Their problems they mention seem to be quite standard for image recognition (scale- and perspective variance), however I could imagine that this could be quite disastrous for plant recognition, given the massive diversity of plants: e.g. a leaf viewed from the side could equally well be a more narrow leaf from a different species.

As they write on [1]: "Our challenge comes from adapting our image recognition platform to recognize different shapes and sizes of the same plants, flowers and trees. We know this is possible, and we are close to cracking it."

[1] https://www.indiegogo.com/projects/plantsnap-identify-plants...

It's a nice idea, but I have a feeling it's never going to be better than human enthusiasts/experts, who you can get to identify plants from photos for free already[1].

I can immediately think of all kinds of challenges that are really hard to overcome: diseased leaves, different seasons, plus all the usual glare/shading/background issues.

[1]: https://gardening.stackexchange.com/q/9355/3346

Botantical classification is interesting but difficult. Once trained, it's fast enough, but can "deep learning" be anywhere near accurate enough?

I'd guess there'd be a focus on salient botantical features for classification, and perhaps the human can be enlisted to circle them out. There could be a "twenty questions"-type narrowing down, perhaps using images.

I want do the same for mushroom, because they are hard to recognize. Google images works quite fine.
A pre-order on an app, seriously?
I would imagine a series of questions that you answer about the plant would help increase the accuracy. (Leaf patterns, etc.) Or, at least, whatever answer it comes up with should confirm the defining characteristics.
Having recently become the owner of a garden, this looks very interesting. I've been using the myGardenAnswers app, which purported to do the same thing. It's been singularly useless so far, though.
I've been using this (free) plantnet app http://identify.plantnet-project.org/
Glad to see this as it is something I have been interested in for a while. There are some online tools I have tried in the past but nothing that really seemed to work well.
or go get a book on plants and you will be able to identify them pretty easy. The guidebooks ask you questions to narrow it down (how many petals does the flower have? for example). you dont need any knowledge going into using one, and you'll learn to observe and appreciate plants more as well through the process of IDing! newcombs wildflower is the go-to for beginners on the east coast but there are many.
I love the idea of an app to identify plants, I hate the idea of having to pay to use it... Definitely, I'm not in the target market.
I wanted this app for a while, but it didn't seem feasible even with modern ML techniques? Also what do I get out of a paid version?
Weirdly, the site disables the normal scrollbar but adds a new one inset, which doesn't respond to SPACE to scroll down.
I wanted to do something similar for snakes once, but after some thought it didn't seem like the safest idea.
Just a quick note to the site creator- the twitter icon in the footer is linked to instagram, and vice-versa.
/r/whatsthisplant FTW
This is really cool. IF only I could find an app that does this with bugs.
I trained my kids, first for mushrooms, then for flowers and trees. They are better then any app :). But definitely would love to have an app to confirm our findings sometimes.
I really want this app. That is all.
WTF 150K of his own money in 5 years? I don't think at a business level this is wise under any circumstances for a bootstrapped project. Unless you have a ton of money to waste and this is your caprice. Of course.