Deep learning outperformed dermatologists in melanoma image classification task

Since this is a journal focused on cancer and not machine learning, I can understand why the editors would see this paper as being worthy for for publication. Unfortunately, many of the readers will read the paper uncritically.

If possible, you should write a critical response to this paper, focusing on its methodological flaws, and send it to the editors. It doesn't have to be long; critical response are usually a couple pages at most. This is likely the most effective way of removing (or at the very least, heavily qualifying) bad science from research journals.

Why would you ever balance your test data? If 80/20 is the actual population distribution, the sample that forms your test set should conform to that. Balance all you want in train/validation sets, but never the test set.

Not balancing and using ROC is a terrible combo, but the metric is the problem, not the lack of artificial balance.

I would say your criticism is way off base. I've developed and fielded ML-based medical devices and this looks like a reasonable study that suggests they have a system worthy of further testing. There's nothing wrong with using an ROC curve here and they document the experience of the doctors, so they weren't hiding that and around 60 or so doctors had greater than 5 years experience. Also, studies like this generally don't use only biopsy-proven negatives, since that would bias the negatives towards those that were suspicious enough to biopsy. Without knowing more details than what the paper provides, I cannot say the results are valid, but I also don't see any terrible errors after a quick scan. The main weakness is probably the fact that the test set came from the same image archive used for development. As a result, there can be all sorts of biases the CNN is using to inflate its performance unbeknownst to the developers. The best way to eliminate that concern is to use a test set gathered through a different data collection effort using different clinics, but that is expensive and time consuming and not something I would do initially. This looks like a good first step and I would encourage the developers to carry it further.

EDIT: I'll add that the ratio of positives to negatives in the training set is irrelevant and in no way invalidates the study. As far as testing goes, there is always a balance you must strike in a reader study involving doctors. Ideally, you would have the exact ratio a doctor would encounter in practice, but for a screening study, that is typically impractical as you would need a huge number of cases and doctor time is expensive. A ratio of 1 positive to 4 negatives is entirely reasonable, although the doctors (particularly the less experienced ones) will almost certainly have an elevated sensitivity and reduced specificity since they will know it is an enriched set, but this is reasonable for ROC comparison purposes as it mostly just selects a different point on the doctor's personal ROC curve. Note that some studies even tell the doctors beforehand what percentage of cases are positive.

Thank you for posting this; I can see that this evaluation came very easily to you because of your experience and expertise but to me it shows how much knowledge is required to evaluate something like this. There really should be a protocol defined around this kind of study that encodes the criticisms that you make here (and others) and stops publication of this kind of thing in its tracks.

Agreed. See this paper for a reputable reference in this space: https://www.nature.com/articles/nature21056

Why can't you use ROC with an imbalanced dataset?

My understanding is the PR curve is preferable to ROC since the ROC can make it difficult to discern differences between models on imbalanced data; but the ROC is still a valid way to compare/measure models.

I work as an ML engineer, some thoughts:

The train/test data being imbalanced in the same way does give the model an advantage, but I don't think that making the test set 50% would solve the issue completely either. Doctors have been "trained" on the true distribution, while which is not 50% (I'd guess that the true distribution is actually extremely unbalanced).

The model isn't simply learning to predict no 80% of the time, it is learning the distribution of the data with respect to the input features. For example, let's say that we have a simple model with only 3 binary features. It may learn that when features X_0, X_1 and X_2 are 1, the probability of cancer is 70%. This isn't a simple multiplication of the true probability by the upscaling factor though--it depends on the percent of negative samples with this feature vector and the percent of positive samples with this feature vector.

If we are to change the test set to be 50% positive and keep the same train distribution, the model no longer has the correct information about cancer rates with respect to feature distributions, but neither does the dermatologist. The specificity and sensitivity continue to not be interpretable as predicted specificity and sensitivity in the real world.

There is no issue with reporting specificity/sensitivity if they had used the true distribution of cases. Yes, the curves/AUCs will look better than the precision/recall rates, but they do not mis-represent what the doctors are interested in (what percent of people will be missed, and what percent of healthy people will be subjected to unnecessary procedures).

Anyways, the classifier doesn't actually seem to be that good, there's actually doctors that were better than the classifiers if you check the paper.

Sensitivity and recall are two names for the same thing, Mr Stats 101 :)

Also, please explain the problem with using ROC here. The probabilistic interpretation of ROC's AUC is the probability of correctly ranking a random mixed pair (i.e. ranking the positive example higher than a negative one). How is that metric affected by the 80/20 split of the test data? Genuinely curios here...

It does not matter whether the data is balanced or not when you report ROC (AUC), sensitivity and specificity for the purpose of comparison of two ways of image interpretation (e.g. humans vs. machines) as long as the evaluation is done on the same dataset with the same methodology. Obviously, the absolute numbers would not mean much outside of the study.

> test data should be balanced or they should correct for this in the analysis.

Why should it be balanced? It should be the expected natural clinical class distribution, no? The humans have priors about this too. If anything, it should be more imbalanced, as I would guess (I would hope!) that less than 20% of scans are malignant.

Very useful, thanks for this level of critique.

I wish they added this context in the limitations section. The paper only says:

"There are some limitations to this system. It remains an open question whether the design of the questionnaire had any influence on the performance of the dermatologists compared with clinical settings. Furthermore, clinical encounters with actual patients provide more information than that can be provided by images alone. Hänßle et al. showed that additional clinical data improve the sensitivity and specificity of dermatologists slightly [5]. Machine learning techniques can also include this information in their decisions. However, even with this slight improvement, the CNN would still outperform the dermatologists."

Your points hit on validity issues. Where would it fit on the errors of omission/commission scale?

While I agree that there are problems with the paper, I think you are confused about suitability of ROC, PR and how test set class imbalance affects them.

Your first two suggestions combined together are very wrong. If you made the test dataset balanced and then measured PR curve the precision would be way too optimistic as it is directly affected by the class imbalance. ROC curve on the other hand is invariant to the test set imbalance.

You can find interesting this short article I have written about this problem: https://arxiv.org/abs/1812.01388

> * They measure the wrong things that reward the network. Because the dataset is imbalanced you can't use an ROC curve, sensitivity, or specificity. You need to use precision and recall and make a PR curve. This is machine learning and stats 101.

A̶F̶A̶I̶K̶,̶ ̶a̶ ̶R̶O̶C̶ ̶c̶u̶r̶v̶e̶ ̶c̶a̶n̶ ̶b̶e̶ ̶m̶i̶s̶l̶e̶a̶d̶i̶n̶g̶ ̶f̶o̶r̶ ̶a̶n̶ ̶i̶m̶b̶a̶l̶a̶n̶c̶e̶d̶ ̶d̶a̶t̶a̶s̶e̶t̶,̶ ̶b̶u̶t̶ ̶t̶h̶e̶ ̶A̶U̶C̶ ̶i̶s̶ ̶s̶t̶i̶l̶l̶ ̶o̶k̶a̶y̶ ̶f̶o̶r̶ ̶s̶e̶l̶e̶c̶t̶i̶n̶g̶ ̶m̶o̶d̶e̶l̶s̶.̶ Edit: This is incorrect, a PR curve + PR AUC should be used for model selection if imbalanced. I agree it would be really misleading if they (say) just reported accuracy (since the null classifier of always guess negative would give 80% overall accuracy). I̶ ̶t̶h̶o̶u̶g̶h̶t̶ ̶t̶h̶a̶t̶ ̶t̶h̶e̶ ̶A̶U̶C̶ ̶f̶o̶r̶ ̶R̶O̶C̶ ̶c̶u̶r̶v̶e̶ ̶s̶h̶o̶u̶l̶d̶ ̶s̶t̶i̶l̶l̶ ̶b̶e̶ ̶a̶ ̶v̶a̶l̶i̶d̶ ̶m̶e̶a̶s̶u̶r̶e̶ ̶s̶i̶n̶c̶e̶ ̶i̶t̶'̶s̶ ̶s̶h̶o̶w̶i̶n̶g̶ ̶h̶o̶w̶ ̶m̶u̶c̶h̶ ̶b̶e̶t̶t̶e̶r̶ ̶t̶h̶e̶ ̶m̶o̶d̶e̶l̶ ̶p̶e̶r̶f̶o̶r̶m̶s̶ ̶t̶h̶a̶n̶ ̶r̶a̶n̶d̶o̶m̶ ̶g̶u̶e̶s̶s̶i̶n̶g̶.̶

How do you usually handle imbalanced data? I've had some success with SMOTE or weighted loss for imbalanced datasets, but I'm embarrassed to say I've been using AUC with ROC curves as the default - if this gives inferior model selection than AUC with PR curve I'll have to start doing that instead.

Thanks for the comments, this is a great summary. Curious what you'd think of a Kappa score given the imbalance?

https://en.wikipedia.org/wiki/Cohen%27s_kappa

>you can't use an ROC curve, sensitivity, or specificity. You need to use precision and recall and make a PR curve

But sensitivity and recall are the same thing...

There is nothing wrong with using ROC for imbalanced data. It is also perfectly reasonable to use an enriched dataset for a reader study, this is the standard practice.

It's almost as if publishing the thing was more important for the authors than the scientific value of the content.

First off, if I'm reading correctly, it outperformed on its test set. This is different as it doesn't get to see that at any point before it's final.

If the authors have done a diligent job here, that should be good evidence of it's accuracy. It's also encouraging to see they do multiple training runs, getting similar accuracy, and that their ROC is generally better than not just the average physician, but almost all.

Isn’t DeepMind about to release a medical product that will do something very similar to this? Right now I wouldn’t doubt how well these systems can perform as compared to trained specialists that rely on their eyes even for reading test results.

This is extremely relevant. Factors like whether the patient has freckles or red hair often drive doctors to resect anything out of an abundance of caution. It doesn't look like these additional demographic factors were made available to clinicians.

That said, the results here are still impressive.

Source: I worked in clinical melanoma research for 2 years.

Maybe? It's hard to say. I had a question about a mole once and what they did was take a photo of it with a special camera apparatus and they sent the image off to be diagnosed while I waited around. The doctor looked at it personally but it seems like the actual diagnosis was made by someone who never even saw me.

There are always potential issues when a machine learning algorithm is applied over time.

Example #1: Let's say that cancer rates are increasing over time and cameras are improving over time. You might end up with a weird artifact in your model that higher resolution images are more likely to indicate cancer.

Example #2: Let's say that cancer-detecting algorithms are widely successful and so someone makes an app that lets you upload images of skin and the app tells you the probability of you having cancer. Suddenly a model that was trained on suspicious lesions is being used on normal freckles that people uploaded for fun. You end up with a lot of false positives. Maybe you try to combat that by including images uploaded to the app (that you somehow obtain labels for). But now you have a model that predicts that photos taken in brightly lit medical offices are likely to be cancer and blurry images taken in bathroom mirrors are not cancer.

You could argue that Example #2 is more about the difference between training data and data to be scored, but the fact remains that outside of tightly controlled scenarios, the way data is collected nearly always changes in time and ends up affecting model performance in unexpected ways.

Yup, this thread has a nice overview of ways performance on a validation set can overestimate clinical performance:

https://twitter.com/IAmSamFin/status/1122271463170564100

Another example of change over time:

> One difficulty in such a comparison is that Gleason grading standards have shifted over time, so that scores below six are now rarely assigned, and assigning a higher grade has become more common

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3775342/

Stationarity isn't really the issue here, as you don't typically analyse this data as measurements from a single stochastic process.

However - you have hit on a very real problem. Imaging systems have got better over time, imaging quality even on the nominally same system can be different from different sites. Image coverage can change by both policy and system capabilities, etc.

It's worse the more sophisticated the imaging systems are. Consider MRI, which is perhaps better thought of as equipment to perform physics experiments than as an imaging device. In that case, nominally equivalent scans from different vendors (even different generation from the same vendor) can have significantly different characteristics. And there is a ton of processing going on, there is no such thing as "raw" data here - even the vendors themselves may no longer be able to really (or at least easily) characterize what is being done.

So yes, in any machine learning applied to these data sets, you have a very real risk of learning odd characteristics of the sample data and hurting your generalization.

Biology isn't as likely to be a problem I think, but biological response to changing treatment protocols, sure.

My guess is largely no. The diagnosis and actions taken are in isolated environments e.g, a cancer in person B is not impacted by a diagnosis and treatment in patient A. This is not the case in trading where everyone is analyzing and influencing a single messy environment.

Overfitting to people with certain complexions of healthy skin seems like a potentially much bigger problem than evolution of ailments.

This isn’t time series data so don’t need to worry about stationarity.

It's not clear that melanoma diagnosis is fairly described as a static image classification task.

Certainly it's a component, but (for example) patient history is, too.

Considering that the ground truth comes from humans I would say that humans always outperform NN's and that results which show otherwise are demonstrating the limitations of the data set or testing process.