▲ 65 points
back
69 comments
Second author here with a link to the arXiv paper:
https://arxiv.org/abs/2008.02637
Although I have to say, VentureBeat did much better than most media outlets I have seen writing about current research and what they write is not only accurate but also largely devoid of hype. Perhaps we actually managed to “keep the hype down” as we intended when writing this piece?
I will check in on this post now and then if you have questions and see if the first author is interested in joining when he wakes up as he really did all the legwork for this one.
Nice paper and important work.
Given that nearest-neighbor outperforms on closed book, is it reasonable to suspect the model is doing NN itself internally (which would explain the good performance on close duplicates?)
And if this is the case do you think training-time processing of data to attempt to move convert it to question/answer form data rather than raw QA would be a reasonable approach towards tackling this?
Hi, first author here.
> Given that nearest-neighbor outperforms on closed book, is it reasonable to suspect the model is doing NN itself internally (which would explain the good performance on close duplicates?)
I think this is definitely the case for the BART model. It is essentially acting as a QA-pair memorizer over the training data, and at test time, it just matches the question onto those seen at training time. Note that the T5-11B+SSM closed-book model was able to do a little better on NQ, so very large models with task-specific pretraining objectives do seem to do something slightly more interesting than just NN, but still really struggle in some settings.
> And if this is the case do you think training-time processing of data to attempt to move convert it to question/answer form data rather than raw QA would be a reasonable approach towards tackling this?
Great question! Converting sentences into a series of QA pairs is something we're really interested in. The T5-11B+SSM model we evaluate in the paper uses a special "Salient span masking" pretraining objective that does this to some extent (only mask words at pretraining time that are likely to be "answers" to factual questions), so in essence the pretraining task becomes pretty standard cloze-question answering, and they find that leads to better downstream results (https://arxiv.org/abs/2002.08910)
Thanks for the response.
> [BART] is essentially acting as a QA-pair memorizer over the training data, and at test time, it just matches the question onto those seen at training time.
Not super-surprising.
> The T5-11B+SSM model we evaluate in the paper uses a special "Salient span masking" pretraining objective that does this to some extent (only mask words at pretraining time that are likely to be "answers" to factual questions), so in essence the pretraining task becomes pretty standard cloze-question answering
This seems an obvious approach for cloze-type questions, but it seems non-obvious how to extent beyond this.
Are you aware of any work probing the differences in the representation using this style of masking vs a more normal language model objective? It would seem to me that this is the key to significant progress here (and of course one would speculate that a representation that works well for this would also work well for all kinds of KB-related tasks).
Thinking about this for a few minutes things like masking names, colors and numbers (the things that neural representations often confuse) and then asking questions based on them might be interesting. I wonder if bAbI could be extended for this?
Did you also try with "retrieval" models models such as RAG, REALM, etc ?
Hi, first author here. Yes, we evaluated 3 retrieval-based models, DPR, RAG and FID - check out the paper for the numbers (https://arxiv.org/pdf/2008.02637.pdf)
Judea Pearl has been bringing up the lack of causal knowledge in ML very often. He has even posted lots of interesting comments in Andrew Gelman's blog, e.g.: https://statmodeling.stat.columbia.edu/2009/07/05/disputes_a...
I tend to think that lots of solutions could come from topics like those discussed in this book, with a lot of further development: http://probmods.org/
Understanding of causality is very likely an emergent property. While extremely important, it's unlikely that we have some hard-coded low level architecture of causal inference in brains. It probably will just arise as a necessity of grounded understanding of the world.
I'm not exactly sure what you mean.
The conditions for causal inference being possible are pretty clear and have to do with the intentional modification of the local environment.
The intention to achieve some new environmental state, and your action to bring it about, is a dynamical activity that enables "deep" model building.
Causal inference is not going to be some "module" of the brain... it requires a body. When you place your hand on a hot surface, once, you immediately understand that it is hot. It does not require "induction" (as hume supposed). That is because our body identifies causes.
Its therefore pretty trivial to observe no NLP system understands language, or even can understand language, because it lacks this capacity to acquire language semantics via participation in environmental exploration. It has no body.
ie., you need to have experienced "on top", "green", etc. to know what "green leaves grow on top of trees" means. There is no meaning in the frequency co-incidence of symbols in text.
So no matter how much you are able to reproduce these patterns, they contain no content. The content is in the reader.
Your comment is interesting but seems to conflate two separate though related areas. The need for a body arises from the embodied cognition school of AI which suggests that intelligence is fundamentally embodied, hence the need for a robot equivalent to a body for truly understanding language.
However this does not necessarily have to be related to causality, and counterfactual statements about a causal model. The math behind counterfactuals and causality is actually well understood now (see any of Pearl's books). It does not actually require that a system be embodied, just that the system have some suitable (and correct) causal model of the world.
It would of course be amazing to have both in one system, but that is not required. An AI system that understood causality and language could be bootstrapped from causal models supplied by humans - or even other AIs :)
I agree that causal inference certainly requires ability to interact with the environment. In do-calculus this follows basically from the definition of the "do" operator.
As other people already mentioned, the "do" operator does not need to be related to the physical human scale environment. Causal inference could be useful also in interacting with the internet or other virtual environments like games.
Additionally, even humans do not have intuitive understanding of physical reality outside of our evolutionary environment. Nobody is able to intuitively understand quantum mechanics or general theory of relativity. Our intuitions of causality can beak even in relatively mundane environment like low Earth orbit. For example, you can fire thrusters to push you towards an object and still miss that object. There are actual missions in LEO that failed do to this type of mistakes of elite test pilots employed by NASA.
Of course, you can use mathematical formalism to reason about unfamiliar environments, but no matter how much time you spend learning about multidimensional spaces you will not be able to imagine 4D space or a quantum wave. But nobody is claiming that we do not understand language like "spin of an electron" because we have never experienced what a spin of an electron is.
Now to get to my original point. My original point was that causal inference is not a basic building block of intelligence. Formalism like do-calculus or logic are too brittle to be a building block of intelligence. You need something that is robust to noise. Something that can consume a tensor of pixels and process it. Once you have a system that is able to deal with this type of inputs then you can strive to do something like do-calculus or logic on top of it. But my argument goes further. My guess is that ability to do formal reasoning will emerge from the need to carry out complex tasks. Nobody will intentionally program models to do logic or do-calculus. Ability to do it will arise in models as a combination of the necessity of solving complex tasks and clever training techniques as it did trough evolutionary process in humans.
PS.
> So no matter how much you are able to reproduce these patterns, they contain no content. The content is in the reader.
To me this type of assertions do not contain any content.
"The content is in the reader." I don't understand that at all. Maybe I'm just a Chinese room :) .
I disagree. There are other ways to observe and learn cause and effect than by physical interaction. We learn new causes and effect of many things each day by reading or watching. There are also most likely examples of humans who never had control over their physical world (through disability or whatnot) that came to understand cause and effect.
Yes, and text can be written in non-causal ways, which means even if you parse things in order, you can easily end up with a non-causal understanding.
“I just saw a huge fish, but then I realized I was out of worms. I should’ve bought more, but I spent $20 on gas instead.”
Here the causal order goes from last to first (minus the first clause)
This sounds like a possible analog to someone who is intelligent but doesn't really understand math or science. This person might be able to BS their way through certain fields, but could never pass a calculus exam.
It seems like there are not many common-knowledge questions that you can't find answers to on the Internet? Most of us rely on web search these days.
Also, making up answers isn't necessarily a good skill to train for if accuracy is needed. It would be more useful to quote and cite the source.
I'm always amazed that some people think performing statistical analysis (training neural networks) on text can lead to actual intelligence and knowledge about the world, if only we increase the number of model parameters by a few more orders of magnitude (GPT-3 and the likely strategy for GPT-4).
Of course, all this without any model of how likely it is that the knowledge is embedded in the text, such as trying to train simple models on descriptions of a simple world. I think it's very likely a model couldn't even learn simple arithmetic (addition, subtraction multiplication, division on the rational numbers, let's say) given all of human writing on arithmetic, nevermind being able to reason about the entire world.
It amazes me people think their brain works differently.
It's pretty close to believing in body/mind dualism, the only thing in neuroscience more outdated than Freud.
Your brain works within the same laws of physics as the outside world. We don't know how the brain works exactly. But once we understand it, it is unlikely to be qualitatively different from a neural network.
On the other side of the equation, the emergent behaviour of deep networks make them fundamentally different from the sort of statistics that came before. So even if the building blocks are old and boring, the larger system isn't.
> it is unlikely to be qualitatively different from a neural network
I'm sorry but this shows a profound misunderstanding of what a NN is, and what the brain is.
There are no "neural network"s. The NN algorithm is a method for optimizing the parameters of a piece-wise linear regression model.
These regression models have no homology to any brain structure and the process of producing them ("training") has no neurological analogue either. They can be produced with a variety of algorithms.
Here's one: f(x) = max(0, 2max(0, 3x - 0.1) - 0.5)
The phrase "neural network" is, as peddled by the media and poorly informed lecturers, a lie. It is neither neural nor a network. It's just gradient descent with more parameters.
A model of the brain would model, at least: neuroplasticity, biochemical signalling, activation frequency, etc.
There is nothing about a peice-wise linear regression model which does any of that. No matter how many circles and lines you draw. (NB. essentially any mathematical function can be drawn as a "neural network". The "network" is just a way of diagramming function application & dot-products).
Aside from all that, the body-brain system of animals is a physical process whose properties are not abstract. The reason we are intelligent is because we have bodies capable of causal analysis; and that capability is biophysical.
To put it another way, no algorithm which runs on a digital computer will turn it into gold. Not even one called, "the midas algorithm".
Causal. Imagine a video. Each frame is plotted in space, so instead of a sequence of 2d images, you have a cube. The z axis of the cube shows each frame of the video in turn. This is an equivalent representation to the representation that we are used to. We have just switched the time dimension for a physical dimension. What you are reverently calling causality, is only correlation along the z spacial dimension. In the same way an image can be compressed due to the similarities across it's 2 dimensions, so can a 3d representation of video be compressed along all 3 of it's axis. An appreciation of causality is only induction and induction is only a correlation, if a then probably b. Finding correlations in one of the dimensions you are more familiar with, like the pixels of an image is no different to finding it in time.
Within language the elements of time are encoded, just as they are within the video represented as a cube. "The boy threw the ball, the ball landed and then rolled."
Just because the time dimension is represented to the neural network within the input parameters of a single iteration of the neural network, does not make it any less able to understand correlations across time.
Very good points! I would add that we don't yet have a complete model of the computational properties of even a non-neural cell, even though we know that all cells exhibit simplistic perception and decision-making.
This is not to say that we should expect some non-turing weirdness or anything 'mystical' like that, but that we just haven't taken the time to analyze the exact mechanisms and to model cells at this level well enough to be able to say that they are similar to any particular computational structure. Thus, to claim that the brain in its entirety is similar to any particular computational structure such as an NN is either wrong or trivial - the only level at which 'similarity' can be claimed is that they are both useful in computation, and likely turing complete. Anything more specific is fanciful thinking.
It's amazing just how many people believe that neural networks have a close resemblance to the human brain.
As another poster wrote, the main problem is not the neural network itself, it is the training set, and very likely the training algorithm as well.
My claim is that working knowledge of the world is simply not encoded in human text. All human text (and speech) presupposes a certain model of the world that is learned from experiencing the world, likely based on a model that itself is partially pre-trained by evolutionary processes.
If a text corpus correlates with human knowledge (your "presupposes a certain model of the world"), then it necessarily encodes the portions of the knowledge it correlates with. That is, correlations drive the search of model parameters that explain the correlations. The stronger or more numerous the correlations, the more accurate a model can be computed. But the search space informed by this network of correlations is much more restricted than a naive brute force search, thus the correlations provide information about, i.e. encodes, the correct model.
What we need here, I think, is a GAN-like approach. Use a generator + a discriminator. The generator could be like GPT-3, but it needs another model to filter out the nonsense. We work the same way - come up with crazy ideas, then check them out to see if they hold ... 'I had an idea, but nah, it wouldn't work'. It's a process supported by different skills.
The issue isn't whether neural networks work, the issue is the dataset. Look again at what the parent wrote:
> I'm always amazed that some people think performing statistical analysis (training neural networks) on text can lead to actual intelligence and knowledge about the world
Emphasis mine. The human brain does not develop what we know as human intelligence by analyzing text.
I don't see why you're "amazed". Neural networks are _provably_ an universal approximator. As such, they can approximate _any_ function to an arbitrary degree of precision. It just might require an impractically large/complex neural net to do so, and you might not be able to train it yet, but it's _provably_ doable in the limit, if you remove size/training constraints.
I'm a practitioner in the field, and an AI skeptic because of it. I think perceptual tasks work reasonably well, NLP barely works for anything, and any sort of cognition doesn't work at all, as of August 2020. I will be the first to admit that we don't have real "AI" yet, and we won't have it in the foreseeable future. But to say that real intelligence is not possible with the current tool set is simply incorrect. It may be impractical. It may take 100+ years to figure out how to build nets this large and train them successfully before sun burns out, but it is _theoretically_ provably doable.
In a way it's kind of like fusion: you have a proof that it can work right above your head during the day. We just don't know how to do it at our "human" scale with our knowledge and capabilities. Our knowledge and capabilities, however, are not a constant - they improve exponentially over time, although most people don't perceive this process as exponential.
For clarity, Universal approximation doesn't mean ANNs can approximate just absolutely any function...
Depending on which proof you're looking at, ANNs can provably approximate any continuous function, or any Lebesgue integrable function, or any real vector-valued continuous function on a compact subset of R^n space [1]. That's certainly a very large class of functions, but it's not just any function.
Also, those proofs only work if you allow ANNs with arbitrary number of layers or layer width. Perhaps the required ANN size to approximate the AI / AGI function is so huge that it's beyond the number of particles on Earth. At what point is it beyond impractical? What if the ANN size (number of layers or neurons) is beyond the number of particles in the galaxy? Maybe it's just the wrong approach.
I haven't studied those proofs in detail, but as existence proofs, they just show that some ANN exists that can approximate a given function (of a certain class). There's no guarantee that there's a general algorithm that can learn the required ANN, is there? Maybe you just have to be really lucky to randomly initialize onto the correct approximating ANN...
Also, I'm not aware of any proof that human or bio NNs implement a Lebesgue integrable function, or etc. It's entirely possible bio NNs uses some weird (quantum?) physical process, and implements a function outside the scope of any ANN universal approximation theorem.
I'm just a little more cautious around those universal approximation theorems. I'm more or less in agreement with your comment otherwise!
[1]: https://en.wikipedia.org/wiki/Universal_approximation_theore...
> Perhaps the required ANN size to approximate the AI / AGI function is so huge that it's beyond the number of particles on Earth
Perhaps there's a teapot orbiting Mars. It's unlikely tho. I think a better hypothesis would be that to model a brain (if we knew how) one would need the same order of magnitude of neurons and connections. This is completely unachievable now, and even if it were technically possible nobody has a faintest clue about how to train such a thing. We do not understand _many_ things about how the brain learns (no backprop!), how it retains and retrieves memories, etc, etc. The best studied system is vision, and computer vision does actually work pretty well by now, achieving superhuman results on some tasks. Thus far the most complex brain we've been able to accurately and completely model is not even a brain at all - it's just a handful of neurons of a worm. I took an online course in neuroscience a few years ago hoping to gain some insight for my ANN work. The situation with our understanding of even primitive brains of i.e. rodents is, well, primitive. We also don't really have math to accurately model more than a handful of real, biological spiking neurons - PDEs get way, way too complex for anybody to handle. But one could argue we don't have to: ANNs effectively operate in the frequency domain instead, where magnitude of activation approximates spike train frequency, and timing effects are ignored.
Don't get me wrong, I'm pretty sure if we do achieve real AGI, it likely won't be with anything even remotely like the current tech and science stack. Energy efficiency and compute density is just not sufficient. But I'm also pretty confident that even with the current stack we will be able to, eventually, approximate intelligence closely enough to suit many practical uses that seem completely out of reach today, including even some limited degree of logical inference, and dare I say, cognition.
If only people (including myself) weren't so chickenshit and actually seriously worked on (and funded) this stuff, that could even happen in my lifetime. As things stand right now, people are afraid to touch anything AGI related due mostly to fear of ridicule.
In my comment above I am merely referring to the conjecture that _if_ we had ANNs that are large enough, and _if_ we knew how to structure and train them, there doesn't seem to be anything that would _fundamentally_ prevent them from supporting AGI. Those are very, very massive "ifs" indeed, without even a faintest glimmer of light at the end of the tunnel.
I'm not claiming that an NN can't approximate human or super human intelligence, I'm claiming that using human writing as the only training set is not going to get you there.
The text simply doesn't contain enough information to draw a model of the world. Human language only makes sense if you understand the world, not the other way around.
Essentially what we're doing is similar to a human trying to learn psychology by reading books in Mandarin on it, while knowing neither the writing system nor the language. Except that human languages share some universal grammar, and share more or less the same notions about the world.
The parameter/dataset arms race producing emergent intelligence rather than just bigger models is a fantasy.
However without a good/better explanation in the meantime , really interesting capabilities emerge at higher parameters
Specifically to your point of arithmetic— here is GPT3 performing “zero shot” (ie no examples) arithmetic:
https://photos.app.goo.gl/pf6fHcrio7QFtU4S8
The “completion” includes an answer to the problem but also provides a natural language summary (GPT3)
GPT3 has ~170billion parameters (GPT2 ~1.5B) & somehow is capable of many-digit arithmetic w/o fine-tuning or task-specific training. (Its training data was “messy” from Common Crawl like for search engines)
Microsoft’s Turing-NLG has ~17billion parameters & was successful only in 1 of every 2 attempts of 2-digit arithmetic and only successful less than 10% of the time for higher numbers of digits.
That's pretty impressive.
Google's Pegusus summarizer learnt to count to 5: https://ai.googleblog.com/2020/06/pegasus-state-of-art-model...
It does lead to intelligence and knowledge of the world. It leads to somewhat similar understanding of the world as our understanding of 4D, quantum mechanics or environment close to a black hole. Pretty bad one, mostly based on repeating whatever we have read about in popular science literature without any intuitive understanding. GPT-3 understanding of physical world is similarly bad. It's just simply a modality not available to the model.
But GPT-3 is able to do few-shots learning. If you show it couple of times how to do something it will try to repeat after you showing some rudimentary understanding of what you are trying to do. It's even able to learn to do simple analogies [0].
GPT-3 is able to do very, very simple arthritics like addition and subtractions with 2-3 numbers. Actually even a lot smaller models trained on arthritics are able to do impressive feats like symbolic integration and solving differential equations.
> Deep Learning for Symbolic Mathematics [1]: Neural networks have a reputation for being better at solving statistical or approximate problems than at performing calculations or working with symbolic data. In this paper, we show that they can be surprisingly good at more elaborated tasks in mathematics, such as symbolic integration and solving differential equations. We propose a syntax for representing mathematical problems, and methods for generating large datasets that can be used to train sequence-to-sequence models. We achieve results that outperform commercial Computer Algebra Systems such as Matlab or Mathematica.
What is true is that GPT-3 is not very good at following instructions. It would not be able to execute an algorithm. So, it would not learn arithmetic from instructions if you would not include explicit examples.
[0] https://medium.com/@melaniemitchell.me/can-gpt-3-make-analog... [1] https://arxiv.org/abs/1912.01412
So you’re saying ai needs to be grounded to the real world to be effective?
Yes. Specifically the way human brain interacts with environment, the same way any AGI will need to sense and react to an environment, the more real the better.
An interesting example would be if the NN learns something from a book, also fed it with the all the critique on that book ( the environment reaction ). So it can possibly learn from a human what is good or bad in that book, etc.
Then both AI and humans can speak the same language ( on terms.of "feeling"/sensing ) and eventually learn the proper context, so less parameters are needed.
It does lead to intelligence and knowledge of the world. It leads to somewhat similar understanding of the world as our understanding of 4D, quantum mechanics or environment close to a black hole. Pretty bad one [...] GPT-3 understanding of physical world is similarly bad.
No, GPT-3 has no "understanding" whatsoever - we at least have a conscious engagement with the topics you cite and a genuine understanding that we lack a directly relatable perspective.
Our understanding in many circumstances - like those you mention, may be severely constrained, but it exists as part of a sentience that there is no reason to believe GPT-3 or any other current AI tool - possesses.
Do we really understand? What's the mechanism for that?
My thinking is that we humans have some ability to do things by rote until we have a good statistical understanding, at which point we use the statistical understanding mostly, and fall back to rote where we have to be careful. But, mostly, we're stats machines too.
It shouldn't be that hard to imagine. If the objective is to predict human text, at some point the best way to predict text is to capture the kinds of knowledge that humans know and use in text generation. Human knowledge is a strong prior for human text generation, and so it shouldn't be surprising that for a large model its parameters would end up in a portion of parameter space that encodes some human knowledge.
By this token, the best way to learn Mandarin would be to start looking at text in Mandarin and trying to assign it meaning. While it theoretically can work, it is vastly more difficult than trying to relate it to a previously learned model. In fact, it is so difficult that we have never successfully understood a lost human language this way (though with the caveat that we don't have the huge corpus of text that GPT-3 was trained on in any of the lost languages we tried it on).
No, I said the best way to learn to predict human text is to capture human knowledge. I did not say how best to capture human knowledge. Obviously doing gradient descent over a huge text corpus is massively inefficient for a human. Whether its the best way for a machine to learn it is another matter. After all, the amount of data that humans experience through their senses during formative years, as well as the "computational work" that went into evolving human brain architectures surely dwarfs GPT-3's training regime.
It is very true that GPT-XYZ is fundamentally limited, like all transformer models, but don't misattribute the root cause to the fact that it's just processing text.
The real limitation is that these are feedforward networks that just do perception without any processing of what they've perceived. You can try to hide that fact for a while by increasing the depth of the perception network, basically hard-coding some processing into the single pass, but you're still not capturing any of the "absolutely requires self-feedback" behavior that we care about that takes a human more than a split second to do (aka almost all actual thought).
A statistical model that made good use of what was coming in could absolutely learn (to take your example) math at every level based on nothing but text, transformers are just nowhere close to having that capability because of their design limitations.
But the fact that it 'lives' in a world of text is much more fundamental. There is simply not enough information in text to draw a working model of the world. We presuppose such a model in our words and in our communication, so fundamentally any algorithm that is solely trained on text can't learn a model of the world from it.
It seems they are on the right direction. But, the number of parameters hit the physical barrier of silicon-based device.
To make the neuron network be a strong AI, we may need the parameters in scales of 10^19~10^44 (uncertainty limit and Planck time) estimated from how many frames per second in real world.
I personally got a feeling that this is bound to how nature calculate the physical phenomenon. For simulation, there are lots of "for loop" in our code/machine. But, in real world, we could not observe any "for loop". It is fast and continuous in the macro scale.
In order to reach strong AI, it is time to focus on quantum computing.
> To make the neuron network be a strong AI, we may need the parameters in scales of 10^19~10^44 (uncertainty limit and Planck time) estimated from how many frames per second in real world.
There is absolutely no basis for claiming this. It might turn out that there are better algorithms for AI than neural nets. Do you even realize just how many parameters even 10^12 is???
For loops in AI are actually only there to implement continuous mathematical operations such as matrix multiplications in a single core. Modern AI chips and implementations use very little to no for loops.
That said the interesting thing about language AIs is that they are by design able to communicate, which leads to interesting results, even though it is increasingly obvious that they don't really have any 'awareness' as such.
The article doesn't give examples, but the paper does. Here are a few examples of question overlap, for context.
Test Question: who plays max voice in a goofy movie
Train Question: who does max voice in a goofy movie
Answer: Jason Marsden
Test Question: when will the 2018 oscar nominations be announced
Train Question: when are the oscar nominations for 2018 announced
Answer: January 23 2018
Test Question: who has scored more goals in the premier league
Train Question: most goals scored by a premier league player
Answer: Alan Shearer
Test Question: where are the cones in the eye located
Train Question: where are cone cells located in the eye
Answer: retina
Test Question: who led the conquest of the incas in south america
Train Question: conquistador who defeated the incan empire in peru
Answer: francisco pizarro
The paper also considers answer overlap; that is, when the answer to the question also occurs in the training set. This alone does not imply memorization, but it does open the door for some shortcuts. Some examples from the paper are:
Open Natural Questions
Duplicated: Phil Simms, Brian Johnson, 8, the Indians, the 1830s
Unique: Cloves, Matt Monro, 1,020 – 1,080 kg, Hermann Ebbinghaus, Matt Flinders
TriviaQA
Duplicated: David Bowie, Battle of camlann, Heligoland, Henry VII, Niagra Falls
Unique: Death in the afternoon, Clash of the Titans, ice-cream sundae, Camshaft, Cumberland
WebQuestions
Duplicated: Harvard, Alderaan, India, 2011, Zeus
Unique: Queen Victoria, Braslia, Paddington, Tom Corbett, Gary
It should be noted that a lot of the discussions about causal knowledge and intelligence aren't too relevant to these questions, as they are largely tests of memory or retrieval. It is expected that these models answer the questions by searching their training data or their document index. The issue is that their training data isn't meant to contain copies of the test questions, just the information sufficient to answer them.
Hi, First author here.
Thanks for the comment and for adding examples, and for your nuanced comments of the answer overlap split.
My position is that these datasets are still useful for QA, but what was lacking was an analysis of how easy/hard the questions in them were, and what kind of modelling was needed to do well. These overlap phenomena are less like "bugs" maybe, but more like poorly understood features.
We need models that can accurately recall QA pairs they have seen before, so being able to score well on "memorizable" QA pairs is still important to do well, but we also want models that can do more than that. One single accuracy number on a leaderboard cannot capture all the behavioural information we need to properly understand the capabilities of these models.
That's useful for chatbots, where you have a big FAQ and people ask questions. That's exactly what RASA does, using TensorFlow. It's reasonably good at matching user questions with stored questions. But that's all the AI part does.