I have often seen people hold the assumption that you have a strong opinion on exactly what the mental difference are between male, female, etc with out good research to back it up then it is likely due to sexism.
What percentage of people are jumping to conclusions arbitrarily rather then allowing their conclusions to biased for their own ends. Most seem to assume the latter is more likely.
The arguments you're making perpetuate structural oppression of women. One might as well talk about how race differences exist and on average different ethnicities are better at different tasks. (That is to say, I find the language you're using morally repugnant.)
"Simply noting a difference" is never simple when dealing in stereotypes driven by historical prejudice.