moultano

Sure, but mixture models and mutually exclusive events are less common to work with than just "models." Obviously there are times when you add probabilities together, but usually you don…

That's true. I left out saying that deeper networks represent a wider variety of functions more easily, because that seems generally intuitive to everyone. But the arguments about how easy it is …

Let us know when polynomial regression succeeds at any machine learning task. A lot of people publish results that say deep learning is "just" something else, where the something else doesn&…

I'm curious which category you put this one in. :D

People coming from the perspectives of general optimization believed that they were impossible to train, due to being very non-convex. People coming from the perspective of classical statistics believ…

Yeah I was thinking about that as I was writing and trying to convey why I feel like deep models are different. I think one way of thinking about it is that protein structure, even though it has lots …

With squared loss where it's easy for the loss to be zero, then yes, it will have lots of global minima, all at a loss of zero. For losses that asympotote, like logloss, they may have no minima.

If that worked, people would use it, but it doesn't, so they don't.

Thank you. Several people have pointed that out, and I'm probably not reading the right papers. Is it common when people introduce a new flavor of adaptive SGD to address how it handles saddles s…

This isn't true. A loss function that asymptotes can also have no minima, which commonly used loss functions do.

That isn't what I'm arguing at all.

Think of the model as a point in high d space that you're trying to trap inside a cage. (Corresponding to it being at a local minima and surrounded by higher points.) So you're trying to bui…

I know a lot of folks that have rented victorian living rooms, sometimes even with a curtain to divide it in two. There just isn't that much else.

That's actually what the title is, mostly I just wanted to write about clams and crabs. The person who submitted it changed the title in a way that got more attention than the original that I sub…

People still eat them, just illegally. The rangers have to bust people for it all the time. https://amp.sanluisobispo.com/news/local/environment/article... …

I'd bet that a shallow transformer that was distilled on gpt2, possibly with quantized weights, would easily win. GPT2 isn't optimized for size at all, but there are tons of ways to do so.

Given the replication crisis, these are the kind of results that I reflexively don't believe anymore. Has anyone read any of the studies? What are n and p for these results? What's the effec…

It was obviously not the status quo, because they were banned from the country. Companies should advocate for the rights of their employees. They should demand their employees treat each other with re…

Having the time and energy to devote to it is not the same thing as whether it materially changes your life. Often those things are anticorrelated.

An Iranian immigrant can't help but advocate for their political views simply by existing. Living and working in the US is an assertion that they should have the legal right to do so. Simply by…

In January 2017, several of my coworkers were banned from reentering the country to go home to their families, while they were on business trips to visit other offices. How is anyone supposed to be ap…

It's kinda crazy that this is the format we've standardized on to carry all of the output of academia into the future.

Betteridge's law of headlines seems to apply. It's a mistake to assume that public policies, such as lockdowns, are the primary thing that changes human behavior, when mobility data has show…