Item — HN Reader

When working with tabular data, there are very few situations where absolute model performance is the only criteria that's important. In practice, the following are equally as important:

- Explainability / debug-ability of models

- Effort to train, deploy, and manage NN models in production

- Capturing, collating, and organizing new & better datasets

- Local developer experience and human-model-iteration time

Building all of your software in C or Assembly will be faster and higher performant. But at what cost and with what tradeoffs? Building a website has a different set of tradeoffs than building a program for the Mars rover.

1 comments

It's funny; as a regular non-ML programmer, the optimum for every one of those factors for "tabular data" would seem to me, to be to "throw the tabular data into a relational data warehouse, and ask your questions in the form of SQL queries."

Or, if the "tabular data" is heavily relationship-based, then possibly replace "relational data warehouse" with "graph database", and "SQL queries" with whatever querying language that graph DB is natively / most expressively queried in.

Of course, this is the most important implicit "equally important" factor, one that an ML dev would think goes without mentioning: the generality or "power" of the model in what questions it can answer. You can only make these trade-offs in the context of knowing what kinds of questions you want your model to solve for! If all your questions are quantitative ones, maybe the right "model" for you is an RDBMS!

---

Though, that being said... why can't a deep-learning model emulate the thing that an RDBMS does, "at runtime", as part of its "mental toolkit" for approaching problems? That would be the best of both worlds, no?

I know that LLMs in particular have been observed to have "emergent numeracy" above a certain training-set size. There is a step function in how they approach such problems, going from their only being able to answer arithmetic questions on numbers of bounded size, and sometimes getting the answers wrong (probably this is due to a memorization-based approach); to being able to answer arbitrary arithmetic questions on operands of unbounded size, and always getting the answer correct.

I would guess that that what's happening, is that they are developing a functional component of their network that works akin to an Arithmetic Logic Unit, operating not on tokens, but on tokens transformed into a "numeric register" representation that is amenable to having math done to it with stable, quantized, position-independent results. (Just like the functional component that human brains develop after seeing enough math problems... probably.)

Do you, as an ML dev, think it would ever be possible for any of the model architectures we're familiar with today, to be trained such that they would develop an analogous emergent functional component for handling tabular-data questions, by transforming its internal working state into relational-DB/graph-DB data structures — e.g. page-heaps of binary-packed row-tuples; B-tree indices; etc — and then manipulating the working state in that form, using learned algorithms applicable to that type of data?

It seems to me (possibly just because I don't know any better) that just as with numeracy, "being able to put the data into a different and better internal representation" is what would be needed for deep-learning models to become truly good at dealing with tabular-data problems.

But, unlike with numeracy, "thinking as if you were a relational database" is not something a single human would ever intuit how to do without being taught. Relational algebra — and the data-structures and algorithms to make it practical to have a Turing machine do said relational algebra — wasn't even a single intuition, but a conscious effort, of multiple humans, working together over years. I strongly doubt that there's any number of "tabular-data problems" that you could show a human being, that would result in them developing an intuitional ability to do what a relational database does with its memory to efficiently answer queries.

(I suppose we could give an ML model an RDBMS, and hardwire it to interact with it. I know there are hybrid ML + formal-logic systems. Are there hybrid ML + data-warehouse systems? Not where the model queries an external DB — while that can be done, it'd be only in the same "stop and do this" way that ChatGPT runs Python code, which wouldn't make it a thinking tool the way that the formal-logic proof engines are for hybrid ML systems. Rather, I mean that some data-warehouse execution engine could be embedded into the ML execution framework itself, deployed as part of the GPU shader-program to each tensor core, such that data-warehouse operations can be done as a native part of the network's per-node instruction-set. Anyone ever tried this?)

> It's funny; as a regular non-ML programmer, the optimum for every one of those factors for "tabular data" would seem to me, to be to "throw the tabular data into a relational data warehouse, and ask your questions in the form of SQL queries."

It's doubly funny; as someone that comes from an ML background, and has developed and maintained multiple ML systems at multiple orgs, that I also think the answer very often is, "throw the tabular data into a relational data warehouse, and ask your questions in the form of SQL queries."

Most problems don’t need complex solutions.

I'll go one better. Many problems aren't worth solving.

>"throw the tabular data into a relational data warehouse, and ask your questions in the form of SQL queries."

You can ask SQL descriptive questions. Can you ask it for predictions? How?

https://www.red-gate.com/simple-talk/blogs/statistics-sql-si...

One of several examples of implementing linear regression in SQL.

This is called extrapolation and can be done with simple linear regression in some cases

>simple linear regression in some cases

You're correct, but "in some cases" is doing a lot of work here.

With the tooling where it's at, how much harder is it to apply xGBoost vs a linear model?

model.predict() is pretty easy to call, but judging the validity of the model is still hard and very manual. Linear model is less flexible and powerful, but easier to analyse/validate.

LLMs don't have any "mental model". They are just text completion with a larger memory. This works for text (unsurprisingly), but for nothing else.

This is pretty much what every isolated brain tissue does - predict the next.

Circular reasoning and begging the question. We don't actually know what brain tissue does and have no way to find out, currently.

Au contraire, there is plenty of findings as to what neurons and clumps of neurons do.

Begging the question again, as there is no evidence that "intelligence" runs on neurons. (And plenty of evidence that "intelligence" can exist without neurons.)

What evidence is there that intelligence can exist without neurons?

> ask your questions in the form of SQL queries

How do you know which questions to ask? This is what ML is good at, finding the right questions which classify the data.

You already made a faulty assumption — that we're interested in "classifying the data" in the first place.

Maybe we already know everything about the dataset. For example, if it's line-of-business customer data gradually built up by a sales team, then the brains of the salespeople have likely already done all the "implicit classification" needed to generate good questions about the dataset.

And this is, by far, the usual scenario for Business Intelligence questions: someone with "business-domain knowledge", e.g. an executive, has formed an intuitional hypothesis about the data based on their personal experience; and so they ask someone with "data-domain knowledge", e.g. a business analyst or data scientist, to test that hypothesis.

It's actually rare, in my experience, to have a tabular-data dataset that someone is motivated to understand, that doesn't also "come with" a set of people who can already act as (good!) models trained on that dataset, to aid them in that understanding. (Sometimes these people can't find each-other — but they do usually exist.)

AFAIK, having reams of entirely opaque and ill-understood tabular data, such that you need classification/clustering to get started on asking questions, only really happens in the sciences: sensor-network climate data; longitudinal-study medical-outcome data; census data; housing-market data; etc. In other words, it's almost always universities and governments — not businesses — that care about analyzing opaque tabular data.

And that's a key to understanding the constraints in play for choosing models! Because business-driven analyses are usually time-constrained in some way (potentially even needing post-training question-answers to be generated in soft-realtime); while institutional analyses usually aren't. Big difference!

I might be misunderstanding your point, but there's use cases that have repeatedly come up for me in multiple businesses, below being some examples, without getting too specific:

- identify latent features of customers via their behavioral data, to be used for profiling customers or recommending products to them

- within a large amount of customer behavioral data, identify potentially fraudulent behavior

- identify causes of seasonality (e.g. temporal patterns) in the data in order to improve forecasting (sales, traffic, whatever)

In those cases part of the investigation is to initially take a hands-off (unsupervised) approach, so that we can compare our initial top-down hypotheses with actual patterns in the data.

In both of those cases there's considerable (and sometimes adversarial) noise in the data.

Answers to these questions are actually Bayesian statistical models ("what is the probability of Y given a high likelihood of X"), treating these problems as unsupervised classification might work, but that's a very crude way of approaching them.

I wouldn't say it's a crude way of approaching the problem, it's a crude way of solving the problem. Taking the fraud example, taking unsupervised approaches to understanding patterns of the data before you impose assumptions on the data is a very useful process. For example, what might be fraudulent behaviors in the first place, assuming you aren't even sure you know what fraud looks like, or that it's actually all been detected? Your goal there might be to detect latent features period, not look at their predictive power for X.

Having understood that question, and built an understanding of what predicts fraud, you would then graduate to build models to understand the extent to which features predict fraudulence.

My point in context of the conversation is that it's useful in a business context to explore and understand that data.

I'm really not clear on why you're arguing against this. A proper data warehouse tackles the known unknowns, i.e. supervised learning. But you can glean new insights using unsupervised learning, like the textbook example of Target knowing a woman is pregnant based on sales data.

https://www.forbes.com/sites/kashmirhill/2012/02/16/how-targ...

>You already made a faulty assumption — that we're interested in "classifying the data" in the first place.

It's not clear what your point is. If you're not interested in the predictions that tree-based models provide, do not use tree-based models on your tabular data. A predictive model and a SQL query are not the same thing.

Data teams in companies often aim to enable the answering of future questions nobody has asked yet, by creating denormalizations of their data that offer maximum flexibility in what classes of questions they can answer. Maximum "power."

Lately, that means they're often spending a lot of resources (and even novel R&D time!) getting various kinds of ML models trained on the data.

My point is that this is often pointless, because, given the type of data they're working with (tabular, quantitative line-of-business data), they won't actually see "arbitrary questions"; they'll see the strict subset of arbitrary questions that could have been solved just as well — if not much better! — with a SQL query. And for much less capital expenditure — because the LOB data usually already lives in an RDBMS in the first place.

What? No! That's not how it works. That's not how anything -- including unsupervised techniques work!