back

by dijksterhuis·6y ago·view on hn ↗
Some of what you're describing in 1-4 is Data Engineering. 5-6 exists (in some form) for most software jobs.

The general breakdown I give people is:

Data Scientists:

* Get data.

* Clean data (~60% - 70% of time required).

* Research.

* Low level data analysis.

* Building models.

It's mostly the "knowing data" and the modelling.

Data Engineers:

* Data storage

* Data processing

* Automation

* Infrastructure

It's about getting the Data Scientist's output into production / making data easily available to them.

This is especially true for big ETL jobs. The more we can automate your ETL jobs, the happier you'll be!