Item — HN Reader

I worked in that field a number of years. My recommendation to you is to start with some form of data that you are passionate about. Baseball statistics, business metrics, investment figures, whatever.

Once you have the data, then figure what you're going to do with it. (Don't agonize over it, this should all take just a day or so.)

Then go after the toolkit. You'll find many interesting questions if you start with the end goal in mind.

Good luck, and have fun!

3 comments

This is the best advice I've seen here. The purpose of data engineering is to support a much larger effort related to scientific discovery or increasing bottom line revenue. Don't start with the language, tooling, libs and architecture, start with a purpose and goal. Most goals relate to mimicking a portion of human cognition in terms of pattern matching.

If you're building a house you don't start by picking out hammers and saws.

This takes being passionate about your data and its pre and postprocessing as a data engineer.

quick question: is there a clearing house of this data? I've been wanting to get at football, hockey or soccer data but finding a data source has been daunting.

I could scrape pages, but don't really feel that is the best way to go about it.

Thanks for the advice !