alt Hacker NewsIt's frustrating how the data discipline is split up into so many different job titles. I regularly have to work with data engineers, data analysts, analytics engineers, data scientists, machine learning scientists, and machine learning engineers. Each view their role really narrowly, so you need to pull in a lot of people to get something done. It's often easier to just do the work yourself.
I get that some specialisation is useful, but I would like to see more T-shaped data folks.
Replies
It used to be the case that a "data engineer" was just a software engineer that was good at data-related things (maybe they were good at Kafka, airflow, DBT, knew why you'd pick postgres for one job and Clickhouse for another, etc).
Then the BI people (who had never written a line of code except maybe some SQL) got jealous, started calling themselves "data engineers", broke everything ("what do you mean 'we need tests???'"), and ruined the title.
My favorite title is 'digital prostitute' - we do anything if you pay us money :D
Yeah, as a "T-shaded data folk" (have worked as a data analyst, data scientist, data engineer, and machine learning engineer, so I think I'm close to a full bingo care) in data, I agree!
I think the specialisms are really helpful for hiring - data engineering and data science involve a lot of different things so it's a lot easier to hire someone if you don't require skill in both.
That said, I think we have a culture problem in defining our roles narrowly, I've seen data scientists look down at data-pipeline parts of their job, and ignoring model latency and deployment; data engineers be deeply uncurious around what happens to the data after they put it in a warehouse; etc.
The best people I've worked with, within data and elsewhere, are the ones who are curious about every aspect of the stack, even if they mostly focus on one side.