Recasting the role of the data scientist

Maybe I have achieved a one-day mind-meld with Azeem Azar, because he also noticed an interesting and timely research paper: What Should Data Science Education Do with Large Language Models?

I say timely because of OpenAI’s release code interpreter plugin for ChatGPT becoming widely available.

Sidenote 1: they’ll likely they’ll follow up with the equivalent functionality (probably superior, per Altman’s mission) via the API. But who knows when.

Sidenote 2: the paper’s title almost feels like an article you would see in a tech blog, not an aggregator of academic research papers. Is Arxiv.org the new Hacker News?

But it’s a much more thorough analysis than your average blog post, though, with coverage of every area in which LLMs are expected to impact data science, from education to professional practice . As to the latter, page 4 contains the passage that probably best sums up the paper:

“LLMs have the potential to revolutionize the data science pipeline by simplifying complex processes, automating code generation, and redefining the roles of data scientists. With the assistance of LLMs, data scientists can shift their focus towards higher-level tasks, such as designing questions and managing projects, effectively transitioning into roles similar to product managers.“

I don’t doubt this – one example of many of the opportunity for the laptop worker to subcontract the dirty work to AI, then align their daily work more closely to designing and delivering solutions.