How’s the Data Science Tech Stack Evolved?
4 min readSep 1, 2022
- Have you felt a bit lost when you are trying to figure out how to set up an endpoint for your model?
- Have you dealt with kubernetes APIs, or a container orchestration system, or the concepts of those?
Are you worried where data science is going?
You are not alone. 😉
This is us today👇, or soon-to-be tomorrow
A data scientist swamped by the tools surrounding
As our data size, model size and number of prototypes grow, we are slowly and inevitably stepping away from the data science work and entering into the realm of engineering.
A data scientist’s narrative:
What are the problems we are facing here?
- “My workflow relies on the robustness of the supporting systems such as the feature store, how should I think about orchestrating my workflow?”
- “Do I need to rewrite my pipeline to run asynchronously with the modern workflow orchestrators, or write docker files and understand what different package managers are there?”