Query Your Dataset with Natural Language Using JupySQL and Haystack Agents This tutorial and complementary scripts were …
Ploomber Joins Y Combinator
tl;dr; Y Combinator funded Ploomber, which guarantees long-term support for our open-source projects. We’ll start working on a platform for enterprises.
I’m proud to announce that Y Combinator, the premier technology startup accelerator globally, is funding Ploomber. This event represents an incredible community milestone; each community member has played an essential role in our journey. So, how did we get here?
Ido and I spent the last seven years working on data projects spanning academic research and model deployment at large enterprises. There is something bizarre about the Data Science practice: we use a tool for development (Jupyter, VSCode, PyCharm, RStudio), but deploy using an entirely new set of tools (Kubernetes, Airflow, etc.); so essentially, whenever a Data Scientist deploys their work, an excruciating refactoring process precedes.
I accepted the mantra for many years: Jupyter (and any other interactive environment by extension) is for prototyping only. Yet, after taking my previous job as a Data Scientist (and tasked with deploying a business-critical model), I challenged the idea: it didn’t feel right to move code around from notebooks to production frameworks. The refactoring process slowed me down, frustrated me, and put my projects at risk.
So I started playing around with some ideas, wrote a prototype, and used it for a project: it allowed me to work a lot faster. Finally, after a few extra weeks of work, I got the courage and open-sourced it. And a few months later, it was no longer a one-person show.
A few more people embraced this idea of bridging the gap between interactive data work and production. They were opening GitHub issues and sending me emails. At that point, I realized I had something valuable in my hands and decided to bring Ido, my long-term friend and one of the most capable people I had the privilege of knowing.
We’ll be honest, Ploomber had a rocky start: many dismissed the project because Jupyter is not for production. But those early adopters motivated us to keep going. Our community attracted more than a hundred like-minded people who believed in the mission in a matter of weeks.
Fast forward, it’s 2022, and Forbes 100 companies standardize their Machine Learning pipelines with Ploomber, startups ship their first models with Ploomber and universities run cluster workloads with Ploomber.
Ploomber is the most exciting project we’ve ever worked on.
The support from Y Combinator will allow us to materialize our vision fully, let us offer long-term support for our open-source software, and accelerate their development. Many teams rely on Ploomber to get their work done, and we want them to move faster. We’ll double down our efforts to work with our community members closely. We’re on a mission to help data teams develop and ship data products at lightning speed. ⚡️
Contents Introduction Securely storing and accessing credentials When to use Jupyter and JupySQL Magics When to use IDEs …