Notebooks are a magnificent tool to explore data, but such a powerful tool can become hard to manage quickly. Ironically, the ability to interact with our data rapidly (modify code cells, run, and repeat) is the exact reason why a notebook may become an obscure entanglement of variables that are hard to understand, even to the notebook’s author. But it doesn’t have to be that way. This post summarizes my learnings over the past few years on writing clean notebooks.

Putting notebooks and production in the same sentence is a sure way to trigger a heated online debate. This topic comes up frequently, and it’s common to hear that teams completely dismiss notebooks because they’re not for production. Unfortunately, the discussion often centers on problems already fixed by current tools, but the solutions are still unknown for many practitioners. This post shows how to overcome some of Jupyter notebook’s limitations, proposes a workflow to streamline their use in production, and discusses problems that remain to be solved.

Given the increase in popularity of technologies such as Snowflake, SQL is more prevalent than ever. That’s excellent news; SQL makes data manipulation a lot easier. Still, a lot of practitioners seem to perceive it as an outdated language. This blog post motivates SQL as the language for data science and provides some practical tips for using it effectively. Why use SQL over Python/R? SQL is a declarative language. Instead of writing a recipe with exact steps (like we do in imperative languages such as Python or R), we describe what we want to accomplish.