Skip to content

09 · Jupyter Notebook Best Practices

Notebooks are great for exploration and terrible for reproducibility if you don't discipline your workflow. This module covers the habits that keep a notebook trustworthy and shareable, not just "worked once on my machine."

Restart and run all, always

The single most valuable habit: before trusting a notebook's output (and definitely before sharing it), use Kernel → Restart & Run All. Cells run out of order all the time during exploration — you delete a cell that defined a variable another cell still depends on, or you rerun cell 12 after editing cell 3. The kernel's in-memory state can silently diverge from what a fresh top-to-bottom execution would produce.

# Anti-pattern: this works right now because you ran an earlier cell
# out of order, but a fresh "Restart & Run All" would raise NameError.
adjusted = raw_value * scaling_factor   # scaling_factor defined three cells below

If "Restart & Run All" fails or gives different numbers than what's currently displayed, the notebook is lying about its own results.

One cell, one idea

# Bad: five unrelated things crammed into one cell
import pandas as pd
df = pd.read_csv("sales.csv")
df["date"] = pd.to_datetime(df["date"])
df = df.dropna(subset=["amount"])
summary = df.groupby("region")["amount"].sum()
print(summary)

# Better: each cell is independently re-runnable and debuggable
df = pd.read_csv("sales.csv")

df["date"] = pd.to_datetime(df["date"])
df = df.dropna(subset=["amount"])

summary = df.groupby("region")["amount"].sum()
summary

Small cells make it obvious where something broke, let you inspect intermediate state without re-running everything, and produce a cleaner diff when the notebook is version-controlled.

Keep heavy logic in .py files, not cells

# analysis/cleaning.py
import pandas as pd

def load_clean_sales(path: str) -> pd.DataFrame:
    df = pd.read_csv(path)
    df["date"] = pd.to_datetime(df["date"])
    return df.dropna(subset=["amount"])

# In the notebook:
from analysis.cleaning import load_clean_sales

df = load_clean_sales("sales.csv")
df.head()

Functions in a .py module are testable with pytest, importable from multiple notebooks without copy-paste, and reviewable in a normal code diff — notebook JSON diffs are close to unreadable. A good rule: if a cell has more than ~15 lines of logic (not display code), it probably belongs in a module.

Pin your environment

# requirements.txt or environment.yml — commit this alongside the notebook
pandas==2.2.2
numpy==1.26.4
scikit-learn==1.4.2
matplotlib==3.8.4

"It works on my machine" almost always means an unpinned dependency changed behavior. Recreate environments with pip install -r requirements.txt (or conda env create -f environment.yml) so a colleague — or you, in six months — gets the same results.

Clear outputs before committing (usually)

jupyter nbconvert --clear-output --inplace notebook.ipynb

Committing a notebook with large embedded outputs (plots as base64 images, long DataFrame dumps) bloats the git history and makes diffs unreadable. The common exception: a final, presentation-ready notebook meant to be viewed directly on GitHub — there, keep outputs so reviewers don't need to re-run anything.

Seed randomness and record versions

import random, numpy as np

SEED = 42
random.seed(SEED)
np.random.seed(SEED)

print(pd.__version__, np.__version__)

Any notebook using np.random, train_test_split, or a stochastic model should fix a seed — otherwise "I got a different number this time" becomes a recurring, unresolvable question. Printing library versions at the top of the notebook (or better, in the pinned requirements file) closes the loop on "what exactly produced this output."

A minimal notebook checklist

  • [ ] Restart & Run All succeeds top to bottom with no errors
  • [ ] Random seeds are set wherever randomness is used
  • [ ] Heavy logic lives in importable functions, not sprawling cells
  • [ ] Dependencies are pinned in a committed requirements file
  • [ ] Outputs are cleared before committing (unless it's a final report)
  • [ ] The first markdown cell states the notebook's purpose and data source

Cheat sheet

Practice Why
Restart & Run All before trusting results Catches out-of-order execution bugs
One idea per cell Isolates failures, cleaner diffs
Move logic to .py modules Testable, reusable, reviewable
Pin dependencies Reproducible environment
Clear outputs before commit Small, readable git history
Seed randomness Reproducible numbers

How It Actually Works

The root cause every practice on this page defends against is the same mechanical fact: a Jupyter notebook's kernel state is independent of cell order on the page. Each cell execution mutates one shared, persistent Python namespace, and the kernel only remembers execution order (the [N] number next to each cell), not visual order. If you define x = 5 in cell 10, then go back and edit cell 3 to use x, cell 3 will run fine — because the namespace still has x = 5 from the earlier execution — even though nobody could get that result by reading the notebook top to bottom. "Restart and Run All" is the only way to prove the notebook's visual order matches its logical dependency order, because it rebuilds the namespace from nothing and executes strictly top-to-bottom.

One cell, one idea matters because Jupyter's error model halts execution of a cell on the first uncaught exception but leaves every prior cell's state intact — so a giant cell that loads data, cleans it, and plots it will lose all three steps' progress on one bad line, while three separate cells let you fix and re-run just the failing step without re-running (and re-paying the cost of) the expensive data load above it.

Pinning dependencies (requirements.txt with exact versions, or a lock file) addresses the fact that pandas.read_csv defaults, sklearn model internals, and even floating-point reduction order in NumPy have changed across versions in ways that silently shift results — "reproducible" code that imports an unpinned library is only reproducible until that library's next release.

Clearing outputs before committing is about git's diff algorithm, which is line-based and has no concept of "this changed because I re-ran the cell, not because I changed the code": a notebook's JSON stores rendered outputs (including base64-encoded images) inline, so a re-run with no code change still produces a full diff of noise, and two people running the same notebook produce spurious merge conflicts purely from output-cell content.

Seeding randomness — anything using NumPy's default_rng(seed) or random.seed() — pins the pseudorandom number generator's internal state so the exact same sequence of "random" draws is produced on every run. Without it, a train/test split, a bootstrap resample, or a model's random weight initialization differs run to run, making "did my change actually help" undiagnosable from noise in the randomness alone.

Exercise

Take any notebook you've written with more than 10 cells. Run Kernel → Restart & Run All. If it fails or produces different numbers, identify the out-of-order dependency that caused it, and fix the cell ordering so a fresh run is deterministic and correct.