Python vs. JS

English

简体中文

English

简体中文

Appearance

Jupyter Notebook: The Swiss Army Knife of Interactive Computing ​

Jupyter Notebook is an open-source, web-based interactive computing environment that allows users to create and share documents containing live code, equations, visualizations, and narrative text. It has become an indispensable tool in data science, machine learning, scientific computing, and academic education.

References:

1. What is a Jupyter Notebook? ​

You can think of a Jupyter Notebook as a digital, executable lab notebook. Unlike traditional Integrated Development Environments (IDEs) that run an entire script at once, Jupyter allows you to break down your code into independent cells, which can then be executed individually and in any order.

This interactivity makes it ideal for exploratory data analysis, rapid prototyping, and educational demonstrations.

2. Core Components ​

A complete Jupyter system consists of three main components:

  1. The Web Application

    • This is the user interface you see in your browser for writing and running code. It provides a file browser and an interactive interface for creating, editing, and running Notebook documents (.ipynb files).

  2. Kernels

    • This section will be refactored and moved to a new chapter below.

  3. Notebook Documents (.ipynb)

    • Every Notebook you create is a file with the .ipynb extension.
    • It is essentially a JSON file that stores all your content in a structured way, including code cells, Markdown cells, the output of each cell, images, and metadata. This format makes Notebook documents easy to share and collaborate on using version control systems like Git.

The Relationship Between Jupyter & Python: The Power of Kernels ​

Understanding the relationship between Jupyter and Python is key to grasping the concept of the Kernel.

Jupyter itself is a language-agnostic platform. It provides a general "shell" or "interface" that allows users to interact with a code execution environment through a browser. The component that is actually responsible for executing the code in the background is the Kernel.

The IPython Kernel: Jupyter's "Python Engine" ​

When you choose to run Python code, the kernel that Jupyter starts is IPython (Interactive Python).

  • IPython is not standard Python: It is an enhanced interactive Python interpreter that provides many powerful features not found in the standard Python REPL, such as:

    • Rich code completion (Tab key).
    • Object introspection (adding a ? after a variable to see its help).
    • Magic Commands: Special commands prefixed with % or %% that control the Notebook's behavior, like %matplotlib inline for displaying Matplotlib plots directly in the notebook.
    • Seamless integration with the system shell.

  • Relationship Summary: Jupyter is the front-stage platform, and the IPython kernel is the backstage Python star. You enter commands through the Jupyter interface, Jupyter passes these to the IPython kernel, the kernel executes them, and then returns the results (text, numbers, plots, etc.) back to the Jupyter interface for display.

The Core Principle: A Decoupled Communication Protocol ​

The communication between the Jupyter front-end (the browser) and the back-end kernel is completely decoupled. They communicate through a well-defined Jupyter Messaging Protocol based on ZeroMQ.

The process works like this:

  1. You type print("Hello") in a code cell and press Shift + Enter.
  2. The Jupyter web app packages this code into an execute_request message that conforms to the protocol.
  3. This message is sent over the network to the running IPython kernel.
  4. The IPython kernel receives the message, executes the code, and captures the output of the print function.
  5. The kernel packages this output into a stream message and sends it back to the front-end.
  6. The Jupyter web app receives the message and renders it below the code cell.

The beauty of this architecture is that as long as a programming language can implement a kernel that adheres to this messaging protocol, it can be seamlessly integrated into the Jupyter ecosystem. This is why Jupyter supports so many languages, like R, Julia, and Scala—they each have their own kernel implementation.

3. The Core Concept: Cells ​

Cells are the fundamental building blocks of a Notebook. There are two primary types:

Code Cells ​

  • This is where you write and execute your code.
  • Each code cell is preceded by an In [ ]: label. When you run the cell (by clicking the "Run" button or using the shortcut Shift + Enter), the code is sent to the kernel for execution.
  • If the code produces an output (e.g., the result of a print() statement or the value of the last line in the cell), it will be displayed directly below the code cell.
  • Once a cell has been run, the brackets will be filled with a number indicating its execution order.
python
# This is a code cell
import numpy as np

def square(x):
    return x * x

x = np.random.randint(1, 10)
y = square(x)

print(f'The square of {x} is {y}')

Markdown Cells ​

  • This is where you can practice "Literate Programming."

  • You can use Markdown syntax in these cells to write richly formatted narrative text, including:

    • Headings
    • Bold and italic text
    • Lists (ordered or unordered)
    • Links and images
    • Tables
    • Even LaTeX mathematical equations

  • This allows you to clearly document your thought process, explain the logic of your code, and present your analysis, turning the entire Notebook into a complete, self-explanatory story.

markdown
# This is a Markdown cell

## Purpose of the Experiment
This experiment aims to verify the correctness of the `square` function.

- We will generate a random number.
- Then we will calculate its square.

The final result should match the expectation: $x^2 = y$.

4. Why Is It So Popular? ​

  • Interactive Exploration: Allows for rapid iteration and experimentation, perfect for tasks like data cleaning and model tuning that require constant trial and error.
  • Results and Code Together: Computational results (including charts and tables) are displayed inline, right below the code that produces them, making the analysis workflow clear and easy to follow.
  • Literate Programming: Combines code, explanations, and visualizations, greatly enhancing the readability and reproducibility of your work.
  • Easy to Share: .ipynb files can be easily shared via platforms like GitHub and NBViewer, allowing others to view your complete work in a browser without needing to install anything.

5. How to Get Started ​

The easiest way to get started is by installing JupyterLab (the next-generation interface for Jupyter Notebook) using pip.

bash
# Make sure you are in a virtual environment
pip install jupyterlab

# Start the JupyterLab server
jupyter lab

After running the last command, your default browser will automatically open a new tab with the JupyterLab interface, and you can start creating your first Notebook!

While installing directly with pip is convenient, a more robust and recommended long-term strategy is to use pipx to install JupyterLab.

Core Philosophy: Treat JupyterLab as a system-level application (like your code editor or browser), not a library that every project needs.

Why is this better?

  • Keeps Environments Clean: You don't need to repeatedly install JupyterLab and its numerous dependencies in every project's virtual environment (venv). Your project environment can contain only the libraries your project actually needs (like pandas or requests).
  • Single Entry Point: No matter which project you are working on, you use the same JupyterLab instance, managed centrally by pipx.
  • Avoids Dependency Conflicts: The dependencies of JupyterLab itself are safely isolated by pipx and will never conflict with the dependencies of any of your projects.

The Workflow: Global Application + Project Kernels

This model works in two steps:

Step 1: Install JupyterLab Once with pipx

bash
# 1. Install pipx (if you haven't already)
python -m pip install --user pipx
python -m pipx ensurepath

# 2. Use pipx to install jupyterlab
pipx install jupyterlab

After this step, jupyter lab becomes a global command you can run from anywhere on your system.

Step 2: Create Independent Kernels for Your Projects

Now, for each specific data science project, you should create a separate virtual environment and register a kernel for it.

bash
# 1. Create and activate a virtual environment for your new project
cd /path/to/my_new_project/
python -m venv .venv
source .venv/bin/activate

# 2. Install the libraries needed for this project in this environment
pip install pandas matplotlib scikit-learn

# 3. Install ipykernel, the bridge between your environment and Jupyter
pip install ipykernel

# 4. Register the current virtual environment as a new Jupyter kernel
#    --name: The internal name for the kernel
#    --display-name: The pretty name shown in the Jupyter UI
python -m ipykernel install --user --name="my_new_project_kernel" --display-name="Python (My New Project)"

Now, when you run the global jupyter lab command, you will see "Python (My New Project)" as an option in the kernel selector on the Launcher page. By selecting it, your Notebook will run in the clean environment you created for my_new_project, which contains pandas and other libraries.

When you're done with the project, you can easily remove the kernel:

bash
jupyter kernelspec uninstall my_new_project_kernel

This "Global Jupyter + Local Kernels" model is widely recognized as the clearest and most error-proof best practice for managing complex Python projects.