Build a Multi-Module Python Project

To build a multi-module Python project means moving beyond the single-file scripts that worked during the early stages of learning and creating a directory structure where each module has a clear responsibility, import paths are predictable, and the project can grow without becoming disorganized. The transition from a flat collection of .py files to a structured project with packages, a clear entry point, and proper configuration is the point where Python stops being a scripting tool and starts being a software development platform.

The goal of this article is to walk through building a multi-module Python project from scratch, making intentional decisions at each step about directory layout, package boundaries, import styles, and project configuration. By the end, you will have a template that you can adapt for your own projects, whether they are command-line tools, data processing pipelines, or libraries that other developers will install and import.

Starting with the right directory layout

The flat layout is the simplest starting point. A single Python script sits at the project root alongside any data files and configuration. Additional modules live in the same directory and are imported by name. This layout works for projects with fewer than five modules and no external consumers. It is the right choice for personal automation scripts, one-off data analysis, and prototypes that may never grow beyond a few files.

When the project outgrows the flat layout, the next step is the package layout. All importable code moves into a named package directory with an init file, and the entry-point script stays at the project root. This separation creates a clear boundary between the application layer, which runs the project, and the library layer, which contains the reusable logic. The package name becomes the top-level namespace, and all internal imports use that name as the root.

python

from taskmanager.tasks import create_task, list_tasks
from taskmanager.storage import save_tasks, load_tasks

A simple multi-module project with this structure might have a task manager package containing modules for task operations and data storage, plus a main script at the root that imports from the package and provides the command-line interface. Each module inside the package handles one aspect of the problem domain, and the main script orchestrates them without containing any business logic itself.

Designing the package hierarchy

The package's internal structure should mirror the conceptual organization of the problem domain. If the project manages tasks, the package might start with a flat collection of modules: one for task creation and manipulation, one for data storage, and one for formatting output. As the project grows, modules that become too large are split, and related modules are grouped into subpackages.

The storage module might grow to handle multiple storage backends, at which point it becomes a storage subpackage containing a base module for the common interface and separate modules for JSON storage, SQLite storage, and cloud storage. The tasks module might grow to include task scheduling and recurring tasks, at which point it splits into a tasks subpackage. The key principle is that the package hierarchy evolves in response to real complexity, not in anticipation of it.

python

from taskmanager.tasks.crud import create_task, delete_task
from taskmanager.storage.json_backend import save_to_json

Each subpackage gets its own init file that re-exports the public API. A developer who needs to work with tasks imports from the tasks subpackage without needing to know which specific module inside it contains each function. The init file acts as a facade that hides the internal module structure and presents a stable interface to the rest of the project. This pattern, covered in detail in the article on the Python init file, is what keeps large packages navigable as they grow.

Setting up the entry point

The main script at the project root should be thin. It imports from the package, parses command-line arguments or reads configuration, calls the appropriate package functions, and exits. It should not contain business logic, data processing, or any code that another script might want to reuse. If the main script starts accumulating logic, that logic belongs in the package where it can be imported and tested independently.

For projects that need to be executable as a command after installation, the entry point can be defined in the project configuration file so that pip creates a wrapper script automatically. This is more robust than relying on users to run the main script directly, and it is the standard approach for Python packages that are distributed through PyPI. The same project can also include a main.py file so that it is runnable with the python -m flag, as explained in the article on the Python main module.

Adding project configuration

The pyproject.toml file is the modern standard for Python project configuration. It lives at the project root and declares the project's name, version, description, Python version requirements, and dependencies. It also tells build tools like pip and setuptools how to find the package directory so that installation works correctly.

A minimal configuration file specifies the build system, the project metadata, and the package location. With this file in place, running pip install in the project directory installs the package into the current Python environment, making it importable from anywhere. The editable install flag links the installed package to the source directory so that code changes take effect immediately without reinstalling, which is essential for development. This configuration step is what separates a collection of scripts from a proper Python project that can be shared, versioned, and deployed reliably across different machines and environments.