The shutil module is Python's high-level toolbox for moving real files and directories around on disk. Where os gives you small surgical operations (rename one entry, remove one file) and pathlib gives you a clean way to write paths, shutil is the layer that handles whole-file and whole-tree work: copy this directory into that one, zip up a folder for download, move a file across drives, delete a tree of generated reports. This lesson walks through the parts of shutil you'll actually reach for in an E-Commerce codebase: copying, moving, removing, archiving, checking disk space, and finding executables.

Python gives you three layers for working with the filesystem, each at a different level of abstraction. os exposes thin wrappers over operating system calls. pathlib gives you Path objects with methods like .exists(), .read_text(), and .glob(). shutil sits on top of both and handles the "whole operation" cases: copying a tree, archiving a folder, moving across devices. They aren't competitors; they cover different jobs.

The diagram shows the typical stack. shutil calls into os for the actual work, and pathlib is mostly about clean path handling. Reach for shutil when an operation describes a whole-file or whole-tree action ("copy this entire folder", "zip this directory", "delete this tree"). Reach for os when you need a single primitive (os.rename, os.remove, os.stat). Reach for pathlib to build, join, and inspect path values.

A small mental rule that works in practice:

This lesson focuses on what shutil adds on top of low-level os calls and pathlib.

There are four functions for copying one file, and the differences matter. They all sit in shutil, and they vary by what metadata they preserve and whether the destination can be a directory or must be a file.

A concrete example with an order CSV. Imagine the daily orders file has just been generated and you want a quick working copy in another directory:

shutil.copy saw that backup_dir was a directory and placed the file inside it, keeping the original file name. The return value is the actual destination path, useful when you let shutil pick the final name.

copyfile is stricter. The destination must be a file path; you can't hand it a directory:

That works because the destination is a path that names a file. The same call with "backups" as the destination would fail with IsADirectoryError, which is the difference from copy.

copy2 is copy plus metadata preservation. If you care about the original mtime (modification time) and atime (access time), and on Linux about extended attributes too, use copy2. Daily-orders backups that need to keep the "when was this file produced" timestamp are a textbook case:

Without copy2, the destination would have the current time as its mtime, and a downstream script that sorts backups by "when the order was produced" would get the wrong answer.

copyfileobj is the low-level option. Instead of two paths, it takes two open file objects and streams bytes from one to the other in a loop. Most of the time you don't need it, but it's the right call when one side isn't a real file on disk (an in-memory buffer, a network stream, a file inside a zip archive). The pattern:

The length argument is the buffer size. The default is fine for most cases. The reason to set it explicitly is when you're copying very large files and want to balance memory use against syscall count.

Single-file copies are useful, but real work usually involves whole folders. A product-images folder, a release of a catalog dump, a staging directory for the CDN. shutil.copytree(src, dst) walks the source tree and recreates it at the destination.

The destination directory (cdn_staging/product_images) didn't exist before the call, and copytree created it. The internal file objects use copy2 by default, so timestamps and permissions carry over.

What's wrong with this code?

The caller created the destination directory before the copy. Historically, copytree would refuse if the destination already existed. The fix in Python 3.8 and later is the dirs_exist_ok=True flag:

Fix:

With dirs_exist_ok=True, copytree will merge into the existing destination, overwriting files with the same name and leaving other files in place. On Python 3.7 and earlier, the workaround was to remove the destination first or to write a manual walk; the flag landed in 3.8 specifically because the "merge into existing" case is so common.

Filtering What Gets Copied

When you stage product images for the CDN, you probably don't want to copy over editing artifacts, source files, or hidden metadata. copytree accepts an ignore callable that returns the set of names to skip in each directory. The easiest way to build one is shutil.ignore_patterns:

The .psd file and the .DS_Store file did not make it across. ignore_patterns uses shell-style globbing (the same syntax as fnmatch), so *.psd matches every file ending in .psd, in any subdirectory. The patterns are matched against the file name only, not the full path.

You can write your own ignore function if globbing isn't enough. The signature is ignore(directory, contents) -> list of names to skip:

That callable would skip anything over 5 MB. You'd pass it as ignore=skip_large_files.

shutil.move(src, dst) renames or relocates a file or directory. On the surface it looks like os.rename, but it does an important extra thing: when the destination is on a different filesystem, os.rename fails with OSError, and shutil.move falls back to copying the bytes over and then deleting the source. That fallback is the whole point of using shutil.move.

The file disappeared from inbox and appeared in processed. shutil.move returns the destination path, so a caller that didn't know whether the destination was a directory or a file still gets the resolved location.

What's wrong with this code?

os.rename is a single syscall. When the source and destination live on different filesystems, the kernel can't move the inode across without copying the data, so it refuses with EXDEV (Invalid cross-device link). The fix is to use shutil.move, which catches that error and falls back to a copy-then-delete:

Fix:

That call works whether the two paths are on the same drive or not. When they're on the same drive, shutil.move uses os.rename (effectively free). When they're on different drives, it copies the bytes and then unlinks the source. The caller doesn't have to know which case applies.

You can also pass a custom copy_function to shutil.move. The default is shutil.copy2, but you can pass shutil.copy if you don't need metadata preserved:

This is rarely needed in everyday code, but it's the lever to pull if cross-device moves are showing up as a bottleneck.

os.remove deletes one file. os.rmdir deletes one empty directory. Neither will remove a directory that has anything inside it. shutil.rmtree(path) is the recursive delete: it walks the tree, removes every file, then removes every directory, working from the leaves up.

The folder and everything inside is gone. There's no undo. Once rmtree finishes, the files are unlinked at the filesystem level.

Handling Errors Gracefully

By default, rmtree raises on the first error: a file that's locked, a permission problem, a symlink loop. In a daily cleanup job you usually want to log the failures and keep going, not abort the whole purge. The onexc parameter (added in Python 3.12, replacing the older onerror) lets you supply a callback that runs whenever a deletion fails.

The callback receives the function that was attempting the deletion (os.unlink, os.rmdir, or os.scandir), the path it was working on, and the exception instance. Returning from the callback lets rmtree continue with the remaining entries.

On Python 3.11 and earlier, the equivalent parameter was called onerror and its signature was slightly different (it received an exc_info tuple instead of the exception instance). Both still work in 3.12+, but onexc is the modern spelling and the one to reach for in new code:

If you're targeting both old and new interpreters, you can guard with a version check; in practice, most projects pick one Python baseline and use the matching parameter.

A common pattern in cleanup scripts is "delete and recreate":

That gives you a known-empty starting point before staging the next release. The if staging.exists() check avoids a FileNotFoundError on the first run.

shutil.make_archive(base_name, format, root_dir) takes a directory and produces a single archive file. It's the easy way to package a release of a product-catalog dump for download, to bundle daily order CSVs into a dated archive, or to create a release artifact for a content drop.

A few details worth knowing:

The function supports several formats out of the box:

The list is available at runtime via shutil.get_archive_formats(), which returns a list of (name, description) pairs:

A backup-the-orders example, using a dated filename and gztar so the file compresses well:

The CSVs are now compressed in one file inside backups, dated, ready to copy off the server or attach to a release.

The inverse of make_archive is unpack_archive(filename, extract_dir, format=None). When the format is omitted, shutil infers it from the file extension.

The files come back out into restore/orders-2026-05-14/, in the same layout they had when the archive was created. If the format isn't obvious from the extension (for example, a .dat file that's actually a tar), pass format="tar" explicitly.

The list of registered unpack formats has the same shape as the archive formats:

There's one historical safety note worth flagging. unpack_archive for tar files originally allowed paths inside the archive to point outside the destination (a path like ../../etc/passwd could escape). Starting in Python 3.12, the safer behavior is opt-in via a filter parameter, and starting in 3.14 it became the default. If you're on 3.12 or 3.13 and you're extracting archives from untrusted sources, pass filter="data":

For archives you produced yourself, like the catalog and orders dumps above, this isn't a concern. The flag matters when the archive came from a customer upload, an external partner, or any other place you don't control.

Before staging a 5 GB catalog dump, knowing whether the destination volume has room saves a half-written archive and a confusing error message later. shutil.disk_usage(path) returns a named tuple of (total, used, free) in bytes for the filesystem that contains path.

The numbers are in bytes, which is awkward to read. Format them yourself:

A practical use: before writing a daily product-catalog dump, refuse to start if there isn't enough headroom on the data volume.

The path you pass doesn't have to be the file you're about to write; any path on the right filesystem will do. shutil.disk_usage queries the filesystem itself, not the individual file.

shutil.which(cmd) is the cross-platform equivalent of Unix which or Windows where. It walks the directories on the PATH environment variable and returns the first matching executable, or None if none is found.

Why this matters in an E-Commerce script: imagine a job that resizes product images before uploading them to the CDN, and you want to call out to ffmpeg or convert. Before invoking it via subprocess, check that the binary is actually present:

The error you get from a missing binary at subprocess.run time (FileNotFoundError) is harder to debug than the explicit "ImageMagick is not installed" you can produce yourself. shutil.which lets you fail loudly and early.

You can also pass path= to search a specific list of directories instead of the system PATH, and mode=os.X_OK (the default) controls the access bits the function checks for. Most code doesn't need these arguments, but they're there if you need them.

shutil.chown(path, user=None, group=None) changes the owning user and/or group of a file or directory. It accepts either names or numeric IDs and is a thin wrapper around os.chown with the convenience of looking up the names for you.

A few things to know:

In an E-Commerce setting, the typical use is in deployment scripts: after a release pulls in fresh files, chown makes sure the runtime user (appuser) owns them so the application can read or write as needed.

A daily backup script for an order-processing system is a natural place for several of these calls to land in the same file. Walk through the steps the script needs:

The implementation, end to end:

Nothing in that script reaches for os or pathlib for actions that shutil already covers, which is the whole point of the module. The script reads top-to-bottom as a sequence of meaningful operations, not a sequence of byte-level calls.

A few shutil corners behave differently across operating systems, worth knowing before you ship a script that runs on developer laptops and Linux servers both.

Most code that uses the functions in this lesson works on all three platforms without changes. The functions to be careful with are chown (Unix only) and anything that relies on metadata semantics specific to one filesystem.