Python for file management


I show how file management can be carried out with Python. Therefore, if you are familiar with Python you won’t need Bash programming for automating complex tasks. Here I compare the Python functions with Bash commands on Ubuntu. These functions probably work in Windows and Mac OS too.


To avoid repetition, the codes in this post assume the libraries below are imported:

import os, shutil, fnmatch, ntpath

Moreover, I assume that users of these codes, employ try blocks to catch exceptions at a higher level because there are always I/O exceptions when reading and writing files.


In the below codes, comments show bash commands which are followed by their Python equivalent.

The current working directory can be found via:

# pwd

Move a file/directory into another directory with:

# mv myFile myDir
shutil.move('myFile', 'myDir')

Copy a file into another directory

# cp myFile myDir
shutil.copy2('myFile', 'myDir')

Copy a directory in another place with the same or different name:

# $ cp -r myDir1 myDir2
shutil.copytree('myDir1', 'myDir2')

If myDir2 is not existed, the command creates it and then the content of myDir1 is copied into it

Remove a file:

# $ rm a_file

Create a file:

# touch myFile
f = open("myFile.txt", "w")
f.write("Hello dear!")

Remove a directory and its content:

# $ rm -r myDir

Create a directory:

# mkdir myDir

Get a list of files and directories inside a directory:

# $ ls myDir

To check if a path is a file:

isIt = os.path.isfile('/dir1/myFile')

To check if a path is a directory:

isIt = os.path.isdir('/dir1/dir2')

To check if a path (file/directory) exists:

Exists = os.path.exists('/dir1/myFile')

Get a file or directory from its path i.e. strip everything from left to the rightest / in a the path:

# basename path
# /dir1/dir2/dir3 -> dir3
# /dir1/dir2/file.txt -> file.txt

Recursively get directories and files within a directory:

for root,dirs,files in os.walk(a_path):
        print (root)
        print (dirs)
        print (files)

os.walk() is perfect for searching files or directories. See its usage in the next section.


Here, I define some functions for higher level programmaing.

To delete the content of a directory:

def removeDirContent(dir):

Get a list of all files within a directory:

def listSubfiles(dir=os.getcwd()):
    for file_path in os.listdir(dir):
        if os.path.isfile(file_path) or os.path.islink(file_path):
    return files

Get a list of directories in a directory (not recursively):

def listSubdirs(dir=os.getcwd()):
    for file_path in os.listdir(dir):
        if os.path.isdir(file_path):
    return dirs

Get all sub-directories of a directory recursively like tree -d in Bash:

def listAllSubdirs(dir=os.getcwd()):
    return [x[0] for x in os.walk(dir)]

Find files with matching a pattern:

def findFiles(pattern, path=os.getcwd()):
    result = []
    for root, dirs, files in os.walk(path):
        for name in files:
            if fnmatch.fnmatch(name, pattern):
                result.append(os.path.join(root, name))
    return result

Find recursively list of directories that their name contains name:

def findDir(name, path=os.getcwd()):
    for dir in allsubs:
        if name in ntpath.basename(dir):
    return result

Remove a list of files

def removeFiles(files):
    for file in files:

Remove a list of directories

def removeDirs(dirs):
    for dir in dirs:

Copy a file in multiple directories

def scatterFile(file, dirs):
    for dir in dirs:
        shutil.copy2(file, dir)

Copy a folder multiple times

def multiCopyDir(dir, destNames, destParent=os.getcwd()):
    for name in destNames:
        shutil.copytree(dir, os.path.join(destParent, name))

For example, to copy ~/a folder into ~ and change the name of copies to b, c, d, the inputs are:

  • dir = ~/a
  • destParent=~
  • destNames=['b', 'c', 'd']
Tags ➡ Linux Python HPC


I notify you of my new posts

Latest Posts


0 comment