Python: System Call Tutorial

, , …,

To call a unix shell command, first import subprocess then call one of the following, depending on whether you want Python to return the exit code or command output.

Call Shell Command, Get Output

subprocess.check_output([cmd, arg1, arg2, …]) → Run shell command cmd arg1 arg2. Wait for command to complete, then return its output as a byte string. If the return code is not 0, then raises CalledProcessError.

# -*- coding: utf-8 -*-
# python

# call unix shell command and get its output

import subprocess

output = subprocess.check_output(["ls", "-al"])

print output

Call Shell Command, Ignore Output, Get Exit Code

subprocess.call([cmd, arg1, arg2, …]) → Run shell command cmd arg1 arg2. Wait for command to complete, then return the shell command exit code.

# -*- coding: utf-8 -*-
# python

# call unix shell command and get its exit code. (ignore output)
import subprocess
exitCode = subprocess.call(["ls", "-al"])
print exitCode

subprocess.call() will run successfully even if the shell command did not run successfully.

# -*- coding: utf-8 -*-
# python

# call unix shell command and get its exit code.

import subprocess

# here, ls will exit with error code 2, because there's no option -y
exitCode = subprocess.call(["ls", "-y"])

# python will not raise error
print exitCode                  # 2

(Linux shell commands have exit code of 0 if ran successfully.)

Call Shell Command, Ignore Output, Stop on Error

subprocess.check_call([cmd, arg1, arg2, …]) → Run shell command cmd arg1 arg2. Wait for command to complete. If the return code is 0 then return 0, else raise CalledProcessError. To get exit code, call CalledProcessError.returncode.

# -*- coding: utf-8 -*-
# python

# call unix shell command. Raise exception if the command failed

import subprocess

# ls does not have -y option
exitCode = subprocess.check_call(["ls", "-y"])

# this will not be printed
print "I finished."

# python exit error
"""
Traceback (most recent call last):
  File "/home/jane/xx-testscript-3392647.py", line 9, in <module>
    exitCode = subprocess.check_call(["ls", "-y"])
  File "/usr/lib/python2.7/subprocess.py", line 511, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ls', '-y']' returned non-zero exit status 2
"""

Advanced: 「subprocess.Popen()」

All the following commands:

are wrappers to the more general function subprocess.Popen(). Here's the code:

def call(*popenargs, **kwargs):
    """Run command with arguments.  Wait for command to complete, then
    return the returncode attribute.

    The arguments are the same as for the Popen constructor.  Example:

    retcode = call(["ls", "-l"])
    """
    return Popen(*popenargs, **kwargs).wait()
def check_call(*popenargs, **kwargs):
    """Run command with arguments.  Wait for command to complete.  If
    the exit code was zero then return, otherwise raise
    CalledProcessError.  The CalledProcessError object will have the
    return code in the returncode attribute.

    The arguments are the same as for the Popen constructor.  Example:

    check_call(["ls", "-l"])
    """
    retcode = call(*popenargs, **kwargs)
    if retcode:
        cmd = kwargs.get("args")
        if cmd is None:
            cmd = popenargs[0]
        raise CalledProcessError(retcode, cmd)
    return 0
def check_output(*popenargs, **kwargs):
    r"""Run command with arguments and return its output as a byte string.

    If the exit code was non-zero it raises a CalledProcessError.  The
    CalledProcessError object will have the return code in the returncode
    attribute and output in the output attribute.

    The arguments are the same as for the Popen constructor.  Example:

    >>> check_output(["ls", "-l", "/dev/null"])
    'crw-rw-rw- 1 root root 1, 3 Oct 18  2007 /dev/null\n'

    The stdout argument is not allowed as it is used internally.
    To capture standard error in the result, use stderr=STDOUT.

    >>> check_output(["/bin/sh", "-c",
    ...               "ls -l non_existent_file ; exit 0"],
    ...              stderr=STDOUT)
    'ls: non_existent_file: No such file or directory\n'
    """
    if 'stdout' in kwargs:
        raise ValueError('stdout argument not allowed, it will be overridden.')
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
    output, unused_err = process.communicate()
    retcode = process.poll()
    if retcode:
        cmd = kwargs.get("args")
        if cmd is None:
            cmd = popenargs[0]
        raise CalledProcessError(retcode, cmd, output=output)
    return output

Examples of using 「subprocess.Popen(…)」

Call and Don't Wait

Suppose you want to decompress a file. gzip -d x.txt.gz

# -*- coding: utf-8 -*-
# python

import subprocess

# call a unix command but not wait for it to finish
subprocess.Popen([r"gzip","-d", "x.txt.gz"])

Call and Wait for Command to Finish

Suppose you want to decompress a file. gzip -d x.txt.gz

# -*- coding: utf-8 -*-
# python

import subprocess
subprocess.Popen([r"gzip","-d", "x.txt.gz"]).wait()

The subprocess.Popen([…]) creates a object. The .wait() makes the code wait until the system call finished.

Get Output

To make a system call and get its output, pass the argument stdout=subprocess.PIPE, and get output from the method communicate()[0].

# -*- coding: utf-8 -*-
# python

# call a unix command, wait for it to finish, get its output

myOutput=subprocess.Popen([r"tail","-n 1", "x.txt"], stdout=subprocess.PIPE).communicate()[0]

The method communicate() returns a tuple (stdoutdata, stderrdata), but you must pass arguments stdout=subprocess.PIPE and stderr=subprocess.PIPE to subprocess.Popen().

Full Example

Following is a complete example. The code makes a system call to decompress a gzip file (and wait for it), then get the last line of the file by calling the unix util “tail”, then gzip the file again.

# -*- coding: utf-8 -*-
# python

import subprocess

# decompress a file
subprocess.Popen([r"gzip","-d", "x.txt.gz"]).wait()

# get last line
last_line = subprocess.Popen([r"tail","-n 1", "x.txt"], stdout=subprocess.PIPE).communicate()[0]

# compress the file
subprocess.Popen([r"gzip","x.txt"])

print last_line

Note: Python has a gzip module. If you really want to compress/decompress a file, use that. 〔☛ Python: Compress/Decompress Gzip Files

Get Output, Stderr, Exit Code, Process Id

Here's a example of getting both output and exit code.

# -*- coding: utf-8 -*-
# python

import subprocess

# call shell command
subP = subprocess.Popen(["du", "--si", "-s", "/usr/bin"],  stdout=subprocess.PIPE , stderr=subprocess.PIPE)

stdout, stderr = subP.communicate()

exitCode = subP.returncode

pid = subP.pid

print "--------------------\n output is:", stdout

print "--------------------\n stderr is:", stderr

print "--------------------\n exit code is:", exitCode

print "--------------------\n pid is:", pid

Reference

17.1. subprocess — Subprocess management — Python v2.7.6 documentation

The module “subprocess” is available only since Python 2.4, and it is intended to replace the following:

blog comments powered by Disqus