execute a shell script from different folder in Python

Question

I'm not a pythonist, just need to fix some script. So I'm want to run some Bash shell script that placed not in the same folder as the Python script but in one of its subfolder.

So that what I did:

app_path = os.getcwd()
path = app_path + os.path.sep + "some/subfolder"
retval = subprocess.call(["script.sh"], cwd=path)

Unfortunately I get an error:

FileNotFoundError: [Errno 2] No such file or directory: 'script.sh'

Adding shell=True doesn't help but the error changes to:

/bin/sh: 1: script.sh: not found

The error looks strange for me especially since my current shell is /bin/bash

echo $SHELL
/bin/bash

So far I've found only this ugly workaround:

app_path = os.getcwd()
path = app_path + os.path.sep + "some/subfolder"
os.chdir(path)
retval = subprocess.call(["./script.sh"], shell=True)

but it seems to me an absolute ugliness.

What I do wrong? How can I execute an external bash script from a Python one?

Answer 1

You don't need shell=True. You don't need os.path.join().

Either fix your bash script so it doesn't assume that its current working directory is the same as the location of its source code, so you can use:

retval = subprocess.call(["./some/subfolder/script.sh"])

...or use:

retval = subprocess.call(["./script.sh"], cwd='./some/subfolder')

...making sure that your script is executable (starts with a shebang, has +x permissions set).

Answer 2

You probably don't want the scripts to be invoked using a relative path as it depends on how your app is started and from where, so if your scripts are in a know directory (this can come from a configuration file, env or a system known location):

os.environ['PATH'] += f'{os.pathsep}/path/to/my/scripts'
...
p = subprocess.run(['script1.sh'], env=os.environ)
...
p = subprocess.run(['script2.sh'], env=os.environ)

Also, I would use run instead of call in new code.

If you never ever are going to pass arguments you can also use

p = subprocess.run('script2.sh', env=os.environ)

PS: All fixes in @CharlesDuffy answer are also needed.