Execute bash script from Python on Windows

Question

I am trying to write a python script that will execute a bash script I have on my Windows machine. Up until now I have been using the Cygwin terminal so executing the bash script RunModels.scr has been as easy as ./RunModels.scr. Now I want to be able to utilize subprocess of Python, but because Windows doesn't have the built in functionality to handle bash I'm not sure what to do.

I am trying to emulate ./RunModels.scr < validationInput > validationOutput

I originally wrote this:

os.chdir(atm)
vin = open("validationInput", 'r')
vout = open("validationOutput", 'w')
subprocess.call(['./RunModels.scr'], stdin=vin, stdout=vout, shell=True)
vin.close()
vout.close()
os.chdir(home)

But after spending a while trying to figure out why my access was denied, I realized my issue wasn't the file permissions but the fact that I was trying to execute a bash file on Windows in general. Can someone please explain how to execute a bash script with directed input/output on windows using a python script?

Edit (Follow up Question):

Thanks for the responses, I needed the full path to my bash.exe as the first param. Now however, command line calls from within RunModels.scr come back in the python output as command not found. For example, ls, cp, make. Any suggestions for this?

Follow up #2: I updated my call to this:

subprocess.call(['C:\\cygwin64\\bin\\bash.exe', '-l', 'RunModels.scr'], stdin=vin, stdout=vout, cwd='C:\\path\\dir_where_RunModels\\')

The error I now get is /usr/bin/bash: RunModels.scr: No such file or directory. Using cwd does not seem to have any effect on this error, either way the subprocess is looking in /usr/bin/bash for RunModels.scr.

SELF-ANSWERED I needed to specify the path to RunModels.scr in the call as well as using cwd.

subprocess.call(['C:\\cygwin64\\bin\\bash.exe', '-l', 'C:\\path\\dir_where_RunModels\\RunModels.scr'], stdin=vin, stdout=vout, cwd='C:\\path\\dir_where_RunModels\\')

But another problem...

Regardless of specifying cwd, the commands executed by RunModels.scr are throwing errors as if RunModels.scr is in the wrong directory. The script executes, but cp and cd throw the error no such file or directory. If I navigate to where RunModels.scr is through the command line and execute it the old fashioned way I don't get these errors.

Answer 1

Python 3.4 and below

Just put bash.exe in first place in your list of subprocess.call arguments. You can remove shell=True, that's not necessary in this case.

subprocess.call(['C:\\cygwin64\\bin\\bash.exe', '-l', 'RunModels.scr'], 
                 stdin=vin, stdout=vout,
                 cwd='C:\\path\\dir_where_RunModels\\')

Depending on how bash is installed (is it in the PATH or not), you might have to use the full path to the bash executable.

Python 3.5 and above

subprocess.call() has been effectively replaced by subprocess.run().

subprocess.run(['C:\\cygwin64\\bin\\bash.exe', '-l', 'RunModels.scr'], 
                 stdin=vin, stdout=vout,
                 cwd='C:\\path\\dir_where_RunModels\\')

Edit:

With regard to the second question, you might need to add the -l option to the shell invocation to make sure it reads all the restart command files like /etc/profile. I presume these files contain settings for the $PATH in bash.

Edit 2:

Add something like pwd to the beginning of RunModels.scr so you can verify that you are really in the right directory. Check that there is no cd command in the rc-files!

Edit 3:

The error /usr/bin/bash: RunModels.scr: No such file or directory can also be generated if bash cannot find one of the commands that are called in the script. Try adding the -v option to bash to see if that gives more info.

Answer 2

A better solution than the accepted answer is to use the executable keyword argument to specify the path to your shell. Behind the curtain, Python does something like

exec([executable, '-c`, subprocess_arg_string])

So, concretely, in this case,

subprocess.call(
    './RunModels.scr',
    stdin=vin, stdout=vout,
    shell=True,
    executable="C:/cygwin64/bin/bash.exe")

(Windows thankfully lets you use forward slashes instead of backslashes, so you can avoid the requirement to double the backslashes or use a raw string.)