Python subprocess.Popen() not running command

Question

I'm trying to use subprocess.Popen() to run a command in my script. The code is:

output = Popen(["hrun DAR_MeasLogDump " + log_file_name], stdout=subprocess.PIPE, stderr = subprocess.PIPE, executable="/bin/csh", cwd=cwdir, encoding='utf-8')

When I print the output, it's printing out the created shell output and not the actual command that's in the list. I tried getting rid of executable='/bin/csh', but then Popen wouldn't even run.

I also tried using subprocess.communicate(), but it didn't work either. I would also get the shell output and not the actual command run.

I want to completely avoid using shell=True because of security issues.

EDIT: In many different attempts, "hrun" is not being recoognized. "hrun" is a Pearl script that is being called, DAR_MeasLogDump is the action and log_file_name is the file that the script will call its action on. Is there any sort of set up or configuration that needs to be done in order for "hrun" to be recognized?

Answer 1

I think the problem is that Popen requires a list of every part of the command (command + options), the documentation for Popen inside subprocess has an example for that. So for that line in your script to work, you would need to write it like this:

output = Popen(["/bin/csh", "hrun", "DAR_MeasLogDump", log_file_name], stdout=subprocess.PIPE, stderr = subprocess.PIPE)

I've removed the executable argument, but I guess it could work that way as well.

Answer 2

Try:

output = Popen(["-c", "hrun DAR_MeasLogDump " +log_file_name], stdout=subprocess.PIPE, stderr = subprocess.PIPE, executable="/bin/csh", cwd=cwdir, encoding='utf-8')

csh is expecting -c "full command here". Without -c I think it just tries to open it as a file.

Answer 3

Specifying an odd shell and an explicit cwd seems completely out of place here (assuming cwdir is defined to the current directory).

If the first argument to subprocess is a list, no shell is involved.

result = subprocess.run(["hrun", "DAR_MeasLogDump", log_file_name],
    stdout=subprocess.PIPE, stderr = subprocess.PIPE,
    universal_newlines=True, check=True)
output = result.stdout

If you need this to be run under a legacy version of Python, maybe use check_output instead of run.

You generally want to avoid Popen unless you need to do something which the higher-level wrapper functions cannot do.

Answer 4

You are creating an instance of subprocess.Popen but not executing it.

You should try:

p = Popen(["hrun", "DAR_MeasLogDump ", log_file_name], stdout=subprocess.PIPE, stderr = subprocess.PIPE, cwd=cwdir, encoding='utf-8')

out, err = p.communicate()  # This will get you output

Args should be passed as a sequence if you do not use shell=True, and then using executable should not be required.

Note that if you are not using advanced features from Popen, the doc recommends using subprocess.run:

from subprocess import run

p = run(["hrun", "DAR_MeasLogDump ", log_file_name], capture_output=True, cwd=cwdir, encoding='utf-8')

out, err = p.communicate()  # This will get you output

Answer 5

This works with cat example:

import subprocess

log_file_name='-123.txt'

output = subprocess.Popen(['cat', 'DAR_MeasLogDump' + log_file_name], 
                                stdout=subprocess.PIPE, 
                                stderr=subprocess.STDOUT)
stdout, stderr = output.communicate()
print (stdout)
print (stderr)

I think you need only change to your 'hrun' command

Answer 6

It seems the same problem that I had at the beginning of a project: you have tried with windows "environment variables". It turns out that when entering the CMD or powershell it does not recognize perl, java, etc. unless you go to the folder where the .exe .py .java, etc. is located and enter the cmd, where the java.exe, python.py, etc. is.

In my ADB project, once I added in my environment variables, I no longer needed to go to the folder where the .exe .py or adb code was located.

Now you can open a CMD and it will execute any command even from your perl , so the interpreter that uses powershell will find and recognize the command.