37

I have a case to want to execute the following shell command in Python and get the output,

echo This_is_a_testing | grep -c test

I could use this python code to execute the above shell command in python,

>>> import subprocess
>>> subprocess.check_output("echo This_is_a_testing | grep -c test", shell=True)
'1\n'

However, as I do not want to use the "shell=True" option, I tried the following python code,

>>> import subprocess
>>> p1 = subprocess.Popen(["echo", "This_is_a_testing"], stdout=subprocess.PIPE)
>>> p2 = subprocess.Popen(["grep", "-c", "test"], stdin=p1.stdout)
>>> p1.stdout.close()
>>> p2.communicate()
(None, None)

I wonder why the output is "None" as I have referred to the descriptions in the webpage : http://docs.python.org/library/subprocess.html#subprocess.PIPE

Had I missed some points in my code ? Any suggestion / idea ? Thanks in advance.

27

From the manual:

to get anything other than None in the result tuple, you need to give stdout=PIPE and/or stderr=PIPE

p2 = subprocess.Popen(["grep", "-c", "test"], stdin=p1.stdout, stdout=subprocess.PIPE)
| improve this answer | |
34

Please look here:

>>> import subprocess
>>> p1 = subprocess.Popen(["echo", "This_is_a_testing"], stdout=subprocess.PIPE)
>>> p2 = subprocess.Popen(["grep", "-c", "test"], stdin=p1.stdout)
>>> 1
p1.stdout.close()
>>> p2.communicate()
(None, None)
>>>

here you get 1 as output after you write p2 = subprocess.Popen(["grep", "-c", "test"], stdin=p1.stdout), Do not ignore this output in the context of your question.

If this is what you want, then pass stdout=subprocess.PIPE as argument to the second Popen:

>>> p1 = subprocess.Popen(["echo", "This_is_a_testing"], stdout=subprocess.PIPE)
>>> p2 = subprocess.Popen(["grep", "test"], stdin=p1.stdout, stdout=subprocess.PIPE)
>>> p2.communicate()
('This_is_a_testing\n', None)
>>>
| improve this answer | |
  • Now I know that for p2.communicate() to work, I must add stdout=subprocess.PIPE in the code p2 = .... Thanks. – user1129812 Feb 22 '12 at 12:06
  • How I can get error if it happens in p1 result? Here is my question, can you help with it? stackoverflow.com/questions/40467105/… – prosto.vint Nov 7 '16 at 17:15
24
>>> import subprocess

>>> mycmd=subprocess.getoutput('df -h | grep home | gawk \'{ print $1 }\' | cut -d\'/\' -f3')

>>> mycmd 

'sda6'

>>>
| improve this answer | |
  • it uses the shell internally. OP explicitly said: "I do not want to use the "shell=True" option" – jfs Sep 14 '15 at 16:27
  • 9
    Thanks. That's exactly what I'm looking for when searching for "how to execute shell commands with pipes" in Python. – xpt Jul 14 '16 at 21:46
  • 3
    I had to use commands.getoutput instead of subprocess.getoutput for this to work – Anake Nov 29 '16 at 12:12
  • 1
    Isn't is subprocess.check_output()? – WesternGun Oct 20 '17 at 8:43
  • 1
    Doesn't exist in Python 2.7, does in Python 3.6.6 – DavidJ Dec 6 '18 at 17:29
6

While the accepted answer is correct/working, another option would be to use the Popen.communicate() method to pass something to a process' stdin:

>>> import subprocess
>>> p2 = subprocess.Popen(["grep", "-c", "test"], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
>>> p2.communicate("This_is_a_testing")
('1\n', None)
>>> print p2.returncode
0
>>>>

This resolves the need to execute another command just to redirect it's output, if the output is already known in the python script itself.

However communicate has the side-effect, that it waits for the process to terminate. If asynchronous execution is needed/desired using two processes might be the better option.

| improve this answer | |
0

Answer is similar to mentioned earlier, with little formatting. I wanted to get exactly same output as normal shell command with pipe on python 3.

import subprocess

p1 = subprocess.Popen(["ls", "-l", "."], stdout=subprocess.PIPE)
p2 = subprocess.Popen(["grep", "May"], stdin=p1.stdout, stdout=subprocess.PIPE)


for s in (str(p2.communicate())[2:-10]).split('\\n'):
    print(s)
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.