130

Can someone explain why the result I want, "hi", is preceded with a letter 'b' and followed with a newline?

I am using Python 3.3

>>> import subprocess
>>> print(subprocess.Popen("echo hi", shell=True,
                           stdout=subprocess.PIPE).communicate()[0])
b'hi\n'

This extra 'b' does not appear if I run it with python 2.7

3
  • 1
    What version of Python are you using?
    – Necrolyte2
    Mar 12, 2013 at 23:27
  • 2
    Not sure about the 'b', but the newline is because echo hi prints hi\r\n. To avoid that, you could add .strip() at the end, or similar fix.
    – azhrei
    Mar 12, 2013 at 23:34
  • 8
    you could use check_output() instead of .communicate() here: print(subprocess.check_output("echo hi", shell=True, universal_newlines=True), end="")
    – jfs
    Mar 13, 2013 at 0:00

4 Answers 4

121

The b indicates that what you have is bytes, which is a binary sequence of bytes rather than a string of Unicode characters. Subprocesses output bytes, not characters, so that's what communicate() is returning.

The bytes type is not directly print()able, so you're being shown the repr of the bytes you have. If you know the encoding of the bytes you received from the subprocess, you can use decode() to convert them into a printable str:

>>> print(b'hi\n'.decode('ascii'))
hi

Of course, this specific example only works if you actually are receiving ASCII from the subprocess. If it's not ASCII, you'll get an exception:

>>> print(b'\xff'.decode('ascii'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0…

The newline is part of what echo hi has output. echo's job is to output the parameters you pass it, followed by a newline. If you're not interested in whitespace surrounding the process output, you can use strip() like so:

>>> b'hi\n'.strip()
b'hi'
4
  • 1
    How do you get the print() function to print a byte string without a preceeding 'b'? Or do you need to convert it to a unicode string first? Mar 12, 2013 at 23:53
  • I'm curious, when os.popen returns text strings, whether there's a way to make subprocess.Popen also return them, instead of the byte strings. Dec 13, 2014 at 20:58
  • 14
    I'll answer myself, there's an option with cryptic name called universal_newlines that causes the Popen object to accept and return text strings. Dec 13, 2014 at 21:24
  • 3
    @PavelŠimerda While os.popen returns text strings, they are apparently being decoded incorrectly for non-ascii characters, at least on Windows. E.g. running check_output("dir"), extracting a file name from the output and then trying to access it with open will fail if the filename contains German umlauts. Might be a bug.
    – kdb
    May 21, 2015 at 17:44
99

As mentioned before, echo hi actually does return hi\n, which it is an expected behavior.

But you probably want to just get the data in a "right" format and not deal with encoding. All you need to do is pass universal_newlines=True option to subprocess.Popen() like so:

>>> import subprocess
>>> print(subprocess.Popen("echo hi",
                           shell=True,
                           stdout=subprocess.PIPE,
                           universal_newlines=True).communicate()[0])
hi

This way Popen() will replace these unwanted symbols by itself.

8
  • 20
    universal_newlines=True worked like a charm. This should be the accepted answer, in my humble opinion... Aug 17, 2018 at 23:55
  • 5
    It produces extra empty lines.
    – LoMaPh
    Mar 13, 2019 at 23:09
  • 7
    You may need both universal_newlines=True in Popen (to get rid of the b'') and a strip() on the resulting string, if you want to chop the terminating newline.
    – arielf
    Feb 5, 2020 at 5:22
  • 1
    FYI, the documentation says universal_newlines is now just a backwards-compatible alias for the text parameter, which is clearer but only in Python 3.7 and above. Feb 7, 2020 at 0:17
  • It produces extra empty lines because it doesn't work. universal_newlines does not remove \n
    – kol23
    Oct 15, 2020 at 12:55
28

The echo command by default returns a newline character

Compare with this:

print(subprocess.Popen("echo -n hi", \
    shell=True, stdout=subprocess.PIPE).communicate()[0])

As for the b preceding the string it indicates that it is a byte sequence which is equivalent to a normal string in Python 2.6+

http://docs.python.org/3/reference/lexical_analysis.html#literals

1
  • 10
    you don't need '\' inside the parentheses.
    – jfs
    Oct 15, 2015 at 8:31
12

b is the byte representation and \n is the result of echo output.

Following will print only the result data

import subprocess
print(subprocess.Popen("echo hi", shell=True,stdout=subprocess.PIPE).communicate()[0].decode('utf-8').strip())
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.