4

I am executing multiple "make" files from my python script. A sample script is as follows :

print("Calling make file")

call(["make"])

and the output will be :

Calling make file

Starting make

cd src && make distclean

make[1]: Entering directory '/home/tejas/WLANrepo/src/3rdparty/redis-stable/src' rm -rf redis-server redis-sentinel redis-cli redis-benchmark redis-check-dump redis-check-aof *.o *.gcda *.gcno *.gcov redis.info lcov-html (cd ../deps && make distclean)'

I want the entire output to be redirected to a log file. I tried :

class Logger(object):

    def __init__(self):
        self.terminal = sys.stdout
        self.log = open("Buildexec_logfile.log", "a")

    def write(self, message):
        self.terminal.write(message)
        self.log.write(message)

but this will redirect only those statements that are under "print". But the output of the make file that gets called is not getting redirected.

Python: python 2.7, OS: CentOS

2 Answers 2

2

You need to redirect output of processes executed by python (in your case, make). See documentation for subprocess module and specifically, the redirection parameters (stdin=..., stdout=..., stderr=...) of its methods. Depending on your needs you may completely discard subprocess output (DEVNULL), or collect it to a variable inside python process (PIPE) or send it to another stream.

3
  • I just tried it and there is a strange (maybe not so strange, the module is called subprocess, so they are probably running in different threads) thing happening - I have print before call, though in log file messages are not in this order. Do you think there's a simple way to arrange them the way they should be? Sep 29, 2015 at 12:56
  • Well, I just could propose the following explanation: when you run a process, it's run asynchronously and output from bother processes goes to the console completely independently. You could first collect all of a subprocess output waiting its termination, and then send everything to console in an order you need. I could discess the subject more specifically, but I need a code to observe. Sep 29, 2015 at 12:57
  • I used stdout itself to call the above mentioned class. from conf.script_logging import Logger sys.stdout = Logger() A log file is created but only print statements are printed.
    – Tejas
    Sep 30, 2015 at 5:28
0

To capture regular output of the command we can use the subprocess.check_output() function. The synchronization is forced because the function must wait for the output of the called sub-process, however if it fails (returns the error message you want to capture) you will need to handle the CalledProcessError that is raised. Fortunately the CalledProcessError exception has an output attribute that stored the error message.

import subprocess
with open('logfile.log','w') as log:
    for makefile in multiple_makefiles:
        log.write("calling {}".format(makefile)
        try:
            log_info = subprocess.check_output(["make",makefile],stderr=STDOUT)
        except subprocess.CalledProcessError as e:
            log_info = e.output
        log.write(log_info)

The try will give log_info the output when the make instance exits cleanly and the except catches the error when it does not. Either way you cant move on until the log_info is written to the log file, so there should be no issues with synchronization

I would also add that if you are writing this script for utility reasons like:

Dang. I really need this compiled now and the makefiles that shipped with it aren't working I need to gather the output to debug this Then go ahead and get it finished in whatever way is most useful or the quickest for you

BUT

If you are really doing something more like:

I need a setup script to automate building my software, and I will ship it with the software for the foreseeable future

Then stop, don't re-invent the wheel you are looking for cmake. (and will log output for you as well)

3
  • Hi Ajay, my bad, should have been more clear. I am executing and installing a host of other scripts as well along with make files. I'll try out the first option.
    – Tejas
    Sep 30, 2015 at 5:10
  • @Tejas ahh I see, well in that case the general format of this code should be able to help out!
    – Ajay
    Sep 30, 2015 at 16:08
  • in the first piece of code, could you please explain the for loop? Why are you creating a list multiple_makefiles?
    – Tejas
    Oct 9, 2015 at 11:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.