129

Obviously Python is more user friendly, a quick search on google shows many results that say that, as Python is byte-compiled is usually faster. I even found this that claims that you can see an improvement of over 2000% on dictionary-based operations.

What is your experience on this matter? In which kind of task each one is a clear winner?

1
  • 2
    Python is drastically faster on text processing, which is a common operation. If I perform the same search 10000 times on each language, on Bash it takes 1m24s, on Python 636ms. This is because Bash use a sub-process for each operation of the text processing, which is slow to create. Oct 22, 2021 at 4:18

11 Answers 11

115

Typical mainframe flow...

Input Disk/Tape/User (runtime) --> Job Control Language (JCL) --> Output Disk/Tape/Screen/Printer
                                   |                          ^
                                   v                          |
                                   `--> COBOL Program --------' 

Typical Linux flow...

Input Disk/SSD/User (runtime) --> sh/bash/ksh/zsh/... ----------> Output Disk/SSD/Screen/Printer
                                   |                          ^
                                   v                          |
                                   `--> Python script --------'
                                   |                          ^
                                   v                          |
                                   `--> awk script -----------'
                                   |                          ^
                                   v                          |
                                   `--> sed script -----------'
                                   |                          ^
                                   v                          |
                                   `--> C/C++ program --------'
                                   |                          ^
                                   v                          |
                                   `--- Java program ---------'
                                   |                          ^
                                   v                          |
                                   :                          :

Shells are the glue of Linux

Linux shells like sh/ksh/bash/... provide input/output/flow-control designation facilities much like the old mainframe Job Control Language... but on steroids! They are Turing complete languages in their own right while being optimized to efficiently pass data and control to and from other executing processes written in any language the O/S supports.

Most Linux applications, regardless what language the bulk of the program is written in, depend on shell scripts and Bash has become the most common. Clicking an icon on the desktop usually runs a short Bash script. That script, either directly or indirectly, knows where all the files needed are and sets variables and command line parameters, finally calling the program. That's a shell's simplest use.

Linux as we know it however would hardly be Linux without the thousands of shell scripts that startup the system, respond to events, control execution priorities and compile, configure and run programs. Many of these are quite large and complex.

Shells provide an infrastructure that lets us use pre-built components that are linked together at run time rather than compile time. Those components are free-standing programs in their own right that can be used alone or in other combinations without recompiling. The syntax for calling them is indistinguishable from that of a Bash builtin command, and there are in fact numerous builtin commands for which there is also a stand-alone executable on the system, often having additional options.

There is no language-wide difference between Python and Bash in performance. It entirely depends on how each is coded and which external tools are called.

Any of the well known tools like awk, sed, grep, bc, dc, tr, etc. will leave doing those operations in either language in the dust. Bash then is preferred for anything without a graphical user interface since it is easier and more efficient to call and pass data back from a tool like those with Bash than Python.

Performance

It depends on which programs the Bash shell script calls and their suitability for the subtask they are given whether the overall throughput and/or responsiveness will be better or worse than the equivalent Python. To complicate matters Python, like most languages, can also call other executables, though it is more cumbersome and thus not as often used.

User Interface

One area where Python is the clear winner is user interface. That makes it an excellent language for building local or client-server applications as it natively supports GTK graphics and is far more intuitive than Bash.

Bash only understands text. Other tools must be called for a GUI and data passed back from them. A Python script is one option. Faster but less flexible options are the binaries like YAD, Zenity, and GTKDialog.

While shells like Bash work well with GUIs like Yad, GtkDialog (embedded XML-like interface to GTK+ functions), dialog, and xmessage, Python is much more capable and so better for complex GUI windows.

Summary

Building with shell scripts is like assembling a computer with off-the-shelf components the way desktop PCs are.

Building with Python, C++ or most any other language is more like building a computer by soldering the chips (libraries) and other electronic parts together the way smartphones are.

The best results are usually obtained by using a combination of languages where each can do what they do best. One developer calls this "polyglot programming".

7
  • 65
    I'm failing to recognise how can it be accepted answer. It does not provide any insights about which tasks those two are more suited for.
    – vigilancer
    Oct 7, 2016 at 23:59
  • 2
    @vigilancer I hope the modifications and additions just posted are helpful. Oct 8, 2016 at 2:20
  • 2
    While I agree with other comments, this doesn't exactly answer the question. This is one of the best answers I have ever read! Feb 13, 2019 at 7:55
  • @vigilancer - yep - my thoughts exactly - this answer is just rambling - there is some insight in there but not too deep.
    – codeepic
    Jun 5, 2021 at 22:31
  • @Daniel Please edit your answer unless 'bwsh ' is some shell I'm unfamiliar with. Assuming you mean 'bash', Python does a good job of straddling the line between an interpreted language and a compiled one with a massive number of libraries that wrap binary C/C++ libraries. Bash does not have that facility and so must execute other commands that do... many of them written in Python since it does a good job with GTK+ graphics (e.g. YAD) Jun 9, 2021 at 2:25
83

Generally, bash works better than python only in those environments where python is not available. :)

Seriously, I have to deal with both languages daily, and will take python instantly over bash if given the choice. Alas, I am forced to use bash on certain "small" platforms because someone has (mistakenly, IMHO) decided that python is "too large" to fit.

While it is true that bash might be faster than python for some select tasks, it can never be as quick to develop with, or as easy to maintain (at least after you get past 10 lines of code or so). Bash's sole strong point wrt python or ruby or lua, etc., is its ubiquity.

3
  • 5
    Isn't Python already on every Linux/Unix, even MacOS? I'm curious what operations are faster in bash - from what I understood, its calling up different separate commands makes it much slower than Python's os or shutil module commands.
    – NoBugs
    Feb 15, 2015 at 4:42
  • 2
    @NoBugs It would definitely not be on every single Linux/Unix distribution. It almost certainly comes on every major linux distribution (e.g. debian-based distributions, slackware, etc.) and Mac OS X, however, if you build your own iso with yocto (yoctoproject.org), then you could not have it, as you customize every package yourself. But it's probably safe to say that for any major Unix OS now-a-days, it will come installed with python2 (at least) and perhaps python3 as well.
    – dylnmc
    Feb 26, 2016 at 13:42
  • 1
    Python is an excellent scripting language for complex tasks such as a full-featured GUI. Equally important, it enforces good programming practices so programs are easier to maintain. Bash requires the imposition of good practices learned elsewhere in order to be maintainable. In so doing, and using a GUI dialog utility or Python for UI, gives superior performance (via extremely fast utility programs called from Bash) as well as a good UX. Jul 22, 2018 at 1:56
42

When you writing scripts performance does not matter (in most cases).
If you care about performance 'Python vs Bash' is a false question.

Python:
+ easier to write
+ easier to maintain
+ easier code reuse (try to find universal error-proof way to include files with common code in sh, I dare you)
+ you can do OOP with it too!
+ easier arguments parsing. well, not easier, exactly. it still will be too wordy to my taste, but python have argparse facility built in.
- ugly ugly 'subprocess'. try to chain commands and not to cry a river how ugly your code will become. especially if you care about exit codes.

Bash:
+ ubiquity, as was said earlier, indeed.
+ simple commands chaining. that's how you glue together different commands in a simple way. Also Bash (not sh) have some improvements, like pipefail, so chaining is really short and expressive.
+ do not require 3rd-party programs to be installed. can be executed right away.
- god, it's full of gotchas. IFS, CDPATH.. thousands of them.

If one writing a script bigger than 100 LOC: choose Python
If one need path manipulation in script: choose Python(3)
If one need somewhat like alias but slightly complicated: choose Bash/sh

Anyway, one should try both sides to get the idea what are they capable of.

Maybe answer can be extended with packaging and IDE support points, but I'm not familiar with this sides.

As always you have to choose from turd sandwich and giant douche. And remember, just a few years ago Perl was new hope. Where it is now.

9
  • 7
    Yes, A code with bash lives forever. I coded a lot of Perl, they're useless now. May 18, 2017 at 12:07
  • 1
    Just for perspective... The current largest script I've wriiten, which I use all day every day, weighs in at 4121 lines of actual, non-comment or blank line bash code. With the extensive comments and such, makes it 7261 lines. It is accompanied by a help file of manpage-like docs for every function that is another 6650 lines. Every function has an option that can instantly retrieve and display its help text in the best available output form that currently includes 3 versions of YAD, Zenity, dialog or just plain CLI text. I call it 'kit'. it's on version 44 as of this writing. Nov 20, 2018 at 6:20
  • This is heavy! (c)
    – vigilancer
    Nov 30, 2018 at 22:24
  • 1
    I don't think LoC is really the decisioning factor to choose Python. More, how complex is the task you are doing? If you're just chaining 100 commands that's probably fine, if it's only 30 LoC in bash but it could be easier to understand in Python - use python.
    – JFord
    Nov 8, 2019 at 19:08
  • @Akito it is okay, when nothing touches it. but there are quit a few situations where things could go wrong. you set it to non-default and forgot to clear it. something outside changed it but your script relies on default, and so on. one always has to keep IFS in mind, because some tools are implicitly use it.
    – vigilancer
    May 21, 2020 at 3:58
39

Developer efficiency matters much more to me in scenarios where both bash and Python are sensible choices.

Some tasks lend themselves well to bash, and others to Python. It also isn't unusual for me to start something as a bash script and change it to Python as it evolves over several weeks.

A big advantage Python has is in corner cases around filename handling, while it has glob, shutil, subprocess, and others for common scripting needs.

1
  • 8
    The question aimed at "performance-wise" comparison which implies machine performance and not developer performance. See my performance tests in another answer. Aug 6, 2014 at 23:08
38

Performance-wise bash outperforms python in the process startup time.

Here are some measurements from my core i7 laptop running Linux Mint:

Starting process                       Startup time

empty /bin/sh script                   1.7 ms
empty /bin/bash script                 2.8 ms
empty python script                    11.1 ms
python script with a few libs*         110 ms

*Python loaded libs are: os, os.path, json, time, requests, threading, subprocess

This shows a huge difference however bash execution time degrades quickly if it has to do anything sensible since it usually must call external processes.

If you care about performance use bash only for:

  • really simple and frequently called scripts
  • scripts that mainly call other processes
  • when you need minimal friction between manual administrative actions and scripting - fast check a few commands and place them in the file.sh
2
  • 2
    ... and /bin/echo outperforms bash by such a magnitude, it's difficult to measure. So instead of running bash, you can use /bin/echo mycommand > named_pipe (output commands/messages to a named pipe or socket) ... and have a background Python process reading commands/instructions from that pipe and running them. So bash is not really a good "startup cost optimization". Jul 4, 2015 at 15:15
  • 1
    Usually you are supposed to use threads instead of processes when the task is really short and quick. Multiple processes are a high level thing and as long as starting one is within a half a second, that seems pretty reasonable for the most part, wouldn't you say? Apr 2, 2018 at 2:58
21

Bash is primarily a batch / shell scripting language with far less support for various data types and all sorts of quirks around control structures -- not to mention compatibility issues.

Which is faster? Neither, because you are not comparing apples to apples here. If you had to sort an ascii text file and you were using tools like zcat, sort, uniq, and sed then you will smoke Python performance wise.

However, if you need a proper programming environment that supports floating point and various control flow, then Python wins hands down. If you wrote say a recursive algorithm in Bash and Python, the Python version will win in an order of magnitude or more.

6
  • 16
    So the whole moral of my rant is: use the right tool for the right job.
    – Justin
    Mar 11, 2010 at 15:40
  • 2
    floating point is supported with tools like awk, bc and with shells like zsh/ksh, so why do you say Python wins hands down?
    – ghostdog74
    Mar 11, 2010 at 15:44
  • 5
    Because those tools are not Bash. I was pointing out a distinct difference. Those tools are used in a shell script, but native Bash itself does not support floating point.
    – Justin
    Mar 11, 2010 at 15:52
  • 3
    No. Try it yourself. gzip a large log file and use zcat, sort, etc to do some filtering and then use the native Python libs. It's significantly faster using the native tools.
    – Justin
    Mar 11, 2010 at 16:26
  • 7
    @justin, yes, these tools are not Bash but they have been around since ancient times and are often used in shell scripting. if you want floating point, use awk/bc. Its a combination of these tools that make shell scripting just as powerful as Python.
    – ghostdog74
    Mar 11, 2010 at 17:06
16

I'm posting this late answer primarily because Google likes this question.

I believe the issue and context really should be about the workflow, not the tools. The overall philosophy is always "Use the right tool for the job." But before this comes one that many often forget when they get lost in the tools: "Get the job done."

When I have a problem that isn't completely defined, I almost always start with Bash. I have solved some gnarly problems in large Bash scripts that are both readable and maintainable.

But when does the problem start to exceed what Bash should be asked to do? I have some checks I use to give me warnings:

  1. Am I wishing Bash had 2D (or higher) arrays? If yes, it's time to realize that Bash is not a great data processing language.
  2. Am I doing more work preparing data for other utilities than I am actually running those utilities? If yes, time again to realize Bash is not a great data processing language.
  3. Is my script simply getting too large to manage? If yes, it is important to realize that while Bash can import script libraries, it lacks a package system like other languages. It's really a "roll your own" language compared to most others. Then again, it has a enormous amount of functionality built-in (some say too much...)

The list goes on. Bottom-line, when you are working harder to keep your scripts running that you do adding features, it's time to leave Bash.

Let's assume you've decided to move your work to Python. If your Bash scripts are clean, the initial conversion is quite straightforward. There are even several converters / translators that will do the first pass for you.

The next question is: What do you give up moving to Python?

  1. All calls to external utilities must be wrapped in something from the subprocess module (or equivalent). There are multiple ways to do this, and until 3.7 it took some effort to get it right (3.7 improved subprocess.run() to handle all common cases on its own).

  2. Surprisingly, Python has no standard platform-independent non-blocking utility (with timeout) for polling the keyboard (stdin). The Bash read command is an awesome tool for simple user interaction. My most common use is to show a spinner until the user presses a key, while also running a polling function (with each spinner step) to make sure things are still running well. This is a harder problem than it would appear at first, so I often simply make a call to Bash: Expensive, but it does precisely what I need.

  3. If you are developing on an embedded or memory-constrained system, Python's memory footprint can be many times larger than Bash's (depending on the task at hand). Plus, there is almost always an instance of Bash already in memory, which may not be the case for Python.

  4. For scripts that run once and exit quickly, Python's startup time can be much longer than Bash's. But if the script contains significant calculations, Python quickly pulls ahead.

  5. Python has the most comprehensive package system on the planet. When Bash gets even slightly complex, Python probably has a package that makes whole chunks of Bash become a single call. However, finding the right package(s) to use is the biggest and most daunting part of becoming a Pythonista. Fortunately, Google and StackExchange are your friends.

12

If you are looking to cobble together a quick utility with minimal effort, bash is good. For a wrapper round an application, bash is invaluable.

Anything that may have you coming back over and over to add improvements is probably (though not always) better suited to a language like Python as Bash code comprising over a 1000 lines gets very painful to maintain. Bash code is also irritating to debug when it gets long.......

Part of the problem with these kind of questions is, from my experience, that shell scripts are usually all custom tasks. There have been very few shell scripting tasks that I have come across where there is already a solution freely available.

9

There are 2 scenario's where Bash performance is at least equal I believe:

  • Scripting of command line utilities
  • Scripts which take only a short time to execute; where starting the Python interpreter takes more time than the operation itself

That said, I usually don't really concern myself with performance of the scripting language itself. If performance is a real issue you don't script but program (possibly in Python).

1
  • But python is a scripting language, so you do script. If you want speed, you go with C, C++, or luajit, if you want to stick to scripting.
    – Daniel
    Jun 6, 2021 at 22:55
4

I don't know if this is accurate, but I have found that python/ruby works much better for scripts that have a lot of mathematical computations. Otherwise you have to use dc or some other "arbitrary precision calculator". It just becomes a very big pain. With python you have much more control over floats vs ints and it is much easier to perform a lot of computations and sometimes.

In particular, I would never work with a bash script to handle binary information or bytes. Instead I would use something like python (maybe) or C++ or even Node.JS.

1
  • Bash arithmetic is strictly integer so you have to do floating point operations by calling something else (like awk or dc) and capturing the output from it. Simple monetary things can often be done internally by just multiplying by 100 and adjusting the decimal point in the output. Sep 22, 2016 at 6:35
-1

Performance wise both can do equally the same, so the question becomes which saves more development time?

Bash relies on calling other commands, and piping them for creating new ones. This has the advantage that you can quickly create new programs just with the code borrowed from other people, no matter what programming language they used.

This also has the side effect of resisting change in sub-commands pretty well, as the interface between them is just plain text.

Additionally Bash is very permissive on how you can write on it. This means it will work well for a wider variety of context, but it also relies on the programmer having the intention of coding in a clean safe manner. Otherwise Bash won't stop you from building a mess.

Python is more structured on style, so a messy programmer won't be as messy. It will also work on operating systems outside Linux, making it instantly more appropriate if you need that kind of portability.

But it isn't as simple for calling other commands. So if your operating system is Unix most likely you will find that developing on Bash is the fastest way to develop.

When to use Bash:

  • It's a non graphical program, or the engine of a graphical one.
  • It's only for Unix.

When to use Python:

  • It's a graphical program.
  • It shall work on Windows.

Not the answer you're looking for? Browse other questions tagged or ask your own question.