123

Is there some sort of character limit imposed in bash (or other shells) for how long an input can be? If so, what is that character limit?

I.e. Is it possible to write a command in bash that is too long for the command line to execute? If there is not a required limit, is there a suggested limit?

1
  • 3
    The input limit is very different from the OS-level argument limit (note that some things other than arguments, such as environment variables, also apply towards that one). The generated command passed to the operating system can have more or fewer characters than the shell command that generated it. Nov 9, 2015 at 22:16

4 Answers 4

163

The limit for the length of a command line is not imposed by the shell, but by the operating system. This limit is usually in the range of hundred kilobytes. POSIX denotes this limit ARG_MAX and on POSIX conformant systems you can query it with

$ getconf ARG_MAX    # Get argument limit in bytes

E.g. on Cygwin this is 32000, and on the different BSDs and Linux systems I use it is anywhere from 131072 to 2621440.

If you need to process a list of files exceeding this limit, you might want to look at the xargs utility, which calls a program repeatedly with a subset of arguments not exceeding ARG_MAX.

To answer your specific question, yes, it is possible to attempt to run a command with too long an argument list. The shell will error with a message along "argument list too long".

Note that the input to a program (as read on stdin or any other file descriptor) is not limited (only by available program resources). So if your shell script reads a string into a variable, you are not restricted by ARG_MAX. The restriction also does not apply to shell-builtins.

13
  • 1
    @KrzysztofJabłoński Unlikely, because the contents of LONG_VAR are passed on stdin--and that is done entirely in the shell; it is not expanded as an argument to cmd, so the ARG_MAX limit for fork()/exec() does not come into play. It is easy to try yourself: create a variable with contents exceeding ARG_MAX and run your command.
    – Jens
    May 14, 2015 at 13:06
  • 2
    Here's the clarification, for the record: for an 8 megabyte m4a file, I did: blah="$(cat /home/schwager/Music/Recordings/20090420\ 131623.m4a)"; cat <<< $blah >/dev/null . Note no error.
    – Mike S
    Nov 9, 2015 at 22:07
  • 4
    Little caveat. The environment variables also count. sysconf manpage > It is difficult to use ARG_MAX because it is not specified how much > of the argument space for exec(3) is consumed by the user's > environment variables.
    – Gerrit
    Jan 19, 2018 at 9:21
  • 3
    @user188737 I feel like it is a pretty big caveat in BUGS. For example xargs on macOS 10.12.6 limits how much it tries to put in one exec() to ARG_MAX - 4096. So scripts using xargs might work, until one day when someone puts too much stuff in the environment. Running into this now (work around it with: xargs -s ???).
    – neuralmer
    Feb 20, 2018 at 22:15
  • 2
    @Jens Your answer talks about fork()/exec() limitations, not about how much a shell can handle on an input line (interactive or not). – So this is not answering the question. (I do see that some commands to the shell invoke other programs and thus arguments get passed there, but this is a different story.) Apr 15, 2019 at 21:31
52

Ok, Denizens. So I have accepted the command line length limits as gospel for quite some time. So, what to do with one's assumptions? Naturally- check them.

I have a Fedora 22 machine at my disposal (meaning: Linux with bash4). I have created a directory with 500,000 inodes (files) in it each of 18 characters long. The command line length is 9,500,000 characters. Created thus:

seq 1 500000 | while read digit; do
    touch $(printf "abigfilename%06d\n" $digit);
done

And we note:

$ getconf ARG_MAX
2097152

Note however I can do this:

$ echo * > /dev/null

But this fails:

$ /bin/echo * > /dev/null
bash: /bin/echo: Argument list too long

I can run a for loop:

$ for f in *; do :; done

which is another shell builtin.

Careful reading of the documentation for ARG_MAX states, Maximum length of argument to the exec functions. This means: Without calling exec, there is no ARG_MAX limitation. So it would explain why shell builtins are not restricted by ARG_MAX.

And indeed, I can ls my directory if my argument list is 109948 files long, or about 2,089,000 characters (give or take). Once I add one more 18-character filename file, though, then I get an Argument list too long error. So ARG_MAX is working as advertised: the exec is failing with more than ARG_MAX characters on the argument list- including, it should be noted, the environment data.

7
  • Hmm. I hadn't read the existing answer to imply that builtins were subject to the constraint in question, but can certainly see how someone could. Nov 9, 2015 at 22:17
  • 6
    Yes, I think it's difficult to remember- especially for newer command-line afficianados- that the situation of calling a bash builtin vs. fork/exec'ing a command is different in non-obvious ways. I wanted to make that clear. One question that I invariably get in a job interview (as a Linux Sysadmin) is, "So I got a bunch of files in a directory. How do I loop over all of them..." The questioner is invariably driving towards the line length limit and wants a find/while or xargs solution. In the future I'm going to say, "ah hell- just use a for loop. It can handle it!" :-)
    – Mike S
    Nov 9, 2015 at 22:28
  • @MikeS while you could do a for loop, if you can use a find-xargs combo you will fork a lot less and will be faster. ;-) Jan 12, 2016 at 0:43
  • 4
    @LesterCheung for f in *; do echo $f; done won't fork at all (all builtins). So I don't know that a find-xargs combo will be faster; it hasn't been tested. Indeed, I don't know what the OP's problem set is. Maybe find /path/to/directory won't be useful to him because it will return the file's pathname. Maybe he likes the simplicity of a for f in * loop. Regardless, the conversation is about line input limit- not efficiency. So let's stay on topic, which pertains to command line length.
    – Mike S
    Jan 12, 2016 at 16:26
  • FWIW, the problem, as I recall, was just trying to write a shell in C, and determine how long I should allow inputs to be. Jul 7, 2017 at 21:13
2

There is a buffer limit of something like 1024. The read will simply hang mid paste or input. To solve this use the -e option.

http://linuxcommand.org/lc3_man_pages/readh.html

-e use Readline to obtain the line in an interactive shell

Change your read to read -e and annoying line input hang goes away.

4
  • 2
    This isn't about read: "I.e. Is it possible to write a command in bash that is too long for the command line to execute?" Aug 6, 2018 at 6:43
  • 1
    @ChaiT.Rex you are sort of correct, but here is the thing: try running Bash interactively without Readline, i.e. bash --noediting, and at the new prompt try running the command echo somereallylongword, where somereallylongword is longer than 4090 characters. Tried on Ubuntu 18.04, the word got truncated, so obviously it does have something to do with Readline not being enabled.
    – Amir
    May 26, 2020 at 12:45
  • @Amir Interesting! You are correct! I attempted to edit the answer, but then I realized that the -e option doesn't apply to bash in this context (in bash, it exits the shell immediately on error). And I'm not sure why Paul pivoted to read. Anyway, there is a buffer limit of between 4-5000 characters when bash is started with --noreadline. That's a side effect that I didn't know about or expect.
    – Mike S
    Oct 20, 2020 at 18:45
  • upvote because read was the problem child I was chasing around the room and this answer got me to the right place. thanks!
    – keen
    Feb 25, 2021 at 17:44
0

In the old days, tcsh had a limit of 1024 characters per command line, which made it difficult if you had a very long $PATH. I was forced to rebuild a private version of tcsh with the buffer size increased to allow users to have long $PATH settings. That was 2 decades ago. That was when I gave up using tcsh, and switched to zsh which did not have that limitation. Now I just use plain old bash because it is good enough.

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.