124

I have a variable like this:

words="这是一条狗。"

I want to make a for loop on each of the characters, one at a time, e.g. first character="这", then character="是", character="一", etc.

The only way I know is to output each character to separate line in a file, then use while read line, but this seems very inefficient.

  • How can I process each character in a string through a for loop?
1
  • 3
    It might be worth mentioning that we see a lot of newbie questions where the OP thinks this is what they want to do. Very often, a better solution which does not require each character to be processed individually is possible. This is known as an XY Problem and the proper solution is to explain what you actually want to accomplish in your question, not just how to execute the steps you think will help you get there.
    – tripleee
    Jan 10, 2018 at 10:07

16 Answers 16

298

You can use a C-style for loop:

foo=string
for (( i=0; i<${#foo}; i++ )); do
  echo "${foo:$i:1}"
done

${#foo} expands to the length of foo. ${foo:$i:1} expands to the substring starting at position $i of length 1.

9
  • Why do you need two sets of brackets around the for statement for it to work?
    – tgun926
    Aug 16, 2015 at 6:41
  • 3
    I know this is old, but, the two parentheses are required because they allow for arithmetic operations. See here => tldp.org/LDP/abs/html/dblparens.html
    – Hannibal
    Apr 23, 2016 at 7:21
  • 9
    @Hannibal I just wanted to point out that this particular use of double parentheses is actually the bash construct: for (( _expr_ ; _expr_ ; _expr_ )) ; do _command_ ; done and not the same as $((expr)) nor ((expr)). In all three bash constructs, expr is treated the same and $((expr)) is also POSIX.
    – nabin-info
    Jan 22, 2017 at 6:26
  • 2
    @codeforester That has nothing to do with arrays; it's just one of many expressions in bash that is evaluated in an arithmetic context.
    – chepner
    Jun 12, 2017 at 0:39
  • 1
    @CiroSantilliOurBigBook.com It absolutely isn't POSIX. POSIX wasn't an attempt at making a "modern" shell; it was just a standardization of existing features, basically a lowest common denominator you could rely on. Remember: the main purpose of a shell is to run other programs, not process data on its own.
    – chepner
    Oct 17, 2022 at 12:49
59

With sed on dash shell of LANG=en_US.UTF-8, I got the followings working right:

$ echo "你好嗎 新年好。全型句號" | sed -e 's/\(.\)/\1\n/g'
你
好
嗎

新
年
好
。
全
型
句
號

and

$ echo "Hello world" | sed -e 's/\(.\)/\1\n/g'
H
e
l
l
o

w
o
r
l
d

Thus, output can be looped with while read ... ; do ... ; done

edited for sample text translate into English:

"你好嗎 新年好。全型句號" is zh_TW.UTF-8 encoding for:
"你好嗎"     = How are you[ doing]
" "         = a normal space character
"新年好"     = Happy new year
"。全型空格" = a double-byte-sized full-stop followed by text description
5
  • 7
    Nice effort on UTF-8. I didn't need it, but you get my upvote anyway.
    – Jordan
    Aug 19, 2013 at 20:27
  • +1 You can use the for loop on the resulting string from sed.
    – Tyzoid
    Jan 25, 2014 at 19:24
  • 1
    Minor point, In your seds, -e for editing is not needed.
    – Timo
    Mar 31, 2022 at 11:53
  • Or just do this: fold -1<<<"你好嗎 新年好。全型句號"
    – roblogic
    Dec 22, 2022 at 14:04
  • Things don't quite work right with combining marks (e.g. UTF-8 characters from Unicode blocks U+0300-036F, U+1AB0-1AFF, U+1DC0-1DFF, U+20D0-20FF, U+FE20-FE2F) such as with the string "Béégashii bitsįʼ yį́yą́".
    – baltakatei
    Feb 8, 2023 at 12:40
44

${#var} returns the length of var

${var:pos:N} returns N characters from pos onwards

Examples:

$ words="abc"
$ echo ${words:0:1}
a
$ echo ${words:1:1}
b
$ echo ${words:2:1}
c

so it is easy to iterate.

another way:

$ grep -o . <<< "abc"
a
b
c

or

$ grep -o . <<< "abc" | while read letter;  do echo "my letter is $letter" ; done 

my letter is a
my letter is b
my letter is c
2
  • 3
    what about whitespace? Aug 7, 2014 at 1:32
  • What about whitespace? A whitespace character is a character and this loops over all characters. (Though you should take care to use double quotes around any variable or string which contains significant whitespace. More generally, always quote everything unless you know what you are doing.)
    – tripleee
    Jan 10, 2018 at 10:09
36

I'm surprised no one has mentioned the obvious bash solution utilizing only while and read.

while read -n1 character; do
    echo "$character"
done < <(echo -n "$words")

Note the use of echo -n to avoid the extraneous newline at the end. printf is another good option and may be more suitable for your particular needs. If you want to ignore whitespace then replace "$words" with "${words// /}".

Another option is fold. Please note however that it should never be fed into a for loop. Rather, use a while loop as follows:

while read char; do
    echo "$char"
done < <(fold -w1 <<<"$words")

The primary benefit to using the external fold command (of the coreutils package) would be brevity. You can feed it's output to another command such as xargs (part of the findutils package) as follows:

fold -w1 <<<"$words" | xargs -I% -- echo %

You'll want to replace the echo command used in the example above with the command you'd like to run against each character. Note that xargs will discard whitespace by default. You can use -d '\n' to disable that behavior.


Internationalization

I just tested fold with some of the Asian characters and realized it doesn't have Unicode support. So while it is fine for ASCII needs, it won't work for everyone. In that case there are some alternatives.

I'd probably replace fold -w1 with an awk array:

awk 'BEGIN{FS=""} {for (i=1;i<=NF;i++) print $i}'

Or the grep command mentioned in another answer:

grep -o .


Performance

FYI, I benchmarked the 3 aforementioned options. The first two were fast, nearly tying, with the fold loop slightly faster than the while loop. Unsurprisingly xargs was the slowest... 75x slower.

Here is the (abbreviated) test code:

words=$(python -c 'from string import ascii_letters as l; print(l * 100)')

testrunner(){
    for test in test_while_loop test_fold_loop test_fold_xargs test_awk_loop test_grep_loop; do
        echo "$test"
        (time for (( i=1; i<$((${1:-100} + 1)); i++ )); do "$test"; done >/dev/null) 2>&1 | sed '/^$/d'
        echo
    done
}

testrunner 100

Here are the results:

test_while_loop
real    0m5.821s
user    0m5.322s
sys     0m0.526s

test_fold_loop
real    0m6.051s
user    0m5.260s
sys     0m0.822s

test_fold_xargs
real    7m13.444s
user    0m24.531s
sys     6m44.704s

test_awk_loop
real    0m6.507s
user    0m5.858s
sys     0m0.788s

test_grep_loop
real    0m6.179s
user    0m5.409s
sys     0m0.921s
3
  • 1
    character is empty for whitespace with the simple while read solution, which may be problematic if different types of whitespace must be distinguished from each other.
    – pkfm
    Aug 29, 2019 at 1:58
  • 2
    Nice solution. I found that changing read -n1 to read -N1 was needed to handle space characters correctly.
    – nielsen
    Feb 3, 2020 at 8:13
  • I like this solution; however I prefer to avoid typos by putting printf "$words" | while read -n1 character; do echo "$character"; done Aug 18, 2022 at 7:25
22

I believe there is still no ideal solution that would correctly preserve all whitespace characters and is fast enough, so I'll post my answer. Using ${foo:$i:1} works, but is very slow, which is especially noticeable with large strings, as I will show below.

My idea is an expansion of a method proposed by Six, which involves read -n1, with some changes to keep all characters and work correctly for any string:

while IFS='' read -r -d '' -n 1 char; do
        # do something with $char
done < <(printf %s "$string")

How it works:

  • IFS='' - Redefining internal field separator to empty string prevents stripping of spaces and tabs. Doing it on a same line as read means that it will not affect other shell commands.
  • -r - Means "raw", which prevents read from treating \ at the end of the line as a special line concatenation character.
  • -d '' - Passing empty string as a delimiter prevents read from stripping newline characters. Actually means that null byte is used as a delimiter. -d '' is equal to -d $'\0'.
  • -n 1 - Means that one character at a time will be read.
  • printf %s "$string" - Using printf instead of echo -n is safer, because echo treats -n and -e as options. If you pass "-e" as a string, echo will not print anything.
  • < <(...) - Passing string to the loop using process substitution. If you use here-strings instead (done <<< "$string"), an extra newline character is appended at the end. Also, passing string through a pipe (printf %s "$string" | while ...) would make the loop run in a subshell, which means all variable operations are local within the loop.

Now, let's test the performance with a huge string. I used the following file as a source:
https://www.kernel.org/doc/Documentation/kbuild/makefiles.txt
The following script was called through time command:

#!/bin/bash

# Saving contents of the file into a variable named `string'.
# This is for test purposes only. In real code, you should use
# `done < "filename"' construct if you wish to read from a file.
# Using `string="$(cat makefiles.txt)"' would strip trailing newlines.
IFS='' read -r -d '' string < makefiles.txt

while IFS='' read -r -d '' -n 1 char; do
        # remake the string by adding one character at a time
        new_string+="$char"
done < <(printf %s "$string")

# confirm that new string is identical to the original
diff -u makefiles.txt <(printf %s "$new_string")

And the result is:

$ time ./test.sh

real    0m1.161s
user    0m1.036s
sys     0m0.116s

As we can see, it is quite fast.
Next, I replaced the loop with one that uses parameter expansion:

for (( i=0 ; i<${#string}; i++ )); do
    new_string+="${string:$i:1}"
done

The output shows exactly how bad the performance loss is:

$ time ./test.sh

real    2m38.540s
user    2m34.916s
sys     0m3.576s

The exact numbers may very on different systems, but the overall picture should be similar.

13

I've only tested this with ascii strings, but you could do something like:

while test -n "$words"; do
   c=${words:0:1}     # Get the first character
   echo character is "'$c'"
   words=${words:1}   # trim the first character
done
11

It is also possible to split the string into a character array using fold and then iterate over this array:

for char in `echo "这是一条狗。" | fold -w1`; do
    echo $char
done
0
10

The C style loop in @chepner's answer is in the shell function update_terminal_cwd, and the grep -o . solution is clever, but I was surprised not to see a solution using seq. Here's mine:

read word
for i in $(seq 1 ${#word}); do
  echo "${word:i-1:1}"
done
6
#!/bin/bash

word=$(echo 'Your Message' |fold -w 1)

for letter in ${word} ; do echo "${letter} is a letter"; done

Here is the output:

Y is a letter o is a letter u is a letter r is a letter M is a letter e is a letter s is a letter s is a letter a is a letter g is a letter e is a letter

2

To iterate ASCII characters on a POSIX-compliant shell, you can avoid external tools by using the Parameter Expansions:

#!/bin/sh

str="Hello World!"

while [ ${#str} -gt 0 ]; do
    next=${str#?}
    echo "${str%$next}"
    str=$next
done

or

str="Hello World!"

while [ -n "$str" ]; do
    next=${str#?}
    echo "${str%$next}"
    str=$next
done
2

sed works with unicode

IFS=$'\n'
for z in $(sed 's/./&\n/g' <(printf '你好嗎')); do
 echo hello: "$z"
done

outputs

hello: 你
hello: 好
hello: 嗎
0

Another approach, if you don't care about whitespace being ignored:

for char in $(sed -E s/'(.)'/'\1 '/g <<<"$your_string"); do
    # Handle $char here
done
0

Another way is:

Characters="TESTING"
index=1
while [ $index -le ${#Characters} ]
do
    echo ${Characters} | cut -c${index}-${index}
    index=$(expr $index + 1)
done
1
  • This would be sloooow, launching one process for each character. Dec 25, 2021 at 7:11
0

fold and while read are great for the job as shown in some answers here. Contrary to those answers, I think it's much more intuitive to pipe in the order of execution:

echo "asdfg" | fold -w 1 | while read c; do
    echo -n "$c "
done

Outputs: a s d f g

-1

I share my solution:

read word

for char in $(grep -o . <<<"$word") ; do
    echo $char
done
1
  • This is very buggy -- try with a string that contains a *, you'll get files in the current directory. Sep 29, 2018 at 14:57
-3
TEXT="hello world"
for i in {1..${#TEXT}}; do
   echo ${TEXT[i]}
done

where {1..N} is an inclusive range

${#TEXT} is a number of letters in a string

${TEXT[i]} - you can get char from string like an item from an array

2
  • 5
    Shellcheck reports "Bash doesn't support variables in brace range expansions" So this will not work in Bash
    – Bren
    Aug 30, 2018 at 15:50
  • @Bren Seems like a bug to me. Jun 17, 2020 at 20:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.