2

I have a text file with records:

  Data1

  Data2

  ...

  Data50

I have to create a .csv file from the above text file in the following format:

Type |  Count | Name

Def |    u1 |    Data1

Def |    u2  |     Data2

....  |  .....   |  ....

Def  |   u50   | Data50

i need a bash script to generate the .csv file from the text file. I am new to shell scripting! I have gone through basics of awk and sed as well. I have a vague idea like:

#!/bin/bash
type="Def"
x=1
count="u"
for F in ../test.txt
do
    {
       read \n
       echo "$type, $count$x, $..." >> ../test.csv
       x=x+1
    } < $F

done 

I do understand that the field separator is '\n'. I am kind of lost after that.

Thanks!

0

4 Answers 4

3

Your for loop will only loop a single time, it loops over the tokens you list and you only listed one (which looks like a file name, so I'm guessing you want to loop over lines in the file):

#!/bin/bash
type="Def"
x=1
count="u"
while read value; do
   echo "$type, $count$x, $value"
   let x++
done <../test.txt > ../test.csv

You could also use an external utility for the running numbers:

nl ../test.txt |
while read -r x value; do
    echo "$type,$count$x,$value"
done >../test.csv

Redirecting outside the loop is more efficient because the shell won't have to close and reopen the output file.

If you want to pass in a variable file name on the command line, just replace the hardcoded ../test.txt with "$1". You could similarly parametrize the output file name, but I would simply remove the output redirection, and leave it to the caller to decide what to do with the script's output.

If you need to read multiple fields and split on comma, tweak your IFS.

nl -s , ../test.txt |
while IFS=, read -r i first second rest; do
    printf "%i,%s,u%i,%s\n" $i "$first" "$second" "$rest"
done

(printf is recommended over echo, especially if your output requirements are nontrivial. I fail to quote $i specifically to get rid of any leading whitespace added by nl before the line number. Otherwise, you should generally always use double quotes around your variables unless you specifically require the shell to perform whitespace tokenization and wildcard expansion on the value.)

9
  • don't forget the csv header line! which is still easy to add in your solutions
    – Roland
    Feb 24, 2015 at 16:37
  • x=((x+1)) did not work for me, too bad, I don't like expr
    – Roland
    Feb 24, 2015 at 16:39
  • 1
    @Roland use x=$((x+1)) or let x++
    – clt60
    Feb 24, 2015 at 16:52
  • 1
    Thank you so much for taking time. You got it correct, i want to read multiple lines in a file. Like I mentioned I am new to bash scripting. So I tried the first code you provided, its giving me syntax error at 'done'! Can you give me some detail about external utility thing. Feb 24, 2015 at 16:54
  • 1
    There are excellent reasons to omit the header line. As the OP's example didn't have one either, I didn't add one; and would recommend against it if there was a header line.
    – tripleee
    Feb 24, 2015 at 17:39
1

If you have perl installed, the

perl -lnE 'say qq{Def,u$.,"$_"}' < inputfile

will do the job.

demo:

the seq -f 'Some Data%g' 50 will generate lines like:

Some Data1
Some Data2
...
Some Data50

so the

seq -f 'Some Data%g' 50 | perl -lnE 'say qq{Def,u$.,"$_"}'

prints

Def,u1,"Some Data1"
Def,u2,"Some Data2"
...
Def,u49,"Some Data49"
Def,u50,"Some Data50"

I quoted the last field, because in the input you could get , or spaces.

based on @Roland comment, adding a header line:

cat data | (echo 'Type,Count,Name' ; perl -lnE 'say qq{Def,u$.,"$_"}')

or

perl -lnE 'BEGIN{say q{Type,Count,Name}}say qq{Def,u$.,"$_"}'

If you want bash solution, simply use:

cat -n filename | sed 's/ *\(.*\)\t\(.*\)/Def,u\1,"\2"/'

or save the

cat -n - | sed 's/ *\(.*\)\t\(.*\)/Def,u\1,"\2"/'

into some file, like "makecsv" and use it as

./makecsv < data

Ps: hmm.. the @tripleee's nl is shorter as cat -n ;)

1
  • it is easy to also add the csv header line here. Sorry for this, but currently I am working a lot with csv files and it is astonishing how often the header line is omitted for no reason. Even for csv files with something like two dozen columns ...
    – Roland
    Feb 24, 2015 at 17:24
0

save this in a file, e.g. makecsv.rc:

#!/bin/sh
echo Type,Count,Name
x=0
for f in `cat`
do
   x=`expr $x + 1`
   echo Def,u$x,$f
done

then run as:

cat  ../test.txt | ./makecsv.rc > ../test.csv

if needed, you do chmod +x makecsv.rc

The advantage is that the input/output file names are not hardcoded

8
  • 1
    Just use "$1" and "$2" if you don't want to hardcode file names. The cat in backticks is rather horrible.
    – tripleee
    Feb 24, 2015 at 17:38
  • This was super helpful, as I would need to execute it at definite intervals with different files! Thanks a lot! Feb 24, 2015 at 17:38
  • @tripleee you are right. But in my timezone it is definitely time to go home. Please propose how to avoid the TWO cats: in the script and on the command line.
    – Roland
    Feb 24, 2015 at 17:42
  • 1
    I added a snippet to my own answer. Yours could be similarly refactored, but then it would be pretty much the same answer...
    – tripleee
    Feb 24, 2015 at 17:55
  • 1
    @Roland my comment is for the other users, who could use the solution and they could have a space. Remember, the answers are for the future users too. I not want be an critic of your answer, but honestly i agree with @tripleee - the simple cat in backtick is really horrible and it is against any good bash practice. Also, in any modern shell the $(command) is more preferred as the backticks.
    – clt60
    Feb 27, 2015 at 15:19
0

You make the second field with:

x = `expr $x + 1`
$count$x

The entire script becomes:

#!/bin/sh

echo Type,Count,Name > test.csv
x=0
for f in `cat test.txt`
do
   x=`expr $x + 1`
   echo Def,u$x,$f >> test.csv
done

Good Luck!

9
  • Wo! Thanks! I missed that. Thanks for that! But I am stuck at how to write the "Data1, Data2 ... Data50" from the .txt to a new .csv file. Feb 24, 2015 at 16:17
  • I tested this on my desktop pc using Cygnus, while doing stackoverflow on a notebook, so I had to type it over. Did I make a typing error?
    – Roland
    Feb 24, 2015 at 16:24
  • you were right. I corrected using cat, with back quotes. Sorry for answering too fast :-)
    – Roland
    Feb 24, 2015 at 16:27
  • I tried your code, but it seems the for loop is not working with the cat command Feb 24, 2015 at 17:06
  • 1
    @Pbms read can read multiple variables, just list the fields you want. while read -r first second rest; do ... done <file which also removes the pesky problems with for behind the link in the previous comment. If your fields are not whitespace-separated, you can use IFS=, to split on comma, for example. All of this is very well covered in various FAQs.
    – tripleee
    Mar 23, 2017 at 11:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.