0

The input text will be:

9842901121
9942501133
9942501199
9942501133
9842901121

At first the input file should be sorted with unique lines only. Then the output to be

'9842901121',()(now,
'9942501133',()(now,
'9942501199',()(now,

If we use ' (a single quote) on the Unix command line, the prompt goes to >. Why? How do we avoid that problem?

2
  • What exactly are you trying to do sort a file from a unix command prompt and remove duplicates?
    – Mech
    Dec 23, 2010 at 21:56
  • 1
    I think you need to take the time to properly explain your problem and how you've attempted to solve it, perhaps adding some formatting to you question using the editor tools. Otherwise, people will simply ignore your question. Dec 23, 2010 at 21:57

2 Answers 2

0

Use a script like this:

#!/bin/sh

sort "$1" | uniq | (
    while read line; do
        echo "'$line',()(now,"
    done
)

Save this in a file called, say, test.sh and then run it like sh test.sh filetoprocess.

1
  • @user394741: I have no idea what you're talking about.
    – cdhowie
    Dec 24, 2010 at 0:45
0

Use sort and sed:

sort -u input.file | sed "s/.*/'&',()(now,/"

Which, it has to be said, is a pretty weird string to concatenate onto the end of anything.

The reason the shell prompt changes to '>' is that it thinks you've not completed the command. It is called the secondary prompt, and is settable via the $PS2 variable. For example, if I typed:

$ sort -u input.file |
> sed "s/.*/'&',()(now,/"
'9842901121',()(now,
'9942501133',()(now,
'9942501199',()(now,
$

When I hit return after the pipe symbol, the shell knows the command is incomplete so it asks for the rest of the input - the sed command in this case.

I used the quotes - double quotes this time - around the sed script to tell the shell where the argument ends. Without those, I get a syntax error, but it is conceivable that the shell would misunderstand things and think the command was incomplete. I normally use single quotes around regex strings; there's less to worry about on the whole. But this time, the replacement text contained single quotes, and then using single quote around the whole string requires:

sort -u input.file | sed 's/.*/'\''&'\'',()(now,/'

which is much harder to write, and read, than the double-quoted version.


From the comments:

cat inputfile.txt | sort | uniq > Inputfile.txt
awk -F" " ' {print (echo ""$1" ,()(now,")}' Inputfile1.txt

In the above command I cannot add ' before and after the number but others work fine.

  1. Please don't abuse cat.
  2. Please remember that sort -u is quicker than sort | uniq.
  3. FYI: There are case-insensitive file systems in the world.
  4. Maybe you had a typo for Inputfile1.txt.
  5. Unless you need the intermediate file, simply pipe the output from the sort phase into the 'mangle' phase.
  6. As I noted in my main answer, you have to be very careful with quotes, especially when you want to print them.
  7. The -F option to awk is only needed when you have multiple fields on a line and the separator is not 'white space' (blanks or tabs).

Using awk instead of sed is perfectly feasible; we just need to be careful with the quotes. The trouble is, the awk script itself needs to include both single and double quotes, which means you have to be ultra-careful. I recommend using sed because you don't have to get both single and double quotes into the script.

sort -u inputfile.txt |
awk '{printf "'\''%s'\'',()(now,\n", $0}'

The first single-quote starts a single-quoted string; there are no special characters in a single-quoted string and the first following single quote terminates it. So, the first double quote is just a regular character. The second single quote is the start of a sequence to remember: '\''. The first of these single quotes terminates the current single-quoted string; the backslash single-quote combo embeds a single quote into the string; the third single-quote in the sequence starts a new single-quoted string. After that, the "%s" is part of the string, then there's another of '\'' sequences to get another single quote into the script; then there is the rest of a printf() format string, followed by a final single quote. By enclosing things in single quotes, we don't have to worry about escaping double quotes, backticks, backslashes and dollar signs in the string. The net result is that awk sees the program text:

{printf "'%s',()(now,\n", $0}

which prints the input data with single quotes around it and the ',()(now,' sequence after it, ending with a newline.

Can you write that with double quotes around the program? Yes, of course you can:

sort -u inputfile.txt |
awk "{printf \"'%s',()(now,\\n\", \$0}"

This is actually two characters shorter than the single quoted version, but there were more opportunities to get it wrong. With single quotes, all I need to do is replace each single quote that should appear in the script with the '\'' sequence; with double quotes, I have to worry about escaping the other special characters too.

A good understanding of single quotes and double quotes is very, very important for shell programming. So too is a clear understanding of which parts of the script are related to the shell sees and which parts are related to what the command (awk in this example, or sed or perl or ...) sees.

3
  • cat inputfie.txt | sort | uniq > Inputfile.txt awk -F" " ' {print (echo ""$1" ,()(now,")}' Inputfile1.txt
    – user394741
    Dec 23, 2010 at 23:58
  • In the above command i cannot add ' before and after the number but others work fine .Kindly help to resolve..
    – user394741
    Dec 24, 2010 at 0:01
  • @user394741: on StackOverflow, the way you show thanks is by upvoting helpful answers, and by accepting the answer that is most helpful. See the FAQ. Dec 24, 2010 at 1:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.