7

I have some text with some leading whitespace on all lines. I want to remove the whitespace from the shortest line (if it's simpler, this requirement could be changed to the first line) and then remove the same amount of whitespace from all other lines.

E.g. I have this text:

    var flatten = function(result, next_array) {
        console.log('current result', result);
        return result.concat(next_array);
    };

    [1, [2], [3, 4]]
        .reduce(flatten, []);

And I want to result in this text:

var flatten = function(result, next_array) {
    console.log('current result', result);
    return result.concat(next_array);
};

[1, [2], [3, 4]]
    .reduce(flatten, []);

Basically, I want to shift the text over until there's at least one line with no whitespace on the left and preserve all other leading whitespace on all other lines.

The use case for this is copying code from the middle of a section of code to paste as an example elsewhere. What I currently do is copy the code, paste into vim with paste insert mode, use << until I get the desired output, and copy the buffer. The same could be done in TextMate with Cmd-[.

What I want is to do this with a shell script so I could, for example, trigger it with a hotkey to take my clipboard contents, remove the desired whitespace, and paste the result.

2
  • One complication for some of the proposed solutions is that the blank line there has zero spaces at the front of it..!
    – beroe
    Commented Oct 12, 2013 at 7:52
  • Very useful question. I like it when answering creates something that I will end up using down the road...
    – beroe
    Commented Oct 12, 2013 at 8:14

5 Answers 5

4

this awk one-liner could do it for you too. it assumes you want to remove at least 1 whitespace. (because I see in your example, there is an empty line, without any leading spaces, but all lines are shifted left anyway.)

test with your example:

kent$  cat f
    var flatten = function(result, next_array) {
        console.log('current result', result);
        return result.concat(next_array);
    };

    [1, [2], [3, 4]]
        .reduce(flatten, []);

kent$  awk  -F '\\S.*' '{l=length($1);if(l>0){if(NR==1)s=l; else s=s>l?l:s;}a[NR]=$0}END{for(i=1;i<=NR;i++){sub("^ {"s"}","",a[i]);print a[i]}}' f
var flatten = function(result, next_array) {
    console.log('current result', result);
    return result.concat(next_array);
};

[1, [2], [3, 4]]
    .reduce(flatten, []);

EDIT

I don't think awk scripts not readable. but you have to know the syntax of awk script. anyway, I am adding some explanation:

The awk script has two blocks, the first block was executed when each line of your file was read. The END block was executed after the last line of your file was read. See commpents below for explanation.

awk  -F '\\S.*'          #using a delimiter '\\S.*'(regex). the first non-empty char till the end of line
                         #so that each line was separated into two fields,
                         #the field1: leading spaces
                         #and the field2: the rest texts

'{                       #block 1
l=length($1);            #get the length of field1($1), which is the leading spaces, save to l
if(l>0){                 #if l >0
if(NR==1)s=l;            #NR is the line number, if it was the first line, s was not set yet, then let s=l
else s=s>l?l:s;}         #else if l<s, let s=l otherwise keep s value
a[NR]=$0                 #after reading each line, save the line in a array with lineNo. as index
}                        #this block is just for get the number of "shortest" leading spaces, in s

END{for(i=1;i<=NR;i++){  #loop from lineNo 1-last from the array a
sub("^ {"s"}","",a[i]);  #replace s number of leading spaces with empty
print a[i]}              #print the array element (after replacement)
}' file                  #file is the input file
11
  • This isn't working for me. Using Ubuntu precise with Bash 4.2. Commented Oct 12, 2013 at 11:49
  • @bashophil I don't know which awk do you use, if you have gnu awk, try awk --re-interval -F.... or report your awk version
    – Kent
    Commented Oct 12, 2013 at 12:05
  • @bashophil I don't have so old version here, so I cannot test it for you. what you can try is, instead of sub("^ {"s"}"..., you build an empty string with length s, it is easy to be done, and sub("^" TheString, "")....
    – Kent
    Commented Oct 12, 2013 at 14:56
  • 1
    @ThomasUpton explanation added. hope it helps.
    – Kent
    Commented Oct 13, 2013 at 10:08
  • 1
    @ThomasUpton I guess "better" means portable. we could remove the gawk specific features and using POSIX awk basic features.
    – Kent
    Commented Oct 14, 2013 at 20:28
2

These functions can be defined in your .bash_profile to have access to them anywhere, rather than creating a script file. They don't require first line to match:

shiftleft(){
    len=$(grep -e "^[[:space:]]*$" -v $1 | sed -E 's/([^ ]).*/x/' | sort -r | head -1 | wc -c)
    cut -c $(($len-1))- $1
}

Usage: shiftleft myfile.txt

This works with a file, but would have to be modified to work with pbpaste piped to it...

NOTE: Definitely inspired by the answer of @JoSo but fixes the errors in there. (Uses sort -r and cut -c N- and missing $ on len and doesn't get hung up by blank lines w/o white space.)


EDIT: A version to work with the contents of the clipboard on OSX:

shiftclip(){
    len=$(pbpaste | grep -e "^[[:space:]]*$" -v | sed -E 's/([^ ]).*/x/' | sort -r | head -1 | wc -c)
    pbpaste | cut -c $(($len-1))-
}

Usage for this version: Copy the text of interest and then type shiftclip. To copy output directly back to the clipboard, do shiftclip | pbcopy

2
  • The first solution will not accept input from standard input, which is however a common use case for a filter. It's not terribly hard to fix, at the expense of losing some elegance (use a temporary file).
    – tripleee
    Commented Oct 12, 2013 at 11:39
  • Cool. Add ` | pbcopy` to the end to shift the clipboard entirely "in place."
    – beroe
    Commented Oct 13, 2013 at 1:51
1
len=$(sed 's/[^ ].*//' <"$file"| sort | head -n1 | wc -c)
cut -c "$((len))"- <"$file"

Or, a bit less readable but avoiding the overhead of a sort:

len=$(awk 'BEGIN{m=-1} {sub("[^ ].*", "", $0); l = length($0); m = (m==-1||l<m) ? l : m; } END { print m+1 }' <"$file")
cut -c "$((len))"- <"$file"
3
  • This isn't working but sounds promising. The brackets aren't fitting. Commented Oct 12, 2013 at 2:06
  • Agree this is close, but ... need to move the $file over before the first pipe and get rid of the closeparen... Also the cut is only returning the first N characters instead of everything afterwards. There needs to be a dash after the number...
    – beroe
    Commented Oct 12, 2013 at 7:39
  • Sorry for posting slightly broken example. Fixed now.
    – Jo So
    Commented Oct 12, 2013 at 18:48
0

Hmmm... this isn't very beautiful, and also assumes you have access to Bash, but if you can live with your "first line" rule:

#!/bin/bash
file=$1
spaces=`head -1 $file | sed -re 's/(^[ ]+)(.+)/\1/g'`
cat $file | sed -re "s/^$spaces//"

It also assumes only the space character (i.e., you need to tweak for tabs) but you get the idea.

Assuming your example is in a file snippet.txt, put the bash code in a script (e.g., "shiftleft") , "chmod +x" the script, then run with:

shiftleft snippet.txt
1
  • remember that #!/bin/bash is always wrong; #!/usr/bin/env bash is better but #!/bin/sh is best; also $1 needs to be quoted and the cat is useless Commented Oct 14, 2013 at 13:26
0

This is a one-liner (albeit mildly cryptic) in perl:

Given a file with indented text (/tmp/indented.txt, say):

    var flatten = function(result, next_array) {
        console.log('current result', result);
        return result.concat(next_array);
    };

    [1, [2], [3, 4]]
        .reduce(flatten, []);

You can trim the maximum common leading whitespace with:

perl -p -0777 -e '/^(\h+).*(\v(\1.*)?)*$/ && s/^$1//mg;' < /tmp/indented.txt

-0777 slurps in the entire input as a single string. The first regex searches for common leading horizontal whitespace, and the second removes it from all lines.

The output now looks like:

var flatten = function(result, next_array) {
    console.log('current result', result);
    return result.concat(next_array);
};

[1, [2], [3, 4]]
    .reduce(flatten, []);

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.