For a full list of BASHing data blog posts see the index page.     RSS

How to move selected lines within a file

If you're working with a GUI text editor and want to move a particular line from one place to another, you might use cut and paste. Cut out the line, delete the gap that's left behind, insert a blank line at the destination place and paste into the blank the line from the clipboard.

If you're working with a text file on the command line (not in an editor like emacs or vim) and don't want to use a clipboard, line-moving can be done with a single AWK command.

For example, suppose I have the following file, called "demo":

146|fish|BG fillets|3|15.00|45.00
119|meat|liver paste|1|2.10|2.10
6253|veg|carrot bunch|2|4.90|9.80
8847|fish|tin tuna|4|1.50|6.00
3776|veg|pak choy|2|2.50|5.00
534|fish|tin tuna|1|1.50|1.50
1221|meat|pork slices|8|4.20|33.60

and I want the "apple" line to follow the second "banana" line without a change in the order of the other lines. The following AWK command will do the job by processing "demo" twice: once to store the "apple" line in a variable, and again to do the insertion and deletion.

awk -F"|" 'FNR==NR {if ($1=="295") {x=$0; next} else next} \
$1=="0039" {$0=$0 RS x} $1!="295" {print}' demo demo


(I've used red arrows above to highlight the "apple" line before and after moving. For another way to highlight the line, see the last section of this blog post.)
The FNR==NR method of processing two files separately is explained here. AWK processes "demo" the first time and looks for the line in which "295" is the entry in the first pipe-delimited field ($1=="295"). When the line is found, the whole line is stored in the variable "x"; instead of printing the line (AWK's default), AWK follows the next instruction and moves to the next line. At any other line, the instruction is just next: don't print, just move to the next line.
When AWK moves to "demo" for the second time (FNR no longer equals NR), it looks for a line in which the field 1 entry is "0039". When that line is found, it's replaced by the same line, a newline (the default record separator, or RS) and the line stored in the variable "x". AWK moves to its final instruction, which is to print any line where the field 1 entry isn't "295", thus deleting the "295" line. The printing will include the modified line beginning with "0039", which is now actually two lines.

Above I've identified lines by the contents of the first field. Lines can also be identified for AWK by line number, but note that in the second pass the current line number is FNR, not NR:

awk 'FNR==NR {if (NR==9) {x=$0; next} else next} \
> FNR==4 {$0=$0 RS x} FNR!=9 {print}' demo demo


The line-number command for moving a single line can be written as a function to save a lot of typing. I call my function "shiftline":

shiftline() { awk -v target="$2" -v putafter="$3" 'FNR==NR {if (NR==target) {x=$0; next} else next} FNR==putafter {$0=$0 RS x} FNR!=target {print}' "$1" "$1"; }
shiftline [filename] [line number to be moved] [line number preceding the new position]


The same command structure can be used to move multiple lines. Suppose in "demo" I want to gather up the two "meat" lines and put them just under the header. This command will do it:

awk -F"|" 'FNR==NR {if ($1=="119") {x=$0; next} \
else if ($1=="1221") {y=$0; next} else next} \
FNR==1 {$0=$0 RS x RS y} $1!="119" && $1!="1221" {print}' demo demo


As a special case, there's a much easier way to move lines if their new position is to be either at the start of a file or the end. Here's the CSV "4firstlast":


To move the "bbb" and "ddd" lines to either the start or the finish I can use cat and grep:

cat <(grep -E "bbb|ddd" 4firstlast) <(grep -vE "bbb|ddd" 4firstlast)
cat <(grep -vE "bbb|ddd" 4firstlast) <(grep -E "bbb|ddd" 4firstlast)


On my system, grep versions are aliased to grep --color=auto. To highlight the "apple" line and also show the rest of the "demo" file, I can use either of these two commands:

grep -E "^295\|.*$|$" demo
grep -E "^295\|.*$|^" demo

In each case grep is looking for either a line that starts with "295|" (^295\|) and finishes with zero or more characters (.*$), or an empty string. "^" is the empty string at the start of every line, and "$" is the empty string at the end of every line. grep will color what it finds, but with empty strings there's nothing to color.


Last update: 2020-05-13
The blog posts on this website are licensed under a
Creative Commons Attribution-NonCommercial 4.0 International License