logo

On this page:
    Field recipes
    Record recipes
    Data item recipes
    Header recipes

On the Data tables 1 page:
    What's a "data table"?
    Why TSV?
    Spreadsheet to TSV
    CSV to TSV

On the Data tables 3 page:
    Tidy data tables

On the Data tables 4 page:
    Inherited spreadsheet problems


TSV This marker means that the recipe only works with tab-separated data tables.
For pattern-matching commands, see the content pages


Field recipes

Print a numbered list of field names with tab-separated numbers   TSV

head -n 1 table | tr '\t' '\n' | nl -w1

Print a numbered list of field names (including empty strings)
in 2 columns (long field names may be truncated)
   TSV

fields() { head -n 1 "$1" | tr '\t' '\n' | pr -t -2 -n; }

fields

A graphical alternative to "fields" is the "fieldlist" function. It uses a YAD dialog to create a window just to one side of a terminal (adjust the "geometry" settings to suit your display), with a scrollable, numbered list of fields. The YAD window is in a background process. To close "fieldlist", bring the process to the foreground with fg and exit with Ctrl+c.

fieldlist() { head -1 "$1" | tr '\t' '\n' | nl | yad --geometry=350x800+1450+100 --text-info --no-focus & }

fieldlist

Print fields 2, 4 and 5 through 8   TSV

cut -f2,4,5-8 table

Delete fields 2, 4 and 5 through 8   TSV

cut -f2,4,5-8 --complement table

Insert a new, blank field between fields 3 and 4 with the field name "newfld"   TSV
(More information here)

paste <(cut -f-3 table) <(echo "newfld") <(cut -f4- table)

Insert the list "insert" (which has the correct number of records for table)
as a new field between fields 3 and 4 and give it the field name "newfld"
   TSV
(More information here)

paste <(cut -f-3 table) <(cat <(echo "newfld") insert) <(cut -f4- table)

Replace field 3 (field name "fld3") with the list "insert"
(which has the correct number of records for table)
   TSV
(More information here)

paste <(cut -f-2 table) <(cat <(echo "fld3") insert) <(cut -f4- table)

Insert a new field (field name "concat56") between fields 3 and 4
that contains the entries in fields 5 and 6 concatenated with a hyphen
   TSV
(More information here)

awk 'BEGIN {FS=OFS="\t"} NR==1 {$3=$3 FS "concat56"} NR>1 {$3=$3 FS $5"-"$6} 1' table

Reverse the order of the fields   TSV

awk -F"\t" '{for (i=NF;i>=2;i--) printf("%s\t",$i); print $1}' table


Record recipes

Print record 3

sed -n '3p' table
 
awk 'NR==3' table

Delete record 3

sed '3d' table
 
awk 'NR!=3' table

Print records 3 and 5

sed -n '3p;5p' table
 
awk 'NR==3 || NR==5' table

Delete records 3 and 5

sed '3d;5d' table
 
awk 'NR!=3 && NR!=5' table

Print records 3 through 8

sed -n '3,8p' table
 
awk 'NR==3,NR==8' table

Delete records 3 through 8

sed '3,8d' table
 
awk 'NR<3 || NR>8' table

After record 4, insert "aaa[tab]bbb[tab]ccc"   TSV
(More information here)

sed '4a\aaa\tbbb\tccc' table
 
awk -v x="aaa\tbbb\tccc" 'NR==4 {$0=$0 RS x} 1' table

Replace record 4 with "aaa[tab]bbb[tab]ccc"   TSV
(More information here)

sed '4s/^.*$/aaa\tbbb\tccc/' table
 
awk -v x="aaa\tbbb\tccc" 'NR==4 {$0=x} 1' table

Delete the records with record numbers listed in the file "list"
(More information here)

sed -e "$(sed 's/$/d/' list)" table
 
awk 'FNR==NR {a[$0]; next} !(FNR in a)' list table

Insert "aaa[tab]bbb[tab]ccc" after each of the records
with record numbers listed in the file "list"
   TSV
(More information here)

sed -e "$(sed 's/$/a\aaa\tbbb\tccc/' list)" table
 
awk -v x="aaa\tbbb\tccc" 'FNR==NR {a[$0]; next} FNR in a {$0=$0 RS x} 1' list table

Replace with "aaa[tab]bbb[tab]ccc" each of the records
with record numbers listed in the file "list"
   TSV
(More information here)

sed -e "$(sed 's/$/s\/^.*$\/aaa\tbbb\tccc\//' list)" table
 
awk -v x="aaa\tbbb\tccc" 'FNR==NR {a[$0]; next} FNR in a {$0=x} 1' list table


Data item recipes

Print the data item in record 37, field 16   TSV

awk -F"\t" 'NR==37 {print $16}' table

Delete the data item in record 37, field 16   TSV

awk 'BEGIN {FS=OFS="\t"} NR==37 {$16=""} 1' table

Replace the data item (or blank) in record 37, field 16 with "aaa"   TSV

awk 'BEGIN {FS=OFS="\t"} NR==37 {$16="aaa"} 1' table

Delete the data items in field 16 in all records
with record numbers listed in the file "list"
   TSV

awk 'BEGIN {FS=OFS="\t"} FNR==NR {a[$0]; next} FNR in a {$16=""} 1' list table

Replace the data items (or blanks) in field 16 with "aaa" in all records
with record numbers listed in the file "list"
   TSV

awk 'BEGIN {FS=OFS="\t"} FNR==NR {a[$0]; next} FNR in a {$16="aaa"} 1' list table


Header recipes

Add the original header to a selection of records

cat <(head -1 table) [selected records from table]

Sort all records except the header

head -n 1 table && tail -n +2 table | sort

Add a new first field (field name "ID") with a serial ID number for each record,
but don't number the header
   TSV

echo -e "ID\t$(head -n 1 table)" && tail -n +2 table | nl
 
awk 'BEGIN {FS=OFS="\t"} {print (NR==1 ? "ID" OFS $0 : NR-1 OFS $0)}' table