For a list of BASHing data 2 blog posts see the index page.

Filling down blanks in multiple fields

The title of this post might be a bit cryptic, so I'll explain it with an example. Here's the tab-separated table "filldowner":

fld1	fld2	fld3	fld4	fld5	fld6
2oQ	3K6		Ef2	I4H	H17
W4o	2iW		U7J	V58
qC5	Ro0	v0L		7XE
0EN	1Ni	J6a		QD4	fS6
VA3	x8V	rL8		6Pv	o5E
DY2	W9c	A6J	o1E	BN5	S0w
SB3		9SS	zA3	T5N	Xz9
1Dh		z8Y	D41	J3M	jY9
O7r		U1x	z2P	2eU	nA5
p2D	vT4	07A	V1Y	RG4	2Xd

I want to replace the blanks in each field with the last non-blank value in that field, like this:

fld1	fld2	fld3	fld4	fld5	fld6
2oQ	3K6		Ef2	I4H	H17
W4o	2iW		U7J	V58	H17
qC5	Ro0	v0L	U7J	7XE	H17
0EN	1Ni	J6a	U7J	QD4	fS6
VA3	x8V	rL8	U7J	6Pv	o5E
DY2	W9c	A6J	o1E	BN5	S0w
SB3	W9c	9SS	zA3	T5N	Xz9
1Dh	W9c	z8Y	D41	J3M	jY9
O7r	W9c	U1x	z2P	2eU	nA5
p2D	vT4	07A	V1Y	RG4	2Xd

And I'd like to do the filling-down with a one-liner, and ignore the header (otherwise I'd be filling down with the field name if the first record had a blank field).

The most direct way to do this job is to use AWK and an AWK array:

awk -F"\t" 'NR>1 {for (i=1;i<=NF;i++) if ($i!="") {a[i]=$i} else {$i=a[i]}} 1' OFS="\t" filldowner

In everyday language the command reads In this tab-separated file, ignore the header line and work through the fields one-by-one in each record. If the data item in the field isn't blank, store the data item in an array "a" indexed with the field number. If the data item is blank, fill the blank with the last array value indexed with that field. Print all records with tab as field separator.

In case you're wondering, I generated the 6 x 10, tab-separated matrix of random 3-character data items in "filldowner" with the command pwgen 3 -N 60 | xargs -n 6 | tr " " "\t". I then manually deleted data items to create blanks, added a header line and HTML'ed the result as a table. pwgen is a marvellous utility from Theodore Ts'o and I use it a lot!

Next post:
2025-09-05 5, 7, 8, 9, 10, 12, 14, 15, 17. Any advance on 17?

Last update: 2025-08-29
The blog posts on this website are licensed under a
Creative Commons Attribution-NonCommercial 4.0 International License