diff options
| author | Adam M. Stück <adam@adast.xyz> | 2022-12-17 15:41:29 +0100 |
|---|---|---|
| committer | Adam M. Stück <adam@adast.xyz> | 2022-12-17 19:26:07 +0100 |
| commit | d1828d9658cb3e17984be757f57224a9e98bce50 (patch) | |
| tree | fdb15fe1d8abf2742e8382792381b48dc1564f9e /cloud-upload | |
| parent | 748bf8ca8e76d7dd48ba744c8a9e405b8ee9de0a (diff) | |
Optimized 'archiver' script with clever use of awkmain
Seems to run about 4 times faster now. Which is a decent time save when
running the script on the cluster.
Uses a magic awk command to concatenate all raw .csv files (excluding
their headers). This replaces the old way which consisted of reading
each file, trimming the header, and appending the lines to a HUGE
variable.
Diffstat (limited to 'cloud-upload')
0 files changed, 0 insertions, 0 deletions