Matchu
88511d3dc6
Ok great, we can now run the delta archive process! It'd be nice to get this running on cron on the impress-2020 server, to a temporary folder? I *do* want to be remembering to run something regularly on my personal machine too though, to keep my own copy up-to-date…
14 lines
No EOL
667 B
Bash
Executable file
14 lines
No EOL
667 B
Bash
Executable file
# List all the files in our bucket. (The CLI handles pagination, thank you!)
|
|
yarn aws s3 ls --recursive s3://dti-archive/ \
|
|
| \
|
|
# Filter out unnecessary lines; just give us lines formatted like results.
|
|
grep -E '^[0-9]{4}-[0-9]{2}-[0-9]{2}\s+[0-9]{2}:[0-9]{2}:[0-9]{2}\s+[0-9]+\s+' \
|
|
| \
|
|
# Replace all the extra info like time and size with "https://".
|
|
sed -E 's/^[0-9]{4}-[0-9]{2}-[0-9]{2}\s+[0-9]{2}:[0-9]{2}:[0-9]{2}\s+[0-9]+\s+/https:\/\//' \
|
|
| \
|
|
# Hacky urlencode; the only % value in URLs list today is %20, so...
|
|
sed -E 's/ /%20/g' \
|
|
| \
|
|
# Output to manifest-remote.txt, and print to the screen.
|
|
tee $(dirname $0)/../manifest-remote.txt |