impress-2020/scripts/archive
Matchu 571949b066 Increase timeout in archive SQL queries
One problem I run into with the archive task is that sometimes the queries time out? My hunch is that maybe some of the assets have like, weirdly big manifest files that are being transferred as surprisingly big text files?

Anyway, I'm increasing the timeout to 20s, which is big, but big feels good for a script that doesn't run often and where failing is not great news!

I'm also idly considering whether I wanna finally put in the work to do a bulk S3 uploader sometime, because the current version that iterates over multiple `aws s3 cp` calls is just real slow, I think because it establishes a new connection each time, and that operation is maybe surprisingly expensive? And the CLI doesn't have a way to do multiple uploads any more precisely than "sync up this whole folder", which is slow when the folder contains a lot of stuff you _know_ you don't want to head up there.
2023-06-12 10:56:40 -07:00
..
create Fix bug in archive:create scripts 2022-10-24 16:56:06 -07:00
prepare Increase timeout in archive SQL queries 2023-06-12 10:56:40 -07:00
upload Implement archive:upload:delta 2022-11-05 02:15:31 -07:00
.gitignore Refactor archive scripts to prepare/create/upload 2022-10-13 16:07:12 -07:00