Matchu
dc22a458bf
Okay, I've simplified the migration to *just* add the column, and instead added a task to find assets without manifest URLs and backfill them. Performance is a lot better now, using the `async-http` library, which as I understand it supports both persistent connections when invoked like this, and maybe also HTTP/2 multiplexing?? (Though I'm not actually sure images.neopets.com does lol) I'm not sure about the number of concurrent tasks I picked here, 100 seems okay for an internet thing and for such small requests, but I worry that the CDN is gonna get annoyed or something. Well, we'll see! This task is very resumable if it turns out we get frozen out or something. |
||
|---|---|---|
| .. | ||
| bundle/ruby/3.1.0/bundler/gems | ||
| cache | ||
| plugins | ||
| .gitignore | ||