Weird! Well! This caused two issues.
One is that we used to filter down to only assets whose urls end in
`.swf`, because I never added support for the sound ones :p
The other is that we were using the SWF URL to infer the potential
manifest URLs, which we can't do when the SWF URL is a weird
placeholder!
Thankfully, just yesterday (wow!) we happened to have Classic DTI keep
track of `manifest_url` during the modeling process, and we backfilled
everything. So the most recent `temp` assets have their manifest! But
slightly older ones (not that old tho!!) didn't, so I manually inferred
the manifest URL from the asset's `remote_id` field instead (which
worked cuz there was no hash component to the manifest URL, unlike some
manifests).
The item "Flowing Black Cloak" (86668) on the Zafara is the one we were
looking at and testing with!
Right, ok, `db.close()` needs to be `db.end()` now.
This probably didn't break the user-syncing cron job though, because that doesn't automatically update I think? so it should still be comfortably running older version of the code that should still work just fine
I'm not getting cron success _or_ cron failure emails for running this script on our Linode box. I was getting failures back when I had the command wrong, though.
My hypothesis is that the script output is too long to email, because of some limit somewhere along the way. I'll update the cron job to use `--skip-unchanged`, in hopes that it helps me get the emails! (I'm not suuure it's running, is the thing... though hey, here's a way to check: as of now, 512,624 of 521,896 assets are converted. If that changes eventually without a manual script run, then the cron is working!)
I want to start running this on a regular cron, and making the script faster (stop sending redundant queries) and clearer (# actually updated) is super useful for that!
Originally, this was sorta a cache warmup script: we wanted to fill in manifests that we hadn't checked for yet.
But now, I want to _also_ check previous cache misses, that we stored in the db as an empty string. Maybe it's been converted now!
I was finding the script too slow running on my local machine, because the SQL RTTs were too slow - and with one connection, they were essentially a serial bottleneck, not taking much advantage of our concurrency.
Here, I instead add a `--dump` option, which outputs SQL to stdout. I then uploaded the resulting SQL to the DTI box, and ran it up there. Doing the network part fast on my machine, and the SQL part fast on the cloud machine!
I first considered uploading this script to the cloud machine, but it's an old Ubuntu and I couldn't figure out how to install a recent NodeJS onto it 🙃
Just gonna bulk load all those manifests into the db, and then that should make most loads notably faster by removing the net request! 🤞
We'll still load manifests inline sometimes, but only the first time anyone pulls up the layer in impress-2020. After that, it should be cached forever!
