cache asset manifests in the db
Here's just some simple caching: we try to load the asset manifest from the db with the rest of the asset. If it's not present, we load it via HTTP, and write it to the database. I might try to do a bulk write of manifests at some point, too. This is because I noticed that one of the main bottlenecks in most of the endpoints now (and definitely the highest-variance) was loading from images.neopets.com. Another approach I considered was HTTP/2 to load the manifests, because it kinda looks like the server is refusing to open all these sockets at once and effectively does the requests in waves? But images.neopets.com doesn't support HTTP/2 right now anyway, so oh well! (And that would have probably cut us down to ~250ms of HTTP time still, instead of ~600–700. Also, why is network out of Vercel so slow? :p)
This commit is contained in:
parent
0f7ab9d10e
commit
a3423cd6d6
1 changed files with 31 additions and 2 deletions
|
@ -382,10 +382,39 @@ const resolvers = {
|
||||||
`/${rid1}/${rid2}/${rid3}/${rid}/${sizeNum}x${sizeNum}.png?v2-${time}`
|
`/${rid1}/${rid2}/${rid3}/${rid}/${sizeNum}x${sizeNum}.png?v2-${time}`
|
||||||
);
|
);
|
||||||
},
|
},
|
||||||
svgUrl: async ({ id }, _, { swfAssetLoader, svgLogger }) => {
|
svgUrl: async ({ id }, _, { db, swfAssetLoader, svgLogger }) => {
|
||||||
const layer = await swfAssetLoader.load(id);
|
const layer = await swfAssetLoader.load(id);
|
||||||
|
let manifest = layer.manifest && JSON.parse(layer.manifest);
|
||||||
|
|
||||||
|
// When the manifest is specifically null, that means we don't know if
|
||||||
|
// it exists yet. Load it to find out!
|
||||||
|
if (manifest === null) {
|
||||||
|
manifest = await neopets.loadAssetManifest(layer.url);
|
||||||
|
|
||||||
|
// Then, write the new manifest. We make sure to write an empty string
|
||||||
|
// if there was no manifest, to signify that it doesn't exist, so we
|
||||||
|
// don't need to bother looking it up again.
|
||||||
|
//
|
||||||
|
// TODO: Someday the manifests will all exist, right? So we'll want to
|
||||||
|
// reload all the missing ones at that time.
|
||||||
|
manifest = manifest || "";
|
||||||
|
const [
|
||||||
|
result,
|
||||||
|
] = await db.execute(
|
||||||
|
`UPDATE swf_assets SET manifest = ? WHERE id = ? LIMIT 1;`,
|
||||||
|
[manifest, layer.id]
|
||||||
|
);
|
||||||
|
if (result.affectedRows !== 1) {
|
||||||
|
throw new Error(
|
||||||
|
`Expected to affect 1 asset, but affected ${result.affectedRows}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
console.log(
|
||||||
|
`Loaded and saved manifest for ${layer.type} ${layer.remoteId}. ` +
|
||||||
|
`DTI ID: ${layer.id}. Exists?: ${Boolean(manifest)}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
const manifest = await neopets.loadAssetManifest(layer.url);
|
|
||||||
if (!manifest) {
|
if (!manifest) {
|
||||||
svgLogger.log("no-manifest");
|
svgLogger.log("no-manifest");
|
||||||
return null;
|
return null;
|
||||||
|
|
Loading…
Reference in a new issue