Oh right, our preview deploy was loading the prod allWakaValues data! Now it uses the VERCEL_URL env variable to request from the current deployment instead.
One day is too long! I'd prefer 1min for just the value itself, but I don't want to bog down all the other metadata with it, it's not _essential_ for it to be faster.
I wanted the ability to clear out closet list text for Support users, and figured I should just build the UI for end users too, and grant Support users the same access!
I narrowed down the problem to the fact that we were joining in pet types against assets, and *then* running GROUP and DISTINCT and everything. Assets x compatible species/color pairs is a LOT of rows!
Here, we instead get all the relevant body IDs first, and *then* match them against pet types—which we fetch in one batch to match body to canonical species/color.
I'm also trashing the weird caching mechanism we did here, because in practice it doesn't seem reliable anyway. If anything, I'd want to look at stronger CDN caching. (I made a small improvement to the caching annotation, but ultimately it still doesn't matter, because this query uses logged-in stuff and always comes out max-age=0 anyway.)
This helps with items like "Living in Watermelon Foreground and Background", which has a species-specific foreground and bodyId=0 background.
With this flag set on the background, it won't appear for pets that don't _also_ have something else that fits. In this case, it hides it from Standard Vandas, and all non-standard colors.
There's some hacky limitations here: the item page still highlights the Vanda, even though clicking gives nothing; and the zone info for it is messy too, with the Background claiming to fit all species, and the LFI claiming to fit 54 specific species. But those don't seem important enough to code for!
The other day, I deleted what was apparently a load-bearing glitch row, lol 😂
We had a row in pet_types that somehow had `body_id = 0`. And I guess that was causing this query to return some species, even though that body has no species.
Here, I'm adding support for the special `representsAllBodies` body's species to be null. The client seems chill with it, we weren't using that property in that situation anyway!
I think this will be generally useful to minimize switching around for common operations, but also I'm thinking of building a bulk assign tool for things with broken body IDs, and this will be the place for it to live, I think
Oops, we weren't doing a good job encapsulating the different conditions in item search. The `OR` in the NC condition was causing a precedence problem!
Now, we wrap all the conditions in parens at the interpolation site, to make it really clear that they all need to be made safe like that!
Now, there's not a bunch of "??????" entries in NC search, oops 😅
This was one more bit that needed fixing for "Flying in an Airplane": it wasn't just the official SVG that was incorrect, but also the official SWF. So our converted PNG was also incorrect!
Here, we now try to use the official Neopets PNG when the manifest provides it, instead of our own.
Inspired by the "Flying in an Airplane" bug (item 82287), where the official SVG (and I think SWF) were visually glitched and included both zones in the image, but the official PNG was correct.
This flag lets us use the PNG, like the official player does—but only for this item, while still keeping SVGs for everyone else!
I'm gonna extend `itemSearch` to also look up the total number of results, and the fragmentation between `itemSearch` and `itemSearchToFit` finally caught up with me :p
I've deprecated `itemSearchToFit`, and moved the fit parameters into a new optional `fitsPet` parameter for `itemSearch`.
I'm going to keep `itemSearchToFit` around for now, because old JS builds still use it, and I'd like to avoid disrupting folks. But I'm not going to add the new total results field to the results object it returns, and that's gonna be okay!
I haven't been keeping these up to date, so at this point they're more overhead than they're worth.
Helpful in the early days when we were iterating fast and making more mistakes, but now we're more solid (and I learned how to just resend queries from devtools :p)
Oops, the new `canonicalAppearance` arguments couldn't handle being omitted rather than being null, due to a low-level SQL call site that cares about the difference.
This meant that loading an item page in an old tab, with an old copy of our JS, could cause a crash.
Now, the backend will be okay with queries from old pages, and respond the same as before!
This isn't a huge deal, because once everyone is on the new JS we won't send queries without this parameter anyway… but I like my optional arguments to actually BE optional, without surprises lurking >.>
Ok cool, so apparently another win we get from using `ts-node` is that I can finally easily use some non-native-Node features like ES module import syntax, for consistency with what I'm doing in the main app source! That was getting on my nerves tbh. Ooh I bet I can finally use `?.` too, I've had to rewrite that a bunch…
Oh yay, I'm pleased with this! I hope it works out well!
stale-while-revalidate is an HTTP caching feature that gives us the ability to still serve relatively static content like item pages ASAP, while also making sure users generally see updates quickly.
The trick is that we declare a period of time where, you can still serve the data from the cache, but you should _then_ go re-fetch the latest data in the background for next time. This works on end users and on the CDN!
I've scanned the basic wardrobe and homepage stuff and brought them up-to-date, and gave particular attention to the item page, which I hope can be very very snappy now! :3
Note to self: Vercel says we can manually clear out a stale-while-revalidate resource by requesting it with `Pragma: no-cache`. I'm not sure it will listen to us for _fresh_ resources, though, so I'm not sure we can actually use that to flush things out in the way I had been hoping until writing this sentence lol :p
Trying to get that Item page fast!
I don't really want to ship this as-is, because I'd really like to get stale-while-revalidate working before shipping a 1-week cache timer… will be tricky though!
So, we've been behind the latest conversion data for a while anyway, because I don't run the sync very often. But I ran the sync today and noticed that the newly converted SVGs weren't showing up in DTI!
This is because TNT changed the asset manifest structure they use for SVG-only assets. Now, we support both!
To test, I checked the Blue Acara (old-style SVG manifest), the Blue Chia (new-style SVG manifest), and the Floating Negg Faerie Doll (animated clip).
Huh, looks like it's possible for a user's NeopetsConnection record to be missing, despite having the ID on their User record!
Here, we handle that case.
Woo, it's looking pretty good, I think!
I didn't bother with pagination yet, since I feel like that'll be a bit of a design and eng lift unto itself... but I figured people would appreciate the ability to look up individual items, even if the rest isn't ready yet 😅
Previously, when you clicked on a saved outfit from Your Outfits, the back button would take you back to the homepage, which was confusing for scanning through stuff! Now, it goes back to Your Outfits if it's yours.
I'm not suuure this is the behavior we want? But it seems intuitive enough!
Oops, the GROUP_CONCAT string was getting cut off! This caused an error trying to look up the name of an item ID that didn't exist, because the ID got truncated partway through.
This query was very slow! I added an index, and now it's fast!
This code change doesn't actually affect anything, but the comment helps explain what happened, since the index isn't stored in code. (Todo: should I start defining some indexes in our setup files?)
The AMFPHP gateway's json.php endpoint has always had a problem parsing pets whose names start with digits… I've dug into it before, and checked again today, and there really is just no way around it: d584b58e95/core/json/app/Actions.php (L43)
And there aren't any reliable AMFPHP Node libraries out there to make the actual native AMF call.
Buuuut! In today's investigation, I noticed the xmlrpc.php endpoint for the first time. And, wouldn't you know it, there's //great// reliability for something as enterprise-standard as that!
So here, I've switched over to using an xmlrpc client library, which simplifies our calling code //and// makes number pets work correctly 😁 I wouldn't have done it just for the simplification, I think bringing in a library is net more complexity… but getting this finally right is a big relief.
Yeah, mm, turns out I don't think it's actually viable to model from Impress 2020, because we can't reasonably set up the SWFs and PNGs in the ways we need, especially for compatibility with Classic DTI.
We can turn this on again later, once Classic DTI is gone, and all assets are converted to HTML5 -- or if we build some kind of bridge to Classic's asset code, or we write new PNG conversion code.
These changes are most relevant for playing around in the dev server, modeing against an empty database. But they'll also help in real-world modeling scenarios! e.g. modeling a new species/color combo is now a bit nicer, we don't show a blank entry in the color picker
this is the last one to get parity with current modeling, I think?? I'm gonna add one more feature though: removing no-longer-used assets from the item
Oops, when building the Support tool to label pet appearances, I didn't realize that there's also a boolean `labeled` field that needs to be true for labeled appearances. Without it, the old app shows the appearance as "Unlabeled".
I also ran this query to fix the rows we'd incorrectly written:
```
mysql> UPDATE pet_states SET labeled = 1 WHERE mood_id IS NOT NULL;
Query OK, 158 rows affected (0.14 sec)
Rows matched: 19640 Changed: 158 Warnings: 0
```
This is mostly because I want to chain the rels after both items and assets save, and I want to be able to specify that stuff a bit more precisely, rather than the like, layers-of-awaits we were building up.
yeah, I had unified Pet into Outfit, but now I think that was overly clever… 😅
Here, I define a new Pet type, and it has some of the fields of Outfit and the deprecated fields still.
I did this because I want petAppearance to work, for UC testing!
We download the schema from prod, and omit real data, but I didn't notice that we were still pulling the metadata of the auto increment counter for IDs! Now, we scrub that from the schema file we save.
This wasn't actually super helpful to read anyway, and I think it was causing us to hit rate limits.
We can maybe add back a limited version to like, add path context of _where_ a span happened in the GQL tree, but like, I feel like that's typically been pretty intuitive so far.
Boom, now we can also run a clean MySQL test db on each test that wants it :)
the test I wrote as a sample is currently marked `it.skip` because it's not passing yet!
This updates the MySQL procedure to get the important special colors, but keeps the GQL behavior the same by only filtering to Blue. Just an incremental step before changing the behavior, to make sure I've gotten it right so far!
Snapshots significantly updated, but, from scanning it, I think that's expected changes from actual modeling progress. Hooray!
I'm using my first ever MySQL Store Procedure for clever cleverness in caching the modeling query!
I realized that checking for the latest contribution timestamp is a pretty reliable way of deciding when modeling data was last updated at all. If that timestamp hasn't changed, we can reuse the results!
I figured that, because query roundtrips are a bottleneck in this environment, I didn't want to make that query separately. So, I built a MySQL procedure to do the check on the database side!
Okay, we handle the new pages correctly! Still some weird bugs when you send requests near each other? Probably wise to migrate to Apollo's new way of doing this
Been bothered by this for a while!
My hope is that this isn't a notable marginal performance hit—we were already walking the table and doing string ops anyway, I can't imagine adding to that is actually that much of a marginal lift, when the main bottleneck was probably reads. And the perf should be identical for simple single-word queries anyway. But we'll see how it feels!
Previously, if you switched species/color such that one of your items was no longer compatible, we _would_ still apply its zone restrictions to the visible layer set.
In this change, we fix that server-side, since I think it makes the most sense for an empty appearance to be truly empty!
This is in preparation for hiding bio zone restrictions but showing item zone restrictions!
I also refactor the build-cached-data script substantially, to run GraphQL against the server instead of a custom query.
okay so the PetAppearance restrictions are stored on the asset, because that's how they're defined on Neopets.com too
but I think that's a confusing API, so here I define `PetAppearance.restrictedZones`, which just maps over the layers and aggregates the zones server-side, same as we would have done on the client
I think that's much easier to understand than having layer contain a field, but having to know that item restrictions _don't_ work that way, you know?
