Oh yay, I'm pleased with this! I hope it works out well!
stale-while-revalidate is an HTTP caching feature that gives us the ability to still serve relatively static content like item pages ASAP, while also making sure users generally see updates quickly.
The trick is that we declare a period of time where, you can still serve the data from the cache, but you should _then_ go re-fetch the latest data in the background for next time. This works on end users and on the CDN!
I've scanned the basic wardrobe and homepage stuff and brought them up-to-date, and gave particular attention to the item page, which I hope can be very very snappy now! :3
Note to self: Vercel says we can manually clear out a stale-while-revalidate resource by requesting it with `Pragma: no-cache`. I'm not sure it will listen to us for _fresh_ resources, though, so I'm not sure we can actually use that to flush things out in the way I had been hoping until writing this sentence lol :p
Trying to get that Item page fast!
I don't really want to ship this as-is, because I'd really like to get stale-while-revalidate working before shipping a 1-week cache timer… will be tricky though!
The Tooltip elements seem to still be taking a bit, I don't really know a great workaround for that… could maybe split it out into a separate box and defer the rendering on it, so it doesn't block the first pageload?
Before the tooltips, I thought the focus state wasn't clear enough at a glance, so I added an extra focus outline to the species faces picker area. Now, I think it's clear enough with the species name tooltip popping up!
Now, when a species isn't compatible with an item, we gray out and sadden the pet, like on Classic DTI!
For now, I've hardcoded only the Zafara body ID to match. Let's do server connection next!
Didn't realize there was a convenient 150x150 face thumbnail we could use, so hey! Nice!
At one point I was considering generating our own thumbnails, but this is making me increasingly interested in just scraping the Rainbow Pool or something :p
Sentry issue IMPRESS-2020-20 doesn't have a clear backtrace, but it looks like the usual thing where we trigger an Apollo query directly, and forget to catch a potential error in the returned promise. I noticed that the last thing the user did was type in the search bar, and got a _caught_ error for the initial search!
Scanning the SearchPanel file, I think it's likely that this was a failure in `fetchMore` for the infinite pagination.
I'm a bit worried as to _why_ we were doing infinite scrolling stuff when there were no results? I wasn't able to repro a scroll event on the empty results list, but it's plausible that it could happen. I've added a gate to not send this request when there's an error!
I think what's happening in Sentry error IMPRESS-2020-1F is that mobile devices are running out of memory, so `canvas.getContext("2d")` returns null.
Now, we have a UI affordance to let you know when this is probably what's happening!
Also, when researching this, I learned about a Safari bug where you need to manually garbage-collect your own canvas data. It's possible that Safari users have been having particular trouble with memory leaks over long sessions? I'm not sure, but it seems like a good idea to add this small garbage-collection code!
Ok I think I've finally narrowed this bug down! We had one more loading case: the items page needs time to figure out which species/color to default the fields to, and passes null into the component while this loads. Now, we wait for that!
Previously, if you typed a pet name on the homepage, but its pose wasn't labeled in our database, you'd get a black empty screen. Now, we redirect to the UNKNOWN pose, or whatever exists for us to use!
I think it's confusing that the poses in the dropdown start with the emotion word, but are grouped by the gender presentation word! It's also different than the precedence order! I've reordered them.
We're occasionally getting errors on the homepage, of the new message I added: `Error loading valid poses for species=, color=108: byteOffset cannot be negative`.
So ok, now we know it's a species undefined bug, coming from `onChangeSpecies`! That suggests we're not finding by ID correctly?
So I'm adding some new logging to help me understand the sequence of actions leading up to this point, and the species data state when the error itself happens!
I've been getting more Sentry errors about JS chunk errors after deploys, and finally looked into it!
Turns out that, our try/catch handling was working great, and the page was reloading correctly for users as expected. But in these scenarios we would _also_ throw and log two uncaught errors!
The first is that, because we're a single-page app, unrecognized routes fall back to the index.html by default (to power our custom client-side routes, like /outfits/123 etc). So this meant that missing JS files, which _should_ be returning a 404, were instead returning 200 OK and an HTML file, which failed to parse. (And running the script isn't included in the catchable part of the `import` promise!)
Now, in our `vercel.json` config, we catch those paths specifically and 404 them. (The exact choice of path is important: on dev, all these routes run _before_ the dev server, which is responsible for serving the static files - but dev doesn't include hashes in the JS file names, so this 404 route only matches built prod JS files, not local dev JS files.)
The second is that we failed to return anything to `@loadable/component` in the error case, so it would try to render `undefined` as if it were a component class. Now, we return a trivial component class that returns null!
I'm getting some vague errors in Sentry about `canvas.getContext` returning null? Weird. (IMPRESS-2020-1F)
I'm not sure what that's about, so I don't want to stop sending it to Sentry. But I do want to make sure we handle this kind of error gracefully! (I'm thinking about how, while I don't think this one was, in the future this _could_ be caused by errors in Neopets movie clip JS, and I don't want our app to start messing up because of it!)
Here, we make sure to log the error to the console with more detail (the library URL), and show feedback to the user, and only log the error once per clip (so that animated ones don't like, send a bunch).
Huh, I'm not sure why Apollo is returning `data: undefined`, when the server is definitely returning the correct no-user-found data in the shape I expect... suspicious :/ well, let's at least stop crashing!
Oops, we were getting errors when people tried to change the species/color picker before the valid pose data was ready!
This was only happening on the homepage and item page, because on the wardrobe page we wait for the valids to load before showing the picker at all (`showPlaceholders` is false).
To fix this, we add the valid poses loading state to our existing `isLoading` state on the selects, which disables the element and adds a loading spinner cursor. This prevents interactions we're not ready for!
I'm not sure why, but people are seeing errors when reading from the /api/validPetPoses binary blob. I think it's the picker not handling loading states well?
In this change, we start by just giving it graceful handling, and improving the logging. I'll also try to fix the cause in the next change!
Oops, we were removing the last word of the search query if you picked a suggestion from Advanced Search! That behavior is meant for the case where you're _typing_ a filter name 😅
Sooo, I added this more graceful regex and error logging… then realized that this shouldn't be happening in the first place, because we should only be removing the last word of the query if you picked the filter via typing, not advanced search!
I'm glad to have the assertion error and the new handler, but I'll fix the cause too in the next change :p
We were getting away with singular stuff like "Hat" in the filter text for a while, but once it became "Hat you own" it got too weird imo!
Now, we say "Hats" and "Hats you own" in the filter text. We keep the singular in the search suggestion, but with the "Zone:" prefix, which is something I've been wanting anyway. (It should help with the show all suggestions UI coming soon, too.)
Oops, the old condition depending on `queryFilterText` to implicitly check for filter presence. But now that we always show "Items" as prefix text for the filters on this page, the reset button was always showing!
Use our new util function instead.
1. Search for something
2. Clear the search bar
3. Quickly start typing something new
Before this change, the results would clear on #2, but then the old results would show up again during #3, before the loading state for the new query.
This matches the logic, right? We hid the results when both the current and debounced query were empty, and, during that time, neither is empty.
Instead, here we update the `useDebounce` hook to have a `forceReset` option, to immediately clear out the value instead of waiting.
I switched from my `_NoAuthRequired` opname hack, to a more robust `context` argument, and it's opt-in!
This should make queries without user data faster by default. We'll need to remember to specify this in order to get user data, but it shouldn't be something we'd like, ship without remembering—the feature just won't work until we do!
Still getting some chunk load errors in my Sentry reporting! My hunch is these are the culprit. I hooope that after this the errors are pretty much gone! If not, then I'm missing something about what causes these failures…
So, we've been behind the latest conversion data for a while anyway, because I don't run the sync very often. But I ran the sync today and noticed that the newly converted SVGs weren't showing up in DTI!
This is because TNT changed the asset manifest structure they use for SVG-only assets. Now, we support both!
To test, I checked the Blue Acara (old-style SVG manifest), the Blue Chia (new-style SVG manifest), and the Floating Negg Faerie Doll (animated clip).
Not really sure how to scale these over time, I feel like some amount of history + blog cuteness could be fun? And like, the ability to catch up if you come back after a couple weeks could be nice. But this seems like a really useful at-a-glancer for folks!
Someone wrote in how, when your search query ends with a string that creates Advanced Search suggestions, clicking on items in the list requires two clicks: one to blur and dismiss the suggestions, and one to actually click the item.
Here, I'm experimenting with just leaving the suggestions open. It doesn't feel _great_, but it definitely feels _better_ than before on this edge case, and I thiiink this only affects this edge case in practice? We'll see if it feels goofy in some cases I forgot tho!
Huh, looks like it's possible for a user's NeopetsConnection record to be missing, despite having the ID on their User record!
Here, we handle that case.
Oops, we hit our Sentry transaction limit after 3 days!
This is in part because we selected a 100% sample rate to start, but also because our app has a lot of client-side navs that don't represent real navigation, and are just to update the state in the URL.
I'm not using most of Sentry's performance tracing features, though! I don't have logging that helps us understand once the page is really done, and I'm only really able to use Web Vitals right now - which only applies to first-time pageload events, anyway. So, that's now all we track!
Woo, it's looking pretty good, I think!
I didn't bother with pagination yet, since I feel like that'll be a bit of a design and eng lift unto itself... but I figured people would appreciate the ability to look up individual items, even if the rest isn't ready yet 😅
When building the code to await auth before sending _any_ GraphQL queries, I didn't realize that auth might be kinda slow. So, I've added a hack to let me mark queries with no user-specific data to skip auth, and applied that to the main queries on the homepage.
I think this is a hint that we might want to change our strategy - e.g. to flip it to hackily mark that auth _is_ required, or to create wrappers or option-builder helpers for logged-in queries, etc.
I also notice that SSR would have resolved this particular case...
This UI generally loads very fast, thanks to the CDN cache, so the flash of skeleton content is more distracting than anything else! We still show it quickly after 300ms, but good network connections should reliably get it loaded before then.
Our host, Vercel, doesn't keep old build files on its CDN after a deploy for very long. This means that, after a deploy that changes a page's bundle, existing sessions that attempt to navigate to it for the first time will fail on the dynamic `import`, because the filename hash has changed.
The best fix I'm aware of for this is to just, reload the page when this happens!
To test this, I did the following:
1. Use `yarn build` to build a prod copy of the site.
2. Use `serve -s build` to start serving it on its own port. (API endpoints won't work, and that's okay!)
3. Don't touch the open copy of the site yet.
4. Make a change to `PrivacyPolicyPage.js`, and `yarn build` again. This simulates a deploy under similar circumstances.
5. Open the Console, tick the "Persist Logs" option, and try to navigate to Privacy Policy. Observe that it logs a ChunkLoadError in the console, and smoothly reloads the page to show you the updated Privacy Policy page.
6. Undo your change 😅
Previously, when you navigated directly to an outfit by typing the URL into the browser or following an external link, the name would stay as "Untitled outfit", even after the outfit loaded.
This was because, when you render an `Editable` Chakra component with `value={undefined}`, it permanently enters "uncontrolled" mode, and providing a value later doesn't change that.
But tbh passing `undefined` down from outfit state wasn't my intention! But yeah, turns out the `?.` operator returns `undefined` rather than `null`, which I guess makes sense!
So, I've fixed this on both ends. I'm now passing more `null`s down via outfit state, because I think that's a more expected value in general.
But also, for the `Editable`, I'm making a point of passing in an empty string as `value`, so that this component will be resilient to upstream changes in the future. (It's pretty brittle to _depend_ on the difference between `null` and `undefined`, as we saw here 😅)
Previously, when you clicked on a saved outfit from Your Outfits, the back button would take you back to the homepage, which was confusing for scanning through stuff! Now, it goes back to Your Outfits if it's yours.
I'm not suuure this is the behavior we want? But it seems intuitive enough!
Previously, if you navigated to /outfits/new without a species or color in the query string, we'd show a blank outfit page, with the species/color picker hidden. Now, we default to a Blue Acara instead!
We don't do anything to handle _invalid_ species/color IDs, but I don't super mind that, because in practice that would require some call site to malform the URL, and I don't super expect that.
This resolves more of the _cause_ of Sentry issue IMPRESS-2020-8, but I'm still wondering how a user got to the URL `/outfits/new?[object+Object]=&objects[]=35185&objects[]=67084`. I'm wondering if the pet loader on the homepage has a bug in Safari? I feel like I heard something like that from the feedback form, too...
If the species/color of the current outfit aren't available yet (e.g. a saved outfit is still loading in), hide the picker altogether. This is because the picker can't handle change events during that time, and it's easier to just hide all this than to add special case handlers like disabled states! (And, while placeholders are often helpful, I'm not sure the placeholder dropdowns are any better than empty space in this case.)
This can also happen when the user loads a page without a species/color ready, like just going straight to `/outfits/new`. I think I might want to add a handler for that, though.
Resolves the direct cause of Sentry issue IMPRESS-2020-8, though I'm not sure how the user got to the URL `/outfits/new?[object+Object]=&objects[]=35185&objects[]=67084` in the first place...
Two fixes in here, for when image downloads fail!
1) Actually catch the error, and show UI feedback
2) Throw it as an actual exception, so the console message will have a stack trace
Additionally, debugging this was a bit trickier than normal, because I didn't fully understand that the image `onerror` argument is an error _event_, not an Error object. So, Sentry captured the uncaught promise rejection, but it didn't have trace information, because it wasn't an Error. Whereas now, if I forget to catch `loadImage` calls in the future, we'll get a real trace! both in the console for debugging, and in Sentry if it makes it to prod :)
I haven't seen anything come in from prod yet, and it's hard to trigger one, maybe because the integration is React-specific? Or maybe it's... not working :p
I can send errors from dev! But just haven't _seeeen_ a prod error come in yet.
Maybe we're just squeaky clean tho :3
My main reason for adding this now is that I'm getting some scattered reports of things not displaying correctly, and I want to start gathering some browser data on that...
I recently confirmed that animations work on iOS (at least one did!), which was going to be my guess of what was breaking...
I took out virtualization for now too, I wanna see how this non-Chakra UI version, with fewer nodes and no tooltips etc, performs on large lists in production.
Huh, I'm not sure why SVGs ever didn't have `crossOrigin: "anonymous"`? The old commit isn't really super helpful for understanding that. Maybe I just didn't notice the problem in that case?
Well, whatever. Let's see if this breaks something else! (I'm also wondering if we should just like, _always_ ask for things with crossOrigin set?)
Oops, if you try to show PosePicker before we have a species/color ready, it sends a bad GraphQL request. No visible user impact, just an unnecessary network call and an error in the console! This happens when you're loading an outfit by ID.
Here, we hide PosePicker if there's no species/color ready yet. This stops the extra request from firing!
When loading an outfit in the wardrobe page, there was an awkward state where the outfit preview loading spinner would vanish and then reappear.
This was because `useOutfitState` briefly reported `loading: false`, then fixed itself after almost immediately—but our OutfitPreview component has a delay before re-showing the spinner.
In this change, we smooth out the loading state, by enabling the second GQL request to start immediately once the first request is done, instead of waiting on a callback to finish.
Oops, when refactoring and adding alt text, I didn't realize the padding for the text would affect the images too! And I forgot to add `overflow: hidden` to round the image's corners. Fixed!
To help the load time for outfits feel shorter, we now reuse the outfit thumbnail from the Your Outfits page as a placeholder!
This doesn't add any overhead when the thumbnail data _isn't_ in your session cache, e.g. if you navigate to the outfit directly. But if we have the thumbnail on hand already, we just show it, easy peasy!
Oops, when switching to @emotion/react, it turns out they no longer support that cute hack I was doing to append suffixes to class names!
Here, I change strategy and let `CSSTransition` apply the plain `exit` and `exit-active` classes to its children, and apply Emotion styles to the child to check for _also_ having those classes.
Oops, the GROUP_CONCAT string was getting cut off! This caused an error trying to look up the name of an item ID that didn't exist, because the ID got truncated partway through.
Oh right, we can't cache objects well when they're missing their ID!
Before this change, selecting an outfit then navigating back would require the outfits to reload. Now, they stay!
That'll still show up when the outfit is still loading, but this lets us use the Apollo cache to show the name instantly if you're clicking through a link from Your Outfits
Still a pretty limited early version, no saving _back_ to the server. But you can click from the Your Outfits page and see the outfit for real! :3 We have a WIPCallout explaining the basics.
This seemed to only show up in dev? But right, I guess it's not happy about passing stuff from ClassNames into a Popover Portal. Move it inside, fixed!
This has been bothering me for a long time, but I couldn't really figure out what to do about it. But tweaking the site bg color a smidge has helped us really add texture to the cards I want to have pop out, like the outfit polaroids!
I kinda went all-in in a burst, but tbh I think it looks great :3
I haven't really touched the wardrobe page with it yet though, that'll probably need some tweaking... for now I'm overriding it to keep the old background!
Looks like there was some kind of runtime conflict when running @emotion/css and @emotion/react at the same time in this app? Some styles would just get clobbered, making things look all weird.
Here, I've removed our @emotion/css dependency, and use the `<ClassNames>` utility element from `@emotion/react` instead. I'm not thrilled about the solution, but it seems okay for now...
...one other thing I tried was passing a `css` prop to Chakra elements, which seemed to work, but to clobber the element's own Emotion-based styles. I assumed that the Babel macro wouldn't help us, and wouldn't convert css props to className props for non-HTML elements... but I suppose I'm not sure!
Anyway, I don't love this syntax... but I'm happy for the site to be working again. I wonder if we can find something better.
Oops, creating a new `SpeciesColorPicker` fn on each render meant that React treated it as a whole new dropdown each time. I've extracted it out into a stable component class, and just pass in the extra props now!
This bug caused changes to kick you out of focus for the dropdown, because it had unmounted and remounted.
This query was very slow! I added an index, and now it's fast!
This code change doesn't actually affect anything, but the comment helps explain what happened, since the index isn't stored in code. (Todo: should I start defining some indexes in our setup files?)
We do animation detection during the preload now, but this wasn't always working correctly: some movies don't actually fully mount the children onto the stage until we start playing. This caused the play/pause button to be missing on the outfit page and the item page, but the animations would still play, depending on the user's saved play/pause state in localStorage.
I saw the short-near-the-front and it just frankly looked awkward? Not sure why I liked it before?
I think this medium at the end of the list is better aesthatically, though it's starting to get a bit messy with the different colors mixed around… but I think there's also a semantic argument that we're keeping the facts about the item together, and the _user-specific_ stuff separate at the end… (putting it at the front would be a good semantic argument too, but I think the NC/NP alignment is too important)
In a previous change, I moved the margin for item badges onto an ItemBadge element… but I didn't think through how that would break the spacing for the loading state of ItemPage. Now, the loading skeleton items _contained_ the badge margin, and so the spacing between badges was shiny skeleton-y.
Here, I replace ZoneBadgesList with a function that just returns the elements, and go back to using Chakra's Wrap component. That will apply the margin to direct children, and the zone badges are direct children now.
One option I'm thinking of in hindsight is an idea I had earlier: Chakra hacks the margin onto _React_ children, but could we use CSS direct child selector instead? A bit trickier to resolve the margin size to the theme's value, but plenty doable… something to consider!
In the previous impl, the buttons variant of the menu would appear on first render, and then the breakpoint stuff would adjust and re-render as the compact nav menu. Now I'm using CSS to show/hide instead!
"Beautiful Green Painting Background" wasn't loading! https://impress-2020.openneo.net/items/75594
```Error building movie clips Error: Expected JS movie library http://images.neopets.com/cp/items/data/000/000/491/491273_31368b3745/491273_2_HTML5%20Canvas.js to contain a constructor named _491273_2_HTML5%20Canvas, but it did not: ssMetadata,Bitmap3,Bitmap5,CachedTexturedBitmap_4183,CachedTexturedBitmap_4184,CachedTexturedBitmap_4185,CachedTexturedBitmap_4186,CachedTexturedBitmap_4187,Symbol20,Symbol8,Symbol4,Symbol7,Symbol2,Symbol1,Symbol9,Symbol2copy,Symbol2_1,_491273_2_HTML5Canvas,properties,Stage```
We already had code to strip out spaces, but not encoded spaces like %20. Now, we decode the URL first, so that space-stripping will work even if it was encoded.
That is, if you're browsing a trade list and you go "oh actually, I _do_ want that!", and click the item page to mark it, then click Back, we'll now update the matching stuff on the trade list page to reflect that it's now a match.
This was just a matter of simplifying the GraphQL query, I think the `currentUserOwnsThis` and `currentUserWantsThis` fields just didn't exist at the time?
We _don't_ yet update your _own_ trade list, if you click through to an item to remove it or something like that. The cache update function isn't too tricky, but it's a bit verbose to implement in Apollo, so I'm not bothering right now!
Oops, the <Wrap> component is nice, but it uses React.Children to apply margin to its _direct_ children, and our badges are not always direct children! (See the new `ZoneBadgeList`.)
I poked my head into how `Wrap` works, and it's honestly pretty simple, so I've applied the same styles manually. Ta da!
Not sure why movie clip building is failing! But it happened outside our try-catch, so it left us in an infinite spinner state.
The repro item is the Spring Topiary Garden Background!
The AMFPHP gateway's json.php endpoint has always had a problem parsing pets whose names start with digits… I've dug into it before, and checked again today, and there really is just no way around it: d584b58e95/core/json/app/Actions.php (L43)
And there aren't any reliable AMFPHP Node libraries out there to make the actual native AMF call.
Buuuut! In today's investigation, I noticed the xmlrpc.php endpoint for the first time. And, wouldn't you know it, there's //great// reliability for something as enterprise-standard as that!
So here, I've switched over to using an xmlrpc client library, which simplifies our calling code //and// makes number pets work correctly 😁 I wouldn't have done it just for the simplification, I think bringing in a library is net more complexity… but getting this finally right is a big relief.
When you navigated directly to ItemPage, the new `safeImageUrl` function would crash during the loading state, because it was trying to safe-ify `undefined`.
Now, I've just made `safeImageUrl` more resilient to that particular kind of unexpected input, by passing through null-y values without change.
When we decided to start out with /api/assetProxy, we didn't know how much the load would be in practice, so we just went ahead and tried it! Turns out, it was too high, and Vercel shut down our deployment 😅
Now, we've off-loaded this to a Fastly CDN proxy, which should run even faster and more efficiently, without adding pressure to Vercel servers and pushing our usage numbers! And I suspect we're gonna stay comfortably in Fastly's free tier :) but we'll see!
(Though, as always, if Neopets can finally upgrade their own stuff to HTTPS, we'll get to tear down this whole proxy altogether!)
I guess something got more picky about the loading sequencing: the fade in animation was happening faster than the cached image could load. Now, we explicitly wait for the image to load (even though we know it's probably cached) before fading it in.
I noticed that, if you're _reading_ the beta callout it's obviously a feedback link, but it's easy to glaze over "Tell us what you think". Here, I've added the word "feedback" to make it stand out on scanning the page, while adding "Got ideas?" to keep it feeling colloquial.
Oops, our movie layer promises don't have a .cancel() method, so calling it crashed our error handler. Now, when there's an error loading a layer and there are HTML5 layers visible, we'll correctly show the "Could not load preview. Try again?" message.
So I broke the Download button when we switched to impress-2020.openneo.net, and I forgot to update the Amazon S3 config.
But in addition to that, I'm making some code changes here, to make downloads faster: we now use exactly the same URL and crossOrigin configuration between the <img> tag on the page, and the image that the Download button requests, which ensures that it can use the cached copy instead of loading new stuff. (There were two main cases: 1. it always loaded the PNGs instead of the SVG, which doesn't matter for quality if we're rendering a 600x600 bitmap anyway, but is good caching, and 2. send `crossOrigin` on the <img> tag, which isn't necessary there, but is necessary for Download, and having them match means we can use the cached copy.)
Oops, I shipped with the images.neopets.com TODO undone! Also, the white background was intersecting with the close X for the feedback form.
In this change, we move the xwee image into our bundle instead of depending on images.neopets.com, and we edit it to have a transparent background, which looks nicer for dark mode. (And we do a srcset!)
Previously I tried to be clever and pre-optimize by putting all the layers onto one canvas… I think this probably helped by batching their paints, but it made fades less smooth by not taking advantage of native CSS transitions, and it made us dip into JS way more often than necessary.
Here, I take the simpler approach: just layers of <img> and <canvas> tags, with each animated layer on its own canvas, and letting the browser handle transitions and compositing, and separate `setInterval` timers to manage their framerates.
I have a suspicion that batching the paints could help performance more, but honestly, maybe that batching is already happening somehow, because things look pretty great on my big-screen stress test now; and so if it _is_ relevant, I want to wait and see after testing on low-power devices.
Not 100% sure on the copy, but I like that it's a bit clearer about the value prop. I tried to work in customization for SEO, but it feels too clunky in a sentence, might need to put it elsewhere in the copy!
Yeah, mm, turns out I don't think it's actually viable to model from Impress 2020, because we can't reasonably set up the SWFs and PNGs in the ways we need, especially for compatibility with Classic DTI.
We can turn this on again later, once Classic DTI is gone, and all assets are converted to HTML5 -- or if we build some kind of bridge to Classic's asset code, or we write new PNG conversion code.
These changes are most relevant for playing around in the dev server, modeing against an empty database. But they'll also help in real-world modeling scenarios! e.g. modeling a new species/color combo is now a bit nicer, we don't show a blank entry in the color picker
this is the last one to get parity with current modeling, I think?? I'm gonna add one more feature though: removing no-longer-used assets from the item
Oops, when building the Support tool to label pet appearances, I didn't realize that there's also a boolean `labeled` field that needs to be true for labeled appearances. Without it, the old app shows the appearance as "Unlabeled".
I also ran this query to fix the rows we'd incorrectly written:
```
mysql> UPDATE pet_states SET labeled = 1 WHERE mood_id IS NOT NULL;
Query OK, 158 rows affected (0.14 sec)
Rows matched: 19640 Changed: 158 Warnings: 0
```
This is mostly because I want to chain the rels after both items and assets save, and I want to be able to specify that stuff a bit more precisely, rather than the like, layers-of-awaits we were building up.
yeah, I had unified Pet into Outfit, but now I think that was overly clever… 😅
Here, I define a new Pet type, and it has some of the fields of Outfit and the deprecated fields still.
I did this because I want petAppearance to work, for UC testing!
We download the schema from prod, and omit real data, but I didn't notice that we were still pulling the metadata of the auto increment counter for IDs! Now, we scrub that from the schema file we save.
I think it's great that we hide the button when it's not relevant, but that makes it hard to know that it exists. Here, we do some cute tricks to blink up the "Paused" button when it first appears, even if the user doesn't have the controls visible right now
We did this a while back too, but I guess something changed in Apollo: I guess it used to return identical item objects from the cache on its own, and now it returns brand new item objects. So we gotta do the object caching hacks ourselves!
This speeds up add/remove item state updates from 500ms to 100ms on my Mac, because we stop re-rendering all the Item components and their complex Chakra children.
This is especially worth doing now, because animations make long updates much more noticeable! (It interrupts the animation 😅)
Honestly kinda surprised this worked on the first go! I was worried something about the process would make the sorta like, instant-cache expectation not work.
Still thinking it might be considerate to like, keep a LRU cache of MovieClip options, so that we don't double-execute these scripts when adding stuff… we even re-execute the ones already applied lol 😅 and that adds lots of script tags to the body!
But yeah I'm not gonna push on it yet until I see evidence that it actually causes performance issues in practice
This is really very cute, but too many items it turns out are lod despite not actually being animated 🙃 it's helpful for looking for test cases tho, so I'm keeping it, but support only!
I also ended up really liking the icon-badge+tooltip design as a way to summarize lil things, so I'm trying Own/Want short badges in the same style.
reflecting further on the abstraction, I'm noticing that this isn't an Easel abstraction like I envisioned early, and that we're baking some Neopets stuff into it. And I think that's the right call, esp with the tricky MovieClip stuff coming up, where I think more barriers would hurt more than they help. So, a new name!
Did it revert? Or did I just never notice that it only worked on mount, not on new loading states?
Also, fixed a bug where we were injecting the script tag way too much, and triggering loading too much that way too!
still just for static stuff, but it's good to be working!
PosePicker got a bit broken, CSS scaling doesn't work quite right anymore, we might need to just up the internal resolution or something?
notable layout change is that the text content will now try to center itself, and we push the buttons off to the right. we also needed to tweak the layout code a bit to get the buttons to feel centered with the top two lines, bc centering against the full block just feels wrong, they want to be top-y in terms of positioning, but still feel centered-y in terms of visual balance
Honestly I don't think this is the long-term home for the Modeling link, I think it'll become a homepage-only link as we add more modules there. But I wanted to get it out of the way!
This wasn't actually super helpful to read anyway, and I think it was causing us to hit rate limits.
We can maybe add back a limited version to like, add path context of _where_ a span happened in the GQL tree, but like, I feel like that's typically been pretty intuitive so far.
Boom, now we can also run a clean MySQL test db on each test that wants it :)
the test I wrote as a sample is currently marked `it.skip` because it's not passing yet!
This updates the MySQL procedure to get the important special colors, but keeps the GQL behavior the same by only filtering to Blue. Just an incremental step before changing the behavior, to make sure I've gotten it right so far!
Snapshots significantly updated, but, from scanning it, I think that's expected changes from actual modeling progress. Hooray!
Fading the whole preview to a black overlay for the loading state was feeling aggressive, especially since loading delay wasn't working correctly!
In this change, I fix loading delay, and I add a nice subtle "corner" variant for outfit preview spinners :)
Here, we add loading skeletons to lots of individual elements, instead of doing a whole item placeholder skeleton. That helps when coming from pages where we have some data, like name and thumbnail, but things is isNc are still missing.
It looks nice, but also particularly means we can handle the loading for the preview separately, get that started faster and iterate better on it in dev!
Initially the spinner was only used in OutfitPreview, where the background was always pretty dark. Now that we use it in more general contexts, we need a light/dark distinction!
Also went and standardized out the `size` props
Here, we extract a lean WardrobePageLayout component, so that we can bundle it into the main app as a loading state for WardrobePage.
This means that clicking Start from the homepage will, instead of flashing the screen to white while WardrobePage loads, show the correctly-sized black/white page layout instead.
I think the Chakra upgrades made these overrides stop working? added !important so that they happen again!
The regression meant the homepage looked worse, always having the selects fade in :/
I'm using my first ever MySQL Store Procedure for clever cleverness in caching the modeling query!
I realized that checking for the latest contribution timestamp is a pretty reliable way of deciding when modeling data was last updated at all. If that timestamp hasn't changed, we can reuse the results!
I figured that, because query roundtrips are a bottleneck in this environment, I didn't want to make that query separately. So, I built a MySQL procedure to do the check on the database side!
I think I got all up in my head about direct queries for this one, because of a previous implementation I had in mind, and I forgot that I could just query species and color from the cache by reference without breaking out of the API provided to the cache function!
I also learned in here that I _can_ look up things from the root by doing `readField("allSpecies", {__ref: "ROOT_QUERY"})`, which I struggled to figure out my previous time. I couldn't figure out how to read an uncached field with arguments (I couldn't quite figure out how to build a proper FieldNode, and passing the string form seemed to provide `null` to the `species` cache field reader), but it's probably doable!
(the permissioning happens on the backend in the prev change! but yeah we send the auth token in the headers, so the backend knows who you are and whether to show you private data)
(also it is just owned items not in any list!)
Essentially I want the center-y visual balance around the image, and the name and _one_ line of tags. If there are extra tags, I want that to go down on its own, rather than bringing down the image+buttons to center against them.
The single-line-of-tags case is the most common, and I think it makes things feel more consistent for all the items to stick to that basic layout, while trying to keep that layout feeling center-y
I was getting annoyed by how, when you're using search, trying on an item will remove conflicting stuff, and then if you decide you don't like what you tried the old stuff _doesn't come back_
As of this change, it does! When you start a new search, we save the outfit state, and then whenever you change the items we ask "hey can these old ones safely be re-worn again?" and re-wear them if so.
On the search panel, not all items have the remove button, and it's confusing to have the other buttons be in inconsistent positions!
Move the Remove button to the left side of the list, so that everything else is positioned the same regardless
Okay, we handle the new pages correctly! Still some weird bugs when you send requests near each other? Probably wise to migrate to Apollo's new way of doing this
This is just an implementation thing, but I realized we can just insert the Zone data into the initial Apollo cache, instead of doing weird field definitions
I _do_ still want the @client tags in the queries though, to tell them not to make server requests at all
Been bothered by this for a while!
My hope is that this isn't a notable marginal performance hit—we were already walking the table and doing string ops anyway, I can't imagine adding to that is actually that much of a marginal lift, when the main bottleneck was probably reads. And the perf should be identical for simple single-word queries anyway. But we'll see how it feels!
This was a subtle little thing for a while! If you switch species/color, such that an item doesn't fit the pet anymore, we used to just hide it. Now, we show it in a list, so that you can understand what went wrong, and have the option to remove it.
Previously, if you switched species/color such that one of your items was no longer compatible, we _would_ still apply its zone restrictions to the visible layer set.
In this change, we fix that server-side, since I think it makes the most sense for an empty appearance to be truly empty!
This was bothering me, I'm surprised and pleased by how easy it seems it was to fix? :)
The strategy is just, look for groups that are provably redundant, and filter them out.
I hope it's correct! It's definitely cozier. Kyrii Mage items are good tests, they have a lot of interesting zones!
I figured that we'd want simpler UI in the ItemsPanel when possible… but now that we've got it pretty simple and comfy, I think the consistency is better
These are nice! :)
The `hideSimpleZones` option I'm not sure about yet, but I figure that:
1. For a new user just doing simple outfits, I feel like the double data on the items page just looks silly, so I want to streamline for that
2. But I _do_ want to let the user think about zone complexity when things _are_ multi-zone.
I did also consider just hiding the zone badge for the header you're under, but I figured the consistency of having the item and its badges look the same in all the places in the list was more important.
This is in preparation for hiding bio zone restrictions but showing item zone restrictions!
I also refactor the build-cached-data script substantially, to run GraphQL against the server instead of a custom query.
okay so the PetAppearance restrictions are stored on the asset, because that's how they're defined on Neopets.com too
but I think that's a confusing API, so here I define `PetAppearance.restrictedZones`, which just maps over the layers and aggregates the zones server-side, same as we would have done on the client
I think that's much easier to understand than having layer contain a field, but having to know that item restrictions _don't_ work that way, you know?
Huh, maybe this is a Firefox bug or something? but the container wasn't applying partial opacity correctly to its children, it was only doing 0 or 1, I think maybe because the children weren't static? I refactor here to make the DOM structure a bit more natural, and fade ins work again 🤷♀️
I'm not sure when this regressed, but changing the outfit was clearing out the whole preview and showing an empty loading state, instead of the intended behavior of showing a loading spinner over the old preview. This affected both the home page and the wardrobe page.
Yeah, so, huh. Fixed! I hope I didn't goof anything else with these effect trigger changes though 😅
The reason I'm doing this now is not just that it's annoying, but as a pair with the color change fix from just now, I want those color changes feeling buttery smooth!
Previously, when changing a pet's color, we would refresh the items panel and send a new network request for the item appearances, even though they're all the same. This is because item appearance data is queried by species/color, for ease of specification.
But! Item appearances are //cached// by body ID. So, if this is a standard color, it's not hard to look in the cache for the standard color's body ID!
Now, most color changes are faster and don't flicker the item panel anymore. We do still refresh the panel and send the requests for color changes that _do_ matter though, like standard <-> mutant!
ahh, in a recent change I made glitched states valid for canonical poses, but didn't make the corresponding change here! This meant that I think the PosePicker showed them, but other ways of getting to them didn't work, including the Candy Acara (who is 100% marked glitched) was no longer pickable at all
Oops, opening the dropdown would auto-focus the left arrow button, because it's now the first focusable element! Make explicit that we want the dropdown instead.
I noticed in prod that the Vercel edge cache can show old data in the Support tool right after you edit it and reload the page, which is super confusing!
In this change, we stop caching the endpoint we use for Support tools, so that the Support tools always feel real-time and trustworthy. (The standard pose picker might still be cached, so it could be a bit confusing for that to be out of sync, but at least you can toggle into Support mode and see that your changes happened _there_, so you don't panic that they're _gone_.)
Previously, I was filtering out glitched appearances from the canonical ones.
But now, I'm thinking it's better to serve glitched ones than no data for a pose at all.
I'm inspired by the case of the Candy Acara, which has _only_ glitched appearances in our db, and I'd like to mark them for reference—but then the site would treat it as no data at all.
Easier to move between appearances quickly! I'm adding this in anticipation of a use case of rapidly labeling Unknown appearances—I want it to be easy to label one and go to the next!
For most users, I want to hide the pose picker if there's not actually anything to pick from.
But I want Support users to be able to open it and use Support mode, to inspect and label Unknown poses!
In this change, I add new UI to show a question mark pose picker button, and a note explaining the empty picker; but only show them for Support users.
I also made a new `useSupport` hook, to replace `useSupportSecret`, now that I have a use case for "is support user?" that doesn't require the secret. Will migrate the rest!
Previously, toggling between the two PosePicker modes could dramatically disrupt the layout, because Popover didn't know to re-measure itself.
In this change, we add a hacky workaround to simulate a window resize event in moments when we know the content is changing. (The realistic ideal solution would still have these manual triggers, but would use an official API in Chakra to notify the Popper instance directly.)
Still just read-only stuff, but now you can look at all the different poses we have for a species/color!
Soon I'll make the pose/glitched stuff editable :3
Some sizable refactors here to add the ability to specify appearance ID as well as pose… most of the app still doesn't use it, it's mostly just lil extra logic to make it win if it's available!
(The rationale for making it an override, rather than always tracking appearance ID, is that it gets really inconvenient in practice to //wait// on looking up the appearance ID in order to start loading various queries. Species/color/pose is a more intuitive key, and works better and faster when the canonical appearance is what you want!)
similar to the layer zoning tools I just rolled out!
not thrilled about the outfit state hacks here bc of how we cache restrict on the appearance rather than the item, but oh well! this escape hatch is pretty easy and solid, and it's a cleanup for another day
Also did a code split here, now that this file is getting larger, to only load this for support users. I don't actually care about restricting console stuff to support users (I'd honestly rather not), but saving the bytes is worth it I think, since support mode is pretty easy to enter when we need to
This is in support of a caching issue in a hack tool coming next! Without this, the change to ItemAppearance restricted zones would make other ItemAppearance fields go missing (bc our hack tool didn't also specify them), so the query would re-execute over the network to find the missing fields we overwrote with nothingness—which would undo the local hack change.
A dev tool to preview what an item would look like in a different zone!
To use, open the browser console, and type `DTIHackLayerZone(layerId, zoneIdOrName)`, but replace `layerId` with the layer's DTI ID, and `zoneIdOrName` with either the zone ID or its name in quotes.
Previously, we were using a custom-y `id` field to help Apollo cross-reference `petAppearance` queries with the results from bulk `petAppearances` queries. Now, instead, we deprecate `petStateId`, and start using `id` to have the same stable value!
This is in anticipation of pet appearance support tools: a stable ID will make it easier to edit them, esp changing their pose (which would otherwise have changed the ID!)
Previously, we would load all `petAppearances` in `PosePicker`, and use cache keys to instantly find it again as a single `petAppearance` in `OutfitPreview` after switching poses.
In this change, we instead have `PosePicker` explicitly load all 6 poses as separate `petAppearance` queries. This simplifies cache sharing between the two components' queries: Apollo can do it automatically, because they were queried the same way in the first place.
I'm doing this in preparation for changing the `id` field of `PetAppearance`, to become `petStateId`. This will help me build pet appearance support tools, by giving the appearances stable identifiers that won't be affected by editing which pose an appearance is!
Huh, some 8-bit species are broken and use the standard body ID!
This was causing our body name query to prioritize 8-bit for standard assets, as the alphabetically-first compatible color; but 8-bit isn't marked standard, so the function kept it labeled 8-bit.
This should fix it and show "Standard Draik" when deleting an asset off the standard draik body!
In practice I saw that this doesn't actually tell you what you _really_ want to know about where the change happened! You want to know it was broken on the Acara or w/e.
Dice reported this, thank you!
My mistake here was that `loadImage` _does_ reject when the image fails to load… but it ends up throwing `undefined`, since I forgot to pass the error along from `onerror` to `reject`!
So we would cancel stuff, but then store `undefined` as our error in state, which our component interprets as no-error.
I tested this by using Firefox DevTools request blocking!
On the homepage, I want to keep the ability to enter invalid species/color pairs, so that you can say "Alien, Aisha" instead of having to pick the Aisha first.
But on the wardrobe page, we were rejecting invalid state changes anyway, so I decided to remove invalid color options from the list. And I added an ability to still switch to any species, and potentially resetting to a basic color automatically to match.
Oops, our loading state logic was eating the error case! I'm not sure exactly where the gap was happening, but I've rewritten the states to be a bit more foolproof, since that first condition was confusing I think.
Ahaha I fucked up a bit! I was indexing into the array of cached zones, instead of looking up by ID. This meant that all zone names were wrong, and some search results weren't loading bc there was no zone data!
I made a fix here, and also added some fallback values, so that if there's an issue in the future we can at least fall back more gracefully than the infinite-spinner case we had here.
In this change, we cache the zones table as part of the JS build process. This keeps the database as our source of truth, while aggressively caching the data at deploy time.
See the new README for some rationale!
I tested this by pulling up dev Honeycomb, and observing that we no longer run db queries to `zones` in the new traces for the wardrobe page. (It's a good thing we did it this way, because I noticed some code in the server that was still loading the zone anyway, and fixed it here!)
This reverts commit 0f7ab9d10e.
The Production Vercel deploys don't seem to like how I did this build trick, even though the Preview deploys seem fine with it 🤔 Reverting for now, sent a message to Vercel support.
Here's just some simple caching: we try to load the asset manifest from the db with the rest of the asset. If it's not present, we load it via HTTP, and write it to the database.
I might try to do a bulk write of manifests at some point, too.
This is because I noticed that one of the main bottlenecks in most of the endpoints now (and definitely the highest-variance) was loading from images.neopets.com.
Another approach I considered was HTTP/2 to load the manifests, because it kinda looks like the server is refusing to open all these sockets at once and effectively does the requests in waves? But images.neopets.com doesn't support HTTP/2 right now anyway, so oh well! (And that would have probably cut us down to ~250ms of HTTP time still, instead of ~600–700. Also, why is network out of Vercel so slow? :p)
I noticed that, while looking up zone data from the db is near instant when you're on the same box, it's like 300ms here!
In this change, we start downloading zone data into the build process. That way, we can have a very fast and practically-up-to-date cache (I'm not sure I've changed it in many years), while being confident that it's in sync with the database source of truth (for things like join queries).
Oops, of course, we weren't actually taking proper advantage of the dataloader here! The queries got over-complicated, but more importantly, subsequent requests to the same loader would re-submit the query!
I noticed it in the SearchPanel operation, in this Honeycomb trace:
https://ui.honeycomb.io/openneo/datasets/dress-to-impress--2020-/trace/aMuhsTjQFZY
We got bit by the "can't run anything after the response finishes" thing
so I'm just forcing the response to wait for Honeycomb submit to finish
I hope this isn't like, just awful for perf lol. but puts to honeycomb seem fast?
An even better resolution mode for "Dyeworks Pink: Peaceful Tree Garland", which doesn't leave any big chunks or holes, and instead takes all the leaves and gives them chunky outlines 😅
The "Dyeworks Pink: Peaceful Tree Garland" was a tricky case, with animated falling leaves…
we decided that having transparency in the main pet area, and some incorrect transparent holes in the trees, was a better conflict resolution for this one
Probably would be good to manually upload a Totally Good version, but like, this flag is probably good to have
Hey wow this was not so hard, just set some global styles, removed some hardcoded colors, and walked through the remaining hardcoded colors to pick a dark mode variant :) neat!!
oops another thing I missed in the refactor! I wasn't providing the selected pose to the hook, so it wasn't styling any of the poses as selected. Now it does again!
Oops, I removed some fields from the pose object, so the aria-label for the options were coming out as "undefined and undefined"!
Now they come out correct!
I tried to get the alignment juuust right in Chrome and Firefox, but Dice's computer was still including a 1px sliver of border.
In this change, I'm adding 1px of extra padding between the border and the Flash area. I figure that, if the screenshots are incorrectly aligned by 1px, that's no big deal and I'm okay accepting that slightly-incorrect upload rather than having folks not be able to use the tool!
I skipped this in the past runs because I had a hard time getting consistency from the results… but they seem to be behaving now?
It really seemed like there were some races on certain query orders… maybe there still is, but my more-reliable connection today is making them resolve in a more consistent order?
Anyway if I see goofs again, I'll consider adding a snapshot matcher that isn't picky about query order 🤔
This happened in an earlier change too, I forget where, but there's some compiler bug that causes comments placed in this kind of spot to yield this error in production, but not in dev:
https://reactjs.org/docs/error-decoder.html/?invariant=152&args
Oh well, moved the comment!
On mobile, it was pretty annoying that you had to show the controls by tapping the preview area to simulate a hover—because it could also click the underlying elements, and do bad stuff!
Here, I add a click capture to prevent the clicks from going down if the controls aren't visible. And I add a toggle, so that you can dismiss the controls, like how YouTube feels :)
Huh, weird how we seem to need preventDefault for buttons, but stopPropagation for links? idk, in any case, the Remove button was doing that thing where it clicks Remove but also bubbles up to the container and clicks it too :p
This was a surprisingly big win! Item is heavier than it looks, because it has like 6 Chakra components, which aren't expensive but aren't _cheap_ in a re-rendered list that needs to be fast, you know?
And it's even more important on search, where there's a lot of items on the page. (we should virtualize it too but that's a thing for another day)
I noticed that item wear/unwear is slow on mobile, because we re-render the whole app tree, and my laptop handles that super fine, but my few-years-old fun takes ~300ms, which is very noticeable.
There's some hacks we could do to get faster feedback, but first I'm diving into the render tree to find the unnecessary renders and stop 'em! That should help build perf across the board, rather than in just one spot, and hopefully be less of a weird sore spot :)
The button section was capable of flex-shrinking, and having those wrap pretty much always looks bad imo. Here, we get more precise about the areas and their flex rules, including only letting the name area grow/shrink. (If the screen is too small for the name to wrap further, the panel area container gains a horizontal scrollbar, which feels like a really good compromise imo)
I think it's more helpful to auto-switch: if you're using the species/color picker for pet compatibility, that's probably what you want! And it's all buffered behind the Save Changes button anyway, so nbd to play a bit fast and loose!
oof, got "too many connections" from mysql, this is probably gonna be a scaling issue in time… for now, stop requesting a pool of 5, even on dev lolol, and just go with a single connection per instance
One fix was that `position: absolute` stuff was appearing over the drawer, including the item search clear button, oof!
Another fix was for a weird bug(?) in Apollo Client, see comments
Note that there's a bug when switching back to the null case… when I look in the Apollo dev tools, it's definitely getting set in the cache correctly at the right time… but the query isn't updating for some reason? I'm hoping it's an Apollo bug that will fix itself someday with an upgrade!
Optimistic UI seems to just be like, not working… I'm seeing some Google results suggesting maybe just get to v3, which is a bit of upgrade work but might be worth it
Oh oops, Chakra UI Next deprecates usePortal for the popover, so it wasn't escaping properly! Add a Portal component to let it escape the top area again!
I guess if you return a reference to an object that doesn't exist, it registers as null; and you need to provide the `true` here to declare that it _is_ real and should be treated as an _insufficiently_ defined object?
Using the newly extracted OutfitPreview! I'm really happy with how this turned out :3
It also makes the pageload after clicking Start super smooth, no spinner! Thanks Apollo cache!!
Kinda surprised our cache redirect isn't good enough for this, but I guess Apollo can't differentiate between a confident empty list vs an uncertain empty list!
