The immediate motivation here is that there's slowness rn, and I'm
digging through the slow query log for suspicious targets (hard to tell
when even normal queries are getting slowed down by whatever the
culprit is!), and Impress 2020's constantly-recalcuated trade activity
dates were one of them.
So, rather than update this page to use the new table columns Classic
now uses, I figured, let's Delete Things and reduce surface area. The
re-merge continues!
If we're lucky, this change alone will stop the bursts of slowness. If
not, we'll keep looking for suspicious targets!
The analytics service is down rn for reasons I haven't had time to investigate, and oops, using the `defer` attribute *does* defer the script until the page has loaded, but it *also* maintains the ordering.
I could also use `async` here, but I don't know enough about the Plausible script to know if it cares about the DOM being loaded. Instead, I'll keep it the same, but move it after the Next.js script tag.
Sometimes we don't have an image for an SWF asset. This can happen when
assets were converted to HTML5 without a PNG specified in the manifest.
(One example is bio/363, the Hind Biology for the Mutant Blumaroo.)
I'm noticing there's a second issue here on DTI's end, where it looks
like *we* have two copies of certain layers? I wonder if this was a bug
in like, the impress-2020 modeling code we tested at one point? Or if
this is in our main modeling code? e.g. the Mutant Blumaroo's bio/363
is in our database with DTI ID 192 and DTI ID 606955, and only the
former is registered as having an image (which we made ourselves and
host on S3).
So, I'm gonna start by just having the more graceful failure mode of
skipping the missing layer. But I'm also concerned about the root here,
I'll investigate!
We're in the process of migrating away from translating these records,
because Neopets hasn't supported non-English languages in many years,
and it'll simplify our code and database lookups.
In Main DTI, we already wrote code to copy these fields onto the main
records and keep them in sync for now; now, once DTI 2020 isn't
referencing them anymore, it should be safe for the main app to drop
the tables altogether.
Note that some Prettier changes got mixed in here and that's fine!
I also wasn't suuuper careful testing these, most of them seem to be
trivially testable by just loading the homepage or doing a few basic
wardrobe actions, and the others are in Discord support log actions
that aren't enabled in development mode, so I'm just like… ehh I'll do
a couple support actions after deploy and see that they don't crash!
Self-hosted Plausible instance! I have need of usage numbers again,
after a good few years of just not using it; but I don't want to send
the data to Google, and I enjoy self-hosting things, so here we have it!
I actually forgot we were still sending the DTI 2020 numbers to
Plausible's cloud instance, oops! I stopped paying for it 😅 They do a
pretty good job anonymizing, so I'm not too worried about the practical
effects of that, but I'll be pleased to keep it more in-house going
forward.
I'm not sure why `async` isn't in the recommended tag anymore, but fine
by me!
Lol I guess we've probably just been having intermittent CORS issues
forever, oops. I hope the cache has been mostly warmed on the right
thing! But today I checked and the entire species/color picker on the
Rails app wasn't working for CORS reasons, so like. Yeah oof.
Without this, 150x150 and 300x300 outfit thumbnails would fail to render new item layers where we didn't have an AWS image layer. Now, they correctly render the new stuff!
I tested this with the new "Spooky Stitches Markings" on the Grarrl, which has a blank image in AWS, but works correctly in the new code by loading the image from neopets.com!
Okay, this is gonna be a drop-in new backend for impress-asset-images.openneo.net, to enable Classic DTI to use the same images as DTI 2020!
This will enable us to stop generating images and uploading them to S3 just for Classic's sake, so we can turn those background processes off! And the new modeling script skips that anyway, so this is an important compatibility step for the new data that went out today!
We're gonna update impress-asset-images.openneo.net to perform redirects and stuff, so Classic DTI can start using the same images that DTI 2020 does.
That should enable us to stop relying on AWS for images, which is important because the new modeling script breaks that anyway :p but this will also let us turn off the image converters that run in the background all the time, and I'm excited for that too!
Oops, Next.js has built-in request body parsing that happens automatically. So it was giving us a `req.body` string, and our code to read in the body and put it in a buffer was waiting forever!
Thankfully, it's much easier than I expected to turn that behavior off for just one route. Now it works like before, so our existing code works again, ta da!
Before, we were using the ContentType from the S3 object, which was unreliable. This helps us behave better for query-string files!
We also add a filename via Content-Disposition for the files that auto-download and for the Save As case. Idk if this is super important exactly, but I feel like it'll be a lifesaver for anyone using this to get at a specific file for their own reference at any point! and it just seems polite lol
Okay so the funny thing is that my upload script is clearly like *super* not working lol, it's been running more than an hour now and still hasn't finished listening the files. So there's only actually a handful of files to test with here, from the `archive:create:upload-test` script!
But anyway, uhh once the archive is actually uploaded, this is a way to read it back! Mainly as a way to assure me that it's all saved correctly, but also as a potential backup for images.neopets.com if it goes down again sometime.
Ok! I think I got it! It's very very nastly tho lmao! But this will merge in the new SSR-provided data before the new page can render, instead of having it sometimes make redundant network requests & show loading spinners in the meantime for data that Next.js already fulfilled for it.
Nasty nasty lil trick. But it seems to be working! Let's see how it does lmao
Ahhh right, a new `initialCacheState` value comes in on every navigation, so if our memoized Apollo client depends on that value, then it's gonna keep getting reset, and thereby dumping everything out of its cache. Rude.
This solution is clearly incomplete, the ideal would be to merge the SSR'd data into the cache each time. But it should be fine in practice I think, we already have good coverage of preloading stuff via GraphQL anyway!
Always been a bit annoyed to have even the item name load in so weird and slow 😅 this fixes it to come in much faster!
This also allows us to SSR the item name in the page title, since we've put it in the GraphQL cache at SSR time!
Whew, setting up a cute GraphQL SSR system! I feel like it strikes a good balance of not having actually too many moving parts, though it's still a bit extensive for the problem we're solving 😅
Anyway, by doing SSR at _all_, we solve the problem where Next's "Automatic Static Optimization" was causing problems by setting the outfit state to the default at the start of the page load.
So I figured, why not try to SSR things _good_?
Now, when you navigate to the /outfits/new page, Next.js will go get the necessary GraphQL data to show the image before even putting the page into view. This makes the image show up all snappy-like! (when images.neopets.com is behaving :p)
We could do this with the stuff in the items panel too, but it's a tiny bit more annoying in the code right now, so I'm just gonna not worry about it and see how this performs in practice!
This change _doesn't_ include making the images actually show up before JS loads in, I assume because our JS code tries to validate that the images have loaded before fading them in on the page. Idk if we want to do something smarter there for the SSR case, to try to get them loading in faster!
Okay so there's a bug here where navigating directly to /outfits/new?species=X&color=Y will reset to a Blue Acara, because Next.js statically renders the Blue Acara on build, and then rehydrates a Blue Acara on load, and then updates the real page query in—and our state management for outfits doesn't *listen* to URL changes, it only *emits* them.
It'd be good to consider like… changing that? It's tricky because our state model is… not simple, when you consider that we have both local state and URL state and saved-outfit state in play. But it could be done! But there might be another option too. I'll take a look at this after moving the home page, which will give me the chance to see what the experience navigating in from there is like!
Hey we're getting real close! :3
I accepted a small bug here where clicking the breadcrumb to "Items X wants" won't scroll down if the page isn't loaded yet. (e.g. you landed on this list page first). If you came *from* the lists index page though, then when you go back your stuff will be there already, so you should be fine. (It might also happen if the page loads fast enough, which in prod it might do?)
Just gonna leave it for now, because the workaround would be a lot! (have the page re-check the anchor once it's done loading)
We're getting so close! :3
There was some shared components in `UserItemListPage` that needed updated too, even though the rest of the page isn't migrated yet.
Okay, when I saw the recipe in the Next.js docs with `getLayout`, I was like "psh this API is so confusing, this should just be a component"
anyway now we see why it wasn't a component: the _whole point_ of it was to circumvent the usual React diffing algorithm's belief that two different components _can't_ ever share UI. But here we were, making different `layoutComponent`s that were meant to share UI, lol!
Anyway, if you just _return JSX in a function_, the React diffing algorithm never sees that it came from a different place, so it's generous when diffing them. Neat!
But I still changed the recipe's `getLayout` name to `renderWithLayout`, because it just confused me so much at first lol, I thought it was going to like, return a layout function? This is much clearer verbing to me imo
Okay I actually screwed up the layouts thing a bit! Because right, they need to *share* a LayoutComponent in order to share the UI across the pages. This gets a bit tricky with wanting to change the margin, too. I'll address this with an upcoming refactor!
The tricky part here was that `returnPartialData` seems to behave differently during SSR. On the page itself, this seems to cause us to always get back at least an empty object, but in SSR we can sometimes get null—which means that a LOT of code that expects the item object to exist while in loading state gets thrown off.
To keep this situation maximally clear, I added a bunch of null handling with `?.` to `ItemPageLayout`. An alternative would have been to check for null and put in an empty object if not, but this feels more resilient and more true to the situation.
The search bar here is a bit tricky, but is pretty straightforwardly adapted from how we did the layouts in App.js. Fingers crossed that it works as smoothly as expected when the search page is migrated too! (Right now typing in there is all messy because it hops over to the fallback route and does its whole separate thing.)
This one is a bit trickier, because it doesn't use a page layout, and we had to make some fixes in OutfitMovieLayer! Nice to get a head-start on that though :3
The first page moved over! Note that this broke navigation on the rest of the app, so don't deploy this until we're done!
The reason it broke was that we had to migrate GlobalHeader and GlobalFooter to the Next.js link & router stuff too, or else it crashed because it wasn't in a react-router-dom context.
The Next.js from react-create-app codemod automatically added this, just in case it mattered for us. It doesn't! I'd rather just avoid the code complexity.
