Oh right, it's possible for `Item#pb?` to return true, but
`Item#pb_color` to return `nil`, if the item has the paintbrush item
description but we can't find a color whose name matches the item name.
This would be expected if a new color were added to Neopets, and PB
items for it were modeled by the community, but we hadn't manually
added the color to the database yet.
Previously, the Item Getting Guide would crash in this scenario. Now,
it correctly handles the possibility of a `nil` value for `pb_color`,
and shows some placeholder info.
To test this, I temporarily edited some item names to not contain the
color name anymore (e.g. "P-rate Elephante Shirt and Vest"), then
loaded the guide and made changes until it no longer crashed.
I noticed in the app that these queries were slowwww! I was able to
track it down to a bad query plan, as we explain in the comment.
I searched online for "mysql query performance filter on one join table
sort by another", and was surprised to find this answer suggest a
subquery, which I've often been told to expect to be slower compared to
joins? But it certainly worked in this case!
https://stackoverflow.com/questions/35679693/mysql-optimize-join-with-filter-and-order-on-different-tables
I think the Rails query cache handled these anyway? But `SwfAsset` has
a `before_save` hook that checks its zone's info, and
`SwfAsset.preload_manifests` saves all the assets, on the assumption
that saving is a no-op when the record didn't change anyway. And it
basically is!
But I figure that, now that I'm realizing hooks exist, simply not
attempting to save unchanged records is probably a better
representation of what we intend to do. So I'm fixing it like that!
Another potential fix would be to preload the zones for these assets,
but I think that confuses the intent too much; the method itself isn't
using the zones, it's just a weird incidental thing that a save hook
happens to use. (Would probably be better to refactor this old save
hook into a different situation altogether, but that's for another
time!)
This is a minor nbd change, I just noticed when playing around in the
console that, unlike most other errors for this model, the `body_id`
being required is _only_ enforced in the database schema, so it isn't
returned with the usual errors. Not a big deal! Just feels like this is
clearer to work with, and more correct to what we *intend*.
This is just a bit of future-proofing! We also add a default thumbnail
URL of the cute "Neopets Circle Background", for cases where the series
name isn't known yet.
Make them the same size, add spacing between them, and also put the
"Get these items!" on the right, because the list is right-aligned and
the Save button has dynamic width (the save vs saving vs saved states),
so this makes things a lot more consistent and stable!
A funny table-layout bug, where the item "Portal to the Unknown" had a
very long Owls listing ("Owls listing: Buyable - Magic Lens + Blank
Grey Tome (NP)"), and so the table layout tried to give it more room by
decreasing the width of the action cell and wrapping the "NC Trades"
action button text onto multiple lines.
The fix: don't allow that! The table layout will figure out how to
handle this being disallowed, and give the actions cell an appropriate
minimum width.
You can now go to `/outfits/new?features=get-these-items` to start
seeing the "Get These Items!" button in the outfit editor!
I haven't tested it a ton yet, but yeah here it is!
Oh oops, I added an optional `subtitle` argument for the item table
list row template, but didn't realize that it would crash in cases when
not provided! Now, we add an initializer to set it to `nil` if
undefined.
This doesn't matter a ton in production, where we already have most of
our manifests loaded! But it matters a lot on my relatively-fresh
development instance, at times like now when images.neopets.com is slow
to respond. A single item search was taking minutes before this change
(5 seconds of timeout per asset for 30 items!), but now takes a few
seconds the first time, as it should!
Note that there's a known performance issue here: we should try to
fetch all the OWLS values at once, instead of doing them in sequence
while rendering the page!
Oh jeez, idk why this was ever in here, but yeah no, I want to be using
default browser focus outlines unless specifically overridden otherwise.
Will help keyboard navigation a lot! Yikes!!
Now, for colors like Mutant or Magma where there's no paint brush
image to show, we use a sample pet image instead, to help it have equal
visual weight and clarity as the cases with the paint brushes.
We do some cleverness in here to make sure to always show the relevant
species, if possible!
Ohh yeah, this helps communicate the process much better, especially
for what the Shops/Trades links mean.
I think I'm gonna also go get the paint brush thumbnail images and add
them to the database too, to help better communicate that this is a
paint brush item situation.
Clearer focus states, row changes bg color on hover/focus to help you
track where you are, remove the redundant image thumbnail link from the
tab order (it's the same as the item name link!)
As part of this, I added a new `search_icon` helper, and a new
`button_link_to` helper, which both styles the link as a button and
accepts an `icon` parameter to make it easier to pass in an icon!
I'm not sure when TNT changed this, but the Shop Wizard has a new
canonical URL now! The previous one redirects, but it does *not* pass
along query string parameters, so these buttons would correctly go to
the Shop Wizard but not auto-fill the name!
Now, we go directly to the new canonical URL, with query params in hand!
This is just a tech update: instead of using hand-built URLs with
`CGI::escape`, I use `Addressable::Template`, which is a more reliable
way to build URLs in general.
The motivation here is that I noticed the Shop Wizard link is actually
broken! And I wanted to fix this while I was here, but I figured let's
split that into a separate commit than this refactor. See next!
This helps us be more efficient with our use of space, keep the CTAs well
aligned, show a clear total, and set up how we might do CTAs for more complex
cases like all the potential Neopoint CTAs like Wiz/Trades/Auction/etc!
Now that we have this helper, we no longer need these stylesheets to
include a `body.controller-action` wrapper to scope all the styles!
Someday we should convert more of our stylesheets to this format,
instead of slamming them all into `application.sass` like we do now.
Ah, well!
NC prices, some CSS, and also a new application-level helper that adds
a feature I've long wanted and been working around for Turbo: the
ability to specific that a stylesheet is specific to the current page,
and should be unloaded when removed!
I use this to write `sources.sass` without the usual
`body.items-sources` scoping that we've historically used to control
what pages a stylesheet applies to. (In the long past, this was because
a lot of stylesheets were—and still are–routed through the
`application.sass` stylesheet! But even for more recent standalone page
stylesheets, I've done the scoping, to avoid issues with styles leaking
beyond the page they're meant for when Turbo does a navigation.)
This'll affect the recommended acquisition method by a lot!
NC Mall info like current price isn't surfaced anywhere else in the app
right now. It'd probably be good to add to the item page, and maybe
some other places too!
Currently we only load the homepage, so there's only actually one
wearable item to sync up! But here's the task to do it!
To do this, we also created the backing model NCMallRecord, where we'll
save the current NC Mall state!
This doesn't connect to anything yet, I'm just doing the beginnings of
loading NC Mall item data!
My intent is to run this regularly to keep our own NC info in the
database too, primarily for use in the Item Getting Guide. (Could be
useful to surface in other places too though!) This will help us split
items into those that can be one-click purchased with the NC Mall
integration, vs NC items that need to be acquired by other means.
TNT requested that we figure out ways to connect the dots between
people's intentions on DTI to their purchases in the NC Mall.
But rather than just slam ad links everywhere, our plan is to design an
actually useful feature about it: the "Item Getting Guide". It'll break
down items by how you can actually get them (NP economy, NC Mall,
retired NC, Dyeworks, etc), and we're planning some cute actions you
can take, like shortcuts for getting them onto trade wishlists or into
your NC Mall cart.
This is just a little demo version of the page, just breaking down
items specified in the URL into NC/NP/PB! Later we'll do more granular
breakdown than this, with more info and actions—and we'll also like,
link to it at all, which isn't the case yet! (The main way we expect
people to get here is by a "Get these items" button we'll add to the
outfit editor, but there might be other paths, too.)
Oops, prior to this commit, searching for "white peach" would return
nothing, whereas now it correctly returns the "Dyeworks White: Just
Peachy Filter", like if you search in the Infinite Closet!
This solution is a bit hacky, wrote some more in the comments about how
to maybe do this better!
Oops, right, this meta tag that runs on all pages currently crashes if
we can't read the credentials file!
Instead, let's just allow this value to be `nil` if not present.
Idk how we got into this state, or if it's environment-dependent or
MySQL-version-dependent or what, but setting up the dev environment on
my macOS machine is complaining that `TEXT` columns can't have default
values.
Well, in that case, let's just have it be a non-nullable field, and add
a note to our code that missing fields *can* cause item saving to fail!
(This was always true, but I'm just extra-noting it because it's
becoming *more* true.)
I refresh the image and UI color here to draw attention to the change!
I also delete the `neopass-thumbnail.png` image, since it's no longer
used anywhere anymore, but I would not be surprised if we want it back
someday and need to revive it from history!
Simple enough to start! If `shadowbanned: true` gets set on a user,
then we show a 404 instead of the actual list page, *unless* you're
logged in as that user, or coming from a known IP of that user.
This isn't a very strong mechanism! Just something to hopefully
increase the costs of messing around with list spam.
Just a lil blurb to make sure it's clear that NC sales and stuff are
forbidden! I imagine the people doing it know this, but I want to make
sure we're being explicit, in case there's any element of
miscommunication.
This hasn't been causing issues as far as I know, I just noticed
*months ago* that I forgot to do this, and have had a sticky note about
it on my desk since then lol.
I tested this by temporarily setting the timeout to `0.5`, and watching
it fail!
A further optimization, this lets us use the image hash as the new hash
for the pet type if it would be useful! (whereas before this change,
we'd dip into `fetch_metadata` and just get back `nil`, which was okay
too but a little bit less helpful!)
Ahh, we recently added a step to pet loading that sends a metadata
request to `PetService.getPet`, which is now (in a sense, correctly!)
raising a `PetNotFound` error when we try modeling with a "pet" that
starts with `@` (a trick we use in situations where we can get an image
hash for a modeling situation, but not an irl pet itself).
In this change, we make it no longer a crashing issue if the pet
metadata request fails: it's not a big deal to have a `PetType` have no
image hash or not have it be up-to-date!
In the next change, I'll also add an optimization to skip fetching it
altogether in this case—but I wanted to see this work first, because
the more general resilience is more important imo!
In particular, we got feedback that it was surprising to not get to
check which NeoPass you wanted to use, and that the permissions were
never prompted again. I figure let's err on the side of ample clarity!
As part of this, I've added the new `external_link_icon` global helper,
which embeds an SVG from Chakra UI. That's just the convenient place I
know to grab that icon, and I did it this way instead of an `img` tag
because that enables the `currentColor` thing to work instead of coming
out black!
Not getting a lot of takers, I think it was wise to start small just in
case, but there doesn't seem to be a floodgate problem, so let's remove
the limitations and increase the ask! (But still not a full launch yet,
because I want to funnel people through the feedback process first.)
Got the icon and background style from Neopets.com! I didn't quite copy
the whole button style, both because getting it to play nice with our
existing styles didn't *immediately* work, but also because I think
this works out as a really good compromise between our two styles
anyway!
Yay, we got the API endpoint for this! The `linkage` scope is the key.
Rather than pulling back the specific fallback behavior we had wrote
for usernames before, which was slightly different and involved
appending `neopass` in there too (e.g. `matchu-neopass-1234`), I
figured let's just use a lot of the same logic, and just use the
preferred name as the base name. (I figure the `neopass` suffix isn't
that useful anyway, `matchu-1234` kinda looks better tbh! And it's all
fallback stuff that I expect serious users to replace, anyway.)
Note: I validated this was working by temporarily changing the URI to
`https://echo.free.beeceptor.com`, which echoes the headers back, then
called `OwlsValueGuide.load_itemdata` directly.
Note: I validated this was working by temporarily changing the URI to
`https://echo.free.beeceptor.com`, which echoes the headers back, then
called `NeopetsMediaArchive.load_file_from_origin` directly.
Oh right, I never did catch this when setting up User-Agent in the app!
(I noticed this because I'm making a new request now, and went to look
how we set it in previous stuff, and was like. Oh. We don't anywhere
right now. Interesting LOL)
Oh right, if you can remove your email, there's a way to fully lock out
your account:
1. Create account via NeoPass, so no password is set.
2. Ensure you have an email saved, then disconnect NeoPass.
3. Remove the email.
4. Now you have no NeoPass, no email, and no password!
In this change, we add a validation that requires an account to always
have at least one login method. This works well for the case described
above, and also helps offer server-side validation to the "can't
disconnect NeoPass until you have an email and password" stuff that
previously was only enforced by disabling the button.
That is, the following procedure could also lock you out before,
whereas now it raises the "Whoops, there was an error disconnecting
your NeoPass from your account, sorry." message:
1. Create account via NeoPass, so no password is set.
2. Ensure you have an email saved, so "Disconnect" button is enabled.
3. Open a new browser tab, and remove the email.
4. In the original browser tab, click "Disconnect".
This is gonna help me in development, to stop having to add stuff to
the URL all the time!! I also considered just always making it
available in development, but I wanted to match production behavior to
help us ensure the hiding behavior is working, to avoid leaking NeoPass
without realizing.
Ahh okay tricky lil thing: if you show the settings page with a partial
change to `AuthUser` that didn't get saved, it can throw off the state
of some stuff. For example, if you don't have a password yet, then
enter a new password but leave the confirmation box blank, then you'll
correctly see "Password confirmation can't be blank", but you'll *also*
then be prompted for your "Current password", even though you don't
have one yet, because `@auth_user.uses_password?` is true now.
In this change, we extend the Settings form to use two copies of the
`AuthUser`. One is the copy with changes on it, and the other is the
"persisted" copy, which we check for parts of the UI that care about
what's actually saved, vs form state.
Ah okay, if you leave the password field blank but don't have one set,
our simple `update` method gets annoyed that you left it blank.
In this change, we simplify the model API by just overriding
`update_with_password` with our own special behavior for the
no-password case.
Simplified this a bit into a helper. It's kinda odd to me, but
convenient for this moment, that Rails allows views to read `params`! I
guess it's for escape hatches exactly like this! lol
including validation logic to make sure it's not already connected to
another one!
The `intent` param on the NeoPass form is part of the key! Thanks
OmniAuth for making it easy to pass that data through!
Ahh I see, if you do a no-op update, it still clears the
`previously_new_record?` state, so our NeoPass controller thinks this
account already existed. Instead, let's only do this update if it's an
account that already exists, instead of depending on the no-op-iness!
I'm getting ready to add handling for "what if you don't *have* a
current password*??", so it seems like the right way to do that is to
just eject the controller and start customizing!
That is, you're required to add a password *or* an email before
disconnecting your NeoPass, but idk, I think it's rude to demand an
email from someone for the sake of *disconnection*. Email is no longer
required for accounts that already exist!
This is more consistent with the `uses_omniauth?` we already have, and
it also will help for the next change, where I want a `uses_password?`
method (and using the name `password?` breaks some of Devise's
validation code).
Ahh right, in development `User` and `AuthUser` will have the same ID,
but that got messed up early on for us in production DTI 😅
Here, we switch the form to reference the `User` instead of the
`AuthUser` (to get the ID right), then we also change how we compare
the IDs, because `User#to_param` appends extra text onto the ID after
the number!
Motivation is that I wanna add NeoPass stuff to here! But also like,
it's looked bad for a long time, let's clean it up!! (I just used the
Devise default without any styling at all lol)
Huh, I was writing up an API inventory doc to send to TNT, and was
gonna explain why we proxy these assets… but turns out we don't need to
anymore! Nice!
This is a bit fragile if they ever change their headers, so I'll
mention that in the doc, but for now, yeah sure let's save the planet
some computational effort!
Previously, the way we loaded the image hash for a given pet was to
navigate to `https://pets.neopets.com/cpn/<pet_name>/1/1.png`, but
*not* follow the redirect, and extract the image hash from the URL
where it redirected us to.
In this change, we refactor to use the AMFPHP RPC `PetService.getPet`
instead. I don't think it had this data last time I looked at it, but
now it does! Much prefer to use an actual RPC than our weird hacky
thing!
(We might also be able to use this call for other stuff, like
auto-labeling gender & mood for pet states, maybe?? That's in this data
too! We used to load petlookups for this, long long ago, before the
petlookup captchas got added.)
I guess this was like, we had some call site that was handling loading
the viewer data itself, and didn't want to have to reload it?
But whatever, not used now, let's simplify! We can rebuild this easily
if we need it again.
Locale is the big one that's not really relevant anymore (I don't want
to be loading non-English item names anymore, now that we've simplified
to only support English like TNT has!), but there was also `item_scope`
and stuff.
The timeout option is technically not used in any call sites, but I
think that one's useful to leave around; timeout stuff is important,
and I don't want to rewrite it sometime if we need it again!
Just a small thing, I guess when I was a kid I did a weird thing where
I attached `origin_pet` to `PetType`, then upon saving `PetType` I
loaded the image hash for the pet to save as the pet type's new image
hash.
I guess this does have the nice property of not bothering to load that
stuff until we need it? But whatever, I'm moving this into `Pet` both
to simplify the relationship between the models, and to prepare for
another potential refactor: using `PetService.getPet` for this instead!
Ahh, I had assumed the `uid` provided by NeoPass would be the user's
Neopets username, but in hindsight that was never gonna work out since
NeoPass doesn't think of things in terms of usernames at all!
For now, we create 100% random NeoPass usernames, of the form
"neopass-shoyru-5812" or similar. This will be an important fallback
anyway, because it's possible to have a NeoPass with *no* Neopets.com
account attached.
But hopefully we'll be able to work with TNT to request the user's main
Neopets account's username somehow, to use that as the default when
possible!
Ah beans, I didn't notice when doing my Turbo fixes in
40804c1543, that I had accidentally
prevented Chakra from applying some of its usual global styles! This
caused some minor visual regressions in various parts of the app, e.g.,
the default border color for the search field in the wardrobe app
became way darker.
Here, instead of copy-pasting the styles and missing some parts, we
scope the global styles a bit more carefully: we first use
`extendTheme` with the same code as Impress 2020 to get a good
`globalStyles` function that includes Chakra's defaults, *then* delete
the key from the theme.
Then, in `ScopedCSSReset`, we use code similar to Chakra's own global
style application code: call the `globalStyles` function with the
current theme and color mode, use Chakra's `css` function to convert
values like `green.800` to CSS values, prepend our scoping rule onto
the selectors, and drop it into our Emotion CSS.
Tbh I was worried because when I first noticed this issue while on my
trip, I saw the black item search box border, and was like "ah dang,
did I destroy all the color in the app by breaking the part where
Chakra defines its CSS color variables??" And no, that's not actually
what happened, a lot of the app still had its colors!
So this was less pressing than I had thought, but still good to get
fixed!
Ah right, I went and checked the Devise source code, and the default
implementation for `password_required?` is a bit trickier than I
expected:
```ruby
def password_required?
!persisted? || !password.nil? || !password_confirmation.nil?
end
```
Looks like `super` does a good enough job here, though! (I'm actually
kinda surprised, I wasn't sure how Ruby's `super` rules worked, and
this isn't a subclass thing—or maybe it is, maybe the `devise` method
adds a mixin? Idk! But it does what I expect, so, great!)
So now, we require the password if 1) Devise doesn't see a UI reason
not to, *and* 2) the user isn't using OmniAuth (i.e. NeoPass).
This had caused a bug where it was impossible to use the Settings page
*without* changing your password! (The form says it's okay to leave it
blank, which stopped being true! But now it's fixed!)
Whew, exciting! Still done nothing against the live NeoPass server, but
we've got this fully working with the development server, it seems!
Wowie!!
This is all still hidden behind secret flags, so it's fine to deploy
live. (And it's not actually a problem if someone gets past to the
endpoints behind it, because we haven't actually set up real
credentials for our NeoPass client yet, so authentication will fail!)
Okay time to lie down lol.
Okay, `sub` seems to be a pretty standard place for user identifiers.
Let's start with that assumption! I override the `oauth2-mock-server`'s
default of `johndoe` with `theneopetsteam`, just to be cute :3
In this change, we wire up a new NeoPass OAuth2 strategy for OmniAuth,
and hook up the "Log in with NeoPass" button to use it!
The authentication currently fails with `invalid_credentials`, and
shows the `owo` response we hardcoded into the NeoPass server's token
response. We need to finally follow up on the little `TODO` written in
there!
If you pass `?neopass=1` (or a secret value in production), you can see
the "Log in with NeoPass" button, which currently takes you to
OmniAuth's "developer" login page, where you can specify a name and
email and be redirected back. (All placeholder UI!)
We're gonna strip the whole developer strategy out pretty fast and
replace it with one that uses our NeoPass test server. This is just me
checking my understanding of the wiring!
This is setting us up for NeoPass, but first we're just gonna try stuff
with the "developer" strategy that's built in for testing, rather than
using the NeoPass dev server!
Oh right, all these cute overrides should be scoped to the page!
I guess we skipped this because we had pulled this out into a
separate stylesheet. Curiously learning more about how Turbo handles
this kind of thing, like that it doesn't *unload* stylesheets that
*leave* the page when you navigate!
I noticed an issue where Turbo-loading between the Your Items page and
the homepage would clobber each other's copy of jQuery, breaking things
sometimes. e.g. go to Your Items, then go to home, then go to Your
Items, and the page's JS fails because `$.fn.live` isn't defined.
I briefly tested the homepage and it didn't seem to actually depend on
any features from the later version of jQuery? At least not that I
noticed! So I'll just downgrade for consistency. (I also tried
upgrading the Your Items page, but there's too much usage of
`$.fn.live`, which is replaced with a notably different syntax in
jQuery 2.0+.)
First one, Turbo reasonably yelled at us in the JS console that we
should put its script tag in the `head` rather than the `body`, because
it re-executes scripts in the `body` and we don't want to spin up Turbo
multiple times!
I also removed some scripts that aren't relevant anymore, fixed a bug
in `outfits/new.js` where failing to load a donation pet would cause
the preview thing to not work when you type (I think this might've
already been an issue?), reworked `item_header.js` to just run once in
the `head`, and split scripts into `:javascripts` (run once in `head`)
vs `:javascripts_body` (run every page load in `body`).
Finally getting around to this because, with Turbo in play, it applies
to subsequent *pages* too, oops! Previously we at least had the blast
radius of this known issue constrained to the item page itself lol :p
Got some questions in Discord about account unlinking, and seeing
people look ahead to other potential integrations. Want to clarify that
unlinking will work here (barring any surprises!), and that there's no
data sharing _just_ yet!
Someone requested this in Discord, and I figured why not! I'm still
planning to move stuff away from Impress 2020 over time, I just figure
may as well have them more linked while this is still The Reality
This doesn't really matter, I just didn't realize the `.html` part was
optional, and I guess I omitted it here without realizing? But let's
add it for consistency.
The modeling code is slow! I think in production it's being cached, and
tbh I though I had development mode caching turned on over here, but
it's quite evidently _not_ doing it if so, so. Okay! Skip for now.
Oh right, we don't have Rails UJS going on anymore, which is what
handled the confirmation prompts for deleting lists. Turbo is the more
standard modern solution to that, and should speed up certain
pageloads, so let's do it!
Here I install the `turbo-rails` gem, then run `rails turbo:install` to
install the `@hotwired/turbo-rails` npm package. Then I move
`application.js` that's run all on pages but the outfit editor into our
section of JS that gets run through the bundler, and add Turbo to it.
I had to fix a couple tricky things:
1. The outfit editor page doesn't play nice with being swapped into the
document, so I make it require a full page reload instead.
2. Prefetching the Sign In link can cause the wrong `return_to` address
to be written to the `session`. (It's a GET request that does, ever
so slightly, take its own actions, oops!) As a simple hacky answer,
we disallow prefetching on that link.
Haven't fixed up the UJS stuff for confirm prompts to use Turbo yet,
that's next!
This `.gif` format is used in the items list "export to petpage"
feature, as the image URL for items whose URLs are known to contain
blocked words that prevent them from being used in petpages.
But when doing some Rails upgrade long ago, we didn't notice the new
security feature that blocks redirects to other sites without a special
flag being set. It was triggering 500 errors, oops.
Now, we set the flag!
I think this is the more canonical place for stuff like this these days!
It's nice to be able to just say the short name when calling `render`.
Here's the answer I looked up about it: https://stackoverflow.com/a/9892081/107415
My immediate motivation is that I'm looking at creating more About
pages, and thinking about where to put them; I think maybe we trash the
`StaticController`, move these partials out to here, and move terms
into a new `AboutController`?
When we moved more logic into the main app, we made some assumptions
about manifest art that were different than Impress 2020's, in hopes
that they would be More Correct for potential future edge cases.
Turns out, they were actually *less* correct for *current* edge cases!
Chips linked us to a few examples, including this Reddit post:
https://www.reddit.com/r/neopets/comments/1b8fd72/i_dont_think_thats_the_correct_image/
Fixed now!
According to our GlitchTip error tracker, every time we deploy, a
couple instances of `Async::Stop` and `Async::Container::Terminate`
come in, presumably because:
1. systemd sends a STOP signal to the `falcon host` process.
2. `falcon host` gives the in-progress requests some time to finish up
3. Sometimes some requests take too long, and so something happens.
(either a timer in Falcon or a KILL signal from systemd, not sure!)
that leads the ongoing requests to finally be terminated by raising
an `Async::Stop` or `Async::Container::Terminate`. (I'm not sure
when each happens, and maybe they happen at different points in the
process? Maybe one happens for the actual long-running ones, vs the
other happens if more requests come in during the meantime but get
caught in the spin-down process?)
4. Rails bubbles up the errors, our Sentry library notices them and
sends them to GlitchTip, the user presumably receives the generic
500 error, and the app can finally close down gracefully.
It's hard for me to validate that this is *exactly* what's happening
here or that my mitigation makes sense, but my logic here is basically,
if these exceptions are bubbling up as "uncaught exceptions" and
spamming up our error log, then the best solution would be to catch
them!
So in this change, we add an error handler for these two error classes,
which hopefully will 1) give users a better experience when this
happens, and 2) no longer send these errors to our logging 🤞❗️
That strange phenomenon where the best way to get a noisy bug out of
your logs is to fix it lmao
Oh rough, when moving an item list entry from one list to another, our
logic to merge their quantities if it's already in that list was just
fully crashing!
That is, moves without anything to merge were working, but moves that
required a merge were raising Internal Server Error 500, because the
`list_id` attribute wasn't present.
I'm not sure why this ever worked, I'm assuming using `list_id` in the
`where` condition would include it in the `select` implicitly in a
previous version of Rails? Or maybe Rails used to have fallback
behavior to run a second query, instead of raising
`MissingAttributeError` like it does now?
Well, in any case, this seems to fix it! Whew!
I don't think people see this very often visually, but it's showing up
in our error logging! The Rails API changed here long ago and we didn't
notice: to render public files, we should use the `file` argument
instead of `template`.
I previously added a warning to the item page, and thought about doing
one here but was sicky and misjudged the complexity and forgot you
don't need to hook into the `knownGlitches` API field to do it! Easy
peasy for a hacky little bug message!
I *think* what I'm observing is that:
1. The zone restrictions are different between these items.
2. The zone restrictions *change* when reloading the page sometimes. (I
assume from remodeling?)
3. The items look very buggy on many pets, because many appearances
seem to expect different zone restrictions than the item actually
has.
I think what this means is:
1. TNT has finally unbound restricted zones from the item level, and
allowed different appearances to have different restrictions. Neat!
2. The API still serves it the same way, as a field on the item.
So I think this means we need to update our schema to reflect the fact
that an item's `zones_restrict` field isn't *really* a property of the
item; it's a property of the combination of the item and the current
body ID.
My gut take here is that maybe this means it's time for the Large
Refactor that I've kinda been interested in for a while, but been
avoiding because of Impress 2020 compatibility issues: instead of a
`body_id` field on assets, and having them directly belong to items,
make an `ItemAppearance` record (closer to how 2020's GQL API modeled
it, I was looking ahead to this possibility!) that's keyed on item and
body ID, and assets belong to *that*.
Then, we could move the zones restriction field onto the
`ItemAppearance` record instead. And then it doesn't really matter to
us how TNT models it internally; whatever we saw is what we use.
(Again, I looked ahead to this in the 2020 app, and tried to use the
`restrictedZones` field on `ItemAppearance` when possible—even though
it secretly just reads directly from the `Item`!)
…but that's a pretty big departure from how things are modeled now, and
isn't something we can just throw together—especially coordinating it
across both apps. I was getting close to being able to shut off 2020
from a *front-facing* perspective (but still keeping a lot of the GQL
endpoints open for the wardrobe-2020 frontend), but I don't think we're
very close to being able to try to target turning off 2020's *backend*
as a prereq to this; or at least, if we do, we should expect that to
take a while. (Counting now, there's still 9 GQL queries—not as many as
I expected tbh, but still quite a few.)
So idk how to sequence this! But for now, let's put out a warning, and
start setting expectations.
The main *intended* user-facing effect of this is that "Items you own"
and "Items you want" filters should work in wardrobe-2020 now!
It is also possible that I messed something up and that this might
break unrelated searches somehow! We'll find out!! 😅
Did this before we had the ability to trigger searches from the app
itself, to allow me to open up the browser JS console and call this
function directly. Now we can just do it in app, goodbye!
Yay, we finally added it, the part where we include the appearance data
for the items based on both the species/color and the alt style! Now,
switching to Faerie Acara correctly filters the search only to items
that would fit (I think literally just only body_id=0 items right now,
but we're not banking on that!)
This only *really* shows up right now in the case where you construct
an Advanced Search form query (which only the wardrobe-2020 app does
now, and in limited form), and we return the query back (which only
gets used by the HTML view for item search, which doesn't have any way
to build one of these requests against it).
This is because, if you just type in `fits:alt-style-87305`, we always
keep your search string the same when outputting it back to you, to
avoid the weirdness of canonicalizing it and changing it up on you in
surprising ways!
But idk, this is just looking forward a bit, and keeping the system's
semantics in place. I hope someday we can bring robust text filter
and Advanced Search stuff back into the main app again, maybe!
I considered this at first, but decided to keep it simple until it
turned out to matter. Oops, it already matters, lol!
I want the item search code to be able to easily tell if the series
name is real or a placeholder, so we can decide whether to build the
filter text in `fits:$series-$color-$species` form or
`fits:alt-style-$id` form.
So in this change, we keep it that `AltStyle#series_name` returns the
placeholder string if none is set, but callers can explicitly ask
whether it's a real series name or not. Will use this in our next
change!
Previously we did this hackily by comparing the ID to a hardcoded list
of IDs, but I think putting this in the database is clearer and more
robust, and it should also help with our upcoming item search stuff
that will filter by it!
Previously, passing in `fits:blue` would cause a crash, because
`species_name` part of the split would be `nil`, oops!
In this change, we use a regex for more explicitness about the pattern
we're trying to match. We'll also add more cases next! (You'll note the
error message mentions `fits:nostalgic-faerie-draik`, which isn't
actually possible yet, but will be!)
Previously, the query wouldn't fill into the search box or page title
if e.g. parsing had failed. Now it does!
I'm not sure why the rescue strategy we previously had here doesn't
work anymore (I'm sure it must've in the past sometime?), but this is
simpler anyway, let's go!
I think this is a bit clearer and lets us clean up some of the syntax a
bit (don't need to always say `filters <<`), and also it will let us
use `return`, which I'm interested in for my next change!
Right, fitting isn't just body_id = this one, it's also body_id=0!
Anyway, doing this query on its own is still deathly slow, I wonder if
the idea I had about left joins (back when I was still working in a
Rails version that didn't support it lol) could help! Might poke at
that a smidge.
Oh right, `imageUrl` is the name of the field relative to what the app
expects, but under the hood `useOutfitAppearance` actually makes that
an alias for `imageUrlV2(idealSize: SIZE_600)`.
So we need to cache it as the same field with the same params, rather
than as just plain `imageUrl`!
This fixes the bug where wearing an item from search would require a
network round-trip and visually remove all items in the meantime.
(Also, none of this issue was visible to most users, because item
search is still feature-flagged onto the old GQL one for most people!)
This makes clicking on search results in the new mode actually work! It
correctly adds it to the outfit, and removes other items.
The thing that's behaving strangely is that, when you add the item, we
visually remove all items until we can finish a fresh network request
for what they should all look like. This probably means that the cache
lookup for `useOutfitAppearance` is not as satisfied with what we cache
here as `findItemConflicts` is? Something to investigate!
It'd be nice to customize the message a bit, but this should be rare
and I'd prefer the simplicity of just going with the default text.
I ran into this when I made a mistake in how I process the return value
of search results, so React Query caught and raised the error via
React, as intended! And I was annoyed that it wasn't logged anywhere,
so that's my motivation for this change—but also, the old message is
pretty meh and has some layout problems anyway.
I feel like this was part of `will_paginate` back before the Rails
community had itself figured out about what belongs in a model?
But yeah, a default per-page value for search results does not belong
here. And I don't think anything references it anymore, because we pass
`per_page` to the `paginate` call in `ItemsController` explicitly! So,
goodbye!
First off, I think our code has converged on a convention of gracefully
returning `nil` for manifest-less situations, so we can do that instead
of raise! And then that lets us just simplify this check to whether
`manifest` is present, instead of `manifest_url`, so we stop crashing
in cases where we get to this point in the code and there's a manifest
URL but not a manifest.
This was a bit tricky! When I initially turned it on, running
`rails swf_assets:manifests:load` would trigger database errors of "oh
no we can't get a connection from the pool!", because too many records
were trying to concurrently save at once.
So now, we give ourselves the ability to say `save_changes: false`, and
then save them all in one batch after! That way, we're still saving by
default in the edge cases where we're downloading and saving a manifest
on the fly, but batching them in cases where we're likely to be dealing
with a lot of them!
Now we're *really* duplicating with Impress 2020's system lol, but I
need a way to not keep trying to load manifests that are actually 404,
which are surprisingly plentiful!
This doesn't actually stop us from loading anything yet, it just tracks
the timestamps and the HTTP status! But next I'll add logic to skip
when it was 4xx recently.
This is both unnecessary now, but also caused a bug in the new search
stuff where searching by zone would pass an extra `locale` argument to
a filter that doesn't need it!
Idk when this regressed exactly, but probably people didn't super
notice because I don't think it's a very common thing to type directly
into the Infinite Closet search box! (It used to be crucial to the old
wardrobe app.)
But I'm using it in the wardrobe app again now, so, fixed!
For now, I'm doing it with a secret feature flag, since I want to be
committing but it isn't all quite working yet!
Search works right, and the appearance data is getting returned, but I
don't have the Apollo Cache integrations yet, which we rely on more
than I remembered!
Also, alt styles will crash it for now!
`is:np` now means "is not NC and is not PB".
Note that it might be good to make NC and PB explicitly mutually
exclusive too? It would complicate queries though, and not matter in
most cases… the Burlap Usul Bow is the only item that we currently
return for `is:pb is:nc`, which is probably because of a rarity issue?
Adding new functionality to the item search JSON endpoint, and adding
an adapter layer to match the GQL format!
Hopefully this will be pretty drop-in-able, we'll see!
Clearing the way to be able to delete the announcement banner, which is
currently the only link!
I feel like there's room to redo the site layout to find a place to
more properly link to this from, but I don't have one yet! And this is
enough of a niche reference that I think this is good enough?
A bit of a hack, because the thing triggering it was also a bit of a
hack? I feel like there's something we gotta do with refactoring how
our multiple concepts of state are managed… but in any case! This seems
to keep basic outfit-loading working, while no longer getting us
trapped in autosave loops!
Here's how I reproduced the bug:
1. Open a saved outfit.
2. Set the browser devtools to apply a latency of 5sec to all requests.
3. Add an item to the outfit, and wait for the autosave to start.
4. While it still says "Saving", remove the item again.
5. Watch how, when the first autosave request comes in, the item is
re-applied to the outfit, then autosave gets stuck looping forever.
The issue was that, when an outfit finishes saving, the change in
outfit data was triggering this effect in `useOutfitState` that was
*meant* to *initialize* local state from the saved outfit, not to keep
them in sync all the time. (In general, when saved outfit data comes
back from the server, we don't want to use it to "fix" local outfit
state in the case of discrepancies, because the most common source of
discrepancy will be the user having made further changes!)
But anyway, one thing I didn't realize is that we *were* depending on
this hacky hook to do more than I thought: it was responsible for
syncing `id` and `appearanceId` to the local state after saving the
outfit. So, I replaced the `rename` action dispatch here with a new
action that explicitly sets all fields the server is responsible for!
Ah yeah, if you're not on the wardrobe page (so we don't need the Alt
Styles UI), and the outfit's `altStyleId` is null (as is the case for
the item preview page), then there's no need to load the alt styles for
that species.
So before this change, going to `/items/123` would include an XHR
request to `/species/<id>/alt-styles.json`, which would not be used for
anything. After this change, that request is no longer sent. Hooray!
The alt styles controller is the one place we use this right now, but
I'm planning to generalize this to loading appearances during item
search, too!
I also add more `only` fields to the alt styles `as_json` call, because
idk it feels like good practice to both 1) say what we need in this
endpoint, rather than rely on default behavior upstream, and 2) to
avoid leaking fields we didn't realize were on there. (And also to
preserve bandwidth, too!)
I think there's no call sites for these anymore, so now I can start
repurposing these methods for the new API endpoints I'm planning! :3
Now, `SwfAsset#image_url` approximately matches Impress 2020 logic: use
the thumbnail PNG from the manifest if one exists, or the Impress 2020
converter for canvas movies, or the old AWS copy generated by gnash if
necessary, or return nil.
I built this API endpoint in anticipation of a change I never actually
made! I'll just remove it for now, leaning toward cleanuppery over
holding onto something I'm not sure about.
I think this used to be used in an API endpoint we've now deleted? I'm
just cleaning up call sites because I intend to refactor the `urls`
method and stuff, so I'm removing cruft that would complicate it!
I'm not certain-certain this is unused, but I did a global search for
`\bimages\b` in the codebase, and didn't find anything that looked like
a match to me!
Doing that sweet, sweet backfill!! It's not exactly *fast*, since
there's about 570k records to work through, but it's pretty good all
things considered! Thanks, surprisingly-reusable async code!
I'm gonna also use this for a task to try to warm up *all* the
manifests in the database! But to start, just a simple one, to prepare
the alt styles page quickly on first run. (This doesn't really matter
in production now that I've already visited the page once, but it helps
when resetting things in dev, and I think more it's about establishing
the pattern!)
The Neopets Media Archive is a service that mirrors `images.neopets.com`
over time! Right now we're starting by just loading manifests, and
using them to replace the hacks we used for determining the Alt Style
PNG and SVG URLs; but with time, I want to load *all* customization
media files, to have our own secondary file source that isn't dependent
on Neopets to always be up.
Impress 2020 already caches manifest files, but this strategy is
different in two ways:
1. We're using the filesystem rather than a database column. (That is,
manifest data is kinda duplicated in the system right now!) This is
because I intend to go in a more file-y way long-term anyway, to
load more than just the manifests.
2. Impress 2020 guesses at the manifest URLs by pattern, and reloads
them on a regular basis. Instead, we use the modeling system: when
TNT changes the URL of a manifest by appending a new `?v=` query
string to it, this system will consider it a new URL, and will load
the new copy accordingly.
Fun fact, I actually have been prototyping some of this stuff in a side
project I'd named `impress-media-server`! It's a little Sinatra app
that indeed *does* save all the files needed for customization, and can
generate lightweight lil preview iframes and images pretty easily. I
had initially been planning this as a separate service, but after
thinking over the arch a bit, I think it'll go smoother to just give
the main app all the same access and awareness—and I wrote it all in
Ruby and plain HTML/JS/CSS, so it should be pretty easy to port over
bit-by-bit!
Anyway, only Alt Styles use this for now, but my motivation is to be
able to use more-correct asset URL logic to be able to finally swap
over wardrobe-2020's item search to impress.openneo.net's item search
API endpoint—which will get "Items You Own" searches working again, and
whittle down one of the last big things Impress 2020 can do that the
main app can't. Let's see how it goes!
Preparing to finally move wardrobe-2020's item search to use the main
app's API endpoints instead!
One blocker I forgot about here: Impress 2020 has actual support for
knowing an item's true appearance, like by reading the manifest and
stuff, that we haven't really ported over. I feel like maybe I should
pause and work on the changes to manifest-archiving that I'd been
planning anyway? I'll think about it.
I changed the type of this tag without realizing the JS references it
by both class and `div`!
I think at the time this was a perf suggestion for jQuery, because the
best way to query by class name was to query by tag first then filter?
It's possible our jQuery still does this, but I don't imagine it's very
relevant today, so I'll just remove that for better guarding against
similar bugs in the future instead.
I've moved the support secret into the encrypted credentials file, and
moved the origin into a top-level custom config value in the
environment files, with different defaults per environment but still
the ability to override it. (I don't use this, but it feels polite to
not actually *demand* that people use port 4000, y'know?)
Okay, so I still don't know why rendering is just so slow (though
migrating away from item translations did help!), but I can at least
cache entire closet lists as a basic measure.
That way, the first user to see the latest version of a closet list
will still need just as much time to load it… but *only* the ones that
have changed since last time (rather than always the full page), and
then subsequent users get to reuse it too!
Should help a lot for high-traffic lists, which incidentally are likely
to be the big ones belonging to highly active traders!
One big change we needed to make was to extract the `user-owns` and
`user-wants` classes (which we use for trade matches for *the user
viewing the list right now*) out of the cached HTML, and apply them
after with Javascript instead. I always dislike moving stuff to JS, but
the wins here seem. truly very very good, all things considered!
From an era when we didn't have that! Now we do!
(My motivation is that I'm trying to add new JS to this page and errors
in stickUp are crashing the page early, womp womp!)
This one is important, I didn't notice that this is a way of setting
attributes that won't be written to both tables! `name` will only be
written to the translation table (which crashes the save), and the
other fields would only be written to the main table. Fixed! (I don't
like the super-dynamic this code was written before, anyway.)
Missed this at first - now that the `name` field is just a normal field
and is always English, it's now an error to provide the locale to it as
a parameter, like we used to for the translated version of the field!
Like with Species, Color, and Zone, we're moving the translation data
directly onto the model, and just using English. This will simplify
some of our queries a lot (way fewer joins!), and it's what Neopets
does now anyway, and I have a secret hope that removing the complexity
along the codepath for `item.name` might help speed up large item lists
if we're lucky?? 🤞
Anyway, this is the first step, performing the migration to copy the
data onto the `items` table, making sure to keep them in sync for the
2020 app for now!
I think this was to explain why `order` wasn't part of this query, and
we probably used to sort in the controller? But now the item search
module takes care of all that, this is just confusing to say now imo!
Impress 2020 has had this for a while, I've wanted it for reference on
occasion, let's bring it in!
Very similar logic, and Ruby & Rails's date affordances are super
helpful for simplifying how to express it!
The homepage used to point to old projects that don't work anymore
anyway! This is the only project that stuck, so just redirect here!
We also remove the openneo.net link from the footer, because there's
nothing useful to say there anymore!
It hasn't been updated in a long time, let's just be rid of it!
It's possible I'll replace it with another blog sometime if we get the
chance to do more development work, it could be a useful way to improve
communication—but not yet!
I think I cleared this from the outfits/new template a while ago, but
never cleaned up this file, because I was too anxious that I was
correctly identifying all its call sites. But now I'm more confident!
At least, they seem unused to me on a quick audit! The scriptaculous
stuff has long been replaced by jQuery UI equivalents. (Wow, so many
generations of libraries! lol)
Mostly this is just me testing out what it would look like to
modularize the app more… I've noticed that some concerns, like
fundraising, are just not relevant to most of the app, and being able
to lock them away inside subfolders feels like it'll help tidy up
long folder lists.
Notably, I haven't touched the models case yet, because I worry that
might be a bit more complex, whereas everything else seems pretty
well-isolated? We'll try it out!
Tbh I'm not sure `special_color` is actually used anywhere? It used to
be how we decide what to show in the previewer on the item page, but
that's been replaced with the 2020 logic, so idk…
But in any case, I noticed that the description doesn't match the
pattern we have, so here's the fix!
I looked at this and was like. "ok literally what is
`nonstandard_colors` trying to do"
reading it again now, I'm realizing the idea is that it probably runs
two queries: one to get nonstandard colors, then depends on
ActiveRecord to implicitly convert the relation to an array and then to
IDs for the second query? Instead of doing a join??
Idk, it's unused, so trash it!
This used to be the behavior, and the site has plenty of graceful
fallbacks for it, I just forgot this one when doing Rails upgrades!
Note that the impress-2020 stuff is *not* as graceful about this, so
the wardrobe page won't show the pet until the color is in the DB. Ah
well, still an improvement!
I thought this refactor of this change was working, but actually it was
just failing to build the JS lmao. Here's a version with correct syntax!
😅
Is there a syntax for this kind of thing that I'm just forgetting? Idk,
oh well!
Okay right, the wardrobe-2020 app treats `state` as a bit of an
override thing, and `pose` is the main canonical field for how a pet
looks. We were missing a few pieces here:
1. After loading a pet, we weren't including the `pose` field in the
initial query string for the wardrobe URL, but we _were_ including
the `state` field, so the outfit would get set up with a conflicting
pet state ID vs pose.
2. When saving an outfit, we weren't taking the `state` field into
account at all. This could cause the saved outfit to not quite match
how it actually looked in-app, because the default pet state for
that species/color/pose trio could be different; and regardless, the
outfit state would come back with `appearanceId` set to `null`,
which wouldn't match the local outfit state, which would trigger an
infinite loop.
Here, we complete the round-trip of the `state` field, from pet loading
to outfit saving to the outfit data that comes back after saving!
Now that DTI 2020 has been deployed without references to the
translations tables, we can stop keeping them in sync!
Next step is to drop the tables and be done with them altogether! (I
have a backup of the public data for this too, as does this repo!)
This happens on the Baby Kougra, where for most poses half of the
assets have a manifest that includes an SVG but no PNG. Skip 'em!
I considered adding a glitch tag for this, but idk I think we can do
that once we're aware of an actual case where this causes visible
issues.
On the small-screen layout, the popover goes down and covers the item
list, which isn't a big deal in context; so I want it to be basically
as tall as it can be without being unwieldy, to give more info.
But on the large-screen layout, it doesn't take long at all for the
popover to start intersecting the pet preview, because it *has* to go
up and cover the pet preview. So, I'm much more reserved about how much
vertical space I'm willing to give it!
(I also considered sending the popover off to the *right* of the button,
to cover the item list, but it felt *way* too weird imo! Especially in
the expressions case.)
The intent of this glitch message was that, when UC or Invisible pets
hide an item because of a zones-restrict thing, it would still show up
in the items panel as fitting a certain zone, whereas it should have
been in the "Incompatible" section and having none of its zones applied.
But the previously implementation would like, show this message even for
items that _were_ correctly marked as Incompatible? And that the server
returned no layers for, because it doesn't fit this body type to begin
with? (e.g. put a Grarrl hat on a Grarrl, then switch to Acara, and the
Grarrl hat is marked Incompatible—but would also show this confusing
message; or similarly with switching to Alt Styles)
So, when the server just returned no layers for this item to begin with,
don't show this message!
Now that we're tracking tab state ourselves, it's pretty easy to just
pass the `initialFocusRef` to the right place instead of to both!
This helps switching between the tabs feel a lot smoother, because we
don't have to re-render and fade-in all the poses again.
I'm keeping it in the same place for now rather than trying to fold it
into Styles, because I think that's net-less-confusing (since Styles
work pretty differently, e.g. different color requirements), and
certainly less work either way lol!
