1
0
Fork 0
forked from OpenNeo/impress
Commit graph

566 commits

Author SHA1 Message Date
18c7a34b8f Update series_name for alt styles to be null, with a fallback string
I considered this at first, but decided to keep it simple until it
turned out to matter. Oops, it already matters, lol!

I want the item search code to be able to easily tell if the series
name is real or a placeholder, so we can decide whether to build the
filter text in `fits:$series-$color-$species` form or
`fits:alt-style-$id` form.

So in this change, we keep it that `AltStyle#series_name` returns the
placeholder string if none is set, but callers can explicitly ask
whether it's a real series name or not. Will use this in our next
change!
2024-02-27 15:48:28 -08:00
9243193bc8 Add support for fits:alt-style-87305 item searches
Easy peasy!
2024-02-27 15:36:45 -08:00
d983a20989 Add support for fits:nostalgic-faerie-draik item searches
Nice and easy now that we have `series_name` in the database!

I didn't add the `fits:alt-style-12345` case yet though!
2024-02-27 15:33:08 -08:00
9f74e6020e Add series_name database field to alt styles
Previously we did this hackily by comparing the ID to a hardcoded list
of IDs, but I think putting this in the database is clearer and more
robust, and it should also help with our upcoming item search stuff
that will filter by it!
2024-02-27 15:28:05 -08:00
cd786ffcb1 Oops, I broke user filters in my previous refactor!
Right, `user` is an important argument that I missed! 😅
2024-02-27 15:08:15 -08:00
76d741091c Extract "raise_search_error" method in item search query parsing
Just to make all this a bit more wieldy, and not repeat the same prefix
all the time!
2024-02-27 15:03:18 -08:00
61b1a1aed1 Improve parsing fits:blue-acara filter
Previously, passing in `fits:blue` would cause a crash, because
`species_name` part of the split would be `nil`, oops!

In this change, we use a regex for more explicitness about the pattern
we're trying to match. We'll also add more cases next! (You'll note the
error message mentions `fits:nostalgic-faerie-draik`, which isn't
actually possible yet, but will be!)
2024-02-27 14:56:36 -08:00
19ebf4d78a Extract item search filter parsing into helper methods
I think this is a bit clearer and lets us clean up some of the syntax a
bit (don't need to always say `filters <<`), and also it will let us
use `return`, which I'm interested in for my next change!
2024-02-27 14:43:42 -08:00
3781c9810a Item search can filter by fitting alt styles (but missing many details!)
A query like this works for finding items containing "hat" that fit
alt style #87296 (the Faerie Acara):

http://localhost:3000/items?q%5B0%5D%5Bkey%5D=name&q%5B0%5D%5Bvalue%5D=hat&q%5B1%5D%5Bkey%5D=fits&q%5B1%5D%5Bvalue%5D%5Bspecies_id%5D=1&q%5B1%5D%5Bvalue%5D%5Bcolor_id%5D=8&q%5B1%5D%5Bvalue%5D%5Balt_style_id%5D=87296

But there's two main missing pieces still:
1. You can't do a text-filter version of this—in fact, clicking Search
   immediately on the page this loads will return an error!
2. We haven't extended this to the `with_appearances_for` parameter,
   so we add a `NotImplementedError` to that bit for now too.

I'm also thinking that the text filter should ideally be like,
`fits:nostalgic-faerie-acara` if we can pull it off; and then fall back
to the plainer `fits:alt-style-87296` if e.g. we don't know the set
it's from yet.
2024-02-27 14:32:54 -08:00
183cb40e74 Oops, don't return body_id=0 items for "-fits:blue-acara"
Right, fitting isn't just body_id = this one, it's also body_id=0!

Anyway, doing this query on its own is still deathly slow, I wonder if
the idea I had about left joins (back when I was still working in a
Rails version that didn't support it lol) could help! Might poke at
that a smidge.
2024-02-27 14:07:20 -08:00
671a79d158 Refactor fits and not_fits in Item::Search::Query
Just restructuring a bit in anticipation of changes we're gonna make
for alt style support in here!
2024-02-27 14:05:37 -08:00
2e5b1c7350 Remove unused Item.per_page attribute
I feel like this was part of `will_paginate` back before the Rails
community had itself figured out about what belongs in a model?

But yeah, a default per-page value for search results does not belong
here. And I don't think anything references it anymore, because we pass
`per_page` to the `paginate` call in `ItemsController` explicitly! So,
goodbye!
2024-02-25 16:16:43 -08:00
fb2bdd6ea5 Fix crash when dealing with 404'd manifests
First off, I think our code has converged on a convention of gracefully
returning `nil` for manifest-less situations, so we can do that instead
of raise! And then that lets us just simplify this check to whether
`manifest` is present, instead of `manifest_url`, so we stop crashing
in cases where we get to this point in the code and there's a manifest
URL but not a manifest.
2024-02-25 16:05:43 -08:00
2cac048158 Save manifest load info when preloading them, too
This was a bit tricky! When I initially turned it on, running
`rails swf_assets:manifests:load` would trigger database errors of "oh
no we can't get a connection from the pool!", because too many records
were trying to concurrently save at once.

So now, we give ourselves the ability to say `save_changes: false`, and
then save them all in one batch after! That way, we're still saving by
default in the edge cases where we're downloading and saving a manifest
on the fly, but batching them in cases where we're likely to be dealing
with a lot of them!
2024-02-25 16:02:36 -08:00
cc33ce1d6e Skip loading manifest if we recently failed
This helps speed up some item search result pages a lot in the new API
endpoint I'm building!
2024-02-25 15:46:50 -08:00
a684c915a9 Track when manifest was last loaded, and what status it returned
Now we're *really* duplicating with Impress 2020's system lol, but I
need a way to not keep trying to load manifests that are actually 404,
which are surprisingly plentiful!

This doesn't actually stop us from loading anything yet, it just tracks
the timestamps and the HTTP status! But next I'll add logic to skip
when it was 4xx recently.
2024-02-25 15:35:04 -08:00
067cee2d41 Oops, fix bug reading manifest assets when .svg is not present
Ah right, I make this mistake a lot when doing `group_by` stuff: if
there's no SVG, then `assets_by_ext[:svg]` is `nil`, not `[]`. Oh well!
2024-02-25 15:02:35 -08:00
e5bf6d6ba1 Oops, fix remaining references to locale in item search
This is both unnecessary now, but also caused a bug in the new search
stuff where searching by zone would pass an extra `locale` argument to
a filter that doesn't need it!
2024-02-25 15:00:22 -08:00
61a4dcad02 Oops, "fits:blue-acara" search should return non-body-specific items
Idk when this regressed exactly, but probably people didn't super
notice because I don't think it's a very common thing to type directly
into the Infinite Closet search box! (It used to be crucial to the old
wardrobe app.)

But I'm using it in the wardrobe app again now, so, fixed!
2024-02-25 14:47:37 -08:00
a8cbce0864 Start working on new item search in wardrobe-2020!
For now, I'm doing it with a secret feature flag, since I want to be
committing but it isn't all quite working yet!

Search works right, and the appearance data is getting returned, but I
don't have the Apollo Cache integrations yet, which we rely on more
than I remembered!

Also, alt styles will crash it for now!
2024-02-25 14:46:27 -08:00
52e81557c2 Update search filters to consider NP and PB mutually exclusive
`is:np` now means "is not NC and is not PB".

Note that it might be good to make NC and PB explicitly mutually
exclusive too? It would complicate queries though, and not matter in
most cases… the Burlap Usul Bow is the only item that we currently
return for `is:pb is:nc`, which is probably because of a rarity issue?
2024-02-25 12:57:04 -08:00
a3dcaa0f0e Add useItemSearch for wardrobe app, but don't use it yet!
Adding new functionality to the item search JSON endpoint, and adding
an adapter layer to match the GQL format!

Hopefully this will be pretty drop-in-able, we'll see!
2024-02-25 12:06:32 -08:00
c5cd1f2f3d Remove SwfAsset#html5_*_url in favor of SwfAsset#urls
The alt styles controller is the one place we use this right now, but
I'm planning to generalize this to loading appearances during item
search, too!

I also add more `only` fields to the alt styles `as_json` call, because
idk it feels like good practice to both 1) say what we need in this
endpoint, rather than rely on default behavior upstream, and 2) to
avoid leaking fields we didn't realize were on there. (And also to
preserve bandwidth, too!)
2024-02-24 16:29:47 -08:00
cd095eefcb Oops, fix crashing bug when loading item appearances endpoint
I missed this because it's only for backgrounds and stuff! But yeah I
added a prefix to the `Body` struct, and forgot to apply it everywhere,
oops!
2024-02-24 16:14:30 -08:00
e0b5382c3f Update SwfAsset#image_url to use all the latest tech
I think there's no call sites for these anymore, so now I can start
repurposing these methods for the new API endpoints I'm planning! :3

Now, `SwfAsset#image_url` approximately matches Impress 2020 logic: use
the thumbnail PNG from the manifest if one exists, or the Impress 2020
converter for canvas movies, or the old AWS copy generated by gnash if
necessary, or return nil.
2024-02-24 16:12:02 -08:00
829d960d70 Move SwfAsset#override_body_id to the bottom of the file
This is a console-y method, it's not important enough to be up here!
2024-02-24 15:24:24 -08:00
cf1cdf7e7d Reorder the higher-up lines of SwfAsset
A lot of stuff was mixed together, including methods interrupting the
scopes and relationships, oops! Fixed!
2024-02-24 15:22:14 -08:00
e8737a520e Remove more unused SwfAsset scopes
Wowzers, a lot of cruft in here! I hope I'm not deleting anything
important, but I think my Ctrl-F audits are checking out?
2024-02-24 15:19:28 -08:00
7fee4e6e41 Remove unused SwfAsset.fitting_standard_body_ids and support methods 2024-02-24 15:16:25 -08:00
c25e015b70 Delete some more unused SwfAsset methods 2024-02-24 15:15:21 -08:00
f8cf8165df Remove unused SwfAsset#images method
I think this used to be used in an API endpoint we've now deleted? I'm
just cleaning up call sites because I intend to refactor the `urls`
method and stuff, so I'm removing cruft that would complicate it!

I'm not certain-certain this is unused, but I did a global search for
`\bimages\b` in the codebase, and didn't find anything that looked like
a match to me!
2024-02-24 15:15:21 -08:00
b7f4c5b6ac Remove asset_hosts initializer
It just contains one constant, used in one place. Inlined and deleted!
2024-02-24 15:05:37 -08:00
992954ce89 Create swf_assets:manifests:load task to save all manifest files
Doing that sweet, sweet backfill!! It's not exactly *fast*, since
there's about 570k records to work through, but it's pretty good all
things considered! Thanks, surprisingly-reusable async code!
2024-02-23 14:06:49 -08:00
9a3b33ea2f Preload many manifests concurrently for the Alt Styles page
I'm gonna also use this for a task to try to warm up *all* the
manifests in the database! But to start, just a simple one, to prepare
the alt styles page quickly on first run. (This doesn't really matter
in production now that I've already visited the page once, but it helps
when resetting things in dev, and I think more it's about establishing
the pattern!)
2024-02-23 13:45:12 -08:00
2cc46703b9 Create NeopetsMediaArchive, read the actual manifests for Alt Styles
The Neopets Media Archive is a service that mirrors `images.neopets.com`
over time! Right now we're starting by just loading manifests, and
using them to replace the hacks we used for determining the Alt Style
PNG and SVG URLs; but with time, I want to load *all* customization
media files, to have our own secondary file source that isn't dependent
on Neopets to always be up.

Impress 2020 already caches manifest files, but this strategy is
different in two ways:

1. We're using the filesystem rather than a database column. (That is,
   manifest data is kinda duplicated in the system right now!) This is
   because I intend to go in a more file-y way long-term anyway, to
   load more than just the manifests.
2. Impress 2020 guesses at the manifest URLs by pattern, and reloads
   them on a regular basis. Instead, we use the modeling system: when
   TNT changes the URL of a manifest by appending a new `?v=` query
   string to it, this system will consider it a new URL, and will load
   the new copy accordingly.

Fun fact, I actually have been prototyping some of this stuff in a side
project I'd named `impress-media-server`! It's a little Sinatra app
that indeed *does* save all the files needed for customization, and can
generate lightweight lil preview iframes and images pretty easily. I
had initially been planning this as a separate service, but after
thinking over the arch a bit, I think it'll go smoother to just give
the main app all the same access and awareness—and I wrote it all in
Ruby and plain HTML/JS/CSS, so it should be pretty easy to port over
bit-by-bit!

Anyway, only Alt Styles use this for now, but my motivation is to be
able to use more-correct asset URL logic to be able to finally swap
over wardrobe-2020's item search to impress.openneo.net's item search
API endpoint—which will get "Items You Own" searches working again, and
whittle down one of the last big things Impress 2020 can do that the
main app can't. Let's see how it goes!
2024-02-23 12:02:39 -08:00
3f449310d6 Refactor item search JSON, add appearances
Preparing to finally move wardrobe-2020's item search to use the main
app's API endpoints instead!

One blocker I forgot about here: Impress 2020 has actual support for
knowing an item's true appearance, like by reading the manifest and
stuff, that we haven't really ported over. I feel like maybe I should
pause and work on the changes to manifest-archiving that I'd been
planning anyway? I'll think about it.
2024-02-23 10:44:50 -08:00
666394de25 Refactor Impress 2020 config
I've moved the support secret into the encrypted credentials file, and
moved the origin into a top-level custom config value in the
environment files, with different defaults per environment but still
the ability to override it. (I don't use this, but it feels polite to
not actually *demand* that people use port 4000, y'know?)
2024-02-22 13:07:43 -08:00
583f3c712f High-level caching for closet lists
Okay, so I still don't know why rendering is just so slow (though
migrating away from item translations did help!), but I can at least
cache entire closet lists as a basic measure.

That way, the first user to see the latest version of a closet list
will still need just as much time to load it… but *only* the ones that
have changed since last time (rather than always the full page), and
then subsequent users get to reuse it too!

Should help a lot for high-traffic lists, which incidentally are likely
to be the big ones belonging to highly active traders!

One big change we needed to make was to extract the `user-owns` and
`user-wants` classes (which we use for trade matches for *the user
viewing the list right now*) out of the cached HTML, and apply them
after with Javascript instead. I always dislike moving stuff to JS, but
the wins here seem. truly very very good, all things considered!
2024-02-20 18:43:39 -08:00
1e6ff4cefc Remove the Item::Translation model entirely
Okay, Impress 2020 is migrated off translations too, so we can start to
wrap this up!
2024-02-20 16:57:45 -08:00
3ac9e7ce69 Migrate item search away from item translations
Lightning fast for simple name queries now, gotta say!!
2024-02-20 16:04:41 -08:00
5ee3b472ec Migrate away from item translations in modeling
This one is important, I didn't notice that this is a way of setting
attributes that won't be written to both tables! `name` will only be
written to the translation table (which crashes the save), and the
other fields would only be written to the main table. Fixed! (I don't
like the super-dynamic this code was written before, anyway.)
2024-02-20 15:52:03 -08:00
0e8f457aa1 Oops, fix bug on item page now that translations aren't available
Missed this at first - now that the `name` field is just a normal field
and is always English, it's now an error to provide the locale to it as
a parameter, like we used to for the translated version of the field!
2024-02-20 15:37:07 -08:00
c75d988497 Migrate away from item translations in the Your Items feature
Just replacing references to the `Item::Translation` model to the
fields on `Item` itself!
2024-02-20 15:36:20 -08:00
a1066d9c8a Add translated item fields directly to the Item model
Like with Species, Color, and Zone, we're moving the translation data
directly onto the model, and just using English. This will simplify
some of our queries a lot (way fewer joins!), and it's what Neopets
does now anyway, and I have a secret hope that removing the complexity
along the codepath for `item.name` might help speed up large item lists
if we're lucky?? 🤞

Anyway, this is the first step, performing the migration to copy the
data onto the `items` table, making sure to keep them in sync for the
2020 app for now!
2024-02-20 15:25:03 -08:00
d39e7cea81 Move fundraising models into the Fundraising module
This was mostly straightforward it seems, whew!
2024-02-18 20:29:31 -08:00
1b22258576 Update special_color logic for Banana Chia Wings case
Tbh I'm not sure `special_color` is actually used anywhere? It used to
be how we decide what to show in the previewer on the item page, but
that's been replaced with the 2020 logic, so idk…

But in any case, I noticed that the description doesn't match the
pattern we have, so here's the fix!
2024-02-17 12:50:35 -08:00
6515e525fb Remove some unused PetType scopes
I looked at this and was like. "ok literally what is
`nonstandard_colors` trying to do"

reading it again now, I'm realizing the idea is that it probably runs
two queries: one to get nonstandard colors, then depends on
ActiveRecord to implicitly convert the relation to an array and then to
IDs for the second query? Instead of doing a join??

Idk, it's unused, so trash it!
2024-02-16 23:32:22 -08:00
15a905b0dd Allow modeling pets for colors we haven't gotten metadata for yet
This used to be the behavior, and the site has plenty of graceful
fallbacks for it, I just forgot this one when doing Rails upgrades!

Note that the impress-2020 stuff is *not* as graceful about this, so
the wardrobe page won't show the pet until the color is in the DB. Ah
well, still an improvement!
2024-02-16 23:28:10 -08:00
118ec6aa1a Oops, update outfit's updated_at when adding/removing items
This bug caused outfit thumbnails to get out of date, because the
outfit's `updated_at` field is used as a cache-busting part of the URL.
2024-02-08 10:32:53 -08:00
5d0848bf28 In development, use local Impress 2020 to generate outfit images
Just so we can see accurate stuff on the outfits page!
2024-02-08 10:03:10 -08:00