1
0
Fork 0
forked from OpenNeo/impress
Commit graph

647 commits

Author SHA1 Message Date
a8cbce0864 Start working on new item search in wardrobe-2020!
For now, I'm doing it with a secret feature flag, since I want to be
committing but it isn't all quite working yet!

Search works right, and the appearance data is getting returned, but I
don't have the Apollo Cache integrations yet, which we rely on more
than I remembered!

Also, alt styles will crash it for now!
2024-02-25 14:46:27 -08:00
52e81557c2 Update search filters to consider NP and PB mutually exclusive
`is:np` now means "is not NC and is not PB".

Note that it might be good to make NC and PB explicitly mutually
exclusive too? It would complicate queries though, and not matter in
most cases… the Burlap Usul Bow is the only item that we currently
return for `is:pb is:nc`, which is probably because of a rarity issue?
2024-02-25 12:57:04 -08:00
a3dcaa0f0e Add useItemSearch for wardrobe app, but don't use it yet!
Adding new functionality to the item search JSON endpoint, and adding
an adapter layer to match the GQL format!

Hopefully this will be pretty drop-in-able, we'll see!
2024-02-25 12:06:32 -08:00
c5cd1f2f3d Remove SwfAsset#html5_*_url in favor of SwfAsset#urls
The alt styles controller is the one place we use this right now, but
I'm planning to generalize this to loading appearances during item
search, too!

I also add more `only` fields to the alt styles `as_json` call, because
idk it feels like good practice to both 1) say what we need in this
endpoint, rather than rely on default behavior upstream, and 2) to
avoid leaking fields we didn't realize were on there. (And also to
preserve bandwidth, too!)
2024-02-24 16:29:47 -08:00
cd095eefcb Oops, fix crashing bug when loading item appearances endpoint
I missed this because it's only for backgrounds and stuff! But yeah I
added a prefix to the `Body` struct, and forgot to apply it everywhere,
oops!
2024-02-24 16:14:30 -08:00
e0b5382c3f Update SwfAsset#image_url to use all the latest tech
I think there's no call sites for these anymore, so now I can start
repurposing these methods for the new API endpoints I'm planning! :3

Now, `SwfAsset#image_url` approximately matches Impress 2020 logic: use
the thumbnail PNG from the manifest if one exists, or the Impress 2020
converter for canvas movies, or the old AWS copy generated by gnash if
necessary, or return nil.
2024-02-24 16:12:02 -08:00
829d960d70 Move SwfAsset#override_body_id to the bottom of the file
This is a console-y method, it's not important enough to be up here!
2024-02-24 15:24:24 -08:00
cf1cdf7e7d Reorder the higher-up lines of SwfAsset
A lot of stuff was mixed together, including methods interrupting the
scopes and relationships, oops! Fixed!
2024-02-24 15:22:14 -08:00
e8737a520e Remove more unused SwfAsset scopes
Wowzers, a lot of cruft in here! I hope I'm not deleting anything
important, but I think my Ctrl-F audits are checking out?
2024-02-24 15:19:28 -08:00
7fee4e6e41 Remove unused SwfAsset.fitting_standard_body_ids and support methods 2024-02-24 15:16:25 -08:00
c25e015b70 Delete some more unused SwfAsset methods 2024-02-24 15:15:21 -08:00
f8cf8165df Remove unused SwfAsset#images method
I think this used to be used in an API endpoint we've now deleted? I'm
just cleaning up call sites because I intend to refactor the `urls`
method and stuff, so I'm removing cruft that would complicate it!

I'm not certain-certain this is unused, but I did a global search for
`\bimages\b` in the codebase, and didn't find anything that looked like
a match to me!
2024-02-24 15:15:21 -08:00
b7f4c5b6ac Remove asset_hosts initializer
It just contains one constant, used in one place. Inlined and deleted!
2024-02-24 15:05:37 -08:00
992954ce89 Create swf_assets:manifests:load task to save all manifest files
Doing that sweet, sweet backfill!! It's not exactly *fast*, since
there's about 570k records to work through, but it's pretty good all
things considered! Thanks, surprisingly-reusable async code!
2024-02-23 14:06:49 -08:00
9a3b33ea2f Preload many manifests concurrently for the Alt Styles page
I'm gonna also use this for a task to try to warm up *all* the
manifests in the database! But to start, just a simple one, to prepare
the alt styles page quickly on first run. (This doesn't really matter
in production now that I've already visited the page once, but it helps
when resetting things in dev, and I think more it's about establishing
the pattern!)
2024-02-23 13:45:12 -08:00
2cc46703b9 Create NeopetsMediaArchive, read the actual manifests for Alt Styles
The Neopets Media Archive is a service that mirrors `images.neopets.com`
over time! Right now we're starting by just loading manifests, and
using them to replace the hacks we used for determining the Alt Style
PNG and SVG URLs; but with time, I want to load *all* customization
media files, to have our own secondary file source that isn't dependent
on Neopets to always be up.

Impress 2020 already caches manifest files, but this strategy is
different in two ways:

1. We're using the filesystem rather than a database column. (That is,
   manifest data is kinda duplicated in the system right now!) This is
   because I intend to go in a more file-y way long-term anyway, to
   load more than just the manifests.
2. Impress 2020 guesses at the manifest URLs by pattern, and reloads
   them on a regular basis. Instead, we use the modeling system: when
   TNT changes the URL of a manifest by appending a new `?v=` query
   string to it, this system will consider it a new URL, and will load
   the new copy accordingly.

Fun fact, I actually have been prototyping some of this stuff in a side
project I'd named `impress-media-server`! It's a little Sinatra app
that indeed *does* save all the files needed for customization, and can
generate lightweight lil preview iframes and images pretty easily. I
had initially been planning this as a separate service, but after
thinking over the arch a bit, I think it'll go smoother to just give
the main app all the same access and awareness—and I wrote it all in
Ruby and plain HTML/JS/CSS, so it should be pretty easy to port over
bit-by-bit!

Anyway, only Alt Styles use this for now, but my motivation is to be
able to use more-correct asset URL logic to be able to finally swap
over wardrobe-2020's item search to impress.openneo.net's item search
API endpoint—which will get "Items You Own" searches working again, and
whittle down one of the last big things Impress 2020 can do that the
main app can't. Let's see how it goes!
2024-02-23 12:02:39 -08:00
3f449310d6 Refactor item search JSON, add appearances
Preparing to finally move wardrobe-2020's item search to use the main
app's API endpoints instead!

One blocker I forgot about here: Impress 2020 has actual support for
knowing an item's true appearance, like by reading the manifest and
stuff, that we haven't really ported over. I feel like maybe I should
pause and work on the changes to manifest-archiving that I'd been
planning anyway? I'll think about it.
2024-02-23 10:44:50 -08:00
666394de25 Refactor Impress 2020 config
I've moved the support secret into the encrypted credentials file, and
moved the origin into a top-level custom config value in the
environment files, with different defaults per environment but still
the ability to override it. (I don't use this, but it feels polite to
not actually *demand* that people use port 4000, y'know?)
2024-02-22 13:07:43 -08:00
583f3c712f High-level caching for closet lists
Okay, so I still don't know why rendering is just so slow (though
migrating away from item translations did help!), but I can at least
cache entire closet lists as a basic measure.

That way, the first user to see the latest version of a closet list
will still need just as much time to load it… but *only* the ones that
have changed since last time (rather than always the full page), and
then subsequent users get to reuse it too!

Should help a lot for high-traffic lists, which incidentally are likely
to be the big ones belonging to highly active traders!

One big change we needed to make was to extract the `user-owns` and
`user-wants` classes (which we use for trade matches for *the user
viewing the list right now*) out of the cached HTML, and apply them
after with Javascript instead. I always dislike moving stuff to JS, but
the wins here seem. truly very very good, all things considered!
2024-02-20 18:43:39 -08:00
1e6ff4cefc Remove the Item::Translation model entirely
Okay, Impress 2020 is migrated off translations too, so we can start to
wrap this up!
2024-02-20 16:57:45 -08:00
3ac9e7ce69 Migrate item search away from item translations
Lightning fast for simple name queries now, gotta say!!
2024-02-20 16:04:41 -08:00
5ee3b472ec Migrate away from item translations in modeling
This one is important, I didn't notice that this is a way of setting
attributes that won't be written to both tables! `name` will only be
written to the translation table (which crashes the save), and the
other fields would only be written to the main table. Fixed! (I don't
like the super-dynamic this code was written before, anyway.)
2024-02-20 15:52:03 -08:00
0e8f457aa1 Oops, fix bug on item page now that translations aren't available
Missed this at first - now that the `name` field is just a normal field
and is always English, it's now an error to provide the locale to it as
a parameter, like we used to for the translated version of the field!
2024-02-20 15:37:07 -08:00
c75d988497 Migrate away from item translations in the Your Items feature
Just replacing references to the `Item::Translation` model to the
fields on `Item` itself!
2024-02-20 15:36:20 -08:00
a1066d9c8a Add translated item fields directly to the Item model
Like with Species, Color, and Zone, we're moving the translation data
directly onto the model, and just using English. This will simplify
some of our queries a lot (way fewer joins!), and it's what Neopets
does now anyway, and I have a secret hope that removing the complexity
along the codepath for `item.name` might help speed up large item lists
if we're lucky?? 🤞

Anyway, this is the first step, performing the migration to copy the
data onto the `items` table, making sure to keep them in sync for the
2020 app for now!
2024-02-20 15:25:03 -08:00
d39e7cea81 Move fundraising models into the Fundraising module
This was mostly straightforward it seems, whew!
2024-02-18 20:29:31 -08:00
1b22258576 Update special_color logic for Banana Chia Wings case
Tbh I'm not sure `special_color` is actually used anywhere? It used to
be how we decide what to show in the previewer on the item page, but
that's been replaced with the 2020 logic, so idk…

But in any case, I noticed that the description doesn't match the
pattern we have, so here's the fix!
2024-02-17 12:50:35 -08:00
6515e525fb Remove some unused PetType scopes
I looked at this and was like. "ok literally what is
`nonstandard_colors` trying to do"

reading it again now, I'm realizing the idea is that it probably runs
two queries: one to get nonstandard colors, then depends on
ActiveRecord to implicitly convert the relation to an array and then to
IDs for the second query? Instead of doing a join??

Idk, it's unused, so trash it!
2024-02-16 23:32:22 -08:00
15a905b0dd Allow modeling pets for colors we haven't gotten metadata for yet
This used to be the behavior, and the site has plenty of graceful
fallbacks for it, I just forgot this one when doing Rails upgrades!

Note that the impress-2020 stuff is *not* as graceful about this, so
the wardrobe page won't show the pet until the color is in the DB. Ah
well, still an improvement!
2024-02-16 23:28:10 -08:00
118ec6aa1a Oops, update outfit's updated_at when adding/removing items
This bug caused outfit thumbnails to get out of date, because the
outfit's `updated_at` field is used as a cache-busting part of the URL.
2024-02-08 10:32:53 -08:00
5d0848bf28 In development, use local Impress 2020 to generate outfit images
Just so we can see accurate stuff on the outfits page!
2024-02-08 10:03:10 -08:00
dc44b4dbb3 Oops, fix bug with saving outfits of pets loaded from Neopets.com
Okay right, the wardrobe-2020 app treats `state` as a bit of an
override thing, and `pose` is the main canonical field for how a pet
looks. We were missing a few pieces here:

1. After loading a pet, we weren't including the `pose` field in the
   initial query string for the wardrobe URL, but we _were_ including
   the `state` field, so the outfit would get set up with a conflicting
   pet state ID vs pose.
2. When saving an outfit, we weren't taking the `state` field into
   account at all. This could cause the saved outfit to not quite match
   how it actually looked in-app, because the default pet state for
   that species/color/pose trio could be different; and regardless, the
   outfit state would come back with `appearanceId` set to `null`,
   which wouldn't match the local outfit state, which would trigger an
   infinite loop.

Here, we complete the round-trip of the `state` field, from pet loading
to outfit saving to the outfit data that comes back after saving!
2024-02-08 09:51:31 -08:00
2667ed49ba Fully disable model translations for Color/Species/Zone
Now that DTI 2020 has been deployed without references to the
translations tables, we can stop keeping them in sync!

Next step is to drop the tables and be done with them altogether! (I
have a backup of the public data for this too, as does this repo!)
2024-02-03 08:13:14 -08:00
c93895b2f7 Fix mistake in pet loading error handling
Oops, warning isn't enough, we also need to stop, right lol!

I moved this into a method, so we could `return` out of there.
2024-02-01 07:21:47 -08:00
c60e222faa Add Alt Style support to outfit saving
Pretty straightforward, just add the field to the record, and wire it
all up! I'm glad this seemed to work out pretty well all-in-all 😅
2024-02-01 05:55:19 -08:00
cb90b79efe Remove logic to auto-detect Unconverted pets (fixes Baby Pteri bug)
Two motivations here:

1. Unconverted pets should no longer exist on Neopets.com (and we
   especially don't expect new ones), so this logic helps no one.
2. The Baby Pteri keeps getting overridden to be marked as Unconverted
   because it has only one asset, which is incorrect.

A decent heuristic for a bygone era, goodbye!
2024-02-01 01:40:50 -08:00
b50bf6ceb8 Add SVG support to alt styles
Just moving our hacks around and adapting them to the SVG case lol!
2024-01-31 03:02:19 -08:00
c2de6f7167 Display alt styles in outfit editor when selected
Yay it works(*)! But two major missing pieces:

- Outfit saving doesn't persist it at all
- Item compatibility is unaffected: items will still appear in search
  and in the preview, even when they don't fit anymore.
2024-01-30 07:01:03 -08:00
8e5939e408 Show alt style name in the pose picker button when selected
To help with space, I'm just showing the word "Nostalgic" (or "???" if
it's from a series we don't recognize, this is hardcoded by ID), and
trusting that from context it will be obvious that it's the "Nostalgic
Faerie" case or whatever. (Moreover, in both the button and the select
we're omitting the species name, by similar reasoning!)

Note that this _still_ doesn't actually apply the style to the outfit
whatsoever; this is all just local state as we're continuing to play
with UI concepts. Actually applying it is probably next though! (Though
there's a couple more UI things I want to do, like some affordances to
clarify that a Style is applied and that Expression changes won't work.)
2024-01-30 05:55:19 -08:00
514c99fb42 Add WIP styles tab to the pose picker
It shows the styles! You can select between them, but it currently does
nothing, womp womp!
2024-01-29 04:26:40 -08:00
72d4bc65ae API endpoints to load alt styles as JSON, and filter by species
Gonna use this in the app for the alt style picker! Still deciding on
the UI details.
2024-01-28 07:30:06 -08:00
7a1239ae5a Strip spaces from thumbnail URLs for colors like Usukiboy 2024-01-24 07:27:07 -08:00
394cc212b3 Add Styling Studio page to show the styles we already have modeled 2024-01-24 06:53:37 -08:00
ba45de583f Oops, don't update body_id when there's an alt style! 2024-01-24 06:12:35 -08:00
40278afd0c Add find_by_name method to AltStyle 2024-01-24 06:12:03 -08:00
e511bdc5e2 Oops right, it's okay to work with an alt style we've already seen! 2024-01-24 04:01:34 -08:00
4e5023288e Track contributions of Alt Styles 2024-01-24 03:54:43 -08:00
5004142dfb Add alt style support to modeling
Nothing to show them yet, but I think this works for loading it all in
the first place?

Still needs contributions tho!
2024-01-24 03:25:23 -08:00
4888a9dcbe Oops, remove stray viewer_data reader from Pet
This is left over from a previous plan I had, before moving
`fetch_viewer_data` to a class method.
2024-01-24 01:00:53 -08:00
b0e7f2ccd5 Move lib/rocketamf -> lib/rocketamf_extensions, to fix reload issues
Something in the Rails loader doesn't like that I have both a gem and
a lib folder named `RocketAMF`, I think? It'll often work for the first
pet load request, then on subsequent ones say `Envelope` is not
defined, I'm guessing because it scrapped the gem's module in favor of
mine?

Idk, let's just simplify all this by making our own module. I feel like
this old lib could use an overhaul and simplification anyway, but this
will do for now!
2024-01-24 00:59:11 -08:00
0406a32444 Disable loading alt style pets specifically 2024-01-24 00:54:30 -08:00
c33f1cb767 Extract Pet.fetch_viewer_data to a class method
This will make it easy to look up pet data from Neopets without messing
around with how the Pet model handles it!
2024-01-24 00:51:20 -08:00
c76e8cd2a9 Allow admin users to load pets
Also allows the new rake task we're building to load, too!
2024-01-24 00:42:35 -08:00
b1eca2f3a5 Disable modeling while we fix Alt Styles rollout bugs 2024-01-24 00:19:28 -08:00
09ad6d5fbb Convert /cpn/ loading errors to warnings
Idk why this broke again, it works locally for me, did our IP get
banned or something? We'll look into it another time!
2024-01-23 22:11:10 -08:00
44d9f98313 Increase pet loading timeout 2024-01-23 22:10:42 -08:00
73af9d4d77 Start migrating off globalize gem for zones
Like in 0dca538, this is preliminary work for being able to drop the
`zone_translations` table! We're copying the field over first, to be
able to migrate DTI 2020 safely before dropping anything.
2024-01-23 05:43:00 -08:00
48f613d9bc Oops, fix recent bug in species:acara item search filter
Ah oops, I just typo'd this and didn't test it properly, womp womp!
2024-01-23 05:31:09 -08:00
ce64f12cc3 Remove references to species/color translations at call sites
We wisely did preloading in some places. Now, we don't need to!
Hopefully this'll even be a free speed boost on some pages!
2024-01-23 05:23:57 -08:00
0dca538b0a Start migrating off globalize gem for species/color names
Non-English languages haven't been supported on Neopets for a while, so
I'd like to remove this extra cross-cutting complexity, especially
since it's now inconsistent with the real site anyway!

The main motivation is that I'd like to do this for items too, because
I have a hunch that all the complexity of `globalize` to read
`item.name` is part of what's making large user lists so slow to
render: lots of little objects getting created down the stack, and
needing to be garbage-collected later.

I'm not sure that's why! But I figure removing this complexity is a
simplicity win anyway, so let's do it!

Note that this doesn't *finish* the migration, it just starts it! The
`Species::Translation` and `Color::Translation` models still exist, and
still have their data, and not all references to them are scrubbed yet.
I especially don't want to delete the backing tables until both DTI and
DTI 2020 are ready for it!

So this change will someday be paired with another change to actually
drop the tables - after backing up the data for future records just in
case, of course!
2024-01-23 05:10:43 -08:00
23240005a2 Oops, fix bug that would sometimes flip own/want in the item bulk menu
If your first wanted list was created before your first owned list,
then `false` would come before `true` in the keys of
`current_user_lists`.

I both fixed this to be more consistent at the model level, because who
likes unpredictable behavior? But also downstream at the view I
hardcoded that true should come before false, because that's a UI
concern that I want to be encoded in the view regardless of what's
upstream.
2024-01-23 04:06:58 -08:00
5d577db649 Oops, load the data for the bulk item quantity form on the trades page! 2024-01-21 06:42:24 -08:00
402e3d4afb Basic trade hangers page, just content and without style
We're just getting started but there we go!! No links or styles yet,
just getting it done!
2024-01-21 03:10:06 -08:00
a03ae90697 Move more of the trade-fetching logic into the model
It was a bit tricky to figure out the right API for this, since I'm
looking ahead to the possibility of splitting these across multiple
pages with more detail, like we do in DTI 2020.

What I like about this API is that the caller gets to apply, or not
apply, whatever scopes they want to the underlying hanger set (like
`includes` or `order`), without violating the usual syntax by e.g.
passing it as a parameter to a method.
2024-01-21 00:39:20 -08:00
b1c1bea7be Remove unused data in items#show controller
Oh right, this was for the old pet previewer. Goodbye!
2024-01-20 23:53:30 -08:00
05c8e7f618 Oops, include default-list hangers in the trade lists
Without the `left_outer_joins`, this query only returns hangers that
have a matching list, which defeats the purpose of two of the three
clauses! lol
2024-01-19 01:27:57 -08:00
bec350e9f2 Filter trade lists to active users only
I don't have the cute fancy UI with trade list names included, but! Now
the trade ratios should match DTI 2020 and that's nice!
2024-01-19 00:38:16 -08:00
16328d3840 Refactor how we load closet hangers
Ahh I see, the way we got away with not having a `trading` scope before
was a weird metaprogramming `{owned/wanted}_trading` situation. Okay,
let's trash that in favor of our new stuff! And that helps us bulk the
queries too which is nice.
2024-01-19 00:28:28 -08:00
470c805880 Save last trade activity time onto User
In impress-2020, we do a big slow query to figure out which users have
been active in trades recently. Now, we cache that timestamp on the
User model.

This won't have any immediate effect; it's to clear the way for Classic
DTI to receive the better trade ratios feature people like from 2020.

I also added some unit testing infra because I finally wanted it! for
all the ways you can trigger this timestamp lol

Note too that this is a bit of an unusually complex migration, but my
hope is that the batching and query structure and such helps it run
surprisingly fast! 🤞
2024-01-19 00:00:46 -08:00
e337767954 Fix "couldn't download the associated Flash files" when modeling
So this was a slightly wrong error message, what was happening was:

1. Trying to load the image hash for this pet, by looking them up at
   https://pets.neopets.com/cpn/PET_NAME/1/1.png and seeing what URL it
   redirects to.
2. But pets.neopets.com was rejecting our User-Agent string, which
   would've been just "Ruby", since we hadn't set it otherwise. I guess
   that's an explicitly banned string?

I also found that the kind of more-helpful User-Agent string I like to
write was being rejected, and I could only get it to accept something
very simple? So that's what we're using now, I guess!!
2024-01-15 01:15:35 -08:00
a656cd511a Create endpoint to show all species appearances for a given color
Building toward replacing more of the 2020 data sources! I think this is
an endpoint that benefits from bulk loading, esp with the way the item
page previews work. I also like taking the concept of "canonical" out of
the GQL interface, and instead just loading for each of the 50 species
and letting the client decide. (And then it can fast-swap between them!)
2023-12-05 18:04:54 -08:00
c8de3dae63 Delete unused PetAttribute model files
Oh right, this powered `Color` and `Species` back before they were
database rows. Now these base classes are unused! Goodbye!
2023-12-05 14:56:34 -08:00
99f5cd717f Move some model mixins to lib/
The models folder is a bit confusingly large, these are more mixins and
kinda clutter it. Push them off into `lib`, I think!

I think they used to be in models mainly because Rails used to handle
`lib` differently with autoloading, and it made for a worse dev
experience. Now it's all the same, though!
2023-12-05 14:55:31 -08:00
c725a11920 Remove unused NC Mall spider code & fields
We haven't used the mall spider in this app in forever (I guess we even
deleted the code at some point?), but there was some vestigial stuff
left. Goodbye!
2023-11-11 15:45:38 -08:00
94d15f005d Delete a couple unused accessors
At least, I think they are!
2023-11-11 15:42:08 -08:00
a31039c4c8 Load item page restricted zones data from Rails app, not impress-2020
Just moving more stuff over! I modernized Item's `as_json` method while
I was here. (Note that I removed the NC/own/want fields, because I
think the only other place this method is still called from is the
quick-add feature on the closet lists page, and I think it doesn't use
these fields to do anything: updating the page is basically a full-page
reload, done sneakily.)
2023-11-11 08:49:19 -08:00
c6cb61ef38 Remove no-op item.thumbnail.secure_url
There was a time when I used an old proxy server to try to fix mixed
content issues, and I eventually removed it but never took the tendrils
out from the code.

We probably _should_ figure out how to secure these URLs! But until
then, we may as well simplify the code.
2023-11-11 08:24:08 -08:00
fe6264b20a Use an actual body with ID "0", instead of the string "all"
I changed my mind again! At first I wanted to make the special case
clearer, and to be able to more strongly assert that the species is
not null. But now I'm like… eh, there's code that references `body.id`
that has no reason _not_ to work in the all-bodies case… let's just
keep the types more consistent, I think.
2023-11-11 08:13:07 -08:00
eaab0b1922 Oops, return an array in the all-bodies case
I was returning just the body itself, oops! For consistency with the
general return value, it should be a one-element array.
2023-11-11 07:58:50 -08:00
133895b213 Refactor appearances data to use a "body" object
This is more similar to what impress-2020 does, I was working on the
wardrobe-2020 code and took some inspiration!

The body has an ID and a species, or is the string "all".
2023-11-11 07:54:56 -08:00
6c31094c2d Add known_glitches to appearances JSON 2023-11-11 07:21:13 -08:00
010f64264f Replace swf_assets#index with item_appearances#index
Preparing a better endpoint for wardrobe-2020 to use! I deleted the
now-unused swf_assets#index endpoint, and replaced it with an
"appearances" concept that isn't exactly reflected in the database
models but is a _lot_ easier for clients to work with imo.

Note that this was a big part of the motivation for the recent
`manifest_url` work—in this draft, I'm probably gonna have the client
request the manifest, rather than use impress-2020's trick of caching
it in the database! There's a bit of a perf penalty, but I think that's
a simpler starting point, and I have a hunch I'll be able to make up
the perf difference once we have the impress-media-server managing more
of these responsibilities.
2023-11-11 07:14:48 -08:00
3b59823fc4 Fix minor item search error message bug
Oops, I got the variable name wrong here! Before this change,
`species:foo` would crash. Now, it shows the error message correctly.
2023-11-11 07:04:57 -08:00
93511b3d51 Delete unused rake tasks 2023-11-10 18:04:24 -08:00
d35b10c1b8 Oops, fix validation bug for SwfAsset
Oh right, saving an asset that doesn't have a manifest will crash this
validation step!
2023-11-10 16:16:32 -08:00
96998643b5 Add manifest_url to swf_assets table
Ok so, impress-2020 guesses the manifest URL every time based on common
URL patterns. But the right way to do this is to read it from the
modeling data! But also, we don't have a great way to get the modeling
data directly. (Though as I write this, I guess we do have that
auto-modeling trick we use in the DTI 2020 codebase, I wonder if that
could work for this too?)

So anyway, in this change, we update the modeling code to save the
manifest URL, and also the migration includes a big block that attempts
to run impress-2020's manifest-guessing logic for every asset and save
the result!

It's uhh. Not fast. It runs at about 1 asset per second (a lot of these
aren't cache hits), and sometimes stalls out. And we have >600k assets,
so the estimated wall time is uhh. Seven days?

I think there's something we could do here around like, concurrent
execution? Though tbqh with the nature of the slowness being seemingly
about hitting the slow underlying images.neopets.com server, I don't
actually have a lot of faith that concurrency would actually be faster?

I also think it could be sensible to like… extract this from the
migration, and run it as a script to infer missing manifest URLs. That
would be easier to run in chunks and resume if something goes wrong.
Cuz like, I think my reasoning here was that backfilling this data was
part of the migration process… but the thing is, this migration can't
reliably get a manifest for everything (both cuz it depends on an
external service and cuz not everything has one), so it's a perfectly
valid migration to just leave the column as null for all the rows to
start, and fill this in later. I wish I'd written it like that!

But anyway, I'm just running this for now, and taking a break for the
night. Maybe later I'll come around and extract this into a separate
task to just try this on all assets missing manifests instead!
2023-11-09 21:42:51 -08:00
3a963c7d25 Fix old .scoped -> .all Rails upgrade bug
Ahh, I guess I missed these, I think they're maybe not actually used in
the app is why? cuz they're all default values that are overridden at
the actual call sites. But I ran into it when running `Pet.load` in the
console, and yeah let's just fix 'em up!
2023-11-09 21:35:42 -08:00
c9dd0f7376 Remove "Import from pets" page
This hasn't worked for a while, and I don't know an API off the top of
my head to drop in for it. Let's just delete it for now, and revisit it
later if we want to!
2023-11-06 13:06:18 -08:00
a2feee2d9b Add support for is_manually_nc
A really really simple change! It works on the item page, the item
index page, item search, the homepage, and the item lists page.

The main reason I avoided this for so long (even before modernizing the
Rails app) was that the ElasticSearch stuff felt like it made it messy?
But now it's pretty simple, and it works in search already cuz I did
that when I implemented item search, so, nice!
2023-11-03 16:27:39 -07:00
5dcb1dedb4 Add Owls values to the item page
Eyy it's time!! Long-requested, finally here lol
2023-11-03 16:20:02 -07:00
7a3aa609ba Use the main app for outfit saving, not impress-2020
This came in a few parts!
1. Add meta tags to let us know we're logged in.
2. Install React Query, which has the data-loading sensibilities I like
   about Apollo without the GraphQL that has honestly been a drag.
3. Replace the outfit-loading and outfit-saving calls with API calls to
   the main app.
4. Update the main app's API calls to use our more flexible data
   constructs like "pose".

Would've loved to do this more incrementally, but it's hard to! You
can't split out outfit-loading and outfit-saving, or auth from any of
that, or the state gets all out-of-sorts.

Still, this is a good nugget we've pulled out all-in-all, and one that
people have been asking for! Can maybe look to logged-in item search
soon too, for own/want data?
2023-11-02 16:54:35 -07:00
7948974949 Do preloading manually on user list pages, to reduce memory usage
I used the new profiler tools on this page, and noticed a lot of
allocations in the Globalize library, which we use for translating
database records. I realized that we were loading all of the fields of
not just all of the items on the page, but all of their translation
records in all locales! We used to scrape data for lots of languages, so
that can be quite a lot!

Unfortunately, Rails's `includes` method to efficiently preload related
records always loads all fields, and simply can't be overridden.

So, in this change we write manual preloading code, to identify the
records we need, load them in big bulk queries, and assign them back to
the appropriate associations. Basically just what `includes` does, but
written out a bit more, to give us the chance to specify SELECT and
WHERE clauses!
2023-10-27 19:42:02 -07:00
22e3f4240a Update most URLs to use HTTPS
I noticed we didn't have the little lock icon in the browser, and yeah
huh there's a lot of `http://` still floating around! Let's fix that!
2023-10-25 15:22:57 -07:00
ee3ffe8afe Delete unused item proxy class
We used to do this for weird clever caching tricks that I don't think
were actually very effective. We stopped using this a few months ago,
and now I'm finally cleaning up this supporting code!
2023-10-25 12:55:30 -07:00
4cd12f6132 Delete unused FragmentExpiration module 2023-10-25 12:51:16 -07:00
f8cf3fd701 Fix crash when searching for "is:nc"
Huh, Arel can *sometimes* handle just having an attribute stand in as
"X is true" in a condition, but sometimes gets upset about it. I guess
this changed in Rails since we recently wrote this?

Specifically, item search would crash on "is:nc" (but *not* "is:np"),
saying:

```
undefined method `fetch_attribute' for #<struct Arel::Attributes::Attribute relation=#<Arel::Table:0x0000000109a67110 @name="items", @klass=Item(…), @type_caster=#<ActiveRecord::TypeCaster::Map:0x0000000109a66e90 @klass=Item(…)>, @table_alias=nil>, name="is_manually_nc">
```

The traceback was a bit misleading (it happened at the part where we
merge all the scopes together), but that hinted to me that it working
with an attribute in a place where it expected a conditional. So I
converted the attribute in the `is_nc` scope to a conditional, and made
the matching change in `is_np`, and that fixed it! Ok phew!
2023-10-25 12:46:48 -07:00
f5b45d2dc8 Delete unused roulette feature
It was a cute idea! But it's been broken for a long time now. Goodbye!
2023-10-24 19:05:18 -07:00
c06cea2a3c Remove slightly-broken edit outfit links from user outfits page
The URL anchors were getting like. double-encoded? The `closet[]` part
was encoding as `closet%255B%255D`. Maybe a thing in Rails, where you
need to mark them `html_safe` to insert them in a URL like that?

Well anyway, those URLs are redundant now, I just have it link straight
to the same outfit page as the big link!
2023-10-24 18:05:51 -07:00
b8a8cb9b20 Stop orphaning hangers when deleting lists
Idk if this used to be different or what, but it looks like the current
behavior is: if you delete a closet list, it'll leave the hangers
present, but Classic DTI would not show them anywhere; but Impress 2020
(until recently) would crash about it.

Now, we use `dependent: :destroy` to delete the hangers when you delete
the list (which I think makes sense, and is different than what I
decided in the past but that's ok, and is what the current behavior
*looks* like to people!), and we add a migration that deletes orphaned
hangers.

The migration also outputs the deleted hangers as JSON, for us to hold
onto in case we made a mistake! I'm also backing up the database in
advance of running this migration, just in case we gotta roll back HARD!
2023-10-24 15:35:21 -07:00
d8b0bf3174 Skip item translations when modeling
Okay, this is a process that idk if it's even been working for a while anyway, I don't think Neopets translates item names anymore?

And it's crashing when I try to model stuff now, so like. yeah ok I'm fine with just skipping this, it's a shame to lose out on potential data going forward but *I think there just isn't data to get anyway*
2023-10-23 19:05:10 -07:00
5f3ce1210a Stop saving local copies of SWFs
I think we used this for both conversion to image, and also for CORS stuff when rendering Flash-based previews… let's trash it, I don't want to be growing our hard drive with files I don't think we use anymore!

If I'm wrong and it turns out we do use them for something, then like. hey I'm sure we'll find out soon enough, and it's very recoverable operation.
2023-10-23 19:05:10 -07:00
966a0025e5 Fix SWF downloads in modeling
A lot simpler with latest Ruby! We can drop the whole SSL workaround yaay
2023-10-23 19:05:10 -07:00
468caea4ae Oops, fix typo include -> includes 2023-10-23 19:05:09 -07:00
3398439bae Fix more modeling bugs
Just find_all_by's that I never cleaned up

Oddly enough, I still got a "neopets seems down" message out of this, idk if that's an actual bug or just sluggishness rn
2023-10-23 19:05:09 -07:00
cbad00f32d Fix AMFPHP requests
Okay, right, if we're just using www.neopets.com (like we are for now), it fails on http://www.neopets.com because it triggers a redirect that we don't follow.

So here I 1) change the default to HTTPS, and 2) add HTTPS support to our little RocketAMF lib
2023-10-23 19:05:09 -07:00
1e690d9f6c Fix old find_or_initialize_by methods
I missed this in the Rails upgrades, oops!
2023-10-23 19:05:09 -07:00
0e7bbd526f Clarify the error behavior on AuthUser syncing
I added bangs to signify they're mutative operations, but also the bang on `create!` helps ensure we'll bail if User creation fails for some reason.
2023-10-23 19:05:08 -07:00
82be3bf5f9 Enable password resets
Nice, just turning it on seemed to do all we need for now!

Fair questions to be asked about like, should you be able to look up by username instead of email? But like idk, this feels simpler *and* more solid, to give you feedback on if it's the right email.
2023-10-23 19:05:08 -07:00
33b2953949 Enable basic AuthUser tracking
Simply turning back on the module that tracks sign-ins and IP addresses.
2023-10-23 19:05:08 -07:00
e79428fa28 Add Remember Me to login
This requires a migration, our first migration against the openneo_id database from this app! Fun!
2023-10-23 19:05:08 -07:00
d65aafdd4c Signup and settings page for OpenNeo ID accounts
Hey nice!!

Note that I removed an account delete button from the settings page. You can still send a DELETE request to the right endpoint to do it, but it's not gonna delete all the associated records, and I wanna think a bit about how to handle that better before exposing that button.
2023-10-23 19:05:07 -07:00
eee097a9f8 Sync AuthUser and User names
Callbacks are handy for this!
2023-10-23 19:05:07 -07:00
75185de957 Create a User when we create an AuthUser
We also update seeds.rb to be up to date! This should make it possible to log in as test/test123 from a fresh database.
2023-10-23 19:05:07 -07:00
621a0bc211 Mark some more relationships as optional
I noticed this was stopping changing your default list visibility bc contact neopets connection can't be empty, so I fixed that!

And then I just decided to scroll through every `belongs_to` relationship and add optional to the ones that jumped out at me lol
2023-10-23 19:05:07 -07:00
83f80facda Can log into OpenNeo ID accounts directly!
A lot of rough edges here (e.g. no styles on the flash messages), but it's working and that's good!!

I tested this by temporarily switching to the production database and logging in as matchu!

Still missing a lot of big features too, like registration, password resets, settings page, etc.
2023-10-23 19:05:07 -07:00
700e26d7df Remove old OpenNeo ID auth code
This removes login/logout/session logic for integrating with OpenNeo ID, replacing them with stubs that just redirect to `/?TODO` when you click login, and helpers that act as if you're not logged in.

This gives us a clean slate to plug in new Devise logic to integrate with the `openneo_id` database directly!
2023-10-23 19:05:07 -07:00
1d5af835a5 Add AuthUser model, connecting to openneo_id db
No user-facing functionality here yet, just configuring the database connection to work with openneo_id records.

This is a first step in integrating Devise stuff into this app instead of connecting with a weird second app.

My basic testing for this was to temporarily connect to production `openneo_id`, and see `AuthUser.first` correctly return a user!
2023-10-23 19:05:07 -07:00
0e4cc80ac8 Remove sanitize_sql_like monkey-patch
I had added this many Rails versions ago during the recent upgrade process, because it was in latest Rails but not in the version of Rails I was using when replacing Elasticsearch with MySQL queries. We can remove it now!
2023-10-23 19:05:07 -07:00
3263a926dc Oops, add species search filter back
lmao I keep forgetting things! note that the negative case of this filter, like the negative case of `fits`, is currently broken because Rails changed the default SQL mode and I didn't notice! We'll need to add a `database.yml` file and set `sql_mode: TRADITIONAL`.
2023-10-23 19:05:07 -07:00
eb5e9d53cb Oops, fix anonymous outfits
These are some old issues I just never tested for I think
2023-10-23 19:05:07 -07:00
cfc5c4ef19 Fix stray bugs on minor codepaths
I ran `rails zeitwerk:check`, which eager-loads the app, and it found two problems: `closet_group.rb` doesn't define `ClosetGroup` (cuz it's empty), and I left in a reference to a cache sweeper observer oops. Goodbye!
2023-10-23 19:05:06 -07:00
62fcc9fe00 Fix moving closet hanger to null-list
Rails 5 added new validation on `belongs_to` to ensure the corresponding record exists. In the case of moving to the null list, this shouldn't trigger!

I wish we could flag that specifically `nil` is okay, but other values should be validated? But oh well, this is fine!
2023-10-23 19:05:06 -07:00
e8131f3608 Fix bug with Arel ordering
Ok so weird little situation, usually Arel will accept an attribute as a param to `order()`, but not when it's in a very specific situation of all of the following:

`Item.joins(:translations).includes(:translations).limit(30).order(Item::Translation.arel_table[:name])`

For some reason, it's all like "hey I can't call `to_sql` on an attribute!", but only in the scenario where all 3 of those other things are present. Weird!

Anyway, explicitly saying `.asc` fixes this. Ok!
2023-10-23 19:05:05 -07:00
d97c32b5da Upgrade to Rails 5.2.8.1
Some important little upgrades but mostly straightforward!

Note that there's still a known issue where item searches crash, I was hoping that this was a bug in Rails 4.2 that would be fixed on upgading to 5, but nope, oh well!

Also uhh I just got a bit silly and didn't actually mean to go all the way to 5.2 in one go, I had meant to start at 5.0… but tbh the 5.1 and 5.2 changes seem small, and this seems to be working, so. Yeah ok let's roll!
2023-10-23 19:05:05 -07:00
f80d220a62 Upgrade to Rails 4.2.11.3 and Ruby 2.4.10
Some tricks required here to get the dependencies to work out, but we got it!!

Oh also, we move away from the rbenv in Ubuntu's package manager, because it doesn't support more recent Rubies like 2.4.10.
2023-10-23 19:05:05 -07:00
a15cbeb307 Remove pet state labeling & Neopets gem
This labeling technique hasn't worked in a long time bc it requires being logged in. These days we just manually label them with the 2020 support tools I think!

Clearing out the Neopets gem should help us manage some gem dep conflicts in the 4.2 upgrade too (I think the nokogiri one gets tricky?)
2023-10-23 19:05:05 -07:00
0d4c6ca077 Delete utf8 backfill for old Ruby
Idk if the replacement `require` is necessary exactly, but it's the one-to-one replacement for this lib, so let's start there for now!
2023-10-23 19:05:05 -07:00
c4f3a472ff Remove RightAWS gem
We're not connecting to AWS directly anymore, now that we deleted the SWF conversion stuff, so we can finally clear this out!
2023-10-23 19:05:05 -07:00
Matchu
d0616b6dfd Delete Camo references & Addressable gem
At one point we piloted a "Camo" service to proxy HTTPS image urls for us, but it doesn't exist anymore.

We already have proxies and stuff for this, so I left `Image` as a placeholder for this, but it's not working yet!

This also deletes our final reference to the Addressable gem, so we can remove it!
2023-10-23 19:05:05 -07:00
Matchu
fd263ea82f Remove mall spider cron jobs
I don't think these work anymore, and our volunteers get new items into the db fast anyway, Impress 2020 is doing better spidering these days. And then we get to remove the cron job `whenever` gem!
2023-10-23 19:05:05 -07:00
Matchu
31ebf7d4eb Remove OutfitImageUploader and Carrierwave
Ok right, missed this in the outfit image stuff deletion, get rid of the code and library we were using to push those images up to S3!
2023-10-23 19:05:05 -07:00
Matchu
38f3c9894c Delete WardrobeTip model
lmao I have zero recollection of this, I've been generally trying to avoid too-far-off-the-path cleanup, but this one just seems silly, goodbye
2023-10-23 19:05:05 -07:00
Matchu
93c596007d Clarify image path methods for SwfAsset
Using `s3_path` and stuff made it sound like we were still referencing the original Amazon S3 images - but actually our new asset proxy just uses the same path structure, and we didn't change anything about it.

Oh also I deleted an after_conversion method that isn't used anymore, forgot about that!
2023-10-23 19:05:05 -07:00
Matchu
8ea74b737e Remove outfit image saving
This has already been moved to Impress 2020 too, so we can delete all the image generation and saving!
2023-10-23 19:05:04 -07:00
Matchu
e121d8bba2 Remove SWF conversion
We've already swapped out the backend for this stuff to Impress 2020, so the resque task and the broken image report UI aren't actually relevant anymore. Delete them!

This helps us delete Resque soon too.
2023-10-23 19:05:04 -07:00
Matchu
a4a0188b1b Stop caching trade users on item page
Idk this one might actually be a bit of a pain to load? But I'd want to optimize it differently anyway, and there's overhauls we're already planning to do here.
2023-10-23 19:05:04 -07:00
Matchu
44341ba731 Stop caching pet type images on item page
That's easy queries and easy templates!
2023-10-23 19:05:04 -07:00
Matchu
b87492d4ee Stop caching item page contributors
This lets us remove the contribution observer too!
2023-10-23 19:05:04 -07:00
Matchu
cf2f78703b Stop caching latest contribution
That's another tiny query that I'm okay with just removing the cache overhead complexity for right now!
2023-10-23 19:05:04 -07:00
Matchu
1bf84b5106 Remove unused "outfits#new newest_items" cache
Huh! This cache key seemed to only be referenced in checks and expirations, but was never actually used! So I guess we've been loading the modeling predictions every time for a while huh??

We'll get smarter about that someday, but anyway, that lets us delete our Item resque tasks and ItemObserver!
2023-10-23 19:05:04 -07:00
Matchu
41fdcb5abc Remove newest_items caching from items page
Yeah I'm very unconvinced of the merit of saving us one items/translations query lmao
2023-10-23 19:05:04 -07:00
Matchu
db74dd1e29 Remove as_json item caching
Again I'm just not convinced of the perf on this, and it enables us to delete some whole infra over it, we can improve it another time if it's useful to!
2023-10-23 19:05:04 -07:00
Matchu
ffa73b6b03 Simplify item page rendering
Just removing some caching and the expiration of it! There's still more superfluous(?) caching on the item page to audit, but these seem a bit more sensible about avoiding loading extra data.
2023-10-23 19:05:04 -07:00
Matchu
02abd4e07f Simplify item_link rendering
In the interest of clearing out Resque, I'm just gonna remove a lot of our more complex caching stuff, and we can do a perf pass for things like big item list pages once everything's upgraded. (I'm hopeful that the upgrades themselves improve perf; and if not, that some improved sensibilities 10 years later can find simpler approaches.)
2023-10-23 19:05:04 -07:00
Matchu
44a00146aa Oops, remove some remaining references to Flex
We uninstalled Flex, our Elasticsearch gem, to replace item search with direct DB queries; but I forgot these calls, oops!

I also kinda want to see about deleting the resque tasks altogether, since I'm not sure how to get Resque installed on latest rails bc there seems to be a conflict over the version of Rack? And it'd be nice to get rid of the complexity if we can.
2023-10-23 19:05:04 -07:00
Matchu
5d1cce293b Reimplement Advanced Search
Oh I uhhh flat out forgot about this LMAO

well it's back now!! and was pleasantly easy to build, following the `from_text` example & abstractions
2023-10-23 19:05:04 -07:00
Matchu
3e728a8e56 Oops, fix ambiguous zones_restrict column
When combining occupies and restricts filters, this wouldn't be sure whether to apply to the items or the swf assets.
2023-10-23 19:05:04 -07:00
Matchu
fd87b41c17 Oops, add unowned & unwanted support to search!
Uhhh idk how I messed this up, but right, wanting is not the opposite of owning, LOL!
2023-10-23 19:05:04 -07:00
Matchu
52d4ec8443 Convert has_many conditions to where block
Looks like `conditions` was removed in this version! This broke outfit saving. Now it's fixed!
2023-10-23 19:05:04 -07:00
Matchu
2dd9cfb3d5 Use all instead of scoped
Back in the day, `all` would immediately load up a query into an array, but now I think it's an alias for what `scoped` used to be: a relation that contains everything.
2023-10-23 19:05:04 -07:00