1
0
Fork 0
forked from OpenNeo/impress
Commit graph

484 commits

Author SHA1 Message Date
a03ae90697 Move more of the trade-fetching logic into the model
It was a bit tricky to figure out the right API for this, since I'm
looking ahead to the possibility of splitting these across multiple
pages with more detail, like we do in DTI 2020.

What I like about this API is that the caller gets to apply, or not
apply, whatever scopes they want to the underlying hanger set (like
`includes` or `order`), without violating the usual syntax by e.g.
passing it as a parameter to a method.
2024-01-21 00:39:20 -08:00
b1c1bea7be Remove unused data in items#show controller
Oh right, this was for the old pet previewer. Goodbye!
2024-01-20 23:53:30 -08:00
05c8e7f618 Oops, include default-list hangers in the trade lists
Without the `left_outer_joins`, this query only returns hangers that
have a matching list, which defeats the purpose of two of the three
clauses! lol
2024-01-19 01:27:57 -08:00
bec350e9f2 Filter trade lists to active users only
I don't have the cute fancy UI with trade list names included, but! Now
the trade ratios should match DTI 2020 and that's nice!
2024-01-19 00:38:16 -08:00
16328d3840 Refactor how we load closet hangers
Ahh I see, the way we got away with not having a `trading` scope before
was a weird metaprogramming `{owned/wanted}_trading` situation. Okay,
let's trash that in favor of our new stuff! And that helps us bulk the
queries too which is nice.
2024-01-19 00:28:28 -08:00
470c805880 Save last trade activity time onto User
In impress-2020, we do a big slow query to figure out which users have
been active in trades recently. Now, we cache that timestamp on the
User model.

This won't have any immediate effect; it's to clear the way for Classic
DTI to receive the better trade ratios feature people like from 2020.

I also added some unit testing infra because I finally wanted it! for
all the ways you can trigger this timestamp lol

Note too that this is a bit of an unusually complex migration, but my
hope is that the batching and query structure and such helps it run
surprisingly fast! 🤞
2024-01-19 00:00:46 -08:00
e337767954 Fix "couldn't download the associated Flash files" when modeling
So this was a slightly wrong error message, what was happening was:

1. Trying to load the image hash for this pet, by looking them up at
   https://pets.neopets.com/cpn/PET_NAME/1/1.png and seeing what URL it
   redirects to.
2. But pets.neopets.com was rejecting our User-Agent string, which
   would've been just "Ruby", since we hadn't set it otherwise. I guess
   that's an explicitly banned string?

I also found that the kind of more-helpful User-Agent string I like to
write was being rejected, and I could only get it to accept something
very simple? So that's what we're using now, I guess!!
2024-01-15 01:15:35 -08:00
a656cd511a Create endpoint to show all species appearances for a given color
Building toward replacing more of the 2020 data sources! I think this is
an endpoint that benefits from bulk loading, esp with the way the item
page previews work. I also like taking the concept of "canonical" out of
the GQL interface, and instead just loading for each of the 50 species
and letting the client decide. (And then it can fast-swap between them!)
2023-12-05 18:04:54 -08:00
c8de3dae63 Delete unused PetAttribute model files
Oh right, this powered `Color` and `Species` back before they were
database rows. Now these base classes are unused! Goodbye!
2023-12-05 14:56:34 -08:00
99f5cd717f Move some model mixins to lib/
The models folder is a bit confusingly large, these are more mixins and
kinda clutter it. Push them off into `lib`, I think!

I think they used to be in models mainly because Rails used to handle
`lib` differently with autoloading, and it made for a worse dev
experience. Now it's all the same, though!
2023-12-05 14:55:31 -08:00
c725a11920 Remove unused NC Mall spider code & fields
We haven't used the mall spider in this app in forever (I guess we even
deleted the code at some point?), but there was some vestigial stuff
left. Goodbye!
2023-11-11 15:45:38 -08:00
94d15f005d Delete a couple unused accessors
At least, I think they are!
2023-11-11 15:42:08 -08:00
a31039c4c8 Load item page restricted zones data from Rails app, not impress-2020
Just moving more stuff over! I modernized Item's `as_json` method while
I was here. (Note that I removed the NC/own/want fields, because I
think the only other place this method is still called from is the
quick-add feature on the closet lists page, and I think it doesn't use
these fields to do anything: updating the page is basically a full-page
reload, done sneakily.)
2023-11-11 08:49:19 -08:00
c6cb61ef38 Remove no-op item.thumbnail.secure_url
There was a time when I used an old proxy server to try to fix mixed
content issues, and I eventually removed it but never took the tendrils
out from the code.

We probably _should_ figure out how to secure these URLs! But until
then, we may as well simplify the code.
2023-11-11 08:24:08 -08:00
fe6264b20a Use an actual body with ID "0", instead of the string "all"
I changed my mind again! At first I wanted to make the special case
clearer, and to be able to more strongly assert that the species is
not null. But now I'm like… eh, there's code that references `body.id`
that has no reason _not_ to work in the all-bodies case… let's just
keep the types more consistent, I think.
2023-11-11 08:13:07 -08:00
eaab0b1922 Oops, return an array in the all-bodies case
I was returning just the body itself, oops! For consistency with the
general return value, it should be a one-element array.
2023-11-11 07:58:50 -08:00
133895b213 Refactor appearances data to use a "body" object
This is more similar to what impress-2020 does, I was working on the
wardrobe-2020 code and took some inspiration!

The body has an ID and a species, or is the string "all".
2023-11-11 07:54:56 -08:00
6c31094c2d Add known_glitches to appearances JSON 2023-11-11 07:21:13 -08:00
010f64264f Replace swf_assets#index with item_appearances#index
Preparing a better endpoint for wardrobe-2020 to use! I deleted the
now-unused swf_assets#index endpoint, and replaced it with an
"appearances" concept that isn't exactly reflected in the database
models but is a _lot_ easier for clients to work with imo.

Note that this was a big part of the motivation for the recent
`manifest_url` work—in this draft, I'm probably gonna have the client
request the manifest, rather than use impress-2020's trick of caching
it in the database! There's a bit of a perf penalty, but I think that's
a simpler starting point, and I have a hunch I'll be able to make up
the perf difference once we have the impress-media-server managing more
of these responsibilities.
2023-11-11 07:14:48 -08:00
3b59823fc4 Fix minor item search error message bug
Oops, I got the variable name wrong here! Before this change,
`species:foo` would crash. Now, it shows the error message correctly.
2023-11-11 07:04:57 -08:00
93511b3d51 Delete unused rake tasks 2023-11-10 18:04:24 -08:00
d35b10c1b8 Oops, fix validation bug for SwfAsset
Oh right, saving an asset that doesn't have a manifest will crash this
validation step!
2023-11-10 16:16:32 -08:00
96998643b5 Add manifest_url to swf_assets table
Ok so, impress-2020 guesses the manifest URL every time based on common
URL patterns. But the right way to do this is to read it from the
modeling data! But also, we don't have a great way to get the modeling
data directly. (Though as I write this, I guess we do have that
auto-modeling trick we use in the DTI 2020 codebase, I wonder if that
could work for this too?)

So anyway, in this change, we update the modeling code to save the
manifest URL, and also the migration includes a big block that attempts
to run impress-2020's manifest-guessing logic for every asset and save
the result!

It's uhh. Not fast. It runs at about 1 asset per second (a lot of these
aren't cache hits), and sometimes stalls out. And we have >600k assets,
so the estimated wall time is uhh. Seven days?

I think there's something we could do here around like, concurrent
execution? Though tbqh with the nature of the slowness being seemingly
about hitting the slow underlying images.neopets.com server, I don't
actually have a lot of faith that concurrency would actually be faster?

I also think it could be sensible to like… extract this from the
migration, and run it as a script to infer missing manifest URLs. That
would be easier to run in chunks and resume if something goes wrong.
Cuz like, I think my reasoning here was that backfilling this data was
part of the migration process… but the thing is, this migration can't
reliably get a manifest for everything (both cuz it depends on an
external service and cuz not everything has one), so it's a perfectly
valid migration to just leave the column as null for all the rows to
start, and fill this in later. I wish I'd written it like that!

But anyway, I'm just running this for now, and taking a break for the
night. Maybe later I'll come around and extract this into a separate
task to just try this on all assets missing manifests instead!
2023-11-09 21:42:51 -08:00
3a963c7d25 Fix old .scoped -> .all Rails upgrade bug
Ahh, I guess I missed these, I think they're maybe not actually used in
the app is why? cuz they're all default values that are overridden at
the actual call sites. But I ran into it when running `Pet.load` in the
console, and yeah let's just fix 'em up!
2023-11-09 21:35:42 -08:00
c9dd0f7376 Remove "Import from pets" page
This hasn't worked for a while, and I don't know an API off the top of
my head to drop in for it. Let's just delete it for now, and revisit it
later if we want to!
2023-11-06 13:06:18 -08:00
a2feee2d9b Add support for is_manually_nc
A really really simple change! It works on the item page, the item
index page, item search, the homepage, and the item lists page.

The main reason I avoided this for so long (even before modernizing the
Rails app) was that the ElasticSearch stuff felt like it made it messy?
But now it's pretty simple, and it works in search already cuz I did
that when I implemented item search, so, nice!
2023-11-03 16:27:39 -07:00
5dcb1dedb4 Add Owls values to the item page
Eyy it's time!! Long-requested, finally here lol
2023-11-03 16:20:02 -07:00
7a3aa609ba Use the main app for outfit saving, not impress-2020
This came in a few parts!
1. Add meta tags to let us know we're logged in.
2. Install React Query, which has the data-loading sensibilities I like
   about Apollo without the GraphQL that has honestly been a drag.
3. Replace the outfit-loading and outfit-saving calls with API calls to
   the main app.
4. Update the main app's API calls to use our more flexible data
   constructs like "pose".

Would've loved to do this more incrementally, but it's hard to! You
can't split out outfit-loading and outfit-saving, or auth from any of
that, or the state gets all out-of-sorts.

Still, this is a good nugget we've pulled out all-in-all, and one that
people have been asking for! Can maybe look to logged-in item search
soon too, for own/want data?
2023-11-02 16:54:35 -07:00
7948974949 Do preloading manually on user list pages, to reduce memory usage
I used the new profiler tools on this page, and noticed a lot of
allocations in the Globalize library, which we use for translating
database records. I realized that we were loading all of the fields of
not just all of the items on the page, but all of their translation
records in all locales! We used to scrape data for lots of languages, so
that can be quite a lot!

Unfortunately, Rails's `includes` method to efficiently preload related
records always loads all fields, and simply can't be overridden.

So, in this change we write manual preloading code, to identify the
records we need, load them in big bulk queries, and assign them back to
the appropriate associations. Basically just what `includes` does, but
written out a bit more, to give us the chance to specify SELECT and
WHERE clauses!
2023-10-27 19:42:02 -07:00
22e3f4240a Update most URLs to use HTTPS
I noticed we didn't have the little lock icon in the browser, and yeah
huh there's a lot of `http://` still floating around! Let's fix that!
2023-10-25 15:22:57 -07:00
ee3ffe8afe Delete unused item proxy class
We used to do this for weird clever caching tricks that I don't think
were actually very effective. We stopped using this a few months ago,
and now I'm finally cleaning up this supporting code!
2023-10-25 12:55:30 -07:00
4cd12f6132 Delete unused FragmentExpiration module 2023-10-25 12:51:16 -07:00
f8cf3fd701 Fix crash when searching for "is:nc"
Huh, Arel can *sometimes* handle just having an attribute stand in as
"X is true" in a condition, but sometimes gets upset about it. I guess
this changed in Rails since we recently wrote this?

Specifically, item search would crash on "is:nc" (but *not* "is:np"),
saying:

```
undefined method `fetch_attribute' for #<struct Arel::Attributes::Attribute relation=#<Arel::Table:0x0000000109a67110 @name="items", @klass=Item(…), @type_caster=#<ActiveRecord::TypeCaster::Map:0x0000000109a66e90 @klass=Item(…)>, @table_alias=nil>, name="is_manually_nc">
```

The traceback was a bit misleading (it happened at the part where we
merge all the scopes together), but that hinted to me that it working
with an attribute in a place where it expected a conditional. So I
converted the attribute in the `is_nc` scope to a conditional, and made
the matching change in `is_np`, and that fixed it! Ok phew!
2023-10-25 12:46:48 -07:00
f5b45d2dc8 Delete unused roulette feature
It was a cute idea! But it's been broken for a long time now. Goodbye!
2023-10-24 19:05:18 -07:00
c06cea2a3c Remove slightly-broken edit outfit links from user outfits page
The URL anchors were getting like. double-encoded? The `closet[]` part
was encoding as `closet%255B%255D`. Maybe a thing in Rails, where you
need to mark them `html_safe` to insert them in a URL like that?

Well anyway, those URLs are redundant now, I just have it link straight
to the same outfit page as the big link!
2023-10-24 18:05:51 -07:00
b8a8cb9b20 Stop orphaning hangers when deleting lists
Idk if this used to be different or what, but it looks like the current
behavior is: if you delete a closet list, it'll leave the hangers
present, but Classic DTI would not show them anywhere; but Impress 2020
(until recently) would crash about it.

Now, we use `dependent: :destroy` to delete the hangers when you delete
the list (which I think makes sense, and is different than what I
decided in the past but that's ok, and is what the current behavior
*looks* like to people!), and we add a migration that deletes orphaned
hangers.

The migration also outputs the deleted hangers as JSON, for us to hold
onto in case we made a mistake! I'm also backing up the database in
advance of running this migration, just in case we gotta roll back HARD!
2023-10-24 15:35:21 -07:00
d8b0bf3174 Skip item translations when modeling
Okay, this is a process that idk if it's even been working for a while anyway, I don't think Neopets translates item names anymore?

And it's crashing when I try to model stuff now, so like. yeah ok I'm fine with just skipping this, it's a shame to lose out on potential data going forward but *I think there just isn't data to get anyway*
2023-10-23 19:05:10 -07:00
5f3ce1210a Stop saving local copies of SWFs
I think we used this for both conversion to image, and also for CORS stuff when rendering Flash-based previews… let's trash it, I don't want to be growing our hard drive with files I don't think we use anymore!

If I'm wrong and it turns out we do use them for something, then like. hey I'm sure we'll find out soon enough, and it's very recoverable operation.
2023-10-23 19:05:10 -07:00
966a0025e5 Fix SWF downloads in modeling
A lot simpler with latest Ruby! We can drop the whole SSL workaround yaay
2023-10-23 19:05:10 -07:00
468caea4ae Oops, fix typo include -> includes 2023-10-23 19:05:09 -07:00
3398439bae Fix more modeling bugs
Just find_all_by's that I never cleaned up

Oddly enough, I still got a "neopets seems down" message out of this, idk if that's an actual bug or just sluggishness rn
2023-10-23 19:05:09 -07:00
cbad00f32d Fix AMFPHP requests
Okay, right, if we're just using www.neopets.com (like we are for now), it fails on http://www.neopets.com because it triggers a redirect that we don't follow.

So here I 1) change the default to HTTPS, and 2) add HTTPS support to our little RocketAMF lib
2023-10-23 19:05:09 -07:00
1e690d9f6c Fix old find_or_initialize_by methods
I missed this in the Rails upgrades, oops!
2023-10-23 19:05:09 -07:00
0e7bbd526f Clarify the error behavior on AuthUser syncing
I added bangs to signify they're mutative operations, but also the bang on `create!` helps ensure we'll bail if User creation fails for some reason.
2023-10-23 19:05:08 -07:00
82be3bf5f9 Enable password resets
Nice, just turning it on seemed to do all we need for now!

Fair questions to be asked about like, should you be able to look up by username instead of email? But like idk, this feels simpler *and* more solid, to give you feedback on if it's the right email.
2023-10-23 19:05:08 -07:00
33b2953949 Enable basic AuthUser tracking
Simply turning back on the module that tracks sign-ins and IP addresses.
2023-10-23 19:05:08 -07:00
e79428fa28 Add Remember Me to login
This requires a migration, our first migration against the openneo_id database from this app! Fun!
2023-10-23 19:05:08 -07:00
d65aafdd4c Signup and settings page for OpenNeo ID accounts
Hey nice!!

Note that I removed an account delete button from the settings page. You can still send a DELETE request to the right endpoint to do it, but it's not gonna delete all the associated records, and I wanna think a bit about how to handle that better before exposing that button.
2023-10-23 19:05:07 -07:00
eee097a9f8 Sync AuthUser and User names
Callbacks are handy for this!
2023-10-23 19:05:07 -07:00
75185de957 Create a User when we create an AuthUser
We also update seeds.rb to be up to date! This should make it possible to log in as test/test123 from a fresh database.
2023-10-23 19:05:07 -07:00