Commit graph

216 commits

Author SHA1 Message Date
61a4dcad02 Oops, "fits:blue-acara" search should return non-body-specific items
Idk when this regressed exactly, but probably people didn't super
notice because I don't think it's a very common thing to type directly
into the Infinite Closet search box! (It used to be crucial to the old
wardrobe app.)

But I'm using it in the wardrobe app again now, so, fixed!
2024-02-25 14:47:37 -08:00
52e81557c2 Update search filters to consider NP and PB mutually exclusive
`is:np` now means "is not NC and is not PB".

Note that it might be good to make NC and PB explicitly mutually
exclusive too? It would complicate queries though, and not matter in
most cases… the Burlap Usul Bow is the only item that we currently
return for `is:pb is:nc`, which is probably because of a rarity issue?
2024-02-25 12:57:04 -08:00
a3dcaa0f0e Add useItemSearch for wardrobe app, but don't use it yet!
Adding new functionality to the item search JSON endpoint, and adding
an adapter layer to match the GQL format!

Hopefully this will be pretty drop-in-able, we'll see!
2024-02-25 12:06:32 -08:00
cd095eefcb Oops, fix crashing bug when loading item appearances endpoint
I missed this because it's only for backgrounds and stuff! But yeah I
added a prefix to the `Body` struct, and forgot to apply it everywhere,
oops!
2024-02-24 16:14:30 -08:00
3f449310d6 Refactor item search JSON, add appearances
Preparing to finally move wardrobe-2020's item search to use the main
app's API endpoints instead!

One blocker I forgot about here: Impress 2020 has actual support for
knowing an item's true appearance, like by reading the manifest and
stuff, that we haven't really ported over. I feel like maybe I should
pause and work on the changes to manifest-archiving that I'd been
planning anyway? I'll think about it.
2024-02-23 10:44:50 -08:00
1e6ff4cefc Remove the Item::Translation model entirely
Okay, Impress 2020 is migrated off translations too, so we can start to
wrap this up!
2024-02-20 16:57:45 -08:00
3ac9e7ce69 Migrate item search away from item translations
Lightning fast for simple name queries now, gotta say!!
2024-02-20 16:04:41 -08:00
5ee3b472ec Migrate away from item translations in modeling
This one is important, I didn't notice that this is a way of setting
attributes that won't be written to both tables! `name` will only be
written to the translation table (which crashes the save), and the
other fields would only be written to the main table. Fixed! (I don't
like the super-dynamic this code was written before, anyway.)
2024-02-20 15:52:03 -08:00
0e8f457aa1 Oops, fix bug on item page now that translations aren't available
Missed this at first - now that the `name` field is just a normal field
and is always English, it's now an error to provide the locale to it as
a parameter, like we used to for the translated version of the field!
2024-02-20 15:37:07 -08:00
a1066d9c8a Add translated item fields directly to the Item model
Like with Species, Color, and Zone, we're moving the translation data
directly onto the model, and just using English. This will simplify
some of our queries a lot (way fewer joins!), and it's what Neopets
does now anyway, and I have a secret hope that removing the complexity
along the codepath for `item.name` might help speed up large item lists
if we're lucky?? 🤞

Anyway, this is the first step, performing the migration to copy the
data onto the `items` table, making sure to keep them in sync for the
2020 app for now!
2024-02-20 15:25:03 -08:00
1b22258576 Update special_color logic for Banana Chia Wings case
Tbh I'm not sure `special_color` is actually used anywhere? It used to
be how we decide what to show in the previewer on the item page, but
that's been replaced with the 2020 logic, so idk…

But in any case, I noticed that the description doesn't match the
pattern we have, so here's the fix!
2024-02-17 12:50:35 -08:00
73af9d4d77 Start migrating off globalize gem for zones
Like in 0dca538, this is preliminary work for being able to drop the
`zone_translations` table! We're copying the field over first, to be
able to migrate DTI 2020 safely before dropping anything.
2024-01-23 05:43:00 -08:00
ce64f12cc3 Remove references to species/color translations at call sites
We wisely did preloading in some places. Now, we don't need to!
Hopefully this'll even be a free speed boost on some pages!
2024-01-23 05:23:57 -08:00
b1c1bea7be Remove unused data in items#show controller
Oh right, this was for the old pet previewer. Goodbye!
2024-01-20 23:53:30 -08:00
c725a11920 Remove unused NC Mall spider code & fields
We haven't used the mall spider in this app in forever (I guess we even
deleted the code at some point?), but there was some vestigial stuff
left. Goodbye!
2023-11-11 15:45:38 -08:00
a31039c4c8 Load item page restricted zones data from Rails app, not impress-2020
Just moving more stuff over! I modernized Item's `as_json` method while
I was here. (Note that I removed the NC/own/want fields, because I
think the only other place this method is still called from is the
quick-add feature on the closet lists page, and I think it doesn't use
these fields to do anything: updating the page is basically a full-page
reload, done sneakily.)
2023-11-11 08:49:19 -08:00
c6cb61ef38 Remove no-op item.thumbnail.secure_url
There was a time when I used an old proxy server to try to fix mixed
content issues, and I eventually removed it but never took the tendrils
out from the code.

We probably _should_ figure out how to secure these URLs! But until
then, we may as well simplify the code.
2023-11-11 08:24:08 -08:00
fe6264b20a Use an actual body with ID "0", instead of the string "all"
I changed my mind again! At first I wanted to make the special case
clearer, and to be able to more strongly assert that the species is
not null. But now I'm like… eh, there's code that references `body.id`
that has no reason _not_ to work in the all-bodies case… let's just
keep the types more consistent, I think.
2023-11-11 08:13:07 -08:00
eaab0b1922 Oops, return an array in the all-bodies case
I was returning just the body itself, oops! For consistency with the
general return value, it should be a one-element array.
2023-11-11 07:58:50 -08:00
133895b213 Refactor appearances data to use a "body" object
This is more similar to what impress-2020 does, I was working on the
wardrobe-2020 code and took some inspiration!

The body has an ID and a species, or is the string "all".
2023-11-11 07:54:56 -08:00
010f64264f Replace swf_assets#index with item_appearances#index
Preparing a better endpoint for wardrobe-2020 to use! I deleted the
now-unused swf_assets#index endpoint, and replaced it with an
"appearances" concept that isn't exactly reflected in the database
models but is a _lot_ easier for clients to work with imo.

Note that this was a big part of the motivation for the recent
`manifest_url` work—in this draft, I'm probably gonna have the client
request the manifest, rather than use impress-2020's trick of caching
it in the database! There's a bit of a perf penalty, but I think that's
a simpler starting point, and I have a hunch I'll be able to make up
the perf difference once we have the impress-media-server managing more
of these responsibilities.
2023-11-11 07:14:48 -08:00
3a963c7d25 Fix old .scoped -> .all Rails upgrade bug
Ahh, I guess I missed these, I think they're maybe not actually used in
the app is why? cuz they're all default values that are overridden at
the actual call sites. But I ran into it when running `Pet.load` in the
console, and yeah let's just fix 'em up!
2023-11-09 21:35:42 -08:00
a2feee2d9b Add support for is_manually_nc
A really really simple change! It works on the item page, the item
index page, item search, the homepage, and the item lists page.

The main reason I avoided this for so long (even before modernizing the
Rails app) was that the ElasticSearch stuff felt like it made it messy?
But now it's pretty simple, and it works in search already cuz I did
that when I implemented item search, so, nice!
2023-11-03 16:27:39 -07:00
5dcb1dedb4 Add Owls values to the item page
Eyy it's time!! Long-requested, finally here lol
2023-11-03 16:20:02 -07:00
ee3ffe8afe Delete unused item proxy class
We used to do this for weird clever caching tricks that I don't think
were actually very effective. We stopped using this a few months ago,
and now I'm finally cleaning up this supporting code!
2023-10-25 12:55:30 -07:00
f8cf3fd701 Fix crash when searching for "is:nc"
Huh, Arel can *sometimes* handle just having an attribute stand in as
"X is true" in a condition, but sometimes gets upset about it. I guess
this changed in Rails since we recently wrote this?

Specifically, item search would crash on "is:nc" (but *not* "is:np"),
saying:

```
undefined method `fetch_attribute' for #<struct Arel::Attributes::Attribute relation=#<Arel::Table:0x0000000109a67110 @name="items", @klass=Item(…), @type_caster=#<ActiveRecord::TypeCaster::Map:0x0000000109a66e90 @klass=Item(…)>, @table_alias=nil>, name="is_manually_nc">
```

The traceback was a bit misleading (it happened at the part where we
merge all the scopes together), but that hinted to me that it working
with an attribute in a place where it expected a conditional. So I
converted the attribute in the `is_nc` scope to a conditional, and made
the matching change in `is_np`, and that fixed it! Ok phew!
2023-10-25 12:46:48 -07:00
468caea4ae Oops, fix typo include -> includes 2023-10-23 19:05:09 -07:00
3398439bae Fix more modeling bugs
Just find_all_by's that I never cleaned up

Oddly enough, I still got a "neopets seems down" message out of this, idk if that's an actual bug or just sluggishness rn
2023-10-23 19:05:09 -07:00
0e4cc80ac8 Remove sanitize_sql_like monkey-patch
I had added this many Rails versions ago during the recent upgrade process, because it was in latest Rails but not in the version of Rails I was using when replacing Elasticsearch with MySQL queries. We can remove it now!
2023-10-23 19:05:07 -07:00
e8131f3608 Fix bug with Arel ordering
Ok so weird little situation, usually Arel will accept an attribute as a param to `order()`, but not when it's in a very specific situation of all of the following:

`Item.joins(:translations).includes(:translations).limit(30).order(Item::Translation.arel_table[:name])`

For some reason, it's all like "hey I can't call `to_sql` on an attribute!", but only in the scenario where all 3 of those other things are present. Weird!

Anyway, explicitly saying `.asc` fixes this. Ok!
2023-10-23 19:05:05 -07:00
d97c32b5da Upgrade to Rails 5.2.8.1
Some important little upgrades but mostly straightforward!

Note that there's still a known issue where item searches crash, I was hoping that this was a bug in Rails 4.2 that would be fixed on upgading to 5, but nope, oh well!

Also uhh I just got a bit silly and didn't actually mean to go all the way to 5.2 in one go, I had meant to start at 5.0… but tbh the 5.1 and 5.2 changes seem small, and this seems to be working, so. Yeah ok let's roll!
2023-10-23 19:05:05 -07:00
Matchu
fd263ea82f Remove mall spider cron jobs
I don't think these work anymore, and our volunteers get new items into the db fast anyway, Impress 2020 is doing better spidering these days. And then we get to remove the cron job `whenever` gem!
2023-10-23 19:05:05 -07:00
Matchu
3e728a8e56 Oops, fix ambiguous zones_restrict column
When combining occupies and restricts filters, this wouldn't be sure whether to apply to the items or the swf assets.
2023-10-23 19:05:04 -07:00
Matchu
e1b17e05be Add fits and not_fits back to item search
Some fun stuff here to figure out how to API this out well, but I'm pretty pleased with where it ended up!
2023-10-23 19:05:03 -07:00
Matchu
62f1d883af Improve not_occupied solution
Not being a subquery is better! I realized later that a LEFT JOIN would probably do it even betterer? with like `HAVING count(x) = 0`? but the `left_outer_joins` method doesn't seem to be in Rails 4, and I don't want to do stringy joins, so this is fine for now!
2023-10-23 19:05:03 -07:00
Matchu
c581b063c4 Oops, fix not_occupies logic error
Right, previously we were querying "has *at least one asset* that is not in zone X" instead of "has NO assets that are in zone X".

I don't know a fast way to query for that, this will have to do for now!
2023-10-23 19:05:03 -07:00
Matchu
b244057808 Add restricts filter back to item search 2023-10-23 19:05:03 -07:00
Matchu
d952685c5d Add occupies filter back to item search
Mostly adapting what was already there!
2023-10-23 19:05:03 -07:00
Matchu
185b4eb2fc Use arel for item translation joins
Just a bit defensive so we aren't setting up the possibility of an ambiguous query someday!
2023-10-23 19:05:03 -07:00
Matchu
a653b0c20d Add is:pb back to item search 2023-10-23 19:05:03 -07:00
Matchu
3e3aa6a126 Move Item name search logic to model scope 2023-10-23 19:05:03 -07:00
Matchu
72461972ca Start building new item search
Just name field right now, more to come! A lot deleted lol
2023-10-23 19:05:03 -07:00
Matchu
a29e016555 Update to new scope syntax
Ohh ok, without this change all of our `scope`s were just immediately evaluating the argument and fetching _all_ such matching records immediately, instead of waiting to actually be called. This led to bugs like `pet_type.as_json` returning ALL pet states in the whole db, because the `PetState.emotion_order` scope was being treated as a single predefined query, rather than a query fragment to merge into the current context.

This also explains what happened in 724ed83: that's why things before the scope in the query were being ignored.
2023-10-23 19:05:03 -07:00
Matchu
f0f9033b8f Fix select bug in Item.all_by_ids_or_children
Idk why, but when the `select` was the first thing in the query, it was getting ignored. I wonder if there's something about the `object_assets` scope that I'm not understanding that's overwriting it? Or the `joins`? But whatever, this works, I'm not worried about it for now!
2023-10-23 19:05:02 -07:00
Matchu
aa3d28c641 Update syntax for set_inheritance_column 2023-10-23 19:05:02 -07:00
Matchu
b35c773be3 Remove flex from the app (breaks search!)
We'll need to replace the item search query stuff with direct MySQL queries, but that's not ready yet bc the app still isn't booting, so we're committing this in a known broken state for now!
2023-10-23 19:05:02 -07:00
Matt Dunn-Rankin
c7d60e3e41 Add db field to override special color
Okay, fine, finally making this controllable from the db without requiring a deploy :P Setting this new field will cause `item.special_color` to return the corresponding color. This mainly affects what we show on the item page, and what colors we request for modeling on the homepage.
2018-05-09 14:10:20 -07:00
Matt Dunn-Rankin
711aca0008 hardcode more baby IDs :P
Interestingly, these items *are* correctly detecting their special
color on the homepage for model progress. So, we *do* have the ability
to detect this. But I don't have good item data locally, so it would
be hard to test this, so I'm just gonna go with the cheap solution
again, sorry XP
2018-02-25 14:45:13 -08:00
Matt Dunn-Rankin
80e74da84f whoops, re-add check for explicitly_body_specific?
In bfd825d, we refactored the "is item body-specific?" check. In the process, we dropped the check for the manual override flag, `explicitly_body_specific?`. Not sure if it was an accident or if I was just _so_ confident that it was gonna work :P In any case, re-add the check!
2017-10-29 15:08:22 -07:00
Matt Dunn-Rankin
bd3b75c675 hardcode some color-specific items 😬 2017-10-21 13:32:01 -07:00
Matt Dunn-Rankin
daa831e2b0 show gear image when no thumbnail URL present
Okay, surprise, the bug was unrelated to Camo config (though I'm glad I cleaned
that up anyway :P). We now, at a low level, serve a placeholder image for item
thumbnail URL if, for some reason, we don't have a good thumbnail URL on hand.
2017-04-01 10:29:12 -07:00
Matchu
b11d7a8c9c oh dang, did we just fix most of the mixed content? 2015-08-05 20:11:08 -04:00
Matchu
c5c587fab1 stop crashing when trying to translate item names 2015-07-27 13:23:46 -04:00
9c6399a23d Baby Raindorf Hoodie Robe is annoying. special case it as baby-fitting in the worst hack possible ;P 2014-12-17 00:54:55 -06:00
0fe31ee79a basic fits functionality in search 2014-04-05 17:43:54 -05:00
03c76fe882 Update missing body ID prediction to handle, say, the Maraquan Mynci.
It turns out that some pets for seemingly nonstandard colors have the
standard body type anyway, and vice-versa. This implies that we should
stop relying on a color's standardness, but, for the time being, we've
just revised the prediction model:

Old model:
    * If I see a body_id, I find the corresponding color_ids, and it's wearable
      by all pet types with those color_ids.

New model:
    * If I see a body_id,
        * If it also belongs to a basic pet type, it's a standard body ID.
            * It therefore fits all pet types of standard color (if there's
              more than one body ID modeled already). (Not really,
              because of weird exceptions like Orange Chia. Should that be
              standard or not?)
        * If it doesn't also belong to a basic pet type, it's a nonstandard
          body ID.
            * It therefore only belongs to one color, and therefore the item
              fits all pet types of the same color.
2014-01-20 15:29:01 -06:00
99b2acd419 attach body id to newest unmodeled item species names 2014-01-10 16:25:03 -05:00
9a4e114964 oh yum, this is really starting to come together :) 2014-01-10 16:25:02 -05:00
7c6e607612 basic neopia api integration 2014-01-10 16:25:02 -05:00
728ff60c5f move item cache sweeping and flex syncing to background tasks 2013-12-09 00:12:05 -06:00
f07996d762 cache pet images on items#show, in case that's what's being a super-slow jerkface 2013-12-05 15:22:43 -06:00
cc7ac363dd WIP commit for speeding up item show pages 2013-12-05 13:27:56 -06:00
0cb7fc87df include zones_restrict in item selector when mall spidering, to avoid flex_source errors 2013-10-08 14:42:46 -05:00
082119afe1 fix some mall spider bugs, including not having all the attributes it needed for search indexing 2013-07-09 21:00:36 -07:00
9bd49aa85d first step in repairing mall spider 2013-07-09 20:01:55 -07:00
5b9394ce82 oops - don't cache as_json's owned/wanted, but instead have the proxy override 2013-06-27 00:10:55 -07:00
9e3cac82ec use proxies for item html, too
Some lame benchmarking on my box, dev, cache classes, many items:

No proxies:
    Fresh JSON:  175,  90,  90,  93,  82, 88, 158, 150, 85, 167 = 117.8
    Cached JSON: (none)
    Fresh HTML:  371, 327, 355, 328, 322, 346 = 341.5
    Cached HTML: 173, 123, 175, 187, 171, 179 = 168

Proxies:
    Fresh JSON:  175, 183, 269, 219, 195, 178 = 203.17
    Cached JSON:  88,  70,  89, 162,  80,  77 = 94.3
    Fresh HTML:  494, 381, 350, 334, 451, 372 = 397
    Cached HTML: 176, 170, 104, 101, 111, 116 = 129.7

So, overhead is significant, but the gains when cached (and that should be
all the time, since we currently have 0 evictions) are definitely worth
it. Worth pushing, and probably putting some future effort into reducing
overhead.

On production (again, lame), items#index was consistently averaging
73-74ms when super healthy, and 82ms when pets#index was being louder
than usual. For reference is all. This will probably perform
significantly worse at first (in JSON, anyway, since HTML is already
mostly cached), so it might be worth briefly warming the cache after
pushing.
2013-06-26 23:50:19 -07:00
e42de795dd Use item proxies for JSON caching
That is, once we get our list of IDs from the search engine, only
fetch records whose JSON we don't already have cached.

It's simpler here to use as_json, but it'd probably be even faster
if I figure out how to serve a plain JSON string from a Rails
controller. In the meantime, requests of entirely cached items
are coming in at about 85ms on average on my box (dev, cache
classes, many items), about 10ms better than the last
iteration.
2013-06-26 23:01:12 -07:00
fb219f82e8 sigh, add another special color description format 2013-06-23 22:58:17 -07:00
a80f70bb88 phew, fixed some issues with contribution saving in rails 3.2 2013-03-05 20:51:24 -06:00
5d618237a9 fix bug in determining special color
Specifically, the Tyrannian Meerca Spear is a pb item that contains
"pea", so its item page is only willing to show a Pea Chia. Now,
a color must be a whole word in the item name for special color
determination to work.
2013-02-03 13:31:22 -06:00
629ff2c45d index items by *actual* species support ids 2013-01-28 17:54:50 -06:00
f484fc5c21 update supported species list on items#show 2013-01-28 16:54:25 -06:00
206811a2fb oops, deal with missing item fields better 2013-01-28 03:01:25 -06:00
0b32e8ba59 cut down on pets#load zone queries 2013-01-28 02:19:37 -06:00
2798ebbd5c fix pb filter for other locales 2013-01-26 09:52:21 -06:00
26ac3782ec move zones to database 2013-01-24 18:26:00 -06:00
ce4e2fd53c move species to database 2013-01-24 18:25:59 -06:00
965465ca51 move species to database 2013-01-24 18:25:59 -06:00
94ecc6f02d remove old item search interface 2013-01-24 18:24:35 -06:00
6e09b8bc10 globalized search first draft
Confirmed features:
    * Output (retrieval, sorting, etc.)
    * Name (positive and negative, but new behavior)
    * Flags (positive and negative)

Planned features:
    * users:owns, user:wants

Known issues:
    * Sets are broken
        * Don't render properly
        * Shouldn't actually be done as joined sets, anyway, since
          we actually want (set1_zone1 OR set1_zone2) AND
          (set2_zone1 OR set2_zone2), which will require breaking
          it into multiple terms queries.
    * Name has regressed: ignores phrases, doesn't require *all*
      words. While we're breaking sets into multiple queries,
      maybe we'll do something similar for name. In fact, we
      really kinda have to if we're gonna keep sorting by name,
      since "straw hat" returns all hats. Eww.
2013-01-24 18:24:35 -06:00
c9ae7155b1 locale metadata, including hidden locales for item loading and selection 2013-01-24 18:24:34 -06:00
ef2423e87f globalize3 for items 2013-01-24 18:24:34 -06:00
29ed401238 load pet by locale, prepare needed translations 2013-01-24 18:24:34 -06:00
9c8f7f498f basic translation infrastructure for items - search is still english-only, and probably will have to be replaced with a legit search engine 2013-01-24 18:24:34 -06:00
d79c225c1e apply globalize3 to items 2013-01-24 18:24:34 -06:00
bfd825d98e attempt to fix new species_support_ids format. sigh. 2013-01-23 00:25:09 -06:00
9701221035 wardrobe now considers item.species_support_ids when deciding compatibility
For example, the Meerca Maid Tray is a foreground item, so the SWF is marked
as compatible with all body types, but the item itself is clearly marked as
Meercas-only. items#show reflected this properly, but the swf_assets#index
call that the wardrobe uses ignored item.species_support_ids.

So, /bodies/:body_id/swf_assets.json?item_ids[]=... was deprecated in favor
of /pet_types/:pet_type_id/items/swf_assets.json?item_ids=[]..., which is
much like the former route but, before loading assets, also loads the pet
type and items, then filters the items by compatibility, then only loads
assets for the compatible items.
2013-01-02 23:15:32 -05:00
f56b544963 brought-to-you-by on items#show lists contributors 2012-10-24 22:09:05 -05:00
671641cc16 a more forgiving "type" search filter 2012-10-08 21:20:18 -05:00
775ef7fa51 finally fix encased in ice - woo! 2012-10-05 20:56:52 -05:00
ddec043209 support pea chia cape in infinite closet 2012-09-29 12:40:55 -05:00
1e3938eea9 improve closet performance by caching item link 2012-08-09 19:34:56 -04:00
72237f225c modeling hub 2012-08-06 21:15:31 -04:00
4d314417e2 fix parent-swf-asset-relationship destruction bug
After changing the database structure, we lost the feature where, once we discover
new assets for an item for a given body ID, we disconnect previously connected
assets. This commit reinstates that feature.
2012-01-26 13:51:30 -06:00
ec3088fdec ensure that pet state is saved before trying to save its assets 2012-01-13 15:56:31 -06:00
171d691a98 fix nc mall spider for remote ID 2012-01-13 15:27:30 -06:00
ec9e997ac5 fix user:owns id ambiguity 2012-01-13 15:20:47 -06:00
9c0c7b78cf another oops, better fix 2012-01-12 22:02:12 -06:00
c2c0fe92e8 oops, be consistent in using remote ID when loading pets 2012-01-12 21:47:17 -06:00