Commit graph

2050 commits

Author SHA1 Message Date
27f21ab775 Delete unused test directory
Until we're actually using the little helpers and scaffolding code,
let's just not have the clutter!
2024-01-18 21:42:20 -08:00
cc2d2906e4 Remove VS Code settings
I don't want to propagate VS Code too much, and this is a trivial file,
goodbye!
2024-01-18 21:37:19 -08:00
6d343989de Extend OWLS caching from 15 minutes to 1 day
Poking at the site, I didn't love that clicking any of the NC items on
the homepage took a few seconds to load cuz of the OWLS request taking
a couple seconds.

If the request were faster to load I might decide otherwise, but for
now, let's just keep that cache long.

Another approach we could take would be to ask for an endpoint that
returns all the values at once, so we could cache and update them all
together, instead of having it all split into separate cache keys!
2024-01-15 06:19:48 -08:00
e337767954 Fix "couldn't download the associated Flash files" when modeling
So this was a slightly wrong error message, what was happening was:

1. Trying to load the image hash for this pet, by looking them up at
   https://pets.neopets.com/cpn/PET_NAME/1/1.png and seeing what URL it
   redirects to.
2. But pets.neopets.com was rejecting our User-Agent string, which
   would've been just "Ruby", since we hadn't set it otherwise. I guess
   that's an explicitly banned string?

I also found that the kind of more-helpful User-Agent string I like to
write was being rejected, and I could only get it to accept something
very simple? So that's what we're using now, I guess!!
2024-01-15 01:15:35 -08:00
03f38c6461 Upgrade to Yarn 4.0.2
Unlike previous attempts to do future-Yarn, this seemed to just mostly
go swimmingly, idk why! (I moved to a new computer and installed new
Yarn without really thinking, and hey nice!)

There was an issue where it was trying to do some glob expansion in
one of the arguments for the `build` script, fixed! (Idk why Yarn has
its own glob expansion and different from previous versions, but ok!)
2024-01-14 23:05:53 -08:00
d4e77b9ad9 Remove Pardon Our Dust message and page
I think it's served its purpose! I'll do more updates in another venue
sometime probably!
2024-01-14 23:00:23 -08:00
2b382d95fb Update my desktop SSH key
I did a pretty thorough reset of my desktop machine, and rather than go
spelunking for the same private key, I just rolled it over to a new
one. Let's set it up!
2024-01-14 03:07:12 -08:00
a656cd511a Create endpoint to show all species appearances for a given color
Building toward replacing more of the 2020 data sources! I think this is
an endpoint that benefits from bulk loading, esp with the way the item
page previews work. I also like taking the concept of "canonical" out of
the GQL interface, and instead just loading for each of the 50 species
and letting the client decide. (And then it can fast-swap between them!)
2023-12-05 18:04:54 -08:00
c8de3dae63 Delete unused PetAttribute model files
Oh right, this powered `Color` and `Species` back before they were
database rows. Now these base classes are unused! Goodbye!
2023-12-05 14:56:34 -08:00
99f5cd717f Move some model mixins to lib/
The models folder is a bit confusingly large, these are more mixins and
kinda clutter it. Push them off into `lib`, I think!

I think they used to be in models mainly because Rails used to handle
`lib` differently with autoloading, and it made for a worse dev
experience. Now it's all the same, though!
2023-12-05 14:55:31 -08:00
af5373c390 Add analytics.openneo.net tags
Self-hosted Plausible instance! I have need of usage numbers again,
after a good few years of just not using it; but I don't want to send
the data to Google, and I enjoy self-hosting things, so here we have it!
2023-12-04 22:44:16 -08:00
29c4e4f3ac Remove mini-survey link
That's honestly a bit longer than I even meant to leave it up, but I've
been sick! Time to close shop and start tallying it all up!
2023-11-20 16:30:20 -08:00
f93944d401 Oops, fix the pet preview on the homepage
Oh right, I forgot we use the `pet_types#show` endpoint there! Welcome
back lol!
2023-11-11 16:12:38 -08:00
75a23dfc57 Delete more unused helpers 2023-11-11 15:55:38 -08:00
fb4e967ea1 Delete unused helper 2023-11-11 15:50:07 -08:00
f9669195ea Remove old feedback UserVoice link 2023-11-11 15:48:05 -08:00
c725a11920 Remove unused NC Mall spider code & fields
We haven't used the mall spider in this app in forever (I guess we even
deleted the code at some point?), but there was some vestigial stuff
left. Goodbye!
2023-11-11 15:45:38 -08:00
94d15f005d Delete a couple unused accessors
At least, I think they are!
2023-11-11 15:42:08 -08:00
644a6acc72 Remove old Image Mode stuff
There was a static page explaining it, which we no longer link to; and
there was an unused field in the User model for who was a beta tester
for it. Goodbye!
2023-11-11 15:21:49 -08:00
c960e831bc Reorganize routes.rb
No changes intended here, just grouping and commenting!
2023-11-11 15:18:01 -08:00
4926996e8e Oh, another unused route
I deleted the unused controller but forgot the route!
2023-11-11 15:13:47 -08:00
e22809deb3 Disable some unused Rails features
I noticed when running `rails routes` that there's a lot of routes for
major unused Rails features, like storage. I didn't look deeply enough
into ActiveStorage to know if I was risking accepting arbitrary file
uploads, I just figured, if I disable it (which simplifies the app
footprint anyway), then I can be certain! So, goodbye!
2023-11-11 15:12:41 -08:00
8f226cd129 Delete more unused routes
Truly don't know what these are, they crash when I go to them so!
2023-11-11 15:11:46 -08:00
5905f4aed1 Delete more unused code
Just some routes and controllers that I think were at various points
serving as API endpoints, but no longer do I think!
2023-11-11 15:04:18 -08:00
aae5c767bb Oops, fix species name in zones tooltip
Oh right, this didn't show up in testing bc I only have a few models in
the local db, but yeah I messed this up a bit!!
2023-11-11 11:57:43 -08:00
f56e1ad426 Merge branch 'item-appearances' 2023-11-11 11:24:46 -08:00
0845881aba Add TIMEOUT parameter to swf_assets:manifests task
At this point, I've gone through all the assets, and the only ones
without manifests are:

1. The ones that truly have no manifest yet (that we know of)
2. The ones where execution happened to time out

I think the 5-second timeout is a very reasonable default for starting
the backfill, in a way that prioritizes moving forward; but now that we
have most things, I'd rather be able to re-run it with a more generous
timeout. So here we are!
2023-11-11 11:04:53 -08:00
b54be0ab5c Load item name from Rails app in item preview page
This is just a silly lil single-field migration!
2023-11-11 10:58:14 -08:00
a31039c4c8 Load item page restricted zones data from Rails app, not impress-2020
Just moving more stuff over! I modernized Item's `as_json` method while
I was here. (Note that I removed the NC/own/want fields, because I
think the only other place this method is still called from is the
quick-add feature on the closet lists page, and I think it doesn't use
these fields to do anything: updating the page is basically a full-page
reload, done sneakily.)
2023-11-11 08:49:19 -08:00
c6cb61ef38 Remove no-op item.thumbnail.secure_url
There was a time when I used an old proxy server to try to fix mixed
content issues, and I eventually removed it but never took the tendrils
out from the code.

We probably _should_ figure out how to secure these URLs! But until
then, we may as well simplify the code.
2023-11-11 08:24:08 -08:00
5e25e0bda6 Load item compatibility data from the Rails app, not impress-2020
Ok progress! We've moved the info about what bodies this fits, and
what zones it occupies, into a Rails API endpoint that we now load at
the same time as the other data!

We'll keep moving more over, too!
2023-11-11 08:15:10 -08:00
fe6264b20a Use an actual body with ID "0", instead of the string "all"
I changed my mind again! At first I wanted to make the special case
clearer, and to be able to more strongly assert that the species is
not null. But now I'm like… eh, there's code that references `body.id`
that has no reason _not_ to work in the all-bodies case… let's just
keep the types more consistent, I think.
2023-11-11 08:13:07 -08:00
eaab0b1922 Oops, return an array in the all-bodies case
I was returning just the body itself, oops! For consistency with the
general return value, it should be a one-element array.
2023-11-11 07:58:50 -08:00
133895b213 Refactor appearances data to use a "body" object
This is more similar to what impress-2020 does, I was working on the
wardrobe-2020 code and took some inspiration!

The body has an ID and a species, or is the string "all".
2023-11-11 07:54:56 -08:00
6c31094c2d Add known_glitches to appearances JSON 2023-11-11 07:21:13 -08:00
010f64264f Replace swf_assets#index with item_appearances#index
Preparing a better endpoint for wardrobe-2020 to use! I deleted the
now-unused swf_assets#index endpoint, and replaced it with an
"appearances" concept that isn't exactly reflected in the database
models but is a _lot_ easier for clients to work with imo.

Note that this was a big part of the motivation for the recent
`manifest_url` work—in this draft, I'm probably gonna have the client
request the manifest, rather than use impress-2020's trick of caching
it in the database! There's a bit of a perf penalty, but I think that's
a simpler starting point, and I have a hunch I'll be able to make up
the perf difference once we have the impress-media-server managing more
of these responsibilities.
2023-11-11 07:14:48 -08:00
3b59823fc4 Fix minor item search error message bug
Oops, I got the variable name wrong here! Before this change,
`species:foo` would crash. Now, it shows the error message correctly.
2023-11-11 07:04:57 -08:00
d2de971a60 Delete more rake tasks
I tried to port the Rainbow Pool ones forward, but ran into issues with the
service that uses browser-specific stuff to check that traffic is valid :/

Incidentally, those were the only places we were using `rest-client`.
Goodbye!
2023-11-10 18:59:46 -08:00
93511b3d51 Delete unused rake tasks 2023-11-10 18:04:24 -08:00
eb4a3ce0d9 More gracefully handle batches that fail to save
I noticed a thing with like, an asset that I think referenced an item that
doesn't exist, which caused an error in the `body_specific?` validation
step?

Tbh that validation step needs fixed up in a number of ways, but I'm
scared to, since it's hard to know what will break modeling lol.

But in any case, more graceful handling is nice! If something happens,
I'd rather leave it null and try again later than have the job crash!
2023-11-10 17:42:56 -08:00
80bd229bc6 Clarify an error message in swf_assets:manifests task
It's not just that none of them were 200 OK, it's that they were all 404.
In the event that something returns not-200 and not-404, we immediately
abort, so we shouldn't get to this case unless they were all 404!
2023-11-10 17:27:35 -08:00
dc22a458bf Move manifest backfill to swf_assets:manifests task
Okay, I've simplified the migration to *just* add the column, and
instead added a task to find assets without manifest URLs and backfill
them.

Performance is a lot better now, using the `async-http` library, which
as I understand it supports both persistent connections when invoked
like this, and maybe also HTTP/2 multiplexing?? (Though I'm not
actually sure images.neopets.com does lol)

I'm not sure about the number of concurrent tasks I picked here, 100
seems okay for an internet thing and for such small requests, but I
worry that the CDN is gonna get annoyed or something. Well, we'll see!
This task is very resumable if it turns out we get frozen out or
something.
2023-11-10 16:52:50 -08:00
d35b10c1b8 Oops, fix validation bug for SwfAsset
Oh right, saving an asset that doesn't have a manifest will crash this
validation step!
2023-11-10 16:16:32 -08:00
96998643b5 Add manifest_url to swf_assets table
Ok so, impress-2020 guesses the manifest URL every time based on common
URL patterns. But the right way to do this is to read it from the
modeling data! But also, we don't have a great way to get the modeling
data directly. (Though as I write this, I guess we do have that
auto-modeling trick we use in the DTI 2020 codebase, I wonder if that
could work for this too?)

So anyway, in this change, we update the modeling code to save the
manifest URL, and also the migration includes a big block that attempts
to run impress-2020's manifest-guessing logic for every asset and save
the result!

It's uhh. Not fast. It runs at about 1 asset per second (a lot of these
aren't cache hits), and sometimes stalls out. And we have >600k assets,
so the estimated wall time is uhh. Seven days?

I think there's something we could do here around like, concurrent
execution? Though tbqh with the nature of the slowness being seemingly
about hitting the slow underlying images.neopets.com server, I don't
actually have a lot of faith that concurrency would actually be faster?

I also think it could be sensible to like… extract this from the
migration, and run it as a script to infer missing manifest URLs. That
would be easier to run in chunks and resume if something goes wrong.
Cuz like, I think my reasoning here was that backfilling this data was
part of the migration process… but the thing is, this migration can't
reliably get a manifest for everything (both cuz it depends on an
external service and cuz not everything has one), so it's a perfectly
valid migration to just leave the column as null for all the rows to
start, and fill this in later. I wish I'd written it like that!

But anyway, I'm just running this for now, and taking a break for the
night. Maybe later I'll come around and extract this into a separate
task to just try this on all assets missing manifests instead!
2023-11-09 21:42:51 -08:00
3a963c7d25 Fix old .scoped -> .all Rails upgrade bug
Ahh, I guess I missed these, I think they're maybe not actually used in
the app is why? cuz they're all default values that are overridden at
the actual call sites. But I ran into it when running `Pet.load` in the
console, and yeah let's just fix 'em up!
2023-11-09 21:35:42 -08:00
c9dd0f7376 Remove "Import from pets" page
This hasn't worked for a while, and I don't know an API off the top of
my head to drop in for it. Let's just delete it for now, and revisit it
later if we want to!
2023-11-06 13:06:18 -08:00
e9034fb085 Fix petpage etc import
Dang, I'm really wishing I'd opened this sooner cuz I didn't realize it
would be THIS easy!!

The bug was that the `t` method started taking Ruby keyword params
instead of a hash object for `options`, so the syntax changed.
Womp womp!
2023-11-06 12:59:28 -08:00
562cc33045 Fix closet list petpage export
I uhhh don't know why this ever worked at all lmao. Simple fix, I'd
been avoiding it for a while assuming it was worse lol!
2023-11-06 12:55:03 -08:00
47ea796af4 Fix outfit saving infinite loop in error case
Really don't know why this wasn't a problem with Apollo (or was it??),
but yeah, don't save when there's a save error!! Then we reset the
mutation state when the outfit state changes.
2023-11-06 12:54:23 -08:00
f8bcf5a0de Don't save the outfit while it's already saving
It's weird to be reading this code and be like. was this not always an
issue? Maybe something in Apollo prevented this? Did we use optimistic
UI or something? Idk?

There's still an issue with it infinitely retrying in an error state
though.
2023-11-06 12:38:38 -08:00