Commit graph

21 commits

Author SHA1 Message Date
7607c2c015 Oops, fix sloppiness about pet service refactor
I guess I like super didn't test this end-to-end, oops!!
2024-10-18 18:14:01 -07:00
abfe1e6df7 Extract neopets_origin into a config value 2024-10-18 18:00:48 -07:00
e36e273d50 Extract Neopets::CustomPets service from the Pet class
Just getting this stuff out of Pet, in part because I want to start
being able to unit test modeling, and that will require stubbing out
what this service returns!
2024-10-18 17:40:31 -07:00
acb52cb870 Move NCMall and NeoPass services into a Neopets module
Just a bit more clarity of grouping! I'm also thinking about extracting
modeling APIs into a service file like this too, in which case I think
this would help clarify what it is.
2024-10-18 17:27:15 -07:00
d056a5e766 Oops, don't show not-directly-for-sale items as being "0 NC"
"Fall Woodland Leaves Filter" is an example, it's part of the two-item
*pack* named "Fall Woodland Minitheus Petpet Foreground". The NC Mall
page for it will include the secondary items in `object_data`, but it's
not part of the storefront itself—and the only thing indicating that is
the `render` list.

Theoretically, we could use this to construct more data about like,
packs and stuff, automatically? But also, I don't want to backfill it
for everything historically, so like. Whatever.
2024-09-27 18:27:12 -07:00
be560e4595 Upgrade async and related gems, and fix async-http response handling
When playing with a Rainbow Pool syncing task, I noticed that error
handling wasn't working correctly for requests using `async-http`: if
the block raised an error, the `Sync` block would never return.

My suspicion is that this is because we were never reading or releasing
the request body.

In this change, I upgrade all the relevant gems for good measure, and
switch to using the response object yielded by the _block_, so we can
know it's being resource-managed correctly. Now, failures raise errors
as expected!

(I tested all these relevant service calls, too!)
2024-09-07 12:14:12 -07:00
46d3325144 Load *all* NC Mall pages in nc_mall:sync
Ta da! Now I can run this and pull 481 records into our database, and
then turn around and run it again and have them all correctly say
"skipped"!
2024-05-10 17:39:40 -07:00
b6e18e10a5 Add bare-bones rails nc_mall:sync task, incl. NCMallRecord model
Currently we only load the homepage, so there's only actually one
wearable item to sync up! But here's the task to do it!

To do this, we also created the backing model NCMallRecord, where we'll
save the current NC Mall state!
2024-05-07 17:40:14 -07:00
1f157b49da Load additional pages via NC Mall scraper service
This is for URLs like this! https://ncmall.neopets.com/mall/ajax/load_page.phtml?type=browse&cat=43&lang=en
2024-05-07 17:38:48 -07:00
7b0b6b70d2 Initial NC Mall scraper service
This doesn't connect to anything yet, I'm just doing the beginnings of
loading NC Mall item data!

My intent is to run this regularly to keep our own NC info in the
database too, primarily for use in the Item Getting Guide. (Could be
useful to surface in other places too though!) This will help us split
items into those that can be one-click purchased with the NC Mall
integration, vs NC items that need to be acquired by other means.
2024-05-07 16:06:37 -07:00
644b181ed0 Use Neopets username as base name for new NeoPass accounts, if possible
Yay, we got the API endpoint for this! The `linkage` scope is the key.

Rather than pulling back the specific fallback behavior we had wrote
for usernames before, which was slightly different and involved
appending `neopass` in there too (e.g. `matchu-neopass-1234`), I
figured let's just use a lot of the same logic, and just use the
preferred name as the base name. (I figure the `neopass` suffix isn't
that useful anyway, `matchu-1234` kinda looks better tbh! And it's all
fallback stuff that I expect serious users to replace, anyway.)
2024-04-09 07:48:13 -07:00
9ed34fa042 Add User-Agent header to our OwlsValueGuide requests
Note: I validated this was working by temporarily changing the URI to
`https://echo.free.beeceptor.com`, which echoes the headers back, then
called `OwlsValueGuide.load_itemdata` directly.
2024-04-09 06:59:44 -07:00
eb5f2a020c Add User-Agent header to our NeopetsMediaArchive requests
Note: I validated this was working by temporarily changing the URI to
`https://echo.free.beeceptor.com`, which echoes the headers back, then
called `NeopetsMediaArchive.load_file_from_origin` directly.
2024-04-09 06:58:03 -07:00
2cac048158 Save manifest load info when preloading them, too
This was a bit tricky! When I initially turned it on, running
`rails swf_assets:manifests:load` would trigger database errors of "oh
no we can't get a connection from the pool!", because too many records
were trying to concurrently save at once.

So now, we give ourselves the ability to say `save_changes: false`, and
then save them all in one batch after! That way, we're still saving by
default in the edge cases where we're downloading and saving a manifest
on the fly, but batching them in cases where we're likely to be dealing
with a lot of them!
2024-02-25 16:02:36 -08:00
a684c915a9 Track when manifest was last loaded, and what status it returned
Now we're *really* duplicating with Impress 2020's system lol, but I
need a way to not keep trying to load manifests that are actually 404,
which are surprisingly plentiful!

This doesn't actually stop us from loading anything yet, it just tracks
the timestamps and the HTTP status! But next I'll add logic to skip
when it was 4xx recently.
2024-02-25 15:35:04 -08:00
9a3b33ea2f Preload many manifests concurrently for the Alt Styles page
I'm gonna also use this for a task to try to warm up *all* the
manifests in the database! But to start, just a simple one, to prepare
the alt styles page quickly on first run. (This doesn't really matter
in production now that I've already visited the page once, but it helps
when resetting things in dev, and I think more it's about establishing
the pattern!)
2024-02-23 13:45:12 -08:00
f6cece9a59 Fix inconsistent indentation in swf_assets.rake
My editor now flags this stuff better, thank you editor!
2024-02-23 13:12:21 -08:00
2cc46703b9 Create NeopetsMediaArchive, read the actual manifests for Alt Styles
The Neopets Media Archive is a service that mirrors `images.neopets.com`
over time! Right now we're starting by just loading manifests, and
using them to replace the hacks we used for determining the Alt Style
PNG and SVG URLs; but with time, I want to load *all* customization
media files, to have our own secondary file source that isn't dependent
on Neopets to always be up.

Impress 2020 already caches manifest files, but this strategy is
different in two ways:

1. We're using the filesystem rather than a database column. (That is,
   manifest data is kinda duplicated in the system right now!) This is
   because I intend to go in a more file-y way long-term anyway, to
   load more than just the manifests.
2. Impress 2020 guesses at the manifest URLs by pattern, and reloads
   them on a regular basis. Instead, we use the modeling system: when
   TNT changes the URL of a manifest by appending a new `?v=` query
   string to it, this system will consider it a new URL, and will load
   the new copy accordingly.

Fun fact, I actually have been prototyping some of this stuff in a side
project I'd named `impress-media-server`! It's a little Sinatra app
that indeed *does* save all the files needed for customization, and can
generate lightweight lil preview iframes and images pretty easily. I
had initially been planning this as a separate service, but after
thinking over the arch a bit, I think it'll go smoother to just give
the main app all the same access and awareness—and I wrote it all in
Ruby and plain HTML/JS/CSS, so it should be pretty easy to port over
bit-by-bit!

Anyway, only Alt Styles use this for now, but my motivation is to be
able to use more-correct asset URL logic to be able to finally swap
over wardrobe-2020's item search to impress.openneo.net's item search
API endpoint—which will get "Items You Own" searches working again, and
whittle down one of the last big things Impress 2020 can do that the
main app can't. Let's see how it goes!
2024-02-23 12:02:39 -08:00
e991eda308 Fix minor indentation inconsistency
I recently tweaked my editor settings to make stuff like this more
obvious lol, but those are spaces in tab-indented file
2024-02-18 20:41:42 -08:00
6d343989de Extend OWLS caching from 15 minutes to 1 day
Poking at the site, I didn't love that clicking any of the NC items on
the homepage took a few seconds to load cuz of the OWLS request taking
a couple seconds.

If the request were faster to load I might decide otherwise, but for
now, let's just keep that cache long.

Another approach we could take would be to ask for an endpoint that
returns all the values at once, so we could cache and update them all
together, instead of having it all split into separate cache keys!
2024-01-15 06:19:48 -08:00
5dcb1dedb4 Add Owls values to the item page
Eyy it's time!! Long-requested, finally here lol
2023-11-03 16:20:02 -07:00