1
0
Fork 0
forked from OpenNeo/impress
Commit graph

962 commits

Author SHA1 Message Date
96998643b5 Add manifest_url to swf_assets table
Ok so, impress-2020 guesses the manifest URL every time based on common
URL patterns. But the right way to do this is to read it from the
modeling data! But also, we don't have a great way to get the modeling
data directly. (Though as I write this, I guess we do have that
auto-modeling trick we use in the DTI 2020 codebase, I wonder if that
could work for this too?)

So anyway, in this change, we update the modeling code to save the
manifest URL, and also the migration includes a big block that attempts
to run impress-2020's manifest-guessing logic for every asset and save
the result!

It's uhh. Not fast. It runs at about 1 asset per second (a lot of these
aren't cache hits), and sometimes stalls out. And we have >600k assets,
so the estimated wall time is uhh. Seven days?

I think there's something we could do here around like, concurrent
execution? Though tbqh with the nature of the slowness being seemingly
about hitting the slow underlying images.neopets.com server, I don't
actually have a lot of faith that concurrency would actually be faster?

I also think it could be sensible to like… extract this from the
migration, and run it as a script to infer missing manifest URLs. That
would be easier to run in chunks and resume if something goes wrong.
Cuz like, I think my reasoning here was that backfilling this data was
part of the migration process… but the thing is, this migration can't
reliably get a manifest for everything (both cuz it depends on an
external service and cuz not everything has one), so it's a perfectly
valid migration to just leave the column as null for all the rows to
start, and fill this in later. I wish I'd written it like that!

But anyway, I'm just running this for now, and taking a break for the
night. Maybe later I'll come around and extract this into a separate
task to just try this on all assets missing manifests instead!
2023-11-09 21:42:51 -08:00
3a963c7d25 Fix old .scoped -> .all Rails upgrade bug
Ahh, I guess I missed these, I think they're maybe not actually used in
the app is why? cuz they're all default values that are overridden at
the actual call sites. But I ran into it when running `Pet.load` in the
console, and yeah let's just fix 'em up!
2023-11-09 21:35:42 -08:00
c9dd0f7376 Remove "Import from pets" page
This hasn't worked for a while, and I don't know an API off the top of
my head to drop in for it. Let's just delete it for now, and revisit it
later if we want to!
2023-11-06 13:06:18 -08:00
e9034fb085 Fix petpage etc import
Dang, I'm really wishing I'd opened this sooner cuz I didn't realize it
would be THIS easy!!

The bug was that the `t` method started taking Ruby keyword params
instead of a hash object for `options`, so the syntax changed.
Womp womp!
2023-11-06 12:59:28 -08:00
562cc33045 Fix closet list petpage export
I uhhh don't know why this ever worked at all lmao. Simple fix, I'd
been avoiding it for a while assuming it was worse lol!
2023-11-06 12:55:03 -08:00
47ea796af4 Fix outfit saving infinite loop in error case
Really don't know why this wasn't a problem with Apollo (or was it??),
but yeah, don't save when there's a save error!! Then we reset the
mutation state when the outfit state changes.
2023-11-06 12:54:23 -08:00
f8bcf5a0de Don't save the outfit while it's already saving
It's weird to be reading this code and be like. was this not always an
issue? Maybe something in Apollo prevented this? Did we use optimistic
UI or something? Idk?

There's still an issue with it infinitely retrying in an error state
though.
2023-11-06 12:38:38 -08:00
4ffd85ade0 Add mini-survey announcement
Just sharing this out to gather info, since this might be coming kinda
soon!

I also moved the announcement higher up in the template, because it
gets broken on the user lists page which uses floats quite a bit for
the site header—and tbh I feel like this is better anyway lol.
2023-11-06 12:12:25 -08:00
2126cefdd4 Add posted-at date to Pardon Our Dust page
Just to like. clarify how up-to-date it is and isn't!
2023-11-06 11:54:56 -08:00
57e262a884 Remove unused initialCacheState code from wardrobe-2020
In the impress-2020 app, we use this to prepopulate certain GraphQL
data into the Apollo cache when SSR'ing a page. We don't do that here,
so, goodbye!
2023-11-03 18:16:57 -07:00
a18ffb22a7 Remove the item page drawer, just link to the item page instead
The wardrobe-2020 app had a cute drawer that embeds the item page, but
honestly I don't think it was that valuable, and especially not when it
means we have to basically maintain two item pages lol. Let's decrease
the surface area!
2023-11-03 16:56:51 -07:00
a2feee2d9b Add support for is_manually_nc
A really really simple change! It works on the item page, the item
index page, item search, the homepage, and the item lists page.

The main reason I avoided this for so long (even before modernizing the
Rails app) was that the ElasticSearch stuff felt like it made it messy?
But now it's pretty simple, and it works in search already cuz I did
that when I implemented item search, so, nice!
2023-11-03 16:27:39 -07:00
5dcb1dedb4 Add Owls values to the item page
Eyy it's time!! Long-requested, finally here lol
2023-11-03 16:20:02 -07:00
494f82601f Set up eslint for wardrobe-2020
Ok cool, I have just not been running any of this since moving out of
impress-2020, but now that we're doing serious JS work in here it's time
to turn it back on!!

1. Install eslint and the plugins we use
2. Set up a `yarn lint` command
3. Set up a git hook via husky to lint on pre-commit
4. Fix/disable all the lint errors!
2023-11-02 18:11:07 -07:00
629706a182 Use the main app for outfit deletion, too 2023-11-02 17:39:26 -07:00
d32c6459b0 Normalize outfit data as we load it into wardrobe-2020
Rather than letting the fact that the server API models outfits a bit
differently (underscore keys, integer IDs for things), I'd rather
convert it to the familiar field names and expected types!
2023-11-02 17:12:59 -07:00
7a3aa609ba Use the main app for outfit saving, not impress-2020
This came in a few parts!
1. Add meta tags to let us know we're logged in.
2. Install React Query, which has the data-loading sensibilities I like
   about Apollo without the GraphQL that has honestly been a drag.
3. Replace the outfit-loading and outfit-saving calls with API calls to
   the main app.
4. Update the main app's API calls to use our more flexible data
   constructs like "pose".

Would've loved to do this more incrementally, but it's hard to! You
can't split out outfit-loading and outfit-saving, or auth from any of
that, or the state gets all out-of-sorts.

Still, this is a good nugget we've pulled out all-in-all, and one that
people have been asking for! Can maybe look to logged-in item search
soon too, for own/want data?
2023-11-02 16:54:35 -07:00
1ab3f21917 Finally remove unused auth0 code
Moreover, this code is in a bit of a flimsy state anyway: it'll kinda
work if you're logged in at impress-2020.openneo.net, but that wasn't
intentionally engineered, and I'm not sure exactly what circumstances
cause those cookies to send vs not send?

My intent is to clean up to replace all this with login code that
references the main app instead… this'll require swapping out some
endpoints to refer to what the main app has, but I'm hoping that's not
so tricky to do, since the main app does offer basically all of this
functionality already anyway. (That's not to say it's simple, but I
think it's a migration in the right direction!)
2023-11-01 15:26:53 -07:00
7948974949 Do preloading manually on user list pages, to reduce memory usage
I used the new profiler tools on this page, and noticed a lot of
allocations in the Globalize library, which we use for translating
database records. I realized that we were loading all of the fields of
not just all of the items on the page, but all of their translation
records in all locales! We used to scrape data for lots of languages, so
that can be quite a lot!

Unfortunately, Rails's `includes` method to efficiently preload related
records always loads all fields, and simply can't be overridden.

So, in this change we write manual preloading code, to identify the
records we need, load them in big bulk queries, and assign them back to
the appropriate associations. Basically just what `includes` does, but
written out a bit more, to give us the chance to specify SELECT and
WHERE clauses!
2023-10-27 19:42:02 -07:00
c496e33c37 Add mini profiler to each page
It shows up in development always, and if you're logged in as Me
Specifically in production!

I'm using this to poke at memory usage for pages that seem suspicious.
I don't know why our app reliably grows so large in RAM, but my hunch is
that maybe there are some pages that just use a truly large amount to
begin with - and I've learned Ruby doesn't release memory back after
it's GC'd, it just grows the process and keeps the free space to itself
in its own heap!

So I'm just eyeing pages that I know *can* have a lot going on, and
seeing what I find!
2023-10-27 19:38:49 -07:00
3243a0fdd9 Remove unused Javascript utility libraries
Some of these just didn't have call sites anymore; the HTML5 shim still
did, but that URL is literally broken now lmao. Goodbye!
2023-10-25 16:24:50 -07:00
22e3f4240a Update most URLs to use HTTPS
I noticed we didn't have the little lock icon in the browser, and yeah
huh there's a lot of `http://` still floating around! Let's fix that!
2023-10-25 15:22:57 -07:00
ee3ffe8afe Delete unused item proxy class
We used to do this for weird clever caching tricks that I don't think
were actually very effective. We stopped using this a few months ago,
and now I'm finally cleaning up this supporting code!
2023-10-25 12:55:30 -07:00
ff1f3aa68c Delete unused localized_fragment_exist? method 2023-10-25 12:54:38 -07:00
8d2ed67d89 Delete unused SweeperController
Something about fragment expiration I guess!
2023-10-25 12:53:06 -07:00
4cd12f6132 Delete unused FragmentExpiration module 2023-10-25 12:51:16 -07:00
f8cf3fd701 Fix crash when searching for "is:nc"
Huh, Arel can *sometimes* handle just having an attribute stand in as
"X is true" in a condition, but sometimes gets upset about it. I guess
this changed in Rails since we recently wrote this?

Specifically, item search would crash on "is:nc" (but *not* "is:np"),
saying:

```
undefined method `fetch_attribute' for #<struct Arel::Attributes::Attribute relation=#<Arel::Table:0x0000000109a67110 @name="items", @klass=Item(…), @type_caster=#<ActiveRecord::TypeCaster::Map:0x0000000109a66e90 @klass=Item(…)>, @table_alias=nil>, name="is_manually_nc">
```

The traceback was a bit misleading (it happened at the part where we
merge all the scopes together), but that hinted to me that it working
with an attribute in a place where it expected a conditional. So I
converted the attribute in the `is_nc` scope to a conditional, and made
the matching change in `is_np`, and that fixed it! Ok phew!
2023-10-25 12:46:48 -07:00
8034e8689a Oops, fix silly bug on /outfits/new
I just plum forgot to put the title under `- if outfit`, oops!!
2023-10-24 19:10:05 -07:00
f5b45d2dc8 Delete unused roulette feature
It was a cute idea! But it's been broken for a long time now. Goodbye!
2023-10-24 19:05:18 -07:00
c06cea2a3c Remove slightly-broken edit outfit links from user outfits page
The URL anchors were getting like. double-encoded? The `closet[]` part
was encoding as `closet%255B%255D`. Maybe a thing in Rails, where you
need to mark them `html_safe` to insert them in a URL like that?

Well anyway, those URLs are redundant now, I just have it link straight
to the same outfit page as the big link!
2023-10-24 18:05:51 -07:00
5ba711a413 Delete outfits/show page, point to the editor instead
Now, like in DTI 2020, opening an outfit will go straight to the editor.

I'm not 100% on whether this is actually like. the superior behavior?
But I think it's good enough, and it's what the wardrobe-2020 code
expects, so let's just roll with it for now!
2023-10-24 18:02:18 -07:00
3d07d7f360 Use /outfits/1234 URLs in the editor
Ohh I see, I made a mistake converting this from Next.js routing. It's
not that we had a URL search parameter named `outfitId`; it's that if
you were coming from the `/outfits/:outfitId` route, it would use that!

I still haven't gotten the rest of the site to point that route to this
page, but I'll do that in a later change.
2023-10-24 17:29:07 -07:00
a983ac9053 Add compatibility with older-style outfit URLs
Notable things:
- We used to have the parameters in the hash (`#`) part of the URL.
- We used to use the key `outfit=123` instead of `outfitId=123`.

In this change, we add backwards-compatibility for these things, while
still keeping the latest behavior too, with no change to the URLs we
generate!
2023-10-24 16:50:07 -07:00
e300b2d342 Run Prettier on all wardrobe-2020 JS
Looks like the version of Prettier I just installed is v3, whereas our
last run in the impress-2020 repo was with v2. I don't think we had any
special config in that project, I think these are just changes to
Prettier's defaults, and I'm comfortable accepting them! (Mostly seems
like a lot of trailing commas.)
2023-10-24 16:45:49 -07:00
b8a8cb9b20 Stop orphaning hangers when deleting lists
Idk if this used to be different or what, but it looks like the current
behavior is: if you delete a closet list, it'll leave the hangers
present, but Classic DTI would not show them anywhere; but Impress 2020
(until recently) would crash about it.

Now, we use `dependent: :destroy` to delete the hangers when you delete
the list (which I think makes sense, and is different than what I
decided in the past but that's ok, and is what the current behavior
*looks* like to people!), and we add a migration that deletes orphaned
hangers.

The migration also outputs the deleted hangers as JSON, for us to hold
onto in case we made a mistake! I'm also backing up the database in
advance of running this migration, just in case we gotta roll back HARD!
2023-10-24 15:35:21 -07:00
ddfdd5fc11 Add outfit layers info modal to download PNGs etc
This is an important workflow for people doing art stuff, I'm told! They used to use the Classic DTI broken image UI for this, but now that that's uhh Fully Gone, let's add this more explicitly!
2023-10-23 19:05:10 -07:00
c5edd20b30 Add outfit context menu, with Download button
A better affordance than the popup telling people not to do this lol

and I'm planning to maybe add the button for SWF etc info here too!
2023-10-23 19:05:10 -07:00
35aaaeba8e Fix Chakra UI styles in portal elements
Ah right, the CSS reset only applies in the ScopedCSSReset container, which doesn't work for elements portaled out with the <Portal> component (which a LOT of Chakra components use for things like tooltips etc).

Here, we take advantage of <Portal> having a hardcoded classname .chakra-portal, and applying it to them too!
2023-10-23 19:05:10 -07:00
2c98c1c636 Remove the now-unused pets#submit action
This was used by the Neopia server to send us the modeling data it requested out-of-band. But now we do all our modeling requests back in-app again, so we don't need this!
2023-10-23 19:05:10 -07:00
d8b0bf3174 Skip item translations when modeling
Okay, this is a process that idk if it's even been working for a while anyway, I don't think Neopets translates item names anymore?

And it's crashing when I try to model stuff now, so like. yeah ok I'm fine with just skipping this, it's a shame to lose out on potential data going forward but *I think there just isn't data to get anyway*
2023-10-23 19:05:10 -07:00
5f3ce1210a Stop saving local copies of SWFs
I think we used this for both conversion to image, and also for CORS stuff when rendering Flash-based previews… let's trash it, I don't want to be growing our hard drive with files I don't think we use anymore!

If I'm wrong and it turns out we do use them for something, then like. hey I'm sure we'll find out soon enough, and it's very recoverable operation.
2023-10-23 19:05:10 -07:00
2e152735c5 Stop referencing Neopia, just do modeling inline
I hope this doesn't cause problems! But yeah, with Puma doing threading, and maybe switching to Falcon someday to get even better concurrency properties, I feel like this will probably be fine?

And it makes the UX a loootttt better, to be back in the world where all these forms just work, whew.
2023-10-23 19:05:10 -07:00
e00ee08ae7 Revert timeout back to 1
Oh okay, I was misinterpreting the error: it was that our NEOPETS_URL_ORIGIN secret value isn't the real Neopets.com IP address anymore, so amfphp requests were just plain *always* failing in production. Oops!

I've remove that environment variable from our production config, and now modeling is working in the bulk thing!

Also I'm noticing that we're using puma these days, which does good threading stuff. I think there might be merit to switching over to Falcon because of just how async-y our stuff is, but having 5 threads going is honestly probably good enough that I don't need to worry too much about mutual blocking, and could probably just write stuff to get Neopia out of the picture like *right now*. Neat!
2023-10-23 19:05:10 -07:00
b885ff6ac0 Increase timeout on modeling
Okay so… I'm worried about this because of Rails whole single-threaded situation, which doesn't really let it handle blocking on external network requests very well.

Ultimately I think we're gonna have to do a clever thing but idk quite what?

I should look into whether like, puma + the new async stuff can enable Rails to be more tolerable about this, and handle a few requests at once, instead of having to have the Neopia server doing it. (Right now, the Neopia server isn't really doing its job quite right, because it depends on the Rails app being *local* to send stuff to it.)

But for now, let's just extend the timeout, cuz it's basically always getting hit in production—because there's currently no other way to do modeling, oops lol
2023-10-23 19:05:10 -07:00
966a0025e5 Fix SWF downloads in modeling
A lot simpler with latest Ruby! We can drop the whole SSL workaround yaay
2023-10-23 19:05:10 -07:00
468caea4ae Oops, fix typo include -> includes 2023-10-23 19:05:09 -07:00
3398439bae Fix more modeling bugs
Just find_all_by's that I never cleaned up

Oddly enough, I still got a "neopets seems down" message out of this, idk if that's an actual bug or just sluggishness rn
2023-10-23 19:05:09 -07:00
cbad00f32d Fix AMFPHP requests
Okay, right, if we're just using www.neopets.com (like we are for now), it fails on http://www.neopets.com because it triggers a redirect that we don't follow.

So here I 1) change the default to HTTPS, and 2) add HTTPS support to our little RocketAMF lib
2023-10-23 19:05:09 -07:00
a21ae014ef Fix crash when rendering modeling errors
Oh right, you can't return text from a format.json, but text can be json!
2023-10-23 19:05:09 -07:00
1e690d9f6c Fix old find_or_initialize_by methods
I missed this in the Rails upgrades, oops!
2023-10-23 19:05:09 -07:00