Added prominent warnings in multiple locations to prevent accidental
database migrations that would break Impress 2020:
README.md:
- Added critical warning in "OpenNeo ID Database" section
- Highlighted that Impress 2020 directly accesses both databases
- Added warning in "Deployment" section about schema compatibility
- Linked to detailed documentation
docs/impress-2020-dependencies.md:
- Clarified both databases are directly accessed by Impress 2020
- Added new "Database Consolidation Blocker" section
- Documented that consolidation migration is ready but blocked
- Provided options to unblock (retire I2020 or coordinated deployment)
This ensures future developers (including future me!) are aware of this
critical dependency before proposing database schema changes.
Related: feature/consolidate-auth-database branch contains a ready-to-go
database consolidation, but it's blocked on Impress 2020 retirement.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Added critical information about database consolidation blocker:
- Impress 2020 directly accesses BOTH openneo_impress and openneo_id databases
- Any database consolidation must wait until Impress 2020 is retired
- Database migration is ready on feature/consolidate-auth-database branch
but is blocked on this dependency
This ensures we don't accidentally deploy the database consolidation
before addressing the Impress 2020 dependency, which would break
authentication for users accessing the site through Impress 2020.
Updated sections:
- Deployment Architecture: Clarified both databases are accessed by I2020
- After Full Migration: Noted database consolidation is ready but blocked
- Added new "Database Consolidation Blocker" section with details
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
First actual feature I'm letting Claude run! We worked the exploration of the updated API together, then it ran with the implementation.
I left this hanging for a long time.... good to finally have it updated!
I'm not sure if this is a Mac-only problem or what, but we were getting incompatible-function-pointer errors when trying to build the RocketAMF C extensions. This fixes that! (Maybe it's like, Mac-only but as of Ruby 3.4 in specific? We're running RocketAMF in production on Ruby 3.4 right now without this. Shrug.)
I'm starting to learn how AI agent stuff works, and a lot of what I'm finding is that rushing them into feature development sets you up for disaster, but that having strong collaboration conversations with helpful context works wonders.
So, I'm starting by creating that context: I had a little "here's the codebase" walkthrough conversation with Claude Code, and it generated these docs as output—which came out solid from the jump, with a few tweaks from me for improved nuance.
My hope is that this can serve both as an improved starting point for human collaborators _and_ if I let future Claude instances play around in here. That's a big theme of what I've found with AI tools so far: don't try to get clever, don't expect the world, just give them the same support you'd give people—and then everybody wins 🤞
Just because I'm poking at archiving again, I'm noticing failures when trying to run the `rails swf_assets:manifests:load` job, something to do with an "unacquired resource" with the connection.
Maybe this is a bug that an update will fix? May as well try!
Oops, I didn't realize that the MySQL function `SUBSTRING_INDEX` always
returns the full string if the split delimiter isn't found.
This meant that, for series names like "Regal", we read the main name as
"Regal" (correct) and the variant name as "Regal" (incorrect).
This caused sort order to be incorrect for some series, e.g.,
- Prismatic Dawn: Regal
- Prismatic Dusk: Regal
- Prismatic Mirage: Regal
- Regal
whereas the main series name is meant to be first, and *does* come first
in cases like "Festive" where the main name sorts before any of the
variant names!
In this change, we update the variant name definition to return an empty
string. That way, when there's no variant name and it's just the main
series, that one sorts to the top of the series variants.
They made a small change to the API call! Fixed now!
The other NC Mall syncing stuff to get current NC cost is broken, but
that seems like a MUCH bigger change, so I won't bother just yet.
Just a small visual cleanup because I happened to click on item trades
today! We don't need to repeat "This Week" a million times. Just output
it for the first row, then hide it for the following. (We still include
it in screen reader output for semantic clarity; this is just a visual
cleanup.)
Before this change, the "Ornamental Lake with Goldies" item would fail
to preview on the item page: the iframe for the animation layer would
display an error page.
The error was:
```
Invalid Content Security Policy script-src: "https://images.neopets.com/cp/items/data/000/000/497/497366_deca9f2827/497366_HTML5 Canvas.js". Directive values must not contain whitespace or semicolons. Please use multiple arguments or other directive methods instead. (ActionDispatch::ContentSecurityPolicy::InvalidDirectiveError)
```
This is because the URL that Neopets sends us for this JS file contains
an unescaped space character. This isn't usually an issue for e.g.
loading a URL in the browser, but it's *not* valid syntax for inclusion
in a Content Security Policy.
In this change, we update our CSP code to parse URLs into
`Addressable::URI` objects, which enables us to call the `normalize!`
method, which fixes oddities like that.
The URL now correctly appears in the CSP as
`https://images.neopets.com/cp/items/data/000/000/497/497366_deca9f2827/497366_HTML5%20Canvas.js`.
Neopets.com recently added some new security rules that, if not
satisfied, cause the request to return 403 Forbidden.
We figured these out through trial and error, and added them to the
`DTIRequests` library, so they would apply to all requests we make.
We also updated our AMFPHP library to use `DTIRequests` as well, as an
easy way to get the same security rules to apply to those requests.
This change was motivated by pet loading being down for the past day or
so, because all pet loading requests were returning 403 Forbidden! Now,
we've fixed it, hooray!
Sigh, the "Valentine Plushie" series is messing with me again! It
doesn't follow the previously established pattern of the names being
"<series> <color> <species>", because in this case the base color is
considered "Valentine".
Okay, well! In this change we add `full_name` as an explicit database
field, and set the previous full name value as a fallback. (We also
extract the generic fallback logic into `ApplicationRecord`, to help us
express it more concisely.)
We also tweak `adjective_name` to be able to shorten custom `full_name`
values, too. That way, in the outfit editor, the Styles options show
correct values like "Cherub Plushie" for the "Cherub Plushie Acara".
I also make some changes in the outfit editor to better accommodate the
longer series names, to try to better handle long words but also to
just only use the first word of the series main name anyway. (Currently,
all series main names are one word, except "Valentine Plushie" becomes
"Valentine".)
I don't know how reliably people actually know that we support NC
Styles, because the token items aren't searchable.
In this change, we add a little message to teach people about this
feature, if they search for words that match known styles, or the
colloquial word "token" (even though style token items aren't actually
called that on-site).
Previously, we took the incoming date as just an integer, which was
raising a database error when we attempted to save it to a timestamp
field. Now, we correctly parse it into a timestamp.
I imagine we just haven't run the import manually while there's a real
discount in the records before? Weird!
Idk a clear and concise way to group all these together, *and* include
`rspec-rails` in both development and test as the install instructions
recommend, but omit `webmock` from development.
Not that it matters a lot, I just want to be minimal about the gems we
include, for the sake of clarity.
This formulation is a bit verbose, but I think it's maximally clear and
self-consistent, and I appreciate that. I might change my mind later
though?
Anyway, I mostly just noticed this because of the redundancy where
`webmock`'s `group: :test` I think doesn't do anything? Shrug
Huh okay, looks like maybe the `protocol-http` or `protocol-http2` gem
made non-backwards-compatible changes that broke `async-http`. But
moving to the very latest `async-http` seems to fix it. Okay!
It's been a while, time to update all our minor/patch versions!
When I first ran this, the dev server stopped working entirely, saying:
```
NoMethodError: undefined method `ip_address' for an instance of Socket
```
I traced this back to what seems to be a bug introduced in
`protocol-rack` version `0.11.0`, so I modified the Gemfile to hold us
back from that version in particular. After that, everything is looking
good!
Now that some of these series names are very long (like "Prismatic
Cocoa: Festive"), they can throw off the layout. Let's just use the
shorter one, it's clear enough!
Previously, when opening the pose picker and looking at Styles, the
Cybunny options were sorted like this:
- Default
- Celebratory 25th Anniversary
- Festive Christmas
- Nostalgic Baby
- Nostalgic Blue
- Nostalgic Christmas
- Nostalgic Darigan
- Nostalgic Faerie
- Nostalgic Grey
- Nostalgic Maraquan
- Nostalgic Mutant
- Nostalgic Plushie
- Nostalgic Robot
- Nostalgic Royalboy
- Nostalgic Royalgirl
- Nostalgic Snow
- Nostalgic Tyrannian
- Prismatic Cocoa: Festive Christmas
- Prismatic Cocoa: Nostalgic Christmas
- Prismatic Tinsel: Festive Christmas
- Prismatic Tinsel: Nostalgic Christmas
- Spooky Halloween
Now, they're sorted like this:
- Default
- Celebratory 25th Anniversary
- Nostalgic Baby
- Nostalgic Blue
- Festive Christmas
- Prismatic Cocoa: Festive Christmas
- Prismatic Tinsel: Festive Christmas
- Nostalgic Christmas
- Prismatic Cocoa: Nostalgic Christmas
- Prismatic Tinsel: Nostalgic Christmas
- Nostalgic Darigan
- Nostalgic Faerie
- Nostalgic Grey
- Spooky Halloween
- Nostalgic Maraquan
- Nostalgic Mutant
- Nostalgic Plushie
- Nostalgic Robot
- Nostalgic Royalboy
- Nostalgic Royalgirl
- Nostalgic Snow
- Nostalgic Tyrannian
Note especially the Christmas case, which is all together now! I think
it's also more in line with people's expectations for Halloween to be
alphabetically among the rest, instead of being at the bottom for being
"Spooky".
There's enough styles now that I'm starting to wonder if there's other
UI affordances worth having here, like e.g. only showing (or at least
prioritizing) styles that match the chosen color? But I don't want to
mislead people about compatibility, either.
Oh, right, silly: my previous version of this change still grouped by
zone, then mapped the zones to their labels. This didn't *merge* the
lists of appearances for zones that share the same label; just one of
the zones would win, and the others would disappear.
In this change, I just go upstream and actually group them by label in
the first place, instead of grouping by zone then trying to merge and
transform them.
For example, the "Red Knitted Beanie with Wig" occupies two different
"Hat" zones: one for behind the head, and one for in front. It's not
useful to split them up!
Instead of offering a form to request a different format, we just
render both in the HTML, and use CSS to swap between the two. Love to
see the `:has` filter come through for us!
Before this change, the sort order when searching for
"Prismatic Pine: Nostalgic" showed:
- [Added Dec 18, 2024] Prismatic Pine: Nostalgic Christmas Flotsam
- [Added Dec 19, 2024] Prismatic Pine: Nostalgic Christmas Gelert
- [Added Dec 18, 2024] Prismatic Pine: Nostalgic Christmas Bruce
- [Added Dec 17, 2024] Prismatic Pine: Nostalgic Christmas Scorchio
- <more>
This is because the Gelert was created at 11:37 NST on Dec 19, whereas
the Flotsam was created at 18:11 NST on Dec 18—but in UTC, which is how
timestamps are stored in the database, these are both Dec 19, so the
Flotsam was sorting first alphabetically.
In this change, we do a hacky transform from UTC to NST-ish. I didn't
want to set up the deploy process to pull named time zones into MySQL,
and then have this as a potential gotcha for the dev environment
later—so instead, I pretend `-08:00` is a good-enough specification of
NST.
This will sometimes create slightly incorrect sort ordering when it
*is* Daylight Savings, and a record was created around midnight. I'm
okay with that!
When there's an unlabeled style, previously we'd include the placeholder
value "<New?>" in the search dropdown, even though we don't actually
support searching by it.
Now, we don't! I did this in part by just refactoring how we look this
stuff up, with queries that don't load *all* alt styles into memory,
which will help perf a bit as more of them are released.
I ran a bad version of this job yesterday, and set a lot of the new
styles to an incorrect "Prismatic" series name. This will help me fix
them!
There's still styles to manually fix, where TNT decided not to put the
full series name in the item name. (Presumably some of the names were
too long? Or maybe they just forgot?) But this is a better starting
point!
Ahh right, that's part of why I skipped Prismatics: it's no longer true
that the first word of the style is its series name.
In this change, I try to parse out everything before the pet name part
of the style name, and default to skipping it if we can't quite get it
right.
Oh oops, Firefox is a bit stricter about interpreting custom elements
as inline elements by default instead of block elements. This messes up
the positioning relative to the container. Fixed!
Ruby 3.3.6 has a warning for this, neat! I apparently didn't notice the
`path` option already there at first, and just added a new one.
This change shouldn't affect behavior, it just is clearer and removes a
warning!
A style not being modeled yet is not a big deal, whereas some of those
other warnings actually require manual intervention. I want to make the
list easier to scan for the warn icons!
Fedora upgraded its system Ruby, and I'm on that laptop today, and I
prefer to have prod keep pace rather than use rbenv to keep myself and
prod knowingly on an older version!
Just a wrapper for the barrier/semaphore thing we're doing constantly!
I only applied it in the importer rake tasks for now. There's other
call sites to move over to it, too!
Previously, "Then: Go to unlabeled appearance" would always take you to
the *first* unlabeled appearance in our database.
Now, we go to the *next* unlabeled appearance in the list, relative to
this one.
I've known there are bugs in the SVGs pretty often, because they're not
very well attended-to—I noticed pretty quick that the Marble Eyrie, for
example, has its Body asset saved correctly in PNG, but its SVG is just
another copy of the head, oops!
I think SVG is still a nice default for this UI, but I added a little
form to switch to PNG, to give us a debugging method and escape hatch
if it starts to get weird.
This washes out the colors a bit more than necessary on lighter pets,
but helps a lot on darker pets. It really kinda pushes everything other
than the lineart all the way to white, which tbh is a pretty neat
sketch-like effect.
I couldn't find a library for this functionality that didn't require
jQuery, and I don't want to be adding *more* jQuery requirements. So, I
decided to throw together my own!
The `<magic-magnifier>` component copies its contents into a "lens"
element, then uses basic JS to track mouse position, then uses CSS to
move the lens and its contents into a helpful position.
One thing I noticed here is that the zoom is a bit crunchy because
we're using PNG images, and it's hard to zoom in even further than we
already are. I might try switching this UI to use the SVG images by
default instead?
When you hover the row for a layer in the table, it highlights the
corresponding layer in the outfit viewer. And when you click anywhere
in the row, it opens the first link (usually the PNG image).
Sometimes I forget like, what the masc/fem variants of a given pet
actually look like? Some are super obvious about things like eyelashes,
and others use more subtle eye differences.
This is a cheap lil hack to make it easier to open a reference! Ideally
I think it would be neat to like, when you hover over an option, have
it show you the reference variant of that pose? But this is good enough
I think!
If the screen is narrow, many of the bubbles will wrap their text onto
two lines, but "Unconverted" won't. Give it equal height to the rest
anyway, for visual consistency!
We copy the same feature from alt styles, now that the UI is shared via
support form helpers! Easy peasy!
This adds a "Then: Go to unlabeled appearance" checkbox next to the
submit button on the pet appearance edit form. If checked, it takes you
to the first unlabeled appearance in the database, and keeps the box
checked for next time. Slam through 'em!
This helped me debug a thing in the upcoming change! It lets you drop a
`debugger` line into the app, then run `rdbg --attach` in another
terminal to get into a debug session. Neat!
Realizing that, with the keyword argument spread syntax, I don't need
to do merging, I can just. spread at the right place!
My rationale for the ordering here is: if the caller theoretically tried
to override the builder (even though I don't see why), I think we would
want to respect that. Whereas the `class` argument should be overridden
because we're safely *merging* our `.support-form` class into it.
I want to reuse this for unlabeled pet styles is why! (That's been the
immediate motivation for this refactor, but also I do just like that
it'll make support forms easier to build.)
I think helpers are fine for the simpler ones that are basically *just*
wrapper tags, but once it starts getting into `concat`, I think that's
too unfamiliar of a syntax for developers; let's bail into our usual
templating system!
I'm not sure about putting them in `application/support_form` like this.
That's cute for one-offs like `application/hanger_spinner`, because
`render partial: "hanger_spinner"` assumes the `application` view folder
by default, but that doesn't work once it's nested: it looks for a
`views/support_form` folder.
I think maybe it could soon be time to bail from the strict "view
folders belong to controllers" thing, similar to how we did for
`SupportFormHelper`, and add a `components` folder or similar? Idk, not
sure yet!
Ah right, `> label` doesn't work with how Rails will wrap broken labels
and inputs each in a `.field_with_errors` element. Fixed, and added
some basic coloring!
Instead of hand-rolling HTML, this offers helpers like `f.field`, to
help ensure the HTML is consistent, and to keep the templates more
focused on the unique form elements.
Most notable change here is extracting the pose option bubbles into a
`data-type="radio-grid"`, and pulling that into the `.support-form`
CSS. My rationale is that, unlike most fields, this field benefits from
being 100%-width, and I don't want to specify that as an override if I
can avoid it, because that's fragile-y.
Instead, I extract this into a generic type of field that
`.support-form` can use (it feels pretty reusable anyway!), and require
the caller to specify how many columns they want as `--num-columns`.
Specifically, I'm going for a more-vertical layout, cuz I want to bring
PetState over to it, and the weird grid situation wasn't gonna fit the
big pose label radios.
There's still plenty left, but we have 213 we "manually" marked as
"done" (I think I ran a batch job on everything Chips told me was on
the page and already done), and that should help a lot!
This keeps causing missing-attribute crashes when I change things, and
I don't think the performance benefit is a big deal for how the page
currently runs, esp as we keep gathering more attributes? I feel like
`description` is the main "large" one we're omitting, and like. Shrug!
Been running into the item "Hanging Plushie Grundo Background Item" not
being modelable, because TNT seem to have left its description blank!
Let's be less picky about what data we take in, but keep the intention
of these validations: to ensure that *we* don't make a mistake and
forget a field when importing items!
This is a basic attempt at the Vandagyre logic, but also things like
"Maraquan items released before the Maraquan X was released"!
I also added a new task, `rails items:update_cached_fields`, which needs
to be run after this change, because it affects the value of
`Item#predicted_fully_modeled?`.
Eyeballing the updated search results for `-is:modeled`, this feels
pretty close? I'm guessing it's not perfect (e.g. maybe a pet type we
got modeled late into its existence, or some items that just never did
fit a certain pet), but feels pretty good.
I also know we had the "modeling hints" override in Impress 2020, which
we aren't reading yet. We should probably take that into account here
too!
We're now caching `predicted_fully_modeled?` on the database record, so
we can query by it in the database!
I'm moving on from the model I did in Impress 2020, of writing really
big fancy single-source-of-truth queries based on the assets themselves.
I see the merit of that in terms of theoretical reliability, but in
practice I think it will be *more* reliable to have one *in-code*
definition of modeling status (which we need anyway for generating the
homepage modeling requests), and just save that in a queryable way.
In our tests, I discovered an unexpected behavior where calling
`item.swf_assets << swf_asset` wasn't updating computed fields
correctly.
This isn't something we actually do in-app, I think the modeling system
happened to trigger the callbacks in a way that still worked fine?
But I think this is a good idea for reliability, since caching is such
a notoriously difficult thing to get right anyway! And it makes our
tests simpler and clearer.
Specifically, `compatible_body_ids` references `swf_assets`, which, I'm
kinda surprised, *doesn't* include the newly-added asset yet when the
`ParentSwfAssetRelationship.after_save` hook runs while calling
`item.swf_assets << swf_asset`. Reloading it fixes this!
I'm grouping some shared behaviors to pull into the different cases, so
that we can check the behaviors of a fully-modeled item vs a
not-fully-modeled item in *all* of the relevant cases.
Specifically, I'm planning to add `is:modeled` search filters, and
creating pending placeholder tests for them!
This doesn't generally happen, but did the other day when I rolled back
some of the database's SWF asset records but kept the items—and it was
a bit confusing that the homepage marked them as fully modeled!
The main thing is that I was getting "RequireNotFound" warnings for
`require 'rails_helper'`, because the LSP seems unaware of how RSpec
offers `spec/` as a root for requires.
I think the `require_relative` is clearer anyway, I'm decently
satisfied with it. And if I decide it's too much ugly, we can try
something else in the Solargraph config or something sometime!
and with a `rails neopets:import` task you can call to do them all at
once!
I'm gonna do some other stuff here too to make `neopets:import` easier
to call all in one go, like prompting for the Neologin cookie just
once at the start.
Note that this changes the cron setup, so you gotta run
`bin/deploy:setup` after this deploys!
I'm running into this with the automated tests and the fixtures I think
sometimes using large auto-generated IDs?
But the point is, our tables generally use Rails's default `:integer`
size for its IDs, and then columns that reference them are *smaller*,
which is… not correct stuff, y'know?
So I figure, let's just expand the columns. We don't have enough data
that being real picky about the integer sizes matters, so let's keep it
simple and more obviously correct.
Oh huh, when doing Rainbow Pool stuff, I put the ordering in the wrong
place! It's a sensible ordering for the Rainbow Pool page, but not so
much for the JSON view!
This is currently crashing the Rainbow Pool when the Anniversary Techo
would appear, because the asset seems to be missing? The SWF doesn't
seem to exist, nor does its manifest.
Oh right, yeah, we like to do things gracefully around here when
there's no corresponding color/species record yet!
Paying more attention to this, I'm thinking like… it could be a cool
idea to, in modeling, *create* the new color/species record, and just
not have all the attributes filled in yet? Especially now that we're
less dependent on attributes like `standard` to be set for correct
functioning.
But for now, we follow the same strategy we do elsewhere in the app: a
pet type can have `color_id` and `species_id` that don't correspond to
a real record, and we cover over that smoothly.
Oh dang, we're on color #120 now, and looks like our maximum value is
127. Let's expand that!
I noticed this because I'm writing tests for some stuff, and used "456"
as a placeholder ID number, and it just fully did not work, and I'm
like. Oh.
Huh, I guess when I reapplied my refactors to modeling disabling the
other day, I didn't notice that I turned it off in production. And I
guess I didn't deploy this at the time cuz it's just refactors, but
when I deployed other changes yesterday this came with it. Whoops!
I only now thought through that I can scrape these instead of enter
them manually, similar to how we did our Rainbow Pool scraper… hooray!
I'm actually writing tests for stuff too, wowie!
This change was modified a bit after cherry-picking, to no longer
include the broken changes to item modeling in 9eaee4a.
(cherry picked from commit 90407403ba)
Okay so, when we reverted a buncha stuff in e3d196f, it was in response
to a bug where item modeling data was getting deleted. And I was tired,
and just took a big simple hammer to it of reverting all the modeling
refactors.
Here, we reintroduce *some* of them: the biology ones before the item
bug. And tests still pass, and in fact I can un-pending some of them!
I might also try to reapply the change where we extract it all into a
new file, but without the item parts.
```shell
git cherry-pick --no-commit 13ceec8fcc
git cherry-pick --no-commit f81415d327
git cherry-pick --no-commit c03e7446e3
git cherry-pick --no-commit 52ca41dbff
```
Also, while we're here! To restore the lost data, I:
1. Downloaded this scheduled public data backup, which was taken
thankfully the day before we updated modeling code!
https://impress.openneo.net/public-data/2024-11-03T08_15_02Z-scheduled.sql.gz
2. Trimmed it just to the section about the `parents_swf_assets` table:
dropping it, then rebuilding it from scratch.
3. Ran this modified backup SQL dump on the production server.
4. Ran the code from `db/migrate/20241001052510_add_cached_fields_to_items.rb`
to bring items' cached fields back into the correct state.
I also had to fix some errors in the item data that prevented some
items from passing the latest validations:
```rb
Item.where(rarity: "").update_all(rarity: "???")
Item.where(description: "").update_all(description: "???")
Item.where(zones_restrict: "").update_all(zones_restrict: "00000
00000000000000000000000000000000000000000000000")
```
Because we ended up with such a big error, and it doesn't have an easy
fix, I'm wrapping up today by reverting the entire set of refactors
we've done lately, so modeling in production can continue while we
improve this code further over time.
I generated this commit by hand-picking the refactor-y commits
recently, running `git revert --no-commit <hash>` in reverse order,
then manually updating `pet_spec.rb` to reflect the state of the code:
passing the most important behavioral tests, but no longer passing one
of the kinds of annoyances I *did* fix in the new code.
```shell
git revert --no-commit 48c1a58df9
git revert --no-commit 42e7eabdd8
git revert --no-commit d82c7f817a
git revert --no-commit 5264947608
git revert --no-commit 90407403ba
git revert --no-commit 242b85470d
git revert --no-commit 9eaee4a2d4
git revert --no-commit 52ca41dbff
git revert --no-commit c03e7446e3
git revert --no-commit f81415d327
git revert --no-commit 13ceec8fcc
```
As I'm writing out my solution for this, I'm almost wondering if it's
time for the refactor I've been Theoretically Planning Someday, to move
items to a real `ItemAppearance` model in the database similar to
`PetState`… Hmm hmm hmm…
For now though, I'm taking a break!
This bug never made it into production I think, it was a consequence of
some of how I refactored stuff in the recent changes? I think??
But yeah, I refactor how we manage `SwfAsset#body_id`, to be a bit more
explicit about when and how it can change, instead of the weird
callbacks that tbqh have bit us too often…
Ah right, the callbacks in `ParentSwfAssetRelationship` don't get
called when Rails does automatic join-model management stuff. We need
the `Item` to call its `update_cached_fields` callback itself, too!
When fixing this, I found a new bug that arose, in how we infer
`body_id` for assets that fit all pets. Fixing that next!
This gives better output when they fail, and also avoids spurious
failures like when an array for `cached_compatible_body_ids` is replaced
by an identical one! (I'm running into this right now, and yeah, it
helps a lot lol)
Hmm, I think I made a mistake on `modeling_snapshot.rb:69`: I'm
assigning the *entire* `item.swf_assets` relation to *just* the assets
for the new model of it, which breaks all the other connections.
First, I'm disabling modeling. Then, I'll restore a backup. Then, I'll
write tests for that case, and fix it up!
Not actually touching alt style yet, just the very basic stuff about
how alt style can cause loading to fail in certain extremely rare cases
(specifically, if it's our first time seeing the underlying
color/species combo too, which… isn't gonna happen irl on DTI for a long
time if ever, I would guess, but hey!)
That is, if everything is the same as before, we don't need to change
anything in our database!
I also learned a bit more about RSpec syntax sugars, it's cute!
The NC Pet Styles sentence getting broken across two lines I think
makes it too hard to notice.
Design-wise, it would be nice to just call better attention to this
feature altogether in some higher-level design-language-y way, but!
Whatever!
If you check this box, it'll keep you in a mode where saving an alt
style redirects you to the *next* one that needs labeling, until
they're all done. Useful for big drops!
I want to not turn it off entirely, so that if there's a nasty one it
becomes visible, but we don't need all that vertical space for this
small test suite rn!
I forget, there was some tricky debugging about getting the fixtures
right, I think the previous commit doesn't *actually* pass from a clean
setting. Ah well, looks good now!
Just getting a basic foothold here. I'm thinking about moving this to
RSpec, cuz I feel like the assertions are gonna get pretty specific
and groupable.
I'm gonna work on adding modeling tests, and I want to not be breaking
them without realizing! The trade history ones are good to be checking
more often like this, too.
In 540ce08caa, I updated the Item class
to be more explicit about what fields are required, so this test would
fail in a more helpful way, instead of just crashing from `name` being
`nil` when trying to infer the Dyeworks info.
Now, we update the test to use Rails's standard "fixture" system to set
up a more-correct placeholder item, instead!
Catch missing fields in validation before sending it to the DB, and
skip the Dyeworks stuff if the name is missing.
I ran into this looking into `test/trade_activity_test.rb`, which fails
right now because we try to create a boring placeholder item with
minimal fields, which Dyeworks can't call `name.match()` on!
Now, the test fails with a more helpful error about the item being
invalid. Next, I'll fix that!
Just getting this stuff out of Pet, in part because I want to start
being able to unit test modeling, and that will require stubbing out
what this service returns!
Just a bit more clarity of grouping! I'm also thinking about extracting
modeling APIs into a service file like this too, in which case I think
this would help clarify what it is.
They're not all Nostalgic anymore! Oh, how the times have changed!
This way, new ones will appear as "<New?>", until support staff come in
and label them (with our cool new tools!)
Whoops, I didn't realize this change I made to validation for the alt
style editing form, was goofing up alt style modeling!
The trick is, the validation was happening before the `before_create`
hook. Now I've reformulated these as `before_validation` hooks, so
we're not rejecting new alt styles for having no thumbnail!
Oh huh, I guess most of the new items we had when I rewrote this were
Maraquan, and I didn't test enough on standard species-specific items.
Before this change, partially-modeled items for standard pets would
appear as fully modeled, because the presence of the "nonstandard"
color Orange (because of the Orange Chia) meant that the "standard" key
didn't actually have any unique bodies (it was all `["standard", 47]`).
Here, I take my own comments' advice and move away from the standard
label as part of the logic. Instead, we look first for nonstandard
colors with unique bodies, which we'll call out as modelable; and then
check whether there are any basic bodies *not* covered by those special
colors.
That way, compatibility with the Maraquan Acara (a unique body) means
we'll treat Maraquan as a modelable color; and then we'll ignore the
basic bodies, even though it *does* fit the basic Mynci, because there
aren't any compatible basic bodies that aren't *also* Maraquan bodies.
This also means that compatibility with both the Blue Acara and Orange
Acara does *not* preclude a normal item from needing basic pets for
models: because, while Orange is a slightly nonstandard color in the
case of Chia, it doesn't have its own unique body for this item, so we
ignore it when predicting compatibility with basic colors.
This is because labeling poses with the Support tools *should*
invalidate the `PetState.all_supported_poses` cache! But the previous
cache key would only invalidate when a new pet state is *added*, not
when one is *edited*.
This has just been absent for too long! We've lost a lot of data about
when poses were first modeled, which is a shame.
But I want this in now, because I was just doing caching on
/rainbow-pool.json, and realized that _labeling_ poses is another way
pet states can update rather than just being created!
So we need an `updated_at` field, to be able to quickly detect edits
that require us to invalidate the cache on
`PetState.all_supported_poses`. I'll do that next!
This clocks in a bit bigger than what Impress 2020 does in terms of
binary encoding (with gzip it's at 11K instead of 4K), but I'm okay
with that for the simplicity win.
Gonna try to swap this in for where we're still using Impress 2020 for
the species/color picker in the outfit editor!
Before this change, pages that opt in with `use_responsive_design`
would often have the top nav be real cluttered for logged-in users. (I
think I happened to first test this responsive design without being
logged in on my dev box, oops!) Because the home link and `#userbar`
were absolutely positioned on the page, they would frequently overlap.
Here, I stop doing our old tricks to make the top nav load last on the
page. (This was to get "main content" loading faster, which I think is
a. not as relevant today with more commonly faster connections, and b.
was a bit naive to think that it'd be helpful to have to wait a long
time to _navigate_ if a page is unexpectedly large.)
These tricks used to leave some padding at the top of the `#container`,
which these elements would then visually fill via `position: absolute`
once they load.
Next, I update the CSS (for the responsive design pages only) to use
the new `#main-nav` container to lay them out in Flexbox: all in one
row if possible, or wrapped if needed.
Some designs hide stuff like this into a hamburger menu or such when
the screen gets small. I haven't done that here! No specific reason,
I'm just not sold that it's that much better, or worth the trouble.
I tested this on the following combinations:
1. Logged out, homepage
2. Logged in, homepage
3. Logged out, `/items`
4. Logged in, `/items`
5. Logged out, `/items/89487-Dyeworks-Purple-Haunted-Sky-Background`
6. Logged in, `/items/89487-Dyeworks-Purple-Haunted-Sky-Background`
Hope it's solid! 🤞
Before this change, a fully-modeled item (Dyeworks Burgundy: Gown of
the Night) was displaying as still needing the Chia. This was because
looking for "standard" body IDs like this caught up some of the weird
Chia bodies.
I think there's probably something here where we need to like, relabel
certain colors? But honestly, the better version of this logic would
probably be to lean more into the `basic` label in this logic.
But hey, that's a refactor for another time. I gotta go eat!
Noticing a lot of Maraquan items on the homepage today, and they're
doing that thing of expecting standard body types to be relevant too,
because I think we wrote this logic before the Maraquan Mynci ended up
having the same standard Mynci body? (Maybe? Or maybe I'm being
ahistorical and I just wrote this wrong to begin with lol)
In any case, this is more accurate, and I think I'm also maybe
incidentally noticing that it's running faster, at least in my brief
before/after production testing? (There's *more* queries, like 100! But
many of them are *very* fast lookups, coming in at under 1ms—and also a
lot of them are dupes being served by Rails's request-scoped query
cache.)
Huh, I hadn't realized that like, we'd already set up the controller to
always *run* basically all of the modeling logic, and the caching in
the view layer wasn't saving us any queries anymore. Kinda silly!
Remove the caching call, just to simplify the codebase (I like to avoid
caching things that don't specifically need it!).
And hey, love that the modeling code in the controller is now *way*
faster to run! You love to see it!
I have some other changes planned too, but these are some easy ones. I
also turn back on this stuff in development, in hopes that my changes
can make these queries fast enough to not be a big deal anymore!
This is the second part of the previous change `efda6d74`, in which we
switch out the item search query conditions!
This was a two-parter to ease deployment: first deploy the change with
the migration, then *run* the migration (because it's an unusually slow
one), then deploy this change that actually uses it.
Another way to approach this would've been to deploy it all in one
commit, but not set it as `current` until we had run the migration.
That would have been a reasonable approach too!
This is the first part of a change to improve search performance, by
caching occupied zone IDs and supported body IDs onto the Item record
itself, instead of always doing joins with `SwfAsset`.
It's unfortunate, because part of the power of SQL is joins! But doing
joins with big tables, in ways that can't take advantage of indexes in
the same ways as we often want to, is… slow.
It's possible there's something I'm misunderstanding about SQL
optimization, and this _could_ be done with query optimization or
indexes instead of duplicating data like this? This complexity carries
the risk of data getting out of sync in unforeseen ways. But this is
what I know how to do, and it seems to be working, so! Okay!
By default, Rails gives this button the name `commit`, so it appears in
the URL the form sends to. By setting the name to `nil`, Rails doesn't
set a `name` attribute on the HTML element, so it's *not* included.
The lists of pet types and pet states had very similar styles, which I
mostly copy-pasted. Now that I want to use them for Alt Styles too, I'm
refactoring!
Ah whoops, I didn't notice that, when Turbo morphs the
`<measured-container>` into what the server HTML returns, it deletes
the `style` attribute we were using.
In this change, I refactor for `MeasuredContainer` to be the component
rather than `MeasuredContent`, so that it can also be responsible for
listening for changes to its own `style` prop, and remeasuring when
they happen.
We're also careful to avoid infinite loops, by only doing this when the
property is missing! (Otherwise, setting `--natural-width` triggers the
callback again, oops!)
This hasn't worked for a while anyway! Let's remove the bits of code
where we deal with it, and the database field that signals it. (We also
make a corresponding change in Impress 2020, so it doesn't crash trying
to query based on the `prank` column.)
I also ran this snippet to clear out all the Nebula stuff in the db:
```rb
Color.transaction do
nebula = Color.where(prank: true).find_by_name("Nebula")
nebula.pet_types.includes(pet_states: :swf_assets).each do |pet_type|
pet_type.pet_states.each do |pet_state|
pet_state.parent_swf_asset_relationships.each do |psa|
psa.swf_asset.destroy!
psa.destroy!
end
pet_state.destroy!
end
pet_type.destroy!
end
nebula.destroy!
end
```
"Fall Woodland Leaves Filter" is an example, it's part of the two-item
*pack* named "Fall Woodland Minitheus Petpet Foreground". The NC Mall
page for it will include the secondary items in `object_data`, but it's
not part of the storefront itself—and the only thing indicating that is
the `render` list.
Theoretically, we could use this to construct more data about like,
packs and stuff, automatically? But also, I don't want to backfill it
for everything historically, so like. Whatever.
See comment for details! I wonder if other items have been affected by
this in the past. I think probably what happened before was that we
successfully created this item, but failed to create the *translation*,
so when migrating over the Patchwork Staff all its translated fields
were empty? (That's what I found looking in the database today.)
But yeah, thankfully our crash logging at health.openneo.net gave me
the name of a pet someone was trying to model, and so I was able to
find the bug and fix it!
Now that we have such a convenient lil outfit viewer component we built
for the item page preview, it's easy peasy to drop it in here too! And
it's all nice and lightweight, since in this case it's basically just.
image tags, with some supporting enhancements.
Anyway, this page has no actual useful styles of its own yet. Gonna
make it look nice and such!
I'm experimenting with a Rainbow Pool ish UI, mainly as a support tool
for exploring and labeling poses—but one we can probably just show to
real users too!
Right now, I just use pet type images as a placeholder, and I polished
up some of the `pet_type_image` API. But we're probably gonna drop
these for a full outfit viewer, now that I think of it.
This is a transitional gem to help with upgrading from old versions of
Rails: it provides a deprecated feature that Rails removed.
I audited and I *think* we only used it in one place, and that this one
place doesn't even use any of its functionality for styling or
scripting? So, begone!
Oh sweet, I learned about a new CSS feature with good-enough support!
This lets you use CSS transitions for an element as it enters the page,
or becomes visible.
Firefox only has partial support for this feature rn, but its partial
support covers our case, I tested to make sure! (Specifically, it
doesn't handle transitioning from `display: none` yet, which isn't what
we're doing.)
Whew, quite a history here! I didn't _extensively_ audit for these, but
I scanned with pretty good searches and hit major pages and they didn't
crash, so. Good enough for me!
Right, yeah, we've been depending on an external CDN for a long time
for jQuery and the jQuery Template library, and I don't like that kind
of external dependency! Let's put it in with the rest of our libs.
It's only actually used in two JS files, so rather than doing a weird
global `$.ajaxSetup` call, let's just inline it into the small handful
of AJAX calls that actually care.
Uhh I guess when I half-removed a feature from the translations list (I
don't remember when?), it left two different dictionaries labeled
`neopets_page_import_tasks.new`, and the second one overwrote the
first. Oops! Yikes!
By removing these, the translations *above* them actually get to apply
to the page correctly. Before this change, the page just showed the
translation keys as placeholders, womp womp.
When I was trying to debug slow view code one time long long ago, I was
like "let's cache any part of the template that's static!"
And like. no that's silly, I don't trust that this speeds anything up,
but it _definitely_ adds complexity. Let's just not.
Just for consistency with the other features we're not using, we turn
off ActionCable when loading the app. I just removed
`config/cable.yml`, so I figure, let's not load a feature without the
config file it expects! (even though that didn't seem to bother it)
We're not using the ActionCable or ActiveStorage Rails features in this
app, so we can clear out these default config files. If we need them
later, it's not hard to re-find / re-generate them!
Our production data now contains basic hashes for all species/color
combinations, and it's easy enough for a dev copy of the site to get
them too by running `rails public_data:pull`. So, I think it's time to
retire this hardcoded set, and get one more file out of our codebase!
No pressing reason, I'm just doing upgrades today, and noticed a new
version is out, and scrolled the patch notes and there's no obvious
breaking changes for my purposes, so. Up we go!
The silly motivation is that I wanted to remove `.prettierignore`,
which just exists to omit that one folder from `npm run format`. But it
also seems like this is the standard place to put them—a standard
created long after we first set this up lol
I forget what this was for, I think part of it was for managing item
names in different languages, and the "private" locale thing was
probably for WIP locales? But yeah, not used, delete!
Yeah, I don't remember why So Many Years Ago I felt it was important to
use the Droid fonts; I adapted this choice into the Noto fonts when
modernizing the other day, but, tbh, the default system fonts are
probably just a better fit for like. everything we do, and then *not*
downloading MB of font files.
I also feel like a lot of the contexts where we used serif fonts were
like, frankly incidental, based on where we chose `<p>` for semantic
reasons? I don't think any of them actually are made much better by
serifs, I'm okay with just simplifying and dropping that, instead of
looking for a better serif font stack to replace it.
There's some funny bugs we had here, like "Relic Elephante Jewellery"
and "Royal Girl Skeith Bodice" getting assigned "Ice", and
"Tyrannian Meerca Spear" being "Pea" lmao
I went and checked all the assignments now and they look good to me!
```ruby
Item.is_pb.order(:name).
map { |i| [i.pb_color&.human_name, i.name] }
```
Ahh right, when you indent stuff underneath a tag in HAML, it does the
same indented form in the output HTML, which adds whitespace that
creates a problem for how we're doing this list.
Before this change, the "Engulfed in Flames Effect" item showed below
the preview: `Occupies: Background Item , Lower Foreground Item`, with
an extra space before the comma.
After this change, it now shows
`Occupies: Background Item, Lower Foreground Item`, as intended.
Fun little bug: viewing the "Engulfed in Flames Effect" item was
showing our "502 Bad Gateway" custom error page in the embed. This is
because the Rails app was providing a `Content-Security-Policy` header
value that was longer than nginx is configured by default to allow, so
it was refusing the response, and showing the same 502 error as if the
app hadn't responded at all. (We discovered this by opening
`/var/log/nginx/error.log`, which explained this very clearly, ty~!)
In this change, we no longer list every `images.neopets.com` asset,
instead marking the entire domain as a valid image source for the
SWF asset embed iframe. I don't _love_ this solution, I liked the
property of specifying literally exactly the assets we allow! But I
don't think there's any practical danger here, and it helps a *lot* for
making this more reliable.
(If we could have solved this reliably by increasing nginx's allowed
response header size, I probably would've done that? But I researched a
bit, and ultimately concluded that I don't trust other intermediary
software like firewalls not to have the same issue. Let's not be
pushing the limits of HTTP headers of all things!)
This shouldn't ever be an issue in practice? I just noticed it because
something funny is going on with the `#userbar` element specifically
not using the Delicious font, and so I figured, hey, this simulates a
very real possible scenario, I'd rather use our consistent sans font
in this case!
Huh okay, moving to my other machine, the change to Noto Sans subtly
broke the homepage layout a bit, wrapping the form buttons to the next
line in the three module sections.
Here, I refactor to more modern grid/flexbox sensibilities. Btw, there
was a Flexbox thing that didn't work quite how I expected? I commented
on my confusion, but checked in Chrome and Firefox and it seems to work
in both, so, ok!
Okay cool, so this was an error that was happening *only* when building
assets for production: Sass's CSS minifier isn't familiar with all
modern CSS syntax (I think is the issue?), and so errors on things that
are actually totally okay.
I had previously worked around this in `swf_assets/show.css` with an
equivalent syntax that Sass recognized. But in this latest case with
the new `fonts.css.erb`, it was upset about the `src` list for the
fonts, and I don't know a workaround for that.
So, let's just disable Sass's CSS minification for now. I imagine the
difference isn't huge when CSS compresses just fine with gzip anyway?
(Most of what you can "minify" in CSS is whitespace, and that largely
seems silly to me when gzip is running.)
I was just scrolling our CSS and surprised to find we use Google Fonts
embeds! I don't like depending on external hosts like that.
Google Fonts doesn't offer the Droid fonts for download anymore,
though—looks like the Noto fonts are their spiritual successor. The
Droid Serif and Noto Serif fonts look visually identical to me, but the
Sans ones are a bit different… I kinda like the charm of the Droid Sans
better, but ah well! I'd rather be moving forward with a more modern
font with more reliable glyph support etc for now.
I think this has just been broken for a long time? And I don't think
it's very useful in a world 15 years later, where our problem *used* to
be giant gaps in our library, which isn't really our data problem
anymore.
No more of this loading everything into `application.css`! I'm
arbitrarily starting here because that's where I've been playing
lately, but this is part of a larger effort to move toward a more
straightforward CSS architecture (and away from Sass even?)
Ran into a funny thing on an upcoming change, where a style on the
page-specific stylesheet was getting undone by the *CSS Reset* of all
things in the application stylesheet. Resets come first!
I haven't audited that I didn't break a ton of stuff with this change,
but. I hope not! :)
Closes#3, by adapting the suggested changes! Thank you!!
We both change how we create pet name preview jobs, by catching the `@`
case early; and we better handle symbols in pet names when showing the
thank you message, by parsing the query string more correctly.
Co-Authored-By: Steve C <diceroll123@gmail.com>
This was always modeling correctly, but not showing the message,
because Turbo doesn't handle anchors in redirect URLs the same way the
browser's full page loads do.
I forget why we had this as a `#` URL anyway to begin with. Use `?`
instead!
I haven't been running Prettier consistently on things in this project.
Now, it's quick-runnable, and I've got it on everything!
Also, I just think tabs are the right default for this kind of thing,
and I'm glad to get to switch over to it! (In `package.json`.)
I skipped this for a bit because I couldn't think of a simple way to
adapt this behavior to a web component + vanilla CSS setting, but then
I thought of CSS variables, and sat down and cranked this out!
I noticed the last row of the species faces required a scroll, I forget
when that happened! But I made some tweaks, most notably widened the
container from the normal 800px, so that on bigger screens everything
lays out and aligns nice, without requiring any scrolling of the face
container!
Oh oops, I forgot one of the kinds of restricted zones when refactoring
how we load search data in wardrobe-2020! This made most items with
restricted zones (like Be Gone items) not work correctly when you
search for them to add them to the item—though it *does* work correctly
when you reload the page or change the species, to get to load a
different way.
If something goes wrong, like the site goes down or has an intermittent
error, try a full pageload. That way, we're both retrying, and in a way
that gives the user more control and visibility into what's going on,
and what they can potentially do about it. (e.g. if there's a useful
error message, they will see it!)
I took this ordering from a specific place on Impress 2020, but I think
that was in a context where the pose mattered more? Here though, I'm
realizing that I'd rather show any known-unglitched pose than the happy
masc or whatever we semi-randomly chose.
instead of doing the random choice we do for most colors.
This is especially noticeable in cases where like, I'm looking at the
Elderlyboy Ogrin and like, it has *work* put into the masc eyes, and
them fem eyes are just the standard ones.
Used to have something like this long ago, now here's the latest
version!
This task can't run autonomously, it needs the human user to provide a
neologin cookie value. So, no cron for us! But we're cleaning up *years*
of lil guys in one swoop now :3
When playing with a Rainbow Pool syncing task, I noticed that error
handling wasn't working correctly for requests using `async-http`: if
the block raised an error, the `Sync` block would never return.
My suspicion is that this is because we were never reading or releasing
the request body.
In this change, I upgrade all the relevant gems for good measure, and
switch to using the response object yielded by the _block_, so we can
know it's being resource-managed correctly. Now, failures raise errors
as expected!
(I tested all these relevant service calls, too!)
Huh, this is a bit odd, I think we took this from Impress 2020's
`canonicalPetStateForBodyLoader` SQL query… but actually, it doesn't
really make sense? and `petStatesForPetTypeLoader` has a more sensible
ordering, and is the one the app uses in more ways. Maybe that's a
mistake we made back then, or a bug we fixed only in one place?
Anyway, this fixes why the item previews were like. using a LOT of
glitched pet states and I was like "dang did a lot of them break
recently?"
Nah we were just. not pulling the right ones lol
Oh right, forgot about this lol!
The specific effect on Impress 2020 where the button label expands is,
kinda hard to implement in normal CSS/JS, and so I'm not in the mood
and I'm settling for the `title` attribute lol
Oh right, I need the error indicator to be part of a container that
also contains the outfit viewer, to appear below it!
I was motivated because I realized I forgot the Customize More button
so now I'm building it lol
The main bottleneck for us is still just uploading the full source code,
there might be some clever option I'm not using for that yet of like,
compression or something? But this change did take the process down
from like 5 minutes to 3 minutes, so, works for me!
My immediate motivation is that I'm going to try turning on the
pipelining setting, to improve performance, and I'd like to have the
consistent place to put it! But also, I like standardizing our setup a
bit more, too
The loading indicator *should* fade in after two seconds, to avoid a
flash of a loading indicator when the page loads quickly - but in some
circumstances it wouldn't delay:
1. Visit an item page. (It delays correctly the first time!)
2. Click "Infinite Closet", then click a link to another item page.
3. The loading indicator appears immediately, because this time the
web component JS is already loaded, so the `outfit-layer` elements
enter `:state(loading)` *immediately*. The element starts at
`opacity: 1`, and the delay doesn't matter, because it was never at
anything else.
In this change, we have the `outfit-viewer` web component take on a
`:state(after-first-frame)`, after a `setTimeout(0)` resolves. That
enables the loading state CSS to *never* apply on the first frame, but
then sometimes kick in on the *second* frame, so that the element is
correctly perceived as "transitioning" from hidden to visible, and the
two-second delay will apply.
When I run `bin/deploy:precompile` on the previous version, I get an
error from libsass that `vw` and `vh` are incompatible units. I don't
get this error in development, only when compiling for production.
My inference is:
1. For the production build, Sass is trying to preprocess even non-SASS
files, maybe to help minify them?
2. In Sass, their `min()` existed before CSS's `min()`, so it's
treating it Like That, and returning a reasonable-in-some-cases-but-
not-here error that `min(100vw, 100vh)` can't be *precomputed*.
Anyway, wrapping it in `calc()` isn't a *problem*, and helps the Sass
compiler not try to precompute it, so. Okay!
https://github.com/sass/node-sass/issues/2815#issuecomment-575926329
We call it enough times on this page, and it *does* have a SQL query,
that I want to cache it! (Also I want to make it fewer species queries
if I can tbh…)
For static image layers, this was *always* logging that we failed to
send the frame a "pause" message. Which, like, of course!
It makes sense to log the notable circumstance where we send a message
we *expect* to arrive, but the frame isn't loaded yet. But if there's
just no frame, ignore it and don't bother to say so.
We add a new `use_responsive_design` helper, for pages to opt into this
new CSS—mostly just because like… it's *worse* to apply these styles
for pages that don't expect it 😅
And then, I fix up a couple things on the item page (including in the
general items layout) to match!
I'm doing this because the species face picker layout is going to want
some responsive awareness, and I want to be doing that from the start!
Here, I remember the trick I learned when building the outfit viewer:
web components are great for making sure stuff stays initialized well
in a Turbo environment!
The problem was, after submitting the form and getting a new preview
loaded via Turbo, the part where we remove `inert` would get undone.
Additionally, this script only loads *once* per session, so if you
Turbo-nav to a different item then that part of the page never ran.
Instead, we use web components to remove the attributes on mount, then
again if they're ever reapplied by Idiomorph.
We mark the options as `inert` and `aria-hidden` while the JS is still
loading—and if the `noscript` tag tells us it's never coming, it covers
up the picker with a brief explainer!
The basics are working great! There's a few known missing things though:
- Add reasonable noscript behavior
- Disable options where there's no valid appearance
- Lay it out actually _good_, instead of just images dumped there
Adapting what the Impress 2020 UI does, but in Ruby instead!
I feel like this is case is really starting to show the power of doing
this stuff in Rails instead of via an API… we can *really* take
advantage of our models and our handy idioms at all points. This is
just so much less *code* than this feature takes in Node + GraphQL +
React.
We used to use this to determine what color to show by default on the
item page preview for, like, Maraquan-specific items. Now, we infer it
from our actual customization data, rather than these heuristics!
There's still a database field for `Item#manual_special_color_id`. We
can still read and write this from the support UI, and Impress 2020
still slightly uses it from the homepage, so I'm not removing from the
database right now.
The `build_on_pet_types` helper used to be reused on the items page, to
generate the list of species to display. We don't use it anymore, so
simplify and remove!
I'm about to reimplement the more-robust version of what this used to
be: how the item page used to say "sometimes" after certain zones in
the occupied list.
Now, we're going to do parity with 2020, and list the actual species!
I like that this takes away the weird `#sometimes` method on the `Zone`
class, which was always an odd hack for just this small thing.
We had this issue on Impress 2020 and I fixed it over there too. I guess
it went less noticed here on Classic, because it's a more
progressively-enhanced site in general (and this failure case is an
interesting argument for that architecture! lol).
On Impress 2020, I wasn't sure if the "waits for the document to load"
behavior of the `defer` attribute was necessary to the script, so I
chose to keep `defer` but move it _after_ the other scripts.
This time, I dug in a bit more, and found a Plausible author saying
that the choice was kinda arbitrary; and another person who had the
same issue as me, who said they switched to `async` and it worked well.
So, that's what we're doing now, too!
https://github.com/plausible/analytics/discussions/1907#discussioncomment-2754499
Closes#2, after making some tweaks to the PR to fit how JS templating
works here. Thanks @dice!!
I had to move `petThumbnailUrl` out of the closure, because this script
does a cute thing of having separate variable scopes for the separate
areas of the page—but this is used by two of them. Arguably it could
make sense to like, put this all in one larger shared IIFE closure that
wraps both of them, to preserve some of this code's intention of
avoiding adding to the global namespace on this page, but like.
*It's fine.*
Co-Authored-By: Steve C <diceroll123@gmail.com>
For the gems, I mostly just ran `bundle update`; with the exception of
`httparty`, because latest Ruby throws a deprecation warning about its
use of the deprecated `csv` stdlib, which the latest version resolves.
One other little thing: this is on my new Fedora workstation, and I had
to deal with a known bug where the `sassc` gem compiles a `libsass.so`
file, but saves it in the wrong place somehow.
Here's the known bug, and the comment that helped me:
https://github.com/sass/sassc-ruby/issues/146#issuecomment-2028974524
And here's what I ran to get it into the right place:
```shell
ln -s ~/.local/share/gem/ruby/3.3.0/extensions/aarch64-linux/3.3.0/sassc-2.4.0/sassc/libsass.so \
~/.local/share/gem/ruby/3.3.0/gems/sassc-2.4.0/lib/sassc/libsass.so
```
This thing about `libsass` isn't reflected in the code changes anywhere
in this commit! I'm just mentioning it so that it's literally written
down anywhere. (I did try other comments' advice to use an older
version of `sassc` first, but I ran into compilation errors, so figured
this machine-side hack was better than untangling that mess.)
Dress to Impress (DTI) is a tool for designing Neopets outfits. Load your pet, browse items, and see how they look together—all with a mobile-friendly interface!
There'll be more to say about it here soon :3
## Architecture Overview
DTI is a Rails application with a React-based outfit editor, backed by MySQL databases and a crowdsourced data collection system.
### Core Components
- **Rails backend** (Ruby 3.4, Rails 8.0): Serves web pages, API endpoints, and manages data
- **React outfit editor**: Embedded in `app/javascript/wardrobe-2020/`, provides the main customization UI
- **Modeling system**: Crowdsources pet/item appearance data by fetching from Neopets APIs when users load their pets
### The Impress 2020 Complication
In 2020, we started a NextJS rewrite ("Impress 2020") to modernize the frontend. We've since consolidated back into Rails, but **Impress 2020 still provides essential services**:
- **GraphQL API**: Some outfit appearance data still loads via GraphQL (being migrated to Rails REST APIs)
- **Image generation**: Runs a headless browser to render outfit thumbnails and convert HTML5 assets to PNGs
See [docs/impress-2020-dependencies.md](./docs/impress-2020-dependencies.md) for migration status.
## Key Concepts
### Customization Data Model
The core data model powers outfit rendering and item compatibility. See [docs/customization-architecture.md](./docs/customization-architecture.md) for details.
**Quick summary**:
- `body_id` is the key compatibility constraint (not species or color directly)
- Items have different `swf_assets` (visual layers) for different bodies
- Restrictions are subtractive: start with all layers, hide some based on zone restrictions
- Data is crowdsourced through "modeling" (users loading pets to contribute appearance data)
### Modeling (Crowdsourced Data)
DTI doesn't pre-populate item/pet data. Instead:
1. User loads a pet (via pet name lookup)
2. DTI fetches appearance data from Neopets APIs (legacy Flash/AMF protocol)
3. New `SwfAsset` records and relationships are created
4. Over time, the database learns which items fit which pet bodies
This "self-sustaining" approach means the site stays up-to-date as Neopets releases new content, without manual data entry.
- Coverage is focused on key areas: modeling, prediction logic, external APIs
- Not comprehensive, but thorough for critical behaviors
## Tech Stack
- **Backend**: Ruby on Rails (Ruby 3.4, Rails 8.0)
- **Frontend**: Mix of Rails views (Turbo/HAML) and React (for outfit editor)
- **Database**: MySQL (two databases: `openneo_impress`, `openneo_id`)
- **Styling**: CSS, Sass (moving toward modern Rails conventions)
- **External Integrations**:
- **Neopets.com**: Legacy Flash/AMF protocol for pet appearance data (modeling)
- **Neopets NC Mall**: Web scraping for NC item availability/pricing
- **NeoPass**: OAuth integration for Neopets account linking
- **Neopets Media Archive**: Local filesystem mirror of `images.neopets.com` (never discards old files)
- **Lebron's NC Values**: Third-party API for NC item trading values ([lebron-values.netlify.app](https://lebron-values.netlify.app))
- **Impress 2020**: GraphQL for some outfit data, image generation service (being phased out)
## Development Notes
### OpenNeo ID Database
The `openneo_id` database is a legacy from when authentication was a separate service ("OpenNeo ID") meant to unify auth across multiple OpenNeo projects. DTI was the only project that succeeded, so the apps were merged—but the database split remains for now.
**Implications**:
- Rails is configured for multi-database mode
- User auth models live in `auth_user.rb` and connect to `openneo_id`
- **⚠️ CRITICAL**: Impress 2020 also directly accesses both `openneo_impress` and `openneo_id` databases via SQL
- **Database migrations affecting these schemas must consider Impress 2020's direct access**
- See [docs/impress-2020-dependencies.md](./docs/impress-2020-dependencies.md) for full details on this dependency
### Rails/React Hybrid
Most pages are traditional Rails views using Turbo for interactivity. The **outfit editor** (`/outfits/new`) is a full React app that:
- Loads into a `#wardrobe-2020-root` div
- Uses React Query for data fetching
- Calls both Rails REST APIs (in `loaders/`) and Impress 2020 GraphQL (being migrated)
The goal is to simplify this over time—either consolidate into Rails+Turbo, or commit fully to React. For now, we're in a hybrid state.
## Deployment
- **Main app**: VPS running Rails (Puma, MySQL)
- **Impress 2020**: Separate VPS in same datacenter (NextJS, GraphQL, headless browser for images)
- **Shared databases**: Both services directly access the same MySQL databases over the network
- `openneo_impress` - Main application data
- `openneo_id` - Authentication data
- ⚠️ **Any database schema changes must be compatible with both services**
---
**Project maintained by [@matchu](https://github.com/matchu)** • **[OpenNeo.net](https://openneo.net)**
