Commit graph

76 commits

Author SHA1 Message Date
b03d9b264a Increase maximum username length to 30
I'm writing some code for default NeoPass usernames, and they can get
kinda long, so I want to clear some extra space for them!
2024-04-01 05:53:38 -07:00
07617fa34f Increase ID column limit for item_outfit_relationships
Ahh wow, we're finally at the point of having enough outfits to need to
increase this ID field!

I got an alert this morning that queries were failing with the MySQL error
"id out of range". I went and found that creating a new outfit still works if
it's empty, but fails once you try to add an item to it. So, I assume this is
the failing column!

Note that I'm writing this from my emergency Steam Deck workstation, and that
I didn't quite get MySQL set up on it yet (I just yolo ran this in
production), so we don't have the corresponding changes to `db/schema.rb` yet.
This might require another commit from my normal workstation later to
complete this change!
2024-03-24 11:09:40 +11:00
ec6dca1c16 Improve Unicode support, emojis don't crash us anymore lol!
A few pieces here:

1. Convert all tables to `utf8mb4`+`utf8mb4_unicode_520_ci` strings.
2. Configure that as the server's default.
3. Configure the Rails database connection to use this encoding too.

Came together pretty well, whew! This has been a LONG time coming,
`latin1` is NOT a good charset for the year 2024!
2024-02-28 18:54:27 -08:00
18c7a34b8f Update series_name for alt styles to be null, with a fallback string
I considered this at first, but decided to keep it simple until it
turned out to matter. Oops, it already matters, lol!

I want the item search code to be able to easily tell if the series
name is real or a placeholder, so we can decide whether to build the
filter text in `fits:$series-$color-$species` form or
`fits:alt-style-$id` form.

So in this change, we keep it that `AltStyle#series_name` returns the
placeholder string if none is set, but callers can explicitly ask
whether it's a real series name or not. Will use this in our next
change!
2024-02-27 15:48:28 -08:00
9f74e6020e Add series_name database field to alt styles
Previously we did this hackily by comparing the ID to a hardcoded list
of IDs, but I think putting this in the database is clearer and more
robust, and it should also help with our upcoming item search stuff
that will filter by it!
2024-02-27 15:28:05 -08:00
a684c915a9 Track when manifest was last loaded, and what status it returned
Now we're *really* duplicating with Impress 2020's system lol, but I
need a way to not keep trying to load manifests that are actually 404,
which are surprisingly plentiful!

This doesn't actually stop us from loading anything yet, it just tracks
the timestamps and the HTTP status! But next I'll add logic to skip
when it was 4xx recently.
2024-02-25 15:35:04 -08:00
c8e53165c7 Drop item_translations table
Okay cool, we're successfully migrated off translations, we can delete
the table now!

I'm not worried about backing up this data as such, because the
impress-2020 repo has a bunch of this data in its
`public-data-from-modeling.sql.gz` file history. Safe to remove from
the live app!
2024-02-20 17:01:54 -08:00
b7296d6a75 Fix default value for Impress 2020's modeling_logs.created_at 2024-02-20 16:46:19 -08:00
a1066d9c8a Add translated item fields directly to the Item model
Like with Species, Color, and Zone, we're moving the translation data
directly onto the model, and just using English. This will simplify
some of our queries a lot (way fewer joins!), and it's what Neopets
does now anyway, and I have a secret hope that removing the complexity
along the codepath for `item.name` might help speed up large item lists
if we're lucky?? 🤞

Anyway, this is the first step, performing the migration to copy the
data onto the `items` table, making sure to keep them in sync for the
2020 app for now!
2024-02-20 15:25:03 -08:00
d6b6ab884d Migration to drop {color,species,zone}_translations
They're no longer referenced in either app! Begone!

(The translated values are still available in the DTI 2020 repository's
history, under `scripts/db`!)
2024-02-03 08:20:18 -08:00
c60e222faa Add Alt Style support to outfit saving
Pretty straightforward, just add the field to the record, and wire it
all up! I'm glad this seemed to work out pretty well all-in-all 😅
2024-02-01 05:55:19 -08:00
4fff8d88f2 Add support_staff flag to user record; they can use Support tools
A little architecture trick here! DTI 2020 authorizes support staff
requests by means of a secret token, instead of user account stuff. And
our support tools still all call DTI 2020 APIs.

So here, we bridge the gap: we copy DTI 2020's support secret to this
app's environment variables (I needed to update
`deploy/files/production.env` and run `bin/deploy:setup` for this!),
then users with the new `support_secret` flag have it added to their
HTML documents in the meta tags. Then, the JS reads the meta tag.

I also fixed an issue in the `deploy/setup.yml` playbook, where I had
temporarily commented some stuff out to skip steps one time, and forgot
to uncomment them after oops lol!
2024-01-29 04:21:19 -08:00
5004142dfb Add alt style support to modeling
Nothing to show them yet, but I think this works for loading it all in
the first place?

Still needs contributions tho!
2024-01-24 03:25:23 -08:00
73af9d4d77 Start migrating off globalize gem for zones
Like in 0dca538, this is preliminary work for being able to drop the
`zone_translations` table! We're copying the field over first, to be
able to migrate DTI 2020 safely before dropping anything.
2024-01-23 05:43:00 -08:00
413d2eed91 Improve species/color name migration performance & correctness
Oh right, this should only run in the up direction! And also we can get
a lot faster by preloading the translations.
2024-01-23 05:30:42 -08:00
0dca538b0a Start migrating off globalize gem for species/color names
Non-English languages haven't been supported on Neopets for a while, so
I'd like to remove this extra cross-cutting complexity, especially
since it's now inconsistent with the real site anyway!

The main motivation is that I'd like to do this for items too, because
I have a hunch that all the complexity of `globalize` to read
`item.name` is part of what's making large user lists so slow to
render: lots of little objects getting created down the stack, and
needing to be garbage-collected later.

I'm not sure that's why! But I figure removing this complexity is a
simplicity win anyway, so let's do it!

Note that this doesn't *finish* the migration, it just starts it! The
`Species::Translation` and `Color::Translation` models still exist, and
still have their data, and not all references to them are scrubbed yet.
I especially don't want to delete the backing tables until both DTI and
DTI 2020 are ready for it!

So this change will someday be paired with another change to actually
drop the tables - after backing up the data for future records just in
case, of course!
2024-01-23 05:10:43 -08:00
470c805880 Save last trade activity time onto User
In impress-2020, we do a big slow query to figure out which users have
been active in trades recently. Now, we cache that timestamp on the
User model.

This won't have any immediate effect; it's to clear the way for Classic
DTI to receive the better trade ratios feature people like from 2020.

I also added some unit testing infra because I finally wanted it! for
all the ways you can trigger this timestamp lol

Note too that this is a bit of an unusually complex migration, but my
hope is that the batching and query structure and such helps it run
surprisingly fast! 🤞
2024-01-19 00:00:46 -08:00
c725a11920 Remove unused NC Mall spider code & fields
We haven't used the mall spider in this app in forever (I guess we even
deleted the code at some point?), but there was some vestigial stuff
left. Goodbye!
2023-11-11 15:45:38 -08:00
644a6acc72 Remove old Image Mode stuff
There was a static page explaining it, which we no longer link to; and
there was an unused field in the User model for who was a beta tester
for it. Goodbye!
2023-11-11 15:21:49 -08:00
dc22a458bf Move manifest backfill to swf_assets:manifests task
Okay, I've simplified the migration to *just* add the column, and
instead added a task to find assets without manifest URLs and backfill
them.

Performance is a lot better now, using the `async-http` library, which
as I understand it supports both persistent connections when invoked
like this, and maybe also HTTP/2 multiplexing?? (Though I'm not
actually sure images.neopets.com does lol)

I'm not sure about the number of concurrent tasks I picked here, 100
seems okay for an internet thing and for such small requests, but I
worry that the CDN is gonna get annoyed or something. Well, we'll see!
This task is very resumable if it turns out we get frozen out or
something.
2023-11-10 16:52:50 -08:00
96998643b5 Add manifest_url to swf_assets table
Ok so, impress-2020 guesses the manifest URL every time based on common
URL patterns. But the right way to do this is to read it from the
modeling data! But also, we don't have a great way to get the modeling
data directly. (Though as I write this, I guess we do have that
auto-modeling trick we use in the DTI 2020 codebase, I wonder if that
could work for this too?)

So anyway, in this change, we update the modeling code to save the
manifest URL, and also the migration includes a big block that attempts
to run impress-2020's manifest-guessing logic for every asset and save
the result!

It's uhh. Not fast. It runs at about 1 asset per second (a lot of these
aren't cache hits), and sometimes stalls out. And we have >600k assets,
so the estimated wall time is uhh. Seven days?

I think there's something we could do here around like, concurrent
execution? Though tbqh with the nature of the slowness being seemingly
about hitting the slow underlying images.neopets.com server, I don't
actually have a lot of faith that concurrency would actually be faster?

I also think it could be sensible to like… extract this from the
migration, and run it as a script to infer missing manifest URLs. That
would be easier to run in chunks and resume if something goes wrong.
Cuz like, I think my reasoning here was that backfilling this data was
part of the migration process… but the thing is, this migration can't
reliably get a manifest for everything (both cuz it depends on an
external service and cuz not everything has one), so it's a perfectly
valid migration to just leave the column as null for all the rows to
start, and fill this in later. I wish I'd written it like that!

But anyway, I'm just running this for now, and taking a break for the
night. Maybe later I'll come around and extract this into a separate
task to just try this on all assets missing manifests instead!
2023-11-09 21:42:51 -08:00
b8a8cb9b20 Stop orphaning hangers when deleting lists
Idk if this used to be different or what, but it looks like the current
behavior is: if you delete a closet list, it'll leave the hangers
present, but Classic DTI would not show them anywhere; but Impress 2020
(until recently) would crash about it.

Now, we use `dependent: :destroy` to delete the hangers when you delete
the list (which I think makes sense, and is different than what I
decided in the past but that's ok, and is what the current behavior
*looks* like to people!), and we add a migration that deletes orphaned
hangers.

The migration also outputs the deleted hangers as JSON, for us to hold
onto in case we made a mistake! I'm also backing up the database in
advance of running this migration, just in case we gotta roll back HARD!
2023-10-24 15:35:21 -07:00
b2a027fbbf Oops, should've used migration version 4.2
Oh I didn't realize the lowest version Rails had for this is 4.2. I wish running `rake db:migrate` checked this, but I'm running into it on another branch when I try to create a *new* migration which for some reason leads it to inspect the old migrations and notice the issue. Weird!
2023-10-23 19:05:08 -07:00
fcf3292448 Add Rails version to old migrations
I'm not sure it's literally true that they were all built against Rails 3.2, but that's what it was at before we upgraded, and like. that's probably fine
2023-10-23 19:05:08 -07:00
Matchu
38f3c9894c Delete WardrobeTip model
lmao I have zero recollection of this, I've been generally trying to avoid too-far-off-the-path cleanup, but this one just seems silly, goodbye
2023-10-23 19:05:05 -07:00
Matchu
8aa4aa6e30 Drop NewsPosts model
We never ended up really using this lol, and it has an attr_accessible I want to be rid of so let's just fully trash it while we're at it!
2023-10-23 19:05:04 -07:00
Matt Dunn-Rankin
c7d60e3e41 Add db field to override special color
Okay, fine, finally making this controllable from the db without requiring a deploy :P Setting this new field will cause `item.special_color` to return the corresponding color. This mainly affects what we show on the item page, and what colors we request for modeling on the homepage.
2018-05-09 14:10:20 -07:00
Matt Dunn-Rankin
f5fb8711fc improved donation campaign nav 2017-01-05 13:06:50 -08:00
Matchu
b5442d0f3f news posts read from the database, with a campaign-supplement classname option :) 2015-09-26 20:37:03 -07:00
Matchu
2ca784c3a2 thank-you message automatically deploys on campaign success 2015-09-22 22:40:51 -07:00
d8038f2fbf prefer scraped rainbow pool images over pet images 2015-09-05 18:48:41 +00:00
Matchu
5d2c226357 yum, new campaigns 2015-08-05 19:26:12 -04:00
0e6f823154 toggle whether a donation campaign is advertised 2015-02-25 13:49:18 -06:00
8e22c271a4 track campaign progress 2014-09-11 17:40:37 -05:00
f11f6374da donation mailer 2014-09-10 14:32:54 -05:00
90b45dcecd edit featured outfits on donation page 2014-09-09 23:16:02 -05:00
59d5e99312 donation show form and basic name updating 2014-09-09 22:04:17 -05:00
595b1c2fc5 charge and record donations 2014-09-09 21:11:55 -05:00
8fc156833f basic tip functionality 2014-04-02 23:00:50 -05:00
f9fa3eb596 prank color artist credit 2014-03-31 21:05:28 -05:00
49a901abfb prank color migration 2014-03-27 22:28:48 -05:00
b2fca6b6c1 closet hangers index uses neopets connections dropdown 2014-01-18 22:50:14 -06:00
72b174c9b3 store all neopets usernames for logged-in users, but breaks closet_hangers#index 2014-01-18 21:55:01 -06:00
8bd7ad5a1e support manually uploaded swf asset images - not the actual uploading, but block them from being reconverted 2013-10-16 15:50:48 -05:00
bf528b06d2 label pet states as glitched, send to bottom of emotion order 2013-04-27 10:21:51 -05:00
a39110884d cut down on pets#load queries 2013-01-28 02:10:25 -06:00
26ac3782ec move zones to database 2013-01-24 18:26:00 -06:00
ce4e2fd53c move species to database 2013-01-24 18:25:59 -06:00
965465ca51 move species to database 2013-01-24 18:25:59 -06:00
d79c225c1e apply globalize3 to items 2013-01-24 18:24:34 -06:00