Commit graph

99 commits

Author SHA1 Message Date
b6e18e10a5 Add bare-bones rails nc_mall:sync task, incl. NCMallRecord model
Currently we only load the homepage, so there's only actually one
wearable item to sync up! But here's the task to do it!

To do this, we also created the backing model NCMallRecord, where we'll
save the current NC Mall state!
2024-05-07 17:40:14 -07:00
0943e2dbba Fix broken default value in schema for item description
Idk how we got into this state, or if it's environment-dependent or
MySQL-version-dependent or what, but setting up the dev environment on
my macOS machine is complaining that `TEXT` columns can't have default
values.

Well, in that case, let's just have it be a non-nullable field, and add
a note to our code that missing fields *can* cause item saving to fail!
(This was always true, but I'm just extra-noting it because it's
becoming *more* true.)
2024-05-02 13:00:10 -07:00
156cabbab4 Add shadowban mechanism for closet lists
Simple enough to start! If `shadowbanned: true` gets set on a user,
then we show a 404 instead of the actual list page, *unless* you're
logged in as that user, or coming from a known IP of that user.

This isn't a very strong mechanism! Just something to hopefully
increase the costs of messing around with list spam.
2024-04-20 20:57:15 -07:00
5cc219c795 Connect a NeoPass to an existing account
including validation logic to make sure it's not already connected to
another one!

The `intent` param on the NeoPass form is part of the key! Thanks
OmniAuth for making it easy to pass that data through!
2024-04-08 05:33:58 -07:00
d5c3bc087e Track neopass_email when logging in with NeoPass
Gonna use this in the Settings UI to communicate what NeoPass you're
connected to!
2024-04-07 07:17:07 -07:00
b03d9b264a Increase maximum username length to 30
I'm writing some code for default NeoPass usernames, and they can get
kinda long, so I want to clear some extra space for them!
2024-04-01 05:53:38 -07:00
7d18da01d8 Update schema.rb for ID column limit increase
Commit 07617fa34f was an emergency commit
while I was on a trip, away from my usual workstation! I was able to
write the migration and run it against production, but I didn't have
the dev server fully set up, so I wasn't able to run the migration in
development, which is when Rails updates `schema.rb`.

Now I'm home and can run it, easy peasy! `rails db:migrate`
2024-03-30 23:38:04 -07:00
07617fa34f Increase ID column limit for item_outfit_relationships
Ahh wow, we're finally at the point of having enough outfits to need to
increase this ID field!

I got an alert this morning that queries were failing with the MySQL error
"id out of range". I went and found that creating a new outfit still works if
it's empty, but fails once you try to add an item to it. So, I assume this is
the failing column!

Note that I'm writing this from my emergency Steam Deck workstation, and that
I didn't quite get MySQL set up on it yet (I just yolo ran this in
production), so we don't have the corresponding changes to `db/schema.rb` yet.
This might require another commit from my normal workstation later to
complete this change!
2024-03-24 11:09:40 +11:00
3eeb5d1065 Actually create user from NeoPass authentication! <3 <3
Whew, exciting! Still done nothing against the live NeoPass server, but
we've got this fully working with the development server, it seems!
Wowie!!

This is all still hidden behind secret flags, so it's fine to deploy
live. (And it's not actually a problem if someone gets past to the
endpoints behind it, because we haven't actually set up real
credentials for our NeoPass client yet, so authentication will fail!)

Okay time to lie down lol.
2024-03-14 19:11:06 -07:00
08b1b9e83b Add OmniAuth plugin to AuthUser
This is setting us up for NeoPass, but first we're just gonna try stuff
with the "developer" strategy that's built in for testing, rather than
using the NeoPass dev server!
2024-03-14 15:06:13 -07:00
ec6dca1c16 Improve Unicode support, emojis don't crash us anymore lol!
A few pieces here:

1. Convert all tables to `utf8mb4`+`utf8mb4_unicode_520_ci` strings.
2. Configure that as the server's default.
3. Configure the Rails database connection to use this encoding too.

Came together pretty well, whew! This has been a LONG time coming,
`latin1` is NOT a good charset for the year 2024!
2024-02-28 18:54:27 -08:00
8f78892212 Remove item_translations from schema.rb??
Yeah what the heck, why is this here, we have the migration to drop it
and it's already dropped in production!

Y'know, sometimes I goof a migration and it sets things in a weird
state in development, maybe we didn't recover correctly from that or
something? But idk how we would have goofed this one. Whatever! I've
manually dropped it from my development machine, and it was already
correctly dropped on production, so, go figure!
2024-02-28 18:53:48 -08:00
371da73615 Revert "Oops, add AltStyle#series_name db change to schema"
This reverts commit cc339672b1.

Oh, wait, no, the state the schema file was in *was* correct. I'm not
sure why… huh ok, I need to debug my local state.
2024-02-28 17:20:43 -08:00
cc339672b1 Oops, add AltStyle#series_name db change to schema
Did we end up behind on this migration in development? Idk! Weird! I'm
doing other migration stuff now and noticing this change slipping in
and I'm like. Huh! You should've already been there!
2024-02-28 15:53:48 -08:00
18c7a34b8f Update series_name for alt styles to be null, with a fallback string
I considered this at first, but decided to keep it simple until it
turned out to matter. Oops, it already matters, lol!

I want the item search code to be able to easily tell if the series
name is real or a placeholder, so we can decide whether to build the
filter text in `fits:$series-$color-$species` form or
`fits:alt-style-$id` form.

So in this change, we keep it that `AltStyle#series_name` returns the
placeholder string if none is set, but callers can explicitly ask
whether it's a real series name or not. Will use this in our next
change!
2024-02-27 15:48:28 -08:00
9f74e6020e Add series_name database field to alt styles
Previously we did this hackily by comparing the ID to a hardcoded list
of IDs, but I think putting this in the database is clearer and more
robust, and it should also help with our upcoming item search stuff
that will filter by it!
2024-02-27 15:28:05 -08:00
a684c915a9 Track when manifest was last loaded, and what status it returned
Now we're *really* duplicating with Impress 2020's system lol, but I
need a way to not keep trying to load manifests that are actually 404,
which are surprisingly plentiful!

This doesn't actually stop us from loading anything yet, it just tracks
the timestamps and the HTTP status! But next I'll add logic to skip
when it was 4xx recently.
2024-02-25 15:35:04 -08:00
c8e53165c7 Drop item_translations table
Okay cool, we're successfully migrated off translations, we can delete
the table now!

I'm not worried about backing up this data as such, because the
impress-2020 repo has a bunch of this data in its
`public-data-from-modeling.sql.gz` file history. Safe to remove from
the live app!
2024-02-20 17:01:54 -08:00
b7296d6a75 Fix default value for Impress 2020's modeling_logs.created_at 2024-02-20 16:46:19 -08:00
a1066d9c8a Add translated item fields directly to the Item model
Like with Species, Color, and Zone, we're moving the translation data
directly onto the model, and just using English. This will simplify
some of our queries a lot (way fewer joins!), and it's what Neopets
does now anyway, and I have a secret hope that removing the complexity
along the codepath for `item.name` might help speed up large item lists
if we're lucky?? 🤞

Anyway, this is the first step, performing the migration to copy the
data onto the `items` table, making sure to keep them in sync for the
2020 app for now!
2024-02-20 15:25:03 -08:00
d39e7cea81 Move fundraising models into the Fundraising module
This was mostly straightforward it seems, whew!
2024-02-18 20:29:31 -08:00
d6b6ab884d Migration to drop {color,species,zone}_translations
They're no longer referenced in either app! Begone!

(The translated values are still available in the DTI 2020 repository's
history, under `scripts/db`!)
2024-02-03 08:20:18 -08:00
c60e222faa Add Alt Style support to outfit saving
Pretty straightforward, just add the field to the record, and wire it
all up! I'm glad this seemed to work out pretty well all-in-all 😅
2024-02-01 05:55:19 -08:00
4fff8d88f2 Add support_staff flag to user record; they can use Support tools
A little architecture trick here! DTI 2020 authorizes support staff
requests by means of a secret token, instead of user account stuff. And
our support tools still all call DTI 2020 APIs.

So here, we bridge the gap: we copy DTI 2020's support secret to this
app's environment variables (I needed to update
`deploy/files/production.env` and run `bin/deploy:setup` for this!),
then users with the new `support_secret` flag have it added to their
HTML documents in the meta tags. Then, the JS reads the meta tag.

I also fixed an issue in the `deploy/setup.yml` playbook, where I had
temporarily commented some stuff out to skip steps one time, and forgot
to uncomment them after oops lol!
2024-01-29 04:21:19 -08:00
5004142dfb Add alt style support to modeling
Nothing to show them yet, but I think this works for loading it all in
the first place?

Still needs contributions tho!
2024-01-24 03:25:23 -08:00
73af9d4d77 Start migrating off globalize gem for zones
Like in 0dca538, this is preliminary work for being able to drop the
`zone_translations` table! We're copying the field over first, to be
able to migrate DTI 2020 safely before dropping anything.
2024-01-23 05:43:00 -08:00
413d2eed91 Improve species/color name migration performance & correctness
Oh right, this should only run in the up direction! And also we can get
a lot faster by preloading the translations.
2024-01-23 05:30:42 -08:00
0dca538b0a Start migrating off globalize gem for species/color names
Non-English languages haven't been supported on Neopets for a while, so
I'd like to remove this extra cross-cutting complexity, especially
since it's now inconsistent with the real site anyway!

The main motivation is that I'd like to do this for items too, because
I have a hunch that all the complexity of `globalize` to read
`item.name` is part of what's making large user lists so slow to
render: lots of little objects getting created down the stack, and
needing to be garbage-collected later.

I'm not sure that's why! But I figure removing this complexity is a
simplicity win anyway, so let's do it!

Note that this doesn't *finish* the migration, it just starts it! The
`Species::Translation` and `Color::Translation` models still exist, and
still have their data, and not all references to them are scrubbed yet.
I especially don't want to delete the backing tables until both DTI and
DTI 2020 are ready for it!

So this change will someday be paired with another change to actually
drop the tables - after backing up the data for future records just in
case, of course!
2024-01-23 05:10:43 -08:00
470c805880 Save last trade activity time onto User
In impress-2020, we do a big slow query to figure out which users have
been active in trades recently. Now, we cache that timestamp on the
User model.

This won't have any immediate effect; it's to clear the way for Classic
DTI to receive the better trade ratios feature people like from 2020.

I also added some unit testing infra because I finally wanted it! for
all the ways you can trigger this timestamp lol

Note too that this is a bit of an unusually complex migration, but my
hope is that the batching and query structure and such helps it run
surprisingly fast! 🤞
2024-01-19 00:00:46 -08:00
c725a11920 Remove unused NC Mall spider code & fields
We haven't used the mall spider in this app in forever (I guess we even
deleted the code at some point?), but there was some vestigial stuff
left. Goodbye!
2023-11-11 15:45:38 -08:00
644a6acc72 Remove old Image Mode stuff
There was a static page explaining it, which we no longer link to; and
there was an unused field in the User model for who was a beta tester
for it. Goodbye!
2023-11-11 15:21:49 -08:00
dc22a458bf Move manifest backfill to swf_assets:manifests task
Okay, I've simplified the migration to *just* add the column, and
instead added a task to find assets without manifest URLs and backfill
them.

Performance is a lot better now, using the `async-http` library, which
as I understand it supports both persistent connections when invoked
like this, and maybe also HTTP/2 multiplexing?? (Though I'm not
actually sure images.neopets.com does lol)

I'm not sure about the number of concurrent tasks I picked here, 100
seems okay for an internet thing and for such small requests, but I
worry that the CDN is gonna get annoyed or something. Well, we'll see!
This task is very resumable if it turns out we get frozen out or
something.
2023-11-10 16:52:50 -08:00
96998643b5 Add manifest_url to swf_assets table
Ok so, impress-2020 guesses the manifest URL every time based on common
URL patterns. But the right way to do this is to read it from the
modeling data! But also, we don't have a great way to get the modeling
data directly. (Though as I write this, I guess we do have that
auto-modeling trick we use in the DTI 2020 codebase, I wonder if that
could work for this too?)

So anyway, in this change, we update the modeling code to save the
manifest URL, and also the migration includes a big block that attempts
to run impress-2020's manifest-guessing logic for every asset and save
the result!

It's uhh. Not fast. It runs at about 1 asset per second (a lot of these
aren't cache hits), and sometimes stalls out. And we have >600k assets,
so the estimated wall time is uhh. Seven days?

I think there's something we could do here around like, concurrent
execution? Though tbqh with the nature of the slowness being seemingly
about hitting the slow underlying images.neopets.com server, I don't
actually have a lot of faith that concurrency would actually be faster?

I also think it could be sensible to like… extract this from the
migration, and run it as a script to infer missing manifest URLs. That
would be easier to run in chunks and resume if something goes wrong.
Cuz like, I think my reasoning here was that backfilling this data was
part of the migration process… but the thing is, this migration can't
reliably get a manifest for everything (both cuz it depends on an
external service and cuz not everything has one), so it's a perfectly
valid migration to just leave the column as null for all the rows to
start, and fill this in later. I wish I'd written it like that!

But anyway, I'm just running this for now, and taking a break for the
night. Maybe later I'll come around and extract this into a separate
task to just try this on all assets missing manifests instead!
2023-11-09 21:42:51 -08:00
b8a8cb9b20 Stop orphaning hangers when deleting lists
Idk if this used to be different or what, but it looks like the current
behavior is: if you delete a closet list, it'll leave the hangers
present, but Classic DTI would not show them anywhere; but Impress 2020
(until recently) would crash about it.

Now, we use `dependent: :destroy` to delete the hangers when you delete
the list (which I think makes sense, and is different than what I
decided in the past but that's ok, and is what the current behavior
*looks* like to people!), and we add a migration that deletes orphaned
hangers.

The migration also outputs the deleted hangers as JSON, for us to hold
onto in case we made a mistake! I'm also backing up the database in
advance of running this migration, just in case we gotta roll back HARD!
2023-10-24 15:35:21 -07:00
e79428fa28 Add Remember Me to login
This requires a migration, our first migration against the openneo_id database from this app! Fun!
2023-10-23 19:05:08 -07:00
b2a027fbbf Oops, should've used migration version 4.2
Oh I didn't realize the lowest version Rails had for this is 4.2. I wish running `rake db:migrate` checked this, but I'm running into it on another branch when I try to create a *new* migration which for some reason leads it to inspect the old migrations and notice the issue. Weird!
2023-10-23 19:05:08 -07:00
60284a87cc Update schema.rb format
Running a no-op `rake db:migrate` happened to reformat this file to be more modern. Ok neat!
2023-10-23 19:05:08 -07:00
fcf3292448 Add Rails version to old migrations
I'm not sure it's literally true that they were all built against Rails 3.2, but that's what it was at before we upgraded, and like. that's probably fine
2023-10-23 19:05:08 -07:00
75185de957 Create a User when we create an AuthUser
We also update seeds.rb to be up to date! This should make it possible to log in as test/test123 from a fresh database.
2023-10-23 19:05:07 -07:00
1d5af835a5 Add AuthUser model, connecting to openneo_id db
No user-facing functionality here yet, just configuring the database connection to work with openneo_id records.

This is a first step in integrating Devise stuff into this app instead of connecting with a weird second app.

My basic testing for this was to temporarily connect to production `openneo_id`, and see `AuthUser.first` correctly return a user!
2023-10-23 19:05:07 -07:00
7e922503b5 Upgrade to Rails 7.0.6
Whew! Seems like a pretty clean one? Ran `rails app:upgrade` and stuff, and made some corrections to keyword arguments for `translate` calls. There might be more such problems elsewhere? But that's hard to search for, and we'll have to see.
2023-10-23 19:05:07 -07:00
Matchu
38f3c9894c Delete WardrobeTip model
lmao I have zero recollection of this, I've been generally trying to avoid too-far-off-the-path cleanup, but this one just seems silly, goodbye
2023-10-23 19:05:05 -07:00
Matchu
5da4006053 Set up development database in Vagrant
Just a lil bit of extra provisioning and config! We also added some more to the `seeds.rb` task, to set up records that the app expects to exist.
2023-10-23 19:05:04 -07:00
Matchu
8aa4aa6e30 Drop NewsPosts model
We never ended up really using this lol, and it has an attr_accessible I want to be rid of so let's just fully trash it while we're at it!
2023-10-23 19:05:04 -07:00
Matt Dunn-Rankin
c7d60e3e41 Add db field to override special color
Okay, fine, finally making this controllable from the db without requiring a deploy :P Setting this new field will cause `item.special_color` to return the corresponding color. This mainly affects what we show on the item page, and what colors we request for modeling on the homepage.
2018-05-09 14:10:20 -07:00
Matt Dunn-Rankin
f5fb8711fc improved donation campaign nav 2017-01-05 13:06:50 -08:00
Matchu
b5442d0f3f news posts read from the database, with a campaign-supplement classname option :) 2015-09-26 20:37:03 -07:00
Matchu
2ca784c3a2 thank-you message automatically deploys on campaign success 2015-09-22 22:40:51 -07:00
d8038f2fbf prefer scraped rainbow pool images over pet images 2015-09-05 18:48:41 +00:00
Matchu
5d2c226357 yum, new campaigns 2015-08-05 19:26:12 -04:00