1
0
Fork 0
forked from OpenNeo/impress
Commit graph

123 commits

Author SHA1 Message Date
08b1b9e83b Add OmniAuth plugin to AuthUser
This is setting us up for NeoPass, but first we're just gonna try stuff
with the "developer" strategy that's built in for testing, rather than
using the NeoPass dev server!
2024-03-14 15:06:13 -07:00
684dcb53ba Add Turbo to speed up the app, and set up for missing UJS features
Oh right, we don't have Rails UJS going on anymore, which is what
handled the confirmation prompts for deleting lists. Turbo is the more
standard modern solution to that, and should speed up certain
pageloads, so let's do it!

Here I install the `turbo-rails` gem, then run `rails turbo:install` to
install the `@hotwired/turbo-rails` npm package. Then I move
`application.js` that's run all on pages but the outfit editor into our
section of JS that gets run through the bundler, and add Turbo to it.

I had to fix a couple tricky things:

1. The outfit editor page doesn't play nice with being swapped into the
   document, so I make it require a full page reload instead.
2. Prefetching the Sign In link can cause the wrong `return_to` address
   to be written to the `session`. (It's a GET request that does, ever
   so slightly, take its own actions, oops!) As a simple hacky answer,
   we disallow prefetching on that link.

Haven't fixed up the UJS stuff for confirm prompts to use Turbo yet,
that's next!
2024-03-13 13:43:48 -07:00
8dc11f9940 Create rails public_data:commit task, to share public data dumps
I'm starting to port over the functionality that was previously just,
me running `yarn db:export:public-data` in `impress-2020` and
committing it to Git LFS every time.

My immediate motivation is that the `impress-2020` git repository is
getting weirdly large?? Idk how these 40MB files have blown up to a
solid 16GB of Git LFS data (we don't have THAT many!!!), but I guess
there's something about Git LFS's architecture and disk usage that I'm
not understanding.

So, let's move to a simpler system in which we don't bind the public
data to the codebase, but instead just regularly dump it in production
and make it available for download.

This change adds the `rails public_data:commit` task, which when run in
production will make the latest available at
`https://impress.openneo.net/public-data/latest.sql.gz`, and will also
store a running log of previous dumps, viewable at
`https://impress.openneo.net/public-data/`.

Things left to do:
1. Create a `rails public_data:pull` task, to download `latest.sql.gz`
   and import it into the local development database.
2. Set up a cron job to dump this out regularly, idk maybe weekly? That
   will grow, but not very fast (about 2GB per year), and we can add
   logic to rotate out old ones if it starts to grow too far. (If we
   wanted to get really intricate, we could do like, daily for the past
   week, then weekly for the past 3 months, then monthly for the past
   year, idk. There must be tools that do this!)
2024-02-29 14:30:33 -08:00
45579167dc Run bundle update
I also put in a manual bump for `falcon`!

The motivation is that I'm working on a Ruby 3.3.0 upgrade in another
branch, and I'm getting deprecation warnings from the `async` gem,
which I think are resolved in the latest version, so I figure, hey,
good time for an update!
2024-02-22 11:48:32 -08:00
ec1cd65a2e Upgrade to Rails 7.1.3.2
In response to:
- https://discuss.rubyonrails.org/t/possible-xss-vulnerability-in-action-controller/84947
- https://discuss.rubyonrails.org/t/possible-redos-vulnerability-in-accept-header-parsing-in-action-dispatch/84946

No evidence that our app has been affected, but I like to move quickly
on these!
2024-02-22 11:25:52 -08:00
f85703d2cc Remove now-unused globalize dependency
We previously used this to translate fields on some models: Color,
Species, Zone, and Item.

We no longer do, so we can safely remove the gem!
2024-02-20 16:58:39 -08:00
7cf3786023 Uninstall puma gem
Falcon works in development too, puma no longer needed!
2024-01-29 01:04:31 -08:00
76af587e7c Replace falcon server with puma
Been wanting this for a while in theory, gonna actually do it now!

The motivation is that I want to turn up the timeout for loading pets,
because the Neopets endpoints are slower today with the NC UC release -
but I can already predict that under our current architecture that will
be a problem, because it'll block up our request queue!

Falcon uses Ruby's relatively-new async system to *not* have requests
block on upstream requests, and my understanding is that this behavior
is plug-and-play. Let's see how it goes!
2024-01-23 21:55:26 -08:00
470c805880 Save last trade activity time onto User
In impress-2020, we do a big slow query to figure out which users have
been active in trades recently. Now, we cache that timestamp on the
User model.

This won't have any immediate effect; it's to clear the way for Classic
DTI to receive the better trade ratios feature people like from 2020.

I also added some unit testing infra because I finally wanted it! for
all the ways you can trigger this timestamp lol

Note too that this is a bit of an unusually complex migration, but my
hope is that the batching and query structure and such helps it run
surprisingly fast! 🤞
2024-01-19 00:00:46 -08:00
d2de971a60 Delete more rake tasks
I tried to port the Rainbow Pool ones forward, but ran into issues with the
service that uses browser-specific stuff to check that traffic is valid :/

Incidentally, those were the only places we were using `rest-client`.
Goodbye!
2023-11-10 18:59:46 -08:00
dc22a458bf Move manifest backfill to swf_assets:manifests task
Okay, I've simplified the migration to *just* add the column, and
instead added a task to find assets without manifest URLs and backfill
them.

Performance is a lot better now, using the `async-http` library, which
as I understand it supports both persistent connections when invoked
like this, and maybe also HTTP/2 multiplexing?? (Though I'm not
actually sure images.neopets.com does lol)

I'm not sure about the number of concurrent tasks I picked here, 100
seems okay for an internet thing and for such small requests, but I
worry that the CDN is gonna get annoyed or something. Well, we'll see!
This task is very resumable if it turns out we get frozen out or
something.
2023-11-10 16:52:50 -08:00
18ff22f211 Add Sentry to Rails
Now we're tracking both JS and Rails errors, phew!
2023-11-06 12:37:40 -08:00
c496e33c37 Add mini profiler to each page
It shows up in development always, and if you're logged in as Me
Specifically in production!

I'm using this to poke at memory usage for pages that seem suspicious.
I don't know why our app reliably grows so large in RAM, but my hunch is
that maybe there are some pages that just use a truly large amount to
begin with - and I've learned Ruby doesn't release memory back after
it's GC'd, it just grows the process and keeps the free space to itself
in its own heap!

So I'm just eyeing pages that I know *can* have a lot going on, and
seeing what I find!
2023-10-27 19:38:49 -07:00
13371e3cf2 Remove unused automated testing files & gems
Look, I'll be real, I have literally not run these automated tests in
probably like a whole decade. Most of these files are empty, the ones
that aren't seem basically trivial, and I bet half of it would fail
anyway.

If I wanted to do real automated testing, I would basically want to
start from scratch anyway, and apply coverage I can trust to the areas
I actually care about.

Until then, I feel like these gems and files are mostly just clutter,
and I don't like them being One More Barrier To Entry. Goodbye, unused
complexity!
2023-10-25 16:09:01 -07:00
56ce32b6cb Upgrade to Rails 7.1.1
The usual stuff! Installed the new gem and its new deps, ran
`bin/rails app:update` and did my best to manually merge the dev/prod
config files with the new canonical defaults, deleted some migrations I
don't think are relevant to us, and yeah!

Also, Rails 7.1 seems to need `libyaml-dev` installed, so I added that
to the `deploy/setup.yml` playbook!

One thing to note is that, while I was here, I turned on some settings
relating to our use of SSL that technically weren't on before. This
should be fine and helpful? But if stuff breaks, well, check those!
2023-10-25 15:05:31 -07:00
661fbd2d03 Security updates
Resolves CVE-2023-40175 and CVE-2023-38037.
2023-10-23 19:08:56 -07:00
8bd7ca167c Remove leftover Capistrano stuff
We were using that for deployment before, and now we're using my Ansible recipes!
2023-10-23 19:05:09 -07:00
74197a6e9f Upgrade to latest Sass and Sprockets
This required a buncha fixes to how SASS scoping works! Needed to add a bunch of imports for stuff that previously would get read from the global scope by being imported *after* the constants and mixins etc.

There's clearly a lot of refactor opportunity here, but I'm not gonna worry about it!!
2023-10-23 19:05:09 -07:00
15002d19db Remove compass-rails
I wasn't sure what we were actually using it for, turns out it was mostly polyfills for CSS features that are very standard now!

I didn't audit these changes very carefully tbqh because they seemed pretty simple? Fingers crossed!
2023-10-23 19:05:09 -07:00
9b68e982e7 Precompile assets when deploying new version
I did some refactoring while here too, of pulling the deploy scripts out of `package.json` and into `bin`, to be a bit more canonically Rails-y. (idk how canonical the colon thing is but, probably fine??)
2023-10-23 19:05:09 -07:00
4ddcb005ea Remove memcache from production config
I don't know enough about our caching situation to know where memcache performs meaningfully better than Rails's in-memory cache. Let's delete it for now and see if there's a problem, to simplify the deploy environment!
2023-10-23 19:05:09 -07:00
6b8fc6407e Use puma web server
Works in dev, and people seem to like it for prod! (I'm working on some deploy stuff is what.)
2023-10-23 19:05:09 -07:00
81b2a2b4a2 Bundle wardrobe-2020 into the app
We add jsbuilding-rails to get esbuild running in the app, and then we copy-paste the files we need from impress-2020 into here!

I stopped at the point where it was building successfully, but it's not running correctly: it's not sure about `process.env` in `next`, and I think the right next step is to delete the NextJS deps altogether and use React Router instead.
2023-10-23 19:05:08 -07:00
83f80facda Can log into OpenNeo ID accounts directly!
A lot of rough edges here (e.g. no styles on the flash messages), but it's working and that's good!!

I tested this by temporarily switching to the production database and logging in as matchu!

Still missing a lot of big features too, like registration, password resets, settings page, etc.
2023-10-23 19:05:07 -07:00
700e26d7df Remove old OpenNeo ID auth code
This removes login/logout/session logic for integrating with OpenNeo ID, replacing them with stubs that just redirect to `/?TODO` when you click login, and helpers that act as if you're not logged in.

This gives us a clean slate to plug in new Devise logic to integrate with the `openneo_id` database directly!
2023-10-23 19:05:07 -07:00
7e922503b5 Upgrade to Rails 7.0.6
Whew! Seems like a pretty clean one? Ran `rails app:upgrade` and stuff, and made some corrections to keyword arguments for `translate` calls. There might be more such problems elsewhere? But that's hard to search for, and we'll have to see.
2023-10-23 19:05:07 -07:00
02a9afbb74 Upgrade to Ruby 3.0.6
Hey nice! We have to add webrick now because it's not included in Ruby 3, but hey just drop it right back in.

Idk how to choose between this or puma or whatever, but in the absence of a specific reason let's just pick the one whose name I know best.
2023-10-23 19:05:07 -07:00
59efb49419 Upgrade to Rails 6.1.7.4
This one was pretty straightforward yaay! Main thing was the change from `render file` to `render template` in a couple places, oh and a thing with complex `order()` clauses.
2023-10-23 19:05:07 -07:00
eaf43128ba Add web console support for Vagrant users 2023-10-23 19:05:06 -07:00
be7e11a0d0 Upgrade to Rails 6.0.6.1
Another pretty easy one! We have the `rails app:update` changes in here too.
2023-10-23 19:05:06 -07:00
bb5065deed Update all our gems as far as we can
Got pretty far, there's a couple still held back by Rails 6+, but yeah doing great!
2023-10-23 19:05:06 -07:00
bc3a9bc33b Fix gems we'd locked to old versions
Now that we're on Ruby 2.6, the constraints on the latest versions of these dependencies are satisfied, so we can stop holding them back!
2023-10-23 19:05:06 -07:00
fc34208db9 Upgrade web-console
Huh, error pages weren't actually working on this older version of web-console, but now they are!

Solution found here: https://discuss.rubyonrails.org/t/question-dev-mode-500-errors-not-displayed-config-consider-all-requests-local-not-honored/78453/7
2023-10-23 19:05:05 -07:00
86edc8584f Run rails app:update
We accepted some changes as-is, but for development.rb and production.rb we read the diff and manually edited them!
2023-10-23 19:05:05 -07:00
d97c32b5da Upgrade to Rails 5.2.8.1
Some important little upgrades but mostly straightforward!

Note that there's still a known issue where item searches crash, I was hoping that this was a bug in Rails 4.2 that would be fixed on upgading to 5, but nope, oh well!

Also uhh I just got a bit silly and didn't actually mean to go all the way to 5.2 in one go, I had meant to start at 5.0… but tbh the 5.1 and 5.2 changes seem small, and this seems to be working, so. Yeah ok let's roll!
2023-10-23 19:05:05 -07:00
685ac2b4ad Make 4.1 -> 4.2 changes
Just following the recommended stuff in the upgrade guide!
2023-10-23 19:05:05 -07:00
f80d220a62 Upgrade to Rails 4.2.11.3 and Ruby 2.4.10
Some tricks required here to get the dependencies to work out, but we got it!!

Oh also, we move away from the rbenv in Ubuntu's package manager, because it doesn't support more recent Rubies like 2.4.10.
2023-10-23 19:05:05 -07:00
4091ce2a5b Remove dotenv-deployment, upgrade dotenv-rails
Idk exactly what's going on with dotenv-deployment, if it turns out it was critical to our deploy process then we'll change the deploy process! It's deprecated and conflicts with gem deps for `dotenv-rails`.
2023-10-23 19:05:05 -07:00
aa001f1e32 Uninstall passenger_monit
I'm not exactly sure what it was doing, whatever bye
2023-10-23 19:05:05 -07:00
375753ef89 Version the http_accept_languages gem
idk why we were pulling from git before, probably to get a feature that wasn't pushed to rubygems yet? But now the latest rubygems version matches the latest repo commit (both back in 2017 lol), so let's do that for greater stability and clarity!
2023-10-23 19:05:05 -07:00
98cbaa0c96 Delete character_encodings gem used only in 1.8
We're not on Ruby 1.8 anymore, so this never happens anyway!
2023-10-23 19:05:05 -07:00
c4f3a472ff Remove RightAWS gem
We're not connecting to AWS directly anymore, now that we deleted the SWF conversion stuff, so we can finally clear this out!
2023-10-23 19:05:05 -07:00
Matchu
fd263ea82f Remove mall spider cron jobs
I don't think these work anymore, and our volunteers get new items into the db fast anyway, Impress 2020 is doing better spidering these days. And then we get to remove the cron job `whenever` gem!
2023-10-23 19:05:05 -07:00
Matchu
b0cceb8ce7 Uninstall Airbrake
Haven't checked it in forever, if we want an error reporting solution we'll create it fresh!
2023-10-23 19:05:05 -07:00
Matchu
fde8ec0c3b Uninstall mini_magick
I think this was probably part of the swf converters or something? Seems unused now though!
2023-10-23 19:05:05 -07:00
Matchu
31ebf7d4eb Remove OutfitImageUploader and Carrierwave
Ok right, missed this in the outfit image stuff deletion, get rid of the code and library we were using to push those images up to S3!
2023-10-23 19:05:05 -07:00
Matchu
1195a6190b Uninstall resque
Yay, we've deleted all our background tasks!

We'll probably want to replace some of the basic functionality like certain caching? But we can deal with that as we run into it.

The direct motivation here was a seeming version conflict between Rails 4.2's rack dependency and latest Resque's rack dependency... but this is just nice complexity elimination regardless, we want this anyway :3
2023-10-23 19:05:04 -07:00
Matchu
e121d8bba2 Remove SWF conversion
We've already swapped out the backend for this stuff to Impress 2020, so the resque task and the broken image report UI aren't actually relevant anymore. Delete them!

This helps us delete Resque soon too.
2023-10-23 19:05:04 -07:00
Matchu
298404208f Remove rails-observers
We deleted all the observers, so now we can delete this too!
2023-10-23 19:05:04 -07:00
Matchu
97bf520674 Update gems to Rails 4.1 (crashes!)
Lookin' good on this end, but the server isn't starting up yet, got some fixes to make!
2023-10-23 19:05:04 -07:00