OpenNeo/impress
OpenNeo/impress
1
0
Fork 1
Code Pull requests Activity
impress/Gemfile

92 lines
2.7 KiB
Ruby
Raw Normal View History

Update most URLs to use HTTPS I noticed we didn't have the little lock icon in the browser, and yeah huh there's a lot of `http://` still floating around! Let's fix that!
2023-10-25 15:16:46 -07:00
source 'https://rubygems.org'
Update Ruby to 3.3.4, and update gems For the gems, I mostly just ran `bundle update`; with the exception of `httparty`, because latest Ruby throws a deprecation warning about its use of the deprecated `csv` stdlib, which the latest version resolves. One other little thing: this is on my new Fedora workstation, and I had to deal with a known bug where the `sassc` gem compiles a `libsass.so` file, but saves it in the wrong place somehow. Here's the known bug, and the comment that helped me: https://github.com/sass/sassc-ruby/issues/146#issuecomment-2028974524 And here's what I ran to get it into the right place: ```shell ln -s ~/.local/share/gem/ruby/3.3.0/extensions/aarch64-linux/3.3.0/sassc-2.4.0/sassc/libsass.so \ ~/.local/share/gem/ruby/3.3.0/gems/sassc-2.4.0/lib/sassc/libsass.so ``` This thing about `libsass` isn't reflected in the code changes anywhere in this commit! I'm just mentioning it so that it's literally written down anywhere. (I did try other comments' advice to use an older version of `sassc` first, but I ran into compilation errors, so figured this machine-side hack was better than untangling that mess.)
2024-08-27 17:06:22 -07:00
ruby '3.3.4'
rails 3
2010-05-14 15:12:31 -07:00
Upgrade to Rails 7.1.3.4 Motivated by a security patch, I don't think we're affected by it but whatever, it's a convenient time to get it done!
2024-06-04 11:32:51 -07:00
gem 'rails', '~> 7.1', '>= 7.1.3.4'
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
Upgrade to Ruby 3.0.6 Hey nice! We have to add webrick now because it's not included in Ruby 3, but hey just drop it right back in. Idk how to choose between this or puma or whatever, but in the absence of a specific reason let's just pick the one whose name I know best.
2023-08-03 16:15:23 -07:00
# The HTTP server running the Rails instance.
Upgrade async and related gems, and fix async-http response handling When playing with a Rainbow Pool syncing task, I noticed that error handling wasn't working correctly for requests using `async-http`: if the block raised an error, the `Sync` block would never return. My suspicion is that this is because we were never reading or releasing the request body. In this change, I upgrade all the relevant gems for good measure, and switch to using the response object yielded by the _block_, so we can know it's being resource-managed correctly. Now, failures raise errors as expected! (I tested all these relevant service calls, too!)
2024-09-07 12:14:12 -07:00
gem 'falcon', '~> 0.48.0'
Upgrade to Ruby 3.0.6 Hey nice! We have to add webrick now because it's not included in Ruby 3, but hey just drop it right back in. Idk how to choose between this or puma or whatever, but in the absence of a specific reason let's just pick the one whose name I know best.
2023-08-03 16:15:23 -07:00
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# Our database is MySQL, in both development and production.
Upgrade to Rails 5.2.8.1 Some important little upgrades but mostly straightforward! Note that there's still a known issue where item searches crash, I was hoping that this was a bug in Rails 4.2 that would be fixed on upgading to 5, but nope, oh well! Also uhh I just got a bit silly and didn't actually mean to go all the way to 5.2 in one go, I had meant to start at 5.0… but tbh the 5.1 and 5.2 changes seem small, and this seems to be working, so. Yeah ok let's roll!
2023-08-02 16:05:02 -07:00
gem 'mysql2', '~> 0.5.5'
support concurrent requests. now demands thin and nginx, but hey. it works
2010-10-10 15:46:58 -07:00
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For reading the .env file, which you can use in development to more easily
# set environment variables for secret data.
Remove dotenv-deployment, upgrade dotenv-rails Idk exactly what's going on with dotenv-deployment, if it turns out it was critical to our deploy process then we'll change the deploy process! It's deprecated and conflicts with gem deps for `dotenv-rails`.
2023-08-02 13:41:37 -07:00
gem 'dotenv-rails', '~> 2.8', '>= 2.8.1'
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For the asset pipeline: templates, CSS, JS, etc.
Upgrade to latest Sass and Sprockets This required a buncha fixes to how SASS scoping works! Needed to add a bunch of imports for stuff that previously would get read from the global scope by being imported *after* the constants and mixins etc. There's clearly a lot of refactor opportunity here, but I'm not gonna worry about it!!
2023-08-22 16:08:11 -07:00
gem 'sprockets', '~> 4.2'
Upgrade haml I was hoping this would fix a character encoding bug, it didn't really? but it doesn't seem to hurt and is probably good to do
2023-07-22 12:58:15 -07:00
gem 'haml', '~> 6.1', '>= 6.1.1'
Upgrade to latest Sass and Sprockets This required a buncha fixes to how SASS scoping works! Needed to add a bunch of imports for stuff that previously would get read from the global scope by being imported *after* the constants and mixins etc. There's clearly a lot of refactor opportunity here, but I'm not gonna worry about it!!
2023-08-22 16:08:11 -07:00
gem 'sass-rails', '~> 6.0'
Precompile assets when deploying new version I did some refactoring while here too, of pulling the deploy scripts out of `package.json` and into `bin`, to be a bit more canonically Rails-y. (idk how canonical the colon thing is but, probably fine??)
2023-08-19 18:51:46 -07:00
gem 'terser', '~> 1.1', '>= 1.1.17'
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
gem 'react-rails', '~> 2.7', '>= 2.7.1'
Move jsbundling-rails higher up in Gemfile Oh I guess I ranthe thing that put it at the bottom!
2023-08-19 17:07:17 -07:00
gem 'jsbundling-rails', '~> 1.1'
Add Turbo to speed up the app, and set up for missing UJS features Oh right, we don't have Rails UJS going on anymore, which is what handled the confirmation prompts for deleting lists. Turbo is the more standard modern solution to that, and should speed up certain pageloads, so let's do it! Here I install the `turbo-rails` gem, then run `rails turbo:install` to install the `@hotwired/turbo-rails` npm package. Then I move `application.js` that's run all on pages but the outfit editor into our section of JS that gets run through the bundler, and add Turbo to it. I had to fix a couple tricky things: 1. The outfit editor page doesn't play nice with being swapped into the document, so I make it require a full page reload instead. 2. Prefetching the Sign In link can cause the wrong `return_to` address to be written to the `session`. (It's a GET request that does, ever so slightly, take its own actions, oops!) As a simple hacky answer, we disallow prefetching on that link. Haven't fixed up the UJS stuff for confirm prompts to use Turbo yet, that's next!
2024-03-13 13:43:48 -07:00
gem 'turbo-rails', '~> 2.0'
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For authentication.
Upgrade to Rails 4.2.11.3 and Ruby 2.4.10 Some tricks required here to get the dependencies to work out, but we got it!! Oh also, we move away from the rbenv in Ubuntu's package manager, because it doesn't support more recent Rubies like 2.4.10.
2023-08-02 15:18:36 -07:00
gem 'devise', '~> 4.9', '>= 4.9.2'
Can log into OpenNeo ID accounts directly! A lot of rough edges here (e.g. no styles on the flash messages), but it's working and that's good!! I tested this by temporarily switching to the production database and logging in as matchu! Still missing a lot of big features too, like registration, password resets, settings page, etc.
2023-08-06 15:52:05 -07:00
gem 'devise-encryptable', '~> 0.2.0'
Add OmniAuth plugin to AuthUser This is setting us up for NeoPass, but first we're just gonna try stuff with the "developer" strategy that's built in for testing, rather than using the NeoPass dev server!
2024-03-13 13:10:59 -07:00
gem 'omniauth', '~> 2.1'
gem 'omniauth-rails_csrf_protection', '~> 1.0'
Refactor to use OpenID Connect OmniAuth gem instead of plain OAuth2 Right, I didn't totally connect the dots that there's some OpenID features in the mix here for how we expect to identify the user once they authenticate. It requires looking up the provider's public key, and validating the JWT they sent us. This gem does all that for us! I don't actually know what a real NeoPass `id_token` looks like yet? But I'll fill in some placeholder stuff for now, and use that for initializing the account!
2024-03-14 18:11:40 -07:00
gem "omniauth_openid_connect", "~> 0.7.1"
move to unstable rocketamf so that it builds hashes from registries, not arrays
2011-01-13 14:22:07 -08:00
Clarify will_paginate's Gemfile comment
2023-08-02 16:30:17 -07:00
# For pagination UI.
gem 'will_paginate', '~> 4.0'
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For translation, both for the site UI and for Neopets data.
Upgrade to Rails 6.0.6.1 Another pretty easy one! We have the `rails app:update` changes in here too.
2023-08-02 17:27:11 -07:00
gem 'rails-i18n', '~> 7.0', '>= 7.0.7'
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
gem 'http_accept_language', '~> 2.1', '>= 2.1.1'
add compass to gemfile
2010-05-15 15:58:28 -07:00
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For reading and parsing HTML from Neopets.com, like importing Closet pages.
Upgrade to webrick 1.8 Literally don't know why this wasn't 1.8 when I first installed it? Whatever!
2023-08-03 16:43:37 -07:00
gem 'nokogiri', '~> 1.15', '>= 1.15.3'
closet page, closet hanger
2011-07-12 16:37:16 -07:00
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For safely rendering users' Markdown + HTML on item list pages.
Update all our gems as far as we can Got pretty far, there's a couple still held back by Rails 6+, but yeah doing great!
2023-08-02 17:16:24 -07:00
gem 'rdiscount', '~> 2.2', '>= 2.2.7.1'
gem 'sanitize', '~> 6.0', '>= 6.0.2'
closet lists, round one
2011-07-29 07:52:04 -07:00
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For working with Neopets APIs.
# unstable version of RocketAMF interprets info registry as a hash instead of an array
gem 'RocketAMF', :git => 'https://github.com/rubyamf/rocketamf.git'
use Neopets::User for username-based closet imports
2012-05-21 09:48:19 -07:00
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For preventing too many modeling attempts.
Update all our gems as far as we can Got pretty far, there's a couple still held back by Rails 6+, but yeah doing great!
2023-08-02 17:16:24 -07:00
gem 'rack-attack', '~> 6.7'
throttle pet loads per ip
2013-07-02 14:10:01 -07:00
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For testing emails in development.
Update all our gems as far as we can Got pretty far, there's a couple still held back by Rails 6+, but yeah doing great!
2023-08-02 17:16:24 -07:00
gem 'letter_opener', '~> 1.8', '>= 1.8.1', group: :development
donation mailer
2014-09-10 12:32:54 -07:00
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For parallel API calls.
Update all our gems as far as we can Got pretty far, there's a couple still held back by Rails 6+, but yeah doing great!
2023-08-02 17:16:24 -07:00
gem 'parallel', '~> 1.23'
phew. rails 3.2.12, including some asset pipeline. still buggy.
2013-03-05 18:08:57 -08:00
Upgrade to Rails 7.1.1 The usual stuff! Installed the new gem and its new deps, ran `bin/rails app:update` and did my best to manually merge the dev/prod config files with the new canonical defaults, deleted some migrations I don't think are relevant to us, and yeah! Also, Rails 7.1 seems to need `libyaml-dev` installed, so I added that to the `deploy/setup.yml` playbook! One thing to note is that, while I was here, I turned on some settings relating to our use of SSL that technically weren't on before. This should be fine and helpful? But if stuff breaks, well, check those!
2023-10-25 15:05:31 -07:00
# For miscellaneous HTTP requests.
Update Ruby to 3.3.4, and update gems For the gems, I mostly just ran `bundle update`; with the exception of `httparty`, because latest Ruby throws a deprecation warning about its use of the deprecated `csv` stdlib, which the latest version resolves. One other little thing: this is on my new Fedora workstation, and I had to deal with a known bug where the `sassc` gem compiles a `libsass.so` file, but saves it in the wrong place somehow. Here's the known bug, and the comment that helped me: https://github.com/sass/sassc-ruby/issues/146#issuecomment-2028974524 And here's what I ran to get it into the right place: ```shell ln -s ~/.local/share/gem/ruby/3.3.0/extensions/aarch64-linux/3.3.0/sassc-2.4.0/sassc/libsass.so \ ~/.local/share/gem/ruby/3.3.0/gems/sassc-2.4.0/lib/sassc/libsass.so ``` This thing about `libsass` isn't reflected in the code changes anywhere in this commit! I'm just mentioning it so that it's literally written down anywhere. (I did try other comments' advice to use an older version of `sassc` first, but I ran into compilation errors, so figured this machine-side hack was better than untangling that mess.)
2024-08-27 17:06:22 -07:00
gem "httparty", "~> 0.22.0"
Add Owls values to the item page Eyy it's time!! Long-requested, finally here lol
2023-11-03 16:20:02 -07:00
gem "addressable", "~> 2.8"
Upgrade to Rails 7.1.1 The usual stuff! Installed the new gem and its new deps, ran `bin/rails app:update` and did my best to manually merge the dev/prod config files with the new canonical defaults, deleted some migrations I don't think are relevant to us, and yeah! Also, Rails 7.1 seems to need `libyaml-dev` installed, so I added that to the `deploy/setup.yml` playbook! One thing to note is that, while I was here, I turned on some settings relating to our use of SSL that technically weren't on before. This should be fine and helpful? But if stuff breaks, well, check those!
2023-10-25 15:05:31 -07:00
Move manifest backfill to `swf_assets:manifests` task Okay, I've simplified the migration to *just* add the column, and instead added a task to find assets without manifest URLs and backfill them. Performance is a lot better now, using the `async-http` library, which as I understand it supports both persistent connections when invoked like this, and maybe also HTTP/2 multiplexing?? (Though I'm not actually sure images.neopets.com does lol) I'm not sure about the number of concurrent tasks I picked here, 100 seems okay for an internet thing and for such small requests, but I worry that the CDN is gonna get annoyed or something. Well, we'll see! This task is very resumable if it turns out we get frozen out or something.
2023-11-10 16:52:50 -08:00
# For advanced batching of many HTTP requests.
Upgrade async and related gems, and fix async-http response handling When playing with a Rainbow Pool syncing task, I noticed that error handling wasn't working correctly for requests using `async-http`: if the block raised an error, the `Sync` block would never return. My suspicion is that this is because we were never reading or releasing the request body. In this change, I upgrade all the relevant gems for good measure, and switch to using the response object yielded by the _block_, so we can know it's being resource-managed correctly. Now, failures raise errors as expected! (I tested all these relevant service calls, too!)
2024-09-07 12:14:12 -07:00
gem "async", "~> 2.17", require: false
gem "async-http", "~> 0.75.0", require: false
Move manifest backfill to `swf_assets:manifests` task Okay, I've simplified the migration to *just* add the column, and instead added a task to find assets without manifest URLs and backfill them. Performance is a lot better now, using the `async-http` library, which as I understand it supports both persistent connections when invoked like this, and maybe also HTTP/2 multiplexing?? (Though I'm not actually sure images.neopets.com does lol) I'm not sure about the number of concurrent tasks I picked here, 100 seems okay for an internet thing and for such small requests, but I worry that the CDN is gonna get annoyed or something. Well, we'll see! This task is very resumable if it turns out we get frozen out or something.
2023-11-10 16:52:50 -08:00
gem "thread-local", "~> 1.1", require: false
Make 4.1 -> 4.2 changes Just following the recommended stuff in the upgrade guide!
2023-08-02 15:41:22 -07:00
# For debugging.
Add web console support for Vagrant users
2023-08-02 17:36:32 -07:00
gem 'web-console', '~> 4.2', group: :development
Make 4.1 -> 4.2 changes Just following the recommended stuff in the upgrade guide!
2023-08-02 15:41:22 -07:00
Upgrade to Rails 5.2.8.1 Some important little upgrades but mostly straightforward! Note that there's still a known issue where item searches crash, I was hoping that this was a bug in Rails 4.2 that would be fixed on upgading to 5, but nope, oh well! Also uhh I just got a bit silly and didn't actually mean to go all the way to 5.2 in one go, I had meant to start at 5.0… but tbh the 5.1 and 5.2 changes seem small, and this seems to be working, so. Yeah ok let's roll!
2023-08-02 16:05:02 -07:00
# TODO: Review our use of content_tag_for etc and uninstall this!
Update all our gems as far as we can Got pretty far, there's a couple still held back by Rails 6+, but yeah doing great!
2023-08-02 17:16:24 -07:00
gem 'record_tag_helper', '~> 1.0', '>= 1.0.1'
Upgrade to Rails 5.2.8.1 Some important little upgrades but mostly straightforward! Note that there's still a known issue where item searches crash, I was hoping that this was a bug in Rails 4.2 that would be fixed on upgading to 5, but nope, oh well! Also uhh I just got a bit silly and didn't actually mean to go all the way to 5.2 in one go, I had meant to start at 5.0… but tbh the 5.1 and 5.2 changes seem small, and this seems to be working, so. Yeah ok let's roll!
2023-08-02 16:05:02 -07:00
Run `rails app:update` We accepted some changes as-is, but for development.rb and production.rb we read the diff and manually edited them!
2023-08-02 16:17:07 -07:00
# Reduces boot times through caching; required in config/boot.rb
Update all our gems as far as we can Got pretty far, there's a couple still held back by Rails 6+, but yeah doing great!
2023-08-02 17:16:24 -07:00
gem 'bootsnap', '~> 1.16', require: false
Add mini profiler to each page It shows up in development always, and if you're logged in as Me Specifically in production! I'm using this to poke at memory usage for pages that seem suspicious. I don't know why our app reliably grows so large in RAM, but my hunch is that maybe there are some pages that just use a truly large amount to begin with - and I've learned Ruby doesn't release memory back after it's GC'd, it just grows the process and keeps the free space to itself in its own heap! So I'm just eyeing pages that I know *can* have a lot going on, and seeing what I find!
2023-10-27 19:38:49 -07:00
# For investigating performance issues.
gem "rack-mini-profiler", "~> 3.1"
gem "memory_profiler", "~> 1.0"
gem "stackprof", "~> 0.2.25"
Add Sentry to Rails Now we're tracking both JS and Rails errors, phew!
2023-11-06 12:37:40 -08:00
# For monitoring errors in production.
gem "sentry-ruby", "~> 5.12"
gem "sentry-rails", "~> 5.12"
Save last trade activity time onto User In impress-2020, we do a big slow query to figure out which users have been active in trades recently. Now, we cache that timestamp on the User model. This won't have any immediate effect; it's to clear the way for Classic DTI to receive the better trade ratios feature people like from 2020. I also added some unit testing infra because I finally wanted it! for all the ways you can trigger this timestamp lol Note too that this is a bit of an unusually complex migration, but my hope is that the batching and query structure and such helps it run surprisingly fast! 🤞
2024-01-19 00:00:46 -08:00
Create `rails public_data:commit` task, to share public data dumps I'm starting to port over the functionality that was previously just, me running `yarn db:export:public-data` in `impress-2020` and committing it to Git LFS every time. My immediate motivation is that the `impress-2020` git repository is getting weirdly large?? Idk how these 40MB files have blown up to a solid 16GB of Git LFS data (we don't have THAT many!!!), but I guess there's something about Git LFS's architecture and disk usage that I'm not understanding. So, let's move to a simpler system in which we don't bind the public data to the codebase, but instead just regularly dump it in production and make it available for download. This change adds the `rails public_data:commit` task, which when run in production will make the latest available at `https://impress.openneo.net/public-data/latest.sql.gz`, and will also store a running log of previous dumps, viewable at `https://impress.openneo.net/public-data/`. Things left to do: 1. Create a `rails public_data:pull` task, to download `latest.sql.gz` and import it into the local development database. 2. Set up a cron job to dump this out regularly, idk maybe weekly? That will grow, but not very fast (about 2GB per year), and we can add logic to rotate out old ones if it starts to grow too far. (If we wanted to get really intricate, we could do like, daily for the past week, then weekly for the past 3 months, then monthly for the past year, idk. There must be tools that do this!)
2024-02-29 14:30:33 -08:00
# For tasks that use shell commands.
gem "shell", "~> 0.8.1"
Add Solargraph autocomplete while in development This is a Ruby language server that integrates with my editor! Static analysis of Ruby and Rails is pretty tricky, but it's working and I think that's neat!!
2024-07-01 15:35:39 -07:00
# For workspace autocomplete.
gem "solargraph", "~> 0.50.0", group: :development
gem "solargraph-rails", "~> 1.1", group: :development
Copy permalink