OpenNeo/impress
OpenNeo/impress
1
0
Fork 1
Code Pull requests Activity
impress/Gemfile

89 lines
2.5 KiB
Ruby
Raw Normal View History

Update most URLs to use HTTPS I noticed we didn't have the little lock icon in the browser, and yeah huh there's a lot of `http://` still floating around! Let's fix that!
2023-10-25 15:16:46 -07:00
source 'https://rubygems.org'
Finish migrating to Ruby 3.3.0 As the comment in `deploy.yml` explains, this was a multi-step process, but it went very smoothly as planned, hooray!! I noticed again while making this change that Bundler doesn't seem to be availing itself of the checked-in dependencies in `vendor/cache`. I think I know the fix for this, I'll toss it into an upcoming change and see if it works!
2024-02-22 12:05:02 -08:00
ruby '3.3.0'
rails 3
2010-05-14 15:12:31 -07:00
Upgrade to Rails 7.1.3.2 In response to: - https://discuss.rubyonrails.org/t/possible-xss-vulnerability-in-action-controller/84947 - https://discuss.rubyonrails.org/t/possible-redos-vulnerability-in-accept-header-parsing-in-action-dispatch/84946 No evidence that our app has been affected, but I like to move quickly on these!
2024-02-22 11:25:52 -08:00
gem 'rails', '~> 7.1', '>= 7.1.3.2'
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
Upgrade to Ruby 3.0.6 Hey nice! We have to add webrick now because it's not included in Ruby 3, but hey just drop it right back in. Idk how to choose between this or puma or whatever, but in the absence of a specific reason let's just pick the one whose name I know best.
2023-08-03 16:15:23 -07:00
# The HTTP server running the Rails instance.
Run `bundle update` I also put in a manual bump for `falcon`! The motivation is that I'm working on a Ruby 3.3.0 upgrade in another branch, and I'm getting deprecation warnings from the `async` gem, which I think are resolved in the latest version, so I figure, hey, good time for an update!
2024-02-22 11:48:32 -08:00
gem 'falcon', '~> 0.43.0'
Upgrade to Ruby 3.0.6 Hey nice! We have to add webrick now because it's not included in Ruby 3, but hey just drop it right back in. Idk how to choose between this or puma or whatever, but in the absence of a specific reason let's just pick the one whose name I know best.
2023-08-03 16:15:23 -07:00
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# Our database is MySQL, in both development and production.
Upgrade to Rails 5.2.8.1 Some important little upgrades but mostly straightforward! Note that there's still a known issue where item searches crash, I was hoping that this was a bug in Rails 4.2 that would be fixed on upgading to 5, but nope, oh well! Also uhh I just got a bit silly and didn't actually mean to go all the way to 5.2 in one go, I had meant to start at 5.0… but tbh the 5.1 and 5.2 changes seem small, and this seems to be working, so. Yeah ok let's roll!
2023-08-02 16:05:02 -07:00
gem 'mysql2', '~> 0.5.5'
support concurrent requests. now demands thin and nginx, but hey. it works
2010-10-10 15:46:58 -07:00
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For reading the .env file, which you can use in development to more easily
# set environment variables for secret data.
Remove dotenv-deployment, upgrade dotenv-rails Idk exactly what's going on with dotenv-deployment, if it turns out it was critical to our deploy process then we'll change the deploy process! It's deprecated and conflicts with gem deps for `dotenv-rails`.
2023-08-02 13:41:37 -07:00
gem 'dotenv-rails', '~> 2.8', '>= 2.8.1'
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For the asset pipeline: templates, CSS, JS, etc.
Upgrade to latest Sass and Sprockets This required a buncha fixes to how SASS scoping works! Needed to add a bunch of imports for stuff that previously would get read from the global scope by being imported *after* the constants and mixins etc. There's clearly a lot of refactor opportunity here, but I'm not gonna worry about it!!
2023-08-22 16:08:11 -07:00
gem 'sprockets', '~> 4.2'
Upgrade haml I was hoping this would fix a character encoding bug, it didn't really? but it doesn't seem to hurt and is probably good to do
2023-07-22 12:58:15 -07:00
gem 'haml', '~> 6.1', '>= 6.1.1'
Upgrade to latest Sass and Sprockets This required a buncha fixes to how SASS scoping works! Needed to add a bunch of imports for stuff that previously would get read from the global scope by being imported *after* the constants and mixins etc. There's clearly a lot of refactor opportunity here, but I'm not gonna worry about it!!
2023-08-22 16:08:11 -07:00
gem 'sass-rails', '~> 6.0'
Precompile assets when deploying new version I did some refactoring while here too, of pulling the deploy scripts out of `package.json` and into `bin`, to be a bit more canonically Rails-y. (idk how canonical the colon thing is but, probably fine??)
2023-08-19 18:51:46 -07:00
gem 'terser', '~> 1.1', '>= 1.1.17'
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
gem 'react-rails', '~> 2.7', '>= 2.7.1'
Move jsbundling-rails higher up in Gemfile Oh I guess I ranthe thing that put it at the bottom!
2023-08-19 17:07:17 -07:00
gem 'jsbundling-rails', '~> 1.1'
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For authentication.
Upgrade to Rails 4.2.11.3 and Ruby 2.4.10 Some tricks required here to get the dependencies to work out, but we got it!! Oh also, we move away from the rbenv in Ubuntu's package manager, because it doesn't support more recent Rubies like 2.4.10.
2023-08-02 15:18:36 -07:00
gem 'devise', '~> 4.9', '>= 4.9.2'
Can log into OpenNeo ID accounts directly! A lot of rough edges here (e.g. no styles on the flash messages), but it's working and that's good!! I tested this by temporarily switching to the production database and logging in as matchu! Still missing a lot of big features too, like registration, password resets, settings page, etc.
2023-08-06 15:52:05 -07:00
gem 'devise-encryptable', '~> 0.2.0'
move to unstable rocketamf so that it builds hashes from registries, not arrays
2011-01-13 14:22:07 -08:00
Clarify will_paginate's Gemfile comment
2023-08-02 16:30:17 -07:00
# For pagination UI.
gem 'will_paginate', '~> 4.0'
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For translation, both for the site UI and for Neopets data.
Upgrade to Rails 6.0.6.1 Another pretty easy one! We have the `rails app:update` changes in here too.
2023-08-02 17:27:11 -07:00
gem 'rails-i18n', '~> 7.0', '>= 7.0.7'
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
gem 'http_accept_language', '~> 2.1', '>= 2.1.1'
add compass to gemfile
2010-05-15 15:58:28 -07:00
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For reading and parsing HTML from Neopets.com, like importing Closet pages.
Upgrade to webrick 1.8 Literally don't know why this wasn't 1.8 when I first installed it? Whatever!
2023-08-03 16:43:37 -07:00
gem 'nokogiri', '~> 1.15', '>= 1.15.3'
closet page, closet hanger
2011-07-12 16:37:16 -07:00
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For safely rendering users' Markdown + HTML on item list pages.
Update all our gems as far as we can Got pretty far, there's a couple still held back by Rails 6+, but yeah doing great!
2023-08-02 17:16:24 -07:00
gem 'rdiscount', '~> 2.2', '>= 2.2.7.1'
gem 'sanitize', '~> 6.0', '>= 6.0.2'
closet lists, round one
2011-07-29 07:52:04 -07:00
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For working with Neopets APIs.
# unstable version of RocketAMF interprets info registry as a hash instead of an array
gem 'RocketAMF', :git => 'https://github.com/rubyamf/rocketamf.git'
use Neopets::User for username-based closet imports
2012-05-21 09:48:19 -07:00
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For preventing too many modeling attempts.
Update all our gems as far as we can Got pretty far, there's a couple still held back by Rails 6+, but yeah doing great!
2023-08-02 17:16:24 -07:00
gem 'rack-attack', '~> 6.7'
throttle pet loads per ip
2013-07-02 14:10:01 -07:00
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For testing emails in development.
Update all our gems as far as we can Got pretty far, there's a couple still held back by Rails 6+, but yeah doing great!
2023-08-02 17:16:24 -07:00
gem 'letter_opener', '~> 1.8', '>= 1.8.1', group: :development
donation mailer
2014-09-10 12:32:54 -07:00
Clean up the Gemfile + add explanations Grouping stuff a bit more sensibly and with explanations! I just finished deleting a bunch of gems we're not really using, and this helped me audit that the ones we still have here we generally actually want.
2023-08-02 13:36:28 -07:00
# For parallel API calls.
Update all our gems as far as we can Got pretty far, there's a couple still held back by Rails 6+, but yeah doing great!
2023-08-02 17:16:24 -07:00
gem 'parallel', '~> 1.23'
phew. rails 3.2.12, including some asset pipeline. still buggy.
2013-03-05 18:08:57 -08:00
Upgrade to Rails 7.1.1 The usual stuff! Installed the new gem and its new deps, ran `bin/rails app:update` and did my best to manually merge the dev/prod config files with the new canonical defaults, deleted some migrations I don't think are relevant to us, and yeah! Also, Rails 7.1 seems to need `libyaml-dev` installed, so I added that to the `deploy/setup.yml` playbook! One thing to note is that, while I was here, I turned on some settings relating to our use of SSL that technically weren't on before. This should be fine and helpful? But if stuff breaks, well, check those!
2023-10-25 15:05:31 -07:00
# For miscellaneous HTTP requests.
gem "httparty", "~> 0.21.0"
Add Owls values to the item page Eyy it's time!! Long-requested, finally here lol
2023-11-03 16:20:02 -07:00
gem "addressable", "~> 2.8"
Upgrade to Rails 7.1.1 The usual stuff! Installed the new gem and its new deps, ran `bin/rails app:update` and did my best to manually merge the dev/prod config files with the new canonical defaults, deleted some migrations I don't think are relevant to us, and yeah! Also, Rails 7.1 seems to need `libyaml-dev` installed, so I added that to the `deploy/setup.yml` playbook! One thing to note is that, while I was here, I turned on some settings relating to our use of SSL that technically weren't on before. This should be fine and helpful? But if stuff breaks, well, check those!
2023-10-25 15:05:31 -07:00
Move manifest backfill to `swf_assets:manifests` task Okay, I've simplified the migration to *just* add the column, and instead added a task to find assets without manifest URLs and backfill them. Performance is a lot better now, using the `async-http` library, which as I understand it supports both persistent connections when invoked like this, and maybe also HTTP/2 multiplexing?? (Though I'm not actually sure images.neopets.com does lol) I'm not sure about the number of concurrent tasks I picked here, 100 seems okay for an internet thing and for such small requests, but I worry that the CDN is gonna get annoyed or something. Well, we'll see! This task is very resumable if it turns out we get frozen out or something.
2023-11-10 16:52:50 -08:00
# For advanced batching of many HTTP requests.
gem "async", "~> 2.6", require: false
gem "async-http", "~> 0.61.0", require: false
gem "thread-local", "~> 1.1", require: false
Make 4.1 -> 4.2 changes Just following the recommended stuff in the upgrade guide!
2023-08-02 15:41:22 -07:00
# For debugging.
Add web console support for Vagrant users
2023-08-02 17:36:32 -07:00
gem 'web-console', '~> 4.2', group: :development
Make 4.1 -> 4.2 changes Just following the recommended stuff in the upgrade guide!
2023-08-02 15:41:22 -07:00
Upgrade to Rails 5.2.8.1 Some important little upgrades but mostly straightforward! Note that there's still a known issue where item searches crash, I was hoping that this was a bug in Rails 4.2 that would be fixed on upgading to 5, but nope, oh well! Also uhh I just got a bit silly and didn't actually mean to go all the way to 5.2 in one go, I had meant to start at 5.0… but tbh the 5.1 and 5.2 changes seem small, and this seems to be working, so. Yeah ok let's roll!
2023-08-02 16:05:02 -07:00
# TODO: Review our use of content_tag_for etc and uninstall this!
Update all our gems as far as we can Got pretty far, there's a couple still held back by Rails 6+, but yeah doing great!
2023-08-02 17:16:24 -07:00
gem 'record_tag_helper', '~> 1.0', '>= 1.0.1'
Upgrade to Rails 5.2.8.1 Some important little upgrades but mostly straightforward! Note that there's still a known issue where item searches crash, I was hoping that this was a bug in Rails 4.2 that would be fixed on upgading to 5, but nope, oh well! Also uhh I just got a bit silly and didn't actually mean to go all the way to 5.2 in one go, I had meant to start at 5.0… but tbh the 5.1 and 5.2 changes seem small, and this seems to be working, so. Yeah ok let's roll!
2023-08-02 16:05:02 -07:00
Run `rails app:update` We accepted some changes as-is, but for development.rb and production.rb we read the diff and manually edited them!
2023-08-02 16:17:07 -07:00
# Reduces boot times through caching; required in config/boot.rb
Update all our gems as far as we can Got pretty far, there's a couple still held back by Rails 6+, but yeah doing great!
2023-08-02 17:16:24 -07:00
gem 'bootsnap', '~> 1.16', require: false
Add mini profiler to each page It shows up in development always, and if you're logged in as Me Specifically in production! I'm using this to poke at memory usage for pages that seem suspicious. I don't know why our app reliably grows so large in RAM, but my hunch is that maybe there are some pages that just use a truly large amount to begin with - and I've learned Ruby doesn't release memory back after it's GC'd, it just grows the process and keeps the free space to itself in its own heap! So I'm just eyeing pages that I know *can* have a lot going on, and seeing what I find!
2023-10-27 19:38:49 -07:00
# For investigating performance issues.
gem "rack-mini-profiler", "~> 3.1"
gem "memory_profiler", "~> 1.0"
gem "stackprof", "~> 0.2.25"
Add Sentry to Rails Now we're tracking both JS and Rails errors, phew!
2023-11-06 12:37:40 -08:00
# For monitoring errors in production.
gem "sentry-ruby", "~> 5.12"
gem "sentry-rails", "~> 5.12"
Save last trade activity time onto User In impress-2020, we do a big slow query to figure out which users have been active in trades recently. Now, we cache that timestamp on the User model. This won't have any immediate effect; it's to clear the way for Classic DTI to receive the better trade ratios feature people like from 2020. I also added some unit testing infra because I finally wanted it! for all the ways you can trigger this timestamp lol Note too that this is a bit of an unusually complex migration, but my hope is that the batching and query structure and such helps it run surprisingly fast! 🤞
2024-01-19 00:00:46 -08:00
Create `rails public_data:commit` task, to share public data dumps I'm starting to port over the functionality that was previously just, me running `yarn db:export:public-data` in `impress-2020` and committing it to Git LFS every time. My immediate motivation is that the `impress-2020` git repository is getting weirdly large?? Idk how these 40MB files have blown up to a solid 16GB of Git LFS data (we don't have THAT many!!!), but I guess there's something about Git LFS's architecture and disk usage that I'm not understanding. So, let's move to a simpler system in which we don't bind the public data to the codebase, but instead just regularly dump it in production and make it available for download. This change adds the `rails public_data:commit` task, which when run in production will make the latest available at `https://impress.openneo.net/public-data/latest.sql.gz`, and will also store a running log of previous dumps, viewable at `https://impress.openneo.net/public-data/`. Things left to do: 1. Create a `rails public_data:pull` task, to download `latest.sql.gz` and import it into the local development database. 2. Set up a cron job to dump this out regularly, idk maybe weekly? That will grow, but not very fast (about 2GB per year), and we can add logic to rotate out old ones if it starts to grow too far. (If we wanted to get really intricate, we could do like, daily for the past week, then weekly for the past 3 months, then monthly for the past year, idk. There must be tools that do this!)
2024-02-29 14:30:33 -08:00
# For tasks that use shell commands.
gem "shell", "~> 0.8.1"
Save last trade activity time onto User In impress-2020, we do a big slow query to figure out which users have been active in trades recently. Now, we cache that timestamp on the User model. This won't have any immediate effect; it's to clear the way for Classic DTI to receive the better trade ratios feature people like from 2020. I also added some unit testing infra because I finally wanted it! for all the ways you can trigger this timestamp lol Note too that this is a bit of an unusually complex migration, but my hope is that the batching and query structure and such helps it run surprisingly fast! 🤞
2024-01-19 00:00:46 -08:00
# For automated testing.
group :test do
gem 'sqlite3', '~> 1.7'
end
Copy permalink