Replace falcon server with puma

Been wanting this for a while in theory, gonna actually do it now!

The motivation is that I want to turn up the timeout for loading pets,
because the Neopets endpoints are slower today with the NC UC release -
but I can already predict that under our current architecture that will
be a problem, because it'll block up our request queue!

Falcon uses Ruby's relatively-new async system to *not* have requests
block on upstream requests, and my understanding is that this behavior
is plug-and-play. Let's see how it goes!
This commit is contained in:
Emi Matchu 2024-01-23 21:47:51 -08:00
parent 0cad34acfc
commit 76af587e7c
18 changed files with 264 additions and 212 deletions

View file

@ -4,7 +4,11 @@ ruby '3.1.4'
gem 'rails', '~> 7.1', '>= 7.1.1' gem 'rails', '~> 7.1', '>= 7.1.1'
# The HTTP server running the Rails instance. # The HTTP server running the Rails instance.
# NOTE: Once we're migrated, remove puma! I have both to allow the upgrade to
# be incremental: push this out with `bin/deploy`, then change the service file
# with `bin/deploy:setup`.
gem 'puma', '~> 6.3', '>= 6.3.1' gem 'puma', '~> 6.3', '>= 6.3.1'
gem 'falcon', '~> 0.42.3'
# Our database is MySQL, in both development and production. # Our database is MySQL, in both development and production.
gem 'mysql2', '~> 0.5.5' gem 'mysql2', '~> 0.5.5'

View file

@ -87,6 +87,9 @@ GEM
fiber-annotation fiber-annotation
io-event (~> 1.1) io-event (~> 1.1)
timers (~> 4.1) timers (~> 4.1)
async-container (0.16.12)
async
async-io
async-http (0.61.0) async-http (0.61.0)
async (>= 1.25) async (>= 1.25)
async-io (>= 1.28) async-io (>= 1.28)
@ -95,6 +98,8 @@ GEM
protocol-http1 (~> 0.16.0) protocol-http1 (~> 0.16.0)
protocol-http2 (~> 0.15.0) protocol-http2 (~> 0.15.0)
traces (>= 0.10.0) traces (>= 0.10.0)
async-http-cache (0.4.3)
async-http (~> 0.56)
async-io (1.37.0) async-io (1.37.0)
async async
async-pool (0.4.0) async-pool (0.4.0)
@ -109,6 +114,7 @@ GEM
bindex (0.8.1) bindex (0.8.1)
bootsnap (1.16.0) bootsnap (1.16.0)
msgpack (~> 1.2) msgpack (~> 1.2)
build-environment (1.13.0)
builder (3.2.4) builder (3.2.4)
concurrent-ruby (1.2.2) concurrent-ruby (1.2.2)
connection_pool (2.2.5) connection_pool (2.2.5)
@ -133,6 +139,19 @@ GEM
ruby2_keywords ruby2_keywords
erubi (1.12.0) erubi (1.12.0)
execjs (2.5.2) execjs (2.5.2)
falcon (0.42.3)
async
async-container (~> 0.16.0)
async-http (~> 0.57)
async-http-cache (~> 0.4.0)
async-io (~> 1.22)
build-environment (~> 1.13)
bundler
localhost (~> 1.1)
openssl (~> 3.0)
process-metrics (~> 0.2.0)
protocol-rack (~> 0.1)
samovar (~> 2.1)
ffi (1.15.5) ffi (1.15.5)
fiber-annotation (0.2.0) fiber-annotation (0.2.0)
fiber-local (1.0.0) fiber-local (1.0.0)
@ -163,6 +182,7 @@ GEM
addressable (~> 2.8) addressable (~> 2.8)
letter_opener (1.8.1) letter_opener (1.8.1)
launchy (>= 2.2, < 3) launchy (>= 2.2, < 3)
localhost (1.1.10)
loofah (2.21.3) loofah (2.21.3)
crass (~> 1.0.2) crass (~> 1.0.2)
nokogiri (>= 1.12.0) nokogiri (>= 1.12.0)
@ -171,6 +191,7 @@ GEM
net-imap net-imap
net-pop net-pop
net-smtp net-smtp
mapping (1.1.1)
marcel (1.0.2) marcel (1.0.2)
memory_profiler (1.0.1) memory_profiler (1.0.1)
mini_mime (1.1.5) mini_mime (1.1.5)
@ -193,8 +214,12 @@ GEM
nokogiri (1.15.3) nokogiri (1.15.3)
mini_portile2 (~> 2.8.2) mini_portile2 (~> 2.8.2)
racc (~> 1.4) racc (~> 1.4)
openssl (3.2.0)
orm_adapter (0.5.0) orm_adapter (0.5.0)
parallel (1.23.0) parallel (1.23.0)
process-metrics (0.2.1)
console (~> 1.8)
samovar (~> 2.1)
protocol-hpack (1.4.2) protocol-hpack (1.4.2)
protocol-http (0.25.0) protocol-http (0.25.0)
protocol-http1 (0.16.0) protocol-http1 (0.16.0)
@ -202,10 +227,13 @@ GEM
protocol-http2 (0.15.1) protocol-http2 (0.15.1)
protocol-hpack (~> 1.4) protocol-hpack (~> 1.4)
protocol-http (~> 0.18) protocol-http (~> 0.18)
protocol-rack (0.4.1)
protocol-http (~> 0.23)
rack (>= 1.0)
psych (5.1.1.1) psych (5.1.1.1)
stringio stringio
public_suffix (5.0.3) public_suffix (5.0.3)
puma (6.4.0) puma (6.4.2)
nio4r (~> 2.0) nio4r (~> 2.0)
racc (1.7.1) racc (1.7.1)
rack (2.2.8) rack (2.2.8)
@ -272,6 +300,9 @@ GEM
actionpack (>= 5.2) actionpack (>= 5.2)
railties (>= 5.2) railties (>= 5.2)
ruby2_keywords (0.0.5) ruby2_keywords (0.0.5)
samovar (2.2.0)
console (~> 1.0)
mapping (~> 1.0)
sanitize (6.0.2) sanitize (6.0.2)
crass (~> 1.0.2) crass (~> 1.0.2)
nokogiri (>= 1.12.0) nokogiri (>= 1.12.0)
@ -338,6 +369,7 @@ DEPENDENCIES
devise (~> 4.9, >= 4.9.2) devise (~> 4.9, >= 4.9.2)
devise-encryptable (~> 0.2.0) devise-encryptable (~> 0.2.0)
dotenv-rails (~> 2.8, >= 2.8.1) dotenv-rails (~> 2.8, >= 2.8.1)
falcon (~> 0.42.3)
globalize (~> 6.2, >= 6.2.1) globalize (~> 6.2, >= 6.2.1)
haml (~> 6.1, >= 6.1.1) haml (~> 6.1, >= 6.1.1)
http_accept_language (~> 2.1, >= 2.1.1) http_accept_language (~> 2.1, >= 2.1.1)

View file

@ -61,6 +61,10 @@ module OpenneoImpressItems
config.middleware.insert_after ActionDispatch::Flash, Rack::Attack config.middleware.insert_after ActionDispatch::Flash, Rack::Attack
# On the Falcon server, requests run on fibers. Isolate Rails internal
# state to the per-fiber level, to avoid conflicts that crash stuff!
config.active_support.isolation_level = :fiber
# It seems like some Neopets servers reject any user agent containing # It seems like some Neopets servers reject any user agent containing
# symbols? So I can't provide anything helpful like a URL, email address, # symbols? So I can't provide anything helpful like a URL, email address,
# version number, etc. So let's only send this to Neopets systems, where it # version number, etc. So let's only send this to Neopets systems, where it

View file

@ -5,7 +5,7 @@ Description=Dress to Impress webapp
User=impress User=impress
Restart=always Restart=always
WorkingDirectory=/srv/impress/current WorkingDirectory=/srv/impress/current
ExecStart=/opt/ruby-3.1.4/bin/bundle exec puma --port=3000 ExecStart=/opt/ruby-3.1.4/bin/bundle exec falcon host
Environment="RAILS_ENV=production" Environment="RAILS_ENV=production"
; Set EXECJS_RUNTIME to save us from needing to install Node ; Set EXECJS_RUNTIME to save us from needing to install Node
Environment="EXECJS_RUNTIME=Disabled" Environment="EXECJS_RUNTIME=Disabled"

View file

@ -7,253 +7,253 @@
email_address: "emi@matchu.dev" # TODO: Extract this to personal config? email_address: "emi@matchu.dev" # TODO: Extract this to personal config?
impress_hostname: impress.openneo.net impress_hostname: impress.openneo.net
tasks: tasks:
- name: Create SSH folder for logged-in user # - name: Create SSH folder for logged-in user
become: no # become: no
file: # file:
name: .ssh # name: .ssh
mode: "700" # mode: "700"
state: directory # state: directory
- name: Copy authorized SSH keys to logged-in user # - name: Copy authorized SSH keys to logged-in user
become: no # become: no
copy: # copy:
dest: ~/.ssh/authorized_keys # dest: ~/.ssh/authorized_keys
src: files/authorized-ssh-keys.txt # src: files/authorized-ssh-keys.txt
mode: "600" # mode: "600"
- name: Disable root SSH login # - name: Disable root SSH login
lineinfile: # lineinfile:
dest: /etc/ssh/sshd_config # dest: /etc/ssh/sshd_config
regexp: ^#?PermitRootLogin # regexp: ^#?PermitRootLogin
line: PermitRootLogin no # line: PermitRootLogin no
- name: Disable password-based SSH authentication # - name: Disable password-based SSH authentication
lineinfile: # lineinfile:
dest: /etc/ssh/sshd_config # dest: /etc/ssh/sshd_config
regexp: ^#?PasswordAuthentication # regexp: ^#?PasswordAuthentication
line: PasswordAuthentication no # line: PasswordAuthentication no
- name: Enable public-key SSH authentication # - name: Enable public-key SSH authentication
lineinfile: # lineinfile:
dest: /etc/ssh/sshd_config # dest: /etc/ssh/sshd_config
regexp: ^#?PubkeyAuthentication # regexp: ^#?PubkeyAuthentication
line: PubkeyAuthentication yes # line: PubkeyAuthentication yes
- name: Update the apt cache # - name: Update the apt cache
apt: # apt:
update_cache: yes # update_cache: yes
- name: Install fail2ban firewall with default settings # - name: Install fail2ban firewall with default settings
apt: # apt:
name: fail2ban # name: fail2ban
- name: Configure ufw firewall to allow SSH connections on port 22 # - name: Configure ufw firewall to allow SSH connections on port 22
community.general.ufw: # community.general.ufw:
rule: allow # rule: allow
port: "22" # port: "22"
- name: Configure ufw firewall to allow HTTP connections on port 80 # - name: Configure ufw firewall to allow HTTP connections on port 80
community.general.ufw: # community.general.ufw:
rule: allow # rule: allow
port: "80" # port: "80"
- name: Configure ufw firewall to allow HTTPS connections on port 443 # - name: Configure ufw firewall to allow HTTPS connections on port 443
community.general.ufw: # community.general.ufw:
rule: allow # rule: allow
port: "443" # port: "443"
- name: Enable ufw firewall with all other ports closed by default # - name: Enable ufw firewall with all other ports closed by default
community.general.ufw: # community.general.ufw:
state: enabled # state: enabled
policy: deny # policy: deny
- name: Install unattended-upgrades # - name: Install unattended-upgrades
apt: # apt:
name: unattended-upgrades # name: unattended-upgrades
- name: Enable unattended-upgrades to auto-upgrade our system # - name: Enable unattended-upgrades to auto-upgrade our system
copy: # copy:
content: | # content: |
APT::Periodic::Update-Package-Lists "1"; # APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1"; # APT::Periodic::Unattended-Upgrade "1";
dest: /etc/apt/apt.conf.d/20auto-upgrades # dest: /etc/apt/apt.conf.d/20auto-upgrades
- name: Configure unattended-upgrades to auto-reboot our server when necessary # - name: Configure unattended-upgrades to auto-reboot our server when necessary
lineinfile: # lineinfile:
regex: ^(//\s*)?Unattended-Upgrade::Automatic-Reboot ".*";$ # regex: ^(//\s*)?Unattended-Upgrade::Automatic-Reboot ".*";$
line: Unattended-Upgrade::Automatic-Reboot "true"; # line: Unattended-Upgrade::Automatic-Reboot "true";
dest: /etc/apt/apt.conf.d/50unattended-upgrades # dest: /etc/apt/apt.conf.d/50unattended-upgrades
- name: Configure unattended-upgrades to delay necessary reboots to 3am # - name: Configure unattended-upgrades to delay necessary reboots to 3am
lineinfile: # lineinfile:
regex: ^(//\s*)?Unattended-Upgrade::Automatic-Reboot-Time ".*";$ # regex: ^(//\s*)?Unattended-Upgrade::Automatic-Reboot-Time ".*";$
line: Unattended-Upgrade::Automatic-Reboot-Time "03:00"; # line: Unattended-Upgrade::Automatic-Reboot-Time "03:00";
dest: /etc/apt/apt.conf.d/50unattended-upgrades # dest: /etc/apt/apt.conf.d/50unattended-upgrades
- name: Configure the system timezone to be US Pacific time # - name: Configure the system timezone to be US Pacific time
community.general.timezone: # community.general.timezone:
name: America/Los_Angeles # name: America/Los_Angeles
- name: Create "impress" user # - name: Create "impress" user
user: # user:
name: impress # name: impress
comment: Impress App # comment: Impress App
home: /srv/impress # home: /srv/impress
create_home: false # create_home: false
shell: /bin/bash # shell: /bin/bash
- name: Create "impress-deployers" group # - name: Create "impress-deployers" group
group: # group:
name: impress-deployers # name: impress-deployers
- name: Add the current user to the "impress-deployers" group # - name: Add the current user to the "impress-deployers" group
user: # user:
name: "{{ lookup('env', 'USER') }}" # name: "{{ lookup('env', 'USER') }}"
groups: # groups:
- impress-deployers # - impress-deployers
append: yes # append: yes
# We use this so the deploy playbook doesn't have to prompt for a root # # We use this so the deploy playbook doesn't have to prompt for a root
# password: this user just is trusted to act as "impress" in the future. # # password: this user just is trusted to act as "impress" in the future.
- name: Enable the "impress-deployers" group to freely act as the "impress" user # - name: Enable the "impress-deployers" group to freely act as the "impress" user
community.general.sudoers: # community.general.sudoers:
name: impress-deployers-as-impress # name: impress-deployers-as-impress
group: impress-deployers # group: impress-deployers
runas: impress # runas: impress
commands: ALL # commands: ALL
nopassword: yes # nopassword: yes
# Similarly, this enables us to manage the impress service in the deploy playbook # # Similarly, this enables us to manage the impress service in the deploy playbook
# and in live debugging without a password. # # and in live debugging without a password.
# NOTE: In the sudoers file, you need to specify the full path to the # # NOTE: In the sudoers file, you need to specify the full path to the
# command, to avoid tricks where you use PATH to get around the intent! # # command, to avoid tricks where you use PATH to get around the intent!
- name: Enable the "impress-deployers" group to freely start and stop the impress service # - name: Enable the "impress-deployers" group to freely start and stop the impress service
community.general.sudoers: # community.general.sudoers:
name: impress-deployers-systemctl # name: impress-deployers-systemctl
group: impress-deployers # group: impress-deployers
commands: # commands:
- /bin/systemctl status impress # - /bin/systemctl status impress
- /bin/systemctl start impress # - /bin/systemctl start impress
- /bin/systemctl stop impress # - /bin/systemctl stop impress
- /bin/systemctl restart impress # - /bin/systemctl restart impress
nopassword: yes # nopassword: yes
- name: Install ACL, to enable us to run commands as the "impress" user # - name: Install ACL, to enable us to run commands as the "impress" user
apt: # apt:
name: acl # name: acl
- name: Install ruby-build # - name: Install ruby-build
git: # git:
repo: https://github.com/rbenv/ruby-build.git # repo: https://github.com/rbenv/ruby-build.git
dest: /opt/ruby-build # dest: /opt/ruby-build
version: 4d4678bc1ed89aa6900c0ea0da23495445dbcf50 # version: 4d4678bc1ed89aa6900c0ea0da23495445dbcf50
- name: Check if Ruby 3.1.4 is already installed # - name: Check if Ruby 3.1.4 is already installed
stat: # stat:
path: /opt/ruby-3.1.4 # path: /opt/ruby-3.1.4
register: ruby_dir # register: ruby_dir
- name: Install Ruby 3.1.4 # - name: Install Ruby 3.1.4
command: "/opt/ruby-build/bin/ruby-build 3.1.4 /opt/ruby-3.1.4" # command: "/opt/ruby-build/bin/ruby-build 3.1.4 /opt/ruby-3.1.4"
when: not ruby_dir.stat.exists # when: not ruby_dir.stat.exists
- name: Add Ruby 3.1.4 to the global PATH, for developer convenience # - name: Add Ruby 3.1.4 to the global PATH, for developer convenience
lineinfile: # lineinfile:
dest: /etc/profile # dest: /etc/profile
line: 'PATH="/opt/ruby-3.1.4/bin:$PATH" # Added by impress deploy setup script' # line: 'PATH="/opt/ruby-3.1.4/bin:$PATH" # Added by impress deploy setup script'
- name: Install system dependencies for impress's Ruby gems # - name: Install system dependencies for impress's Ruby gems
apt: # apt:
name: # name:
- libmysqlclient-dev # - libmysqlclient-dev
- libyaml-dev # - libyaml-dev
- name: Create the app folder # - name: Create the app folder
file: # file:
path: /srv/impress # path: /srv/impress
owner: impress # owner: impress
group: impress # group: impress
mode: "755" # mode: "755"
state: directory # state: directory
- name: Add a convenient .bash_profile for when we log in as "impress" # - name: Add a convenient .bash_profile for when we log in as "impress"
copy: # copy:
owner: impress # owner: impress
group: impress # group: impress
dest: /srv/impress/.bash_profile # dest: /srv/impress/.bash_profile
content: | # content: |
set -a # Export all of the below # set -a # Export all of the below
RAILS_ENV=production # RAILS_ENV=production
EXECJS_RUNTIME=Disabled # EXECJS_RUNTIME=Disabled
source /srv/impress/shared/production.env # source /srv/impress/shared/production.env
set +a # set +a
- name: Create the app's "versions" folder # - name: Create the app's "versions" folder
become_user: impress # become_user: impress
file: # file:
path: /srv/impress/versions # path: /srv/impress/versions
state: directory # state: directory
- name: Create the app's "shared" folder # - name: Create the app's "shared" folder
become_user: impress # become_user: impress
file: # file:
path: /srv/impress/shared # path: /srv/impress/shared
state: directory # state: directory
- name: Check for a current app version # - name: Check for a current app version
stat: # stat:
path: /srv/impress/current # path: /srv/impress/current
register: current_app_version # register: current_app_version
- name: Check whether we already have a placeholder app # - name: Check whether we already have a placeholder app
stat: # stat:
path: /srv/impress/versions/initial-placeholder # path: /srv/impress/versions/initial-placeholder
register: existing_placeholder_app # register: existing_placeholder_app
when: not current_app_version.stat.exists # when: not current_app_version.stat.exists
- name: Create a placeholder app, to run until we deploy a real version # - name: Create a placeholder app, to run until we deploy a real version
become_user: impress # become_user: impress
copy: # copy:
src: files/initial-placeholder # src: files/initial-placeholder
dest: /srv/impress/versions # dest: /srv/impress/versions
when: | # when: |
not current_app_version.stat.exists and # not current_app_version.stat.exists and
not existing_placeholder_app.stat.exists # not existing_placeholder_app.stat.exists
- name: Configure the placeholder app to run in deployment mode # - name: Configure the placeholder app to run in deployment mode
become_user: impress # become_user: impress
command: # command:
chdir: /srv/impress/versions/initial-placeholder # chdir: /srv/impress/versions/initial-placeholder
cmd: /opt/ruby-3.1.4/bin/bundle config set --local deployment true # cmd: /opt/ruby-3.1.4/bin/bundle config set --local deployment true
when: not current_app_version.stat.exists # when: not current_app_version.stat.exists
- name: Install the placeholder app's dependencies # - name: Install the placeholder app's dependencies
become_user: impress # become_user: impress
command: # command:
chdir: /srv/impress/versions/initial-placeholder # chdir: /srv/impress/versions/initial-placeholder
cmd: /opt/ruby-3.1.4/bin/bundle install # cmd: /opt/ruby-3.1.4/bin/bundle install
when: not current_app_version.stat.exists # when: not current_app_version.stat.exists
- name: Set the placeholder app as the current version # - name: Set the placeholder app as the current version
become_user: impress # become_user: impress
file: # file:
src: /srv/impress/versions/initial-placeholder # src: /srv/impress/versions/initial-placeholder
dest: /srv/impress/current # dest: /srv/impress/current
state: link # state: link
when: not current_app_version.stat.exists # when: not current_app_version.stat.exists
# NOTE: This file is uploaded with stricter permissions, to help protect # # NOTE: This file is uploaded with stricter permissions, to help protect
# the secrets inside. Most of the app is world-readable for convenience # # the secrets inside. Most of the app is world-readable for convenience
# for debugging and letting nginx serve static files, but keep this safer! # # for debugging and letting nginx serve static files, but keep this safer!
- name: Upload the production.env file # - name: Upload the production.env file
become_user: impress # become_user: impress
copy: # copy:
dest: /srv/impress/shared/production.env # dest: /srv/impress/shared/production.env
src: files/production.env # src: files/production.env
mode: "600" # mode: "600"
notify: # notify:
- Reload systemctl # - Reload systemctl
- Restart impress # - Restart impress
- name: Create service file for impress - name: Create service file for impress
copy: copy:

12
falcon.rb Normal file
View file

@ -0,0 +1,12 @@
#!/usr/bin/env -S falcon host
# frozen_string_literal: true
load :rack, :supervisor
hostname = File.basename(__dir__)
rack hostname do
endpoint Async::HTTP::Endpoint.parse('http://localhost:3000').
with(protocol: Async::HTTP::Protocol::HTTP1)
end
supervisor

BIN
vendor/cache/async-container-0.16.12.gem vendored Normal file

Binary file not shown.

BIN
vendor/cache/async-http-cache-0.4.3.gem vendored Normal file

Binary file not shown.

Binary file not shown.

BIN
vendor/cache/falcon-0.42.3.gem vendored Normal file

Binary file not shown.

BIN
vendor/cache/localhost-1.1.10.gem vendored Normal file

Binary file not shown.

BIN
vendor/cache/mapping-1.1.1.gem vendored Normal file

Binary file not shown.

BIN
vendor/cache/openssl-3.2.0.gem vendored Normal file

Binary file not shown.

BIN
vendor/cache/process-metrics-0.2.1.gem vendored Normal file

Binary file not shown.

BIN
vendor/cache/protocol-rack-0.4.1.gem vendored Normal file

Binary file not shown.

Binary file not shown.

BIN
vendor/cache/puma-6.4.2.gem vendored Normal file

Binary file not shown.

BIN
vendor/cache/samovar-2.2.0.gem vendored Normal file

Binary file not shown.