impress/deploy/files/impress.service

[Unit]
Description=Dress to Impress webapp

[Service]
User=impress
Restart=always
WorkingDirectory=/srv/impress/current
ExecStart=/opt/ruby-3.1.4/bin/bundle exec puma --port=3000
Environment="RAILS_ENV=production"
; Set EXECJS_RUNTIME to save us from needing to install Node
Environment="EXECJS_RUNTIME=Disabled"
EnvironmentFile=/srv/impress/shared/production.env

; Try not to go over 75% of the server's RAM, and kill the app at 80%.
; NOTE: This assumes this is the main thing running on the box! If it were
; running alongside e.g. a database, you'd want to constrain it differently.
MemoryHigh=75%
MemoryMax=80%

; Some security directives, adapted from Akkoma's service file, they seem like sensible defaults!
; Use private /tmp and /var/tmp folders inside a new file system namespace, which are discarded after the process stops.
PrivateTmp=true
; The /home, /root, and /run/user folders can not be accessed by this service anymore. If your Akkoma user has its home folder in one of the restricted places, or use one of these folders as its working directory, you have to set this to false.
ProtectHome=true
; Mount /usr, /boot, and /etc as read-only for processes invoked by this service.
ProtectSystem=full
; Sets up a new /dev mount for the process and only adds API pseudo devices like /dev/null, /dev/zero or /dev/random but not physical devices.
PrivateDevices=true
; Ensures that the service process and all its children can never gain new privileges through execve().
NoNewPrivileges=true
; Drops the sysadmin capability from the daemon.
CapabilityBoundingSet=~CAP_SYS_ADMIN

[Install]
WantedBy=multi-user.target
Extract some files out of the deploy script Okay, there's enough going on in here now that I don't like it inline anymore! Welcome to `files`! 2023-10-25 15:40:43 -07:00			`[Unit]`
			`Description=Dress to Impress webapp`

			`[Service]`
			`User=impress`
			`Restart=always`
			`WorkingDirectory=/srv/impress/current`
			`ExecStart=/opt/ruby-3.1.4/bin/bundle exec puma --port=3000`
			`Environment="RAILS_ENV=production"`
			`; Set EXECJS_RUNTIME to save us from needing to install Node`
			`Environment="EXECJS_RUNTIME=Disabled"`
			`EnvironmentFile=/srv/impress/shared/production.env`

Add RAM constraints to impress service on in production I just restarted the impress app in production! First I logged in to see why it wasn't responding, and I saw that there was almost no free RAM left, and that the Rails app had grown to eat it all up! So in this change, we set a memory limit: if the impress app is taking up more than 75% of the machine's RAM, systemctl will try to shrink it down; if it can't, then it will kill the app at 80%. I'm not totally sure whether these bounds are tight enough? I didn't look closely enough at the numbers to see what the app's actual usage was according to systemctl at the time (`sudo systemctl status impress`), so my hope is this is enough. But if we run into a memory leak crash like that again, because it turns out even existing at 75% RAM freezes the machine when running alongside its other processes, we can decrease these numbers! I also don't know the nature of the memory leak, and that could be worth investigating—the app pretty cleanly fits into ~500–600MB when it starts up, but then does seem to slowly but steadily grow. If it could be kept at that size, it's possible we could downgrade the server and save some costs—but that's a question for another day, since making sure we handle memory leaks when they do happen is a more important robustness fix! 2023-10-26 13:51:47 -07:00			`; Try not to go over 75% of the server's RAM, and kill the app at 80%.`
			`; NOTE: This assumes this is the main thing running on the box! If it were`
			`; running alongside e.g. a database, you'd want to constrain it differently.`
			`MemoryHigh=75%`
			`MemoryMax=80%`

Extract some files out of the deploy script Okay, there's enough going on in here now that I don't like it inline anymore! Welcome to `files`! 2023-10-25 15:40:43 -07:00			`; Some security directives, adapted from Akkoma's service file, they seem like sensible defaults!`
			`; Use private /tmp and /var/tmp folders inside a new file system namespace, which are discarded after the process stops.`
			`PrivateTmp=true`
			`; The /home, /root, and /run/user folders can not be accessed by this service anymore. If your Akkoma user has its home folder in one of the restricted places, or use one of these folders as its working directory, you have to set this to false.`
			`ProtectHome=true`
			`; Mount /usr, /boot, and /etc as read-only for processes invoked by this service.`
			`ProtectSystem=full`
			`; Sets up a new /dev mount for the process and only adds API pseudo devices like /dev/null, /dev/zero or /dev/random but not physical devices.`
			`PrivateDevices=true`
			`; Ensures that the service process and all its children can never gain new privileges through execve().`
			`NoNewPrivileges=true`
			`; Drops the sysadmin capability from the daemon.`
			`CapabilityBoundingSet=~CAP_SYS_ADMIN`

			`[Install]`
			`WantedBy=multi-user.target`