Emi Matchu
bc8d265672
According to our GlitchTip error tracker, every time we deploy, a couple instances of `Async::Stop` and `Async::Container::Terminate` come in, presumably because: 1. systemd sends a STOP signal to the `falcon host` process. 2. `falcon host` gives the in-progress requests some time to finish up 3. Sometimes some requests take too long, and so something happens. (either a timer in Falcon or a KILL signal from systemd, not sure!) that leads the ongoing requests to finally be terminated by raising an `Async::Stop` or `Async::Container::Terminate`. (I'm not sure when each happens, and maybe they happen at different points in the process? Maybe one happens for the actual long-running ones, vs the other happens if more requests come in during the meantime but get caught in the spin-down process?) 4. Rails bubbles up the errors, our Sentry library notices them and sends them to GlitchTip, the user presumably receives the generic 500 error, and the app can finally close down gracefully. It's hard for me to validate that this is *exactly* what's happening here or that my mitigation makes sense, but my logic here is basically, if these exceptions are bubbling up as "uncaught exceptions" and spamming up our error log, then the best solution would be to catch them! So in this change, we add an error handler for these two error classes, which hopefully will 1) give users a better experience when this happens, and 2) no longer send these errors to our logging 🤞❗️ That strange phenomenon where the best way to get a noisy bug out of your logs is to fix it lmao
60 lines
1.2 KiB
HTML
60 lines
1.2 KiB
HTML
<!doctype html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="utf-8" />
|
|
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
|
<title>Dress to Impress: Oops, caught in a reboot!</title>
|
|
<style type="text/css">
|
|
body {
|
|
background-color: #fff;
|
|
color: #666;
|
|
font-family: arial, sans-serif;
|
|
padding: 2em 1em;
|
|
}
|
|
|
|
main {
|
|
border: 1px solid #ccc;
|
|
margin-inline: auto;
|
|
padding: 1em;
|
|
max-width: 600px;
|
|
|
|
display: grid;
|
|
grid-template-areas: "illustration body";
|
|
grid-template-columns: auto 1fr;
|
|
column-gap: 1em;
|
|
}
|
|
|
|
h1 {
|
|
font-size: 1.5em;
|
|
margin: 0;
|
|
margin-bottom: 0.5em;
|
|
}
|
|
|
|
p {
|
|
margin-bottom: 0.5em;
|
|
}
|
|
</style>
|
|
</head>
|
|
<body>
|
|
<main>
|
|
<img
|
|
width="100"
|
|
height="100"
|
|
alt="Distressed Grundo programmer"
|
|
src="/images/error-grundo.png"
|
|
/>
|
|
<div>
|
|
<h1>Oops, caught in a reboot!</h1>
|
|
<p>
|
|
Oh wow, hi! We're deploying a new version of DTI
|
|
<em>right now</em>, and your pageload got caught in the
|
|
middle of the restart process, sorry about this!
|
|
</p>
|
|
<p>
|
|
Reloading the page now should hopefully get you back where
|
|
you were going! 🤞
|
|
</p>
|
|
</div>
|
|
</main>
|
|
</body>
|
|
</html>
|