Commit graph

1453 commits

Author SHA1 Message Date
c4e32d92f1 Fix Typescript error with caching + new Apollo
Okay, right, I guess the Apollo Server upgrade broke type checking in our weird cache control hacks! This should force them to be compatible!
2021-11-26 15:22:01 -08:00
c032d4a00b Merge branch 'main' of github.com:matchu/impress-2020 into main 2021-11-26 15:09:53 -08:00
f32de55ed0 Handle more lists, longer list names, on item page
Give the grid a fixed size, have the list name stuff get ellipsis when it's too long, and try to show all list names (which will almost certainly too long for the space) to give a better hint of what's in there.
2021-11-26 15:05:07 -08:00
ad249d03ab WIP list editing on item page
Gonna have this button pop up a little thing of your lists, with checkboxes, and I guess probably quantities once I add that concept back to 2020 😅
2021-11-26 14:49:21 -08:00
80162e4c92
Merge pull request #13 from matchu/dependabot/npm_and_yarn/lodash-4.17.21
Bump lodash from 4.17.15 to 4.17.21
2021-11-26 14:48:12 -08:00
e8f6050bc0
Merge pull request #12 from matchu/dependabot/npm_and_yarn/aws-sdk-2.814.0
Bump aws-sdk from 2.726.0 to 2.814.0
2021-11-26 14:47:36 -08:00
dependabot[bot]
824a3d0fa2
Bump lodash from 4.17.15 to 4.17.21
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.21)

---
updated-dependencies:
- dependency-name: lodash
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:47:18 +00:00
dependabot[bot]
6f3ea6311e
Bump aws-sdk from 2.726.0 to 2.814.0
Bumps [aws-sdk](https://github.com/aws/aws-sdk-js) from 2.726.0 to 2.814.0.
- [Release notes](https://github.com/aws/aws-sdk-js/releases)
- [Changelog](https://github.com/aws/aws-sdk-js/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-js/compare/v2.726.0...v2.814.0)

---
updated-dependencies:
- dependency-name: aws-sdk
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:47:07 +00:00
894fe7d3df
Merge pull request #11 from matchu/dependabot/npm_and_yarn/glob-parent-5.1.2
Bump glob-parent from 5.1.0 to 5.1.2
2021-11-26 14:46:42 -08:00
2a25eeb529
Merge pull request #10 from matchu/dependabot/npm_and_yarn/browserslist-4.16.6
Bump browserslist from 4.16.0 to 4.16.6
2021-11-26 14:46:33 -08:00
1506299cf0
Merge pull request #9 from matchu/dependabot/npm_and_yarn/path-parse-1.0.7
Bump path-parse from 1.0.6 to 1.0.7
2021-11-26 14:46:02 -08:00
9b63471583
Merge pull request #8 from matchu/dependabot/npm_and_yarn/apollo-server-2.25.3
Bump apollo-server from 2.19.2 to 2.25.3
2021-11-26 14:45:39 -08:00
dependabot[bot]
7212b36a9d
Bump glob-parent from 5.1.0 to 5.1.2
Bumps [glob-parent](https://github.com/gulpjs/glob-parent) from 5.1.0 to 5.1.2.
- [Release notes](https://github.com/gulpjs/glob-parent/releases)
- [Changelog](https://github.com/gulpjs/glob-parent/blob/main/CHANGELOG.md)
- [Commits](https://github.com/gulpjs/glob-parent/compare/v5.1.0...v5.1.2)

---
updated-dependencies:
- dependency-name: glob-parent
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:45:27 +00:00
dependabot[bot]
cca45ca63b
Bump browserslist from 4.16.0 to 4.16.6
Bumps [browserslist](https://github.com/browserslist/browserslist) from 4.16.0 to 4.16.6.
- [Release notes](https://github.com/browserslist/browserslist/releases)
- [Changelog](https://github.com/browserslist/browserslist/blob/main/CHANGELOG.md)
- [Commits](https://github.com/browserslist/browserslist/compare/4.16.0...4.16.6)

---
updated-dependencies:
- dependency-name: browserslist
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:45:23 +00:00
dependabot[bot]
8413d601ad
Bump apollo-server from 2.19.2 to 2.25.3
Bumps [apollo-server](https://github.com/apollographql/apollo-server/tree/HEAD/packages/apollo-server) from 2.19.2 to 2.25.3.
- [Release notes](https://github.com/apollographql/apollo-server/releases)
- [Changelog](https://github.com/apollographql/apollo-server/blob/main/CHANGELOG.md)
- [Commits](https://github.com/apollographql/apollo-server/commits/apollo-server@2.25.3/packages/apollo-server)

---
updated-dependencies:
- dependency-name: apollo-server
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:45:08 +00:00
dependabot[bot]
e6b6ab6945
Bump path-parse from 1.0.6 to 1.0.7
Bumps [path-parse](https://github.com/jbgutierrez/path-parse) from 1.0.6 to 1.0.7.
- [Release notes](https://github.com/jbgutierrez/path-parse/releases)
- [Commits](https://github.com/jbgutierrez/path-parse/commits/v1.0.7)

---
updated-dependencies:
- dependency-name: path-parse
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:44:50 +00:00
be16bb9cc0
Merge pull request #7 from matchu/dependabot/npm_and_yarn/tar-4.4.19
Bump tar from 4.4.13 to 4.4.19
2021-11-26 14:44:49 -08:00
c335924bb6
Merge pull request #6 from matchu/dependabot/npm_and_yarn/ws-5.2.3
Bump ws from 5.2.2 to 5.2.3
2021-11-26 14:44:35 -08:00
7314598444
Merge pull request #5 from matchu/dependabot/npm_and_yarn/object-path-0.11.8
Bump object-path from 0.11.4 to 0.11.8
2021-11-26 14:44:15 -08:00
dependabot[bot]
a318f87552
Bump tar from 4.4.13 to 4.4.19
Bumps [tar](https://github.com/npm/node-tar) from 4.4.13 to 4.4.19.
- [Release notes](https://github.com/npm/node-tar/releases)
- [Changelog](https://github.com/npm/node-tar/blob/main/CHANGELOG.md)
- [Commits](https://github.com/npm/node-tar/compare/v4.4.13...v4.4.19)

---
updated-dependencies:
- dependency-name: tar
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:43:06 +00:00
dependabot[bot]
ad1965615a
Bump ws from 5.2.2 to 5.2.3
Bumps [ws](https://github.com/websockets/ws) from 5.2.2 to 5.2.3.
- [Release notes](https://github.com/websockets/ws/releases)
- [Commits](https://github.com/websockets/ws/compare/5.2.2...5.2.3)

---
updated-dependencies:
- dependency-name: ws
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:43:06 +00:00
dependabot[bot]
8f1f809c55
Bump object-path from 0.11.4 to 0.11.8
Bumps [object-path](https://github.com/mariocasciaro/object-path) from 0.11.4 to 0.11.8.
- [Release notes](https://github.com/mariocasciaro/object-path/releases)
- [Commits](https://github.com/mariocasciaro/object-path/commits/v0.11.8)

---
updated-dependencies:
- dependency-name: object-path
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:43:05 +00:00
2e41f7bb0b Send Vary: Authorization cache header
I don't think this is actually relevant in-app right now, but I figured sending it is More Correct, and is likely to prevent future bugs if anything (and prevent future question about why we're _not_ sending it).

I also removed the `maxAge: 0` on `currentUser`, now that I've updated Fastly to no longer default to 5-minute caching when no cache time is specified. I can see why that's a reasonable default for Fastly, but we've been pretty careful about specifying Cache-Control headers when relevant, so the extra caching is mostly incorrect.
2021-11-23 13:00:56 -08:00
d4b115e805 Merge branch 'ansible' of github.com:matchu/impress-2020 into ansible 2021-11-23 12:42:59 -08:00
96d6d42120 Use persisted queries in dev, bc Next.js support
We had previously configured the client to not bother to try a GET request for GraphQL queries, and just jump straight to POST instead, because the `vercel dev` server for create-react-app reloaded the backend code for every request anyway, which doubled the dev response time.

The Next.js server is more efficient than this, and keeps some memory, so GET requests work similarly in dev as on prod now! (i.e. it fails the first time, but then succeeds on the second)

In this change, we remove the code to skip `createPersistedQueryLink` in development, and instead always call it. We simplify the code accordingly, too.
2021-11-23 12:35:20 -08:00
b941dce9fa Private cache headers in item search
If the user is searching for things they own or want, make sure we don't CDN cache it!

For many queries, this is taken care of in practice, because the search result includes `currentUserOwnsThis` and `currentUserWantsThis`. But I noticed in testing that, if the search result has no items, so those fields aren't actually part of the _response_, then the private header doesn't get set. So this mainly makes sure we don't accidentally cache an empty result from a user who didn't have anything they owned/wanted yet!
2021-11-16 13:09:45 -08:00
b73e2e1123 Send cache-control header for max-age=0, private
Some queries, like on `/your-outfits`, had the cache hint `max-age=0, private` set. In this case, our cache code sent no cache header, on the assumption that no header would result in no caching.

This was true on Vercel, but isn't true on our new Fastly setup! (Which makes sense, Vercel was a bit more aggressive here I think.)

This was causing an arbitrary user's data to be cached by Fastly as the result for `/your-outfits`. (We found this bug before launching the Fastly cache though, don't worry! No actual user data leaked!)

Now, as of this change, the `/your-outfits` query correctly sends a header of `Cache-Control: max-age=0, private`. This directs Fastly not to cache the result.

To fix this, we made a change to our HTTP header code, which is forked from Apollo's stuff.
2021-11-16 12:34:11 -08:00
cadf7487af Mark currentUser GQL as non-cacheable
Comments explain most of this! Vercel changed around the Cache-Control headers a bit to always essentially apply max-age:0 when scope:PRIVATE was true.

I'm noticing this isn't *fully* working yet though, because we're not getting a `Cache-Control: private` header, we're just getting no header at all. Fastly might aggressively choose to cache it anyway with etag stuff! I bet that's the fault of our caching middleware plugin thing, so I'll check on that!
2021-11-16 12:12:51 -08:00
002af474f8 Don't HTTP cache currentUserOwnsThis/wants
Hmm, I see, Vercel chews on Cache-Control headers a bit more than I'm used to, so anything marked `scope: PRIVATE` would not be cached at all.

But on a more standard server, this was coming out as privately cacheable, but for an actual amount of time (1 hour in the homepage case), because of the `maxAge` on other fields. That meant the device browser cache would hold onto the result, and not always reflect Own/Want changes upon page reload.

In this change, we set `maxAge: 0`, because we want this field to be very responsive. I also left `scope: PRIVATE`, even though I think it doesn't really matter if we're saying the field isn't cacheable anyway, because I want to set the precendent that `currentUser` fields need it, to avoid a potential gotcha if someone creates a cacheable `currentUser` field in the future. (That's important to be careful with though, because is it even okay for logouts to not clear it? TODO: Can we clear the private HTTP cache somehow? I guess we would need to include the current user ID in the URL?)
2021-11-16 12:04:16 -08:00
a67509aaec Merge branch 'main' into ansible 2021-11-16 11:23:01 -08:00
bf8fd81305 Post a message linking to Metaverse revocation 2021-11-15 17:42:02 -08:00
5039371a1d Simplify the page pool
Yeah ok, let's just run one browser instance and one pool.

I feel like I observed that, when I killed chromium in prod, pm2 noticed the abrupt loss of a child process and restarted the whole app process? which is rad? so maybe let's just trying relying on that and see how it goes
2021-11-13 02:27:24 -08:00
0c2939dfe4 Use Puppeteer instead of Playwright
We used Playwright in the first place to try to work around a Vercel deploy issue, and I'm not sure it really ended up mattering lol :p

But yeah, I'm putting the new Puppeteer code through the same prod stress test, and it just doesn't seem to be getting into the same broken state that Playwright was. I'm guessing it's just that Puppeteer has more investment in edge-case handling? (There's also the fact that we're no longer running things as root, which could have been a fucky problem, too?)
2021-11-13 02:16:58 -08:00
587aa09efc Oops, fix bug in pm2 setup
Oh, I made a typo that caused pm2 to be running our processes as `root` instead of `matchu`! Let's very fix that!! 😳

I noticed this because I'm trying Puppeteer, and it got upset about running in sandboxed mode as root, and I'm like "as root??"

So yeah, good, fixed, lol 😬
2021-11-13 02:12:05 -08:00
20fff261ef Switch to a small page pool
Hmm I am really not managing to keep the processes under control… maybe I'll try Puppeteer and see if it's just a bit more reliable??
2021-11-13 01:45:27 -08:00
18bc3df6f4 Use browser pooling for /api/assetImage
I tried running a pressure test against assetImage on prod with the open-source tool `wrk`:

```
wrk -t12 -c20 -d20s --timeout 20s 'https://impress-2020-box.openneo.net/api/assetImage?libraryUrl=https%3A%2F%2Fimages.neopets.com%2Fcp%2Fitems%2Fdata%2F000%2F000%2F522%2F522756_2bde0443ae%2F522756.js&size=600'
```

I found that, unsurprisingly, we run a lot of concurrent requests, which fill up memory with a lot of Chromium instances!

In this change, we declare a small pool of 2 browser contexts, to allow a bit of concurrency but still very strictly limit how many browser instances can actually get created. We might tune this number depending on the actual performance characteristics!
2021-11-12 23:35:30 -08:00
3ec0ae7557 Use localhost in /api/assetImage
Just another VERCEL_URL removal!
2021-11-12 22:08:06 -08:00
0a81f07849 Remove Waka values
The motivation is that I want VERCEL_URL and local net requests outta here :p and we were doing some cutesiness with leveraging the CDN cache to back the GQL fields. No more of that, folks! lol
2021-11-12 22:06:50 -08:00
991defffa1 /api/outfitImage makes direct GQL queries
Previously we were using HTTP queries to keep individual function bundle sizes small, but that doesn't matter in a server where all the code is shared!

The immediate motivation is that I want /api/outfitImage requesting against the same server, not impress-2020.openneo.net. For other stuff I'm probably gonna fix this by replacing VERCEL_URL with something else, but here I figured this was a change worth making anyway.
2021-11-12 21:53:22 -08:00
afd23fb4dd Bump version of graphql
This was actually kinda accidental, I thought I could uninstall it but then realized I couldn't. Anyway, it's updated now!
2021-11-12 21:52:14 -08:00
eaadfd09ef Delete outfitPageSSR
Oh right, we implemented this with Next.js SSR in `/pages/outfits/[id].js`, so we don't need this anymore!
2021-11-12 21:41:17 -08:00
9753cbe173 /api/assetImage fixes in production
Now that we're not on Vercel's AWS Lambda deployment, we can switch to something a bit more standard!

I also tweaked up our version of Playwright, because, hey, why not?

Getting the package list was a bit tricky, but we got there! Left a comment to explain where it's from.
2021-11-12 21:39:35 -08:00
5470a49651 Use utf8 in API error messages
I noticed this when Playwright was trying to draw cute ASCII art and it wasn't showing up right! Not a big deal, but it's a bit more correct to do this, so let's do it!
2021-11-12 21:17:20 -08:00
07d54b9a9e Add SSH keys to deploy-setup
This helps me set up new devices, while still keeping passworded SSH access locked down!
2021-11-12 20:14:44 -08:00
30d582a9c4 Re-add support for nice outfit image URLs
We removed this earlier in 7205455ccb, but now it's time to re-add it!
2021-11-12 19:53:34 -08:00
d37d958a36 Enable automatic updates & reboots on deploy box
Oh right, like the SSH stuff, I did this the first time I set up, but didn't add it to the script! I like having things in the script :3 (I also had forgotten to check on the time zone last time, nice to have it with some rigor!)
2021-11-04 19:17:35 -07:00
8f28f87bee Close most ports on the deploy box by default
I noticed that incoming port 3000 connections were being allowed, oops! Not a huge deal, but I don't want to allow connections without HTTPS, and I don't want surprise surface area even if I'm not currently aware of attacks on it. Close it out!
2021-11-04 18:57:00 -07:00
9310a250d6 Fix some bugs running deploy-setup from scratch
As an exercise, I've wiped the box clean, and I'm reinstalling from the scripts! :3

I added the SSH hardening rules to the playbook instead of doing them by hand this time.

I made a mistake with creating `/srv/impress-2020`, right, you need to *say* what it should be created *as* for the creation step to work!

I also guess my recent pm2 changes made it not actually be willing to start the app anymore, because `/srv/impress-2020/current` doesn't exist or have `node_modules` yet. I'm doing a cute thing where I create a placeholder app during setup, so there's always something to run, without introducing the complexities of a real deploy to the setup process.

And right, of course, we need to install nginx before running certbot! But we need to add certbot config *after* running certbot!

And then just some misc cleanups for consistency and correctness!
2021-11-03 23:11:50 -07:00
1e3e8391b4 Oops, fix YAML + bash mistake
Okay so you know how in 3f07933f7a I switched the newline stuff here?

Yeah, right, I forgot, newlines are significant in bash :p I forgot this because I've never used them inside a `bash -c` invocation, but like, of course

Now, I'm still using `|` for clarity and reduced dependence on magic, but getting my lines right :p
2021-11-03 17:34:07 -07:00
32ae1accce Oops, fix testing for node_modules
I somehow had it in my head that `realpath` would crash if the file didn't exist, but that's super not true! It returns the tentative path for if you _were_ gonna create this!
2021-11-03 17:28:10 -07:00