Commit graph

1375 commits

Author SHA1 Message Date
c608f01a7a Accept more HTTPS layer URLs in /api/outfitImage
I also added some docs for what these patterns are! Mainly because I wanted to add that TODO note, but felt it needed more context to make sense!
2022-08-03 15:06:38 -07:00
b9ba650992 Use HTTPS for AMFPHP requests
This was a bit trickier to figure out how to upgrade, it's not in the `xmlrpc` package's README, but I found the answer here: https://github.com/baalexander/node-xmlrpc/issues/142
2022-08-03 14:40:00 -07:00
bd9017796e Use HTTPS for images.neopets.com and pets.neopets.com
Tbh I didn't even really validate these changes, or that the codepaths right now aren't working, they just seem like clear drop-in upgrades now that HTTPS works and HTTP requests are redirected. Simplify!
2022-08-03 14:36:07 -07:00
6df1f49208 Add a limit to the modeling query
Right now it returns 50 rows; each item that needs modeling returns 1–4 rows, usually 1. So a limit of 200 should be pretty dangerous, while also creating a release valve if there's another future bug: it'll just have the problem of returning too few items, instead of the problem of crashing everything! 😅
2022-06-23 11:57:32 -07:00
c585b1236f Oops, fix crashes in the Modeling Hub!
Neopets released a new Maraquan Koi, and it revealed a mistake in our modeling query! We already knew that the Maraquan Mynci was actually the same body type as the standard Mynci colors, but now the Koi is the same way, and because there's _two_ such species, the query started reacting by assuming that a _bunch_ of items that fit both the standard Mynci and standard Koi (which is a LOT of items!!) should also fit all _Maraquan_ pets, because it fits both the Maraquan Mynci and Maraquan Koi too. (Whereas previously, that part of the query would say "oh, it just fits the Maraquan Mynci, we don't need to assume it fits ALL maraquan pets, that's probably just species-specific.")

so yeah! This change should help the query ignore Maraquan species that have the same body type as standard species. That's fine to essentially treat them like they don't exist, because we won't lose out on any modeling that way: the standard models will cover the Maraquan versions for those two species!
2022-06-20 15:21:38 -07:00
43d091c546
Merge pull request #15 from matchu/dependabot/npm_and_yarn/nanoid-3.2.0
Bump nanoid from 3.1.30 to 3.2.0
2022-03-11 09:26:40 -08:00
3e1537edf5
Merge pull request #18 from diceroll123/patch-1
Update Shop Wizard URL
2022-03-11 09:26:22 -08:00
6c3f2ac61c
Update Shop Wizard URL 2022-03-11 08:37:13 -05:00
cc9a540cec Fix Juppie Swirl bug in /api/validPetPoses
Oops, the new Juppie Swirl color has ID 114, and there's no released color #113 yet. But our `/api/validPetPoses` code, when deciding how large to make the byte array, uses the _number_ of colors in the database.

This meant that, when Juppie Swirl was released, there wasn't a 114th slot allocated, and the loop stopped at color ID #113—so the new Juppie Swirl color #114 wasn't included in the results. This made it impossible to select Juppie Swirl as a starting color. (You could, however, model a Juppie Swirl Chia, and the wardrobe would load it successfully; and you would see the color/species picker with the correct options selected, but in a red "invalid" state.)

Now, we instead use the largest ID in the database to determine the size of the array. This means Juppie Swirl is now included correctly!

There would be network perf implications if the color IDs were a sparser space, but it's dense enough to be totally fine in practice. (But let's not release an April Fools color #9999 or anything!)
2022-02-23 00:01:10 -08:00
c1eef6222b Oops, fix db pooling for scripts
Right, ok, `db.close()` needs to be `db.end()` now.

This probably didn't break the user-syncing cron job though, because that doesn't automatically update I think? so it should still be comfortably running older version of the code that should still work just fine
2022-02-03 16:14:40 -08:00
dependabot[bot]
573c26bb96
Bump nanoid from 3.1.30 to 3.2.0
Bumps [nanoid](https://github.com/ai/nanoid) from 3.1.30 to 3.2.0.
- [Release notes](https://github.com/ai/nanoid/releases)
- [Changelog](https://github.com/ai/nanoid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ai/nanoid/compare/3.1.30...3.2.0)

---
updated-dependencies:
- dependency-name: nanoid
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-01-22 09:39:35 +00:00
3744a476e5 Fix db transactions with pooling
Ah okay, pools support `query` and `execute` the same way connection objects do (as a shorthand for acquiring, querying, and releasing), but it doesn't have the same helpers for transactions. Makes sense: you need those queries to go to the same connection, and an API where you just call it against the pool object can't tell that it's part of the same thing!

Now, we have our transaction code explicitly acquire a connection to use for the duration of the transaction.

An alternative considered would have been to have `connectToDb` acquire a connection, and then release it at the end of the GraphQL request. That would have made app code simpler, but added a lot of additional potential surprise failure points to the infra imo (e.g. what if we're misunderstanding the GraphQL codepath and the connection never gets released? whereas here it's relatively easy to audit that there's a `finally` in the right spot.)
2022-01-08 18:49:00 -08:00
4ed6344b3d Use a connection pool
This should both fix cases where the connection closes for various reasons, by having the pool reconnect; and also should be a second way of solving some of the blocking issues we were having with large queries, by letting faster queries use parallel connections.

Idk what a reasonable number is, 10 seems to be what various guides are saying? Might tune it down if it ends up pushing various connection limits? (We could also constrain it on dev specifically, if that matters.)
2022-01-08 09:20:45 -08:00
ca812784a6 Reinstall yargs
Uhh hmm, I don't remember when we removed it from package.json, I guess
maybe I thought it was unused and didn't look carefully enough?

Anyway, this fixes the export-users-to-auth0 script, which was crashing
because yargs wasn't installed, oops!
2022-01-07 18:08:04 -08:00
e051290df4 Better-scoped queries for currentUserOwnsThis etc
I hypothesize that loading people's full trade lists more often than necessary is part of the cause of the recent mega slowdown!

My hypothesis is that we're clogging up the MySQL connection socket with a ton of data, which blocks all other queries until the big ones come through and parse out. (I haven't actually validated my assumption that MySQL connections send query results in serial like that, but it makes sense to me, and fits what I've been seeing.)

There's more places we could potentially optimize, like the trade list page itself… (we currently aggressively load everything when we could limit it and load the rest on the followup pages, or even paginate the followup pages…)

…but my hope is that this helps enough, by relieving the load on the homepage (latest items) and on item searches!
2022-01-07 11:37:27 -08:00
6ce8a5aea2 Update lists after click item page own/want button
This is a bit hacky, but I want to ship and I'm not in a mood for a refactor :P

Before this change, you could see a bug by doing the following:

1. Click "I own this" to own an item.
2. Click "Add a list" and add it to a list.
3. Click "I own this" to un-own the item. (This deletes it from all lists.)
4. Observe that the "Add a list" dropdown disappears.
5. Click "I own this" to own it again.
6. Observe that, before this change, the dropdown would reappear, but incorrectly say it was still in the old list. After this change, it appears with the blank "Add to list", as intended.
2021-11-30 16:52:38 -08:00
29d44c10bd Fix double-click bug unchecking own/want an item
Oops, I used the wrong property to control the checkbox state! This made it an uncontrolled component. It would always start unchecked when the page loads, regardless of actual own/want state, and then toggle based on physical clicks.

This meant that things generally worked correctly if you didn't own/want the item when you first loaded the page; but if you already did, then you would click once and send an *add* mutation instead of a remove; and then click again and be able to remove.

Now, removes only take one click!
2021-11-30 16:51:20 -08:00
e95f6abbe4 Closet list dropdowns on item page
Oooh this feature is feeling very nice :) :) We hid "not in a list" pretty smoothly I think!

A known bug: If you have the item in a list, then click the big colorful button, it will remove the item from *all* lists; and then if you click it again, it will add it to Not in a List. But! The UI will still show the lists it was in before, because we haven't updated the client cache. (It's not that bad in the middle state though, because the list dropdown stuff gets hidden.)
2021-11-30 16:36:00 -08:00
e6a94eaf80 Move a comment to satisfy VS Code emmet extension
My cute keybind to quickly wrap stuff in <Box> wasn't working in this file, and I figured out from deleting stuff and narrowing down that it was this comment. I guess Emmet's JSX parser doesn't like a comment being there! I moved it up a bit instead.
2021-11-30 15:01:05 -08:00
7c3298d5b2 Oops, fix SVG message and build error
Ah hm, not sure why this only showed up once I tried a prod deploy, but I declared `hiResMode` twice in there, because we already had fixed this bug for item layers but not pet layers!

In this change, I fix the duplicate `hiResMode` declaration, and update the new pet layers message to match the item layers message.
2021-11-27 20:01:38 -08:00
3642e4c32a Only show SVG glitch messages if SVGs are on
It's confusing to see a message that says "instead we're showing a PNG" if you don't have hi-res mode on and actually everything is PNG anyway!
2021-11-27 19:57:49 -08:00
19f1ec092e Turn on Honeycomb instrumentation again
Well, instrumentation seems to be working fine again! The bug we ran into during commit e5081dab7e is gone. Cool!

I want to be able to see what's making the new box slow. My hypothesis was (and it seems to be right) that communication with the database on the Classic DTI server is slow.

But now that they're on the same Linode account and region, I think I can set up a private VLAN to make them muuuch faster. We'll try it out!
2021-11-26 23:41:22 -08:00
c4e32d92f1 Fix Typescript error with caching + new Apollo
Okay, right, I guess the Apollo Server upgrade broke type checking in our weird cache control hacks! This should force them to be compatible!
2021-11-26 15:22:01 -08:00
c032d4a00b Merge branch 'main' of github.com:matchu/impress-2020 into main 2021-11-26 15:09:53 -08:00
f32de55ed0 Handle more lists, longer list names, on item page
Give the grid a fixed size, have the list name stuff get ellipsis when it's too long, and try to show all list names (which will almost certainly too long for the space) to give a better hint of what's in there.
2021-11-26 15:05:07 -08:00
ad249d03ab WIP list editing on item page
Gonna have this button pop up a little thing of your lists, with checkboxes, and I guess probably quantities once I add that concept back to 2020 😅
2021-11-26 14:49:21 -08:00
80162e4c92
Merge pull request #13 from matchu/dependabot/npm_and_yarn/lodash-4.17.21
Bump lodash from 4.17.15 to 4.17.21
2021-11-26 14:48:12 -08:00
e8f6050bc0
Merge pull request #12 from matchu/dependabot/npm_and_yarn/aws-sdk-2.814.0
Bump aws-sdk from 2.726.0 to 2.814.0
2021-11-26 14:47:36 -08:00
dependabot[bot]
824a3d0fa2
Bump lodash from 4.17.15 to 4.17.21
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.21)

---
updated-dependencies:
- dependency-name: lodash
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:47:18 +00:00
dependabot[bot]
6f3ea6311e
Bump aws-sdk from 2.726.0 to 2.814.0
Bumps [aws-sdk](https://github.com/aws/aws-sdk-js) from 2.726.0 to 2.814.0.
- [Release notes](https://github.com/aws/aws-sdk-js/releases)
- [Changelog](https://github.com/aws/aws-sdk-js/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-js/compare/v2.726.0...v2.814.0)

---
updated-dependencies:
- dependency-name: aws-sdk
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:47:07 +00:00
894fe7d3df
Merge pull request #11 from matchu/dependabot/npm_and_yarn/glob-parent-5.1.2
Bump glob-parent from 5.1.0 to 5.1.2
2021-11-26 14:46:42 -08:00
2a25eeb529
Merge pull request #10 from matchu/dependabot/npm_and_yarn/browserslist-4.16.6
Bump browserslist from 4.16.0 to 4.16.6
2021-11-26 14:46:33 -08:00
1506299cf0
Merge pull request #9 from matchu/dependabot/npm_and_yarn/path-parse-1.0.7
Bump path-parse from 1.0.6 to 1.0.7
2021-11-26 14:46:02 -08:00
9b63471583
Merge pull request #8 from matchu/dependabot/npm_and_yarn/apollo-server-2.25.3
Bump apollo-server from 2.19.2 to 2.25.3
2021-11-26 14:45:39 -08:00
dependabot[bot]
7212b36a9d
Bump glob-parent from 5.1.0 to 5.1.2
Bumps [glob-parent](https://github.com/gulpjs/glob-parent) from 5.1.0 to 5.1.2.
- [Release notes](https://github.com/gulpjs/glob-parent/releases)
- [Changelog](https://github.com/gulpjs/glob-parent/blob/main/CHANGELOG.md)
- [Commits](https://github.com/gulpjs/glob-parent/compare/v5.1.0...v5.1.2)

---
updated-dependencies:
- dependency-name: glob-parent
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:45:27 +00:00
dependabot[bot]
cca45ca63b
Bump browserslist from 4.16.0 to 4.16.6
Bumps [browserslist](https://github.com/browserslist/browserslist) from 4.16.0 to 4.16.6.
- [Release notes](https://github.com/browserslist/browserslist/releases)
- [Changelog](https://github.com/browserslist/browserslist/blob/main/CHANGELOG.md)
- [Commits](https://github.com/browserslist/browserslist/compare/4.16.0...4.16.6)

---
updated-dependencies:
- dependency-name: browserslist
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:45:23 +00:00
dependabot[bot]
8413d601ad
Bump apollo-server from 2.19.2 to 2.25.3
Bumps [apollo-server](https://github.com/apollographql/apollo-server/tree/HEAD/packages/apollo-server) from 2.19.2 to 2.25.3.
- [Release notes](https://github.com/apollographql/apollo-server/releases)
- [Changelog](https://github.com/apollographql/apollo-server/blob/main/CHANGELOG.md)
- [Commits](https://github.com/apollographql/apollo-server/commits/apollo-server@2.25.3/packages/apollo-server)

---
updated-dependencies:
- dependency-name: apollo-server
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:45:08 +00:00
dependabot[bot]
e6b6ab6945
Bump path-parse from 1.0.6 to 1.0.7
Bumps [path-parse](https://github.com/jbgutierrez/path-parse) from 1.0.6 to 1.0.7.
- [Release notes](https://github.com/jbgutierrez/path-parse/releases)
- [Commits](https://github.com/jbgutierrez/path-parse/commits/v1.0.7)

---
updated-dependencies:
- dependency-name: path-parse
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:44:50 +00:00
be16bb9cc0
Merge pull request #7 from matchu/dependabot/npm_and_yarn/tar-4.4.19
Bump tar from 4.4.13 to 4.4.19
2021-11-26 14:44:49 -08:00
c335924bb6
Merge pull request #6 from matchu/dependabot/npm_and_yarn/ws-5.2.3
Bump ws from 5.2.2 to 5.2.3
2021-11-26 14:44:35 -08:00
7314598444
Merge pull request #5 from matchu/dependabot/npm_and_yarn/object-path-0.11.8
Bump object-path from 0.11.4 to 0.11.8
2021-11-26 14:44:15 -08:00
dependabot[bot]
a318f87552
Bump tar from 4.4.13 to 4.4.19
Bumps [tar](https://github.com/npm/node-tar) from 4.4.13 to 4.4.19.
- [Release notes](https://github.com/npm/node-tar/releases)
- [Changelog](https://github.com/npm/node-tar/blob/main/CHANGELOG.md)
- [Commits](https://github.com/npm/node-tar/compare/v4.4.13...v4.4.19)

---
updated-dependencies:
- dependency-name: tar
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:43:06 +00:00
dependabot[bot]
ad1965615a
Bump ws from 5.2.2 to 5.2.3
Bumps [ws](https://github.com/websockets/ws) from 5.2.2 to 5.2.3.
- [Release notes](https://github.com/websockets/ws/releases)
- [Commits](https://github.com/websockets/ws/compare/5.2.2...5.2.3)

---
updated-dependencies:
- dependency-name: ws
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:43:06 +00:00
dependabot[bot]
8f1f809c55
Bump object-path from 0.11.4 to 0.11.8
Bumps [object-path](https://github.com/mariocasciaro/object-path) from 0.11.4 to 0.11.8.
- [Release notes](https://github.com/mariocasciaro/object-path/releases)
- [Commits](https://github.com/mariocasciaro/object-path/commits/v0.11.8)

---
updated-dependencies:
- dependency-name: object-path
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 22:43:05 +00:00
2e41f7bb0b Send Vary: Authorization cache header
I don't think this is actually relevant in-app right now, but I figured sending it is More Correct, and is likely to prevent future bugs if anything (and prevent future question about why we're _not_ sending it).

I also removed the `maxAge: 0` on `currentUser`, now that I've updated Fastly to no longer default to 5-minute caching when no cache time is specified. I can see why that's a reasonable default for Fastly, but we've been pretty careful about specifying Cache-Control headers when relevant, so the extra caching is mostly incorrect.
2021-11-23 13:00:56 -08:00
d4b115e805 Merge branch 'ansible' of github.com:matchu/impress-2020 into ansible 2021-11-23 12:42:59 -08:00
96d6d42120 Use persisted queries in dev, bc Next.js support
We had previously configured the client to not bother to try a GET request for GraphQL queries, and just jump straight to POST instead, because the `vercel dev` server for create-react-app reloaded the backend code for every request anyway, which doubled the dev response time.

The Next.js server is more efficient than this, and keeps some memory, so GET requests work similarly in dev as on prod now! (i.e. it fails the first time, but then succeeds on the second)

In this change, we remove the code to skip `createPersistedQueryLink` in development, and instead always call it. We simplify the code accordingly, too.
2021-11-23 12:35:20 -08:00
b941dce9fa Private cache headers in item search
If the user is searching for things they own or want, make sure we don't CDN cache it!

For many queries, this is taken care of in practice, because the search result includes `currentUserOwnsThis` and `currentUserWantsThis`. But I noticed in testing that, if the search result has no items, so those fields aren't actually part of the _response_, then the private header doesn't get set. So this mainly makes sure we don't accidentally cache an empty result from a user who didn't have anything they owned/wanted yet!
2021-11-16 13:09:45 -08:00
b73e2e1123 Send cache-control header for max-age=0, private
Some queries, like on `/your-outfits`, had the cache hint `max-age=0, private` set. In this case, our cache code sent no cache header, on the assumption that no header would result in no caching.

This was true on Vercel, but isn't true on our new Fastly setup! (Which makes sense, Vercel was a bit more aggressive here I think.)

This was causing an arbitrary user's data to be cached by Fastly as the result for `/your-outfits`. (We found this bug before launching the Fastly cache though, don't worry! No actual user data leaked!)

Now, as of this change, the `/your-outfits` query correctly sends a header of `Cache-Control: max-age=0, private`. This directs Fastly not to cache the result.

To fix this, we made a change to our HTTP header code, which is forked from Apollo's stuff.
2021-11-16 12:34:11 -08:00
cadf7487af Mark currentUser GQL as non-cacheable
Comments explain most of this! Vercel changed around the Cache-Control headers a bit to always essentially apply max-age:0 when scope:PRIVATE was true.

I'm noticing this isn't *fully* working yet though, because we're not getting a `Cache-Control: private` header, we're just getting no header at all. Fastly might aggressively choose to cache it anyway with etag stuff! I bet that's the fault of our caching middleware plugin thing, so I'll check on that!
2021-11-16 12:12:51 -08:00