OpenNeo/impress-2020
OpenNeo/impress-2020
1
0
Fork 0
Code Pull requests Activity
impress-2020/package.json

138 lines
6.5 KiB
JSON
Raw Normal View History

Initialize app with create-react-app
2020-04-21 17:49:52 -07:00
{
"name": "impress-2020-app",
"version": "0.1.0",
"private": true,
"dependencies": {
Upgrade @apollo/client to 3.6.9 Mainly because I want the `reset` function from `useMutation` for the login form work 😅 but also it's good to keep pace!
2022-08-16 23:27:30 -07:00
"@apollo/client": "^3.6.9",
got apollo server running as a vercel function!
2020-04-22 14:02:23 -07:00
"@apollographql/graphql-playground-html": "^1.6.24",
add auth0 for login Gonna need to import our folx into it, but gosh it would be nice to not be responsible for this anymore 😬
2020-09-02 03:06:09 -07:00
"@auth0/auth0-react": "^1.0.0",
/api/readFromArchive to serve a backed up image Okay so the funny thing is that my upload script is clearly like *super* not working lol, it's been running more than an hour now and still hasn't finished listening the files. So there's only actually a handful of files to test with here, from the `archive:create:upload-test` script! But anyway, uhh once the archive is actually uploaded, this is a way to read it back! Mainly as a way to assure me that it's all saved correctly, but also as a potential backup for images.neopets.com if it goes down again sometime.
2022-09-24 12:44:13 -07:00
"@aws-sdk/client-s3": "^3.178.0",
Update to latest Chakra UI I found a bug, and just wanted to make sure the latest version didn't have it! So hey, update time!
2021-02-02 16:00:49 -08:00
"@chakra-ui/icons": "^1.0.4",
Upgrade Chakra to 1.6.0 Need this for an upcoming bug fix!
2021-04-30 12:48:03 -07:00
"@chakra-ui/react": "^1.6.0",
Update to latest Chakra UI I found a bug, and just wanted to make sure the latest version didn't have it! So hey, update time!
2021-02-02 16:00:49 -08:00
"@chakra-ui/theme-tools": "^1.0.3",
upgrade Chakra packages
2020-12-25 09:07:59 -08:00
"@emotion/react": "^11.1.4",
"@emotion/styled": "^11.0.0",
code-split OutfitControls
2020-05-18 00:11:18 -07:00
"@loadable/component": "^5.12.0",
feedback form now sends actual emails!
2020-10-09 08:55:03 -07:00
"@sendgrid/mail": "^7.2.6",
Add Sentry error logging + privacy statement My main reason for adding this now is that I'm getting some scattered reports of things not displaying correctly, and I want to start gathering some browser data on that... I recently confirmed that animations work on iOS (at least one did!), which was going to be my guess of what was breaking...
2021-01-16 10:59:54 -08:00
"@sentry/react": "^5.30.0",
"@sentry/tracing": "^5.30.0",
Initialize app with create-react-app
2020-04-21 17:49:52 -07:00
"@testing-library/jest-dom": "^4.2.4",
"@testing-library/react": "^9.3.2",
"@testing-library/user-event": "^7.1.2",
Finally finish TS setup, build works! `yarn build` was crashing on my `build-cached-data` script, because we were trying to run the Typescript file uncompiled! Now, we run it with `ts-node`, which transparently compiles Typescript files before execution. Phew! `react-scripts build` made some automated changes to `tsconfig.json` for compatibility with `create-react-app`, and I also added a `ts-node` section to override one of them so we can compile to CommonJS for `ts-node` script execution!
2021-02-02 21:38:57 -08:00
"@types/node": "^14.14.22",
"@types/react": "^17.0.1",
"@types/react-dom": "^17.0.0",
http caching for all color/species requests
2020-05-14 15:51:08 -07:00
"apollo-link-persisted-queries": "^0.2.2",
Bump apollo-server from 2.19.2 to 2.25.3 Bumps [apollo-server](https://github.com/apollographql/apollo-server/tree/HEAD/packages/apollo-server) from 2.19.2 to 2.25.3. - [Release notes](https://github.com/apollographql/apollo-server/releases) - [Changelog](https://github.com/apollographql/apollo-server/blob/main/CHANGELOG.md) - [Commits](https://github.com/apollographql/apollo-server/commits/apollo-server@2.25.3/packages/apollo-server) --- updated-dependencies: - dependency-name: apollo-server dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 14:45:08 -08:00
"apollo-server": "^2.25.3",
Bump apollo-server-core from 2.25.3 to 3.12.1 Bumps [apollo-server-core](https://github.com/apollographql/apollo-server/tree/HEAD/packages/apollo-server-core) from 2.25.3 to 3.12.1. - [Release notes](https://github.com/apollographql/apollo-server/releases) - [Commits](https://github.com/apollographql/apollo-server/commits/apollo-server-core@3.12.1/packages/apollo-server-core) --- updated-dependencies: - dependency-name: apollo-server-core dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
2023-08-30 15:13:33 -07:00
"apollo-server-core": "^3.12.1",
Add stale-while-revalidate cache headers Oh yay, I'm pleased with this! I hope it works out well! stale-while-revalidate is an HTTP caching feature that gives us the ability to still serve relatively static content like item pages ASAP, while also making sure users generally see updates quickly. The trick is that we declare a period of time where, you can still serve the data from the cache, but you should _then_ go re-fetch the latest data in the background for next time. This works on end users and on the CDN! I've scanned the basic wardrobe and homepage stuff and brought them up-to-date, and gave particular attention to the item page, which I hope can be very very snappy now! :3 Note to self: Vercel says we can manually clear out a stale-while-revalidate resource by requesting it with `Pragma: no-cache`. I'm not sure it will listen to us for _fresh_ resources, though, so I'm not sure we can actually use that to flush things out in the way I had been hoping until writing this sentence lol :p
2021-02-02 19:07:38 -08:00
"apollo-server-env": "^3.0.0",
Bump aws-sdk from 2.726.0 to 2.814.0 Bumps [aws-sdk](https://github.com/aws/aws-sdk-js) from 2.726.0 to 2.814.0. - [Release notes](https://github.com/aws/aws-sdk-js/releases) - [Changelog](https://github.com/aws/aws-sdk-js/blob/master/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-js/compare/v2.726.0...v2.814.0) --- updated-dependencies: - dependency-name: aws-sdk dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
2021-11-26 14:47:07 -08:00
"aws-sdk": "^2.814.0",
outfit-images: use node-canvas to render image
2021-01-03 20:16:48 -08:00
"canvas": "^2.6.1",
Use chrome-aws-lambda for assetImage Okay cool, this one worked! We use this special Chrome package with AWS Lambda support, and then we use normal Playright in dev, and then we exclude `playwright` from the deployment (even though it got auto-detected by `require("playwright")`) to just barely sneak in under the 50MB limit for this function. Phew! The preview deploys for this seem to be, actually working? So that's exciting!
2021-08-19 16:12:12 -07:00
"chrome-aws-lambda": "^10.1.0",
move to vercel now function api structure I haven't really tested the API yet, because it's hard to query GraphQL directly, but I'll set up the client in a bit for it!
2020-04-22 13:03:32 -07:00
"dataloader": "^2.0.0",
add descriptions to closet lists (formatted! :3)
2020-10-28 00:00:14 -07:00
"dompurify": "^2.2.0",
Bundle CreateJS, instead of loading async So I finally started looking into the race condition that makes item previews sometimes fail to load, and as expected, it was that we were trying to load the movie before CreateJS had necessarily loaded. Usually the timing worked out, esp after a reload, but not under certain circumstances! Anyway, I've been wanting for a while to just bundle them instead. That'll help us more eagerly load them when we need them, and not depend on external CDNs, and remove a bunch of loading state! So yeah, I had to learn how the `easeljs` and `tweenjs` NPM packages did their bundling, and how to use `imports-loader` to let them just register straight onto `window`! But we got there and it's pretty nice tbh!
2021-06-16 18:00:25 -07:00
"easeljs": "^1.0.2",
Add sharing meta tags to outfit pages Meta tags are a bit tricky in apps built with `create-react-app`! While some bots like Google are able to render the full page when crawling, not all bots are. Most will just see the empty-ish index.html that would normally load up the application. But we want outfit sharing to work! And be cool! And use our new outfit thumbnails! In this change, we add a new server-side rendering API route to handle `/outfits/:id`. It's very weak server-side rendering: it just loads index.html, and makes a few small tweaks inside the `<head>` tag. But it should be enough for sharing to work in clients that support the basics of Open Graph, which I think most major providers respect! (I know Twitter has their own tags, but it also respects the basics of OG, so let's see whether there's anything we end up _wanting_ to tweak or not!)
2021-05-14 19:51:48 -07:00
"escape-html": "^1.0.3",
Upgrade Chakra to 1.6.0 Need this for an upcoming bug fix!
2021-04-30 12:48:03 -07:00
"framer-motion": "^4.1.11",
Use browser pooling for /api/assetImage I tried running a pressure test against assetImage on prod with the open-source tool `wrk`: ``` wrk -t12 -c20 -d20s --timeout 20s 'https://impress-2020-box.openneo.net/api/assetImage?libraryUrl=https%3A%2F%2Fimages.neopets.com%2Fcp%2Fitems%2Fdata%2F000%2F000%2F522%2F522756_2bde0443ae%2F522756.js&size=600' ``` I found that, unsurprisingly, we run a lot of concurrent requests, which fill up memory with a lot of Chromium instances! In this change, we declare a small pool of 2 browser contexts, to allow a bit of concurrency but still very strictly limit how many browser instances can actually get created. We might tune this number depending on the actual performance characteristics!
2021-11-12 23:35:05 -08:00
"generic-pool": "^3.8.2",
Add stale-while-revalidate cache headers Oh yay, I'm pleased with this! I hope it works out well! stale-while-revalidate is an HTTP caching feature that gives us the ability to still serve relatively static content like item pages ASAP, while also making sure users generally see updates quickly. The trick is that we declare a period of time where, you can still serve the data from the cache, but you should _then_ go re-fetch the latest data in the background for next time. This works on end users and on the CDN! I've scanned the basic wardrobe and homepage stuff and brought them up-to-date, and gave particular attention to the item page, which I hope can be very very snappy now! :3 Note to self: Vercel says we can manually clear out a stale-while-revalidate resource by requesting it with `Pragma: no-cache`. I'm not sure it will listen to us for _fresh_ resources, though, so I'm not sure we can actually use that to flush things out in the way I had been hoping until writing this sentence lol :p
2021-02-02 19:07:38 -08:00
"graphql": "^15.5.0",
Try updating honeycomb-beeline? In production we're suddenly getting errors in module wrapping in honeycomb-beeline. I wonder if it's like, an incompatibility with Vercel's version of Node? Well, this new version seems to still be playing nice on dev, so hopefully that's all it is and this fixes it! I give it like a 35% chance lol :p
2021-08-07 23:13:52 -07:00
"honeycomb-beeline": "^2.7.4",
Bump immer from 8.0.1 to 9.0.6 Bumps [immer](https://github.com/immerjs/immer) from 8.0.1 to 9.0.6. - [Release notes](https://github.com/immerjs/immer/releases) - [Commits](https://github.com/immerjs/immer/compare/v8.0.1...v9.0.6) --- updated-dependencies: - dependency-name: immer dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
2021-09-02 18:34:24 -07:00
"immer": "^9.0.6",
Finish support tool to upload manual PNGs Great for fixing those ones that the auto converters can't do!
2020-08-04 18:39:31 -07:00
"jimp": "^0.14.0",
set up auth on the server + test utils
2020-09-02 23:00:16 -07:00
"jsonwebtoken": "^8.5.1",
"jwks-rsa": "^1.9.0",
LRU caches to speed up outfit layer preload The layer preloader already takes advantage of, and primes, the HTTP cache. But we still do duplicate work, on every OutfitPreview render, to re-execute movie clip libraries, and create a movie clip to test for animations. The former is nontrivial cost, and the latter is often large cost. This can make even basic outfit changes slow, when there's no change to the movie clip layers and the player is paused! Here, we add an LRU cache for movie clip libraries, and for the question of "is it animated?". This should speed up a number of places where we would reload the movie (including between toggling the item), and various changes that were triggering full movie clip rebuilds unnecessarily. We _aren't_ solving here for the fact that toggling an animated item requires rebuilding the movie clip, which could conceivably be cached—but with some state management trickiness, because ideally it should be a separate clip for each context where it's being shown. Imo not yet worth the effort! (esp because I think users understand that toggling an animated item can be slow, whereas this was affecting _other_ actions way too much)
2021-04-16 20:16:56 -07:00
"lru-cache": "^6.0.0",
move to vercel now function api structure I haven't really tested the API yet, because it's hard to query GraphQL directly, but I'll set up the client in a bit for it!
2020-04-22 13:03:32 -07:00
"mysql2": "^2.1.0",
[WIP] Run cra-to-next codemod to be on Nextjs I'm interested in ejecting from Vercel, so I'm trying to get off their proprietary-ish create-react-app + Vercel API thing, and onto Nextjs, which is very similar in shape, but more portable. I had to disable `craCompat` in `next.config.js` to stop us from crashing on their webpack config, see https://github.com/vercel/next.js/discussions/25858#discussioncomment-1573822 The frontend seems to work at a basic level, but network requests fail, and images don't seem to be working. I'll work on those next! Note that this commit was forced through despite failing lint checks. We'll need to fix that up too! Also, after the codemod, I moved `src/pages` to the more canonical location `pages`. Lint tooling seemed surprised to not find a `pages` directory, and I didn't see a config that was making it work correctly in the other location, so I figured it's that Next is willing to check `pages` or `src/pages`? But this is more canonical so yeah!
2021-11-01 21:48:09 -07:00
"next": "12.0.2",
move neopets.com loading into server the client stuff worked locally, but not in prod because you can't request http from https, even with cors set up correctly 😅
2020-04-25 06:50:34 -07:00
"node-fetch": "^2.6.0",
Use Puppeteer instead of Playwright We used Playwright in the first place to try to work around a Vercel deploy issue, and I'm not sure it really ended up mattering lol :p But yeah, I'm putting the new Puppeteer code through the same prod stress test, and it just doesn't seem to be getting into the same broken state that Playwright was. I'm guessing it's just that Puppeteer has more investment in edge-case handling? (There's also the fact that we're no longer running things as root, which could have been a fucky problem, too?)
2021-11-13 02:16:58 -08:00
"puppeteer": "^11.0.0",
upgrade to React 17 Another longshot attempt to fix the prod-only emotion bug for free :p This should be a very easy upgrade, bc I don't think we depend on the minor stuff that had breaking changes.
2020-12-28 14:07:03 -08:00
"react": "^17.0.1",
WIP: search suggestions
2020-09-01 17:59:04 -07:00
"react-autosuggest": "^10.0.2",
upgrade to React 17 Another longshot attempt to fix the prod-only emotion bug for free :p This should be a very easy upgrade, bc I don't think we depend on the minor stuff that had breaking changes.
2020-12-28 14:07:03 -08:00
"react-dom": "^17.0.1",
Add disabled save UI
2021-03-29 19:46:21 -07:00
"react-icons": "^4.2.0",
First draft of UserItemListPage A lot is missing! No descriptions, no support for the "Not in a list" case, no scroll performance windowing, no editing! But it's a start :3
2021-06-12 04:45:23 -07:00
"react-router-hash-link": "^2.4.3",
use xmlrpc to load pets, not json.php The AMFPHP gateway's json.php endpoint has always had a problem parsing pets whose names start with digits… I've dug into it before, and checked again today, and there really is just no way around it: https://github.com/silexlabs/amfphp-1.9/blob/d584b58e9581ecbd72def15d1fd31eb2408d050f/core/json/app/Actions.php#L43 And there aren't any reliable AMFPHP Node libraries out there to make the actual native AMF call. Buuuut! In today's investigation, I noticed the xmlrpc.php endpoint for the first time. And, wouldn't you know it, there's //great// reliability for something as enterprise-standard as that! So here, I've switched over to using an xmlrpc client library, which simplifies our calling code //and// makes number pets work correctly 😁 I wouldn't have done it just for the simplification, I think bringing in a library is net more complexity… but getting this finally right is a big relief.
2020-10-22 23:22:04 -07:00
"react-transition-group": "^4.3.0",
Virtualize item list scrolling This helps the render time by a lot!
2021-06-19 12:36:19 -07:00
"react-virtualized": "^9.22.3",
Install Sharp for production Oh neat, when trying `yarn build && yarn start` locally, I got a message about installing Sharp for better image optimization performance in production. It mentions that this isn't relevant for Vercel, where it's auto-added. But it's good to get on it now anyway!
2021-11-02 01:00:52 -07:00
"sharp": "^0.29.2",
add descriptions to closet lists (formatted! :3)
2020-10-28 00:00:14 -07:00
"simple-markdown": "^0.7.2",
Bundle CreateJS, instead of loading async So I finally started looking into the race condition that makes item previews sometimes fail to load, and as expected, it was that we were trying to load the movie before CreateJS had necessarily loaded. Usually the timing worked out, esp after a reload, but not under certain circumstances! Anyway, I've been wanting for a while to just bundle them instead. That'll help us more eagerly load them when we need them, and not depend on external CDNs, and remove a bunch of loading state! So yeah, I had to learn how the `easeljs` and `tweenjs` NPM packages did their bundling, and how to use `imports-loader` to let them just register straight onto `window`! But we got there and it's pretty nice tbh!
2021-06-16 18:00:25 -07:00
"tweenjs": "^1.0.2",
Fix pet loading for most pets Sigh, I guess xmlrpc was deleted? Back to the method that doesn't work for pets with leading digits in their names, sobbe Dice has a neat idea for how to work around that, but I'm not sure how to fit it in our architecture, let's take a look!
2023-08-29 14:40:59 -07:00
"typescript": "^4.4.4"
Initialize app with create-react-app
2020-04-21 17:49:52 -07:00
},
"scripts": {
Add a pre-commit lint hook I've got it in my IDE too, but I want more safeguards lol, I'm tired of goofing up the pushes :p I don't allow warnings here, but still have them as warnings not errors; because I don't want them to block build, but I _do_ want them to block commit.
2021-05-04 18:46:02 -07:00
"prepare": "husky install",
Fix pm2 monitoring Okay huh, while digging a bit into another issue, I found what was wrong with our config and pm2's built-in monitoring! You can't use `yarn start`, because the wrapper script breaks its ability to look inside and see what's happening. I also removed the compiler flag thing from the `start` script in `package.json`, because I think it's redundant? There's no compilation to be done in a live server. I think I might remove monit after this? It's nice extra resilience in a sense, but it feels like extra complexity when it's doing the job `pm2` is supposed to do. (And tbh I've almost never heard of nginx crashing, and if it does it's probably a scenario worth investigating by hand.)
2021-11-03 16:46:35 -07:00
"start": "next start",
Add installation guide to README I tried it in my lil Ubuntu WSL box and hey, it worked great! Neat!! Pretty neat to have just sat down and fixed up the dev environment for other people?? I'll see about what it would take to actually invite people in…
2022-11-13 09:09:36 -08:00
"dev": "DB_ENV=development yarn build-cached-data && DB_ENV=development REACT_APP_IMPRESS_LOG_IN_AS=$IMPRESS_LOG_IN_AS TS_NODE_COMPILER=typescript-cached-transpile next dev",
"local-prod": "DB_ENV=production yarn build-cached-data && DB_ENV=production REACT_APP_IMPRESS_LOG_IN_AS=$IMPRESS_LOG_IN_AS TS_NODE_COMPILER=typescript-cached-transpile next dev",
[WIP] Run cra-to-next codemod to be on Nextjs I'm interested in ejecting from Vercel, so I'm trying to get off their proprietary-ish create-react-app + Vercel API thing, and onto Nextjs, which is very similar in shape, but more portable. I had to disable `craCompat` in `next.config.js` to stop us from crashing on their webpack config, see https://github.com/vercel/next.js/discussions/25858#discussioncomment-1573822 The frontend seems to work at a basic level, but network requests fail, and images don't seem to be working. I'll work on those next! Note that this commit was forced through despite failing lint checks. We'll need to fix that up too! Also, after the codemod, I moved `src/pages` to the more canonical location `pages`. Lint tooling seemed surprised to not find a `pages` directory, and I didn't see a config that was making it work correctly in the other location, so I figured it's that Next is willing to check `pages` or `src/pages`? But this is more canonical so yeah!
2021-11-01 21:48:09 -07:00
"build": "yarn build-cached-data && next build",
outfit-images: fix prod-only bugs We needed an extra build step to get node-canvas working! And `setHeader` is fluent in dev, but not in prod? Go figure!
2021-01-03 22:58:09 -08:00
"vercel-build": "yum install libuuid-devel libmount-devel && cp /lib64/{libuuid,libmount,libblkid}.so.1 node_modules/canvas/build/Release/",
[WIP] Run cra-to-next codemod to be on Nextjs I'm interested in ejecting from Vercel, so I'm trying to get off their proprietary-ish create-react-app + Vercel API thing, and onto Nextjs, which is very similar in shape, but more portable. I had to disable `craCompat` in `next.config.js` to stop us from crashing on their webpack config, see https://github.com/vercel/next.js/discussions/25858#discussioncomment-1573822 The frontend seems to work at a basic level, but network requests fail, and images don't seem to be working. I'll work on those next! Note that this commit was forced through despite failing lint checks. We'll need to fix that up too! Also, after the codemod, I moved `src/pages` to the more canonical location `pages`. Lint tooling seemed surprised to not find a `pages` directory, and I didn't see a config that was making it work correctly in the other location, so I figured it's that Next is willing to check `pages` or `src/pages`? But this is more canonical so yeah!
2021-11-01 21:48:09 -07:00
"test": "jest test --env=jsdom",
[WIP] Fix eslint for Next.js Tweaked some of the default Next.js rules, fixed lint-staged for `next lint`, made a few small easy lint fixes. Feels good! Note that using the `dirs` option in `next.config.js` was causing `lint-staged` to lint _everything_. That's why I edited `yarn lint` to specify the dirs instead: that way, that command will lint all those dirs, but they won't get included in invocations with `--file`. There are still a few lint errors left after this commit, because our <img> tags aren't working (@next/next/no-img-element). I'll fix those when we figure out what's wrong with images!
2021-11-01 22:07:38 -07:00
"lint": "next lint --dir src --dir pages",
Fix some bugs running deploy-setup from scratch As an exercise, I've wiped the box clean, and I'm reinstalling from the scripts! :3 I added the SSH hardening rules to the playbook instead of doing them by hand this time. I made a mistake with creating `/srv/impress-2020`, right, you need to *say* what it should be created *as* for the creation step to work! I also guess my recent pm2 changes made it not actually be willing to start the app anymore, because `/srv/impress-2020/current` doesn't exist or have `node_modules` yet. I'm doing a cute thing where I create a placeholder app during setup, so there's always something to run, without introducing the complexities of a real deploy to the setup process. And right, of course, we need to install nginx before running certbot! But we need to add certbot config *after* running certbot! And then just some misc cleanups for consistency and correctness!
2021-11-03 23:04:25 -07:00
"deploy-setup": "echo $'Setup requires you to become the root user. You\\'ll need to enter the password for your account on the remote web server below, and you must be part of the `sudo` user group.' && ansible-playbook -K -i deploy/inventory.cfg deploy/playbooks/setup.yml",
Add deploy playbook: pulls git and installs deps
2021-11-02 16:29:52 -07:00
"deploy": "ansible-playbook -i deploy/inventory.cfg deploy/playbooks/deploy.yml",
Refactor deploy to build locally, not remotely
2021-11-02 18:47:13 -07:00
"deploy-skip-build": "ansible-playbook -i deploy/inventory.cfg deploy/playbooks/deploy.yml --extra-vars='{\"skip_build\": true}'",
add a convenient `yarn mysql-admin` just for me I use this as an easy shortcut to log into the prod database, with credentials that have more permissions than the app currently does 😅
2020-08-11 22:07:56 -07:00
"mysql": "mysql --host=impress.openneo.net --user=$(dotenv -p IMPRESS_MYSQL_USER) --password=$(dotenv -p IMPRESS_MYSQL_PASSWORD) --database=openneo_impress",
Add openneo_id to the dev database Specifically, we rename the dev database to `openneo_impress` for consistency with the main app, and create a second schema file for `openneo_id`, so we can do local account creation.
2022-09-13 20:39:45 -07:00
"mysql-dev": "mysql --host=localhost --user=impress_2020_dev --password=impress_2020_dev --database=openneo_impress",
create cache-asset-manifests script Just gonna bulk load all those manifests into the db, and then that should make most loads notably faster by removing the net request! 🤞 We'll still load manifests inline sometimes, but only the first time anyone pulls up the layer in impress-2020. After that, it should be cached forever!
2020-08-17 18:23:39 -07:00
"mysql-admin": "mysql --host=impress.openneo.net --user=matchu --password --database=openneo_impress",
Update dev mysql schema I took away the column-statistics option because the mysqldump version I'm using from mariadb doesn't seem to support it. Whatever!
2022-09-13 00:00:54 -07:00
"mysqldump": "mysqldump --host=impress.openneo.net --user=$(dotenv -p IMPRESS_MYSQL_USER) --password=$(dotenv -p IMPRESS_MYSQL_PASSWORD)",
start setting up a local dev database
2020-09-18 05:24:03 -07:00
"setup-mysql": "yarn mysql-admin < scripts/setup-mysql.sql",
Add openneo_id to the dev database Specifically, we rename the dev database to `openneo_impress` for consistency with the main app, and create a second schema file for `openneo_id`, so we can do local account creation.
2022-09-13 20:39:45 -07:00
"setup-mysql-dev": "yarn mysql-dev < scripts/setup-mysql-dev-constants.sql && yarn mysql-dev < scripts/setup-mysql-dev-schema-impress.sql && yarn mysql-dev --database=openneo_id < scripts/setup-mysql-dev-schema-id.sql",
Oops, remove leftover debug code
2022-12-22 18:44:47 -08:00
"run-script": "ts-node --compiler=typescript-cached-transpile --transpile-only -r dotenv/config",
Refactor package.json scripts a tiny bit We extract a `run-script` command that contains the big `ts-node` incantation!
2022-09-24 10:33:16 -07:00
"build-cached-data": "yarn run-script scripts/build-cached-data.js",
"cache-asset-manifests": "yarn run-script scripts/cache-asset-manifests.js",
"delete-user": "yarn run-script scripts/delete-user.js",
"export-users-to-auth0": "yarn run-script scripts/export-users-to-auth0.js",
Auto-modeling script?? It seems to be working!! How exciting!! I'm just letting it run on stuff now :3 One important issue is that Classic DTI doesn't show images for items modeled this way, because we don't download the SWFs for it. But I wanna update it to stop using AWS anyway and do the same stuff 2020 does, I think we can do that pretty sneakily!
2022-10-11 11:13:10 -07:00
"model-needed-items": "yarn run-script scripts/model-needed-items.js",
Refactor package.json scripts a tiny bit We extract a `run-script` command that contains the big `ts-node` incantation!
2022-09-24 10:33:16 -07:00
"validate-owls-data": "yarn run-script scripts/validate-owls-data.js",
Use AWS CLI for archive upload instead of s3cmd Okay there we go, I was following Linode's guidance and ended up using a non-Amazon S3 client, but it turns out you can get the official Amazon AWS client to play with private services too, and it doesn't do the thing s3cmd does of trying to list every single file before doing anything 😅 This command _is_ doing weird stall-outs here and there, but is mostly just chugging along. It's not exactly fast, I imagine it'll take some time, but the fact that it's like. working. is huge lmao
2022-09-24 13:18:40 -07:00
"aws": "AWS_ACCESS_KEY_ID=$(dotenv -p ARCHIVE_STORAGE_READWRITE_ACCESS_KEY) AWS_SECRET_ACCESS_KEY=$(dotenv -p ARCHIVE_STORAGE_READWRITE_SECRET_KEY) aws --endpoint=https://$(dotenv -p ARCHIVE_STORAGE_HOST)",
Refactor archive scripts to prepare/create/upload Sat down and thought about the structure here and how to make the full/delta stuff make more sense together! Here's what I came up with! In both full and delta archiving, we prepare the manifest, we create the local archive, then we upload it to remote.
2022-10-13 16:07:12 -07:00
"archive:full": "yarn archive:prepare:full && yarn archive:create:full && yarn archive:upload:full",
"archive:delta": "yarn archive:prepare:delta && yarn archive:create:delta && yarn archive:upload:delta",
"archive:prepare:full": "yarn run-script scripts/archive/prepare/full.js",
"archive:prepare:remote": "dotenv -- ./scripts/archive/prepare/remote.sh",
"archive:prepare:delta": "dotenv -- ./scripts/archive/prepare/delta.sh",
"archive:create:full": "dotenv -- ./scripts/archive/create/full.sh",
"archive:create:delta": "dotenv -- ./scripts/archive/create/delta.sh",
"archive:upload:full": "dotenv -- ./scripts/archive/upload/full.sh",
"archive:upload:delta": "dotenv -- ./scripts/archive/upload/delta.sh",
Script to set up dev db with public DTI data Now, someone with production DB access can run `yarn db:export:public-data` to create `public-data-constants.sql` and `public-data-from-modeling.sql`. Then, someone setting up their dev database can run `yarn db:setup-dev:full` and get all the wearables data imported right into their dev database! I'm noticing just how poorly I'm keeping up with my own goals for finishing up DTI, and wondering if now is a good time to circle back to some old offers for code contributions I got last year… I also just figure that making this app Possible To Run with a backup of the basic public database is like. a pretty handy thing to have for archival's sake imo Note that, for this change, we also set up Git LFS (Large File Storage). Github should be automatically compatible with this! It's a way to not write the whole 30MB database dump into the repository history, and instead keep it in a secondary filestore, because Git's core algorithms aren't really built to handle large blobs of data very well. Users setting up their dev environment will therefore also need to have Git LFS installed for this script to work! (Otherwise, they'll see a "pointer" file in `public-data-from-modeling.sql.gz` that contains some metadata about the file state but not the data itself.)
2022-11-13 07:03:35 -08:00
"archive:upload:test": "dotenv -- ./scripts/archive/upload/test.sh",
"db:export:schema": "./scripts/db/export/schema.sh",
"db:export:public-data": "./scripts/db/export/public-data.sh",
"db:setup-dev:minimal": "./scripts/db/setup-dev/minimal.sh",
Add installation guide to README I tried it in my lil Ubuntu WSL box and hey, it worked great! Neat!! Pretty neat to have just sat down and fixed up the dev environment for other people?? I'll see about what it would take to actually invite people in…
2022-11-13 09:09:36 -08:00
"db:setup-dev:full": "./scripts/db/setup-dev/full.sh",
"hmm": "echo $FOO",
"hmmm": "yarn hmm"
Initialize app with create-react-app
2020-04-21 17:49:52 -07:00
},
"browserslist": {
"production": [
">0.2%",
"not dead",
"not op_mini all"
],
"development": [
"last 1 chrome version",
"last 1 firefox version",
"last 1 safari version"
]
Layout prototype, items panel
2020-04-21 20:32:53 -07:00
},
"devDependencies": {
Fix dev command for TS support Gosh, I'm really down this rabbit hole, huh? :p Anyway, turns out the `vercel dev` command contains a _fork_ of ts-node that doesn't respect the `ts-node` override key in `tsconfig.json`. But it does respect the env var! So I've now set up `yarn start` to invoke `vercel dev` with the new env, which in turn invokes `yarn dev` to start the create-react-app dev server. Whew! (Probably a better dev experience tbh.)
2021-02-02 22:25:54 -08:00
"5to6-codemod": "^1.8.0",
[WIP] Fix eslint for Next.js Tweaked some of the default Next.js rules, fixed lint-staged for `next lint`, made a few small easy lint fixes. Feels good! Note that using the `dirs` option in `next.config.js` was causing `lint-staged` to lint _everything_. That's why I edited `yarn lint` to specify the dirs instead: that way, that command will lint all those dirs, but they won't get included in invocations with `--file`. There are still a few lint errors left after this commit, because our <img> tags aren't working (@next/next/no-img-element). I'll fix those when we figure out what's wrong with images!
2021-11-01 22:07:38 -07:00
"@typescript-eslint/eslint-plugin": "^5.3.0",
"@typescript-eslint/parser": "^5.3.0",
move to vercel now function api structure I haven't really tested the API yet, because it's hard to query GraphQL directly, but I'll set up the client in a bit for it!
2020-04-22 13:03:32 -07:00
"apollo-server-testing": "^2.12.0",
redo auth0 export script to use APIs, not JSON
2020-09-02 15:26:33 -07:00
"auth0": "^2.28.0",
add `yarn mysql` shortcut script!
2020-04-30 00:01:49 -07:00
"dotenv-cli": "^3.1.0",
create cache-asset-manifests script Just gonna bulk load all those manifests into the db, and then that should make most loads notably faster by removing the net request! 🤞 We'll still load manifests inline sometimes, but only the first time anyone pulls up the layer in impress-2020. After that, it should be cached forever!
2020-08-17 18:23:39 -07:00
"es6-promise-pool": "^2.5.0",
[WIP] Run cra-to-next codemod to be on Nextjs I'm interested in ejecting from Vercel, so I'm trying to get off their proprietary-ish create-react-app + Vercel API thing, and onto Nextjs, which is very similar in shape, but more portable. I had to disable `craCompat` in `next.config.js` to stop us from crashing on their webpack config, see https://github.com/vercel/next.js/discussions/25858#discussioncomment-1573822 The frontend seems to work at a basic level, but network requests fail, and images don't seem to be working. I'll work on those next! Note that this commit was forced through despite failing lint checks. We'll need to fix that up too! Also, after the codemod, I moved `src/pages` to the more canonical location `pages`. Lint tooling seemed surprised to not find a `pages` directory, and I didn't see a config that was making it work correctly in the other location, so I figured it's that Next is willing to check `pages` or `src/pages`? But this is more canonical so yeah!
2021-11-01 21:48:09 -07:00
"eslint": "<8.0.0",
"eslint-config-next": "12.0.2",
Add a pre-commit lint hook I've got it in my IDE too, but I want more safeguards lol, I'm tired of goofing up the pushes :p I don't allow warnings here, but still have them as warnings not errors; because I don't want them to block build, but I _do_ want them to block commit.
2021-05-04 18:46:02 -07:00
"husky": "^6.0.0",
Fix dependencies for prod Hmm, prod builds weren't happy with how I specified the version here …do prod builds not listen to the lockfile? Suspicious :p
2021-06-16 18:28:40 -07:00
"imports-loader": "^1.2.0",
script to export users to auth0
2020-09-02 03:49:58 -07:00
"inquirer": "^7.3.3",
outfit-images: placeholder function and test!
2021-01-03 19:57:02 -08:00
"jest-image-snapshot": "^4.3.0",
Add a pre-commit lint hook I've got it in my IDE too, but I want more safeguards lol, I'm tired of goofing up the pushes :p I don't allow warnings here, but still have them as warnings not errors; because I don't want them to block build, but I _do_ want them to block commit.
2021-05-04 18:46:02 -07:00
"lint-staged": "^10.5.4",
enable Fast Refresh Following this guide: https://dutzi.party/react-fast-refresh/ This enables code changes in dev to appear faster and preserve state, instead of reloading the page!
2020-07-22 21:03:04 -07:00
"prettier": "^2.0.5",
Finally finish TS setup, build works! `yarn build` was crashing on my `build-cached-data` script, because we were trying to run the Typescript file uncompiled! Now, we run it with `ts-node`, which transparently compiles Typescript files before execution. Phew! `react-scripts build` made some automated changes to `tsconfig.json` for compatibility with `create-react-app`, and I also added a `ts-node` section to override one of them so we can compile to CommonJS for `ts-node` script execution!
2021-02-02 21:38:57 -08:00
"react-is": "^16.13.1",
Fix dev command for TS support Gosh, I'm really down this rabbit hole, huh? :p Anyway, turns out the `vercel dev` command contains a _fork_ of ts-node that doesn't respect the `ts-node` override key in `tsconfig.json`. But it does respect the env var! So I've now set up `yarn start` to invoke `vercel dev` with the new env, which in turn invokes `yarn dev` to start the create-react-app dev server. Whew! (Probably a better dev experience tbh.)
2021-02-02 22:25:54 -08:00
"ts-node": "^9.1.1",
Reinstall yargs Uhh hmm, I don't remember when we removed it from package.json, I guess maybe I thought it was unused and didn't look carefully enough? Anyway, this fixes the export-users-to-auth0 script, which was crashing because yargs wasn't installed, oops!
2022-01-07 18:06:46 -08:00
"typescript-cached-transpile": "^0.0.6",
"yargs": "^17.3.1"
Initialize app with create-react-app
2020-04-21 17:49:52 -07:00
}
}
Copy permalink