1
0
Fork 0
forked from OpenNeo/impress
Dress to Impress, a big fancy Neopets customization tool!
Find a file
Emi Matchu 8dc11f9940 Create rails public_data:commit task, to share public data dumps
I'm starting to port over the functionality that was previously just,
me running `yarn db:export:public-data` in `impress-2020` and
committing it to Git LFS every time.

My immediate motivation is that the `impress-2020` git repository is
getting weirdly large?? Idk how these 40MB files have blown up to a
solid 16GB of Git LFS data (we don't have THAT many!!!), but I guess
there's something about Git LFS's architecture and disk usage that I'm
not understanding.

So, let's move to a simpler system in which we don't bind the public
data to the codebase, but instead just regularly dump it in production
and make it available for download.

This change adds the `rails public_data:commit` task, which when run in
production will make the latest available at
`https://impress.openneo.net/public-data/latest.sql.gz`, and will also
store a running log of previous dumps, viewable at
`https://impress.openneo.net/public-data/`.

Things left to do:
1. Create a `rails public_data:pull` task, to download `latest.sql.gz`
   and import it into the local development database.
2. Set up a cron job to dump this out regularly, idk maybe weekly? That
   will grow, but not very fast (about 2GB per year), and we can add
   logic to rotate out old ones if it starts to grow too far. (If we
   wanted to get really intricate, we could do like, daily for the past
   week, then weekly for the past 3 months, then monthly for the past
   year, idk. There must be tools that do this!)
2024-02-29 14:30:33 -08:00
.devcontainer Save last trade activity time onto User 2024-01-19 00:00:46 -08:00
.husky Set up eslint for wardrobe-2020 2023-11-02 18:11:07 -07:00
app Update GitHub links to point to our self-hosted OpenNeo Code 2024-02-29 11:24:21 -08:00
bin Upgrade to Rails 7.1.1 2023-10-25 15:05:31 -07:00
config Create rails public_data:commit task, to share public data dumps 2024-02-29 14:30:33 -08:00
db Improve Unicode support, emojis don't crash us anymore lol! 2024-02-28 18:54:27 -08:00
deploy Create rails public_data:commit task, to share public data dumps 2024-02-29 14:30:33 -08:00
lib Create rails public_data:commit task, to share public data dumps 2024-02-29 14:30:33 -08:00
public Add handlers for requests that were stopped during the reboot process 2024-02-28 13:50:13 -08:00
test Save last trade activity time onto User 2024-01-19 00:00:46 -08:00
vendor Create rails public_data:commit task, to share public data dumps 2024-02-29 14:30:33 -08:00
.eslintrc.json Set up eslint for wardrobe-2020 2023-11-02 18:11:07 -07:00
.gitignore Create rails public_data:commit task, to share public data dumps 2024-02-29 14:30:33 -08:00
.ruby-version Finish migrating to Ruby 3.3.0 2024-02-22 12:05:02 -08:00
.yarnrc.yml Upgrade to Yarn 4.0.2 2024-01-14 23:05:53 -08:00
config.ru Upgrade to Rails 6.1.7.4 2023-10-23 19:05:07 -07:00
falcon.rb Remove supervisor from the Falcon process? 2024-01-24 00:20:23 -08:00
Gemfile Create rails public_data:commit task, to share public data dumps 2024-02-29 14:30:33 -08:00
Gemfile.lock Create rails public_data:commit task, to share public data dumps 2024-02-29 14:30:33 -08:00
LICENSE.md Update GitHub links to point to our self-hosted OpenNeo Code 2024-02-29 11:24:21 -08:00
package.json Upgrade to Yarn 4.0.2 2024-01-14 23:05:53 -08:00
Procfile.dev Fix bin/dev to use the right settings in development 2023-11-02 16:54:39 -07:00
Rakefile Uninstall resque 2023-10-23 19:05:04 -07:00
README.md Oops, needs to be a README.md file! 2023-10-25 16:31:41 -07:00
yarn.lock Upgrade to Yarn 4.0.2 2024-01-14 23:05:53 -08:00

Dress to Impress beach logo

Dress to Impress

Oh! We've been revitalizing the Rails app! Fun!

There'll be more to say about it here soon :3