OpenNeo/impress
OpenNeo/impress
1
0
Fork 1
Code Pull requests Activity
impress/vendor/cache
History
Emi Matchu 8dc11f9940 Create rails public_data:commit task, to share public data dumps 
I'm starting to port over the functionality that was previously just,
me running `yarn db:export:public-data` in `impress-2020` and
committing it to Git LFS every time.

My immediate motivation is that the `impress-2020` git repository is
getting weirdly large?? Idk how these 40MB files have blown up to a
solid 16GB of Git LFS data (we don't have THAT many!!!), but I guess
there's something about Git LFS's architecture and disk usage that I'm
not understanding.

So, let's move to a simpler system in which we don't bind the public
data to the codebase, but instead just regularly dump it in production
and make it available for download.

This change adds the `rails public_data:commit` task, which when run in
production will make the latest available at
`https://impress.openneo.net/public-data/latest.sql.gz`, and will also
store a running log of previous dumps, viewable at
`https://impress.openneo.net/public-data/`.

Things left to do:
1. Create a `rails public_data:pull` task, to download `latest.sql.gz`
   and import it into the local development database.
2. Set up a cron job to dump this out regularly, idk maybe weekly? That
   will grow, but not very fast (about 2GB per year), and we can add
   logic to rotate out old ones if it starts to grow too far. (If we
   wanted to get really intricate, we could do like, daily for the past
   week, then weekly for the past 3 months, then monthly for the past
   year, idk. There must be tools that do this!)
2024-02-29 14:30:33 -08:00
..
actioncable-7.1.3.2.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
actionmailbox-7.1.3.2.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
actionmailer-7.1.3.2.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
actionpack-7.1.3.2.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
actiontext-7.1.3.2.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
actionview-7.1.3.2.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
activejob-7.1.3.2.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
activemodel-7.1.3.2.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
activerecord-7.1.3.2.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
activestorage-7.1.3.2.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
activesupport-7.1.3.2.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
addressable-2.8.6.gem Run bundle update 2024-02-22 11:48:32 -08:00
async-2.8.1.gem Run bundle update 2024-02-22 11:48:32 -08:00
async-container-0.16.13.gem Run bundle update 2024-02-22 11:48:32 -08:00
async-http-0.61.0.gem Move manifest backfill to swf_assets:manifests task 2023-11-10 16:52:50 -08:00
async-http-cache-0.4.3.gem Replace falcon server with puma 2024-01-23 21:55:26 -08:00
async-io-1.41.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
async-pool-0.4.0.gem Move manifest backfill to swf_assets:manifests task 2023-11-10 16:52:50 -08:00
babel-source-5.8.35.gem
babel-transpiler-0.7.0.gem
base64-0.2.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
bcrypt-3.1.20.gem Run bundle update 2024-02-22 11:48:32 -08:00
bigdecimal-3.1.6.gem Run bundle update 2024-02-22 11:48:32 -08:00
bindex-0.8.1.gem
bootsnap-1.18.3.gem Run bundle update 2024-02-22 11:48:32 -08:00
build-environment-1.13.0.gem Replace falcon server with puma 2024-01-23 21:55:26 -08:00
builder-3.2.4.gem
concurrent-ruby-1.2.3.gem Run bundle update 2024-02-22 11:48:32 -08:00
connection_pool-2.4.1.gem Run bundle update 2024-02-22 11:48:32 -08:00
console-1.23.4.gem Run bundle update 2024-02-22 11:48:32 -08:00
crass-1.0.6.gem
date-3.3.4.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
devise-4.9.3.gem Upgrade to Rails 7.1.1 2023-10-25 15:05:31 -07:00
devise-encryptable-0.2.0.gem
dotenv-2.8.1.gem
dotenv-rails-2.8.1.gem Remove dotenv-deployment, upgrade dotenv-rails 2023-10-23 19:05:05 -07:00
drb-2.2.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
e2mmap-0.1.0.gem Create rails public_data:commit task, to share public data dumps 2024-02-29 14:30:33 -08:00
erubi-1.12.0.gem
execjs-2.9.1.gem Run bundle update 2024-02-22 11:48:32 -08:00
falcon-0.43.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
ffi-1.16.3.gem Run bundle update 2024-02-22 11:48:32 -08:00
fiber-annotation-0.2.0.gem Move manifest backfill to swf_assets:manifests task 2023-11-10 16:52:50 -08:00
fiber-local-1.0.0.gem Move manifest backfill to swf_assets:manifests task 2023-11-10 16:52:50 -08:00
globalid-1.2.1.gem Security updates 2023-10-23 19:08:56 -07:00
haml-6.3.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
http_accept_language-2.1.1.gem
httparty-0.21.0.gem Upgrade to Rails 7.1.1 2023-10-25 15:05:31 -07:00
i18n-1.14.1.gem
io-console-0.7.2.gem Run bundle update 2024-02-22 11:48:32 -08:00
io-event-1.4.4.gem Run bundle update 2024-02-22 11:48:32 -08:00
irb-1.11.2.gem Run bundle update 2024-02-22 11:48:32 -08:00
jsbundling-rails-1.3.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
json-2.7.1.gem Run bundle update 2024-02-22 11:48:32 -08:00
launchy-2.5.2.gem
letter_opener-1.9.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
localhost-1.2.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
loofah-2.22.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
mail-2.8.1.gem
mapping-1.1.1.gem Replace falcon server with puma 2024-01-23 21:55:26 -08:00
marcel-1.0.2.gem
memory_profiler-1.0.1.gem Add mini profiler to each page 2023-10-27 19:38:49 -07:00
mini_mime-1.1.5.gem Security updates 2023-10-23 19:08:56 -07:00
minitest-5.22.2.gem Run bundle update 2024-02-22 11:48:32 -08:00
msgpack-1.7.2.gem
multi_xml-0.6.0.gem Upgrade to Rails 7.1.1 2023-10-25 15:05:31 -07:00
mutex_m-0.2.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
mysql2-0.5.6.gem Run bundle update 2024-02-22 11:48:32 -08:00
net-imap-0.4.10.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
net-pop-0.1.2.gem
net-protocol-0.2.2.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
net-smtp-0.4.0.1.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
nio4r-2.7.0.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
nokogiri-1.16.2-x86_64-linux.gem Run bundle update 2024-02-22 11:48:32 -08:00
openssl-3.2.0.gem Replace falcon server with puma 2024-01-23 21:55:26 -08:00
orm_adapter-0.5.0.gem
parallel-1.24.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
process-metrics-0.2.1.gem Replace falcon server with puma 2024-01-23 21:55:26 -08:00
protocol-hpack-1.4.2.gem Move manifest backfill to swf_assets:manifests task 2023-11-10 16:52:50 -08:00
protocol-http-0.25.0.gem Move manifest backfill to swf_assets:manifests task 2023-11-10 16:52:50 -08:00
protocol-http1-0.16.1.gem Run bundle update 2024-02-22 11:48:32 -08:00
protocol-http2-0.15.1.gem Move manifest backfill to swf_assets:manifests task 2023-11-10 16:52:50 -08:00
protocol-rack-0.4.1.gem Replace falcon server with puma 2024-01-23 21:55:26 -08:00
psych-5.1.2.gem Run bundle update 2024-02-22 11:48:32 -08:00
public_suffix-5.0.4.gem Run bundle update 2024-02-22 11:48:32 -08:00
racc-1.7.3.gem Run bundle update 2024-02-22 11:48:32 -08:00
rack-3.0.9.1.gem Run bundle update 2024-02-22 11:48:32 -08:00
rack-attack-6.7.0.gem
rack-mini-profiler-3.3.1.gem Run bundle update 2024-02-22 11:48:32 -08:00
rack-session-2.0.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
rack-test-2.1.0.gem
rackup-2.1.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
rails-7.1.3.2.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
rails-dom-testing-2.2.0.gem
rails-html-sanitizer-1.6.0.gem
rails-i18n-7.0.8.gem Upgrade to Rails 7.1.1 2023-10-25 15:05:31 -07:00
railties-7.1.3.2.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
rake-13.1.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
rdiscount-2.2.7.3.gem Run bundle update 2024-02-22 11:48:32 -08:00
rdoc-6.6.2.gem Run bundle update 2024-02-22 11:48:32 -08:00
react-rails-2.7.1.gem
record_tag_helper-1.0.1.gem
reline-0.4.2.gem Run bundle update 2024-02-22 11:48:32 -08:00
responders-3.1.1.gem Upgrade to Rails 7.1.1 2023-10-25 15:05:31 -07:00
ruby2_keywords-0.0.5.gem Upgrade to Rails 7.1.1 2023-10-25 15:05:31 -07:00
samovar-2.2.0.gem Replace falcon server with puma 2024-01-23 21:55:26 -08:00
sanitize-6.1.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
sass-rails-6.0.0.gem
sassc-2.4.0.gem
sassc-rails-2.1.2.gem
sentry-rails-5.16.1.gem Run bundle update 2024-02-22 11:48:32 -08:00
sentry-ruby-5.16.1.gem Run bundle update 2024-02-22 11:48:32 -08:00
shell-0.8.1.gem Create rails public_data:commit task, to share public data dumps 2024-02-29 14:30:33 -08:00
sprockets-4.2.1.gem Run bundle update 2024-02-22 11:48:32 -08:00
sprockets-rails-3.4.2.gem
sqlite3-1.7.2-x86_64-linux.gem Run bundle update 2024-02-22 11:48:32 -08:00
stackprof-0.2.26.gem Run bundle update 2024-02-22 11:48:32 -08:00
stringio-3.1.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
sync-0.5.0.gem Create rails public_data:commit task, to share public data dumps 2024-02-29 14:30:33 -08:00
temple-0.10.3.gem Run bundle update 2024-02-22 11:48:32 -08:00
terser-1.2.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
thor-1.3.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
thread-local-1.1.0.gem Move manifest backfill to swf_assets:manifests task 2023-11-10 16:52:50 -08:00
tilt-2.3.0.gem Run bundle update 2024-02-22 11:48:32 -08:00
timeout-0.4.1.gem Upgrade to Rails 7.1.3.2 2024-02-22 11:25:52 -08:00
timers-4.3.5.gem Move manifest backfill to swf_assets:manifests task 2023-11-10 16:52:50 -08:00
traces-0.11.1.gem Move manifest backfill to swf_assets:manifests task 2023-11-10 16:52:50 -08:00
tzinfo-2.0.6.gem
warden-1.2.9.gem Run bundle update 2024-02-22 11:48:32 -08:00
web-console-4.2.1.gem Security updates 2023-10-23 19:08:56 -07:00
webrick-1.8.1.gem Upgrade to Rails 7.1.1 2023-10-25 15:05:31 -07:00
websocket-driver-0.7.6.gem
websocket-extensions-0.1.5.gem
will_paginate-4.0.0.gem
zeitwerk-2.6.13.gem Run bundle update 2024-02-22 11:48:32 -08:00