dice/impress
1
0
Fork 0
forked from OpenNeo/impress
Code Pull requests Activity

Add TIMEOUT parameter to swf_assets:manifests task

Browse source 
At this point, I've gone through all the assets, and the only ones
without manifests are:

1. The ones that truly have no manifest yet (that we know of)
2. The ones where execution happened to time out

I think the 5-second timeout is a very reasonable default for starting
the backfill, in a way that prioritizes moving forward; but now that we
have most things, I'd rather be able to re-run it with a more generous
timeout. So here we are!
This commit is contained in:
Emi Matchu 2023-11-11 11:04:53 -08:00
parent 3b59823fc4
commit 0845881aba
1 changed files with 3 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
lib/tasks/swf_assets.rake
View file

@ -4,6 +4,8 @@ require 'async/http/internet/instance'
namespace :swf_assets do namespace :swf_assets do
desc "Backfill manifest_url for SwfAsset models" desc "Backfill manifest_url for SwfAsset models"
task manifests: [:environment] do task manifests: [:environment] do
timeout = ENV.fetch("TIMEOUT", "5").to_i
assets = SwfAsset.where(manifest_url: nil) assets = SwfAsset.where(manifest_url: nil)
count = assets.count count = assets.count
puts "Found #{count} assets without manifests" puts "Found #{count} assets without manifests"
@ -32,7 +34,7 @@ namespace :swf_assets do
semaphore.async do |task| semaphore.async do |task|
manifest_url = nil manifest_url = nil
begin begin
task.with_timeout(5) do task.with_timeout(timeout) do
manifest_url = infer_manifest_url(asset.url, internet) manifest_url = infer_manifest_url(asset.url, internet)
end end
rescue StandardError => error rescue StandardError => error