Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

192-Frame-Limit Bug Fix + Fixed Seed Not Randomizing + Adjustable Video Birate with FFMPEG + Auto-Detecting/Aborting/Retrying Bad Renders #83

Open
wants to merge 38 commits into
base: main
Choose a base branch
from

Conversation

pftq
Copy link

@pftq pftq commented Feb 25, 2025

This pull request fixes a few crucial things + quality of life improvements. The 192-frame limit bug is fixed with Riflex extrapolation that only kicks in when frames > 192. The seed also never randomizes (defaults to 42 and randomizer checks for seed<0). Lastly the video output is poor quality, so I reimplemented it with FFMPEG. The FFMPEG will be used only when available, otherwise defaulting to the OpenCV video export you had before (but actually still higher bitrate at ~7mbps instead of 2.5mbps because of setting the cap higher there as well).
See:
#72
#79

@pftq pftq changed the title Adjustable Video Birate with FFMPEG and Fixed Seed Not Randomizing 192-Frame-Limit Bug Fix + Fixed Seed Not Randomizing + Adjustable Video Birate with FFMPEG Mar 9, 2025
@pftq
Copy link
Author

pftq commented Mar 9, 2025

I've added a fix to the 192-frame-limit bug by applying the Riflex extrapolation technique by thu-ml (credit Kijai for using it in ComfyUI and making me aware of it). This is a pretty solid workaround until there's a true fix for why the video turns to static noise on frame 193 and resets.
https://github.com/SkyworkAI/SkyReels-V1/pull/83/files#diff-23418e8cc57144ed095f778f599e57792d2c651852c1fe66419afaa2cf2cf878

I posted the bug earlier here: #63

@pftq
Copy link
Author

pftq commented Mar 21, 2025

I added another useful (probably the most useful) parameter "--detect_bad_renders" for automatically detecting, aborting, and retrying a videos that become random still images or scene changes (or is likely to become so based on latent analysis early in the sampling process). This saves you time by aborting early if it is detecting a bad video and also retries with different seed automatically.
See: #99

There are 4 parameters:

  • "--detect_bad_renders" enables this functionality. It does a check during the sampling process and another after the video is fully decoded. The processing time is minimal, only scanning the first 2 seconds of the video once during sampling and once again after the video is done. It looks for major frame-to-frame changes, consecutive 24-frames of still image, and major deviation from the input image within the first 2 seconds as triggers for a "bad render." If it detects early and aborts, this saves you 75% of the render time on a video you won't want anyway.
  • "--bad_render_retries 5" will let it retry up to 5 times (or whatever number you pick) using the same settings but different seed. This lets you avoid requesting a batch of 3 videos and getting zero if all 3 are bad renders. Can be used in conjunction with "--variety_batch" to move onto the next video with differing CFG/steps.
  • "--bad_render_threshold 0.02" is the default threshold for checking a video early against the input image and is conservative. 0.03 often leads to bad renders as well, so increase it if you want. 0.04 and above generally do not become stills or scene changes.
  • "--save_bad_renders" can be added if you want to save the bad render in case for manual inspection. If the video is already in the decoding phase, it'll still be watchable, but it'll be mostly noise if the render was aborted early in sampling.

Example prompt now with this and other features in this fork:
Sample prompt, single GPU, variety batch of 10-sec videos with Riflex (automatic at frames>193)

python video_generate.py \
--gpu_num 1 \
--quant \
--offload \
--high_cpu_memory \
--parameters_level \
--sequence_batch \
--model_id "Skywork/SkyReels-V1-Hunyuan-I2V" \
--task_type i2v \
--guidance_scale 8 \
--embedded_guidance_scale 1 \
--width 720 \
--height 720 \
--num_frames 241 \
--num_inference_steps 100 \
--seed -1 \
--image "image.jpg" \
--prompt "FPS-24, ..." \
--negative_prompt "chaotic, distortion, morphing, shaky camera, panning, zoom, glare, lens flare, camera movement, blur, out of focus, low quality, low resolution, static image, overexposed, deformation, bad hands, bad teeth, bad eyes, bad limbs" \
--color_match \
--variety_batch \
--detect_bad_renders \
--bad_render_retries 5 \
--bad_render_threshold 0.02 \
--mbps 15 \
--video_num 10

For multi-GPU, variety batch of 10-sec videos with Riflex (automatic at frames>193)

python video_generate.py \
--gpu_num 8 \
--model_id "Skywork/SkyReels-V1-Hunyuan-I2V" \
--task_type i2v \
--guidance_scale 8 \
--embedded_guidance_scale 1 \
--width 960 \
--height 960 \
--num_frames 241 \
--num_inference_steps 100 \
--image "image.jpg" \
--prompt "FPS-24, ..." \
--negative_prompt "chaotic, distortion, morphing, shaky camera, panning, zoom, camera movement, blur, out of focus, low quality, low resolution, static image, overexposed, deformation, bad hands, bad teeth, bad eyes, bad limbs" \
--color_match \
--variety_batch \
--detect_bad_renders \
--bad_render_retries 5 \
--bad_render_threshold 0.02 \
--mbps 15 \
--video_num 10

@pftq pftq changed the title 192-Frame-Limit Bug Fix + Fixed Seed Not Randomizing + Adjustable Video Birate with FFMPEG 192-Frame-Limit Bug Fix + Fixed Seed Not Randomizing + Adjustable Video Birate with FFMPEG + Auto-Detecting/Aborting/Retrying Bad Renders Mar 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant