Made with the help of Lilien 
With this pack of nodes, you can analyze audio, extract drums, bass, vocal tracks, and use the scheduled masks and weights to create AI-generated audio-reactive animations in ComfyUI
Workflow File (click on the link, then on "download raw file" button at the right and drop the file into ComfyUI) 📜
AudioReactive_ImagesToVideo_Yvann.json
ImgToVid.mov
Workflow File (click on the link, then on "download raw file" button at the right and drop the file into ComfyUI) 📜
AudioReactive_VideoToVideo_Yvann.json
Demo_Lilien.mp4
Nodes
Analyzes audio to generate reactive weights and graph. Can extract specific elements like drums, vocals, bass. Parameters allow manual control over audio weights
Node Parameters
- audio_sep_model: Loaded model from "Load Audio Separation Model"
- audio: Input audio file
- batch_size: Number of frames to associate with audio weights
- fps: Frames per second for processing audio weights
Parameters:
- analysis_mode: Select audio component to analyze
- threshold: Minimum weight value to pass through
- multiply: Amplification factor for weights before normalization
Outputs:
- graph_audio: Graph image of audio weights over frames
- processed_audio: Separated or processed audio (e.g., drums, vocals)
- original_audio: Original unmodified audio input
- audio_weights: List of audio-reactive weights based on processed audio
Load an audio separation model. If unavailable, downloads to
ComfyUI/models/audio_separation_model/
Node Parameters
- model: Audio separation model to load
- HybridDemucs: Most accurate, fastest, and lightweight
- OpenUnmix: Alternative model
Outputs:
- audio_sep_model: Loaded audio separation model
Connect it to "Audio Analysis" or "Audio Remixer"
Detects peaks in audio weights based on a threshold and minimum distance. Identifies significant audio events to trigger visual changes or actions.
Node Parameters
- peaks_threshold: Threshold for peak detection
- min_peaks_distance: Minimum frames between consecutive peaks
Outputs:
- peaks_weights: Binary list indicating peak presence (1 for peak, 0 otherwise)
- peaks_alternate_weights: Alternating binary list based on detected peaks
- peaks_index: String of peak indices
- peaks_count: Total number of detected peaks
- graph_peaks: Visualization image of detected peaks over audio weights
Uses "peaks_weights" from "Audio Peaks Detection" to control image transitions based on audio peaks. Outputs images and weights for two IPAdapter batches, logic from "IPAdapter Weights" or IPAdapter_Plus.
Node Parameters
- images: Batch of images for transitions (loops images to match peak count)
- peaks_weights: List of audio peaks from "Audio Peaks Detection"
Parameters:
- blend_mode: Transition method applied to weights
- transitions_length: Frames used to blend between images
- min_IPA_weight: Minimum weight applied by IPAdapter per frame
- max_IPA_weight: Maximum weight applied by IPAdapter per frame
Outputs:
- image_1: Starting image for transition (connect to first IPAdapter batch "image")
- weights: Blending weights for transitions (connect to first IPAdapter batch "weight")
- image_2: Ending image for transition (connect to second IPAdapter batch "image")
- weights_invert: Inversed weights (connect to second IPAdapter batch "weight")
- graph_transitions: Visualization of weight transitions over frames
Associates "prompts" with "peaks_index" into a scheduled format. Connect output to "batch prompt schedule" of Fizz Nodes. Add an empty line between each individual prompt.
Node Parameters
- peaks_index: Frames where peaks occur from "Audio Peaks Detection"
- prompts: Multiline string of prompts for each index
Outputs:
- prompt_schedule: String mapping each audio index to a prompt
Modify input audio by adjusting the intensity of drums, bass, vocals, or other elements.
Node Parameters
- audio_sep_model: Loaded model from "Load Audio Separation Model"
- audio: Input audio file
Parameters:
- bass_volume: Adjusts bass volume
- drums_volume: Adjusts drums volume
- others_volume: Adjusts other elements' volume
- vocals_volume: Adjusts vocals volume
Outputs:
- merged_audio: Composition of separated tracks with applied modifications
Repeats images N times. Cycles inputs if N > number of provided images.
Node Parameters
- **mask**: Mask input to convertOutputs:
- float: Float value
Inverts each value in a list of floats.
Node Parameters
- floats: List of float values to invert.
Outputs:
- inverted_floats: Inverted list of float values.
Generates a graph from floats for visual data comparison.
Useful to compare audio weights.
Node Parameters
- floats: Primary list of floats to visualize
- floats_optional1: (Optional) Second list of floats
- floats_optional2: (Optional) Third list of floats
Parameters:
- title: Graph title
- x_label: Label for the x-axis
- y_label: Label for the y-axis
Outputs:
- visual_graph: Visual graph of provided floats
Converts a mask into a float. Works with a batch of masks.
Node Parameters
- mask: Mask input to convert.
Outputs:
- float: Float value representing the average value of the mask.
Converts a list of floats into an IPAdapter weights strategy format.
Use with "IPAdapter Weights From Strategy" or "Prompt Schedule From Weights Strategy" to integrate output into IPAdapter pipeline.
Node Parameters
Inputs:
- floats: List of float values to convert
Outputs:
- WEIGHTS_STRATEGY: Dictionary of the weights strategy
- Install ComfyUI & ComfyUI-Manager
- Launch ComfyUI
- Click on "🧩 Manager" -> "Custom Nodes Manager"
- Search for
ComfyUI_Yvann-Nodesin the manager and install it
