Dick (FOV) | 6 years ago | 16 comments | 1.6K views
I like video editing on the beat (kind of music videos).
BluffTitler is great in the way it works and almost every parameter can be "animated", so it has potential for music videos too (f.i. effects on the beat).
The Ultimate version has realtime "VJ effects" (in the layer menu) to animate parameter changes by the audio. It is possible to limit the frequencies the VJ "detectors" respond on, but real onset/beat detection is far more complicated than realtime detecting the peaks of the audio. If you want to animate one parameter on the bass, another on the vocals and another on the hihats of the music, then this is difficult (even with different VJ settings).
There is excellent open source software available that can analyze music in all kind of ways and can write the "detection" results in a simple text/CSV file. See: sonicvisualiser.org
Wishlist:
1. (best and most general solution) A button near the keyframes area, which 'll import a common text file with keyframe positions (just lines with the time in msec?) for the current selected property (parameter).
2. (second best solution) A menu option in the VJ dialog that let you choose the audio layer (even if it is muted / not visible) the audio detector of the current VJ effect uses for the detection (menu choices f.i. "mix" (= all audio layers), name audio layer 1, name audio layer 2, name audio layer 3 , etc.). It is important that audio detection 'll happen even if the selected audio layer isn't visibile (muted). In this way it is possible in another application to make multiple "perfect" detection versions of the music for different parameters and to use multiple (muted) audio layers.
Any chance one of these options 'll be implemented (at least in the Ultimate version)?
Thank you for your requests!
Why don't you post some examples of your work in this community? I would love to see what you do with the current VJ features. That's always the best inspiration/motivation for improving a feature.
Are you aware that the .bt file format is a simple text format? No need to import another text format when it's already a text format. You sound like a coder so it must be easy for you to write a script to generate a .bt file based on sonicvisualiser data. At the bottom of this page you can find an example in PHP:
michiel, 6 years ago
Thanks for your answers.
It is difficult to upload a show with a music file to demonstrate what can be improved with VJ if the upload filesize of a zip file is limited to 500 kB.
My main "complaint" of VJ is that it is difficult (at least for me) to animate in a consistent and tight way at the musical "logical" spots of the music (sometimes on the beat, every 4 beat, sometimes on the "off beat"). Yes, it detects peaks, but not always at the wanted spots (even if you try to find the right frequency range) and you can't correct anything afterwards because it is realtime. For me VJ is nice to get an idea for a musicial "BT animation", but at the moment not for the final animation (for all kind of music).
My idea / goal is to animate properties / parameters synchronous with the music, not on every peak / beat. For (semi automatic) video editing on the beat with Magix MovieStudio / Vegas Pro I have developed my own solution, see the embedded video. I want to do something similar for animating properties / parameters of BluffTitler (the keyframe solution).
Yes, I know the BT files are textfiles.
But to do it all myself I have to deal with about 300 parameters, which can have multiple properties (like X value, Y value and Z value) and which have different valid minimum and maximum values.
So "detected beat data" in the music must be "mapped" to the valid (and wanted) value range for the choosen parameter(s) to animate.
To make nice customizable "animation peaks" probably there should be at least 3 keyframes around the beats. With new BluffTitler versions, the tool must be updated to support the animation of new parameters / properties.
So I was hoping that a simple keyframe import button near the " keyframes bar" in BluffTitler, that would import the keyframes of a textfile for the current selected parameter would do a part of the job for me.
Dick (FOV), 6 years ago
But why not upload a YouTube video of your work? That would immensely inspire us to improve the VJ effects.
Here's a show by Komies. He has connected the TIME DILATION property of the particle layer to an audio frequency using the VJ dialog. That's a great effect and I'm sure there are many equally effective (or even better!) VJ effects still waiting to be discovered. Maybe you have discovered a few. You would make me and other users very happy if you would be willing to share them in this community!
michiel, 6 years ago
Controlling 300 parameters is not necessary at all. An easy way would be to control 1 and find a way to poke it at the right place into an existing .bt file. That would be a quick way to proof the concept.
michiel, 6 years ago
Great Show by Komies.
But try the same with some other (for the audio detectors more "difficult") music like "Mad About You" as in my video. I didn't find the right settings in VJ for the effect I wanted with this kind of music.
Controlling 300 parameters at the same time isn't necessary, but an external conversion tool, that modifies the BT show file with additional keyframes, must "know" the name and range (min and max value) of the values of the properties to animate (or the user must specifiy these).
I 'll try to make 2 very simple shows with the music "Mad about you" and 1 animated parameter; one show with the "optimal" VJ settings and one show with a modified (poked) BT file with keyframes at the time spots of the externally detected beat data.
I'll share these videos and BT files (if I can upload a zip file with the BT showfile and music that is bigger than 500 kB).
Dick (FOV), 6 years ago
To support the "proof of concept" I made a little quick and dirty tool that generates animation keyframes from the input of an external "Beat detector" (based on the beatroot algorithm). Actually the tool "constructs" a peak with 3 keyframes around the "beats", see picture.
I noticed that I had to slide the keyframe values for about 300 msec to "synchronize" the audio and the animated video in Blufftitler (preview). But after an export as an uncompressed avi and an import in Vegas Pro the synchronization was 300 msec "off"! I think this is related to some "buffering" of audio or video in BluffTitler for the preview? After adapting the keyframe values to the original values (buffer compensation = 0) and exporting it seems to be OK.
This video uses the "keyframe animation method" without VJ.
(Detection of every 4 th beat and an offset of 3 beats.)
Dick (FOV), 6 years ago
This is the video with only VJ audio detection.
Even if I change the parameters I don't get it how I want it with this kind of music.
Dick (FOV), 6 years ago
This is the video with VJ audio detection PLUS keyframes from the external audio detector.
The combination seems to give the right balance.
My verdict (at the moment): both methods can exist together and can enhance each other!
Dick (FOV), 6 years ago
Nice work!
peak detection has always been a problem, i remember the first analog guitar synthesizers, the most difficult was to detect guitar pick on the string and prevent vibration aberrations, there was the same problem on vocoders where it was necessary to build one envelope followers (pic) per band to drive voltage controlled filters (VCF).
i suppose digital technology simplified that but i'm sure peak detection satys a problem.
vincent, 6 years ago
Aha, I think I now understand what you mean with "beat detection".
Since all dance music uses a fixed BPM (beats per minute), isn't this the same as "BPM detection"?
If this is true, we don't need complex audio analysis and a BPM textbox in the VJ dialog would be enough.
This way it even works without audio files and can be used to cycle property values for all kinds of reasons.
michiel, 6 years ago
@Vincent
Thanks for showing one of the "problems" with "normal" peak detection.
@Michael
Complex audio analysis is needed to nail the exact "beat" positions. As you can hear in the song "Mad About You" the audio frequencies are not the same at every beat. Beat time intervals are not always consistent in a whole song. Often the beat intervals are on purpose made not exact the same (to "humanize" the music).
So I think just a simple workaround (BPM textbox) wouldn't do the trick.
However I like the idea to add the possibility in VJ of "artificial peaks / values" that are not direct audio driven, especially if you add this feature for every VJ effect. Because multiple VJ effects can animate the same parameter, the chance to get the animation of a parameter "as wanted" grows. The current VJ audio detection and the "artificial peaks generation" can work together (adding animated parameter values).
My suggestion for the "new controls" of a VJ effect (for every VJ effect) for a "proof of concept":
1 [checkbox] Basic VJ Audio peaks detection enable
2 [checkbox] Artificial Peaks Generation Enable
(if both checkboxes are checked Basic VJ Audio peaks detection and Artificial Peaks Generation work together)
3 [numeric input] start time (msec)
(start time artificial peaks adding in timeline (first peak); default 0)
4 [numeric input] end time (msec)
(end time artificial peaks adding in timeline; default end of timeline)
5 [numeric input] time distance between peaks (msec)
(= peaks interval; wide range to control, default ?)
6 [numeric input] peak width (msec)
(default 300 msec?)
7 [numeric input] parameter value peak
(default current value choosen parameter?>)
8 [numeric input] parameter value no peak
(default current value choosen parameter?>)
With these controls a user has maximum flexibility:
a. If the user only wants artificial peaks on a part of the timeline, it is possible.
b. If the choosen peaks interval is just valid (synchronous with the beats) for a part of the song; the user can use peak effect 1 with peak interval 1 and peak effect 2 on the same parameter with peak interval 2.
d. If the user wants short sharp peaks or wide soft peaks, it can be adapted.
e. The user can choose with the controls the "synchronization point" for the peaks in the music (f.i. peak on every beat, peak on every 3 th beat of a 4 beats interval).
Because "artificial peaks adding" is a workaround, I think you should not limit the controls of the user to get it "as wanted".
I am curious!
Dick (FOV), 6 years ago
An audio file is nothing more than a list of peaks, so peak detection is easy. However this is the sum of all frequencies. BluffTitler uses basic Fourier analysis to extract the peaks for 19 frequency ranges. This works very well for many tracks. But not for your favorite tune. Many refinements have been made since monsieur Fourier discovered the algorithm about 200 years ago. We will take a close look at those techniques to improve the VJ dialog.
Beat detection is something else than peak detection. This requires some further analysis. Maybe a beat detection checkbox is all the GUI needs.
Thank you for your suggestions and request!
michiel, 6 years ago
Maybe I am wrong, but I expect adding only a beat detection checkbox (and a sensivity control?) 'll only satisfy overall if you use real beat detection that is based on a good beat detection algorithm.
I have good experiences with the Beatroot detection algorithm.
Documentation (see the PDF links on the right of the page [more details]):
code.soundsoftware.ac.uk...
You can test this algorithm with Sonic Visaliser ( sonicvisualiser.org... ) with the "Beatroot Vamp" plugin ( vamp-plugins.org... ). Sonic Visualiser lets you export the beat locations in a text file so you can compare it with your own implementation of beat detection / "parameter values peaks generation". Please compare with different kinds of music.
In practice for a nice animation flow you want to happen the "parameter value peaks" not on every beat but f.i. every 4 th beat (onset) starting with detected beat 3 (so not always on the first beat; depends on the music and is a matter of personal taste).
If you manage to extend VJ with good beat detection (that can be used for animation of parameter values) for a wide variety of music kinds with just some controls, all users 'll benefit.
Dick (FOV), 6 years ago
Thanks for the links!
Have you already played around with the Special\Oscillator effect? The current version only animates the size, but it could easily be extended to do other props. This way it could become a VJ alternative for fixed BPM tracks.
michiel, 6 years ago
I didn't notice the oscillator effect before.
I did a little test with it.
I think, before it can serve as a VJ alternative for fixed BPM tracks, the characteristics / shape of the parameter animation should be something more like the shape in the picture I made:
- not sine shape parameter animation but more parameter peak shapes at the "supposed" peaks interval (for instance 4 beats; period time of oscillator);
- possibility to set a "FXPeriod" time > 1 second (till 3 seconds?).
Dick (FOV), 6 years ago
With the 1st slider of the FX HOLD property you set the time the unadjusted state is held. In your example this is about 3 beats.
Prop values can be set outside their slider range with the textbox next to the slider.
michiel, 6 years ago
