Right I remember that now. The numpy version was actually very slow (I think I did the numpy version to prototype the FFTProcessable stuff and you fixed it with the Mojo-native FFT). If I remember correctly, it was almost jittery-slow, and that was using it every “hopsize” samples, not even every single sample.
1 Like
This should only need to be done once. I’ll take a look at those other examples as a starting point when I find some more time!
Finally getting back to this but I’m running into a small problem. I’ve gotten scipy’s Delaunay implementation to work inside mojo (yay!) but for some reason mojo refuses to cast a numpy int into a mojo Int…
For example:
for triplet in delaunay.simplices:
var first_speaker = Int(triplet[0]) # triplet is an ndarray, triplet[0] is an numpy.int32
fails to compile.
Is there a reason for this?
1 Like
can you try:
for triplet in delaunay.simplices:
var first_speaker = Int(py=triplet[0])
2 Likes
That did it! Thanks for the help.
2 Likes
Alright, I think I’ve ironed everything out (at least it appears so by the numbers now.) I’m going to try to get access to a suitable array to test soon.
After a working version is up, I’m going to look into adding GPU acceleration. VBAP would be a good candidate for this as it requires a bunch of matrix math.
1 Like
Regarding the GPU acceleration, recently, I have looked into this and I believe @spluta has as well. For real-time applications it is just too much overhead to have the CPU and GPU communicate with each other. It takes longer for data to travel to the GPU and back that just computing a real-time result on the CPU. Now, if the number of audio channels is like 100+, maybe it would be faster but even then I’m not sure.
How big are the matricies?
I’ve been writing GPU kernels recently, so I’d be glad to take a think with you if useful.
1 Like
Ah, that’s unfortunate to hear. For VBAP, every speaker triplet has a 1x3 3x3 matrix multiplication. So for a 40 speaker array that would be roughly ~78 triplets. These only need to be calculated when the azimuth and height change. Right now I’m storing the matrices as arrays of MFloat[4]s. I’m assuming here, but SIMD parallelization is probably going to be faster than sending to the GPU and getting things back?
Although, for VBAP, the 3x3 matrices are calculated at initialization and so could live in VRAM while the 1x3 matrix is the 3D unit vector of the azimuth/height pair. I just started looking at mojo’s GPU coding stuff the other day, but would this make it so that there’s only the GPU->CPU transfer? So the steps would be:
- Initialization (calculate inverse matrices for all speaker triplets)
- Load matrices to a buffer on the GPU
- DSP Loop
-
- Perform matrix calculations on the GPU
-
- Copy the GPU’s buffer back to the CPU
SIMD is much faster than GPU for this size data set. I’ll send you a mockup that might be helpful.
2 Likes
thinking about this some more. there is no reason for the matrices to be float64. if you change them to float32 or even float16, you should be fine and you would be able to deeply parallelize the multiplication. gpu is only float32 anyway.
i think 65536 points between each speaker is probably plenty. hell, you could go 8 bit.
1 Like
That’s really clever, I hadn’t even considered it. For SIMD it would make the most sense to go down to a float32 though right? Each vector has 4 components, so reducing it further wouldn’t see any gains if I’m understanding SIMD correctly? Maybe combining the vectors into a single SIMD vector and then multiplying by another, but I’m still a little fuzzy on how that works. So for example:
[a1, a2, a3, 0.0]
[b1, b2, b3, 0.0]
[c1, c2, c3, 0.0]
becomes
[a1, a2, a3, b1, b2, b3, c1, c2, c3, 0, 0, 0, 0, 0, 0, 0]
I think these would need to be 8bit floats if we’re taking Apple’s 128bit SIMD registers as a yardstick. Then it would just need to be multiplied by a SIMD vector holding the source vector’s components:
[x, x, x, y, y, y, z, z, z, 0, 0, 0, 0, 0, 0, 0]
That seems pretty doable. The resulting 1x3 vector would need to be cast back to a Float64 at the end but I don’t see that offsetting the performance gains of calculating 9 multiplications at once.
I’ve got an on paper (as in everything outputs the expected number in mojo and it pans correctly between the 0 and 1 speakers) working version of 3d VBAP now. I’ll try to get into LSU’s immersive studio in the next couple days to confirm it doesn’t blow anything up and then make a pull request. I also realized I had discarded a commit that included 3d DBAP at some point and need to test that as well. If that’s working properly I’ll include it in the pull request (unless you’d like me to split them).
Is there an easy way to gauge the performance? Some way to check the delta time of calculating a DSP block?
1 Like
on ARM, 64 bit SIMD vectors are 2 wide, 32 bit are 4 wide, 16 bit are 8 wide, etc. so for 16 bit, 16 wide will be performant and 32 wide might be moreso.
a simple mojo test file that loads the struct, checks the time with perf_counter then runs the function 100000 times and checks the perf_counter again should do the trick
1 Like
awesome that this is now in dev. i look forward to trying it out!
1 Like
After some preliminary testing, 32 bit SIMDs seem to be the best choice for optimizing this. They’re ~50-200% faster than 64bit floats, and 16bit SIMD vectors are usually roughly equivalent in performance to 32bit and are more volatile (more often producing results that are worse than 32bit or 64bit versions on my machine.). I’m going to set up a script to run these a ton of times to make sure, but 32bit seems to be the sweet spot. After, I’ll add a comptime parameter to VBAP3D that allows the user to choose between 64 and 32bit vectors, although I’m not sure if there will be much audible difference between the two in terms of ability to localize a sound. Either way, the typical use case for many VBAP instances (lots of sources) would limit localization anyway, so it seems a worthwhile tradeoff. Very cool to be able to make that decision as the user though!
Okay after actually running the numbers, 32bit is ~30% faster than 64bit and 16bit ~30-50% faster than 64bit. 32bit was pretty consistently around ~30% faster but 16bit jumped around a lot (some times resulting in values worse than 32bit and sometimes much better). I think the stability of 32bit is probably more important than a potential extra ~10% gain.
Are you doing lots of copies? Those are expensive. Whatever you can do to avoid copies will help.
But this does seem like a dramatic improvement!
I don’t believe it’s copying, at least it’s not having me explicitly copy anything. I found the results I was getting from perf_counter() to be really inconsistent so I spent some time learning (on a very basic level) mojo’s benchmarking tools, and trying a different approach to perparing the matrices and wound up getting some really impressive performance improvements at 16bit. Ignore the axis labels: y-axis is the average time to calculate a 4 triplet speaker array, x-axis is the implementation. The lower the bar, the faster it’s doing it.
For each of these, the Xbit is the current implementation with that bit float and the X-transposed bases is the new way. I ran these benchmarks ~20 times (each benchmark does >100,000 iterations) and the results were really consistent. I’m not certain as to why 32bit is just less efficient than 64bit but it seems like there must have been something else going on with the way I was testing performance going on because it was consistently the worst. I’m planning on writing a paper on this, I think it really highlights the flexibility and approachability to MMMAudio and Mojo.
1 Like
After banging my head against this over the weekend, I unfortunately am not able to replicate these results within the VBAP struct itself. It’s possible that the matrix calculations are such a small portion of the processing time that it’s irrelevant. I’m now only getting marginal if any performance boost from switching to 16bit, which I don’t think is statistically significant.
I feel that there must be a way to realize the gains from the 16bit-transposed bases implementation as they are significantly better when done out of context. It’s possible that the compiler is just really good at making 64bit SIMD operations very efficient (that would make sense to me?) and that the time chasing this could be spent better.
Regardless, I think it was a good learning experience and I have a much better workflow for benchmarking now. I also got some time in LSU’s immersive studio today and can confirm it works with no hiccups on my end!
This is very possible.
Are you testing it as a whole MMMAudio process, because in that case, there is probably so much computation going on and VBAP is such a small part of what “needs to get done” that it is not noticeable. Fluxuations in the CPU’s load might cause greater deviations that the differences you’re trying to measure.
Have you tried running, like, 1000, VBAP structs at once inside a MMMAudio process? Or as many as you can on 1 CPU? That might tease it out.
But, in any case, if you can demonstrate that a small block of code is more efficient one way instead of another, that’s valid enough to decide to prefer it. All those small gains can add up.
This is always time well spent.
If you find a moment, I’d be curious to see. I’ve been doing some benchmarking as well (I’d be glad to share too!). Maybe we can find a “best practices” for benchmarking MMMAudio, maybe even put together a framework for it.
1 Like
Are you testing it as a whole MMMAudio process, because in that case, there is probably so much computation going on and VBAP is such a small part of what “needs to get done” that it is not noticeable. Fluxuations in the CPU’s load might cause greater deviations that the differences you’re trying to measure.
I was actually trying to isolate it as much as possible, so I had moved it into a separate file and was using the benchmark function to run VBAP3D’s next function. It looks something like this:
from std.time import perf_counter
from std.random import random_float64
from std.benchmark import benchmark, keep, run
def main() raises:
var speaker_array : Array[MFloat[2], 6] = [
MFloat[2](-0.5 * pi, 0),
MFloat[2](0.0, 0.0),
MFloat[2](0.5 * pi, 0.0),
MFloat[2](0.0, -0.5 * pi),
MFloat[2](0.5 * pi, 0.5 * pi),
MFloat[2](-0.5 * pi, 0.5 * pi),
]
var vbap_64 = VBAP3D[6, 8, DType.float64](speaker_array)
var az = random_float64(-2 * pi, 2 * pi)
var ht = random_float64(-1 * pi, pi)
@always_inline
def vbap_test() raises {var vbap_64^, var az, var ht}:
keep(vbap_64.next(1.0, az, ht))
var report = run(vbap_test)
report.print()
VBAP3D currently doesn’t take a world argument, but for other structs I believe the only change needed to run them this way would be to remove that when testing. It’d probably be good to add a world argument to VBAP3D’s init anyway, so I’ll make sure to do that soon.
Have you tried running, like, 1000, VBAP structs at once inside a MMMAudio process? Or as many as you can on 1 CPU? That might tease it out.
I haven’t tried this but that’s a great idea! I’ll get around to checking this this weekend. That should be as easy as modifying the above code to something like this:
from std.time import perf_counter
from std.random import random_float64
from std.benchmark import benchmark, keep, run
def main() raises:
var speaker_array : Array[MFloat[2], 6] = [
MFloat[2](-0.5 * pi, 0),
MFloat[2](0.0, 0.0),
MFloat[2](0.5 * pi, 0.0),
MFloat[2](0.0, -0.5 * pi),
MFloat[2](0.5 * pi, 0.5 * pi),
MFloat[2](-0.5 * pi, 0.5 * pi),
]
var vbap_64 := Array[VBAP3D[6, 8, DType.float64], 200](fill= VBAP3D[6, 8, DType.float64](speaker_array))
var az = random_float64(-2 * pi, 2 * pi)
var ht = random_float64(-1 * pi, pi)
@always_inline
def vbap_test() raises {var vbap_64^, var az, var ht}:
for i in range(200):
keep(vbap_64[i].next(1.0, az, ht))
var report = run(vbap_test)
report.print()
I’d be curious to see how you’ve been doing it as well! I think my approach is really only suitable for testing single structs/methods/functions and not more complex scenarios like the entire MMMAudio instance. Some quick thoughts:
I found the approach I took to be useful for quickly iterating; my compile times were typically very fast and it was faster (for me) to find where errors in my code were when I was making changes. I was also running different functions on different reports in one pass, although I’m unsure if that was influencing anything. (I didn’t notice a difference between running the benchmarking separately or at the same time). This allowed me to run a bunch of different scenarios and compare their benchmarks directly which was pretty cool. I could also compile down to a file and then make a bash script to run that file like 1000 times and get the average speed of all of those runs.
edit: Running the multi-instance tests (at least initially) reproduce the previous performance gains! I’m curious if there’s any reason that the for loop would be producing this change though?
another edit: And the gains are gone again? I’m pretty confused by this… However, by pre-formatting the matrices and using 64bit floats (how I was doing it with 16bit floats) I’m getting better performance than the previous implementation.
I think world arguments should only happen when they are needed. I know it’s a convention now that many things take it, but if it’s not needed I think it keeps clutter away. Also, it becomes meaningful, when that needs to be passed as an argument or not, it actually tells the user something about what is happening inside where world is getting passed.
// ==================== //
I wasn’t using the language’s benchmark tool. That’s awesome. I will try it!
// ==================== //
No idea. I’d have to see it more in action. Compilers are wild. In the loop, it could be that some things are getting cached or branches are being predicted making it look like a performance gain when really it’s a CPU being smart, however those can certainly be considered performance gains if the code is written in a way that can take advantage of that.
When it’s not in a for loop, there are probably many other computations that need to happen before the next next happens which means the cache will very likely not hold the relevant variables anymore…
That’s all to say that testing outside of context can be deceiving. However, using the language’s benchmark tool probably has ways of trying to avoid these pitfalls! I’m looking forward to checking it out!
1 Like