Additional panning algorithms (VBAP, DBAP)? "Plugins"?

Contributing to MMMAudio
1

Are there any plans at the moment to include panning algorithms like VBAP, DBAP, LBAP, etc. in the core MMMAudio library? (If so I’d be happy to take this on!) If not, are there any thoughts on how “plugins” (with the understanding that a “plugin” is just a .mojo file) will be handled? Separate git repos for every “plugin” that is made? A collected git repo for “official plugins” like sc3-plugins? I realize this is a pretty early question to be asking but I’m curious what the plans are for the future!

1 Like
2

Hi Drew!

There’s been some discussion on this but I think no settled plan. It’s likely that there will be some things that don’t get merged into the core repo in order to manage maintainability. However, the panning algorithms you describe seem to me like they’re worth having in the core repo.

I’d say go for it and make a PR!

T

ps. Thanks for checking in re: contributions, that’s always wise!

1 Like
3

Sounds great! Pull requests should be made on the dev branch correct?

1 Like
4

In general yes, I think it’s a good question for @spluta, regarding how dev relates to main and how releases fit in etc. I usually make my PRs on whatever branch seems to have the most recent pushes on it.

1 Like
5

It would be awesome if you could work on these.

All commits should go to dev. I got a little cavalier about things last week and pushed to main when I shouldn’t have, but from now on, we should push to dev then push into main once we know the code is solid.

For VBAP/DBAP, take a look at pan_az. (In writing this message, I looked at it and pushed a second version that is more efficient (more calculations at compile-time), but you can’t change the number of channels after compile. I think we will just get rid of the first one.). A couple of features of it to consider:

  1. it is just a function, not a struct, and I think you can avoid a struct
  2. the second version uses a lot of compile time parameters and calculations to be very efficient.
  3. in both versions, the number of output channels is a power of 2, while the number of speakers is any value. this is necessary. the number of channels of the output vector needs to be 2, 4, 8, 16, etc. but you could have 11 speakers in your calculations.
  4. your code should go in Pan.mojo. We are getting to the point where we need a MMMAudioPlugins repo, but we aren’t quite there yet.

Let us know if you have questions. You are digging into a difficult are where SIMD and num channels don’t totally align, so there may be multiple ways to proceed.

Sam

1 Like
6

This is because MMMAudio uses SIMD vectors to represent multi-channel audio streams and SIMD vectors always need to be a power of 2 in length. In MMMAudio a SIMD vector can also be created/used as an MFloat[n] where n is the power of two indicating the length of the SIMD vector.

1 Like
7

Thanks for all of the guidance! I’ll get working on this and hopefully have something up quickly! Is there anything to look out for when working with SIMD vectors? Or do they just function like any other indexed collection? (Arrays, Lists, Tuples, etc.)

8

You can index them totally normally as you suggest. Also, the power of them comes from treating them like “vectors” which enables the parallel processing on the CPU that they’re designed for (it’s also often very convenient to treat them like vectors). For example:

a = MFloat[2](2.0,3.0)
b = MFloat[2](5.0,6.0)

c = a * b

c will equal MFloat[2](10.0,18.0)

//=============================

also:

a = MFloat[2](2.0,3.0)
b = MFloat[1](5.0)

c = a * b

c will equal MFloat[2](10.0,15.0)

//==============================

For what you’re looking to do, you could think about masking certain channels (I’m just shooting from the hip here, trying to give you a sense of how you might think about things, I’m not trying to suggest an implementation at all):

sig_dup4 = MFloat[4](sig,sig,sig,sig)
chan_mask = MFloat[4](0.0,0.5,0.5,0.0)

c = sig_dup4 * chan_mask

now in c you have sig at -6 dB in chans 1 and 2 and none at all in chans 0 and 3.

//========================================
this isn’t in the repo but here’s a matrix mixer implementation that relies on SIMD and also uses the .reduce_add() to sum all the values in a vector. Maybe it’s a good example of thinking of SIMD in terms of vectors.

struct SIMDMatrixMixer[inputs: Int, outputs: Int](Movable, Copyable):
    var coeffs: List[SIMD[DType.float64, Self.inputs]]
    var output: SIMD[DType.float64, Self.outputs]

    def io_to_idx(self, i: Int, o: Int) -> Int:
        return o * Self.inputs + i

    def __init__(out self):
        self.coeffs = List[SIMD[DType.float64, Self.inputs]](length=Self.outputs,fill=SIMD[DType.float64, Self.inputs](0.0))
        self.output = SIMD[DType.float64, Self.outputs]()

    def next(mut self, input: SIMD[DType.float64, Self.inputs]) -> SIMD[DType.float64, Self.outputs]:
        comptime for i in range(Self.outputs):
            self.output[i] = (input * self.coeffs[i]).reduce_add()
        return self.output

    def update_coeffs(mut self, new_coeffs: List[Float64]):
        for o in range(Self.outputs):
            for i in range(Self.inputs):
                idx = self.io_to_idx(i, o)
                self.coeffs[o][i] = new_coeffs[idx]

Feel free to come here with questions. We’ll be excited to help!

2 Likes
9

Ah thanks that’s very helpful! So only operations performed on the entire vector (or I guess there must be a way to address power of 2 segments of the vector?) to get performance gains, good to know. I’ve got a working version of DBAP going that I’ll make a pull request for. Also, I added a function for converting InlineArrays to MFloat vectors. This was the cleanest way I could think of to get a comptime SIMD from an array. If this would be better placed in the functions file, or removed entirely and some other solution implemented, let me know. Also, if there’s a better way to do this please let me know! (using next_power_of_2() was throwing errors for me for a reason that I could not understand).

10

This is awesome. I made some tweaks that are mostly just SIMD things.

  1. I SIMD optimized the math. Doing the vectorized math is totally fine when SIMD vectors are small (1-4 values), but when they are bigger than 4, and they probably will be with VBAP, then you want to break them down into separate calculations of 2 or 4 values. I’m not sure why the compiler doesn’t do this optimization itself, but it doesn’t.
  2. Uncle GPT tells me that k = 1 / sqrt(denom). You had k = 1/denom. Which is correct?
  3. I moved the very excellent array_to_mfloatto functions, since this is generally valuable.
  4. I removed the default parameters because I’m not sure they make sense and I wasn’t clear that the InlineArray would populate correctly. Anyways, these should always be provided.

Could you make an example so we can try this out?

1 Like
11

Ah that’s good to know as well, thanks for the help on the SIMD things. I’ll make sure to study it closer for the VBAP implementation. It’s definitely supposed to be k = 1 /sqrt(denom), I think I accidentally removed the sqrt when attempting to SIMD optimize it. I’ll get an example up ASAP, I can also add a unit test for array_to_mfloat() if you’d like.

1 Like
12

Dope.

Yeah, its always good to ask an LLM after you think you are done, just to ask if the math is right. They say a lot of dumb things, especially about SIMD, but they catch little errors like this. I wouldn’t have seen it either.

I think it is so awesome that this is just a simple function. The comptime stuff makes this stuff super clear. I finally just figured out how to make an env without using the Env struct.

Sam

1 Like
13

This is super.

I think the documentation can be yet improved some. Are these good questions / improvements? Some of these are genuine questions I have, some are rhetorical questions, but the kind that a user might have, so good to answer in the documentation!

//==================================

num_speakers: The number of speakers as an integer. Must be <= simd_out_size.

simd_out_size is no longer a parameter so should be removed from this description.

//==================================

weights: An InlineArray of Float64s defining speaker weights for DBAP.

What are the weights for? How do I know what to put in?

//==================================

blur: Blur between speakers. Values > 0 spread the source to more speakers.

What about values < 0? Is that valid? Useful? What’s the range of possible values? I see that the default is 0.1. Is that number specifically meaningful or just a “good” heuristic? Is 0 or 1 meaningful in some way?

//=================================

rolloff: The dB Rolloff (defaults to 6db).

It’s not necessary to indicate the default in the docstring because the documentation automatically pulls it from the function signature and displays it on the web page. Not putting it in the docstrings reduces noise and prevents possibly the codebase and documentation getting out of sync.

Also, is this for a filter? Is it dB / octave? How is this number used?

1 Like
14

I’ll update the documentation as well! (I think I got a little over-excited about making a commit) In the future I’ll spend more time testing and looking over things. Also, I’m noticing that the next_power_of_two function is causing a crash when used to determine the simd_out_size. Changing back to explicitly stating the simd_out_size fixes this for now, not sure why though.

15

I haven’t tested it yet. Are you and Sam on the same version of Mojo?

16

That is an excellent question, let me make sure I’m using the latest version of it. I’ll also make a pull request of a quick and dirty example. I intend to update the example with a more demonstrative gui once all is said and done.

I’m on Mojo 1.0.0b1 (the latest stable release)

I figured it out after looking at the pan_az example more carefully… Should be working as intended now. I’ll get around to updating the documentation soon.

1 Like
17

Yeah. The next_power_of_two thing is weird. If the function returns an MFloat[next_power_of_two(8)], that isn’t the same as MFloat[8], even though they both have 8 values.

So we might want to rething pan_az and DBAP. Maybe we have to explicitly give both the num_chans and the num_simd_chans.

I would ask the forum, but I know no one would answer.

2 Likes
18

Yeah, it’s strange that it works that way (at least to me). Maybe the mojo team will change this? For what it’s worth, as a user I think I would prefer to explicitly give the simd_out_size and have they way to format outputs consistent across all panning functions instead of having to assign the values to a new MFloat. It’d also make it less confusing as to what is actually causing the error (in my opinion).

Regardless, let me know which way you’d like to take it and I can adjust the XBAP functions to be consistent.

19

Can you elaborate on this, maybe sketch out what you would like the ideal use-case to look like?

20

Sure. I think I would prefer if all of the panning functions were used in the same way even if their arguments are all different (especially if it stops me from doing something in an inefficient way like calling something at runtime that could have been done at compile time).

For instance, when using all of pan2, pan_stereo, and the splay functions I would use the same pattern:

def next() -> MFloat[2]:

    sig = self.osc.next()

    out = pan2(sig, 0.0) 

    return out

I would prefer if all of the panning functions followed this pattern (or a consistent pattern), regardless of the style of panning. It would eliminate confusion (such as how it took me extra time to figure out why the dbap2D function itself wasn’t throwing errors but using it like pan2 was.) So instead of the current solution which saves some redundant typing I’d rather explicitly state the simd_out_size and retain the same programming pattern:

def next() -> MFloat[8]:

    sig = self.osc.next()

    out = dbap2D[
    num_speakers=5,
    simd_out_size=8,
    speaker_pos= [MFloat[2](-3, 10), ....],
    ](
    sig, 
    MFloat[2](0.0, 0.0)
    )

    return out

At least for myself, this would reduce the likelihood of making a mistake when using a new panning function.

The only issue is that a user might not provide a valid simd_out_size and this will throw the same kinds of errors that the current solution causes. Though maybe some error handling would help? I’m not really familiar with how Mojo’s error handling works. It’d be great if it could catch the error at compile time and spit out an error. It would be fairly easy to check with a comptime conditional as well:

comptime if simd_out_size < num_channels or simd_out_size % 2 != 0:
    #Throws an error

I’m not sure how feasible this is, I’m just spitballing. Regardless, it would be easier to look at the documentation and see that simd_out_size must be a power of 2 and greater than num_channels than explaining in text how the current solution is working.

1 Like