Sigmoid-Logit Mapping

Download

Sigmoid Logit Demo.maxpat
Sigmoid Logit Speed Demo.maxpat
tim.sig.algeb.maxpat
tim.sig.logis.maxpat
tim.sig.trigo.maxpat
README
LICENSE

Github repo

Do you also find yourself often looking for elegant parameter mapping solutions to complex input behaviours? At least I have had this issue numerous times before; enough to warrant keeping a module at hand for most purposes. In this article I want to present some of these tools and explain a bit what is meant by the term sigmoid mapping, a type of "S-curve" mapping that provides greater flexibility to model more complex behaviours.

Mapping a value is understood as the action of modifying a range of input values to a target range. Most commonly the operations are linear, meaning you only shift (addition) and scale (multiplication) the input range to fit the target. For example, MIDI only sends integer values from $0$ to $127$. So, in order to use the velocity information coming from a MIDI keyboard, you might want to scale those values down in the range $0.0$ to $1.0$ and use that as a scaling factor to control a sound's amplitude.

However, in some cases, linear mapping is just not enough. For example, mapping the MIDI values to frequency values requires the MIDI range to be mapping to the logarithmic frequency space, where each octave is a doubling in Hz. Max/MSP provides objects for these common use cases, such as [mtof], or providing the addition of an exponent argument to the [scale] object. But, as things become even more complex, more complex mappings suddenly become more necessary to achieve more obscure tasks. In particular, when composing procedurally or aleatorically, values are often mapped to different distribution curves of various shapes to achieve a number of outcomes.

Exponentials and Powers

My journey towards discovering the class of sigmoid functions begins with the exponential curve. While the classic $e^x$ may seem like the obvious choice – and it may be the most accurate one in some use cases – it offers little flexibility out of the box. In order to map it easily to any target range, it would first have to be fitted to a $[0,1]$ range. You do this by shifting the function by the value it has at $0$, then dividing it by it value at $1$. Moreover, you may want to alter the curvature, so you parameterise the base. The resulting function might look something like this: $${a^x-1}/{a-1}.$$

However, this is just a power function with extra steps. You can achieve a similar result by simply switching $x$ from the exponent into the base and do the opposite to the parameter $a$. The resulting function $x^a$ is already fixed to $[0,0]$ and $[1,1]$, so no shifting nor scaling is necessary! Sure, the parameter $a$ does not map the same way, being much more dramatic in the $x^a$ case, and the curvature is slightly different... but it usually does the job in most cases... and, especially in Max/MSP, it is plopped down so much more quickly, using a simple [pow 2.] and connecting $a$ to the second inlet! Below you can play around with the comparison between the two and have the difference between them visualised:

This is all well and good, but the values are merely either distorted towards the lower bound (if $a>1$) or towards the upper bound (if $0<a<1$). While more complex that a linear mapping, it is still pretty limiting. Surely it should be possible to describe more complex curves within the $[0,1]$ range mathematically? I mean, of course there is and let's explore one of them in this article. Going forward, the above example also reminds us there is usually more than one way to solve a mapping problem, as long as we approach the problem pragmatically.

The Signed Power Function

I always wondered if it would be possible to make some curve that has a kind of plateau, or saddle point, in the middle, instead, where most values would be distorted against or pushed away from. Initial tries combined the square root and power functions to achieve this. However, the ending curvature of the square root does not align well with the start of the power function, and vice versa, creating an uneven curve that is both ugly to program and ugly mathematically.

Then, I came across the signed power function: $$sgn(x-b)|x-b|^a.$$ It is an elegant solution for negative bases, all real-numbered exponents, and produces a smooth transition at the $[0,0]$ point. More importantly, it basically mirrors the power function at the $[0,0]$ point, creating the smooth "S-curve" type sigmoid I was looking for. It even smoothly transitions into a logit function if $a>1$. Granted, like my sqrt/pow solution above, it maps an input from $[-1,-1]$ to $[1,1]$, but a bit of input scaling and output scaling is fine. However, I got curious, what if we not only confine it to $[0,1]$, but also make the saddle point shiftable. For this, we can use the remapping formula: $${g(x-b)-g(0-b)}/{g(1-b)-g(0-b)},$$ which gives us the shiftable signed power function: $${sgn(x-b)|x-b|^a+b^a}/{(1-b)^a+b^a}.$$ You can play around with the function and the parameters in the widget below. $a$ controls the curve's curvature, while $b$ control the position of the saddle point.

In case you have not tried the following, I would like you to set $a>1$ and then play around with $b$. At first you might not notice anything... however, while having $b≠0.5$, change $a$ instead and see how the saddle point moves – which ideally it shouldn't, since $a$ should change the curvature and $b$ should change the saddle point alone! While in "logit mode" (i.e. $a>1$), the saddle point shifts up and down when changing $a$ and makes $b$ much more sensitive... whereas in "sigmoid mode" (i.e. $0<a<1$) the vertical saddle remains fixed in place. The asymmetry comes from the fact that the signed power function is not symmetric around $b$ on the $[0,1]$ domain. The distances from $b$ to the two anchors $0$ and $1$ are $b$ and $1-b$ respectively. Raising those to the power $a$ doesn't preserve the midpoint relationship unless $b=0.5$.

Sigmoid vs. Logit

The solution to this is to take the inverse function to the shiftable signed power function, which means solving it for $y$ and use one of the two functions for the sigmoid case, while its inverse is used for the logit case (ideally whichever has either case in the $0<a<1$ range, respectively). This also requires shifting $a$ around a bit in the background to make the input "feel" the same, as previously in the pure signed power solution. It may show $a>1$ in the bottom right corner of the widget below, but in actuality it is $0<a≤1$ internally. Play around with the parameters and see how the entire curve is much more stable, particularly in the logit case (the sideways "S"-curve). You will also see the first derivative plotted for the function. Have a look at the derivative if you spot anything strange...

Now, if you set, say, $b=0.75$, the saddle point also stays at $0.75$ no matter which value you give for $a$ – for both the sigmoid and logit cases, i.e. at $0.75$ along either axis respectively! Pretty good. However, the whole function became much more complex. Granted, the condition is in the parameter $a$ and not the variable $x$, as was the case in the sqrt/pow solution, but we do require two functions to map the entire behaviour as compared to a single one using pure signed power approach. The whole lot is also pretty ugly to program in Max/MSP! Moreover, you might have noticed the discontinuity, as demonstrated by the derivative that was added. While not problematic from a practical sense, still a bit of a thorn in my eye. Lastly, while the saddle point is fixed along x-axis in the sigmoid case and along the y-axis in the logit case, you may have noticed that it wanders along the other respective axis in each case as you modify $b$. This can be a good or a bad feature, depending on your use case.

The Hyperbolic Tangent Function

So, I poked around some more. One function that I did not consider before and which has a natural sigmoid shape is the hyperbolic tangent, $\tanh$. We can constrain the $\tanh$ function in the $[-1, 1]$ interval by dividing it by its value at $1$. By scaling the argument by $a$ we can modify the curvature of the function: $${\tanh(x·a)}/{\tanh(a)}.$$ To get the logit version, we, once again, have to compute the inverse to this function. Have a look below how this complex of functions behaves for different values of $a$ and have a look at the first derivative as well:

Now, I do not know if this matters, but not only is the function continuous for all values of $a$, the maximum of the first derivative is also nearly equal the value of $a$! Doing the math, it is obvious, of course: $$f'(0)={a(1-\tanh^2(0))}/{\tanh(a)}={a·1}/{\tanh(a)}={a}/{\tanh(a)}.$$ For large values of $a$, the term $\tanh(a)$ tends towards $1$ and, thus, we are pretty much left with $a$ (or $1\/a$ in the logit case).

Moreover, though, this also implies we always have a slope around the saddle point. While the signed power function's discontinuity guarantees a point where the function is truly horizontal or vertical, this new $\tanh$ solution will give you something that is never exactly horizontal nor vertical, no matter how high you set $a$ to. Again, this can either be a bug or a feature, depending on your use case.

Of course, I then had to see what happens if we were to shift the $[0,0]$ point across the x or the y axis, respectively. The simplest approach was to use the same remapping formula as above and plug our new $\tanh$ into it. Then we end up with this very neat and tidy formula for the shiftable hyperbolic tangent solution (sigmoid version): $${\tanh(a(x-b))-\tanh(ab)}/{\tanh(a(1-b))-\tanh(ab)}.$$ Here, we only really need to compute a single instance of $tanh$ for every remapping task, i.e. if $x$ changes. The rest only depends on $a$ and $b$, which can be considered constants for all intents and purposes.

Arguably, its inverse function to get the logit version is not as neat. By shifting everything over on the other side of the equation to solve it for $y$ and we define $C=\tanh(a(1-b))$ and $D=-\tanh(ab)$ for a cleaner look, we get: $${\atanh(x(C-D)+D)}/{a}+b.$$ The good news is that we can reuse all constants from the sigmoid case! These only need re-computation if $a$ and $b$ change. So, the only trigonometric function left to compute on every new value of $x$ is the somewhat daunting $\atanh$...

Below, you have the final trigonometric solution using $\tanh$ and $\atanh$ in each the sigmoid and logit case respectively. You can play around with the $a$ and $b$ parameters and see how the curve behaves. Personally, it is just so satisfying how it shifts across the respective axis while modifying $b$, has it's inflection points right on the set value for $b$ and does not change along the other axis... but maybe that is just pure curve aesthetics and has no real use in real world applications...

Also, you can toggle the signed power function solution from before to compare the two. Have a look how they dance around each other, when one is higher or lower than the other and how their curvature differs. Consider that $a$ is modified to be $10^{a\/10}$ instead, in the background, in this case. Additionally, you can include the difference curve between the trigonometric and arithmetic solutions to analyse the distance between them.

The Logistic Function

Then, I paid attention to the logistic function. I had known about the logistic function before, but in the context of this endeavour it had escaped me that the logistic function also has this characteristic S-shape. It has a similar result to the $\tanh$ solution, but manages to achieve this result using only powers. Confining it between $[0,1]$ and controlling it's slope with $a$ and shift with $b$ gives: $${1-e^{ax}}/{1-e^a}·{1+e^{a(1-b)}}/{1+e^{a(x-b)}}.$$ For the logit version, we solve it for $y$ and define $C=e^{a(1-b)}+1$ and $D=e^a-1$ for a cleaner look: $$1/a\ln({C+xD}/{C-xDe^{-ab}}).$$ In the widget below you can choose to compare it against the $\tanh$ and signed power approach, and visualise the difference between the two:

It was actually around this point that I came across the term "sigmoid" to describe these S-shapes curves in the first place. Discovering this term opened the door to a whole slew of ways to generate sigmoid curves! Digging deeper into the logistic curve, for example, you may find the generalised logistic formulation: $$A+{K-A}/{(C+e^{-B(x-M)})^{1/v}}.$$ It features many parameters to allow a high degree of flexibility when designing your sigmoid shaped curve. Some parameters have varying roles, depending on the values of the other. All this makes the generalised logistic function a rather complex beast to properly tame if you're merely moving sliders around by hand. You can modify the range, slope and symmetry of the curve by using specific combinations of parameters. However, for the purpose what we are trying to achieve in this article, i.e. where we try to keep the mapping in the range $[0,1]$, there is probably no good reason to use this function, since adapting the ranges to the desired input and output ranges using the parameters at hand is not very intuitive (apart from simply slapping the ol' range limiter on it). Nonetheless, have a go at in the widget below:

Moreover, you might notice that it does not give us a logit solution out of the box. This is understandable, since the original purpose of this function is growth modelling, not parameter mapping. However, to be honest, I mainly wanted to include the generalised logistic function here to show off the adaptability and range of functionality of my little graphing widget. So, in this vain, let's test the widget further! Here is the solution for the logit version of the generalised logistic function: $$M-1/B\ln(({K-A}/{x-A})^v-C),$$ and a widget for you to play with as well:

So, you might ask: "Well, can we combine them into a single sigmoid-logit function, make it shiftable and restrict it to the $[0,1]$ range?" Of course we can and of course I will let you play with it. However, keep in mind that all I did is plug the two functions into the remapping formula and slapped a shift-parameter $b$ on $x$. The parametere $a$ here merely controls the linear interpolation from sigmoid to linear to logit in the range $[-1,1]$. Interestingly, the parameters $A$ and $K$ cancel out in the remapped sigmoid case and the parameters $B$ and $M$ do the same in the remapped logit case. Also, you will most likely encounter issues with different parameter combinations, in particular with the logit case. Here, many combinations between the shift parameter $b$ with $A$ ($D$), $K$ ($N$) and even $C$ often cause impossible curves. t is out of scope to investigate deeper into what is happening, since I only included this widget for fun anyway. Afterall, we're only interested in mapping parameters here. Since the whole thing is already so unwieldy I doubt there is a real application in this context. Nevertheless, I hope you like sliders!

Honorable Mentions

To wrap this up, I want to leave you with a few other alternatives that deserve an honorable mention. Some can also be found in a list on wikipedia, for the interested reader:

• Softsign approach: has the generalised form $(x-b)\/(1+|x-b|^a)^(1\/k)$. Unlike the logistic function it has no transcendentals, but it is algebraic like the signed power function. It is used in AI applications as an activation function, and in some other hardware applications where its specific slope is desired. Limiting the formula to $[0,1]$ for all values of $a$ and $b$ using the remapping formula is pretty ugly. Contrary to most other sigmoid curves, the function slopes more the smaller $a$ gets. Additionally, it's inverse solution is pretty weird, in that it does not turn linear for any value of $a$. So, combining the two smoothly seems impossible. However, for negative values of $a$ the sigmoid and logit functions have an unusual discontinuity that is cool to look at.

• Arctangent approach: uses $atan$ instead of $tanh$ to achieve a similar shape. Being very similar to the $tanh$ approach, its inverse is also nearly identical. The curvatures are a minutely different. It is said that $tanh$ is faster than $atan$, which is why the $tanh$ approach should be generally preferable, but we're probably splitting hairs at this point...
• Gudermannian function: similarly, the Gudermannian approach achieves comparable results with an even bigger smorgasbord of trigonometric functions. It is defined as $\atan(\sinh(x))$, while the inverse results to $\asinh(\tan(y))$. It is essentially the same approach as the $\tanh$ or the $\atan$ one, but now the argument is wrapped into two trigonometric functions, doubling the computational complexity, more or less. Nevertheless, at least it is interesting to compare the resulting curve when clamped into $[0,1]$ to the $\atan$ and $\tanh$ approaches. Moreover, it can demonstrate that you can get very creative, if you need to shape your mapping curve in a specific way and have the compute available.

• Soboleva modified hyperbolic tangent (SMHT) approach: an interesting approach with a lot of flexibility to shape the curve in more minute detail without introducing much more complexity. The base function has the form $smht(x) = (e^(Ax)-e^(-Bx))\/(e^(Cx)+e^(-Dx))$. If you're familiar with the hyperbolic tangent function you might quickly realise that if we set $A=B=C=D$ it is the same as $\tanh$. We can plug this into our remapping formula to receive a clamped version, with a shift parameter $b$ as well. However, as far as I am aware, there it is no simple logit version to this solution without using numerical approaches, or constraining the parameters $A$, $B$, $C$ and $D$ somehow... but then we just find ourselves in $\tanh$ territory again, giving us a scaled $\atanh$! Thus, the SMHT must remain in the honorable mention section, as appealing as it might seem. Nonetheless, if sigmoids are all you're looking for, this is not a bad approach, so have a go at the different parameters.

• Error function: uses an integral and can thus be only be approximated. Often used in statistics, including machine learning and other fields. The curve is similar to that of the $\tanh$ solution. However, as far as I know, it does not have a useful logit form for our purposes.
• Smoothstep polynomial: uses the Hermite interpolation with zero slopes at the endpoints: $3x^2-2x^3$. It is often used in graphics and animation. Now we're really far from the originally stated goal! It is another sigmoid curve that does not have a useful logit version. When clamping it to $[0,1]$, some solutions for different values of $a$ and $b≠0.5$ produce results that are neither sigmoid shaped anymore, nor is the output bound to $[0,1]$. Nevertheless, for the fun of playing with the graphing widget, here you go and have fun!

Needless to say, there are probably more sigmoid curves significant enough to make the list. Maybe some that are even worth implementing in Max/MSP that I have not come across yet. It is impressive how a topic so niche can have such a huge barrel that the bottom seems virtually endless with possibilities. Moreover, the different ways of achieving sigmoids hopefully also shows that the base functions used in this article can serve as starting points for more specific or more complex curves, depending on the use case. But, for now, lets look at the functions that were actually implemented as abstractions in Max/MPS.

Max/MSP Implementations

When it comes to implementing these curves in Max/MSP your first thought might be to "simply" write these equations into an [expr] object and call it a day. However, using [expr] is often not the best choice and programming Max/MSP proper, i.e. using the built in math objects is actually preferable. In the particular case of this article there are several hurdles that would need to be overcome and which do not make using [expr] any more appealing to implement than using the math objects in Max/MSP directly:

• There is no $\atanh$ function in [expr] (nor $\arctanh$ if you're wondering that I made a simple syntax mistake). Instead, you would have to use the alternative form $\atanh(x)=\0.5\ln((1+x)\/(1-x))$, meaning you have to copy the entire expression inside of $\atanh$ twice!
• There is no $\sig$ function either and the expression $(x>0)-(x<0)$ needs to be used.
• [expr] has a character limit. Particularly in the algebraic case this was a problem and the entire expression had to be broken down into branches of several [expr] objects.
• The code written using [expr] is a little over twice as slow than the abstractions written in Max/MSP proper. A speed test patch is provided that compares the abstractions against each other and against an [expr] implementation for download:

Because the curves are slightly different in shape and the shape might be important so some, I offer three implementations of the sigmoid-logit abstraction in the code package: the trigonometric, the algebraic and the logistic approach.

The trigonometric approach is by far the cleanest and is also proven to be the fastest of the three approaches. The better performance has several reasons, in my view. For one, the trigonometric uses less objects to implement, causing less overhead. Secondly, the objects used seem to be lighter on the processor. It is definitely my general go-to abstraction if I need some type of sigmoid mapping.

The algebraic approach, on the other hand, is a complete mess to program in Max/MSP. I tried hard to minimize the code and make it more readable, but I seem unable to make this look any nicer. Even the [expr] version, provided in the speed test file, is a mess and cannot reasonably fit into a single [expr] object! It is also the only approach of the three that expects an exponential slope parameter $a$, which means that the standard input for $a$ needs to be scaled by $10^(a\/10)$. While this has no impact on the performance, as this is only calculated on a change of $a$, not $x$, it still makes the code much more bulky than it needs to be.

The logistic approach is also not as satisfying as the trigonometric one, when it comes to coding it in Max/MSP. While more organised than the algebraic one, there is no explicit $\exp$ nor $\log$ function object in Max/MSP. For the $\exp$ case, the only object available for the job would be [pow]. However, due to Max/MSP handling of "hot" and "cold" inlets, only the base can trigger an output of the object, meaning the number $e$ would have needed to be injected every time some $e^k$ is computed somewhere. More than making the code worse, though, is the fact that [pow] seems to be slower than [expr exp()]. This is why I opted for using the $\exp$ function provided by [expr]. For the $\log$ case there is no good alternative to using the [expr] object either, which, from a coding philosophy, is a bit of a bummer...

Finally, yes, there are other ways to implement this, most notably using [gen]. I think I will want to explore this option and potentially others in the near future to have clarity as to which implementation is the best approach. Nevertheless, at least Max/MSP provides us with the ability to create abstractions. This turns a patch into a black box. The beauty of this approach, as you can probably guess, is that if a better approach is found in the future, a simple replacement of the file will update all projects and have them benefiting from the performance increase almost automatically. Until then, I will leave it at that.