Hyperbolic Tangent Activation Function
So well this an age comes from geometrical interpretation. So if you have your outputs a z so if you represent and it’s of Z, so it is nothing but your sine h of Z upon cos h of Z. It is a hyperbolic function. So you have your unit you compute your linear part. So say you have your biases the b for w 0 then you have W1 W2 X2 and X1 you compute your linear bar W 0 plus W1 X1 plus W2 X2, and then comes here and its function and It’s of Z and it gives you some output at the output layer.
If you have a large pool of networks after all computations, it will give you the ol output or in fact, y predicted. So in mathematically how does it look and H of Z looks like he raised to minus Z minus E raised to e raised to z- e raised to minus X upon e raised to Z Plus e raised to minus Z. So this is how your tanh function looks like so it is also an exponential function which is in terms of Z. So now what is the peculiarity of tanh its function that it ranges from minus 1 to 1 so it does not give your signal into the range 0 to 1, but it can expand to minus 1 also so it will just shift this particular axis.
So if you want the geometrical interpretation to say this is minus Z and this is positive. Of Z so it will have a bend at zero so it will look something like this so it attains – one at this particular point so we’re and of minus infinity is negative 1 and tan of positive Infinity is 1 and this is your X is so for this particular tanh function you have this geometrical interpretation. Now this again, you can see this is this fits into the criteria of nonlinear. This is nonlinear and also it is differentiable. So since we have said it is differentiable we can see how it is seen.
So if we want to differentiate this with respect to z so the ol is differentiated in this way. So you have e raised to z- e raise to minus it upon e raised is 2 Z 2 plus e raise to minus z, so we again use u by v rule that we have seen in the previous video for a sigmoid function. So that is simple so You take the denominator. So you have denominator Square. So first you have denominator, then you take the derivative of numerator. So if you take the derivative of numerator for e raised to 0 it is e raised to set then for arrays to – it you have minus E raise to minus a but you have a negative sign in front of that so it becomes positive.
So that is nothing but e raise to 0 The series 2 – it – you have numerator you take the derivative of the denominator. So but here the sign changes since you have a positive sign so here is to z- e raise to minus it. So well now if you closely inspect the numerator, so this is a two-product. So we have here is 2z + e raise to minus z square minus E raise to z- e raise to minus Z Square upon e raise to Z Plus e raise to minus Z Square. Now, what I basically do is I’ll just split the denominator so it becomes A born here is to set plus e raise to minus Z Square minus E raise to Z minus E raise to minus z square upon e raised to Z minus a raise to Z Square this to just cancels out so that becomes 1 so you have one – now this particular portion.
I can write it as e raised to z- e raise to minus Z upon erased. Say + e raise to minus z whole Square. This is nothing but your tan H so I can write the this as 1 minus tan H Square h of Z especially so that is nothing but 1 minus here output Square. So this is differentiative of your tanh function. So if you want to see the geometrical interpretation, so how it basically looks like is you have this particular scale minus Z 2 Z you have one then you have 0.5 and then you have your minus 1. So this will form a curve something like this.
Now again, if you see the board extremities, it will attain 0 at some particular point or at some particular interval. So it is also a victim of your Vanishing gradient. Did you again cannot solve but it has one important property like you can scale your output in the range of minus 1 to 1. Now, you can see the particular sinh and cosh functions.
This is initially came from the Euler’s constant. That is nothing but Euler’s constant. So again, some people call it as Euler’s constant. I call it as Euler’s constant because this is a scientist from Germany. So here we pronounce it as Euler for EU. So that is a different story. So we’ll Euler’s constant or Euler’s constant is given for these two functions. So if you want to see how this sign is written is so I’ll just write this here. So basically what you yield is you can have two parts that are one part is the real part.
And another part is I imaginary part. So if you have sine h of Z that is given as e raise to I is e minus E raise to minus. iz upon 2 y + 4 cos H. Is it you have erased 2i Z minus. This is Plus. – iz upon 2 For now, when you take the ratio of these two sinhz upon coshz exit you can achieve that is tanhz=(zi)= – i tanh(iz). So basically if you see the Z part is your real part so that you mainly see here that it is across our functions. We mainly use our real part but in mathematical interpretation, you have an imaginary part as well. So if you want to visualize this particular thing that is the real part and imaginary part in three dimensions.
So in two Dimensions, we saw that is this particular function. So that was with respect to our Z. So that is same as our normal sigmoid interpretation. So you have two small mountains, which is narrow in two Dimension. Now, this is for Z that is the real part. But now what you have is you have any imaginary part as well? So It has a much larger Mountain which looks something like this. So it is very much broader from the top which is compared to your normal other small function. So this is a real part. So but for computation, we don’t mainly use our imaginary part.
We just go with the real part and that is sufficient for our calculation. So well, that was all regarding the tanh function in activation functions for deep learning.
reference – Hyperbolic Tangent Activation Function
Total creator. General coffe buff. Award-winning internet trailblazer. Devoted tv practitioner. Gamer. Communicator. Travel fan. AI and machine learning are everyday!