Activation Function: Sigmoid Activation Function: * Widely used as it resembles probability with a value between 0 and 1. tf.math.sigmoid(z) g(z) = 1/(1+e^-z) Hyperbolic Tangent: tf.math.tanh(z) g(z) = (e^z - e^-z)/(e^z+e^-z) ReLU (Rectified Linear Unit) * Piecewise non linearity with a single non-linearity tf.nn.relu(z) g(z) = max(0,z) Whats the importance of an activation function? * Introduce non linearity