Introduce non-linearity

Enhance computational efficiency

Guarantee vanishing gradient

• Eliminate overfitting

Sine

Cosine

Hyperbolic Tangent (tanh)

Hyperbolic Secant (sech)

Adjust weights to minimize errors

Control learning rate

Prevent overfitting

Guarantee non-linearity

Sigmoid

Hyperbolic Tangent (tanh)

Rectified Linear Unit

Softmax

Adversarial training, by adjusting weights based on the gradient of the loss function.

Genetic algorithms, by evolving a population of networks over generations.

Gradient Descent, by iteratively updating weights in the direction of steepest descent of the loss function.

Reinforcement learning, by using reward signals to adjust weights.

The rule for adjusting weights based on the magnitude of the input features.

A strategy for handling imbalanced datasets during training.

The iterative process of updating weights to minimize errors and improve model accuracy.

A method for introducing randomness in weight adjustments to prevent overfitting.