Comparative Analysis of different Activation Function for Pattern Classification Problem

Abstract

The main objective of this paper is to investigate the appropriate activation function to improve the overall performance for deep neural networks. The great success of deep neural networks depends on several aspects in which the development of Activation Function is one of the most important elements. The comprehensive summary of various AFs has been discussed and compared by applying it for both two-class classification (iris dataset) and multiclass classification (breast cancer dataset). Furthermore, the advantages and disadvantages of a few activation functions are discussed in this paper.