Finding optimal hyperparameters of feedforward neural networks for solving 2D Laplace's equation using a genetic algorithm

Abstract

In this work, feedforward neural networks were used to solve 2D Laplace's equation on rectangular and irregular domains. Optimal values of weights and biases of a network were recursively computed during the network training by minimizing a cost function using data at collocation points to approximate the true solution. The performance of the network largely depends on network architecture and model capability. In this work, an optimal set of hyperparameters was searched on various measures of relative errors. The genetic algorithm was used to find the optimal activation function, optimization algorithm, and weight initialization. In addition, it also searched for the optimal number of hidden layers when the total number of parameters is fixed. The optimal total number of parameters for a specific number of hidden layers was also searched. Numerical results showed that an optimal set of hyperparameters that are consistent across many values of relative errors could be successfully obtained.