Using deep reinforcement learning with smooth constraint to invert magnetotelluric data

Deep Q-network (DQN) is a classical deep reinforcement learning algorithm, which is an unsupervised inversion approach that has recently been applied to invert one-dimensional magnetotelluric data. It has the advantages of not requiring a training dataset, being less dependent on the initial model, and being able to obtain the probability distribution of inversion results through multiple inversions. However, it suffers from the issue that the inversion results are not concentrated.

To address this issue, this paper proposes the use of deep reinforcement learning with a smooth constraint to invert magnetotelluric data (Smooth DQN, SDQN). This method is based on the framework of reinforcement learning, considers the inversion problem as a Markov decision problem, and defines the terms environment, reward, agent, and so on. Then, the model constraint term of regularized inversion is introduced into the reward function, guiding the agent to continuously adjust the resistivity parameters of the prediction model to obtain results that are more consistent with the model constraints.

The experimental results of the theoretical model inversion show that, compared with the DQN inversion and Occam inversion methods, the results of the proposed method are more stable when the observed data are inverted with the same number of iterations and different noise levels. The inversion results of the magnetotelluric measured data in the Tashi Kang Mine area of Tibet are largely consistent with the Occam inversion results and align with the existing geological interpretations.

The experimental results show that this method has the advantages of more concentrated inversion results and stronger anti-noise capability for the observed data, and it is a new tool for solving the problem of magnetotelluric inversion.