Solving the 2-level atom non-LTE problem using soft actor-critic reinforcement learning
Loading...
Files
Authors
Author (Corporation)
Publication date
2026
Type of student thesis
Course of study
Collections
Type
01A - Journal article
Editors
Editor (Corporation)
Supervisor
Parent work
RAS Techniques and Instruments
Special issue
DOI of the original publication
Link
Series
Series number
Volume
5
Issue / Number
Pages / Duration
Patent number
Publisher / Publishing institution
Oxford University Press
Place of publication / Event location
Edition
Version
Programming language
Assignee
Practice partner / Client
Abstract
We present a novel reinforcement learning (RL) approach for solving the classical 2-level atom non-LTE radiative transfer problem by framing it as a control task in which an RL agent learns a depth-dependent source function S(tau) that self-consistently satisfies the equation of statistical equilibrium (SE). The agent’s policy is optimized entirely via reward-based interactions with a radiative transfer engine, without explicit knowledge of the ground truth. This method bypasses the need for constructing approximate lambda operators (Lambda^*) common in accelerated iterative schemes. Additionally, it requires no extensive precomputed labelled data sets to extract a supervisory signal, and avoids backpropagating gradients through the complex RT solver itself. Finally, we show through experiment that a simple feedforward neural network trained greedily cannot solve for SE, possibly due to the moving target nature of the problem. Our Lambda^*-Free method offers potential advantages for complex scenarios (e.g. atmospheres with enhanced velocity fields, multidimensional geometries, or complex microphysics) where Lambda^* construction or solver differentiability is challenging. Additionally, the agent can be incentivized to find more efficient policies by manipulating the discount factor, leading to a reprioritization of immediate rewards. If demonstrated to generalize past its training data, this RL framework could serve as an alternative or accelerated formalism to achieve SE. To the best of our knowledge, this study represents the first application of reinforcement learning in solar physics that directly solves for a fundamental physical constraint.
Keywords
Event
Exhibition start date
Exhibition end date
Conference start date
Conference end date
Date of the last check
ISBN
ISSN
2752-8200
Language
English
Created during FHNW affiliation
Yes
Strategic action fields FHNW
Publication status
Published
Review
Peer review of the complete publication
Open access category
Gold
License
Citation
Panos, B., & Milić, I. (2026). Solving the 2-level atom non-LTE problem using soft actor-critic reinforcement learning. RAS Techniques and Instruments, 5. https://doi.org/10.1093/rasti/rzag005