|
Scientific Discovery and the Cost of Measurement -- Balancing Information and Cost in Reinforcement Learning
C. Bellinger, A. Drozdyuk, M. Crowley, I. Tamblyn Canadian AI (2022) In many scientific applications, measuring the state of a system is costly and time-consuming, yet standard reinforcement learning algorithms require a state measurement after each time step. This creates a significant barrier to deploying RL in real-world scientific settings such as materials design and automated chemistry. In this work, we develop a framework that incorporates measurement costs as explicit components of the reward structure, enabling standard deep RL algorithms -- including Dueling DQN and PPO agents -- to learn dual policies: one for selecting actions and another for deciding whether to measure the system state. The results demonstrate that agents trained under this framework can learn optimal action policies while making up to 50% fewer state measurements, and when augmented with recurrent neural networks, achieve a greater than 50% reduction in measurements. These substantial reductions in measurement frequency could help lower the barrier to applying RL to real-world scientific applications. |
