Quantum reinforcement learning
Quantum Reinforcement Learning (QRL) is an interdisciplinary field that integrates the principles of quantum computing (superposition, entanglement, and interference) with classical reinforcement learning (RL). The primary goal is to leverage quantum mechanics to accelerate the training, enhance the performance, or increase the complexity-handling capability of an RL agent in sequential decision-making tasks.
How QRL Works
Traditional reinforcement learning involves an agent operating in a state, getting rewarded (or punished), and then changing states. The agent seeks the optimum policy or plan to optimize its cumulative reward.
QRL radically alters the computation of the RL algorithm’s essential parts:
Quantum State Encoding: Qubits are used to encode the agent’s policy parameters or the environment’s classical state into a quantum state. Quantum parallelism is made possible by the simultaneous representation of 2^n classical states in a superposition by an n-qubit system.
Quantum Processing: The fundamental function of the agent (such as the Q-function or the policy) is modeled using a Variational Quantum Circuit (VQC) or Parametrized Quantum Circuit (PQC) in place of a traditional neural network. A sequence of quantum gates with adjustable parameters makes up this circuit.
Quantum Operations: Unitary operations, also known as quantum gates, can be used to describe the agent’s choice of action and the state transition of the environment. This could enable effective exploration of the state-activity space.
Measurement and Update: By measuring the quantum state, the superposition is collapsed into a classical output, such as the probability of performing an action or the Q-value for a particular action. The settings of the PQC are then updated using this classical result in a classical optimization cycle.
Types of QRL Implementations
Generally speaking, QRL techniques are divided into groups according to the level of quantum involvement:
Also Read About Quantum Circuit Complexity Reveals Hidden Quantum Phases
The NISQ Approach to Hybrid Quantum-Classical
QRL for modern Noisy Intermediate-Scale Quantum (NISQ) computers is the most often used method.
Mechanism: The primary RL control loop, which includes gathering experience, figuring out loss, and adjusting parameters, remains traditional. Only the neural network’s basic function approximator is substituted with a tiny, parametrized quantum circuit (VQC/PQC).
Examples:
- Quantum Deep Q-Network (QDQN), which employs a VQC as the approximator for the Q-function.
- A VQC is used by the Quantum Policy Gradient (QPG) to directly reflect the policy.
- Quantum Advantage Actor-Critic (QA2C): Makes use of VQCs for the critic (value function) or actor (policy) networks.
Fully Quantum QRL (Theoretical)
The goal of this method is to apply the whole Markov Decision Process (MDP) to the quantum realm.
Mechanism: Quantum operations are used to encode and process all states, actions, rewards, and transition probabilities. is frequently used to determine optimal policies or Q-values tenfold faster than traditional approaches by utilizing strong, fault-tolerant quantum algorithms such as Grover’s search or Quantum Amplitude Estimation (QAE).
Example: To determine the ideal order of states and actions, use Grover’s technique for an optimal trajectory search.
Advantages and Disadvantages
| Feature | Advantages of QRL | Disadvantages of QRL |
| Speed/Efficiency | Quantum Speedup: Potential for exponential or polynomial speedup in specific subroutines (e.g., using Grover’s search for optimal actions). | Lack of Hardware: Fully quantum algorithms require a large-scale, fault-tolerant quantum computer, which is not yet available. |
| Complexity | Efficient State Encoding: Qubits can encode an exponentially larger state space than classical bits (2^n vs n), potentially addressing the “curse of dimensionality” in high-dimensional problems. | Data Encoding: Efficiently mapping a high-dimensional classical state (e.g., an image) into a quantum state is a significant, unsolved challenge. |
| Training | Enhanced Exploration: Quantum superposition allows the agent to explore multiple paths/actions simultaneously, potentially leading to faster discovery of the optimal policy. | Noise and Decoherence: Current NISQ devices are extremely noisy, which can destroy the delicate quantum states, making it hard to train models accurately. |
| Optimization | Parameter Efficiency: VQCs may require fewer trainable parameters than classical deep neural networks to achieve similar performance. | Barren Plateaus: Training VQCs can be plagued by the “barren plateau” phenomenon, where gradients vanish exponentially with the number of qubits, stalling learning. |
Also Read About NIST NCCoE Releases Draft Guidance On PQC Migration
Current Events and News (2025 Outlook)
The creation of new theoretical primitives and realistic, hybrid implementations on existing hardware have been major components of recent advances in QRL:
Entanglement-Enhanced Photonic QRL: Researchers have recently shown Quantum Optical Projective Simulation (QOPS), a useful, noise-resistant framework that makes use of single-photon entanglement to improve decision-making. Even on a noisy quantum processor simulator, this was demonstrated to converge more efficiently than classical methods on the cooperative solution in the Prisoner’s Dilemma game. This demonstrates that in situations involving complicated decision-making, entanglement can offer a real benefit.
Predictions Using Quantum Reservoir Computing (QRC): According to a 2025 partnership between Telstra and Silicon Quantum Computing (SQC), a quantum reservoir system named Watermelon might match the network performance estimates of Telstra’s deep learning models while requiring less hardware and training times. When processing time-series data, QRC takes advantage of the inherent dynamics of quantum systems, demonstrating a definite quantum advantage in sample efficiency for practical applications such as managing a telecommunications network.
Theoretical Foundations for QML: In late 2025, a major theoretical breakthrough was made when a true Quantum Bayes’ Rule was derived from a basic physical principle. This could have a big impact on creating more rigorous and effective Quantum Machine Learning algorithms, including those for sequential decision-making in QRL.
Workforce Development: A rising corporate and academic commitment to educating the next generation of professionals in QRL and associated quantum applications is indicated by the announcement of advanced certificate programs in Quantum Computing: Algorithms and AI/ML by top universities such as IIT Roorkee in late 2025.
