Digital Twin Reinforcement Learning

Project Description

• Develop new AI deep reinforcement learning (DRL) approaches that will use OT/DER network and targeted physical process data to detect sophisticated, previously unknown threats and deploy appropriate response actions
• Digital twin capabilities using LLNL’s NeMS tool and automated ns-3 model instantiation will be leveraged to create high-fidelity co-simulation environment needed to train the DRL algorithms
• Before alerting the operator, DRL will take a series of escalating actions, making a decision in each step to increase the confidence that the system is under attack
• If attack is detected, DRL will take appropriate active defense and corrective actions to prevent wide-spread compromise and minimize attack impacts

Value Proposition

• Enabling advanced threat detection and mitigation from sophisticated, previously unknown threats
• Cost effective: Real time operation, with high accuracy and low false alarm rate reducing cost to the operators
• Adaptive: DRL improves over time with feedback from real data and operators
• Well suited for complex environments, such as DER networks
• Increases resiliency: by deploying active defense and corrective actions minimizes impacts of cyber attacks and increases the cost to the adversary

Project Objectives

• Develop and implement new AI DRL to detect new, advanced cyber threats
• Use digital twin technology to train DRL algorithms
• Identify and implement active defense and corrective actions
• Live demonstration at Plum Island to enable and facilitate technology transition