Methods for bridging reality gaps

Expected Impact:
The outcomes should contribute to:
Reduce dependencies on non-European suppliers by boosting the EDTIB and promoting the development of a European solution.
Faster and better planning and decision making (with less personnel) during mission planning and execution, resulting in higher mission success.
Leverage Reinforcement Learning towards largely automating the modelling and implementation of expert-level (or beyond) competent battlespace agents, thereby greatly reducing the time and cost of course of action (COA) development and wargaming.
Deliver a proof-of-concept demonstrator at least of TRL 5.
Increase the opportunities for various smaller actors, including those not previously active in the defence sector, to adapt and apply innovative simulation technologies for defence applications.
Increase business opportunities in the defence sector for EU and Associated Countries companies and promote technological edge in the field.
Increasing the interoperability between EU armed forces and with NATO Allies.
Increase opportunities and future involvement for third parties participating in FSTP in the field of simulation and training within tasks described previously in the call text under “Conditions related to FSTP”.
Objective:
Mission planning and execution in the present and future multi-domain operation environment (MDO) employing manned and unmanned force elements demand that the human decision makers are very well supported to be able to handle the complexity and dynamics of the battlespace and make decisions faster and better than the adversary.
In mission planning different types of operational capabilities need to be carefully coordinated in time and space to achieve mission goals and counter expected threats. Labour intensive manual planning is infeasible within the constraints of available time and resources. The general objective is to develop advanced automated support tools for the generation and evaluation of courses of action (COAs) in an MDO context. The toolset is expected to support wargaming of the candidate COAs to ensure that commanders and staff can assess the plan and options in detail before final decision making.
Specific objective
This call aims to explore technologies, concepts, products, processes and services towards a common simulation framework for wargames/combat simulations with the potential to facilitate reinforcement learning for mission planning and execution support.
Re-planning and decision-making during mission execution are likely to be challenged in the interconnected, manned-unmanned, automated and high-speed battlespace. In the future, the clear distinction between mission planning and execution is expected to be challenged by exploiting battlespace information and predictive capabilities. Proper support is needed to speed up the OODA-loop to outpace the adversary in the planning phase as well as in the execution phase.
The development and use of a computer-based decision support system that leverages AI, machine learning, wargames/combat simulations and digital twins of the battlespace has the potential to change the military planning and decision-making concept of operations (CONOPS).
Reinforcement Learning (RL) in Artificial Intelligence (AI) has shown a huge potential for solving planning problems in civilian applications. However, despite its headline success in video games, strategy games and other planning domains over the last few years, RL is not making similar progresses in the realm of wargames/combat simulations for military operations planning. Videogames leave a lot of margin when it comes to critical (life or death) simulation. Nevertheless, if access to classified data from the field is not possible, videogames data may be used for a proof of concept.
Simulation frameworks tailored to particular domains have played a major role in facilitating reinforcement learning in those domains, as witnessed by the impact of e.g., OpenAI Gym and the Arcade Learning Environment (ALE).
A common simulation framework for wargames/combat simulations has the potential of similarly facilitating reinforcement learning–support in mission planning and execution.
As it is related to EUDIS, this topic aims to support, in addition to the research activities, the creation of an innovation test hub in the field of simulation and training. To achieve this objective, financial support to third parties (cascade funding) (FSTP) is included as part of the grant. This should increase the opportunities for various smaller actors, including those not previously active in the defence sector, to adapt innovative simulation technologies for defence applications and to identify potential business opportunities in the defence sector.
Scope:
Proposals must address studies and design of a reinforcement learning environment/testbed or framework for training of AI agents to develop courses of actions in mission planning, including a flexible and open combat simulation framework fit for RL. It must address the need for rapid and user-friendly creation of scenarios, considering commander’s objectives and intent, rules of engagement and other mission constraints (e.g., speed, resources, attrition). It must also include studies and design of a combat simulation system (not necessarily the same used for AI agent training) including trained AI agents to support mission planning. For the support to mission execution the scope includes studies and design of a digital twin of the ongoing mission for prediction and decision-making support. The proposal must establish a proof-of-concept demonstrator for verification, validation and demonstration.
The learning environment, including the combat simulation framework must be flexible and adaptive for different scenarios and domains. It must take advantage of open standards and open-source frameworks both within AI, simulation technologies (including C2-Simulation interoperability) and mission sensor and mission data to th