The objective of ATFM is to balance in space and time air traffic demand with available capacity and services, providing feasible levels of workload to the ATC system. Then, not only safety is insured by limiting the occurrence and the complexity of traffic conflicts, but also original flight efficiency is sustained by avoiding large deviations from flight plans leading to the use of uneconomic flight levels, in-flight holdings and other traffic congestion induced effects.

ATFM activities include medium term strategic planning where traffic flows and ATC capacity are managed, pre-tactical planning where flights are eventually rescheduled and rerouted and when tactical decisions such as ground delay departures are pre-established, and finally, on the day of implementation, in a close interaction with ATC, the monitoring of air traffic.

This study considers mainly the pre-tactical planning level where the global balance of traffic flows is already fixed by the strategic level (flows structures and levels, ATC sectors and capacity) and where flights are treated on an individual basis in the space between the different departure/arrival airports. Then for each flight, decisions such as ground delayed departure (resulting in flight rescheduling) and/or rerouting may be taken to cope with predicted traffic congestion situations. An explicit connection of the pre-tactical decision framework developed in this study with ATFM tactical and ATC operations is realized through the adoption of a common set of intrinsic criticality measures.

In the last decades, exact and approximate mathematical programming approaches have been developed to tackle this problem, where the optimization objective is in general to minimize the total delay incurred by the considered air flows under flights separation and sectors capacity constraints. This has led to large scale optimization problems where computing time is a limiting issue. More recently, Artificial Intelligence techniques such as Genetic Algorithms, Machine Learning, Reinforcement Learning and Deep Reinforcement Learning have been considered to contribute to the solution of ATFM problems.

In this study, the adopted objective is the global reduction of interactions between flights while maintaining the original efficiency of flight plans computed by airlines taking into account aircraft performances and short-term weather forecasts. For that, a multidirectional flight centric criticality index is adopted to assess the degree of interaction between flights and standing as a possible common driver for ATFM and ATC actions.  A dynamic traffic connectivity graph built from foreseen criticality levels along the flight plans allows to define connectivity components composed of interacting aircraft at a given time.

A flight scheduling policy based on the global degree of asymmetry of the criticality indexes of the flights is designed to process the whole set of considered flights. Then, for each flight during congestion periods, the need to apply a ground delayed departure is assessed based on the level of asymmetry of the longitudinal criticality indexes. When considered necessary for a flight to perform safely the take-off and climb phases towards enroute level, a ground delay at departure is established using a simple proportional rule.

Then, local directional deviation procedures are computed when the criticality level exceeds a given threshold at waypoints. Here it has been supposed that there is an immediate rejoinder to the next waypoint of the flight plan having a low criticality level and that the original airspeed between two waypoints is maintained constant. Local directional deviations are also computed here using simple proportional rules applied to the criticality directional indexes.

The parameters of the associated ATFM procedures (ground delays at departure, local deviations) are tuned using Reinforcement Learning techniques which ally some basic principles optimization in Mathematical Programming with the computation power of IA techniques allowing to overcome nonlinearities of models as well as discontinuity of data. The proposed solution approach is qualified of ‘structured” since it avoids the Black Box image of many IA techniques which turn difficult validation efforts, and it improves its scalability.

The paper is composed of the following sections after the introduction: selection of a set of multidirectional flight centric criticality indexes, definition of an initial scheduling procedure for planned flights, design of ground delayed departure and local deviation decision procedures, tuning of the procedure parameters using Reinforcement Learning and illustration of the proposed approach through different case studies. Finally in the conclusion, the strengths and weaknesses of the proposed approach are discussed for further improvement as well as its connection and possible integration with the ATC procedures with the perspective of building a digital assistant to Air Traffic Controllers (ATCOs), Flight Management Positions (FMPs) and Network Manager (NM).

Keywords: Air Traffic Flow Management, Traffic Criticality Measures, Reinforcement Learning, Flight Scheduling.