Portal de Conferências da UFSC, XX Sitraer

Tamanho da fonte: 
A REINFORCEMENT LEARNING APPROACH FOR OPTIMIZING THE RUNWAY CAPACITY UTILIZATION UNDER UNCERTAINTY
Lucas Orbolato Carvalho, Mayara Condé Rocha Murça, Marcelo Xavier Guterres

Última alteração: 2023-09-04

Resumo


In the air traffic management (ATM) field, uncertainty is everywhere, since air transport activities are subject to intrinsically uncertain parameters, like passengers' demand for flights, operational instabilities and capacity constraints. However, it is not always noticed or properly considered when dealt with. Hence, although many studies have tried to include this factor in the problems' formulations, there is not an ultimate and unquestionable approach to do so in most, if not all, of the cases. Thus, this study brings a new way to introduce uncertainty in a primary ATM problem: runway capacity utilization. Even though this subject was widely explored over the years, the prevailing methods are still deterministic, and those who tried a stochastic formulation focused on trying to model the problem's behavior and solving it with dynamic programming. Nonetheless, this way of facing it is subject to high dimensionality and modeling limitations. Both issues can be eliminated with direct reinforcement learning methods. This tool is able to learn by experience within uncertain and unknown environments. So, a Q-Learning tabular method with Eligibility Traces and a decaying-epsilon-greedy value-based policy is employed to solve the problem for a given runway configuration with two capacity envelopes for different weather conditions. Dynamic storage of states and actions is also proposed to reduce the problem's dimensionality. With this framework, the agent could learn the optimal policy fast and without trouble, allowing the air traffic managers to define the best actions in advance in a real situation. Unlike linear and dynamic programming methods, another important upside of this approach is its flexibility, making it possible to easily change the environment or the reward function. Future improvements can be made by introducing regression models to generalize the learning process and expanding the problem to different runway configurations.

Referências


Dell’Olmo, P. & Lulli, G. (2003). A dynamic programming approach for the airport capacity allocation problem, IMA Journal of Management Mathematics 14(3), 235–249.

Gilbo, E. (1993). Airport capacity: representation, estimation, optimization, IEEE Transactions on Control Systems Technology 1(3), 144–154.

Gilbo, E. (1997). Optimizing airport capacity utilization in air traffic flow management subject to constraints at arrival and departure fixes, IEEE Transactions on Control Systems Technology 5(5), 490–503.

Gilbo, E. & Howard, K. W. (2000). Collaborative optimization of airport arrival and departure traffic flow management strategies for cdm Anais do Proceedings[...], 3rd USA/Europe ATM Seminar FAA and EUROCONTROL Online.

Hall, W. D. (1999). Efficient capacity allocation in a collaborative air transportation system PhD thesis, Massachussets Insitute of Technology (MIT).

Jacquillat, A. & Odoni, A. R. (2015). An integrated scheduling and operations approach to airport congestion mitigation, Operations Research 63(6), 1390–1410.

Jacquillat, A. & Odoni, A. R. (2018). A roadmap toward airport demand and capacity management, Transportation Research Part A: Policy and Practice 114, 168–185.

Jacquillat, A., Odoni, A. R. & Webster, M. D. (2016). Dynamic control of runway configurations and of arrival and departure service rates at JFK airport under stochastic queue conditions, Transportation Science 51(1), 155–176.

Shone, R., Glazebrook, K. & Zografos, K. G. (2019). Resource allocation in congested queueing systems with time-varying demand: An application to airport operations, European Journal of Operational Research 276(2), 566–581.

Shone, R., Glazebrook, K. & Zografos, K. G. (2021). Applications of stochastic modeling in air traffic management: Methods, challenges and opportunities for solving air traffic problems under uncertainty, European Journal of Operational Research 292(1), 1– 26.

Sutton, R. S. & Barto, A. G. (2018). Reinforcement Learning: An Introduction 2nd edition, MIT Press, Cambridge.

Teodorovic, D., Trani, A., Kane, A. & Baik, H. (2004). Fuzzy mathematical programming model for optimizing airport capacity utilization Anais do Proceedings[...], Proceedings of the Triennial Symposium on Transportation Analysis, TRISTAN V French West Indies Le Gosier.


Um cadastro no sistema é obrigatório para visualizar os documentos. Clique aqui para criar um cadastro.