Internships and Master Projects
I am happy to give internships (paid) and Master projects (min. 6 months, preferably mathematicians with very good skills in CS) to motivated non-HEIG-Vd students. If you do want to do an internship or a Master project with me, please send me a resume with your academic records and work or project experience, and explain me your motivation.
Data mining for emergencies
Within the framework of a new European Interreg project, our research team has been entrusted with the mission of setting up a new artificial intelligence software for medical decision support that will allow the regulation centers (144 for example) to adapt « very quickly » to each situation. The trainee will be integrated in a project that consists in finding patterns in our dataset (emergencies recorded at the hospital). Thus, we will try to answer the following questions: Which parameter has an impact on the number of emergencies? Which parameter has an impact on the number of P1/P2/P3 emergencies? Which parameters have an impact on the emergence of risk states (which still need to be rigorously defined)? More specifically, we would like to find and quantify the influence of certain parameters. This problem will be very valuable for the following aspects: 1. For example, it is obvious that heavy traffic conditions have an influence on the number of incidents. But to what extent? Moreover, if we find an unexpected pattern, it could be very useful for the Emergency Medical Services (EMS), because one has to take into account these parameters for the development of the software. 2. This information could also be very useful for a feature selection (it is possible that our model would be better using a well-chosen feature selection). But to do so, we need to choose the most relevant model. 3. To make our predictions, we make extensive use of deep learning. However, the black box nature of this approach raises the question of the reliability of our predictions. If we are able to detect certain factors that affect the number of incidents, we could test whether changing this parameter in the input data changes the output as expected. Such behavior will be desired, as it will suggest that our model has learned a relevant pattern. 4. Finally, it is worth mentioning that more than one person could work on this problem. For example, one person could work on finding the parameters that impact the number of incidents and another could work on the emergences of risky states.
Management of real data
Within the framework of a new European Interreg project, our research team has been entrusted with the mission of setting up a new artificial intelligence software for medical decision support that will allow the regulation centers (144 for example) to adapt « very quickly » to each situation. The trainee will be integrated in a project where we work with real data. Although this project is really exciting, it also comes with some challenges. In particular, how do we deal with missing data? For example, in the data provided by MeteoSuisse, sometimes a sensor does not work for a while. Obviously, we could replace it with the latest measured data, but since a sensor can be down for a long time, this does not seem to be a good approach. So how do we deal with this missing data? Could we generate synthetic data to replace this missing data? What is the best strategy? The project will consist to set up a clean dataset which will be very useful for us.
The intern will be integrated into a reinforcement learning (RL) project, in which we choose to attack the real time allocation problem using an offline risk averse RL approach, which is a special case of safe RL. We could use other reinforcement learning techniques to solve our problem. In particular, we found a very interesting paper (Garrett Thomas, Yuping Luo, and Tengyu Ma. Safe reinforcement learning by imagining the near future. Advances in Neural Information Processing Systems, 34, 2021). In this paper, the author presents SMBPO, an algorithm that penalizes unsafe paths. With the proposed approach, they are able to train a policy that avoids risky states. We could try to implement this algorithm in our real-time resource allocation problem. However there will be some challenges: how to define a risky state? How to detect them in our context? And then there is always the same problem. SMBPO works well with a small representation of the environment, but how can we extend this algorithm to handle high dimensional states? The aim of the project is to answer these questions
Safe imitation learning
Within the framework of a new European Interreg project, our research team has been entrusted with the mission of setting up a new artificial intelligence software for medical decision support that will allow the regulation centers (144 for example) to adapt « very quickly » to each situation. If we have a dataset of experts, we can use a truly natural approach and train a model to imitate the expert’s behavior. This approach is called imitation learning or behavior cloning. However, in general, imitation learning does not care about safety. For our problem, safety is really important. Therefore, we need to train our imitation agent with some safety parameters in mind. This approach is called safe imitation learning.
Convergence of Markov Chains to self-similar Processes
Self-similar processes are stochastic processes that are invariant in distribution under suitable scaling of time and space. These processes can be used to model many space-time scaling random phenomena that can be observed in physics, biology and other fields. One could mention stellar fragments, growth and genealogy of populations, option pricing in finance, various areas of image processing, climatology, environmental science, . . . Self-similar processes appear in various parts of probability theory, such as in Lévy processes, branching processes, statistical physics, fragmentation theory, random fields, … Some well known examples are: stable Lévy process, fractional Brownian motion,. . . but it has also been shown that relatively simple Markov models can produce self-similarity. Even though the cardinality of the state space increases to infinity, it has also been shown that its rate is quite low. The aim of the study is to prove how theses Markov models converge to self-similar limit processes, under which conditions.
Small Datasets Reinforcement Learning for Emergencies
The intern will be incorporated in a project, which aims to find optimal solutions to guide and involve the most appropriate vector (helicopter, ambulance) in the face of an emergency, accident or disaster. For moderately severe cases, it is often preferable to use a vector that is not the closest, but the one which minimizes the redistribution of the vectors on the territory. We believe that machine learning and more precisely reinforcement learning could provide an interesting framework to solve this problem. Unfortunately, reinforcement learning needs millions of samples to achieve good performance and we only have access to a few hundreds of thousands. The intern will explore methods, which aims to use reinforcement learning with a (reasonably) small dataset. The precise project will be determined with the intern according to his background and ideas.