ESR 4

YAXIN HU

Behavioral and gaze-data analysis with deep learning neural networks

I've joined the Pattern Recognition Company in Lübeck, Germany since January 1st, 2022. And I am enrolled as a PhD student at Lübeck University. Here I am supervised by Prof. Erhardt Barth. 


Personal Background and Interest:

I have spent 22 years living in China, and I have got my Bachelor’s Degree in Electrical Engineering and Automation from School of Mechatronic Engineering and Automation, Shanghai University. Then I went to Singapore and got my Master’s Degree in Mechanical Engineering from National University of Singapore. Since then, my research interests have become computer vision and deep learning for medicine. I would like to analyze medical data and design medical toolboxes by using artificial intelligence.

 

Aim of the project:

The objectives of my project are to 1) perform a comprehensive evaluation of different deep-learning approaches for analyzing behavioral and gaze-data and, 2) develop a machine-learning toolbox that can be used to discover diagnostic or rehabilitation relevance in these data.

 

Current activities:

To date, I have used machine learning to reveal task-relevant patterns and features in the gaze-data for videos. In particular, I have designed a network architecture that can extract temporal features from gaze-data. This architecture has been applied to the Weizmann dataset with a near 100% accuracy. On the larger UCF101 dataset, an improved slicing architecture increased the accuracy from 61% to 81%. 


To further the accuracy for the UCF101 dataset, I used UniFormerV2 (a Transformer architecture) with the accuracy to 94.61%. Currently, I’m trying to improve the architecture to reach even higher accuracy.


In collaboration with ESR7 (Safa), I applied deep learning networks on VR navigation data to find some facts and features of glaucoma behavior that can improve the experiment. Once a best performing architecture is obtained, it can be used to extract new patterns and features to predict a patient’s functional vision or other diseases.


Furthermore, I have collaborated with Lübeck University. Here I developed a machine learning method to classify First-Episode Psychosis, Clinic High Risk of Psychosis and Healthy Control based on brain Magnetic Resonance Imaging (MRI) images.


I have also started my secondment in IIT, Italy where together with ESR6 (Kurt) we collected audio-spatial compression data from individuals with scotomas.


Future directions:

In the near future I’ll continue my collaboration with ESR6 and analyze the audio-spatial compression data. I’ll also continue the  collaboration with ESR7 and find the differences between the daily life behavior of health control and glaucoma patients. And I will further design the brain-MRI model to make it not only diagnose psychosis, but also predict the transition of clinical high risks and localize the most related brain region to psychosis. I will still further improve the accuracy of video recognition architecture.

Project output

No output yet.

Contact

Interested in my work and want to get in touch? Send me an e-mail to yaxin.hu@student.uni-luebeck.de

Share by: