Personal Identification in Virtual and Augmented Reality: Control Systems Perspective

Abstract

Virtual and Augmented reality is a rapidly growing, thriving domain. While a lot of benefits come from using VR/AR, involved technology poses significant privacy and security risks. Even biometric data collected during trivial experiments, where subjects perform repetitive tasks, might be used for unique human identification, which makes it a candidate for ethical and legal discussions. The research question of this thesis is if we can consider data collected during trivial VR experiences as personal data: this thesis strives to establish if personal identification in a Virtual Reality environment using data collected during trivial experiments is possible and thus state if such data is sensitive. In other words, if it can be used to identify a human being.

Even though a number of researches have been conducted on the topic of human identification in VR/AR, experiments were designed to enhance the possibility of such identification. On the contrary, this work is dealing with natural, real-life behaviour of subjects interacting with so called digital twins of control objects control systems in Virtual Reality in the area of automatic control systems design.

Research was conducted in such stages: experiment preparation, experiment and data collection, data pre-processing, data analysis, gathering and presenting the results. 37 subjects took part in the experiment, out of which: 17 identify as female, 20 identify as male; 33 are right-handed, 4 are left-handed. Approximately 40% of the participants had experience in VR/AR in the past. Hypothesis is the thesis is that data collected during a trivial experiment can show solid identification accuracy among a group of participants. This thesis compares results given by five machine learning classifiers and 40 different variations of features and metrics, achieving the accuracy of 75% success rate of the identification for the group of 32 people.

Project results

During the development of this student project, an experiment was conducted that allowed to collect the necessary data to publish several research papers on a related topic.

The thesis was also a runner-up at the School of IT Innovation Festival 2021.

Related hardware

Related publications

• S. Alsaleh, A. Tepljakov, M. Tamre, and E. Petlenkov, “Towards Artificial Intelligence Driven Immersive Environments in Virtual Reality for Industrial Applications,” in 2021 44th International Conference on Telecommunications and Signal Processing (TSP). IEEE, 2021, pp. 340–345. [DOI] [Bib]

Project videos