Abstract
Virtual reality (VR) is a rapidly growing technology that catapults our 2 dimensional interactions with technology into 3 dimensional immersive experiences. VR is applied across many domains including digital economies, entertainment, gaming, treatment and training in healthcare, aerospace, defence, disaster safety and emergency procedures, driving, education and design. Unfortunately, an emerging issue is that VR immersion leads to a type of virtual motion sickness known as Cybersickness (CS). Just like traditional motion sickness (MS), CS comes with a cluster of symptoms related to disorientation and nausea with added oculomotor problems. Theories of its pathogenesis involve brain and body reactions to virtual environments. These involve a mismatch of different sensory perceptions, as well as susceptibility to CS due to postural instability. In line with the theory of sensory conflict, VR developers have opted to change the way virtual content is displayed to make VR environments more ergonomic. This may work for some use cases like entertainment, but falls short where content cannot be altered in realistic simulations and training operations. Thus, the solution must be two-fold, not just in alteration of environmental factors but in human physiological adaptation.
The first stage in building a solution for CS is in the identification of its associated physiological correlates or ‘biomarkers’. Researchers have been studying ways to automate the tracking of these biomarkers. The reason for the automated approach is to develop timely warning systems that allow for intervention, rather than for retrospective analysis which is conducted well after the event. To do this, machine learning is used as a diagnostic tool to classify ‘on’ and ‘off’ disease states and their severity. VR technology is a recent adoption by consumers, and as such this form of sickness is a relatively new occurrence and literature on this topic remains scarce. There is no standardization across studies on how to classify CS. In addition, beyond subjective questionnaires evaluating previous MS and CS, literature still lacks automated and objective predictive tools for CS susceptibility. This means that without subjective input, it has so far not been possible to do an on-the-spot assessment of how a person might feel in VR. Additionally, while alleviatory treatments for MS are available in the form of drugs and acupoint stimulation, treatments specifically for CS still remain lacking. Specifically, literature lacks preventative treatments for CS, which is of priority over discovery of alleviatory treatments that counteract symptoms after emergence.
The overall aim of this thesis was to develop interpretable machine learning models that use biosignal data to both predict CS before VR usage, and detect CS during the event. In addition, we sought to use the information generated by the interpretable models to guide clinical intervention.
A machine learning model was built based on a modified NeuCube spiking neural network architecture, making it in essence a neuromorphic data processing software. It models and learns CS related spatiotemporal brain data such as electroencephalogram (EEG), and thus provides brain templates specific to CS that reveal important cortical areas for classification. The final model predicts (85.9%) CS prior to VR usage and detects (76.6%) CS events during VR immersion to a relatively high accuracy, whilst only needing one to two EEG channel data. It was also found that electrocardiogram (ECG) data in the form of heart rate variability parameters (HRV) could predict (74.2%) and detect (72.6%) CS, although fusion of these features with EEG features did not lead to significantly improved accuracies and in some cases worsened them. In addition, no significant HRV differences were found between cybersick and control groups, meaning that a treatment assessed by HRV changes would not be viable. Focusing on EEG data, we used the most important feature revealed by the model (Cz) to define the target cortical area of clinical intervention. This area, involved in motor planning and processing, was hypothesised to be a key region acting as a hub in CS brain networks pre-VR usage and during the CS event. It was also hypothesized that modulating activity in this area would disrupt the functional CS network and therefore the emergent property of the feeling of CS. It was found that application of cathodal high definition transcranial direct current stimulation (HD-TDCS) centred on Cz (with FC1, FC2, CP1, CP2 surrounding electrodes) significantly reduced the risk of CS compared to placebo and anodal (opposite polarity) stimulation, with 47% of cases experiencing very light to no CS after 10 minutes of VR immersion. The results provided evidence for acute prevention of CS via neuromodulation. In addition, this solidified the usefulness of the SNN model to reveal key information about the spatiotemporal brain dynamics of CS. Further on, this thesis describes the utilization of New Zealand eScience Infrastructure (NeSI) high-capacity computing (i.e. a super computer) to improve and optimize the model. Instead of relying on one snapshot in time of EEG data to define important cortical areas, as was done previously to build the model, an exhaustive analysis was conducted with multiple new models. This analysis trained multiple different models on different time segments and data lengths to find the best combination of features and parameters for classification. This new analysis revealed that the same channels could be used to predict and provide preventative treatment for CS using HD-TDCS. It also revealed the importance of the cortical region (F7) involved in processing incongruent information in the environment, which could be a potential new target for intervention in future experiments.
Overall, the thesis provides a stepping stone towards further research into neuromorphic computing for the prediction and detection of CS. It also opens up the field to the generation of new knowledge about CS using interpretable machine learning models, which will help guide potential new treatments for CS. From a translatable aspect, this research opens the possibility for wearable devices to use just one or two EEG channels to collect, classify, update, learn and reveal new information on the pathophysiology of CS all in the same framework.