Prospects for use of extended reality technology for ship passenger evacuation simulation

Safety of passengers on ships is usually investigated based on data available from post-accident reports, experimental research and/or numerical modelling of emergencies. As for the numerical modelling, ship passenger evacuation falls within a greater set of pedestrian evacuation research in which extended reality (XR) technology is playing important role lately. However, XR still strives to find its place in the modelling of ship passenger evacuation. This paper brings review of literature published on the topic of XR in pedestrian evacuation with special focus on the use of these technologies (e.g. virtual reality, augmented reality) in shipping industry. Findings are put in the context of IMO’s guidelines for evacuation analysis and prospect for use of XR for ship passenger evacuation simulation are presented.


Introduction
World cruise industry witnessed a steady rise of passengers between 2009 and 2019, from roughly 17.8 million to 29.7 million (Statista, 2021). The pandemic in 2020 plummeted the numbers, but with the prospects of health situation improving it is expected that the tourist will be fleeing to cruisers once again. Cruise ships can, on average, accommodate 3,000 passengers whose safety must be taken as imperative in order to avoid fatal scenarios (Casareale et al., 2017) that can jeopardize their lives. Therefore, passenger evacuation process should be rigorously studied in the phases of ship design and exploitation.
The Maritime Safety Committee of the International Maritime Organization (IMO) issued the guidelines for evacuation analysis for new and existing passenger ships (IMO, 2016) with the method to determine the travel duration (duration it takes for all persons on board to move from where they are upon notification to the assembly stations) by simulation tools for the advanced evacuation analysis. Simulation parameters to be used within this method form four categories: geometrical, popula-tion, environmental and procedural. Geometrical category consists of layout of escape routes, their obstruction and partial unavailability, as well as initial passenger and crew distribution conditions. The population category is concerned with parameters of persons and population demographics. The environmental category includes static and dynamic conditions of the ship. The procedural category includes availability of crew members to assist in emergency.
In order to obtain useful information regarding ship evacuation process, many scientists gathered data by investigating actual accidents of various passenger ships or performing researches comprised of evacuation experiments and/or numerical modelling of evacuation.
Investigations of actual accidents on ships (Vanem and Skjong, 2006;Wang et al., 2021) prove helpful since they provide insight into real-life behaviour of crowds but, on the other hand, researcher rely solely on the post-accident available data (reports, testimonials, video footage). That means that some vital information might be missing in order to properly understand causes and consequences of accidents. Further, repeatability of the event is practically impossible.
As for the experiments, they can be performed in laboratory or field environment. Experiments in laboratories range from experiments with crowds of social insects and rodents which are used as a proxy for the movement of human crowds (Haghani, 2020a). Laboratory environment is preferred by researchers in order to ensure controlled setting and repeatability of experiments. Experimental work in the laboratories was mainly concerned with topics like single-file movements, step taking behaviour, bidirectional flows, conflict avoidance and bottleneck flows. Field experimental research on the ships is rare, due to high cost of performing such events and challenges in of ensuring repeatability of the process. Nevertheless, an experimental evacuation was performed on two cruise ships and gathered relevant data concerning evacuation simulation index (Galea et al., 2013(Galea et al., , 2014. Also, small scale experimental evacuation was performed on a passenger ship to investigate passengers' motions in the case of emergency (Gwynne et al., 2003). Further, inclining condition of the passenger was taken to study the passenger evacuation during tilting and sinking conditions (Kim, Roh and Han, 2019). As for the numerical modelling of evacuation, several models have been developed and used lately by the researchers concerned with simulations of pedestrian evacuation. These models are: cellular automata model as the most frequently used (Miyagawa and Ichinose, 2020), agent-based model (Yang et al., 2018), fluid mechanics model (Helbing, 1992), lattice gas model (Nagai, Fukamachi and Nagatani, 2005), the multi-granularity quality function deployment model (Mei, Liang and Tu, 2018), centrifugal force model (Yu et al., 2005), social force model (Helbing and Molnár, 1995).
Due to the high cost of performing field experiments on ships and limitations of numerical analysis, many researchers turned to study ship evacuation problems through blended experimental-numerical technique called extended reality (XR). Extended reality is a term referring to all possible combinations of real and virtual environments and human-machine interactions that are generated by computer software and wearable technology.
In this paper, use of XR is discussed in the context of shipping industry. This technology is already finding its application in the assessment of evacuation routes in the urban and building environment (Ding et al., 2021), but yet waits to make an impact in traditionally conservative maritime domain that includes shipbuilders and shipowners (Nasso et al., 2019). Possibilities of use of XR are given in the period of pre-accident prevention, accident response and post-accident analysis along with the discussion and conclusions.

Extended reality
As stated, extended reality fits in the concept of reality-virtuality continuum, Fig. 1. The reality-virtuality con-  (Milgram and Kishino, 1994). The 'X' in XR represents a variable for current or future spatial computing technologies and nowadays it can be substituted with an "A" standing for augmented reality (AR), "V" for virtual reality (VR) or "M" for mixed reality (MR), but it can cover also the areas between them.
Emergency management and safety research community has put considerable effort in implementing VR and AR technologies into their research procedures in recent years. Efforts are being put into making VR and AR effective and affordable, yet flexible platforms for safety scenarios training (Ronchi et al., 2015;Li et al., 2018) and useful research tools that could balance experimental control and economical validity in studies performed (Feng et al., 2021).
New generation of interactive consumer hardware and software has raised interest in VR and AR technology in recent years. Both technologies can immerse user into the virtual environment, but in a different manner. In VR, environment is fully virtual and all the objects inside of it are digitally created enabling users to fully immerse themselves into the virtual reality and not being able to perceive real surroundings (Milgram and Kishino, 1994), Fig. 2a. Virtual content is typically shown via stereoscopic displays coming in a form of a wearable hardware (e.g. head-mounted displays), Fig. 3a, large screens or wall projections (e.g. virtual rooms). Having this in mind, VR can be used in pedestrian evacuation training in fully computer-generated environment, thus enabling practically indefinite number of different emergency scenarios.
In AR, instead, just part of the content is virtual so users are able to view the real surroundings on top of which virtual elements are superimposed or blended with (Azuma, 1997) , Fig 2b. These virtual elements can come in a 2D or 3D form. 3D forms (virtual objects) are placed within a real-world environment and thus are enriching its architecture, while 2D forms (text, signs, etc.) come without any interaction with the architecture of the real world (Carmigniani et al., 2011). AR, comparing it to VR, offers a user closer experience of the real environment because it enhances the physical real-world environment rather than constructing computer-generated virtual world (Carmigniani et al., 2011). AR hardware comes in a form of wearable technology like head-mounted displays (HMD) (e.g. Google Glass, Microsoft HoloLens, Magic Leap One), Fig. 3b, but these are still seeking to make a deeper market impact among consumers. Main disadvantage is that they offer a relatively small field of view. More affordable and momentarily widespread solutions include handheld mobile video-see-through (VST) devices (e.g. smartphones, tablets) that don't have a problem of limited view field. In VST devices cameras capture live video of surroundings on top of which virtual content is added and can be monitored on screen in real time. Having this in mind, pedestrian evacuation training using AR can be made in the real environment (e.g. urban spaces, building, different means of transportation), but practically without any danger to trainees. Previous researches on the matter include adding information about egress routes over a live video of the real-world surrounding pedestrians guiding them to safe area (Ahn and Han, 2012; Mitsuhara et al., 2016).
As much as AR applications allow user navigation in real environment, they must deal with challenges of tracking and recognizing. Since the user is moving in real environment, AR application needs to continuously position the device of the user and track his movement. Tracking can be accomplished via GPS sensors built in the AR devic-es, but GPS availability can be limited in the closed spaces. Another option is using markers placed in the environment when AR device is continuously searching for them as reference points (Carmigniani et al., 2011). Second challenge, recognition of the surroundings, needs to be answered in order to reconstruct the shape of the environment to be able to add virtual elements.
MR merges real and virtual world producing new environments where physical and digital objects co-exist and interact in real time. In MR, virtual objects can be placed into a real environment or real objects can be incorporated into a virtual world in various extent (Milgram and Kishino, 1994). In the research of pedestrian evacuation, use of MR is still limited and true application is yet to be found (Zhu and Li, 2021).

Prospects of XR application for ship passenger evacuation
XR technology has been used in various extent in emergency management of pedestrians in urban areas (Fujimi and Fujimura, 2020) and enclosed spaces (Guo et al., 2020).
Shipping industry yet waits for this type of technology to be implemented widely in the process of ship design op- Source: freepik.com timization, training of ship crew and passengers, and developing salvation scenarios for various types of accidents (e.g. grounding, collision, fire). Here, possible application of XR technology is outlined in three different stages: accident prevention, accident response and accident analysis, Fig. 4.
or fire evacuation skills (Bulgakov et al., 2019), even the emergency procedures in case of earthquakes (Feng et al., 2020). Previous research shows that serious games concept raises interest in participating in safety routine drills and provided users with better understanding of the potential risks in the surroundings (Kawai, Mitsuhara and Shishibori, 2016). Accident response refers to the actions taken during an accident, with focus on evacuation and rescue. XR technologies offer a chance to perform these tasks without putting persons in any actual danger. Evacuation procedures of passengers onboard can be improved analysing the results of virtual experiments where humans are exposed, for instance, to fire and smoke and their level of stress is monitored (Cosma, Ronchi and Nilsson, 2016;Zhao et al., 2020). Also, VR/AR environment can be complemented with spreading of fire and smoke based on the results reached using computational fluid dynamics (CFD) procedures (Yan et al., 2020). Further, factors such as position and availability of emergency lighting, distance to exists or even sound alarm can be altered to develop better evacuation procedures (Lovreglio, Fonzone and dell'Olio, 2016).
As for the rescue procedures, training can be performed using AR navigation and tracking elements placed over real-world video feed (Goldiez, Ahmad and Hancock, 2007). Various scenarios can be developed to train rescue crews for emergency situations involving fire or smoke in enclosed spaces (Xu et al., 2014). In general, comparison of experiments performed using XR technologies with the field or laboratory experiments (Haghani, 2020a) and real case studies (Haghani, 2020b) show that real-life behaviour of pedestrians can be relatively accurately tested and observed using XR technologies.
Accident analysis refers to the detection of consequences and outcomes of the accident. XR technology can be used to document the damaged architecture of a vessel, transform it to digital three-dimensional models that can consequently be analysed. This is something that is currently being done by 3D scanning, but with the further advancements of the technology, resources needed to perform such task could be greatly reduced. Past research dealing with damage detection and documentation includes using AR technology to measure inter-story drift of a multi-story building (Dong, Feng and Kamat, 2013). Combination of XR technologies can be used to reconstruct damaged sites after a disaster (Carrozzino et al., 2012). Also, AR has been used for structural integrity assessment of a building hit by an earthquake (Kim, Kerle and Gerke, 2016).

Discussion and conclusion
Reviewing the existing references on the topic of "pedestrian evacuation", through Web of Science indexing platform, Fig. 5a, it is clearly visible that there is a considerable amount of work performed by research- Fig. 4 Possible application of XR technology in stages of: accident prevention, accident response and accident analysis.

Source: Authors
Accident prevention stands for any type of preventive actions that can be performed to avoid losses of human lives, equipment or negative environmental impact when a real accident happens. Accident prevention includes risk assessment and any kind of training. Risk assessment can be performed during ship design stage and/or ship exploitation. Using XR technologies, various scenarios including different emergencies (e.g. fire, smoke, collision, flooding, etc.) can be played in order to determine and asses risks for successful passenger evacuation. Based on the results of the analyses, ship design can be improved (e.g. corridor outline, position of exits or assembly stations) in order to achieve improved safety for passengers. Previous research on accident risk assessment includes, among others, development of an AR emergency system for safe guidance of pedestrians in case of a fire incident (Park et al., 2018). Further, XR technologies have been used to raise awareness of potential accidents in construction sites (Lovreglio et al., 2018). AR technology was used to establish routines for fire extinguishing equipment maintenance and operation during fire hazards (Chen, Lai and Lin, 2020).
As for the training, in addition to usual drills performed aboard, XR technologies can be used to acquaint the passengers with evacuation procedures in less stressful way and practically without any danger. For that purpose, XR technologies can be used to build serious games (Catal et al., 2020) that are realistic and timeand resource-efficient. This concept has already been used to train people in fire safety (Lovreglio et al., 2020) Fig. 5b. However, as this paper is concerned with "evacuation modelling", most specifically "extended reality (VR&AR)", this initial volume of references was filtered using mentioned keywords, Fig. 6. Further, results were filtered using keyword "ship", Fig. 6 As for the geometrical parameters, in VR environment, layout of the escape routes can be relatively easily altered to adhere to primary (where all the escape routes should be assumed to be in operation) or secondary evacuation case (where some of the escape route should be assumed to be unavailable). As for the population category, age, gender, physical attributes and response durations of the passengers can be altered artificially in VR applications. That way, demographic of the users who are engaged in VR could be uniform (e.g. students), but behaviour of their avatars in the VR can be adjusted to fit certain demographic category. As for the environmental category of the IMO guidelines, static and dynamic conditions of the ship should influence the passengers' moving speed. Guidelines state that these parameters could not yet be considered since no reliable data are available to assess this effect. However, with the use of VR, or even AR in real surroundings, this type of data could be collected with reasonable accuracy.
It can be concluded that, with the further development of wearable electronics and accompanying software, XR technologies will be put in use in greater extent in the field of pedestrian evacuation simulation. Evacuation of passengers on ships can be seen as a subset of pedestrian evacuation. Every ship passenger evacuation simulation should adhere to specific set of rules. Great advantage of XR technologies is that they allow performing experiments without any real danger to the occupants. Results obtained by the use of XR technologies could be used in the ship design process, to improve safety procedures or even to do post-accident analysis.

Funding:
The research presented in the manuscript has been supported by the European Social Fund (ESF) within the project "Promoting skills excellence for the labour market through the institutionalization of professional practice in maritime education (PANDORA)".