VIRTUAL REALITY AND EXERCISES FOR PARETIC UPPER LIMB OF STROKE SURVIVORS

Preliminary communication The article presents the issues of biofeedback in virtual reality (VR). This VR module in ELISE robot gives a possible re-arrangement of the damaged motor cortex which can be activated with the mediation of mirror neurons or through the subject’s motor imagery. This ELISE robot will help to accelerate the recovery from various kinds of neurological disorders, especially from the effects of stroke. Four physical/education games in virtual reality are described in more detail. This paper presents the main technical characteristics of the project, especially design, VR module, rubber expander module for the spastic hand rehabilitation and the details of the hardware/software system. Moreover, psychological rehabilitation aspect is introduced through an innovative virtual assistant of physiotherapist. The functionality of the mechatronic device for hand, forearm and arm rehabilitation has been presented during the first tests, and preliminary assessment of usability and acceptance is promising.


Introduction
Biofeedback is very important in the rehabilitation of stroke survivors and all of the patients for whom it is necessary to rebuild lost neuronal connections in brain [1÷4].Biofeedback provides information about changes of physical condition and patient's reaction during rehabilitation exercises [5,6,7].In many cases, range of motion exercises alone is insufficient in recuperation after different types of neurological disorders [8,9].The patient must be fully focused on exercises [10÷15].You might have noticed that reconstruction process of lost nerves after a stroke is not only based on passive and active exercises [16,17].Fig. 1 shows the relationships between the patient and motor training.The heavy line shows the area of the relationships (implementation of movement and biofeedback) which occur during the proper performance of the upper limb movement.Any damages to them require the recovery process which is realized by rehabilitation.The process of rehabilitation is supported by the physiotherapist or the ELISE robot.The traditional method of rehabilitation of stroke survivors takes place in the presence of a therapist.The therapist is an intermediary between the patient and a properly executed motor training.Another possibility to enrich the recovery process is with the use of rehabilitation robot.The ELISE robot -patient interaction compensates for some intrinsic limitations of traditional treatments.In this case, it is possible to realize a motor training and receive objective (numerical) data pertaining to the patient's health condition at the beginning, during and at the end of the rehabilitation process.This is particularly important during diagnostics.Moreover it is possible to realize the motor training in virtual reality (VR) [2].VR provides additional impulses for brain stimulation e.g.visual and auditory.With the aid of these simulations, a robot [17,18,19] for rehabilitation is intended to speed up the process of recovery from damaged neuron connections in the brain.During active exercises, the ELISE robot does not act directly on the upper limb leading the exercise, but only reproduces the patient's movements in virtual reality [4].During passive exercises, the ELISE robot affects the patient by realizing previously imposed trajectories of the exercises in virtual reality.These kinds of exercises are prepared to stimulate reconstruction of neural connections.The ELISE robot contains all of these methods.The ELISE robot is a modular device that provides the ability to diagnose, perform active and passive exercises with different kinds of biofeedback in VR under constant control of a physiotherapist.The module of virtual reality exercises presented in this article is a part of ELISE robot.This innovative and modular robot [20÷23] consists of a robot frame and the robot arm with supports for attachment of the spastic upper limb.The prototype of ELISE robot (Fig. 2) provides rehabilitation of the entire paretic upper limb including shoulder joint, elbow joint, wrist joint and hand.A single ELISE robot can be used to conduct both passive and active exercises of the right or left limb for stroke victims (includes hemiplegia and hemiparesis).Original solution of ELISE frame gives the possibility to conduct exercises in ten different planes (vertical plane (Fig. 2), horizontal plane (Fig. 3) and intermediate (Fig. 4).The ELISE robot includes a TG Drives digital servo controller with specialized software (TG motion) [26].Moreover a specialized software application for the control of this multi-module rehabilitation system was created.This software application was designed to control all servomotors using a Microsoft .NET Framework 4.0.Communication between modules was programmed through TCP/IP sockets.
To realize the exercises with a spastic hand, an innovative solution of a rubber expander with a pneumatic system was used.

Rubber expander analysis
The second functional unit of robot arm is the socalled "ending", which is held by the patient in his hand.It consists of a "rubber expander" with a pneumatic system.These innovative components were specifically created for the spastic hand rehabilitation.Gradual increase in pressure in "rubber expander" will increase its diameter.At the same time the patient's spastic hand is slowly opening.In the case of this paralysis, the spastic hand is closed automatically.The Rubber expander can be activated independently or it can be also activated with the ELISE robot at the same time.It depends on the set of exercises which are proposed by the therapist.The design process was conducted in light of the anthropometry of the hand, the traditional methods of rehabilitation, literary reviews, opinions and expectations of the therapists and patients.Based on these features, the rubber expander must take into account the assumptions: • The minimum diameter of the rubber expander: 20 ÷ 55 mm.

•
Thumb and index finger Flash forms a diameter 34 mm (for low women according to the EIM 979) and 46 mm (for tall men according to the EIM 979).

•
The maximum diameter of the rubber expander: 100 ÷ 110 mm.
• Maximum force of clenched hand: 300 N.
The rubber expander's design is based on a cylindrical grip hand, which plays a very important role in everyday life.As it turned out, in the case of a clenched hand, this shape is not exactly cylindrical.The little finger (the pinky) forms a circle with the smallest diameter.The ring finger forms a circle with a bigger diameter than the little finger.The prototype of the ELISE robot with a rubber expander, subjected to its first test, is shown in Fig. 3.The rubber expander with a pneumatic system consists of the following elements: • Electro-pneumatic regulator SMC series ITV1010-01F1N-Q, provides step-less control of air pressure proportional to an electrical signal.Operating pressure range: 0.005 ÷ 0.1 MPa.
• SMC series AS2002F-06, in-line type, compact speed controller for proper work of the rubber expander.• Source of compressed air will be a small compressor or tank.
Preliminary analysis of the rubber expander was carried out in two stages.The first part of the tests was focused on the selection of a suitable blend of rubber.This test used the rubber products available in cylindrical shape made of different blends of rubber.The second part of the test was devoted to the analysis of the shape of the truncated cone with a small angle of aperture.This part of the test was made based on the results obtained from the first part of the test.A two-stage nature of research was introduced in order to reduce the costs associated with the preparation of several injection moulds for the rubber expander.
In the first part of the test, several blends of rubber of different hardness levels and theoretical elongation were analysed.The blend of rubber, wall thickness and diameters of rubber expanders were also analysed.The rubber expanders had 170 mm in length.Diameters of 20 mm, 25 mm; wall thickness: 0.3 mm, 1 mm, 2 mm; elongation theoretical: 300 %, 417 %; rubber hardness: 52 Sh, 62 Sh were analysed.As it turned out, the most important parameters for spastic hand rehabilitation are: wall thickness and hardness of the rubber.Finally, the rubber expander with a wall thickness of 1 mm and rubber hardness of 52 Sh is the best solution for hand exercises.
In the case of wall thickness of 0.3 mm, the rubber expander had a problem with retaining a desired shape during exercises.The rubber expander with a wall thickness of 2 mm and rubber hardness of 62 Sh also did not meet the presumptions.The diameter and elongation of the rubber expander had only theoretical importance for achieving the targeted, maximum diameter of 100 ÷ 110 mm.In case of the biggest diameter (55 mm), it was not a problem with the correct execution of the exercises.The largest diameter of the rubber expander of 55 mm requires a long time to put it in the spastic hand.Additionally, the rubber expander with a 25 mm diameter and 417 % theoretical elongation did not receive the desired diameter of an open hand.The last part of the tests concerns analysis of the shape of the truncated cone with a small angle of aperture.Fig. 5 illustrates the diagram of the diameter of the base of a cone and pressure relationship in the rubber expander.The obtained results meet lub fulfils the assumptions.The obtained results satisfy the assumptions.The maximum operating pressure range is 0.035 MPa.It is even smaller than the assumed 0.05 MPa, for this maximum operating pressure range, diameter of rubber expander is 138 mm.This value is higher than the assumed value (more than 28 mm).This allows hand rehabilitation an even greater range of motion than initially expected.The starting diameter of the rubber expander has 55 mm for pressure p = 0.These values are related to the rubber expander at rest.Indirect measurement of force was based on ISE30A-01-F.This force is defined by information about the air pressure in the clenched hand and surface area of the hand.It should also be noted that the rubber expander is made of a blend of rubber which can be squeezed and put in the spastic hand.There is no problem to receive a smaller diameter than 20 mm.
Moreover, ELISE robot includes controls and software [27÷30], which are friendly for patients and therapists.The ELISE robot contains modules of diagnostics, continuous monitoring and report of rehabilitation progress.These modules provide numerical information about the convalescence of patient including ranges of motion and strength.As it turns out the balance between body and mind is the twenty-first century philosophy.Therefore, ELISE robot presents cooperation between the modules responsible for the movement (exercise) and the modules responsible for the biofeedback implemented in virtual reality [19].

Biofeedback in virtual reality of the Elise robot
The article presents four games which belong to VR package ELISE project.The first three games were included into the group of manual tasks.These include rubber game, balloons game and a maze game.The fourth is the word game and is an example of educational activity.This game was developed based on a training of social skills activities.In the first game, patient has to discover what is behind the grey field (Fig. 6).To discover a photo, patient has to hold and move the ending of robot arm.The rubber game is divided into levels, which has a larger percentage of higher screen to discover.The picture to be uncovered by the patient can be freely changed by the therapist.This photo can represent, for instance, topics related to the hobby of the patient or his family and friends.The rubber game was designed for the patient to practice his maximum range of motion.The patient is performing a rehabilitation exercises using the entire paretic upper limb.
The second exercise in VR is the balloons game (Fig. 7).An executive element in this game is the ending of robot arm which is a "rubber expander" with special shape.In this game, the patient is required to let the air in and out of the balloon using the ending of robot arm.Before the patient starts exercising the spastic hand, he must first select the desired balloon with the "rubber expander" cursor.The balloon will be highlighted and patient is possible to let the air out of the balloon by crimping his hand or to fill the balloon by opening his hand.In this balloons game, patient has to practice cylindrical grip, a range of motion and strength in hand.Additionally, there is a possibility to introduce adaptation exercises by changing the rules of this game during rehabilitation (e.g. when patient opens his hand, balloon shrinks on the screen).The last game in the group of manual tasks is maze game (Fig. 8).In this game, the patient has to plot the displayed trajectory using the tip of the robot.This task is focused on precision movement of the entire paretic upper limb.Patient practices his hand -eye coordination.All of the games from the group of manual tasks can be expanded with additional modules in a very easy way.It should be noted that the maze game is the perfect game for people who have problems with writing.The maze game tasks were designed for reciprocal relationship and cooperation between eyes and precise movements of hand.In all games, the patient can monitor his/her progress on the left side of the screen.These results contain information on the position of the hand (exact ending of robot arm).In the first game, these results contain information of the percentage of exposed surface of the photo.In the balloons game, results present the position of "rubber expander", range of motion in cylindrical grip and the strength in hand.In the last game, results contain information of the number of outputs beyond the path which the patient had to reconstruct.
Additionally, a complete report will be presented at the end of the rehabilitation session.All of this information should motivate and encourage the patient to carry out the rehabilitation tasks more effectively.This paper also presents an example of educational game -words (Fig. 9).This game has been prepared for stroke survivors suffering from aphasia.According to the National Aphasia Association, about 25 % to 40 % of people who survive a stroke get aphasia.Aphasia gets in the way of a person's ability to use or understand words.They may also have problems understanding reading and comprehending written words.In this game, patient has to find a word that does not fit into the other.The task of the patient is to identify an incorrect word.Then he has to confirm his choice by squeezing the ending of robot arm.The game connects movement exercises with brain training program.For example sets of words can be: • Bus, car, shampoo, motor.
If the patient correctly identifies the word which does not correspond with the other three, the screen will display "CORRECT ANSWER".If the patient makes a mistake, he will be informed.In this situation, the patient obtains next chance to correct his answer.Information about the number of errors is displayed on the left side of the screen.The therapist can modify range of motion on the screen display.All the time, a patient has the ability to select any of the answers because his range of motion is sufficient to do this.This game is concentrated only on patients memory.As in previous games, therapist can modify screen display contents.
VR was introduced for simulation of the real environment that is generated by dedicated computer software and can be experienced via a human -machine user-friendly interface in ELISE robot.The graphical user interface was built using WPF (Windows Presentation Foundation) technology, which is a module for creation of graphical user interfaces based on XML.It is part of the .NET 4.0 Framework.All of the games are based on WPF.Communication in ELISE robot was programmed using TCP/IP sockets.To exchange information between the client and server, it is necessary to create a connection between them.XML schema is used for the interchange of data over this rehabilitation system.The server sends data about the current position of the manipulator to the client.The client sends the data on the progress in the game in virtual reality to the server.At the moment, the ELISE system runs properly on Windows operating system.However, there are no technical limitations to generate an equivalent version for internet explorer.
All of the reports were recorded in the database.The database report was based on server Mysql5/MSSQL 2008 Express.There is also an option to generate a detailed report and a summary report of rehabilitation exercises.These reports contain data of rehabilitation exercises in different periods of time.Their structures are the same.Typical report contains numerical data and their graphic illustrations.Summary report can be prepared after a few days or weeks of rehabilitation process.Figs.10÷14 illustrate a summary report of the rehabilitation exercises of three persons.Fig. 10 shows the graph during the rubber game in the 14 days of exercises.The relationship between the % discover of photos of days can be observed on the graphs.All of the persons showed progress.All of the graphs showed the progress of about 25 % depending on the starting point.Fig. 11 and Fig. 12 show graphs of the spastic hand active rehabilitation during the balloons game.Fig. 11 shows the graph during exercises of the opening of this hand.Fig. 12 shows the graph during exercises of the squeezing of this hand.The graph of the person 3 shows the progress of the open hand about 40 % (Fig. 11).All of the graphs during exercises of the squeezing of this hand show the progress (Fig. 12).Fig. 13 shows the graph during the maze game.The person 1 has done this exercise best.Fig. 14 shows how the number of correct answers changed over time, and that for every participant the trend was growing.Moreover the person 1 has done this exercise best again.The trend for all the persons was growing, which indicated that the exercises had a positive effect on their motion motor skills and memory.Therapist can modify Technical Gazette 24, Suppl.2(2017), 451-458 screen display contents.Questions can be repeated or can be new each time.Therapist prepares individual plan of rehabilitation for each patient.
Figure 12 The graph during the balloons game (squeeze hand) Figure 13 The graph during the maze game Figure 14 The graph during the word game Psychological rehabilitation is another very important part of the VR module in ELISE robot.It has a huge impact on the whole process of recovery.It does not make sense to conduct physical therapy without patient participation and his motivation to fight the disease.Dispiritedness and passivity of patient may be a bigger problem to return to health as compared to the brain damage after stroke.The patient will have strength to recovery after stroke only if he accepts his current state of health and he takes actions.Sometimes when patient realizes another stage of rehabilitation exercises, he requires suggestions, encouragement or assistance in the implementation of motion exercises.The therapist is responsible for all these tasks.In order to partially relieve the therapist in these activities, the virtual assistant of therapist was introduced.This innovative virtual assistant of therapist oversees the psychological aspect of rehabilitation.The virtual assistant introduces a nice atmosphere and encourages the patient to rehabilitation exercises.It provides the guide on basic operation of ELISE robot and principles of VR games.In case of inability of cooperation between the virtual assistant of therapist and the patient, virtual assistant will inform the therapist about this problem.Via the introduction of virtual assistant, the therapist gets more time to supervise the rehabilitation process or to introduce any corrections in exercises or work of several patients, simultaneously.Virtual assistant of therapist was designed in .NET Minimum 100 MB free hard disk space.

Conclusion
All of the rehabilitation games in ELISE robot were designed for the reconstruction of lost functions in the brain resulting from stroke.They are realized by stimulation-compensation processes and the use of physiological phenomenon of adaptation of the system.
With the first group of manual task games, the patient has an opportunity to improve mechanisms of perception and motion.He has a chance to tie the sensory data from robot with the motion data from body.
Process of patient convalescence can be compared to a child's development from an early age.When the patient realizes a rehabilitation exercise with the aid of rubber game, he can try all the possible reactions of his muscles.Further, he will gradually discover his range of motion.Thus he develops the correlation between the incoming sensory stimuli and the motion reaction.The rubber game is also a perfect example to learn more about the VR module of the ELISE robot.
Another very important issue in rehabilitation is the process of adaptation.Patient has to adapt to the changing environment.The balloons game is excellent example of this process because it develops limb dynamics in particular hand.The process of adaptation in rehabilitation is shown in two ways, i.e. changes in the visual environment and changes in the mechanical environment.For example these changes relate to different ways of filling the balloon.The patient is required to respond appropriately to changes introduced by the program.The balloons game is specifically dedicated for patient with the spastic hand.The last game in the group of manual task games is the maze game.In this game, the patient exercises the ability to coordinate vision with the precise movements of the upper limb.The maze game was developed to create a good base for learning how to write.Proper coordination is the foundation for further rehabilitation.These exercises are the basis for the next word game.The word game is an example of educational game.In this game, the patient can remind himself of the words from different parts of social life.Stroke survivors have problems with naming objects, forgetting the meaning of objects or not being able to distinguish words within the same thematic group.Additionally, this game provides a reading comprehension.Finally, the word game helps in communicating with other people.
In developing the rehabilitation process by using ELISE robot, there are taken into account a variety of stimuli in this paper.The primary purpose of stimuli delivered from ELISE robot is to improve the patient's capacity to perform the set task.When rehabilitation is limited to a narrow range of stimuli, the effectiveness of convalescence becomes smaller.The patient performs rehabilitation exercises automatically.This is unacceptable.This key observation is reflected in each of the games available in ELISE robot.For instance, it is possible to manipulate the pictures in the rubber game or the rules of scoring points in the balloons game.In the case of the last game (words), there exists a possibility of classifying words from different aspects of life.Therefore, the patient gets the opportunity to re-learn words related with e.g.family, house, or shopping.In VR, the physiotherapist has unlimited possibilities to simulate everyday activities.This makes the rehabilitation process more interesting and friendly to the patient in comparison to the traditional treatments.
However, there may be instances when the patient exhibits negative attitude and when he/she does not want to cooperate with the physiotherapist.Sometimes, the patient has a fear of something new.He cannot understand and accept his health condition after stroke.It has a huge impact on the whole process of recovery.To address this, innovative virtual assistant of the therapist was introduced.The virtual assistant explains rules of exercises and encourages the patient to perform rehabilitation exercises.Consequently, the rehabilitation process becomes more pleasant and efficient.
All of the virtual reality games are dedicated to patients with active exercises and active exercises without loading.Connection between virtual reality games and the virtual assistant of therapist causes the rehabilitation to be no longer boring.It is possible that the patient's motor skills can be described by numbers.All of this information is accessible to the therapist in the final report of the virtual reality exercises.These virtual reality games have been designed for simultaneous stimulation of patient's brain as well as correct motion of the upper limb.These games and activities are intended to challenge the patient and to develop patient's creativity, imagination, thinking skills, and social skills.Additionally, these games were designed to increase the range of motion and to improve the precision of patient's movements.
Last part of this project pertains to the first tests of the prototype of ELISE robot.This ELISE robot was verified in a group of 3 patients.The trend for all the persons was growing, which indicated that the exercises had a positive effect on their motion motor skills and memory.
It should be emphasized, however, that the robot is not intended to replace the therapist.Instead, this robot is prepared to aid the therapist in long and exhausting process of rehabilitating the patient.

Figure 1
Figure 1 Relationships in the upper limb rehabilitation process

Figure 2 Figure 3 Figure 4
Figure 2 The robot arm is set in the vertical plane

• 3 /
2 solenoid valve SMC series SYJ314M-5LOU-Q.Compact speed controller to ensure proper functioning of the rubber expander from the therapist's software.Operating pressure range: 0.15 ÷ 0.7 MPa.• Compact proportional solenoid valve SMC series 2/2 PVQ31-5G-23-01.Compact speed controller to ensure proper functioning in the diagnostic module.This valve is controlled by the therapist software.

Figure 5
Figure 5 Relationships in the upper limb rehabilitation process

Figure 6
Figure 6 Rubber game -patient exercises the maximum range of motion

Figure 7 Figure 8
Figure 7 Balloons game -patient exercises the spastic hand

Figure 9
Figure 9 Word game -patient exercises his brain

Figure 10 Figure 11
Figure 10The graph during the rubber game Framework 4.0 and programmed using Visual Studio 2010 and Microsoft Expression Studio 3.0.System Requirements for ELISE robot are: • Install .NET Framework 4.0.• Windows XP with SP3 and Microsoft's latest operating system.• 800 × 800 screen resolution.• Minimum requirements 512 MB RAM.