Parallel Motion Images in Visual and Near Infrared Spectrum

This paper presents parallel film recording of sunlight reflection from the same matter, in visual (V) and near infrared (Z) spectrum. We have designed a ZRGBMotion GoPro video camera for parallel high quality motion recording in dual spectrum. Recorded are two parallel video twins, presenting different information in the grey area. The usage of extended reality in the infrared spectrum could be used in artistic as well as identification purposes. The scenography and costume designs give the technical solution in the making of a parallel motion image. The importance of parallel reproduction of motion images lies in the differences that create a new expanded space in communication design.


INTRODUCTION
Today's current modified cameras are able to capture infrared photography.They produce a still image of near infrared image (NIR) used in different fields of science, i.e. nature observing, documentation of monetary paper, in the military industry, archaeology and the identification of the originality of paintings in the arts field.In this paper we discuss the video recording of motion, in dual spectrum, giving dual information at the same time.In an experiment, we record a play at the theatre, using the ZRGB-M GoPro video cameras.The scenography is made intentionally in dual spectrum for this purpose and gives the play a double meaning.The audience are shown the live performance and the NIR recording at the same time.With ZRGB-M camera, we create a visual and infrared area in the purpose of developing a new communication design in every motion video art.
These cameras only have a single lens and during the daylight, they capture the image in the visual spectrum, either in colour or in black and white, depending on the type of camera.In the absence of sunlight, the camera captures near infrared reflection ranging from 850 to 1400 nm.The separate recording of visual and infrared spectra, with one camera, is possible due to the technical characteristics of the CCD chip inside the camera.The camera captures the image on the CCD chip, with the light sensitivity up to 1400 nm [1,2].During the night recording, a filter for controlling near infrared area is placed on the lens.This way the spectra range from 700 to 800 nm, between the end of visual and beginning of infrared area that would show the remains of absorption as a continuation of the visible spectrum, is avoided.This ZRGB camera was designed for making still images in dual spectrum.
ZRGB-M motion recorded camera is composed of two separate camera lenses where one lens is recording the visual spectrum (RGB: 400 to 700 nm), while the other records the absorption of near infrared materials through a single channel Z at 1000 nm.Both cameras record video motions that are indicated with M (motion).ZRGB-M camera simultaneously captures RGB and Z motion picture of the environment.The camera uses daily light without embedded infrared source that allows it to record both, immediate vicinity and objects at a great distance [1,3].
During the parallel recording, a clapperboard was used in order to synchronize two shots taken at the same place and time in two different spectra.With the help of cameras, the terms of passive and active monitoring and recording of environment were defined [4][5][6].Passive observation was used to examine the reflection of near infrared spectrum of flora and fauna.The absence of absorption was noticed with all living plants, while the fauna demonstrated camouflage properties in the infrared area of observation [5,8,9].ZRGB-M camera was used as a tool for passive observation of students during their visit to a company [5].The recording was made during the daylight, therefore with no artificial IR sources.Depending on the textile dyes used, the differences in the students' clothing were noticed.Colour management was used to achieve perfectionism for visual experience of colour [10].Research has shown that some animals (cats) can see in the near infrared range, which developed a further research that has shown that the nature around us looks different depending on the eye of the beholder [11,12].A new method of active information management was suggested, taking into consideration the properties of matter in the near infrared spectrum [10].The use of diversity is anticipated in the visual and invisible spectrum in the field of visual arts and design [5].The properties of dye pigment enabled an active influence on the near infrared range of art [7,8,13].Creation of paintings with the near infrared properties was defined as a new way of painting that includes the deliberate creation of copies [9].Double art paintings were demonstrated.These paintings are conquering the new visual space, intriguing and provoking the viewer with a double message while uniquely and impenetrably protecting the original painting [11].The Infrared information includes individualized choice of colours and hidden information protected from copying or reproduction [9].By defining the CMYKIR separation (INFRAREDESIGN technique) the door to a diverse and creative protection of graphic products, packaging, textiles, leather and art was opened [2,11,14].The knowledge on printing possibilities of the infrared spectra enabled the creation of the security elements [15,16].Further research defined the branding and individualization in the infrared spectrum.The hidden infrared graphics were built into the luxury designer product.The use of these techniques as standard techniques in business documents and communication as well as protection of well-known international brands was proposed [8,17,18].In order to use the new method of communication design in the visual and infrared spectrum of matter, it is necessary to upgrade the surveillance cameras for parallel (simultaneous) recording of the objects/subjects and space in double spectrum for the purpose of obtaining the complete information.

EXPERIMENTAL WORK DESIGN
By parallel recording matter in double spectrum, we create video materials for expanded communications.By research findings of self-properties of matter in its infrared region, which are not visible to the naked eye, we upgrade space of information.
In order to analyse and determine the details of matter in the visual and infrared spectrum, the "GoPro Hero 4 Black" camera that records the image on the CCD chips sensitive to the light up to 1400 nm, was reconstructed [19,20,21] (the video with the double film in V and Z recording can be found on the web site http://zrgbfilm.svemir.hr/goprozrgb/,[23] which is an integral part of this article.)For the modification of the Z camera, the extended spectrum lens for modification of Z camera was replaced by lens and filter that only records the spectrum from 800 ÷ 1100 nm [21,22].The second camera lens was replaced for the purposes of recording frames of the same focal length as the one of the Z camera.By analysing infrared images, the need for parallel display of double spectrum for the positioning of an object in space, presented itself [2,3,5].The constructed ZRGB-M camera with two lenses is connected to a common trigger that allows the parallel start of both cameras.The previous use of a clapperboard is upgraded in order to describe the recorded material.The cameras are recording parallel highdefinition videos.We analysed three separate situations of passive video recording with ZRGB-M camera.For the purpose of avoiding misinterpretation of infrared images as copies of its black and white twin, ZRGB field of visual image was converted to grey.
When converting ZRGB image, in order to convert the brightness value, the calculation of brightness (Luma = Y) Y = 0,212•R + 0,701•G + 0,087•B' was used.[24] Usage of different ZRBG value, when converting to grey scale image, is due to the fact that human eye is more sensitive to the green and red components than to the blue ones.Colour images enrich the human experience of the observed material more than the grey ones [3,25].The parallel recordings of twins recorded with ZRGB-M camera are analysed in their grey area.The differences between the video recorded in grey image of the visual spectrum, and its near infrared twin were detected.Active influence on the infrared spectrum of matter is introduced.By demonstration and analysis of selected images from the video, the information recorded in the dual range is presented.

Redesign of a GoPro Camera in ZRGB-M Camera
The redesign of a "GoPro Hero 4 Black" camera with a feature for recording photos and videos in high-resolution 3840×2160 (4K) was performed.In its original configuration, camera lenses recorded the near infrared (NIR) area, but were unable to block the sunlight for the visual part of the spectrum.By replacing the lens and filters, used exclusively for recording the near infrared area, a part of ZRGB-M camera for recording the NIR twin was created.Fig. 1 shows the reconstruction process of GoPro cameras.(Full video is available on the website: http://zrgbfilm.svemir.hr/goprozrgb/).On the other GoPro camera the lens with the identical RGB focal length and NIR lens was replaced, in order to record the twins of the same visual area.In order to record the equivalent horizontal picture, the lenses of the ZRGB-M camera were positioned as closely as possible, and the cameras were placed inversely -one camera was recording a vertically oriented picture.Fig. 2 shows the casing with a reverse ZRGB-M camera lens and the feature for the automatic recognition of the reversed camera.ZRGB-M camera is placed on the chest so the cameraman can move around and passively observe the environment while recording without using a tripod.The cameras are closed inside the case with no access to either camera's triggers.Described cameras are connected to a remote control through a wireless network.Wireless communication of GoPro camera in the ZRGB-M system gives a dual functionality -simultaneous management of both cameras via remote control as well as creating a connection with a smart phone application for viewing the recorded video.Using the remote control enables both cameras to start recording at the same time.In the previous work, a movie clapperboard was used to mark the start of the recording.When using GoPro camera as ZRGB-M camera, clapperboard is still used as a means to synchronize external devices for sound recording as well as to describe and identify the recording staff.The camera is located on the cameraman's chest, thus using the ability to connect the camera to a mobile device in order to watch the recording of one of the cameras on the phone as shown in Fig. 3.One mobile device can monitor only one camera at a time, thus a further development of the applications for parallel viewing of the twin ZRGB-M system lens is recommended.

Recording the Environment with ZRGB-M Video Camera
Featured and attached are three different situations of parallel passive environment observation with the reconstructed ZRGB-M camera.In the first situation, the outdoor ice-rink was recorded on a sunny day at 12 pm on January 7 th 2016.It was snowing the night before therefore the majority of the observed surroundings were covered in snow, thus white.The video analyses pedestrians recorded during the passive environment observation.A few days later during the "Audio, video and image processing" class at the Zagreb University of Applied Sciences on January 12 th 2016 at 9:30 AM the indoor scenes were recorded in the classroom where several students were wearing Infraredesign uniforms in the service of teaching and learning about Infraredesign technology.Due to indoor recording, the artificial light was used.
Featured are also the materials recorded on November 8 th 2014 at 3 PM at an estate in Osijek.The recording shows people, animals and nature in the same environment.Clips from Osijek were shot with a different ZRGB camera -a modified photo camera with photo and video options for recording.The third footage resolution is lesser than the resolution of images recorded with the GoPro ZRGB-M camera.
The value of this version of the camera is in the real time recording of dual spectrum which is seen as a different image in the V/Z scenography and V/Z costume design, as well as directing.It can be viewed at this web address: http://zrgbfilm.svemir.hr/goprozrgb/.

VIDEO MATERIAL ANALYSIS
Video materials were recorded on memory cards for each camera.Adobe Premiere CC 2015 was used for processing the video and extracting the static images.Adobe Premiere CC 2015 is a high-quality (4K) tool for photo and video processing.Preparation of all materials in Adobe Premiere program is a part of the process of creation of video twins recorded with the ZRGB-M camera.Using a video processing application, black and white video effect was used to remove the purple tone within the infrared recordings.In the next phase, the video material was transferred to a black and white mode and compared with the near infrared recording (also in black & white mode).
Using simultaneously the display of the images recorded with the ZRGB-M camera (Fig. 7) the differences in grey tone of visual and infrared were analysed.While in the recordings of the ice rink the environment is white due to the snow cover, both visual and NIR are equally grey areas.However, analysing the clothes made of different materials and different dyes, the ZRGB-M capture shows a black hat, which appears white (Fig. 7A); a black pouch on a skater's back that is "merged" with the black jacket, however in the NIR image the jacket appears white while the pouch is black (Fig. 7B).A tiger patterned jacket has no identifiable dots in its infrared video twin (Fig. 7C).When placed in the grey scale in the visual spectrum, all living flora appears as grey background.At the same time, in the NIR range, depending on the health and age of the flora, it appears brighter (Fig. 8A).Elements of clothing display the same principles as the ones shown in the previous recording of clothes dyed with the dye of a various origin (Fig. 8B).Observation of fauna confirms already proven camouflage element in animals, but also their identical image in the grey scale area of ZRGB images.(Fig. 8C) Colour images enable better space orientation and easier object identification and are thus more accepted and widely used.The same cameras, but in low light conditions (night vision), automatically generate infrared rays; therefore the final recordings are shown in black and white (grey).The introduction of new clothing design and visual scene elements to our environment creates the need for a parallel monitoring system in both spectra.The usage of the ZRGB-M camera, as an upgrade of the existing surveillance cameras, using the daily light to simultaneously record visual material and near infrared area, was proposed.This way of observation expands the information system and visual communication in terms of clothing, colour of cars and all the visual elements in the environment.

CONCLUSION
Simultaneous videorecording and reproducing the dual spectrum of visual and near infrared reflection of sunlight created the expanded information.When designing a visual communication it is necessary to conceive the new informational space invisible to the naked eye.For parallel video recording of double environment spectrum, GoPro RGB video cameras were redesigned into ZRGB-M cameras.For all further exploration of parallel visual and near infrared videos the use of high-resolution images is recommended in order to generate better conditions for the analysis.
The videocameras used in this research have the ability to connect to a smart phone application, but are restricted to viewing only one spectrum at the time.We suggest the upgrade of the application in order to enable viewing the images in two spectra at the same time.Creating black and white photos from images conducted with the RGB camera, we can easily observe and define the differences between the image and its infrared twin.With the active management of the properties of matter in the near infrared area, INFRAREDESIGN technique created a new area of design and communication.We recommend the expanded display of the information to designers of all visual elements in our environment.As the expanded system information will be assimilated into everyday life, in terms of surveillance of certain objects, it is necessary to observe not only visible but also invisible infrared spectrum.The use of the ZRGB-M camera as an upgrade for the surveillance cameras in order to monitor the movements of people and the environment is proposed.The creation of new art forms by reproduction of the video materials in visual colour spectrum, its black and white (grey) versions and their near infrared twin is predicted.

Figure 1
Figure 1The process of replacing the lens and filters on a GOPRO 4 camera

Figure 2 A
Figure 2 A system of double reverse camera placed on the cameraman's chest

Figure 3
Figure 3 Live viewing of recorded material

Figs. 4 ,
Figs.4, 5 and 6 show the separate frames of the parallel ZRGB-M video camera recording while the full video footage can be viewed at the same address.

Figure 7
Figure 7 Parallel display of ZRGB (B&W/Z) Fig. 8 shows the flora and fauna recorded in Osijek.When placed in the grey scale in the visual spectrum, all living flora appears as grey background.At the same time, in the NIR range, depending on the health and age of the

Figure 7
Figure 7 Parallel display of ZRGB (B&W/Z)4 APPLICATION OF ZRGB-M CAMERA AS SURVEILLANCE CAMERASZRGB-M camera recorded motions in the environment of two areas: visual and infrared.Previously used surveillance cameras placed in the environment for security purposes have only one lens that enables video recording of a single spectrum.Depending on the type of camera, in good lighting conditions, it records the visual spectrum, either in colour or just the grey component of the visual spectrum.Colour images enable better space orientation and easier object identification and are thus more accepted and widely used.The same cameras, but in low light conditions (night vision), automatically generate infrared rays; therefore the final recordings are shown in black and white (grey).The introduction of new clothing design and visual scene elements to our environment creates the need for a parallel monitoring system in both spectra.The usage of the ZRGB-M camera, as an upgrade of the existing surveillance cameras, using the daily light to simultaneously record visual material and near infrared area, was proposed.This way of observation expands the information system and visual communication in terms of clothing, colour of cars and all the visual elements in the environment.