Transferring an Interdisciplinary Student Product Development Project to Full Online Conduction

: " Product Innovation" is a 7-month product development course, conducted at the Institute of Innovation and Industrial Management with international and interdisciplinary student teams. A close connection and interaction between industry, students and research has been an important part of this project-based learning course. Due to the COVID-19 pandemic and the worldwide occurring lockdowns, students were not able anymore to interact and conduct their projects in the proven manner. Being in a critical phase of the project, the course concept had to be reorganized and transferred to online conduction by using computer supported collaboration work within a few days. Both observations and surveys were used to compare changes in student´s behaviour and results of the project. This paper will describe 1) the original situation and course concept, 2) show how the course was transformed to online conduction, 3) describe the observations made and 4) analyse how the students experienced the transformation.


INTRODUCTION
This paper deals with the transformation of the projectbased learning course "Product Innovation" that relies on physical interaction between the stakeholders into a fully online conducted course. Project based learning can encourage collaboration and negotiation within a group and a crucial element of project based learning is the creation of tangible artifacts that solve authentic problems [1]. In order to support these elements, student teams within "Product Innovation" (PI) get full access to the Schumpeter Laboratory of Innovation -an academic makerspace facilitated by the Institute of Innovation and Industrial Management at Graz University of Technology (TUG). Equipped with modern equipment it serves as the central facility for PI and enables meetings, project work and rapid prototyping. Due to the impacts of COVID-19 and the subsequent lockdowns occurring all over the world throughout the first half of 2020, access to the Schumpeter Laboratory for Innovation (SLFI) was no longer possible for the student teams and major parts of the rest of the course had to be conducted online. This interruption of the course happened during a crucial time in the project that is heavily dependent on the usage of the facilities of the SLFI. The paper starts with an overview of the initial situation of the course "Product Innovation" and its supporting facility, the Schumpeter Laboratory for Innovation. The aim is to give an overview over the measures taken to continue PI despite access restrictions for the students. Further, the impact of the transformation on the student's behavior and project results should be investigated, to derive insights for future online project based learning projects. As input data for the investigation both observations as well as a conducted survey with the participants of PI are used. In the end the findings of the paper should be concluded as well as limitations be discussed and an outlook on further investigations given.

INITIAL SITUATION
The observations and investigations made in this paper deal with the course "Product Innovation" -a project based learning course by the Institute for Innovation and Industrial Management (IIM). The period under review is the project year 2019/2020 -starting midst of October 2019 until June 2020.

"Product Innovation"
Product Innovation, former "Product Innovation Project" is a course that is conducted by the IIM since the winter term 2006. Until today valuable learnings, insights, product ideas and even patents were delivered within this course for students as well as partner companies. It is a course that is originally based on the ME310 program from Stanford University, where Design Thinking ( Fig. 1) is teached via project-based learning and the provided work environment is important as it affects students in various ways such as their behavior, their creativity or their independence. [2,3] [4] Within the course of Product Innovation teams of interdisciplinary and international students work on real-life tasks that are provided by industrial partners. They are backed by a budget of 10.000 € to realize ideas, develop fully working prototypes and conduct experiments. While the majority of students is placed in Graz and works together at the same place, there are some remote members from other universities and countries, who collaborate remotely and just visit Graz for a few weeks within the project time . The course  is facilitated by the Institute for Innovation and Industrial  Management at Graz University of Technology and  throughout the course students have full access to the  facilities of the institute, including the Schumpeter  Laboratory for Innovation (chapter 2.2.).
Over time, incremental improvements have taken place but the overall process is still based on design thinking. The project lasts a full academic year and starts in October with the organisational preparation, meaning a recruiting and application phase. Teams are assembled in an international and interdisciplinary way and each team is working on a different task from a different company. To give them guidance and ensure good communication, each team additionally is supported by an academic supervisor. The official project kick-off takes place in November and startsas soon as the teams have been gathered -with a problem analysis phase. During this phase, teams empathize on their problem, make field research, conduct benchmarking and finalize the phase by refining their initial problem statement and goals. At the end of this phase, usually midst of January, the teams enter the ideation process -where they try to create as many ideas as possible for their projects. They evaluate the ideas, conduct first experiments and combine them to a concept before entering prototyping -typically midst of March. The prototyping phase is used to get valuable data about the concepts, test them and iterate the concept for improvement. The project ends after roughly 7 months, when the teams present their final product in front of a public audience of industry representatives, university staff and students in the beginning of June. During all the phases, students are supported with topic related workshops, where they are provided with suitable knowledge, methods and techniques. [5]

Schumpeter Laboratory for Innovation
The Schumpeter Laboratory for Innovation (SLFI) is an academic makerspace with offices, multimedia presentation technology and conferencing rooms as well as digital production machines. Its machine park for prototyping includes among others laser cutters, 3D printers with different technologies (SLA, FDM) up to 1m³ in printing volume, a vinyl cutter, electronic workspaces, a waterjet cutter, CNC milling machines and various hand tools. First opened for students in fall 2018 it serves as central base of the product development course "Product Innovation". After the project year 2018/2019 and along with a lot of insights on the usage of the SLFI, the full potential of this modern facility should be unlocked for the first time in 2019/2020. [5] Besides the machine park used during the prototyping phases, state-of-the-art multimedia equipment allows to conduct meetings with remote team members, sitting all over the world, in an easy and efficient way. The multimedia equipment includes 2 meeting rooms with video conferencing hardware as well as software. With over 200m² in size and a 30m² LED video wall, the biggest room, called design lab, is equipped with cutting edge communication and presentation technology. The room can be used versatile for prototyping, workshops, presentations and meetings and is open for the students of the PI. Making use of the space and the equipment, allows for more interactive meetings as several people can present live from the stage into a video conference. A representation of the layout of the SLFI is depicted in Fig. 2. After mid of March 2020, students were no longer able to access the facilities of the university for the rest of the course and it took until mid of April 2020 until the staff of the institute was able to make use of the extensive equipment again. It was therefore necessary to conduct major changes to the project´s procedure and processes and transfer it to online conduction within a few days in order to finish the project in time.

RESEARCH QUESTIONS AND METHODOLOGY
This paper deals with the transformation of PI into an online conducted project-based learning course due to the COVID-19 restrictions in Austria and should investigate the following research questions: • RQ1: How can a highly interactive course be conducted in a full online setting? • RQ2: Which impacts does the change to a virtual setting have on the student projects?
In order to investigate research question 1 different ways of computer supported collaboration work are investigated. Central processes that need to be supported are defined on base of the course structure of PI, as well as computer supported cooperative theory and compared with the abilities of software tools. Afterwards a set of tools is defined according to both suitability for the pre-mentioned processes, as well as the availability for project's participants and teaching staff.
In order to investigate research question 2 observations were made and compared to previous project volumes. In addition, a survey is conducted with the participants of PI who were sitting in Graz and therefore directly affected by the closing of the SLFI. The survey is developed based on OECD guidelines [6] and should also test insights and hypotheses from the observations.

LISTING AND EVALUATION OF COMPUTER SUPPORTED COOPERATIVE WORK SYSTEMS
In this chapter functions that need to be provided in order to conduct the course online should be listed and different solutions are shown. Especially the field of computer supported cooperative work was investigated to classify the requirements and investigate possible solutions. By using the categorization according to the computer supported collaboration matrix by Johansen (Fig. 3) [7] the project had to be transferred from mainly face to face interaction as well as asynchronous interaction into synchronous and asynchronous distributed interaction. Due to the fact that some of the international teams also work with remote members, there was already some pre-experience in some teams on how to work in a distributed way.  [7] In addition to the classification made by Johansen, possible groups of information and communications technology (ICT) processes were defined by Andriessen [8] for further categorization: Person interchange processes like communication, task oriented processes like cooperation or coordination or information sharing and group oriented processes like social interaction. These processes represent important functions that need to be supported throughout the Product Innovation course. For each process several solution technologies and systems have been listed, as seen in Fig. 4. As time was a crucial factor for the transition, the matrix has been assembled by gathering systems and technologies, the IIM already had some experience with.
The listed solutions were then evaluated. In a first step, knockout-criteria were used in order to reduce the number of potential usable systems. As the transformation was not planned, no costs for additional software or tools was accounted in the team's budgets. Therefore, the solution system needed to be available for both the teams as well as the institute's staff for free. Additionally, systems that already have been in place before the transition and were familiar to the whole project team should be maintained. The used systems -especially for information sharing -further needed to be in compliance with the non-disclosure agreements (NDAs) that are part of the contract between institute, students and partner companies and are a critical part to secure sensitive information, exchanged or developed throughout the "Product Innovation". The remaining solutions were then transferred into a final concept that provides all processes and functions needed throughout the project, with a focus on implementing as little systems unknown by the students, as possible and only systems which were evaluated as "easy-to-use" by the institute's staff.

TRANSFORMATION AFTER FIRST LOCKDOWN DUE TO COVID-19
Time was the most crucial factor for the transformation of this highly interactive course into a full online conduction, in order to provide students as well as partner companies with the information they needed to continue their project independent of the uncertain external conditions. Therefore, the final concept (as seen in Fig. 5) was developed in less than a week and introduced to the students in form of a workshop where they also received information on the usage and on online-collaboration best practices in general.

Figure 5 Final solution concept
For communication, E-Mails -including a regular newsletter -were already used throughout the whole year and supplemented by WebEx video conference, which was then used for the rest of workshops and meetings. TUG provided all students with a full licence for WebEx, which was the main reason to use it over alternatives. For the same reason, Nextcloud -an online file storing/sharing system -was used for information sharing. It additionally ensured that the shared data was secure and in compliance with NDAs and delivered sufficient storage capacity without additional costs. Set in place already at the beginning of the project it was well known by all participants. As an addition it provided "only office", an office tool that allows to collaborate synchronous on various office files like documents, spreadsheets, diagrams or presentations and was introduced as additional cooperation system besides the shared whiteboard provided in WebEx. For coordination, Trello was already used as project management tool and was maintained, regular updates with newsletters via mail were introduced. For social interaction, regular gaming evenings were organized over WebEx video conferences combined with free multiplayer browser apps. It became clear after a few weeks that the facilities would remain closed until the end of the project, leading to prototyping possibilities that were added to the concept. The students were given access to virtual machines that enabled 3D CAD modelling and could send orders to the SLFI, which were processed by the institute staff and made available for pick-up. Most of the provided methods and techniques could be used further, with just slight adoptions to the online conduction. E.g. for brainstorming -instead of having face to face sessions -the students were introduced to brain writing techniques, still applying the same familiar rules of brainstorming, but making documentation during online sessions easier.

OBSERVATIONS AND SURVEY REGARDING IMPACTS 6.1 Observations
Although observations became hard after transition to online conduction, due to the fact that people spent no time at the SLFI anymore, it still was possible to recognize differences during checkpoint meetings and workshops. Comparisons with previous volumes of PI, especially regarding the work progress, showed that there were positive as well as negative impacts by the transition. It seemed that especially during the beginning the motivation of the participants decreased which could be observed by a reduced number of contacts between the institute´s supervisors and the team as well as by a slowed down project progress. Online organised social events received little response in the beginning. Overall, the teams seemed to struggle to adapt to the given online setting, which could be observed by the little use of the given software (low traffic on the file sharing platform Nextcloud). On the other hand, it was observable that especially remote members, who often lacked information because of bad communication before, were more included in the process of the project and were able to participate at the virtually held workshops -which was not the case before. Also meetings between the team members were conducted more often as feedback from the teams showed. One struggle was especially immanent. When it came to physical prototyping, it could be observed that, since the teams were no longer to use the facilities of the SLFI, the activities reduced significantly. Previous years have shown that teams usually start their prototyping phase latest with beginning to midst of March and make extensive use of the digital production machines, hand tools and workbenches provided at the SLFI. During the underlying project year, teams were at the very beginning of testing and prototyping, not yet making full use of the SLFI capacities when the lockdown occurred. It could be observed during checkpoints, that although the teams tried to transfer their prototyping efforts into virtual environments, for some tests it was indispensable to have physical prototypes. Further, it could be observed that several teams wanted to design prototypes in 3D-CAD software but lacked powerful computing machines. These observations led to the introduction of additional measures as mentioned in chapter 5. Besides access to virtual machines, the students received the possibility of consultation hours with institute's staff where they could discuss, which physical prototypes are crucial and how they could be realized. Besides advice, the staff also processed the data given by the students, ran the machines and made parts and tools available for pick-up, so that the students were able to build and test some physical prototypes as well. Regarding project results it could be observed that all teams provided good quality in results, comparable with previous project years. However, due to the limited possibilities in physical prototyping, the quality decreased in that criteria but this was compensated by an increase in terms of virtual prototypes. Students receive grades for conducting PI, ranging from 1 (best) to 5 (worst). Although there are many influence factors for student grades, a comparison regarding the grading with previous years showed that there was just a minor deviation of 0,003 in the grades mean value for the investigated period.

Survey Development
In addition to the observations, a survey was developed to get more insights. The survey is based on the six steps to better survey design by OECD, as depicted in Fig. 6.

Figure 6
Six steps to better survey design, OECD [6] Step 1: Define survey objectives and target group -The survey objectives are to get insights about the student's perspective and perception of the transition to a virtual Step 1: Define survey objectives and target group Step 2: Draft survey questions Step 3: Pilot and re-adjusting the questionnaire Step 4: Select respondents and the data collection method Step 5: Run the survey Step 6: Analyse the results conduction. Additionally, the survey should support or withdraw conclusions that were made from observations and deliver information that can be used to improve the online conduction of the project. It is not the goal of the survey to test if the chosen technologies are the best technologies available, or to compare them to others. Regarding the target group, it should be stated, that remote students were already used to a mainly online conduction, therefore they should be excluded from the target group.
Step 2: Draft survey questions -The survey questions were mostly derived from interviews and discussions conducted after previous project volumes with the students. Key issues and aspects were identified and translated and categorized into the questions of the survey. Besides a general category where different aspects of the students are asked, the included categories are institute related communication/collaboration, company related communication/collaboration, the structure of PI in general, and facilities and culture. Due to the special challenges and for the investigation of the questions discussed in this paper a new category "virtual conduction" was added. The survey consists both of open and closed questions.
Step 3: Pilot and re-adjusting the questionnaire -The survey was tested partly already in previous years. Additionally, and especially regarding the new category of virtual conduction as well, a test-run with the survey was made by former students of the PI project as well as with supervisors and other staff from the institute. Feedback was included and the survey adapted accordingly.
Step 4: Select respondents and the data-collection method -Due to the fact that the survey is project-related and the target group well defined, the respondents should be all students that were participating in the PI project year 2019/2020, with the exception of exchange and remote students. As it was possible to use the facilities of the SLFI again after the project was finished and due to the fact that a final event was made after the ending of the project it was decided that the data should be collected by handing out a printed version of the survey to the participants.
Step 5: Run the survey -The survey was run at a final event that took place in October 2020, after the project already finished a few months earlier. Due to the fact that the survey was handed out physically, exchange students who took part in the project were no longer available for questioning. Staff from the institute was available during the students filled out the survey to answer questions, if something was not clear.
Step 6: Analyse the results -The survey was filled out by 26 students in total. Two responses were excluded from the results before the analysis as the given answers were inconsistent, therefore leaving a total of 24 questionnaires to be analysed.

Survey Analysis
In the following selected topics from the survey are discussed and analysed. The results are already summarized and categorized, only frequent mentions are to be discussed within this paper. For the survey scale a Likert scale from 1 to 5 was used where 1 was the worst and 5 the best rated value.
Perceived motivation influencers (negative) -the biggest influence on the student´s motivation was COVID-19 and the subsequent lockdown situation. 54% of the surveys stated that. More specific, the resulting lack of prototyping possibilities and the limited possibilities to see each other or get into contact with the companies were named. 21% of the students stated that they experienced some issues within the team, e.g. team member underperforming or not being active any longer.
Transition speed -With a mean value of 4.04 the transition speed was perceived as fast enough, but with slight room for improvement.
Newsletter (E-Mail) vs Trello Board -Students were asked about the usefulness of Newsletters (E-Mail) and the Trello Board -both used for coordination. Overall both showed plenty room for improvement with the newsletter being rated 3,58 on average and the Trello Board just slightly better with 3,83. Possible improvements named were for the newsletter to include a timeline and compress the information to an absolute minimum, while on Trello students wished for a more interactive and faster platform.
Perception of online conduction (chance or hindrance)on the question (different scale used) if students saw the online conduction as a chance (1) or hindrance (5), the mean value was 2,91 showing that it was seen very ambivalent with a slight preference of seeing it as a chance.
Perception of difficulty of switch -the mean value of the perception is 3,26 but 50% of the students answered this question with a value of 3 or less, making it the worst rated value in the questionnaire. Obviously it was, despite the fact that students already were familiar with many of the used systems, the switch to an online conduction was unusual and difficult to implement.
Perception about final solution concept -the final solution concept with the different tools and systems in place was rated with a mean value of 4,17, showing that student´s satisfaction with the concept was high. Only 25% of the students evaluated it with 3 or lower. On the question which additional software the students used or would´ve wished for, alternative video conference software was named (MS Teams, Zoom) and instant messaging within the teams, thus showing that some students were not fully satisfied with the video conferencing software and an instant messaging system could be complemented in the future.
Perception of virtual workshops -with a mean value of 3,74 the virtual workshops are rated good but also leave room for improvement. The reasons named are that it is harder to keep concentrated during a virtual workshop, the interaction is less, more people should be involved during the workshop in general and there should be more time planned in general, due to lags and interruptions.
Positive impacts on the project -the answers have been categorized, multiple answers were possible: 50% of the students did not observe any positive impact on the project. 13% stroke out the better inclusion of remote students within the project. 25% of the students mentioned that they experienced more time for working on tasks of the project on their own. Additionally, it was mentioned that more team meetings were conducted, no more time for commute needed, virtual prototyping made easier and learning about online collaboration valued.
Negative impacts on the project -the answers have been categorized, multiple answers were possible: 33% of the students did not observe or mention any negative impact on the project. 21% of the students stated that they needed to invest more time due to bad communication. 17% experienced issues due to a less motivated team, the same amount experienced issues with prototyping and testing. Other things mentioned were the restricted use of the SLFI, cancelled workshops and the continued conduction despite COVID.
Biggest challenges throughout the project -the answers have been categorized, multiple answers were possible: 25% of the students saw prototyping as their biggest challenge. 17% of the students had issues to keep the motivation high, the same amounts saw the communication within the team as the biggest challenge, as well as the lack of personal/physical social meetings. Another 17% mentioned organisational structure (time management, keeping structured, effectiveness) as their biggest challenge and furthermore presentations and keeping overview of the different tools used were mentioned.

DISCUSSION AND CONCLUSION
This paper showed how a highly interactive course, due to its project-based learning character, needed to be transformed into virtual conduction because of COVID-19 pandemic. It underlies the measures taken in order to make the transition fast and effective as the first lockdown happened in a crucial phase of the project, where usually an increased demand for the supporting facilities is seen. The measures were developed within just one week and implemented within 2 weeks after the lockdown started. Transition speed was seen fast by the students but they would have wished for more transparency. The final solution system used, was well accepted by the students, despite its short development time. A helping fact was that most systems used were already familiar to some extend to the students as many were already used throughout the rest of the project. Additional measures to enable prototyping efforts by the students allowed them to finish their projects without major delay. Nevertheless, observations and a conducted survey showed some difficulties and challenges that the students needed to cope with and that need to be addressed in future online conductions. First insights in the analysis of the survey are discussed within this paper but further investigations need to be taken in order to analyze interdependencies between the student's academic background or with project details with the impact of online conduction. It has also not been analyzed yet how the structure of the team influenced the perception of the changes. Learnings described in this paper have been taken to adapt the online conduction of the course "Product Innovation" further. Due to the fact that physical conduction of lectures and courses stayed restricted throughout the academic year 2020/2021, PI had to be conducted mostly online again in 2020/2021 and a re-evaluation for the impacts of the adaptions is therefore planned.

Notice
The paper was presented at MOTSP 2021 -12 th International Conference Management of Technology -Step to Sustainable Production, which took place in Poreč/ Porenzo, Istria (Croatia), on September 8-10, 2021. The paper will not be published anywhere else.