MEASURING THE KNOWLEDGE TRANSFER PERFORMANCE AT UNIVERSITIES

Measuring the universities' knowledge transfer performance is important for both policymakers and universities due to the recognized social and economic impact of the knowledge transfer process. The aim of this paper is to investigate and discuss the contemporary knowledge transfer metrics used for measuring the knowledge transfer activities at universities. The research results show that the universities need to consider several important aspects when selecting and reporting on their knowledge transfer activities, such as the purpose and continuity of data collection and reporting on knowledge transfer activities, internal and external context in which universities operate, and implementation of common definitions for knowledge transfer indicators. Additional aspects identified from the conducted research which are important for the overall assessment of the universities’ knowledge transfer performances are the collection of both quantitative and qualitative data on knowledge transfer activities, and harmonization of the knowledge transfer metrics that will enable the universities to measure and compare their knowledge transfer activities, nationally and internationally.


INTRODUCTION
Nowadays, a significant number of universities perform knowledge and technology transfer activities beside their core activities of teaching and research. The purpose of the knowledge transfer process (i.e. technology transfer, commercialization of the research results, etc.) is to promote, disseminate and use the knowledge created at universities as a result of their research and development activities. Many universities have established their technology transfer offices -TTOs (i.e. knowledge transfer offices -KTOs) to manage the overall knowledge and technology transfer process. Knowledge and technology transfer from universities to industry, managed by technology transfer offices, is receiving more attention in recent years since scientific knowledge generated at universities can incentivise innovation, foster competitiveness and promote both social and economic development (Algieri, Aquino & Succurro, 2013;Romano, Passiante, Del Vecchio & Secundo, 2014;Secundo, Elena-Perez, Martinaitis & Leitner, 2015). The main drivers of total productivity of economies are increased quality and quantity of education and R&D which fasten knowledge and technology transfer, as well as commercialisation of new goods and services (Dobrzanski & Grabowski, (2019); Borovic, Rebic & Tomas, 2020).
Monitoring and evaluation of universities' knowledge transfer (KT) performance became very important for policymakers in numerous countries due to a recognized economic impact that is associated with the universities' involvement in KT activities. For this purpose, policymakers have designed tools for measuring and evaluating the universities' KT performance through a systematic data collection, and universities are required to provide the quantitative data on their implementation of various KT activities.
It is very difficult to measure KT activities at universities, unlike the measuring of the research excellence through publications quality and citations numbers, or measuring the excellence in teaching through the student-faculty ratio and other indicators. The selection of the appropriate and relevant indicators for measuring the universities' KT performances will enable universities to provide the accurate presentation of their KT activities. Standard metrics that are collected and reported for the KT activities can be useful in measuring outputs. Universities and policymakers are continually searching for the best mechanisms to evaluate and enhance the effectiveness of the university technology transfer processes.
The aim of this paper is to analyse, present and discuss the currently used KT indicators for measuring the implementation of KT activities at universities, supported by the existing theoretical and empirical studies. The results of the analysis will help universities not only in selecting the appropriate KT indicators to measure their KT activities, but also in considering and including other relevant quantitative and qualitative aspects for better assessment of their KT processes.

THEORETICAL BACKGROUND
The studies on university technology transfer received more attention in the academic research after the adoption of the Bayh-Dole Act in 1980 (Link, Rothaermel & Siegel, 2008). Universities have an important role of transforming the knowledge generated by the university scientists in order to create both social and economic value. The transfer of knowledge and technology is a valuable process that improves the local economic development, generates new products and services, and improves the quality of life through various spill-over effects (Shane, 2004). University knowledge transfer is recognized as an important linkage between universities and industry (Yuan, Li, Vlas & Peng, 2016).
Universities are organizations that generate new knowledge. Commercialization of scientific knowledge was recognized as important in the academic literature, but also for the policymakers (O'Shea, Chugh &Allen, 2008;Phan & Siegel, 2006;Rothaermel, Agung & Jiang, 2007). Markman, Siegel and Wright (2008) consider commercialization as a main activity for generation of the academic impact since commercialization enables a direct and measurable market adoption and implementation of academic research outputs. Siegel, Veugelers and Wright (2007) define university knowledge transfer as a process of transferring, converting and commercializing the results of the conducted basic research.
Many universities have incorporated the knowledge and technology transfer activities in their policy objectives. The universities with developed and incorporated strategic dimension of university technology transfer, and particular incentives for technology transfer put in place, will be more successful in the implementation of technology transfer activities (Link & Siegel, 2005). Technology transfer offices at universities facilitate and implement the technology transfer process. According to Markman, Phan, Balkin and Gianiodis (2005), the main responsibility of TTOs is the protection of university created IP and commercialization process management.
Knowledge transfer activity is of particular interest for the governments who provide funding for research activities at universities, since they need to demonstrate the impact of the universities research output on both regional and national economic activity (Phan & Siegel, 2006).
In general, the role of metrics is to measure the planned outcomes which are in line with the particular institutional mission. Policymakers, universities and other relevant stakeholders need to define their primary motives when deciding on the use of specific metrics to measure the KT performance.
Despite the growing number of researches on TTOs performance measurements, there is still a lack of the overall holistic measurement of the TTOs performance. Commercialization activities performed by TTOs were used as performance indicators, but mostly in isolation (Arque-Castells, Cartaxo, García-Quevedo & Godinho, 2016;Baldini, Grimaldi & Sobrero, 2006;Nosella & Grimaldi, 2009;Thursby, Jensen & Thursby, 2001;Siegel, Waldman, Atwater & Link, 2004). The focus on a very narrow range of activities and impacts limits the ability to draw relevant conclusions on TTOs performances (Rossi & Rosli, 2015). The function of a particular TTO varies from one university to another and from one country to another. The overall function of a TTO is to provide a formal and effective mechanism for commercialization of research results created by the university scientists. Knowledge transfer activities are difficult to measure both quantitatively and qualitatively, which makes the comparability between the institutions problematic. Certain associations such as Association of European Science and Technology Transfer Professionals (ASTP) in Europe and Association of University Technology Managers (AUTM) in the United States have developed the indicators for measuring the KT performances which are widely accepted. These associations collect data on particular KT activities, normalize these data versus research spend for particular university, and use these data for comparison. Indicators that are usually collected and reported are the number of patents filed, the number of invention disclosures, the number of spin-off established and the number and types of collaboration agreements with the industry.
The most commonly used indicators by TTOs to measure the KT activities are patents, licenses and spin-outs. However, these indicators are limited and do not completely encompass the complete effort needed by universities to engage into the commercialization process of the universities' research results. Measuring the impact of technology transfer activities is difficult due to the complexities of different technology transfer channels. Therefore, it is difficult to measure the impact of technology transfer activities in a quantitative way.
Currently, there is still a relatively low volume of theoretical research dealing with the definition of the appropriate indicators to measure universities' KT performance activities. Most of the indicators that were identified as most appropriate were defined based on the conducted empirical studies and the data that became available over time.

EMPIRICAL OVERVIEW
Measuring the universities' KT performance and choosing the appropriate indicators is a complex task that needs to take into account the variety and particularities of the implemented KT activities, and to consider several different aspects such as identifying what is the impact of implemented KT activities and how to capture it as both output indicator and as an economic value.
Gardner, Fong and Huang (2010) identified several important reasons for measuring the effectiveness of knowledge transfer activities at universities: to benefit the overall society from technology and knowledge advances, to educate on society's need for innovation, to provide an adequate return on investments, to provide benchmarks for comparison among universities, to promote global competitiveness, and to support future knowledge transfer funding opportunities. They also suggest that certain distinctive aspects should be considered when measuring the effectiveness of knowledge transfer activities: inputs vs. outputs, quality vs. quantity, subjectivity vs. objectivity, and time-series vs. crosssectional analysis. The universities, when deciding on what knowledge transfer metrics to use for measuring their KT performances, need to consider how to assess the impact of knowledge transfer on both the society and the economy.
Rossi and Rosli (2015) outlined several dimensions that should be considered in order to accurately measure the university KT performance: variety of KT activities, variety of impacts, comparability between institutions and behavioural incentives. They analysed the data of the United Kingdom's Higher Education Business and Community Interaction (HE-BCI) survey, the 2010/2011 edition, addressed to all universities in UK, and argued that KT activities can unfold in various ways and generate a variety of impacts. Choosing the indicators that are focused on a narrow range of KT activities and impacts will potentially limit the universities in presenting their KT performances in an accurate way. They also concluded that even though the choice of indicators used in the HE-BCI survey is much wider than in other running surveys, it is still not broad enough related to the types of KT activities and impact what consequently raises the question of comparability between the institutions. Rosli and Rossi (2015) discussed the most appropriate way of monitoring and measuring the universities' KT performance. They suggested that since there is no one-to-one correlation between knowledge transfer activities and theoretical models of knowledge transfer, the specific indicators for each KT activity should be carefully considered: all KT activities cannot be appropriately measured with the same indicators. In addition, they argue that output-oriented indicators alone are inadequate to capture the overall impact of universities' KT activities. Scanlan (2018) identified two critical challenges in measuring the knowledge transfer activities: how to compare KT data between universities for benchmarking purposes with the aim of establishing best practice and set appropriate targets for a particular university, and how to establish and improve the quality of KT essential practices and management within the university itself. He developed and proposed a knowledge transfer capability maturity framework consisting of five levels of maturity: from basic (initial) level in which the processes are not well defined or controlled, to a mature level where robust and repeatable processes are well defined based on the previous experience and practices. There are seven core competencies identified in the model (TT staff experience, spin-outs/LOA activity 1 , industry engagement, consultancy activity, TT/KT culture, IP management processes, and transaction speed and quality) that can be subdivided into additional sub-competencies. Even though this model can be considered as quite subjective since it is based on author's personal experience, it is based on the existing capability maturity framework which is a credible and reliable management tool (Secondo, de Beer & Passiante, 2016).
Another contribution of research conducted by Scanlan (2018) in measuring the KT activities is the development of particular ratios based on the standard metrics reported by TTOs (such as number of invention disclosures, number patent applications filed, number/revenues from research collaboration agreements with the industry, number/revenues from technology licenses, number of spin-off companies founded, etc.) which are normalized to the research expenditure. Normalization provides a measure on how much each of these ratios occur for a particular research expenditure. Theoretically, using such particular ratios can enable a comparison of a large university with some smaller universities, which can be very useful since it can also provide the exchange and transfer of best practices. Such comparison can omit certain potential differences between institutions, such as the core mission of the institution, the research funding source, the size of an institution and TTO, the region or country where institution is operating, and the maturity level of both the institution and the TTO. Many TTOs are reporting on these metrics on the annual basis. The conclusion of his research is that the KT maturity framework is best used as an internal tool for quality management, while the use of metrics ratios is best used for benchmarking purposes and for the detection of potential anomalies in the implementation of KT activities in line with the institution mission.
O'Reilly, Robbins and Scanlan (2019) developed a ratio for knowledge transfer (licenses, options and assignments -LOA) normalized per 10 million euro of research spent in order to get an overview of the universities' knowledge transfer performance efficiency. They collected data from the Ireland Annual Knowledge Transfer Survey (AKTS) in 2014 for 7 Irish universities. The survey has provided an overview of relevant KT performance indicators from publicly funded Irish universities. The focus of their study was on the licenses, options and assignments (LOA) indicator, but other indicators such as the number of patents filed, number of invention disclosures, number of spin-offs created, and number of collaboration agreements with the industry were also captured. The results of the study showed that the ratio of LOA normalized by the university research spent provides a better understanding of the university KT performance related to their research expenditure, but is still a relatively primitive indicator for the evaluation of the university engagement in collaboration with industry.
Choundry and Ponzio (2019) explored and examined the current technology transfer metrics, and developed the metrics for the holistic assessment that will enable U.S. federal agencies to evaluate and measure the effectiveness of their technology transfer programs. They proposed and evaluated two metrics: the Filing Ratio as a measure of the prudent use of resources, and the Transfer Rate as a broad measure of effectiveness. The Filing Ratio is calculated as a ratio of number of patent applications divided by a total number of new invention disclosure in a particular fiscal year, and indicates how prudent the federal agencies and laboratories in their use of patenting resources are. The Transfer Rate is calculated as a ratio of number of new patent licenses granted over the total number of patent applications filed, and provides specific information on how many patented technologies were actually licensed by the agencies. Both proposed metrics can be calculated from the currently reported standard data. Additionally, they proposed the use of expanded standard metrics: number of partnership agreements, transactional efficiency and proximal return on investment (ROI). Their conclusion is that all proposed metrics can be easily applied and calculated by using the existing standard data and would incentivise agencies and laboratories to reflect and implement the processes for better assessment of outcomes resulted from internal scientific research.
Kreilling and Bounfour (2020) propose the use of holistic, practicebased TTO maturity model in order to improve the current understanding of the TTOs performances. The model considers both input and output KT metrics and enables their contextualization. They collected and analysed the data from 17 European TTOs covering the five model dimensions: resources and competences, organizational parameters, generic practices, outputs and outcomes. Based on the analysis of the collected data, it can be seen that the proposed model has several novel elements that extend and complement the existing studies on TTO performances and practices since it provides a more detailed and comprehensive evaluation of resources and competences, organizational context and impacts compared to the previous TTO performance studies. Their model suggests how TTOs and their respective institutions can holistically assess their KT performances, and support policymakers in the creation of KT impact metrics.

ANALYSIS AND DISCUSSION
Knowledge and technology transfer as an activity at the universities can be measured and reported through a set of defined knowledge transfer indicators. Finne, Arundel, Balling, Brisson, and Erselius (2009) addressed the need identified by the European Commission for the development of comparable KT metrics across Europe related to the implementation of KT activities at public research organizations (PROs) 2 , including universities. They proposed seven core (standard) KT indicators to measure KT activities, additional supplementary indicators that will provide valuable information for better monitoring of the core indicators, and emphasized the importance of the basic data on the characteristics of a particular KTOs and PROs for the overall valuation of their KT performance. With the aim of enabling the comparability of KT activities between both individual PROs and Member States, they proposed and agreed on the harmonized set of definitions. The core indicators, additional supplementary indicators and defined basic data on the characteristics of the particular KTOs and PROs are presented in Table 1. Source: Finne et al. (2009)

, adapted by Authors
An important aspect for enabling the comparability of KT indicators is to define an appropriate denominator that will normalize and scale the outputs from institutions that are significantly varying in size. Finne et al. (2009) propose two potential denominators that can serve for calculating the standardized core performance indicators: research expenditures in the reference year and research personnel (number of researchers) in the reference year. Beside normalization, another aspect important for comparability is the time period for data collection. They recommend that all data should be collected for a reference period of one year, and if the reference year is not a calendar year, then the start and end dates for data collection should be clearly defined. Campbell, Cavalade, Haunold, Karanikic, and Piccaluga (2020) reviewed the core and supplementary KT performance indicators recommended by Finne et al. (2009), and proposed a set of harmonized core indicators for measuring the KT activities together with a methodology for their adoption. The proposed indicators for measuring KT activities should reflect various KT channels which include publications and presentations, teaching, networking/events, consultancy, professional development, collaborative research, contract research, licensing, and company creation. Additional supplementary KT indicators were also proposed. The proposed core indicators are presented in Table 2, and include both input and output indicators. Source: Campbell et al. (2020),

adapted by Authors
The findings of the presented empirical studies in Chapter 3, together with the analysis of the KT indicators for measuring the KT activities at universities proposed by Finne et al. (2009) andCampbell et al. (2020) provide a basis for further discussion and implementation recommendations.
In both studies, additional supplementary KT indicators were proposed to be collected in order to obtain a better insight and provide an added value to the implementation of the KT activities at universities. However, all proposed supplementary indicators are not appropriate for all universities/TTOs, therefore it is advisable that particular universities/TTOs select the appropriate supplementary indicators that will reflect their objectives and needs in the best way.
For the purpose of comparability and harmonization of the collected KT performance indicators, the adoption of the common definition for each KT indicator is highly important. According to Gardner et al. (2010), minor differences in definitions can result in large differences in results, but also concern the question of quality vs. quantity . Finne et. al. (2009) proposed the definitions for all KT indicators, and Campbell et. al. (2020) suggest that these definitions should be adopted so that the collection of the proposed core KT indicators can be implemented.
Universities need to consider several important aspects and issues when deciding on which KT indicators they will use for measuring their KT performances. The important aspect of data collection on KT activities at universities that needs to be taken into account is the purpose of the data collection. The collected data can be used as inputs for universities' institutional policy development and improvement, and for their funding decisions. Additionally, benchmarking the particular university on the implemented KT activities can also provide important information on the quality of implemented KT activities, and allow the comparison with other universities, both nationally and internationally. Another important aspect to be considered is the context, both internal and external, in which a particular university operates. As emphasized by Campbell et al. (2020), it would be very difficult to consider the outputs of implemented KT activities without the consideration of the university internal factors and external environment.
In order for KT indicators to have an adequate use in measuring the KT performances at universities, they should be defined in a way that their collection can be relatively easy to facilitate by a large number of universities and their respective TTOs which operate in different environments and circumstances.
An important part of the data collection on KT activities at universities is its continuity. As previously mentioned, it is recommended to collect these data on a yearly basis. The continuous data collection and reporting will provide the adequate quantity of the collected data that is important for further development of universities' KT performances measuring models.
From the analysis of the recent empirical studies presented in this paper, is evident that authors use the currently collected and reported particular data on KT indicators as a basis for the development of their models on measuring of the KT performances at universities. Continuous data collection and reporting, common definitions for KT indicators and involvement of the larger number of universities that will collect and report data on their KT activities will raise the quality of the collected data, which will consequently result in the development of the new and improved empirically based universities' KT performances measuring models. Additionally, higher quality of the collected data regarding universities' KT activities will provide better inputs for policy and funding decisions for both universities and government funding bodies. Quantitative KT performance indicators are not sufficient to demonstrate the complexity and the impact of the KT process. Qualitative indicators such as collection and share of the best practices, evidence-based case studies and success stories will help in the assessment of the overall KT processes at universities.
KT indicators should be considered as tools, since their use offers a broader view on the KT processes at universities and can impact both institutional policies and KT practices. It is important to emphasize that KT indicators reflect the KT performances of universities, and not their respective TTOs, even though this is not usually perceived in practice. TTOs are the facilitators and are managing the KT processes at universities in accordance with the universities' mission, strategies and policies within the universities context.

CONCLUSION
Universities today have an important role in supporting innovation and the economic development through the implementation of the knowledge and technology transfer process. In order to effectively facilitate and manage the knowledge transfer processes, many universities have established their respective technology transfer offices.
Monitoring and evaluation of the universities' knowledge transfer activities is very important for both universities and policymakers, since knowledge transfer process has a recognized social and economic impact. Knowledge transfer activities implemented at universities are very difficult to measure due to the complexity and the nature of the knowledge transfer process. The currently used KT indicators for measuring the universities' KT activities are presented in this paper. Both theoretical and empirical studies demonstrated that these indicators are useful for universities to assess their KT performances. Universities need to consider many aspects of KT indicators discussed in this paper when selecting, collecting and reporting the data on their KT performances. The main aspects they should take into account are their internal and external contexts in which they operate.
The quality assessment of universities' KT performance requires the selection and collection of both input and output KT indicators. It is advisable that a particular university collects data on internal context core indicators and activity core indicators as KT input indicators, and environment core indicators and impact core indicators as KT output indicators. This will provide to universities a quality overview on the implementation of their KT activities, and enable the comparability between different universities, both nationally and internationally. Additional supplementary KT indicators should be selected and collected by universities themselves based on their policies and objectives related to the KT activities.
Furthermore, the quality assessment of the universities' KT performances requires consideration of some additional aspects. Quantitative data on universities' KT performances are not sufficient to capture the overall impact of KT process, so the qualitative data on implemented KT activities should also be collected.
Future research on the implementation of KT indicators for measuring the KT performances at universities should be directed toward the harmonization of the KT metrics that will enable individual universities to evaluate and compare their KT performances both nationally and internationally. Harmonization of the KT metrics could also have a wider future application through, for example, the inclusion of particular KT indicators in the global and EU university ranking lists. However, the harmonization of KT indicators is still in its infancy, and requires a significant effort by many relevant stakeholders.