DEVELOPING A COMPLEX EXAMINATION SYSTEM TO EVALUATE THE INNOVATION ACTIVITIES OF HUNGARIAN AGRICULTURAL MACHINERY MANUFACTURERS

Original scientific paper In the past decade there was a growing interest in complex examination methods. The application of different complex indices and indicators has made it possible to examine such complex areas as innovation and research-development which are very difficult to measure. The use of thoroughly elaborated indicators helps understanding to a great extent as the characteristics of these complex categories and notions are expressed in a digit or number. In our paper we present such a complex method of examination that can be used to collect and evaluate primary data of the innovation processes of Hungarian agricultural machinery manufacturers. As part of the method we introduce the concept of a sectorspecific complex index, which, apart from assessing innovation potential on a corporate level is also able to create an order of competitiveness in the sector.


Introduction
The global social and economic changes that could be felt in the past decade forced the players of the public and private sector to base their decision making mechanisms on information of better quality while reacting to the challenges of the era.The topicality of our paper lies in the fact that the need to have punctual information disclosed about the single organisations, sectors of the national economy and the innovation activity as well as the features of the entire economy is more and more underpinned by recognising the economic role of innovation.The surveys/examinations on innovation that give us a thorough picture about the forms, sources, influencing factors and results of innovations help meet current requirements.It can also become apparent what innovation activities enterprises carry out, whether they are involved in research and development or make use of knowledge generated outside their organisations, how much attention they pay to organise regular trainings for the employees, i.e. what they do to preserve the ability of the organisation for continuous renewal.
The 'traditional' interpretation of statistics and innovation examines the processes basically from the points of view of revenues and expenditures.However, processes, factors and results cannot, or in the case of certain activities only to a certain depth, be explored this way.Due to the present methodology focusing on expenditures even innovation performance of significant volume can still remain undisclosed.Performance measurement, evaluation, management are key factors in every single business unit, forming a part of the corporate control process.A performance measurement method created suitably can supply feedback and information about where we are as opposed to the goals set.To measure strategy efficiency (performance) there are several approaches available with mainly controlling and financial aspects [3].
There are several factors in technical innovations utilised in agriculture that make measuring and assessing innovation processes and performance complicated and uncertain.One of the reasons for this can be that agricultural innovation processes are in a complex relationship with some other areas, and, what is more, this activity has significantly been expanded and gone beyond technical innovation in its traditional sense.All this, of course, makes the creation of the methodology of innovation performance more difficult.
The objective of the paper is to establish such a complex examination methodology based on international examination methods which is applicable to collecting and evaluating the primary data on the innovation activities of Hungarian agricultural machinery producers.As part of the method we have created the conception of a sector-specific complex index, which, apart from assessing the innovation performance of agricultural machinery manufacturers on a corporate level is also able to create an aggregate order of competitiveness in the sector.

Methods
No examples for innovation examination methods that concentrate only on agricultural machinery manufacturers have been found in the previous international innovation measuring practices.
As agricultural machinery production has a few such characteristics that differentiate it from the other manufacturing industries, it has been justified to establish methodology especially based on agricultural machinery manufacturers by considering the following features: -According to our previous experience it is sometimes difficult for the small and medium sized enterprises (SME) consisting of the significant part of the sector to decide whether their activities can be regarded as innovative or not whose impact could presumably be felt in the statistical results.-One of the most important indicators of innovation is the proportion of innovative enterprises in the national economy.However, the single national economic sectors and branches do not carry out innovations with the same intensity as they also have a different range of products and services.It is also a fact that smaller scale enterprises usually are not so innovative, they have fewer innovations and companies with fewer employees also have a substantially smaller ratio of innovations than in the case of bigger ones.
There are no or not enough data applicable on several areas having an impact on innovation or emerging as its consequence, of which it is worth emphasizing the topic of human resources and cooperation in innovation.
The theoretical conclusions drawn from synthesizing the innovation models and the specialist literature, the previous professional reports and the experience as well as methodological principles of the national and international empiric researches served as a basis for establishing research methodology.
The basis of our methodology is the compilation of a questionnaire which considers the connection between the running of the company and other external factors.Questions were directed at exploring the innovation activities and results of the national agricultural machinery producing enterprises.By means of its application data can be gathered on the extent of innovation activity, its main characteristics and partners taking part in the processes as well as the impact of innovation on the general situation of the enterprise.In addition to new or considerably improved product and services introduced and technological proceduresinnovations, organisational and marketing activity as well as innovation environment are also stressed in the questionnaire.
In Hungary more than 100 enterprises deal with producing agricultural machinery.A decisive part of enterprises have a wide range of activities as a lot of small-scale enterprises perform other tasks apart from machinery manufacturing.That is why it is difficult to determine the exact number of agricultural machinery manufacturers.Experts estimate that the number of companies engaged in agricultural machinery production as the main profile is approximately forty.A great number of organisations (80 %) being examined is small and medium sized enterprises whose annual income do not reach 1 billion HUF (3,3 million EUR).There is no list available on all the companies that could serve as the basis for compiling a probability sample so enterprises had to be selected for the questionnaire in other ways.The contact addresses of all the enterprises necessary for the questionnaire were gained through MEGOSZ (Association of Hungarian Agricultural Machinery Manufacturers).
A simple random sample technique was applied, i.e. n < N items were selected without repetition where each subset of individuals has the same probability of being chosen for the sample as any other subset of individuals.The multi-channel approach was used when recording the data of the research whose main points are the following: -15 machine manufacturers were interviewed personally.-Questionnaires were sent to 25 organisations by post asking them the send it back after filling in the questionnaire.Nine of the 25 organisations involved were also asked in person either because they had some difficulty to fill it in (3 cases) or further information was necessary for the unambiguous interpretation of the answers (6 cases).
-The electronic version of our questionnaire was sent to organisations that were incorporated in the MEGOSZ database.Altogether 18 questionnaires were returned.
The same questionnaire was used in all three approaches so figures can be compared.Data were recorded between March 2010 and August 2010.Fifty eight organisations supplied data in the examination.An approach based on proportion estimate was selected to ensure the reliability and accuracy of the research.The accuracy level for infinite population was calculated on the basis of the following equation (1).
where: z − quintile of standard normal spread, p − reliability,  � − sample size (in case of infinite population).
The following correlation corrects the defined population size ( �) to finite population (n) in the case of an infinite population (2).
where:  � − sample size (in the case of infinite population), n − sample size (in the case of finite population), N − number of items in the population.
As there are no reliable and accurate data on the extent of the population in the specialist literature we calculated with N=90 on the basis of different professional reports and statistical data.In this way the accuracy level of the entire sample is ±7,7 per cent points with fixed 95 % reliability, i.e. if the proportion of the 'yes' answer to a Yes/No question is 100p per cent, 100p ± =7,7 of the population gave an affirmative answer on 95 % reliability level.

The structure of the indicator
The following part introduces the concept that helped prepare the complex indicator typical of the sector.
In the past decades the so-called complex indexes (or indicators) that were used in Hungary became very fashionable by more and more people.Examples are the indicators of economic growth and development such as GDP or GNI or competitiveness indicators such as World Competitiveness Index (IDM) and Global Competitiveness Index (World Economic Forum).Furthermore, the aggregate innovation index (EIS) prepared by OECD can also be regarded as a complex indicator [4,9].The basic objective of preparing an indicator is to describe the performance of the examined units (countries, sectors or even companies) from a different point of view, but which, at the same time, can be interpreted as a complex unit.The indicator is such a parameter or value that indicates, describes or gives information about the condition and situation of a phenomenon, the environment or area.Indexes are such summarising indicators that are able to present the positive or negative changes of the phenomena by accompanying the phenomenon which is being examined.Indicators are the measurable aspects of society, economy, project etc. and monitor the directions and trends of the phenomena [8].
Indicators in the statistical system mean a level which is better processed and whose function is to decrease the amount of information that must be considered.While statistical data are of descriptive nature, indexes are interpreting indicators selected by a defined criterion, point of view or standard [1].Regarding the complexity of indicators there can be simple and complex ones.In contrast with simple indicators, the complex ones incorporate information on various aspects into one single index.The complex indicators analyse the different areas of society and the environment, their relations and mutual impacts so in their case special attention must be paid on the selection of the single compounds [7].Of the criteria defined for indicators the professional literature emphasizes specific nature, measurability, accessibility, reliability and its validity for a limited period in time.One of the main functions of indicators is data reduction, i.e. the requirement to reduce the amount of information which has to be considered for the decision makers.
Despite their numerous advantages the use of indicators also has some hidden threats.We must not forget that indicators are the simplified indexes of reality that was made measurable; they are of normative nature and have a defined value system.
The following tasks must be performed to work out an indicator which is typical of the sector by describing the innovation activity of agricultural machinery manufacturers: 1) Working out a conception.
While working out our indicator the Faber-Hensen model and the R&D activity matrix based on it served as the basis.[2,6] We were striving at keeping the advantages that the above mentioned methods have but, at the same time, sector-specific features were also considered.
The complex innovation index indicator which is being created relies on incorporating different innovation features (Fig. 1).

Innovation Efficiency Features
Figure 1 The theoretical model of the Complex Innovation Index During the creation of our theoretical model we tried to include innovation indicators that are based on quantitative indicators expressed in natural measures as well as qualitative innovation features that express efficiency.Such general corporate characteristics that have an impact on the innovation processes of agricultural machinery manufacturers in our examination were also included in the methodology.
The information content of the Complex Innovation Index: -The quantitative figures describing innovation input and output were incorporated in the so-called activity features.In the case of product and process innovations their novelty was considered, which added a weighed difference to the index.-Qualitative features such as revenue related innovation and R&D expenditure or the number of researchers and developers as of the total number of employees were considered in the indicator of innovation efficiency.Innovation strategy and cooperations were also part of qualitative features during the analysis.-The features of general corporate processes were summarised in corporate efficiency.The variables (though of different strengths) selected on the basis of the examinations carried out so far are in a significant relationship with innovation activity so their application is justified.

Standardisation
Most of the selected variables differ in their unit of measurement (e.g.revenue, number etc.) so they must be standardised before their integrated use.One of the most common ways of standardisation is to transfer the value of each variable into a standard value (Z scores method) in a way that the average (̅ ) is deducted from certain values (  ) and then this difference is divided by spread () [10].After transformation the different units of measurements disappear, the expected value of variables is 0 while its spread is 1, and positive values are above the average and the negative ones are below the average:

Testing the model
The method of principal component analysis was used to reduce the amount of information piling up in the variables without significant losses into fewer uncorrelated variables, principal components.During the process of the principal component analysis the different parts of information are summarised with the slightest losses possible (i.e. by maximising variance).In this way the selected method is excellent for constructing different performance indicators.
On the basis of the theoretical model of Fig. 1 the single (already standardised) variables were added by using principal component analysis and the newly created variables were also subject to analysis (altogether there were four principal component analyses), then the given principal component score was assigned to every organisation which incorporated the innovation features typical of the selected organisation as an aggregated variable.
To test the validity of the model, i.e. to decide whether the variables concerned are suitable for principal component analysis (based on their correlation and significance) the Kaiser-Meyer-Olkin (hereinafter referred to as KMO) criterion and the significance level of the Bartlett's test were used.KMO criterion is one of the most important indicators to decide whether variables can be used for factor analysis.Its high values (0,5÷1,0) signal that factor analysis can be implemented while values below 0,5 show that it cannot be carried out.[5] Bartlett's test examines if the variables in the population are not correlated (null hypothesis), i.e. it tests if the elements of the correlated matrix besides the main diagonal divert from zero by accident.We would like to reject H0 as the condition of factor analysis is that variables should correlate to the greatest extent possible.
The indicators of communality and factor loading were used to interpret principal component analysis.Communality measures how much percent of the spread of the original variable is explained by the factors introduced.The higher the given value is (maximum 1), the better.[3] The principal component analysis calculates the value of communalities for each variable, which are then summarised in the final communalities.Factor loadings reflect the correlation between the original variables and the principal component.High factor loading is the obvious expression of the significant, positive and strong relationship between the indicator generated and the original variables.
Four principal component analyses had to be carried out to create and test Complex Innovation Index.
The results obtained are summarised as follows: 1) The principal component 'Innovation activity' incorporates the input and output features of innovation activity, and was justified by KMO (0,551) and the rejection of the hypothesis of Bartlett's sphericity (Sig.0,000).The generated principal component preserved 70,7.% of the information content of the original variables.The factor loadings of the original variables are strongly correlated with the principal component.2) In the case of the feature entitled 'Innovation efficiency' the examination (KMO=0,641) is justified by the 0,000 value of Bartlett's spherical test.The generated principal component incorporated five variables during the examination, its factor loading is high (0,864) and the final communality is 0,749.
3) The suitability of 'Corporate process features' is satisfactory (KMO = 0,625), the significance level of Bartlett's test is 0,001.Four variables were incorporated into the principal component generated in the given examination; the factor loading is high (0,837) and its final communality is 0,702.According to the analysis the generation of the principal component on the basis of strategic behaviour, marketing processes, number of employees and variables of revenue was successful.4) Finally the indicators of innovation activity, efficiency and corporate efficiency were incorporated into the principal component of the would-be complex innovation index.Kaiser-Meyer-Olkin criterion was tested again according to which the suitability of the examination was 0,532 and Bartlett's test was also successful (Sig.0,002).The principal component retained 75,6.% of the information of the original variables.
The examination system of 14 variables could be reduced to one principal component with proper information content.On the basis of the principal component analyses carried out the indicator managed to retain the information content expected (each principal component retained more than 70 % of the information content of the descriptive variables).

Application of the method
With the help of the generated Complex Innovation Index the principal component value of each organisation was calculated and assigned to the organisation in question.As a result, every enterprise can be characterised by complex values of innovation activity, efficiency and corporate efficiency.By putting the values of generated principal components in order a sector specific innovation (competitiveness) rank could be created so the negative side of the axis shows low, while the high values of the positive side reflect powerful innovation potentials.The principal component values of the calculated Complex Innovation Index range from -2,03 to 1,05 and their spread is on the left sliding in the negative direction of the histogram (Fig. 2).Innovation activity hardly exceeds 40 % and it can only be considered a novelty on corporate level only.In their case there is a need for changing their attitude to corporate culture besides providing them with sources for development to increase innovation ability.
The corporate level method of forming groups that was worked out together with its analysis pointed out that regarding innovation ability branches cannot be regarded as units.The Complex Innovation Index, generated from the different dimensions of companies as an evaluation system stressed this internal heterogeneity.In innovation activity, organisational and innovation processes there are significant differences reflected between the single groups.That is why it is also worth dealing with working out cluster specific instruments that urge innovation.

Conclusion
Acknowledging the stressed importance of information has generated an increasing need for information recently in Hungary, too.The experience of analysing innovation data survey has directed attention to the wider correlations of the area which decision makers and participants of innovation activity would like to get acquainted with.
Despite the developments of the past decades there are several gaps and undisclosed areas of measuring innovation.New needs may presumably be generated by extending knowledge about the nature and internalexternal relations of innovation as well as changes in economic and social life.The objective of the paper was to present the concept of such a complex index that is applicable to evaluate the innovation activities of Hungarian agricultural machinery producers within the framework of a complex examination method.
On the basis of the principal component analyses carried out the indicator managed to retain the information content expected (each principal component retained more than 70 % of the information content of the descriptive variables).
Fourteen variables were integrated in the newly generated complex indicators (termed as Complex Innovation Index).On the basis of the principal component of analyses carried out the indicator managed to retain the information content expected (each principal component retained more than 70 % of the information content of the descriptive variables).
The practical benefit of the method is that by applying it the micro-level analysis of companies was carried out and then by aggregating the results an order of agricultural machinery manufacturers was established that can be subject to further analyses.
The methodology of the examination worked out, the questionnaire and the Complex Innovation Index (as was set in our research objectives) took the characteristics of Hungarian agricultural machinery manufacturing into consideration so it has some restrictions in using it as a comparison with other sectors.The questionnaire was compiled according to OECD principles so the sectoral data gained can also serve as a basis for an international comparison.However, the index created is an indicator that considers the features typical of the sector so its values cannot be generalised for other sectors.At the same time, as a method, care must be taken when using it in terms of primary data.
To sum up the methodological experience of the primary research we can conclude that one of the ways to develop the research in the future is to increase the automation of data collection and speed of data processing.One of our further plans is to work out a more modern and flexible way of internet-based interviews that make the annual monitoring cycle in the topic possible (based on the existing methodology).To meet this end, the questionnaire generated must be implemented on the internet and suitable storage and technology should also be worked out.
Another way to go with the research can be to test Complex Innovation Index in other sectors.The experience gained this way could serve as the basis for improving the content of methodology on theoretical basis.
The research could also be extended to other European countries in the future and comparative analyses could be carried out.The determining impact of the global economic recession makes international overview and implication especially topical as getting to know and adapting best practices are only possible by paying attention to social-political and economic special features.

Figure 2
Figure 2 Corporate ranking based on the values of the Complex Innovation IndexThree groups were created on the basis of the axis and histogram of the complex innovation index.1) Top performers have higher values (above 0,5) and their ratio in the sample is 10,3 %.Their innovation activity can be described by high numbers both in quantitative and qualitative terms, which makes high level R&D knowledge integration possible.Their operation is characterised by efficient organisational coordination and advanced communication between the functions.They have acknowledged the importance of innovation and integrated their ideas for long-term development into the corporate strategy.2) Those that catch up can be described by average values (−0,5 and 0,5) and their ratio is significant in the sample, almost 34,5 %.Their innovation performance is high in quantity but rather satisfactory in quality.Their attention is drawn to technological innovation which is aimed at improving efficiency indicators of production and catching up.In their case the knowledge necessary for technological development is integrated on individual level.Higher corporate efficiency and better coordination as well as ensuring additional sources would be the conditions for improving their innovation abilities.3) Those that lag behind comprise a group with low values below −0,5 and comprise more than half of the pattern (55,2 %).Their innovation activity is characterised by low qualitative and quantitative indicators.