A Mixed-Method Study on Measuring Epistemic Emotions as a Trait

.


Introduction
In recent years, more and more studies acknowledge the importance of emotions in the learning process (e.g., Burić, 2015).Emotions are no longer perceived as a side part of learning, but as one of the main variables assessed in this context.The most prominent area of research are academic emotions, which pertain to the learning and classroom activities, and achievement outcomes (Pekrun et al., 2002).Academic emotions are defined as multifaceted phenomena consisting of affective, cognitive, physiological, motivational, and expressive components (Scherer, 2009).Besides this, they can be differed among valence (i.e., positive and negative) and activation dimensions (i.e., activating and deactivating; Scherer, 2000) and grouped according to their object focus (activity or outcome; Pekrun & Perry, 2014).Based on the latter one, Pekrun and Stephens (2012) distinguish between four groups of academic emotions, i.e., achievement, epistemic, topic and social emotions which can be assessed as a state (momentary response to changes in situation) or as a trait (a more general emotional response which is the same over time; Rosenberg, 1998).In recent years, there are much research assessing achievement emotions (e.g., Forsblom et al., 2021), while the other three groups are still largely unexplored.In this research, we are mainly focused on epistemic emotions, which main object focus is on learning and cognitive activities (Pekrun & Stephens, 2012).

Epistemic Emotions
Epistemic emotions are related to the knowledge-generation aspects of studying and cognitive activities and arise as a result of cognitive qualities of task information and processing of that information (Pekrun & Stephens, 2012).More specifically, epistemic emotions are "emotions that result from information-oriented appraisals (i.e., the cognitive component of an emotion) about the alignment or misalignment between new information and existing beliefs, existing knowledge structures, or recently processed information."(Muis, Chevrier, et al., 2018, p. 5).There are seven well-established epistemic emotions often examined in previous research in the context of academic emotions (e.g., Vogl et al., 2019): surprise, curiosity, enjoyment, frustration, anxiety, confusion, and boredom.One can experience surprise, curiosity, and confusion when new information is not in line with their prior knowledge (D'Mello & Grasser, 2012) or with their epistemic beliefs (Muis, Pekrun, et al., 2015), same as anxiety and frustration, while enjoyment, curiosity and lower levels of boredom occur in opposite situations.

Measures of Epistemic Emotions
Epistemic emotions are examined within two research directions: (1) research focused on detailed analyses of a small number of affective states which occur in a specific situation of learning, primarily in the context of learning with digital technologies, and (2) research of academic emotions which assess a wide array of determinants and outcomes of such emotions (D'Mello, 2013).Within the first research domain, emotions are typically examined by single-item instruments in the form of checklists (i.e., participants are given a checklist with the names of emotions and their short description, e.g., confusion "defined as a noticeable lack of understanding"; D 'Mello & Graesser, 2012, p. 6).
In the second research direction, within the context of academic emotions, the most typical measure of epistemic emotions is the multi-item self-report instrument Epistemically-Related Emotion Scales (EES) developed by Pekrun et al. (2016) aimed at measuring seven aforementioned epistemic emotions.It consists of 21 items, with three items measuring each epistemic emotion.Items are formed as oneword adjectives describing those emotions (e.g., an example of items measuring confusion is "puzzled"), which Pekrun et al. (2016) selected based on their frequent usage as emotion words and in existing questionnaires measuring emotions.The EES can be also used as a short version with only one item assessing each emotion.In the EES, participants have to estimate the intensity of each emotion on a 5-point scale (from not at all to very strong).The validation study showed that the EES represents a reliable measure for assessing various major epistemic emotions, with adequate internal and external validity (Pekrun et al., 2016).
Both types of instruments described above assess the epistemic emotions related to a specific task (i.e., as a state), and there are no studies, to our best knowledge, assessing them as a trait.When measuring epistemic emotions as a momentary state, specific tasks that are used to induce epistemic emotions are often exclusively prepared for research purposes and are not always typical for learning situations which students encounter in their everyday school work.This disables the generalizability of those results to an authentic school context, as also proposed by Muis, Sinatra, et al. (2018).Therefore, the aim of this research was to develop a new trait-oriented instruction of the EES for assessing epistemic emotions in a typical learning situation in the school context.

Epistemic Emotions in This Study
Besides the state or trait-oriented nature of epistemic emotions, there are a few more characteristics that need to be defined.Earlier research on achievement emotions showed that emotions are subject-specific (e.g., Goetz et al., 2007).In this research, epistemic emotions will be assessed in the context of physics among middle school students.We have chosen physics since students often have problems with understanding the content (e.g., Erceg & Aviani, 2014) and consider physics a difficult subject, while they perceive their self-efficacy in physics as quite low (e.g., Jokić et al., 2019).That is a typical situation in which epistemic emotions occur (Pekrun & Stephens, 2012).Regarding the dimension of activation and valence, we will follow Pekrun et al. (2016) classification and define enjoyment and curiosity as positive activating emotions, anxiety, frustration and confusion as negative activating emotions, and boredom as negative deactivating emotion.Surprise is typically defined as a neutral activating emotion since its valence depends on the learning context (Muis, Sinatra, et al., 2018).

The Control-Value Theory of Achievement Emotions
In order to test the criterion validity of the EES, correlations between proposed antecedents and outcomes of epistemic emotions will be examined.The most comprehensive theory for analysing antecedents and outcomes of all academic emotions is the control-value theory of achievement emotions (Pekrun, 2006;Pekrun & Perry, 2014).According to this theory, the proximal antecedents of epistemic emotions are cognitive appraisals of control and value, i.e., the students' beliefs of being in control of or out of control of activity and the outcomes that are important to them (Pekrun et al., 2011).
In this study, we will examine the relationship between epistemic emotions and cognitive appraisals.Based on the theoretical assumptions, both cognitive appraisals should have a positive relationship with epistemic curiosity and enjoyment, and a negative with confusion (Muis, Psaradellis, et al., 2015).On the other hand, a low perception of control, and a high perception of value could lead to anger, frustration, and anxiety.If both cognitive appraisals are low, that could lead to the epistemic boredom.Surprise is a neutral epistemic emotion so it is predicted that it will not have a significant relationship with cognitive appraisals (Di Leo et al., 2019;Muis, Chevrier, et al., 2018).There is little research testing these assumptions which are not always consistent in their findings (e.g., Muis, Sinatra et al., 2018).
On the other hand, academic emotions affect different cognitive and motivational aspects of learning and academic achievement (Pekrun & Perry, 2014).In this study, we will examine achievement in physics as an outcome of epistemic emotions.There are few studies on epistemic emotions predicting cognitive and motivational aspects of learning and achievement (e.g., Nerantzaki & Efklides, 2019).Results of those studies show somewhat inconsistent results, e.g. in one research curiosity had a positive relationship with success in learning (Muis, Psaradellis, et al., 2015), while in another that relationship was non-significant (Di Leo et al., 2019).The same as with cognitive appraisals, inconsistent results are not unexpected since there are still few studies assessing those correlations, but in significantly different task-specific situations.

Aims of the Present Research
Research on epistemic emotions is still scarce and there are only a few measuring instruments (e.g., D 'Mello & Graesser, 2011;Pekrun et al., 2016) available for assessing those emotions as momentary states in situation-specific context.The most frequently used one in the field of academic emotions is the EES (Pekrun et al., 2016) which is aimed to assess the epistemic emotions as a state and as a trait, i.e., as emotions that occur during working on some momentary task or as a typically emotional functioning in a specific learning situation (Pekrun et al., 2018).Since there is no research assessing epistemic emotions as a trait with the EES, the main aim of this research was to develop trait-oriented instruction in order to examine a typical emotional experience in the epistemic context in the authentic school setting.Therefore, we implied a mixed-method study methodology in order to achieve the proposed aim.The reasons for that were twofold: (1) there were no data on epistemic emotions among elementary or secondary school students or college students in Croatia, and (2) there was no research examining epistemic emotions as a trait.The first study was a qualitative one, where we explored situations in which students typically experience epistemic emotions in school setting.Based on these results we developed a new trait-oriented instruction for students, aimed at assessing typical epistemic emotional experiences in an everyday school context.The second study was a cross-sectional quantitative study aimed at analysing psychometric properties of the EES with newly formed trait-oriented instruction.To test the structural and internal construct validity, we analysed correlations and confirmatory factor analysis (CFA).Furthermore, we tested criterion validity by analysing the relationships between the epistemic emotions and their antecedents, i.e. cognitive control and value appraisals, and outcomes, i.e., achievement in physics.

Study 1
In Study 1, we conducted focus groups with elementary school students in order to get acquainted with the typical situations in which they experience epistemic emotions during school classes in physics.Based on the qualitative analysis of the data, we extracted one typical learning situation which integrates several of those typical situations and could induce a wide range of different epistemic emotions, not only the specific ones (e.g., when students do not know the answer to a question, they will typically experience negative epistemic emotions, but not the positive ones).Besides this, it enables successful differentiation between students' typical emotional experiences in an epistemic context based on the individual differences in proposed determinants of those emotions.

Participants and Procedure
Thirty-one 8 th -grade students (16 of them were female) from two different elementary schools in Zagreb, Croatia, participated in this study.The students were 14 years old on average (M = 13.68,SD = 0.48), had a grade point average (GPA) in physics M = 4.16 (SD = 0.97) at the end of the 7th grade, while their overall GPA at the end of 7th grade was M = 4.58 (SD = 0.48).In Croatia, grades are ranging from 1 (insufficient) to 5 (excellent).GPA from 3.50 to 4.49 is considered very good and GPA from 4.50 to 5.00 is considered excellent.
Both studies were approved by the Ethical Committee of the Catholic University of Croatia, the Faculty of Humanities and Social Sciences University of Zagreb and the Croatian Ministry of Science and Education.The research was conducted in October 2019 at two elementary schools in Zagreb.Each focus group consisted of 4 to 7 students from the same class departments.When students came to the school classroom or office which was intended for the focus group, they were once again familiarised with all the details about the research and the audio recording of the conversation.All students had parental written permission to participate in the study and they also gave their permission after all the necessary information was presented to them.The first author conducted all focus groups following focus group protocol and students voluntary answered the questions.Along with this, students filled out questionnaires about demographic data.Focus groups lasted 45 minutes on average.

Focus Group Protocol
Students answered structured questions regarding situations in which they experienced epistemic emotions in the context of physics, their perception of the frequency of experiencing those emotions and the relationship between motivation for learning physics and epistemic emotions.Specifically, questions were divided into the following thematic groups: (1) general students' attitude regarding physics, (2) learning-related situations in a classroom or at home in which students experience emotions (i.e., different learning-related situations from physics, such as learning new material, misconceptions, etc., were presented to students, and they were asked to name the emotions they experienced in those situations), (3) specific situations in which students experience epistemic emotions (i.e., students were asked to list and describe the situations in which they experienced each of the seven target epistemic emotions, such as: "Can you remember the situations in which you experienced surprise in physics class?"), (4) epistemic emotions that students experience while resolving physics problems, (5) students' perception of the frequency of experiencing specific epistemic emotions, (6) students' perception of the determinants of epistemic emotions related to physics, and (7) the effects of epistemic emotions on their motivation for learning physics.

School-Related and Socio-Demographic Data
Students filled out a questionnaire assessing questions regarding their sex, age, GPA from physics and overall GPA at the end of the previous school year.

Analyses
We conducted thematic analyses of transcribed data following guidelines from Braun and Clarke (2006).We were specifically focused on analysing the situations in which the students experience seven different epistemic emotions classified by Pekrun et al. (2016).Besides this procedure, to test the reliability between different researchers, we extracted 10% of the students' statements, which were randomly ordered.The task of the other researcher was to relate those statements to a specific code.We calculated the Kappa coefficient, which shows the degree of agreement between researchers (Cohen, 1960) which we then corrected for the possibility of random guessing.The Kappa coefficient was .64,indicating a significant rate of agreement.After the discussion and agreement between the two researchers, the Kappa coefficient was .93,indicating an almost perfect degree of agreement (Landis & Koch, 1977).

Results and Discussion
To determine typical situations in which students experience epistemic emotions in the context of physics, we analysed all parts of transcribed data related to the experience of those emotions.Students experienced surprise when they encountered information they did not expect or could not explain (e.g., when the new information was unknown to them or when they heard contradictory information related to the learning material), which are the typical situations in which surprise occurs (Foster & Keane, 2015;Meyer et al., 1997).In addition, an interesting finding was that students experienced surprise while studying materials that were interesting to them.The appraisal of interestingness is a part of the appraisal of cognitive value (Pintrich et al., 1991), which is one of the main antecedents of positive and negative academic emotions (Pekrun, 2006), except for the surprise, since it is a neutral emotion (Muis, Chevrier, et al., 2018).Earlier quantitative research also found contradictory results regarding the value appraisal as an antecedent of surprise (e.g., Muis, Psaradellis et al., 2015) so we were interested in comparing this result with the results of the following quantitative research.
Regarding the positive epistemic emotions, the enjoyment occurred when the new information was in line with students' prior knowledge (e.g., when giving the right answer to the question) or when they successfully resolved the task and overcame challenges (e.g., when knowing how to solve the task), which are typical situations in which enjoyment occurs (D'Mello & Graesser, 2011;Muis, Pekrun, et al., 2015).Besides this, an interesting finding was that the students experienced enjoyment in challenging learning situations, when they needed to work hard to resolve the physics problem.This was in line with earlier findings, which showed that the perception of a challenge was a positive predictor of enjoyment in activities that were intrinsically motivated and goal-oriented (Abuhamdeh & Csikszentmihalyi, 2012).On the other hand, curiosity, as a second positive epistemic emotion, occurred when students had the desire to find out some specific information (e.g., when the learning material was interesting), or a desire for new information which was induced by novelty, complexity, and ambiguity of that information (e.g., when learning material based on the examples from everyday life).Those situations represent examples of the two theoretical perspectives explaining the occurrence of curiosity (Litman & Jimerson, 2004;Loewenstein, 1994).Besides this, students experienced curiosity when they were faced with the difference between their prior knowledge and what they still do not know, which is another typical situation for curiosity to occur according to Markey and Loewenstein (2014).
Related to the negative epistemic emotions, anxiety occurred in only two situations, which were characterized by the difference between new information and students' prior knowledge (e.g., when students did not understand the material or they did not know how to solve the problem), which Chevrier et al. (2019) state as a usual setting for the anxiety to occur.In our study, boredom was mostly experienced when students lacked interest for some specific material, which resulted in the lack of challenge in those tasks, and rarely because of demanding work on some task that they could not resolve for a long time, as it is proposed by D'Mello and Graesser (2012).In the end, in our study, frustration occurred when students were not able to resolve the impasse in learning or resolving the task (e.g., when solving problems they did not know how to solve), while confusion occurred because of the lack of understanding (e.g., giving the wrong answer), which were also typical situations for those emotions to occur (D'Mello & Graesser, 2012).
After analysing the situations in which the epistemic emotions occur, we concluded that the students experienced some of the epistemic emotions in similar situations (e.g., surprise, curiosity and confusion when giving a wrong answer to the question), while they experienced other emotions in situations specific to a particular epistemic emotion (e.g., surprise when encountering with contradictory information).Among all the analysed situations, we tried to extract one typical situation in the physics class in which students could experience a wide array of epistemic emotions, which would differ in their intensity due to students' individual differences in proposed determinants of those emotions, i.e., cognitive appraisals of control and value (Pekrun & Perry, 2014).We did not want to focus on specific situations that are prototypical for the occurrence of similar epistemic emotions among different people, regardless of the level of their cognitive appraisal.For example, when students do not understand the explanation of some learning material, they will typically experience confusion no matter their level of cognitive control or value.
Based on these findings, we concluded that the common characteristic of the analysed situations is that each of them could appear while studying new material in physics' class.That is, the students might give a right or wrong answer to the question, they could do experiments, encounter contradictory information, find the topic interesting or boring etc.Moreover, during focus groups, we noticed that students differed in experienced epistemic emotions in those situations based on their love for the subject of physics and their perception of themselves as being successful in physics.Although we did not intentionally examine those determinants in the qualitative research, which prevented us from making concrete conclusions, it provided us with some important information and was in line with the theoretical assumptions of the control-value theory of achievement emotions (Pekrun, 2011).Learning new material is in some way a neutral situation which will not induce similar emotional experiences among a majority of people, in contrast to, for example, the situation in which student is struggling with a difficult task which would induce frustration among the majority of them.While learning new material, people could differ in their emotional reactions due to other characteristics which determine the occurrence of the specific emotion, in this case appraisals of control and value, as claimed above.Also, it was proposed that the differences in the level of those proximal antecedents among students will result in experiencing a wide array of epistemic emotions.
In addition to our assumptions, Muis, Chevrier, et al. (2018) claim that there are five information-oriented appraisals as determinants of epistemic emotions.Besides appraisals of control and value, those include the appraisals of novelty and complexity and achievement or impasse of the learning aim.The newly formed instruction in the EES focused on learning new material in the physics class covers all of those proposed appraisals, while we assumed that the students will experience a different intensity of a wide array of epistemic emotions based on the differences in the perception of those appraisals.That is, we assume that students will differ in the intensity of those emotions regarding their perception of the controllability of epistemic actions and outcomes (i.e., control appraisal), and their perception of the importance of those activities and outcomes (i.e., value appraisal).Moreover, this instruction includes the appraisal of a novelty since it is specifically related to learning a new material.Furthermore, students in Croatia find physics very challenging and demanding, and their self-efficacy in physics is very low (Jokić et al., 2019) which covers the assumption of complexity.

Study 2
After forming the new instruction in the EES, the main goal of the Study 2 was to test its psychometric properties.That is, we analysed its structural, internal construct, and criterion validity to check if the instrument is appropriate for assessing epistemic emotions as a trait, not only as a state.

Participants and Procedure
A convenience sample of 8 th -grade students from 12 elementary schools in Zagreb and Zagreb County participated in this study.In total, 545 students participated (268 female).Students had 14 years of age on average (M = 13.99,SD = 0.39), a very good GPA in physics (M = 3.92, SD = 0.99) and a very good overall GPA (M = 4.28, SD = 0.68) at the end of 7th grade.
After obtaining relevant approvals for the conduction of the study, one of the parents gave written consent for their children's participation in the research at the beginning of the school year 2019/2020.The study was conducted in February 2020, right before the start of the pandemic of the novel coronavirus.The students who had parental approval for participation in the study also gave their written consent and filled out the questionnaire during regular school classes.In total, only six students with parent consent refused to participate in the research.The whole procedure lasted about 45 minutes.

Epistemic Emotions
The epistemic emotions were assessed with Epistemically-Related Emotion Scales by Pekrun and colleagues (2016).The scale was doubly translated from English to Croatian and this was the first time it has been used in the research with Croatian elementary school students.We used the original scale, which measured seven epistemic emotions: surprise (e.g., surprised), curiosity (e.g., curious), enjoyment (e.g., happy), confusion (e.g., confused), anxiety (e.g., anxious), frustration (e.g., frustrated), and boredom (e.g., dull).Each emotion was measured with three items in the form of an adjective and the students gave their answers on a 5-point Likert scale ranging from 1 -not at all to 5 -very strong.The original instruction of the Scale was adapted for assessing epistemic emotions in the situations of learning new material at the physics class based on the results of the qualitative study (i.e., "Remember the situations when you were learning new material at the physics class.Please indicate the strength of each emotion by circling the number that best describes the intensity of your emotional response.").The average score is calculated for each emotion, and a higher score indicates a higher intensity of each epistemic emotion.

Control Appraisal
Students' self-efficacy, as an indicator of appraisal of cognitive control, was assessed with the Self-efficacy for Learning and Performance Scale (Pintrich et al., 1991).It consists of 8 items which were adapted to the context of physics (e.g., "I believe I will receive an excellent grade in physics.").The students gave their answers on a 7-point Likert scale (1 -not at all true of me, 7 -very true of me), and the total score is calculated as an average of answers on all items.A higher score indicates higher self-efficacy.The Cronbach alpha coefficient was .93.

Value Appraisal
The appraisal of cognitive value was assessed with the subscale Values from the Components of the Self-regulated Learning Scale (Niemivirta, 1996;Croatian version -Rijavec et al., 2003).It has 6 items which were also adapted to the context of physics (e.g."In my opinion, what we learn from physics is useful.").The students gave their answers on a 5-point scale ranging from 1 -completely disagree to 5completely agree.In calculating the total result as an average of all answers, three items needed to be reversely coded.A higher score indicates a higher value appraisal.The Cronbach alpha coefficient was .90.

Achievement in Physics
Achievement in physics was measured with all the grades the students got from the beginning of the school year (in September 2019) until the moment of the research.Students in Croatia accomplish grades in physics in 3 different categories: (1) knowledge and skills, (2) conceptual and numerical tasks, and (3) research of physical phenomena.In this research, they had to write down all the grades separately for each category in order to form the latent variable from those three categories as indicators.It should be noted that only 259 students from the overall sample gave information about their grades.The Cronbach alpha coefficient was .76.

Statistical Analyses
We conducted confirmatory factor analysis (CFA) and structural equation modelling (SEM) in Mplus version 8.4, while other analyses were conducted in IBM SPSS Statistics program version 23.The following fit indicators were used in order to test if the model has a good fit to the data: χ 2 had to be non-significant (Brown, 2015), RMSEA had to be below .06 to indicate a good fit, or below .08 to indicate acceptable fit, CFI and TLI had to be higher than .95 to show a good fit (Hu & Bentler, 1999) or higher than .90as an indicator of an acceptable fit (Bentler, 1990), and SRMR below .08 to indicate a good fit.In order to compare the fit of the two models, we used χ 2 difference test, which tests if the difference of χ 2 values of the two tested models exceeds the critical p value for the difference between the degrees of freedom between the two models.If the two models differ significantly, the model with a lower χ 2 value is better (Brown, 2015).

Descriptive Statistics, Reliability, and Manifest Correlations between Epistemic Emotions
Descriptive data shows that scores vary across the whole range of the scale, except for the surprise whose maximum value was 4.67 (Table 1).All scales have Cronbach's alpha coefficients above .70and range from .71 to .88.Only surprise had two insignificant correlations with confusion and anxiety, while it correlated positively with curiosity and enjoyment and negatively with frustration and boredom.

Structural Validity of the Epistemically-Related Emotion Scales
To test the structural validity of the EES we conducted confirmatory factor analyses to test the differences in those models.Firstly, we tested the differences between 1-factor (all emotions loaded on the same factor), three variations of 2-factor models and 7-factor model (each of the seven epistemic emotions loaded on one of the seven factors).Regarding 2-factor models, we were focused on the valencerelated distinction between emotions on positive and negative emotions.Therefore, we defined three models with regard to emotion surprise, which is classified as a neutral emotion, following the same procedure as Pekrun et al. (2016).That is, in one 2-factor model, positive emotions (curiosity and enjoyment) and negative emotions (anxiety, frustration, confusion and boredom) loaded on two factors, whereas surprise was not included.In the second 2-factor model, surprise was included as an indicator of negative emotions, while in the third, surprise was included as an indicator of positive emotions.The fit indices for all tested models are presented in Table 2.The results show that the 7-factor model fit the data better than the 1-factor (Δχ 2 = 2302.16,Δdf = 21, p < .01), the 2-factor model without surprise (Δχ 2 = 350.865,Δdf = 34, p < .01), the 2-factor model with surprise as an indicator of negative emotions (Δχ 2 = 1205.78,Δdf = 20, p < .01)and the 2-factor model with surprise as an indicator of positive emotions (Δχ 2 = 606.676,Δdf = 20, p < .01).Although the 7-factor model had an acceptable fit to the data, it could not be accepted because the warning about high correlations between some latent variables.In this study, that was related to the latent correlations between frustration and anxiety (r = .97)and anxiety and confusion (r = .91).Those high correlations indicated low discriminant validity (Brown, 2015).All other correlations were below .85.Frustration, anxiety, and confusion are all negative and activating epistemic emotions.Therefore, we tested the model in which frustration and anxiety were comprised into one latent factor.The model showed an acceptable fit to the data, but the correlation between the new variable and confusion was .88.Based on these results, all three emotions were loaded onto a single factor named negative activating emotions.The model also showed an acceptable fit to the data, but the χ 2 difference test was significant (Δχ 2 = 43.667,Δdf = 5, p < .01).For that reason, we decided to check the differences in CFI values between the two models since χ 2 difference test can be affected by the sample size (Cheung & Rensvold, 2002).The results showed that ΔCFI is .006indicating that the two models do not differ significantly since the difference is lower than .01(Chen, 2007).In line with these results, we have decided to maintain the 5-factor model, consisted of emotions of surprise, curiosity, enjoyment, negative activating emotions and boredom, as a final model (χ 2 = 646.389,df = 179, p < .01,RMESA = .071,CI [.065-.076],p < .01;CFI = .927;TLI = .914,SRMR = .062).
These results implicate that students could not distinguish between various negative activating epistemic emotions.As D'Mello and Graesser (2012) claim, surprise is the first emotion that occurs in the case of cognitive incongruity.If incongruity cannot be resolved, confusion occurs.Prolonged confusion can lead to frustration, and if incongruity is intensive and is significantly violating the existent beliefs, anxiety occurs.This description based on the model of affective dynamics during complex learning (D'Mello & Graesser, 2012) assumes that all three emotions occur in similar learning situations, which might make it hard for younger students to differentiate them since their emotion regulation and learning strategies are still developing (Di Leo et al., 2019).Further, Nook et al. (2018) showed that the differentiation of negative emotions has a quadratic correlation with age, in a way that it drops from childhood to adolescence and rises again in the adulthood.Children can easily differentiate emotions if they experience one emotion at a time, while adults probably have better-developed skills for differentiation of emotions which appear simultaneously.Therefore, emotions in adolescence often appear at the same time, while the skills for their differentiation are still not developed enough.The participants in our study were also adolescent students and this might be the reason why the correlations between these three negative activating emotions, which appear in similar learning situations, were high, and students could not differentiate them successfully.Regarding the internal construct validity, we analysed latent correlations between factors (Table 3).Correlations between all emotions are significant (p < .01)and in expected direction.

Criterion Validity of the Epistemically-Related Emotion Scales
To test the criterion validity, we analysed latent correlations between epistemic emotions with cognitive appraisals and achievement in physics (Table 3).The cognitive appraisals were positively correlated with surprise and positive epistemic emotions, while negatively with negative epistemic emotions.These results are mostly in line with the control-value theory of achievement emotions (Pekrun & Perry, 2014).The theory assumes that control and value appraisals are not determinants of surprise since it is a neutral emotion, which is not always the case in previous studies.Specifically, neither appraisal of control nor appraisal of value were predictors of surprise (Muis, Psaradellis, et al., 2015), or only control (Di Leo et al., 2019) or value appraisal (Muis, Sinatra, et al., 2018) predicted surprise.Additionally, the model with surprise as an indicator of positive affect fitted better to the data than the model in which surprise was an indicator of negative affect (Δχ 2 = 599.104,Δdf = 0, p < .01),but again neither model had an acceptable fit.Qualitative data from the Study 1 also showed that students perceive value appraisal as an antecedent of surprise.As proposed by Pekrun and colleagues (2016), surprise could have positive valance in an epistemic context, but research is still scarce and this assumption should be further tested.
On the other hand, a positive relationship between positive activating epistemic emotions and control and value appraisals is expected and in line with theoretical assumptions, the same as a negative relationship with boredom as deactivating emotion (Pekrun, 2006).Earlier research is mostly in line with these results, i.e., value appraisal positively predicted or was positively correlated with curiosity and enjoyment, and negatively with boredom.In contrast to that, control appraisal mostly was not correlated with those emotions, or it negatively predicted boredom (e.g., Muis, Psaradellies, et al., 2015).
For negative activating emotions, different combinations of correlations with appraisals of control and value are expected.That is, low control and high value could predict anxiety, frustration and confusion.Other combinations are also possible, such as low control and low value as predictors of confusion.In this study, negative activating emotions correlated negatively with both cognitive appraisals, but we did not have the possibility to test their individual relationships since all three negative activating epistemic emotions formed a single latent factor.Therefore, these assumptions should be tested in future studies since the results of all earlier research were focused on state measure of emotions and their determinants (e.g., Di Leo et al., 2019), and in this research we used a trait-oriented approach.

General Discussion
In this paper, we presented two studies in which a mixed-method approach with an independent sample of elementary school students was utilised.We suggested new instructions for the EES, which is intended for measuring epistemic emotions as a trait, not as a state.Although Pekrun et al. (2018) claim that epistemic emotions can be measured as a trait or as a state, this study is, to our knowledge, the first one measuring epistemic emotions in a trait-oriented approach.Furthermore, this was one of the rarely conducted studies based on a mixed-method approach, and the first one using focus groups to analyse specific situations in which students experience different epistemic emotions.The qualitative approach gave us important insight into the variety of situations in which epistemic emotions occur and made the quantitative data from this paper and earlier studies more comprehensible.Besides this, the Croatian translation of EES was used for the first time, in order to test its psychometric properties.Although the structure of the Scale was not the same as in the original version, we assume this was the case because of the adolescent age of the participants, in which they are still not able to differentiate between simultaneously occurring similar emotions (Nook et al., 2018).In the other two studies with elementary school students (i.e., Di Leo et al., 2019;Muis, Psaradellis, et al., 2015), the authors did not present the results of confirmatory factor analyses, which would enable us to compare those results with the results from our studies.The only information given was that the authors had to modify the Scale by simplifying the items so that the younger students could understand them, and the Cronbach alpha's reliabilities, which were in an acceptable range.In some other studies conducted with older students, i.e., 21 years old on average (e.g., Pekrun et al., 2016), the measurement models or confirmatory factor analysis showed the expected 7-factor structure of the Scale.It would be useful to test the factor structure of the EES with students of different age in order to test if there are differences between them in the ability to differentiate between similar epistemic emotions.Regarding the criterion validity of the Scale, epistemic emotions mostly showed expected correlations with cognitive appraisals as the main determinants of those emotions, and achievement in physics, which is one of the proposed outcomes.
Besides the important findings discussed above, there are some limitations that should be taken into consideration.In both studies, a convenient sample of elementary school students participated which could lead to biased results (Ajduković et al., 2021).Furthermore, in each of our studies the participants were students of similar age.Therefore, we could not examine whether the factor structures of the EES are different between younger and older students.Another shortcoming is that the epistemic emotions were assessed exclusively in the context of the subject of physics due to the subject-specific nature of academic emotions (e.g., Goetz et al., 2006).This did not allow us to test whether there are differences between epistemic emotions among different school subjects.Besides this, the newly developed trait-oriented instruction was also formed based on the situations in which students experience epistemic emotions in the context of physics.This might raise questions about the application of this instruction in other contexts in order to measure epistemic emotions as a trait.
Nevertheless, these results gave important insight into measuring epistemic emotions in a trait-oriented way and their relationship with their antecedents and outcomes.Since the research on epistemic emotions has recently gained more interest, primarily in a state-oriented way, we hope that this newly developed instruction will be useful for researchers to examine epistemic emotions in a traitoriented approach.Besides this, to our best knowledge, this was the first study of the epistemic emotions in Croatia, and these results could encourage researchers to use this instrument to assess them in different school subjects with different populations for testing theoretical assumptions.This will expand the knowledge about this largely unexplored, but important area of emotional experiences in the school context.

Table 2
Fit Indices for all Tested Models ** p < .01.