Dear Label_Name,

Greetings! I hope you are well.

I am writing on behalf of an international publishing house, Science and Education Publishing, USA (http://www.sciepub.com).

Science and Education Publishing (SciEP) is a multidisciplinary open access publisher of Journals covering the fields of Science, Technology and Medicine. All SciEP publications are subject to high-quality peer review, editorial and production processes and are freely available online.

Welcome to the April, 2013 edition of SciEP newsletter, providing monthly summaries of articles on science & education related topics of interest to scientist and professionals. The mission of our journals is to provide an open access forum for publishing new findings on issues pertinent to scientific and academic field. We also invite you to submit original research papers, review articles, short reports, or case studies are considered for publication in our Multidisciplinary Journals.

American Journal of Public Health Research
DOI: 10.12691/ajphr-1-1-3

The Implementation of a Tele-Homecare System with Service Oriented Architecture and HL7 Message Transmission Standard
Ching-Sung Wang

American Journal of Medical and Biological Research
DOI: 10.12691/ajmbr-1-1-2

The Influence of Ethanol on Pyruvate Kinases Activity in Vivo, in Vitro, in Silico
Lelevich Sergey Vladimirovich, Khrustalev Vladislav Victorovich, Barkovsky Eugene Victorovich, Shedogubova Tatyana Aleksandrovna

American Journal of Medical Sciences and Medicine
DOI: 10.12691/ajmsm-1-1-1

Clinical and Histological Features in Wegener Granulomatosis
Li Cavoli Gioacchino, Passantino Rita, Bono Luisa, Tortorici Calogera, Ferrantelli Angelo, Giammarresi Carlo, Rotolo Ugo

American Journal of Infectious Diseases and Microbiology
DOI: 10.12691/ajidm-1-1-2

The Curse of Lymphatic Filariasis: Would the Continual Use of Diethylcarbamazine Eliminate this Scourge in Papua New Guinea?
Anne Lanham, Lillian Mwanri

We have released issues of some journals on February 2013.
American Journal of Public Health Research
American Journal of Medical Sciences and Medicine
American Journal of Medical and Biological Research
American Journal of Pharmacological Sciences
American Journal of Clinical Medicine Research
American Journal of Nursing Research
American Journal of Cancer Prevention
American Journal of Medicine Studies
American Journal of Infectious Diseases and Microbiology
American Journal of Cardiovascular Disease Research
American Journal of Biomedical Research
American Journal of Mechanical Engineering
American Journal of Microbiological Research
Journal of Preventive Medicine
American Journal of Food and Nutrition
American Journal of Hypertension Research
American Journal of Sports Science and Medicine
Journal of Cancer Research and Treatment
American Journal of Zoological Research
Journal of Biomedical Engineering and Technology
American Journal of Microbiological Research
Biomedical Science and Engineering

We are pleased to announce that submissions are now being accepted for peer review for all the journals across all our subject areas in SciEP.

Please submit your manuscript via our online submission system:
http://www.sciepub.com/author/signin

To unsubscribe from our mailing list, please click on the following link: unsubscribe

I am looking forward to hearing from you.

Kind regards,

David Rain
Email:join@sciepub.com
Science and Education Publishing (http://www.sciepub.com)

Effectiveness of Literature Critique Peer Discussions to Build Scientific Literacy Skills, Engagement and Improve Learning-Related Emotions during COVID-19-Associated Online Learning

Jennifer M. Monk*, David M. Beauchamp, Rachel K. von Holt, Kelsey Van

Department of Human Health and Nutritional Sciences, University of Guelph, Guelph, ON, Canada

Abstract: Learning-related emotions can affect student skill development, engagement, and academic performance. During the COVID-19 pandemic, studies have reported increased student anxiety and social isolation concurrent with reduced engagement during online learning. Therefore, the purpose of this study was to determine the effectiveness of optional literature critique and peer discussion assignments to promote student engagement, scientific literacy (SL) skill development and positive attitudes toward COVID-19-assocaited online learning. Students’ (n=161) learning-related achievement emotions, SL capabilities, learning approach (deep and surface), and perceived stress were assessed using online surveys. On average students competed 3.5 out of 5 of the optional literature critique and peer discussion assignments. Higher engagement with these assignments was positively associated with achieving higher final grades. Experiencing higher/more frequent positive learning-related emotions (enjoyment, hope, pride) were i) positively associated with students’ SL capabilities, deep learning approach, and final grade, and ii) inversely associated with perceived stress and surface learning approach. Conversely, experiencing higher/more frequent negative learning-related emotions (such as anxiety, anger, shame, hopelessness, boredom) were i) positively associated with perceived stress and surface learning approach, and ii) negatively associated with SL skills, deep learning approach, and final grade. These findings demonstrate that optional literature critique assessments can promote online learning engagement and highlight the impact of students’ attitudes towards learning on skill development, stress, and grades. Therefore, the positive learning-related emotions of hope and pride were positively associated with higher final grades in the course, whereas the negative learning-related emotions of anger, shame, hopelessness, and boredom were all inversely related with final grades in the course. Students experience of anxiety exhibited no relationship with final grades in the course, demonstrating that despite an emphasis in higher education to reduce students’ academic anxiety consideration of other learning-related emotions may be more relevant for impacting academic outcomes.

Keywords: learning-related emotions, stress, scientific literacy, engagement, COVID-19, online learning, literature critique

Received April 08, 2023; Revised May 13, 2023; Accepted May 23, 2023

Citation: Jennifer M. Monk, David M. Beauchamp, Rachel K. von Holt, Kelsey Van, Effectiveness of Literature Critique Peer Discussions to Build Scientific Literacy Skills, Engagement and Improve Learning-Related Emotions during COVID-19-Associated Online Learning. American Journal of Educational Research, 11 (5). 303-315. January 2023. doi:10.12691/education-11-5-8

1. Introduction

Developing scientific literacy (SL) skills are an important part of an undergraduate science curriculum [1] and include, but are not limited to, understanding of the nature of science and its applications, critical thinking capability, and the ability to apply scientific knowledge to solve problems [1,2,3]. SL proficiency is important for success in a career in the scientific field [2,4]. Engagement with the scientific literature during or outside of undergraduate lectures has been shown to be an effective method for promoting the development of students’ SL skills [5,6].

Learning-related emotions can be a significant factor influencing students. Emotions can affect students’ self-efficacy, academic engagement, and academic achievement [7]. Achievement emotions are emotions students experience while in class, while learning, or during a test [8]. The control-value theory of achievement emotions suggests that positive emotions are related to students having control over and placing positive value in the activity or outcome [8]. Positive learning-related emotions experienced by students, such as pride and enjoyment, have previously been associated with higher academic performance (i.e., grades) in their courses, whereas negative learning-related emotions, such as hopelessness and anger, did not display any relationship with academic performance [7]. Reduced academic performance has been shown in some students experiencing anxiety, wherein the consequence of experiencing anxiety reduced their motivation to learn [9,10,11,12]. Anxiety is predominantly classified as a negative learning-related emotion [8]; however, anxiety can provide motivation for students to complete assignments or study for assessments [10,11]. Self-efficacy, which is the self-defined ability to succeed at a given task, is an important skill for success in undergraduate education [13,14]. Importantly, higher levels of self-efficacy have been shown to mitigate the negative effects of anxiety [12], which infers a critical relationship between self-efficacy and learning-related anxiety that could extend to other learning-related emotions beyond an anxiety-centric focus. Learning-related emotions can either adversely or positively impact students’ self-efficacy [7], which in turn, can impact their approach to learning. Thus, undergraduate students exhibiting a high degree of self-efficacy is associated with higher engagement in the course and the adoption of deeper learning strategies [14,15]. In online learning, particularly asynchronous online learning where there are no set lecture times, self-efficacy is especially important for success, as students must create their own schedule [16,17]. The learning-related emotions students experience can also be associated with or influenced by the learning environment, as lecturers can motivate students to feel excitement and enjoyment for the course material and promote students’ motivation to learn in the course [18,19]. Students who exhibit a deep learning approach have demonstrated greater motivation and interest in what they are learning [20]. Conversely, a poor or negative learning environment, which can include low instructor presence and/or a reliance on passive lecturing with no student interaction, can promote students to experience more negative learning-related emotions, including anxiety and boredom [18,21]. Peer interaction through group learning has been shown to support students’ experience of positive learning-related emotions, including pride and happiness, and can encourage students to adopt a deeper learning approach [19,22]. High levels of participation in group learning activities, such as online discussions, has been shown to foster students’ engagement with the course content, and, promote positive learning-related emotions that are associated with greater academic performance [23]. Peer learning can also reduce the anxiety experienced when learning new concepts, as students may feel more comfortable and able to share ideas and discuss topics in smaller groups compared to a large classroom environment [24,25].

The COVID-19 pandemic resulted in a rapid shift to online learning in March of 2020 [26]. Consequently, the number of undergraduate students experiencing stress and negative emotions such as anxiety was shown to increase, which was attributed to social isolation and the shift to an online-only learning environment [27,28]. In addition to anxiety, other negative emotions experienced by students such as hopelessness, can lead to academic burnout and lower self-efficacy [19,29]. This contrasts with feelings of learning-related enjoyment and pride, which can improve students’ self-efficacy [19,29]. High stress levels have also been shown to negatively impact students’ self-efficacy [17,30]. Thus, ensuring that students have a high degree of self-efficacy is important for success in both face-to-face and online learning [14,16].

Previous research has demonstrated that peer learning in small groups using literature critique activities, both in-person and online, are an effective tool for promoting the development of students’ SL capabilities [5,31] and can improve their perceptions of COVID-19-associated online learning [32]. Therefore, utilizing assessment strategies, such as peer learning in small online discussion groups, in a scaffolded manner to promote the development of SL capabilities could represent an approach to concomitantly encourage students’ feelings of positive learning-related emotions and the adoption of a deeper approach to learning to mitigate stress and promote self-efficacy. Thus, by utilizing optional online literature critique peer discussion activities to promote engagement and foster positive attitudes toward online learning, the aim of this study was to determine the influence of positive and negative learning-related emotions on students’ i) SL skill development, ii) learning approach (i.e., deep versus surface approaches), iii) perceived stress levels, and iv) academic performance.

2. Methods

2.1. Participants, Course Assessments, and Online Surveys

Participants in this study were undergraduate students enrolled in the Fall 2021 semester in a fourth-year level toxicology course at a research-intensive university. Due to the COVID-19 pandemic, the course was delivered in an online learning environment with two weekly asynchronous lecture videos uploaded to the course website during the 12-week semester. Critical analysis of the scientific literature was a primary focus of the course learning outcomes. Course assessments included i) optional literature critique peer discussion assignments (5 x 3%), ii) a career readiness reflection assignment (5%), iii) two SL quizzes (at the start and end of the semester; 2 x 2.5%) assessing students’ practical skills using the validated Test of SL Skills (TOSLS) [33], iv) a SL reflection assignment (5%), v) a literature critique test (15%), and vi) a midterm and final exam (25% and 30%, respectively).

Students were invited via a private link sent to their university email addresses to participate in two online surveys administered at the beginning (Survey 1 in week 1) and end (Survey 2 in week 12) of the semester. Each survey included questions pertaining to students’ i) perceptions of their SL literature comprehension capabilities developed by the research team [5,31,34], ii) learning approach using the Revised Two-Factor Study Process Questionnaire (R-SPQ-2F) [20], iii) overall perceived stress experience using the Perceived Stress Scale (PSS) [35], and iv) their learning-related emotions using the Shortened Achievement Emotion Questionnaire (AEQ-S) [36], which is a condensed version of the original Achievement Emotions Questionnaire [37], an approach that has been used previously [38,39]. The online surveys utilized the Qualtrics Insight Platform (Provo, UT, USA). Of the 196 students enrolled in the course, 161 students (82.1%) completed both surveys and were included in the data analysis. All participants gave their informed consent to participate in the study and to utilize their responses to the online surveys, the content of all assignments and examinations (quizzes and tests), and their final grade in the course. A 2% participation bonus was added to students’ midterm exam grades (for Survey 1) and a 2% bonus was added to student’s final exam grades (for Survey 2). Alternative assignments were provided in the place of both online surveys for students to earn the participation bonus while abstaining from participating in the research project. This study was approved by the institutions Research Ethics Board (REB#21-06-022).

2.2. Literature Critique and Peer Feedback Assignments

To practice and develop scientific literature critique and SL skills during the semester, a scaffolded approach was implemented wherein students could complete up to five optional literature critique peer discussion assignments (that included a peer-feedback component) prior to a formal skill assessment on both a literature critique test (which provided specific feedback to each student from the instructor and/or teaching assistant) and the final exam. The optional literature critique peer discussion assignments required students to read a scientific abstract and respond to a series of questions to i) interpret study results, ii) identify the experimental model and associated strengths and weaknesses of the study design, and iii) to pose a follow-up research question and design an experimental study design to answer their follow-up research question. Students shared and discussed their answers in an online discussion board forum (n=8 students/online group) and provided feedback to their group members to improve upon their work and foster further development of SL, critical thinking, and written communication skills. The assignments were optional, as optional assignments have been demonstrated to improve engagement and mitigate the negative effects of online learning, such as low motivation to complete assignments or a disengaged student learning environment [40,41]. Following the peer (optional literature critique assignment) and instructor/teaching assistant (literature critique test) feedback opportunities during the semester, the formal assessment of students’ practical SL and critical thinking skills occurred at end of the course on the final exam. On the final exam students were required to analyze and critique a scientific study summary and provide written answers to questions requiring them to i) correctly identify the type of research study design, ii) identify both the strengths and weaknesses in the study design and discuss the impact of each strength and weakness on the interpretation of the results, and iii) based on the results of the study summarized identify a research question that builds on the experimental findings and then design and describe a follow-up study to answer their novel research question.

2.3. Statistical Analysis

Statistical analysis was conducted using GraphPad Prism (San Diego, CA, USA). For all data, the predefined upper limit of probability for statistical significance was P<0.05. Values are presented in tables and figures as means ± SEM. Paired t-tests were used to determine changes during the semester (i.e., at the start and end of the semester). Pearson correlation analyses were conducted to determine the relationships between parameters.

3. Results

3.1. Assessment of Students’ Engagement and Learning Approach

The number of optional literature critique peer discussion assignments completed by each student was used as a peer discussion engagement score to reflect their level of engagement in the course activities that centered on building SL and critical thinking capabilities. The average number of optional literature critique peer discussion assignments completed by students during the semester was 3.5 out of 5 (or 70% of the assignments). All five literature critique assignments were completed by 46% (n=74) of students in the course and 84.5% (n=136) of students completed at least one of the assignments during the semester (Figure 1). Deep and surface learning approach scores, which provide insight into students’ level of learning engagement [20], did not change during the semester (Figure 1; P<0.05).

Figure 1. Distribution of the number of optional literature critique assessments completed by students (out of a total of 5) during the semester and changes during the semester (i.e., between Survey 1 and Survey 2) in students’ deep learning approach, and surface learning approach scores. Data are presented as means ± SEM and were analyzed using a paired t-test

3.2. Changes in Students’ Perceived and Practical SL Skills

Changes during the semester in students’ perceived SL skill capabilities are shown in Table 1. There were improvements in all skill categories assessed (P<0.05) with the exception of the ability to interpret data presented in a graphical format (P=0.28). Conversely, students’ practical SL skills, assessed using TOSLS [33], did not change over the semester (P=0.63), and remained high with an average score of 81% of the questions answered correctly (Figure 2).

Table 1. Changes in students’ perceived SL skill capabilities at the start and end of semester¹

Figure 2. Changes in students’ practical SL skills assessed using TOSLS. Data are presented as means ± SEM and were analyzed using a paired t-test

Practical SL and critical thinking skills were tested at the end of the course on the final exam that required the analysis and critique an unfamiliar scientific study summary and provide written answers to questions such as i) identify the type of research study design, ii) identify the study strengths and weaknesses and discuss how they influence the interpretation of the results, and iii) identify a novel follow-up study research question and then design and describe a follow-up study to answer their research question. On this component of the final exam the average student grade was 70%. Correlative analyses determined that at the end of the semester, students’ perceived SL skill capabilities (namely their overall perceived SL skill score) was positively associated with their practical skill scores assessed using TOSLS (P<0.05; Table 2), which indicated that students’ self-assessment of their capabilities was similar to their practical skills.

Table 2. Correlation between students’ perceived or practical SL skill capabilities with tested SL skills, and learning appraoch¹

Conversely, there was no association between perceived SL skills and any of the practical SL and critical thinking skills assessed on the final exam (P>0.05; Table 2). Higher perceived SL capabilities were positively associated with a higher deep learning approach score and negatively associated with students’ surface learning approach score (P<0.05; Table 2). This indicated that students utilizing a deep learning approach had higher self-assessed SL capabilities, whereas students with a higher surface learning approach had lower self-assessed SL capabilities. Importantly, practical SL skills assessed using TOSLS were positively associated with the tested skill of identifying study design strengths and weaknesses and identifying and translating a research question into a follow-up experiment (P<0.05; Table 2); however, there was no association between TOSLS score and students’ ability to identify an experimental study design or either deep or surface learning approach (P>0.05; Table 2). Additionally, none of the tested SL and critical thinking skills were associated with either a deep or surface learning approach (P>0.05; Table 2).

The relationships between SL skills and learning approaches (both deep and surface approaches), with students’ final grade in the course or their peer discussion engagement score with the optional literature critique peer discussion assignments are shown in Table 3.

Table 3. Correlations between students’ final grade or peer discussion engagement score with learning approach, perceived and practical SL skill capabilities¹

Students’ practical SL skills assessed using TOSLS were positively associated with their final grade in the course (P<0.05; Table 3), whereas there was no relationship between final grade and students’ perceived SL skill capabilities or any tested SL and critical thinking skills on the final exam, namely the ability to identify types of study designs or experimental strengths and weaknesses or to identify and translate a research question into a follow-up study (P>0.05). There was no relationship between final grade and deep learning approach scores; however, final grade and surface learning approach scores exhibited a significant negative association (P<0.05; Table 3), indicating that students who utilized a surface learning approach had lower grades in the course, which may reflect a lower level of engagement with the course. In this connection, students’ peer discussion engagement score was strongly associated with final grade (P<0.05; Table 3), indicating that students who completed more of the optional literature critique peer discussion assignments had higher final grades in the course. The only other statistically significant relationship that was identified with students’ peer discussion engagement score was with practical SL skill of identifying the type of experimental study design (P<0.05; Table 3), which was a central component of the scientific literature critique peer discussion engagement assessments. Conversely, there was no relationship between peer discussion engagement scores and the other assessments of practical SL skills, perceived SL skills or learning approach (P>0.05; Table 3).

3.3. Changes in Students’ Perceived Stress During the Semester and Its Relationship with SL Skills, Engagement, and Final Grades

Students’ overall self-reported stress experience during the semester (both academic and non-academic stress combined) was assessed using the PSS [35], wherein perceived stress levels were found to significantly increase during academic semester (P<0.05; Figure 3). Correlative analyses determined that students’ perceived stress scores at the end of the semester were negatively associated with their practical skill competency in identifying and translating a research question into a follow-up experiment (P<0.05; Table 4), which was the most difficult SL and critical thinking skill assessed. Perceived stress scores were not related to any other SL skills (perceived or practical), learning approach, peer discussion engagement score, or final grade in the course (P>0.05; Table 4).

Figure 3. Changes during the semester (i.e., between Survey 1 and Survey 2) in perceived stress scores. Data are presented as means ± SEM and were analyzed using a paired t-test. Statistically significant differences (P<0.05) between Survey 1 and Survey 2 scores are denoted with an asterisk (*)

Table 4. Correlation between students’ perceived stress score and perceived and practical SL skill, learning approach, final grade and peer discussion engagement score at the end of the semester¹

3.4. Changes in Students’ Learning-Related Emotions During the Semester and Its Relationship with SL Skills, Learning Approach, Engagement, and Final Grade

In addition to students’ increased perceived stress (Figure 3), there are a range of learning-related emotions (both positive and negative) that can be experienced and impact academic outcomes, as shown in Table 5. Out of the eight emotions assessed only learning-related enjoyment (P=0.02) and anger (P<0.01) significantly increased during the semester and there was no significant change in students experience of hope, pride, anxiety, shame, hopelessness, or boredom associated with learning.

Table 5. Changes in students’ learning-related achievement emotion score between the start and end of the semester¹

Students’ SL skill competencies were associated with their learning-related emotions. Specifically, all positive learning-related emotions (namely enjoyment, hope and pride) were positively correlated with perceived SL skills (Table 6, P<0.05). In contrast, all negative learning-related emotions (anxiety, anger, shame, hopelessness and boredom) were negatively correlated with perceived SL skill capabilities (Table 6, P<0.05). Collectively, this indicates that higher perceived SL skill competencies were independently associated with experiencing higher/more frequent positive learning-related emotions and concomitantly experiencing lower negative learning-related emotions. Conversely, students with low perceived skill competencies experienced higher negative learning-related emotions. There were no significant relationships between learning-related emotions and students’ practical SL skills assessed using TOSLS or skill competencies tested on the final exam (Table 6).

Table 6. Correlation between learning-related achievement emotions at the end of the semester and students’ perceived SL skill scores, learning approach and final grade¹

Correlative analyses demonstrated that deep learning approach scores were consistently positively associated with all positive learning-related emotions and inversely associated with all negative learning-related emotions (Table 7). Therefore, students with higher deep learning approach scores experienced higher or more frequent positive learning-related emotions, whereas students with lower deep learning approach scores experienced a higher frequency of negative learning-related emotions.

The reverse relationship was apparent for surface learning approach scores, which were inversely associated with all three positive learning-related emotions and positively associated with experiencing four of the five negative learning-related emotions (namely anger, shame, hopelessness and boredom) (Table 7). Therefore, students utilizing a surface learning approach exhibited a lower frequency of positive learning-related emotions and a higher frequency of negative learning-related emotions. Not unexpectedly, learning emotions were also related to students’ perceived stress scores. Again, all three positive learning-related emotions exhibited a significant inverse relationship with perceived stress (Table 7). Conversely, higher or more frequent feelings of anxiety, shame and hopelessness were all positively associated with higher perceived stress scores (Table 7), whereas anger and boredom had no relationship with perceived stress (P>0.05). Collectively, this indicates that students with higher stress levels experienced more negative learning-related emotions and lower positive learning-related emotions. Students’ experience of learning-related emotions was also related to their engagement in the optional peer feedback discussion assignments. Specifically, students’ peer discussion engagement score was positively associated with enjoyment, whereas engagement score was negatively associated with learning-related feelings of boredom and anger (Table 7).

Table 7. Correlation between learning-related achievement emotions at the end of the semester and students’ learning approach, perceived stress score, peer discussion engagement score and final grade¹

There were no other significant relationships between the other learning-related emotions and engagement score. Finally, students’ experience of both positive and negative learning-related emotions were related to their final grade attained in the course. Specifically, students with higher or more frequent experiences of hope and pride earned higher final grades in the course, whereas an inverse relationship existed between students’ final grade and experiencing the negative learning-related emotions of anger, shame, hopelessness, and boredom (Table 7). Interestingly, there was no relationship between anxiety associated with learning and students’ final grades in the course (P>0.05).

4. Discussion

In an online asynchronous course format literature critique online group discussions that involved peer feedback were assigned to students with the intention to i) promote the development of SL skills in a low-stake (i.e., low grade value) setting, ii) foster community in the online learning environment when students are working remotely, and iii) promote students’ engagement in the course. Low-stake (i.e., worth a small percentage of students’ final grade) discussion activities involving peer feedback between students, such as the assignments used in this study, have been demonstrated to encourage critical thinking capabilities (which is a critical component of SL), the development of a deeper learning approach among undergraduate students [42], and to reduce academic stress [43,44]. Moreover, low-stake assessments can promote students’ experiencing positive learning-related emotions, such as enjoyment, which can promote learning engagement and higher final grades [19,45,46]. Therefore, the current study aimed to determine the influence of optional literature critique peer discussion assignments and students learning-associated emotions on SL skills and engagement.

Self-efficacy is important for undergraduate students and can be defined as a student’s self-confidence in accomplishing an assigned task [13]. Both self-efficacy and motivation are important attributes for academic success [14,16]. Group learning activities that provide opportunities for peer interaction have been shown to improve students’ engagement with course content, especially in online learning [17,47]. Group learning has also been linked to improved self-efficacy among undergraduate students [48]. Our data showed that students’ confidence in their perceived SL capabilities improved in nearly all domains assessed during the semester (Table 1), and that confidence in these skills was positively correlated with a deeper learning approach and negatively correlated with a surface learning approach (Table 2). These associations highlight the influence of learning approach type (deep versus surface), which is, in part, reflective of student engagement, on students’ SL skill capabilities or confidence in perceived capabilities. This demonstrates that engagement with the course material, which can involve adopting a deeper approach to learning, is associated with greater learning self-efficacy [49,50]. Therefore, from an instructional standpoint, encouraging and emphasizing the adoption of a deeper learning approach can provide a means to support the development of critical skills, such as SL skills [1], in undergraduate education. SL capabilities include, but are not limited to, the ability to critically evaluate and understand scientific literature [2,51] and apply this information to a greater context [1,2]. Ensuring that students’ SL skills are developed during an undergraduate degree program is important for their future success [2,51]. Perceived SL skills were positively associated with practical SL skills assessed using TOSLS (Table 2). Although aggregate TOSLS scores did not change between the start and end of the semester, they were positively associated with students’ tested SL skills on the final exam (Table 2). Moreover, TOSLS scores and peer discussion engagement scores were both positively correlated with students’ final grades in the course (Table 3). This indicated that students with higher practical SL skills who also engaged more in the peer discussion literature critique assignments that provided a low-stakes opportunity for students to practice and further develop their SL skills, earned higher grades in the course. This interpretation is further supported by the positive correlation between students’ peer discussion engagement score and their tested SL practical skill of identifying experimental designs (Table 3), which was one of three central components of the literature critique assessments. Finally, these components are also related, in part, to learning approach, wherein both final grade and peer-discussion engagement score were negatively correlated with surface learning score (Table 3), indicating that students who are less engaged in learning and exhibit lower self-efficacy through surface learning strategies participated to a lesser degree in the optional skill building assessments and earned lower final grades in the course. This is in line with previous studies demonstrating that higher self-efficacy is associated with greater academic performance [53].

Stress has been shown to adversely impact students’ engagement and self-efficacy [17,30,54] and therefore, may also influence their development of SL skills. Students’ perceived stress levels (from both academic and non-academic sources combined) increased during the semester (Figure 3). This is not unexpected given the background influence of the COVID-19 pandemic, as students were learning exclusively remotely and had limited access to social interaction coping mechanisms to alleviate stress [55,56] and experienced higher stress, anxiety, and depression levels [57,58,59,60,61,62,63]. Our findings align with other studies demonstrating elevated academic stress levels experienced by students in pandemic-associated online learning [56,64]. Since the perceived stress scale combines stress from all sources and is a general assessment of stress, we cannot determine the specific sources of stress that contributed to the higher levels assessed at the end of the semester; however, these results were comparable to previous studies assessing stress levels in health science students using the same survey scale [35,65]. Despite the increased stress levels during the semester, correlative analyses determined the only significant relationship that emerged was a negative relationship between perceived stress levels and students’ ability to identify a novel research question and translate it into a follow-up research study (Table 4). Importantly, this SL skill was assessed in an examination setting, which is typically associated with higher stress levels and there was no relationship with any of the other SL skills assessed on that exam. Therefore, despite increasing stress levels over the semester there was no effect on students’ perceived or practical SL skills assessed using TOSLS, learning approach, engagement score, or final grade.

In addition to stress, the emotions students experience while learning can influence skill development and engagement. A considerable research and resource emphasis in higher education is placed on management of students’ anxiety associated with learning [66,67], and anxiety and stress levels have been shown to be positively correlated [68]. Although frequently characterized as a negative emotion associated with learning that can reduce students’ engagement and academic performance, anxiety can also serve as a motivator for some students to engage or re-engage with learning and coursework [10,11,69]. Although student anxiety is important, consideration of other learning-related emotions that can impact students’ academic engagement and outcomes (i.e., skill development and grades) are also important, although frequently overlooked. Moreover, a more comprehensive consideration of the factors that influence students’ engagement and academic outcomes may also provide a means to help manage academic stress and/or anxiety. This is particularly important in relation to the impact of the COVID-19 pandemic on students’ and the shift to online learning, which was shown to increase negative learning-related emotions of anxiety, hopelessness, and anger, in addition to negative feelings not measured in the current study such as confusion, discouragement, loneliness, sadness, and depression [27,63]. Pekrun’s control-value theory of achievement emotions suggests that emotions can be defined by both students’ perceived control over the activity or outcome, and the value they place on doing or finishing the activity or outcome [8]. For example, enjoyment is felt when students perceive they have control over their learning and feel that what they are learning is valuable and worth doing, whereas boredom is experienced when students perceive that what they are learning has no value and/or they feel that they have no control over their learning [8]. In the current study, the only two learning-related emotions that significantly changed between the start and end of the semester were enjoyment and anger (Table 5). Enjoyment represents an important learning-related emotion as it reflects a student’s interest in the course content, motivation to learn, and self-efficacy [70]. In online education, ensuring students are experiencing learning-related enjoyment is important as higher enjoyment of courses has been shown to reduce dropout rates [71]. The increase in students’ feelings of learning-related anger is in line with other research that demonstrates how students are experiencing greater feelings of anger associated with the COVID-19 pandemic [27,63,72].

There were no relationships between students’ learning-related emotions and practical SL skills (Table 6); however, a greater level of self-confidence in SL skills was positively correlated with positive learning-related emotions, and negatively correlated with negative learning-related emotions (Table 6). This indicated that students with greater confidence in their SL skills had greater feelings of positive learning-related emotions and experienced fewer feelings of negative learning-related emotions. These results align with other studies that show experiencing positive learning-related emotions are associated with improved confidence and self-efficacy [8,19,73]. Students utilizing a deep learning approach are typically highly engaged, motivated and interested in the topic, seek a deeper understanding of course content and exhibit greater self-regulation in learning [20]. All positive learning-related emotions were positively correlated with deep learning approach scores and negatively correlated with surface learning approach scores (Table 7), indicating that students who are more engaged and aim for a deeper understanding of concepts experience more positive emotions while learning, which aligns with previous findings that show students who utilize deep learning experience more enjoyment in courses [8,20]. Conversely, students that utilize surface learning approaches (e.g., minimal engagement and learning by rote memorization tactics) have been shown to experience more boredom while learning [8,20]. Another measure of engagement was based on students’ completion of the optional peer feedback scientific literature discussion assignments, wherein students’ engagement scores were positively correlated with experiencing learning-related enjoyment and negatively correlated with experiencing learning-related anger and boredom (Table 7). These findings align with previous work that demonstrated how engagement and feeling connected during remote online learning was an important factor impacting student learning, wherein students feeling disconnected experienced higher rates of annoyance, discouragement, and confusion [27]. Collectively, this demonstrates that higher engagement (either via learning approach or active participation in optional low-stakes assessments designed to help students practice SL skills) supports students experience of more positive-learning related emotions and lower or less frequent negative learning-related emotions. Previous studies have shown that learning-related emotions can affect students’ ability to cope with stress [74,75]. High stress levels are associated with lower emotional well-being, leading to increased feelings of negative learning-related emotions [74]. In the current study, perceived stress levels were positively correlated with feelings of learning-related anxiety, shame, and hopelessness (Table 7). This finding aligns with research demonstrating that the COVID-19 pandemic increased students’ learning-related anxiety and health- and work-related stress [58,60]. Furthermore, in the current study, experiencing learning-related anxiety was negatively associated with a deep learning approach (Table 7), which aligns with previous research demonstrating lower test-related anxiety levels in students that utilize a deep learning approach [75], as deep learning approaches have been suggested to assist students in coping with academic anxiety [76]. Conversely, positive learning-related emotions are associated with the ability to cope with high levels of stress [77]. In this connection, higher perceived stress levels were negatively correlated with experiencing the positive learning-related emotions of pride, hope and enjoyment (Table 7), and it is possible that encouraging positive learning-related emotions may provide a method to reduce students’ academic stress. Students’ satisfaction and academic achievement have been shown to be negatively impacted by stress [54,56,77], and although there was no direct relationship between perceived stress levels and students’ final grade in the current study (Table 4), final grades were related to students learning-related emotions. Specifically, higher feelings of pride and hope during learning were positively correlated with final grades, whereas the negative learning-related emotions anger, shame, hopelessness, and boredom were all negatively correlated with final grades (Table 7). Thus, in addition to managing students’ stress, fostering positive learning-related emotions could also improve students’ academic performance. In this connection, feelings of hope have been demonstrated previously to have the greatest relationship to academic performance compared to other learning-related emotions [78,79].

There were some limitations in this study. Practical SL skills were assessed using the validated TOSLS survey [33] and students’ scores remained high with 81% of questions answered correctly at both the start and end of the semester. Therefore, it is possible that the TOSLS survey was not the optimal practical SL skill assessment tool for fourth-year undergraduate students in biological science majors. Among lower-year undergraduate students, TOSLS scores have been shown to improve after completion of course assignments, such as inquiry-based laboratory modules and exposure to the scientific curriculum in science courses [80]. However, the average score of lower-year undergraduate students reflects a lower level of SL capabilities (average score of approximately 62% of questions answered correctly) [80], which is substantially lower compared to the older and more academically experienced students in the current study. The original TOSLS survey was tested using students in a first-year biology course [33], and therefore, may not provide the necessary complexity in the SL questions to challenge the skill competencies of upper-year undergraduate students. Additionally, this study was conducted during the period of the COVID-19 pandemic when in-person classes were not permitted, and students were learning remotely online. Although timely and relevant to conduct this study during COVID-19 associated online learning, the results may not reflect non-pandemic associated online and/or in-person learning. Furthermore, the elevated stress levels and/or negative learning-related emotions observed at the end of the semester may have been influenced by external pressures associated with the COVID-19 pandemic, which include, but are not limited to, increased anxiety and stress [57,58,59,60,61,62,63], uncertainty regarding future courses, career prospects, and the impact of the pandemic [81,82].

5. Conclusion and Implications for Teaching Practice

This research demonstrates the utility of low-stakes peer discussion engagement assessments centered on literature critique activities to help promote the development of SL skills capabilities in upper-year students in biological science programs. The results demonstrate that employing instructional approaches that promote students’ engagement and a deeper learning approach directly influences the student learning experience and that experiencing positive learning-related emotions can beneficially impact academic performance (i.e., final grades), while concomitantly counteracting the adverse effects of experiencing negative learning-related emotions. In this connection, anxiety was the only negative learning-related emotion that showed no relationship with students’ final grade in the course (Table 7). Students’ attitudes and emotions experienced while learning can impact skill development and academic performance or outcomes such as grades. Given the higher levels of stress and anxiety reported by students in COVID-19-associated online learning [57-63,82], identifying instructional approaches to minimize these experiences (in addition to other negative learning-related emotion experiences) and promote feelings of positive learning-related emotions may ultimately improve academic outcomes. Many universities emphasize reducing students’ academic stress and anxiety [66,67]; however, this research demonstrates that additional elements of the student learning experience may also need to be considered by university educators beyond anxiety, particularly when considering the relationship between other positive and negative learning-related emotions on final grades.

Acknowledgements

D.M.B. is supported by the Scholarship of Teaching and Learning Graduate Research Assistantship from the University of Guelph.

Statement of Competing Interests

The authors have no conflicts of interest to disclose.

Abbreviations

Shortened Achievement Emotion Questionnaire (AEQ-S); Perceived Stress Scale (PSS); Revised Two-Factor Study Process Questionnaire (R-SPQ-2F); Scientific Literacy (SL); Test of Scientific Literacy Skills (TOSLS).

References

[1]   | De Boer, G. E. (2000). Scientific Literacy: Another look at its historical and contemporary meanings and its relationship to science education reform. Journal of Research in Science Teaching, 37(6), 582-601.
[2]   | Britt, M. A. (2014). Scientific Literacy: The role of goal-directed reading and evaluation in understanding scientific information. Educational Psychologist, 49(2), 104-122.
[3]   | Holbrook, J., & Rannikmae, M. (2009). The Meaning of Scientific Literacy. International Journal of Environmental and Science Education, 4(3), 275-288.
[4]   | Turiman, P., Omar, J., Daud, A. M., & Osman, K. (2012). Fostering the 21st Century Skills through Scientific Literacy and Science Process Skills. Procedia - Social and Behavioral Sciences, 59, 110-116.
[5]   | Cartwright, N. M., Liddle, D. M., Arceneaux, B., Newton, G., & Monk, J. M. (2020). Assessing scientific literacy skill perceptions and practical capabilities in fourth year undergraduate biological science students. International Journal of Higher Education, 9(6), 64-76.
[6]   | Kararo, M., & McCartney, M. (2019). Annotated primary scientific literature: A pedagogical tool for undergraduate courses. PLOS Biology, 17(1), e3000103.
[7]   | Putwain, D., Sander, P., & Larkin, D. (2013). Academic self-efficacy in study-related skills and behaviours: Relations with learning-related emotions and academic success. British Journal of Educational Psychology, 83(4), 633-650.
[8]   | Pekrun, R. (2006). The Control-Value Theory of Achievement Emotions: Assumptions, Corollaries, and Implications for Educational Research and Practice. Educational Psychology Review, 18(4), 315-341.
[9]   | Chapell, M. S., Blanding, Z. B., Silverstein, M. E., Takahashi, M., Newman, B., Gubi, A., & McCann, N. (2005). Test Anxiety and Academic Performance in Undergraduate and Graduate Students. Journal of Educational Psychology, 97(2), 268-274.
[10]   | England, B. J., Brigati, J. R., & Schussler, E. E. (2017). Student anxiety in introductory biology classrooms: Perceptions about active learning and persistence in the major. PLOS ONE, 12(8), e0182506.
[11]   | Hancock, D. R. (2001). Effects of Test Anxiety and Evaluative Threat on Students’ Achievement and Motivation. The Journal of Educational Research, 94(5), 284-290.
[12]   | Barrows, J., Dunn, S., & Lloyd, C.A. (2013). Anxiety, Self-Efficacy, and College Exam Grades. Universal Journal of Educational Research, 1(.3), 204-208.
[13]   | Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191-215.
[14]   | Zimmerman, B. J. (1995). Self-efficacy and educational development. In Self-efficacy in changing societies (pp. 202-231). Cambridge University Press.
[15]   | Ferla, J., Valcke, M., & Schuyten, G. (2008). Relationships between student cognitions and their effects on study strategies. Learning and Individual Differences, 18(2), 271-278.
[16]   | Jensen, S. A. (2011). In-class versus online video lectures: similar learning outcomes, but a preference for in-class. Teaching of Psychology, 38(4), 298-302.
[17]   | Wang, C., Shannon, D. M., & Ross, M.E. (2013). Students’ characteristics, self-regulated learning, technology self-efficacy, and course outcomes in online learning. Distance Education, 34(3), 302-323.
[18]   | Pekrun, R. (2007). Emotions in Students’ Scholastic Development. In R. P. Perry & J. C. Smart (Eds.), The Scholarship of Teaching and Learning in Higher Education: An Evidence-Based Perspective (pp. 553-610). Springer Netherlands.
[19]   | Rowe, A. D., Fitness, J., & Wood, L. N. (2015). University student and lecturer perceptions of positive emotions in learning. International Journal of Qualitative Studies in Education, 28(1), 1-20.
[20]   | Biggs, J., Kember, D., & Leung, D. Y. P. (2001). The revised two factor study process questionnaire: R-SPQ-2F. British Journal of Educational Psychology, 71, 133-149.
[21]   | Doucet, M., Vrins, A., & Harvey, D. (2009). Effect of using an audience response system on learning environment, motivation and long-term retention, during case-discussions in a large group of undergraduate veterinary clinical pharmacology students. Medical Teacher, 31(12), 570-579.
[22]   | Tsaushu, M., Tal, T., Sagy, O., Kali, Y., Gepstein, S., & Zilberstein, D. (2012). Peer Learning and Support of Technology in an Undergraduate Biology Course to Enhance Deep Learning. CBE—Life Sciences Education, 11(4), 402-412.
[23]   | Volet, S., Seghezzi, C., & Ritchie, S. (2019). Positive emotions in student-led collaborative science activities: Relating types and sources of emotions to engagement in learning. Studies in Higher Education, 44(10), 1734-1746.
[24]   | Eren-Sisman, E. N., Cigdemoglu, C., & Geban, O. (2018). The effect of peer-led team learning on undergraduate engineering students’ conceptual understanding, state anxiety, and social anxiety. Chemistry Education Research and Practice, 19(3), 694-710.
[25]   | Stone, R., Cooper, S., & Cant, R. (2013). The Value of Peer Learning in Undergraduate Nursing Education: A Systematic Review. ISRN Nursing, 2013, 1-10.
[26]   | Aguilera-Hermida, A. (2020). College students’ use and acceptance of emergency online learning due to COVID-19. International Journal of Educational Research Open, 1, 100011.
[27]   | Baltà-Salvador, R., Olmedo-Torre, N., Peña, M., & Renta-Davids, A.-I. (2021). Academic and emotional effects of online learning during the COVID-19 pandemic on engineering students. Education and Information Technologies, 26(6), 7407-7434.
[28]   | Camacho-Zuniga, C., Pego L., Escamilla, J., & Hosseini, S. (2021). The impact of the COVID-19 pandemic on students’ feelings at high school, undergraduate, and postgraduate levels. Heliyon, 7(3), e06465.
[29]   | Moreno-Fernandez, J., Ochoa, J. J., Lopez-Aliaga, I., Alferez, M. J. M., Gomez-Guzman, M., Lopez-Ortega, S., & Diaz-Castro, J. (2020). Lockdown, Emotional Intelligence, Academic Engagement and Burnout in Pharmacy Students during the Quarantine. Pharmacy, 8(4), 194.
[30]   | Rossi, R. A., Krouse, A. M., & Klein, J. (2021). Undergraduate Student Stress, Classroom Engagement, and Self-Directed Learning Postcurricular Revision. Journal of Nursing Education, 60(10), 566-569.

[31]  
Beauchamp, D. M., Newton G., and Monk, J. M. (2021). Adapting Literature Critique Engagement Activities for Online Learning Due to COVID-19: Use of Online Learning Groups to Promote Scientific Literacy Capabilities in Undergraduate Nutrition Education. International Journal of Higher Education, 10(7), 19-33.

[32]   | Beauchamp, D. M., & Monk, J. M. (2022). Effect of Optional Assessments on Student Engagement, Learning Approach, Stress, and Perceptions of Online Learning during COVID-19. International Journal of Higher Education, 11(5), 87-101.
[33]   | Gormally, C., Brickman, P., & Lutz, M. (2012). Developing a test of scientific literacy skills (TOSLS): Measuring undergraduates' evaluation of scientific information and arguments. CBE Life Science Eduation, 11(4), 364-377.

[34]  
Monk, J. M., & Newton, G. (2018). Use of scaffolded case study assignment to enhance students’ scientific literacy skills in undergraduate nutritional science education: Comparison between traditional lecture and distance education course formats. International Journal of Higher Education, 7(2), 95-106.

[35]   | Cohen, S., Kamarck, T., Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behaviour, 24, 385-396.
[36]   | Bieleke, M., Gogol, K., Goetz, T., Daniels, L., & Pekrun, R. (2020). The AEQ-S: A Short Version of the Achievement Emotions Questionnaire. Contemporary Educational Psychology, 65, 101940.
[37]   | Pekrun, R., Goetz, T., Frenzel, A. C., Barchfeld, P., & Perry, R. P. (2011). Measuring emotions in students’ learning and performance: The Achievement Emotions Questionnaire (AEQ). Contemporary Educational Psychology, 36(1), 36-48.
[38]   | Daniels, L. M., Stupnisky, R. H., Pekrun, R., Haynes, T. L., Perry, R. P., & Newall, N. E. (2009). A longitudinal analysis of achievement goals: From affective antecedents to emotional effects and achievement outcomes. Journal of Educational Psychology, 101(4), 948-963.
[39]   | Acee, T. W., Kim, H., Kim, H. J., Kim, J.-I., Chu, H.-N. R., Kim, M., Cho, Y., & Wicker, F. W. (2010). Academic boredom in under- and over-challenging situations. Contemporary Educational Psychology, 35(1), 17-27.
[40]   | Gillis, A., & Krull, L. M. (2020). COVID-19 Remote Learning Transition in Spring 2020: Class Structures, Student Perceptions, and Inequality in College Courses. Teaching Sociology, 48(4), 283-299.
[41]   | Holmes, N. (2014). Student perceptions of their learning and engagement in response to the use of a continuous e-assessment in an undergraduate module. Assessment & Evaluation in Higher Education, 40(1), 1-14.
[42]   | Afify, M. K. (2019). The Influence of Group Size in the Asynchronous Online Discussions on the Development of Critical Thinking Skills, and on Improving Students’ Performance in Online Discussion Forum. International Journal of Emerging Technologies in Learning, 14(5), 132-152.
[43]   | Cardozo, L. T., de Avezedo, M. A. R., Carvalho, M. S. M., Costa, R., de Lima, P. O. & Marcondes, F. K. (2020). Effect of an active learning methodology combined with formative assessments on performance, test anxiety, and stress of university students. Advances in Physiology Education, 44, 744-751.
[44]   | Venton, B. J., & Pompano, R. R. (2021). Strategies for enhancing remote student engagement through active learning. Analytical and Bioanalytical Chemistry, 413, 1507-1512.
[45]   | Schrank, Z. (2016). An Assessment of Student Perceptions and Responses to Frequent Low-Stakes Testing in Introductory Sociology Classes. Teaching Sociology, 442(2), 118-127.
[46]   | Sotola, L. K., & Crede, M. (2021). Regarding Class Quizzes: A Meta-analytic Synthesis of Studies on the Relationship Between Frequent Low-Stakes Testing and Class Performance. Educational Psychology Review, 33(2), 407-426.
[47]   | Shen, D., Cho, M.-H., Tsai, C.-L., & Marra, R. (2013). Unpacking online learning experiences: Online learning self-efficacy and learning satisfaction. The Internet and Higher Education, 19, 10-17.
[48]   | Maraj, M., Hale, C. P., Kogelbauer, A., & Hellgardt, K. (2019, June), Teaming with Confidence: How Peer Connections in Problem-based Learning Impact the Team and Academic Self-efficacy of Engineering Students. Paper presented at 2019 ASEE Annual Conference & Exposition, Tampa, Florida.
[49]   | Linnenbrink, E. A., & Pintrich, P. R. (2003). The Role of Self-Efficacy Beliefs in Student Engagement and Learning in the Classroom. Reading & Writing Quarterly, 19(2), 119-137.
[50]   | Wester, E. R., Walsh, L. L., Arango-Caro, S., & Callis-Duehl, K. L. (2021). Student Engagement Declines in STEM Undergraduates during COVID-19–Driven Remote Learning. Journal of Microbiology & Biology Education, 22(1), ev22i1.2385.
[51]   | Porter, J. A., Wolbach, K. C., Purzycki, C. B., Bowman, L.A., Agbada, E., & Mostrom, A.M. (2017). Integration of information and scientific literacy: promoting literacy in undergraduates. CBE-Life Sciences, 9(4), 536-542.
[52]   | Newton, G., Bettger, W., Buchholz, A., Kulak, V., & Racey, M. (2015). Evidence-informed strategies for undergraduate nutrition education: a review. Applied Physiology, Nutrition, and Metabolism, 40(7), 652-661.
[53]   | Honicke, T., & Broadbent, J. (2016). The influence of academic self-efficacy on academic performance: A systematic review. Educational Research Review, 17, 63-84.
[54]   | Kahu, E. R., & Nelson, K. (2018). Student engagement in the educational interface: understanding the mechanisms of student success. Higher Education Research & Development, 37(1), 58-71.
[55]   | Clabaugh, A., Duque, J. F., & Fields, L. J. (2021). Academic Stress and Emotional Well-Being in United States College Students Following Onset of the COVID-19 Pandemic. Frontiers in Psychology, 12, 628787.
[56]   | Rogowska, A. M., Kusnierz, C., & Bokszczanin, A. (2020). Examining Anxiety, Life Satisfaction, General Health, Stress and Coping Styles During COVID-19 Pandemic in Polish Sample of University Students. Psychology Research and Behaviour Management, 13, 797-811.
[57]   | Aslan, I., Ochnik, D., & Çınar, O. (2020). Exploring perceived stress among students in Turkey during the covid-19 pandemic. International Journal of Environmental Research and Public Health, 17(23), 8961.
[58]   | Fitzgerald, A., & Konrad, S. (2021). Transition in learning during COVID-19: Student nurse anxiety, stress, and resource support. Nursing Forum, 56(2), 298-304.
[59]   | Odriozola-González, P., Planchuelo-Gómez, Á., Irurtia, M. J., & de Luis-García, R. (2020). Psychological effects of the COVID-19 outbreak and lockdown among students and workers of a Spanish university. Psychiatry Research, 290, 113108.
[60]   | Pennino, E., Ishikawa, C., Ghosh Hajra, S., Singh, N., & McDonald, K. (2022). Student Anxiety and Engagement with Online Instruction across Two Semesters of COVID-19 Disruptions. Journal of Microbiology & Biology Education, 23(1), 1-21.
[61]   | Saravanan, C., Mahmoud, I., Elshami, W., & Taha, M. H. (2020). Knowledge, Anxiety, Fear, and Psychological Distress About COVID-19 Among University Students in the United Arab Emirates. Frontiers in Psychiatry, 11, 582189.
[62]   | Son, C., Hegde, S., Smith, A., Wang, X., & Sasangohar, F. (2020). Effects of COVID-19 on college students’ mental health in the United States: Interview survey study. Journal of Medical Internet Research, 22(9), e21279.
[63]   | Wood, C. I., Yu, Z., Sealy, D.-A., Moss, I., Zigbuo-Wenzler, E., McFadden, C., Landi, D., & Brace, A. M. (2022). Mental health impacts of the COVID-19 pandemic on college students. Journal of American College Health.
[64]   | Biber, D. D., Melton, B., Czech, D. R. (2020). The impact of COVID-19 on college anxiety, optimism, gratitude, and course satisfaction. Journal of American College Health, 70(7), 1942-1950.
[65]   | Amhare, A.F., Jian, L., Wagaw, L.M., Qu, C., & Han, J. (2021). Magnitude and associated factors of perceived stress and its consequence among undergraduate students of Salale University, Ethiopia: cross-sectional study. Psychology, Health & Medicine, 26(10), 1230-1240.
[66]   | Regehr, C., Glancy, D., & Pitts, A. (2013). Interventions to reduce stress in university students: A review and meta-analysis. Journal of Affective Disorders, 148(1), 1-11.
[67]   | Rith-Najarian, L. R., Boustani, M. M., & Chorpita, B. F. (2019). A systematic review of prevention programs targeting depression, anxiety, and stress in university students. Journal of Affective Disorders, 257, 568-584.
[68]   | Misra, R., & McKean, M. (2000). College students’ academic stress and its relation to their anxiety, time management, and leisure satisfaction. American Journal of Health Studies, 16(1), 79-86.
[69]   | Stockinger, K., Rinas, R., & Daumiller, M. (2021). Student adaptability, emotions, and achievement: Navigating new academic terrains in a global crisis. Learning and Individual Differences, 90, 102046.
[70]   | Ainley, M., & Hidi, S. (2014). Interest and enjoyment. In R. Pekrun & L. Linnebrink-Garcia (Eds.), International Handbook of Emotions in Education (pp. 205-227). Routledge/Taylor & Francis Group.
[71]   | Brubacher, M. R., & Silinda, F. T. (2019). Enjoyment and Not Competence Predicts Academic Persistence for Distance Education Students. The International Review of Research in Open and Distributed Learning, 20(3).
[72]   | Shanahan, L., Steinhoff, A., Bechtiger, L., Murray, A. L., Nivette, A., Hepp, U., Ribeaud, D., & Eisner, M. (2022). Emotional distress in young adults during the COVID-19 pandemic: Evidence of risk and resilience from a longitudinal cohort study. Psychological Medicine, 52(5), 824-833.
[73]   | Villavicencio, F. T., & Bernardo, A. B. I. (2016). Beyond Math Anxiety: Positive Emotions Predict Mathematics Achievement, Self-Regulation, and Self-Efficacy. The Asia-Pacific Education Researcher, 25(3), 415-422.
[74]   | de la Fuente, J., Martinez-Vicente, J.M., Lopez-Garcia, M., Zapata, L., & Mariano-Vera, M. (2017). Personal self-regulation, learning approaches, resilience, and test anxiety in psychology students. Estudios sobre Educación, 32, 9-26.
[75]   | Rozgonjuk, D., Kraav, T., Mikkor, K., Orav-Puurand, K., & Täht, K. (2020). Mathematics anxiety among STEM and social sciences students: The roles of mathematics self-efficacy, and deep and surface approach to learning. International Journal of STEM Education, 7(1), 46.
[76]   | Cipra, C., & Müller-Hilke, B. (2019). Testing anxiety in undergraduate medical students and its correlation with different learning approaches. PLOS ONE, 14(3), e0210130.
[77]   | Gloria, C. T., & Steinhardt, M. A. (2016). Relationships Among Positive Emotions, Coping, Resilience and Mental Health. Stress and Health, 32(2), 145-156.
[78]   | Gallagher, M. W., Marques, S. C., & Lopez, S. J. (2017). Hope and the Academic Trajectory of College Students. Journal of Happiness Studies, 18(2), 341-352.
[79]   | Carmona-Halty, M., Salanova, M., Llorens, S., & Schaufeli, W. B. (2021). Linking positive emotions and academic performance: The mediated role of academic psychological capital and academic engagement. Current Psychology, 40(6), 2938-2947.
[80]   | Segarra, V. A., Hughes, N. M., Ackerman, K. M., Grider, M. H., Lyda, T., & Vigueira, P. A. (2018). Student performance on the Test of Scientific Literacy Skills (TOSLS) does not change with assignment of a low-stakes grade. BMC Research Notes, 11(1), 422.
[81]   | Dhawan, S. (2020). Online Learning: A Panacea in the Time of COVID-19 Crisis. Journal of Educational Technology Systems, 49(1), 5-22.

[82]  
Hawley, S.R., Thrivikraman, J.K., Noveck, N., St. Romain, T., Ludy, M-J., Barnhart, L., Chee, W.S.S., Cho, M.J., Chong, M.H.Z., Du, C., Fenton, J.I., Hsiao, P.Y., Hsiao, R., Keaver, L., Lee, H.S., Shen W., Lai, C-C., Tseng, K-W., Tsen, W-C., & Tucker, R.M. (2021). Concerns of college students during the COVID-19 pandemic: Thematic perspectives from the United States, Asia and Europe. Journal of Applied Learning & Teaching. 4(1), 1-10.