Dear Label_Name,

Greetings! I hope you are well.

I am writing on behalf of an international publishing house, Science and Education Publishing, USA (http://www.sciepub.com).

Science and Education Publishing (SciEP) is a multidisciplinary open access publisher of Journals covering the fields of Science, Technology and Medicine. All SciEP publications are subject to high-quality peer review, editorial and production processes and are freely available online.

Welcome to the April, 2013 edition of SciEP newsletter, providing monthly summaries of articles on science & education related topics of interest to scientist and professionals. The mission of our journals is to provide an open access forum for publishing new findings on issues pertinent to scientific and academic field. We also invite you to submit original research papers, review articles, short reports, or case studies are considered for publication in our Multidisciplinary Journals.

American Journal of Public Health Research
DOI: 10.12691/ajphr-1-1-3

The Implementation of a Tele-Homecare System with Service Oriented Architecture and HL7 Message Transmission Standard
Ching-Sung Wang

American Journal of Medical and Biological Research
DOI: 10.12691/ajmbr-1-1-2

The Influence of Ethanol on Pyruvate Kinases Activity in Vivo, in Vitro, in Silico
Lelevich Sergey Vladimirovich, Khrustalev Vladislav Victorovich, Barkovsky Eugene Victorovich, Shedogubova Tatyana Aleksandrovna

American Journal of Medical Sciences and Medicine
DOI: 10.12691/ajmsm-1-1-1

Clinical and Histological Features in Wegener Granulomatosis
Li Cavoli Gioacchino, Passantino Rita, Bono Luisa, Tortorici Calogera, Ferrantelli Angelo, Giammarresi Carlo, Rotolo Ugo

American Journal of Infectious Diseases and Microbiology
DOI: 10.12691/ajidm-1-1-2

The Curse of Lymphatic Filariasis: Would the Continual Use of Diethylcarbamazine Eliminate this Scourge in Papua New Guinea?
Anne Lanham, Lillian Mwanri

We have released issues of some journals on February 2013.
American Journal of Public Health Research
American Journal of Medical Sciences and Medicine
American Journal of Medical and Biological Research
American Journal of Pharmacological Sciences
American Journal of Clinical Medicine Research
American Journal of Nursing Research
American Journal of Cancer Prevention
American Journal of Medicine Studies
American Journal of Infectious Diseases and Microbiology
American Journal of Cardiovascular Disease Research
American Journal of Biomedical Research
American Journal of Mechanical Engineering
American Journal of Microbiological Research
Journal of Preventive Medicine
American Journal of Food and Nutrition
American Journal of Hypertension Research
American Journal of Sports Science and Medicine
Journal of Cancer Research and Treatment
American Journal of Zoological Research
Journal of Biomedical Engineering and Technology
American Journal of Microbiological Research
Biomedical Science and Engineering

We are pleased to announce that submissions are now being accepted for peer review for all the journals across all our subject areas in SciEP.

Please submit your manuscript via our online submission system:
http://www.sciepub.com/author/signin

To unsubscribe from our mailing list, please click on the following link: unsubscribe

I am looking forward to hearing from you.

Kind regards,

David Rain
Email:join@sciepub.com
Science and Education Publishing (http://www.sciepub.com)

The Bottom up Evaluation at Universities

Marek Botek*, Tomáš Macák

Department of Economics and Management of Chemical and Food Industry, Institute of Chemical Technology, Prague, the Czech Republic

Abstract: Evaluation is one of the most important tasks of a manager. Multi-criteria method is often used to increase objectivity. Opinion and satisfaction of students with their classes, courses, teaching and teachers is often used as a part of an evaluation at universities. This article wants to show that student surveys are not representative and objective. The opinion carried in this article is based on study which was made at a prestigious technical university in Prague, the Czech Republic. The paper has been processed based on the analysis and evaluation of secondary sources and outcome synthesis. Survey has been realized in June 2014. It included all subjects which are taught at the university during winter semester 2012/2013. Only laboratories were excluded. There was total of 330 subjects which were part of the study. In a conclusion of the article there are discussed ways for increasing representativeness of the bottom-up evaluation and also possibilities how to take the finding into evaluation in managerial practice.

Keywords: bottom up evaluation, students satisfaction, 360 degree feedback, ICT Prague, students surveys

Received July 26, 2014; Revised September 07, 2014; Accepted September 12, 2014

Citation: Marek Botek, Tomáš Macák, The Bottom up Evaluation at Universities. American Journal of Educational Research, 2 (9). 823-827. January 2014. doi:10.12691/education-2-9-18

1. Introduction

Schools and universities use evaluation regularly and often but there is up to bottom evaluation, it means teachers evaluate students. They usually use grades but at some schools (particularly basic ones) there is used descriptive evaluation, which is more individualized and has a positive effect on learners [1].

Teachers evaluate level of knowledge or techniques they have taught. Therefore they usually know the maximal level (the ideal level of knowledge) and they are able to valuate student´s performance objectively.

Evaluation is also one of personal activities, what are obliged to do all managers (including the ones at education sector). Researchers tend to agree that teacher quality is an important determining factor in influencing student outcomes [2]. Evaluation should be objective, focused on main aspects and serves to evaluee’s development and/or remuneration. Superiors can sometimes have skewed point of view and therefore they exist efforts to objectivize the evaluation. One of them is using more evaluators. In the case of university teachers, one group of evaluators can be students. One such technique is 360° feedback. It is multi-criteria evaluation and evaluators are supervisor (boss), colleagues, and subordinates, also self-evaluation is included [3,4].

The boss should be able to evaluate objectively [5]. Also colleagues´ and self- evaluation can bring relevant data, if raters are reminded to put aside personal bias (such as centralizing, grouping, halo/horns effect, guilt by association, holding a grudge, sunflower effect [6]) when performing evaluations [7] and if they believe that results will be used only for personal development of evaluee. If not, evaluation is often biased. Atkins et al [8] claim that the method is not good for employees’ evaluation; friends usually give better results; also colleagues try to help each other. For example, if somebody is asked to participate in evaluation of his colleague (friend) close to firing, he usually (subconsciously) increases friends´ performance or improves his behavior. On the other hand, if a reward can become only one of the team, it is hard to ask team members for objective evaluation. Atkins et al [8] investigated the consensus between self-evaluation, other evaluations in 360° feedback process and assessment centre results. They found that the self-ratings were negatively and nonlinearly related to performance with some of those who gave themselves the highest ratings having the lowest performance on the assessment centre. Colleagues often overestimated performance for poor performers [8].

The most problematic is evaluation from subordinates/ students. If the evaluation is anonymous, they can try to avenge. Kacmar et al [9] found that subordinates who receive positive feedback from their supervisor will rate the supervisor higher than the subordinates who receive negative feedback. Straka [10] claims that accepting of positive changes increase satisfaction of stakeholders. If there is not anonymous evaluation subordinates/students can afraid to give objective but worse grades. It is enough if they have feeling, that feedback is not anonymous and evaluee can blow their cover. In accordance with [11] mutual trust amongst subordinates and manager is very important. Sometimes can managerial style of leading influence turnover of employees [12].

Next big problem in bottom up evaluation is that evaluators are not able to appraise evaluee´s performance [13]. Therefore their answers should be focused only on description of manager/teacher behavior particularly in relationship to subordinates/students, or on satisfaction with partial aspects of his work.

Using a bottom up teaching evaluation is problematic. Berk [14] thinks that they exists no objective method for measuring teaching performance. Students usually are not able to evaluate the level of teachers knowledge objectively. They are focused only on teacher´s approach, behavior and how much the subject interested them. It is possible to say, they evaluate their own satisfaction with education.

Students satisfaction is very important factor, they exist findings suggest a significant relationship between student satisfaction and academic performance [15] or academic burnout [16]. But there are many definitions of student satisfaction (see [17,18]) and it is sure that students´ satisfaction is influenced by many factors [19]. Choudhury [20] assumes four dimensions (teaching, facilities, attitude and convenience), Maceli, Fogliasso and Baack [21] confirmed gender influence, Marsh [22] discussed the impact of the grade which students expect to receive. Maceli, Fogliasso and Baack [21] claim, that students compare their expectation and lecturers´ performance. It seems that this evaluation cannot be objective. Kesselheim et al [23] claim, that trainees hold mismatched impression of their training program. In spite of this many evaluation procedures focus solely on student feedback without correction from other sources [19]. It also agreed with [24] idea that the performance feedback data lead the training plans, job rotation efforts and career development activities. On the other hand Nair, Bennett and Mertova [25] found that many universities around the world collect student feedback, but they do not use it for institutional changes. It correspondent with fact, that some universities do not managed all projects what they realized [26].

This article aims to illustrate the non-representativeness of students inquires using the example of ICT Prague, a technical university in the Czech Republic.

2. Materials and Methods

Evaluation of student satisfaction at ICT Prague is realized since 2004. Previously it was pen-paper modification consist of three claims evaluated on four-point scale (strongly agree; agree; disagree; strongly disagree):

•Education has been well prepared, information were logical and clear.

•Presentation of information has been understandable and punchy.

•The teacher has tried to increase interest in the subject and supported activities of the students.

Since 2005/2006 it is computerized. From the 10th week of semester (semester consists of 14 weeks for lectures and 5 weeks for exams) until the end of semester students can evaluate each subject (lecturer) they enrolled. The evaluation consists of 8 items of satisfaction evaluated on five-point Likert scale (1 is the best, 5 is the worst):

•Evaluate the teacher from the teaching methods and clarity point of view. - method

•Evaluate the teacher’s approach to students and his/her rapport with students. - approach

•Has the subject broadened your knowledge in the frame of the study program? (1 I have learned a great deal of new information, 3 average, 5 useless subject) - news

•What is your general impression of the laboratories? (equipment, working environment, organization) - impression

•How does the subject link to your previous knowledge? (1 perfect link, 3 average link, 5 no link) - link

•Are the study materials appropriate? - materials

•Has the laboratory work enhanced your knowledge of the subject? - complete

•Is the number of credits adequate? (1 should be increased significantly, 2 should be increased, 3 adequate, 4 should be reduced 5 should be reduced significantly) - credits

Data for this article were collected under winter semester 2012/2013. 247 lectures, 158 exercises, 7 seminars and 18 laboratories were evaluated, it is 430 items. Because a lot of subjects consist of lecture and exercise, there were 330 evaluated subjects, some of them taught in more teaching groups (e.g. Mathematics I in 7 per lectures and 20 per exercises; Chemical Calculations in 30 per exercises). Students should add the name of teacher to subject evaluation, but they sometimes forget to do it. Then the system creates new group of evaluation (see example in Table 1). Subject Chemical Engineering I consists of lecture (lecturer P.) and exercises (seven groups because of number of students – lecturers Z., O., J., V., F., L., N.). Because some students forgot fulfill the name of lecturer, this subject is divided into 10 items.

Table 1. Results of Chemical Engineering education evaluation

In total 1 114 items of students´ evaluation of lectures and exercises were collected.

3. Results

The idea of students´ feedback is very good, but somewhere is a problem and students do not use this tool. Therefore data of these surveys are not representative. The system permits to know only total numbers of respondents (students). It is sufficient if there is only one teacher per group and it is no problem if one teacher leads more groups (e.g. exercises). In that case is in the system feedback from all students it means all received feedbacks and an information how many students should evaluate. It is only impossible to know what group students attended.

Example presented in Table 1 shows two problems. The first one is very low number of feedbacks and the second one is impossibility to find the size of taught groups. The knowledge of number of students in the group is necessary to judge an importance of evaluation. For example ten evaluators from seventeen enrolled students is much more important than 16 evaluators from 176 (see Table 1).

In some cases it would be possible to increase the number of feedbacks a bit, because a lot of students forgot to put teachers´ name to the feedback. If there is subject with only one teacher it is possible to add these anonymous feedbacks to his ones.

The students in Table 1 have only one lecturer but they can choose one of seven groups per exercises. Theoretically it is possible to find the real number of students they should evaluate each subject in another item of Educational info system. It is Schedule for a semester with taught subjects divided into learning groups and numbers of enrolled students for each group. But students can as change their group so attend lectures or exercises for another group. And they always can evaluate all teachers in list of teacher for a subject. Changing a group is often if students prefer another time or “style of teaching” or if they are not satisfied with the teacher. The exemplary subject - Chemical Engineering is quite small subject, e.g. Mathematics had 1 450 and General and Inorganic Chemistry I 1 272 enrolled students.

Similar situation is in case of participation two or more lecturers. If lectures are e.g. taught by two teachers (the first one lead 10 weeks and the second one 4 remaining ones) then students can evaluate both two lecturers. The system gives only general results; e.g. the first lecturer was evaluated by 7 students and the second one by 8 students from a group of 17 enrolled students. It is impossible to find if some students evaluated both teachers. The situation is much complicated in case of labs because here is common a cooperation of 5 or more employees (teachers). Each of them leads only some lab works and students are not obliged to realize all works. Therefore some students do not meet all teachers and it is impossible indirectly find the size of evaluators´ group.

Next problematic group are different projects and seminars, where is one guarantor but a lot of possible people who really lead students. Students can evaluate all teachers who are in a group of lecturers for the subject. Sometimes are therefore evaluated also teachers who did not lead any lecture or exercise.

For illustration the problem with small number of respondents it is used an example of Faculty of chemical Engineering, one of four Faculties at ICT Prague. In the winter semester 2012/2013 there was taught 141 subjects in this Faculty in 350 groups, excluding laboratories, seminars and semestral projects, because in these events it is impossible to find how many teachers student should evaluate. Table 2 shows the numbers of enrolled students in all Faculty departments (signed by numbers 401-407).

Table 2. Enrolled students at Faculty of Chemical Engineering

Table 3 shows the numbers of received feedback in the semester.

Table 3. Nr. of feedbacks at Faculty of Chemical Engineering

If we compare these Tables we can see e.g. that at the department 402 was taught 15 groups (11+2+1+1) with more than 20 enrolled students but only in 2 cases there were received feedbacks from more than 20 students per taught subject. The majority of evaluation comes from 1 up to 5 respondents. There is no evaluation of more than 50 students from the same taught group although at the Faculty there were 6 groups with more than 200 enrolled students.

Another example can give the view to history of evaluation. In Table 4 there are numbers of feedbacks for haphazardly chosen subject from the ones with higher number of enrolled students. It is Management of Enterprise Processes (MEP) taught at Faculty of Chemical Engineering. During the years there were changed teachers and amounts of taught group of students. Some teachers were leading the exercises only in one or two semesters, so in the Table 4 will be used initials only for the teacher, who taught the subject in more years. Under the symbols 1-5 are other teachers, always one symbol for one teacher.

Table 4. History of MEP evaluation

It is possible to see that after two, maximally three years the willingness of students to give feedbacks decrease a lot.

Apart from evaluation of education realized directly by universities, there are some informal evaluations. In the Czech Republic is probably the most popular source for informal evaluation of universities webpages “Primat”. Here it is possible to find not only evaluation of teachers and subjects from all Czech universities but also buy textbooks or notes from courses, find roommates or flats for rent etc. For each activity, for example subject evaluation, user receives a benefit – “bananas”. This is the currency for these webpages and it can be used for buy notes or preparations to exams, essays etc.

Also in this source the reliability of evaluation is very poor. The “Primat” statistics does not distinguish evaluation of lectures and exercises and use only general results for subject. If there are more teachers participated in a course, it is possible to receive only total number of evaluators and general evaluation for subject. For each teacher it exist information about general evaluation for the whole history of webpages. If someone would like to find teachers´ evaluation, he can receive only average grade for all taught subjects in the current year or general evaluation for each taught subject per whole history of webpages.

The positive but also problematic point of evaluation is the fact, that evaluator has to fulfill not only grades for 5 factors (clearness, approach, interest, usefulness and demandingness) but also write at least 50 symbols note.

4. Discussion

The problem with small number of students evaluated lecturers is possible to solve using mandatory evaluations. But there are some disadvantages, here. At one private Prague University the students were obliged to evaluate each lecture. After the lecture they were automatically asked for feedback in each login into University info system until they fulfill the feedback form. They had to give the feedback because without it they cannot continue use the system. The number of evaluations was close to 100 percent but validity was very low. Feedbacks gave also students they didn’t attend the lecture. Some of them chose for all items the best or the average rating but not all students. Author of this article had two times zero attendance in group consists of 10 enrolled students, but in feedbacks he found, for instance high level of “student´s interest of lecture” and sometimes two or three high ratings of items in set of average scored feedbacks (See Table 5). Therefore significant were particularly seldom added notes from students.

Table 5. Bottom up evaluation (no student was attended)

In this example it is clear that mandatory feedback is not good solution (since 2013 the University did not use it).

In Table 4 was shown that in first three years of new (computerized) form of bottom up evaluation was the participation of students quite good. They probably believed that their feedback is important and useful. After some time without changes they resigned. Only since 2012 is this evaluation partially used to formal evaluation of teacher. It can lead to situation that teachers will more ask for the feedback or on opposite they will not inform students about this possibility.

Separate area is selection of questions for evaluation. Unsuitable can be query to interest of explanation because some subjects are naturally more interesting than other, independently on teacher´s explanation. Interest is expressed differently in the case of obligatory or facultative subject. On other hand students can judge if the teacher is prepared for lecture, what is the quality of learning materials or if the subject connects to previous subjects.

5. Conclusions

Bottom up evaluation is an interesting approach to employees´ evaluation. It can bring new and important data. Each manager has to cooperate with his subordinates and because of motivation it is easier and more useful with satisfied employees.

Also (university) teachers should cooperate with students and they need motivated students. Bottom up evaluation at universities, it means a feedback of students, is also important. But it is necessary to find objective and representative data.

It is not enough only ask subordinates/students for feedback. If they do not feel meaning and importance of this activity, their feedbacks are useless.

In the case of ICT Prague students´ surveys was shown that participation in this activity is very low and therefore it is not possible to use these data. This problem increases the impossibility to determine number of students who should provide feedback, what can be solved by modification of data collection system. But students think that the feedback does not have an impact to change style of teachers´ explanation or even change of teacher.

Mandatory evaluation is not a solving; better approach to increasing validity and reliability of these surveys is to explain the importance of bottom up feedback and show changes in relation to the feedbacks. It correspondents with survey at the University Centre for Higher Education Quality (CHEQ) [25], where the rate of responses increases from 33.4 in 2005 to 51.9 in 2008. They used the results to preparing of Strategic Policy for Quality Improvement in Teaching and Learning and students belief that staff will act on their feedback.

Very important is to choose correct question for bottom up feedback. Subordinates/students cannot qualifiedly assess all managers/teachers competencies and activities. It is also important to hold the anonymousness of respondents.

5.1. Limitation

The results of this survey are possible to use in managerial bottom up evaluation but it is important to think about limitations.

It was focused only on university students. They very often cooperate with the teacher only once or twice per their studies but subordinates are in much closer and longer relationship with a manager. The number of taught students is much higher than number of managed employees. And the teacher leads each semester/year a new group of students but manager cooperate with same subordinates for long time.

All these three reasons lead to easier receiving of feedbacks at universities. Students are not so afraid to give a feedback; they are not encumbered by previous experience with the teacher and it is (usually) possible to hold anonymousness of respondents because of number of students.

On the other hand employees should be more motivated to give a feedback. If the results are used only to show the manager a possibility to his growth, the bottom up evaluation should be used.

Future studies should be focused on comparing student satisfaction, attendance and performance to verify if indirect methods of satisfaction detection are usable for teachers´ evaluation. Then it can be realized a comparative study of teachers and employers bottom up evaluation.

References

[1]   | Rizi, H.A. et al., “The effect of descriptive evaluation of elementary school third grade subjects on academic achievement of students,” Journal of Education and Practice, 5 (4), 6-13, 2014.1
[2]   | Zaharah, “The Effect of Certification and Competency towards Teachers Quality in MAN 3 High School Tangerang, Indonesia,” American Journal of Educational Research, 2 (5), 267-271, 2014.
[3]   | Daft, R.L., Management, Thomson South-Western, Mason, USA, 2008.
[4]   | Dessler, G., Human resource management, Pearson Prentice Hall, New York,USA, 2010.
[5]   | Bednaříková, M., Ehlová, Z., “Employee–important stakeholder in chemical industry enterprise,” Journal of MANagement and ECOnomics, (2), 109-117, 2009.
[6]   | Armstrong, Sh., Mitchell, B., The Essential HR Handbook, The Career Press, Inc., Franklin Lakes, N.J., USA, 2008.
[7]   | Vukotich, G., “The 360° process,” OD Practitioner, 42 (3), 24-29, 2010.
[8]   | Atkins, P.W.B., Wood, R.E., “Self- versus others' ratings as predictors of assessment center ratings: validation evidence for 360-degree feedback programs,” Personnel Psychology, 55 (4), 871-904, 2002.
[9]   | Kacmar, K.M., Wayne, S.J., Wright, P.M., “Subordinate Reactions to the Use of Impression Management Tactics and Feedback by the Supervisor*[dagger],” Journal of Managerial Issues, 21 (4), 498-517, winter 2009.
[10]   | Straka, M., “Alfa, a.s. distribution logistics system,” Acta Montanistica Slovaca, 15 (1), 34-43, 2010. (In Slovak).
[11]   | Asumeng, M., “The Effect of Employee Feedback-Seeking on Job Performance: An Empirical Study,” International Journal of Management, 30 (1), 373-388, Mar 2013.
[12]   | Vnoučková, L., “Factors of Personality leading to employee turnover in organization,” Acta Universitatis Bohemiae Meridionales, 16 (2), 185-198, 2013.
[13]   | Bracken, D.W., Rose, D.S., “When Does 360-Degree Feedback Create Behavior Change? And How Would We Know It When It Does?,” Journal of Business and Psychology, 26 (2), 183-192, June 2011.
[14]   | Berk, R. A., Thirteen Strategies to Measure College Teaching: a Consumer’s Guide to Rating Scale Construction, Assessment, and Decision Making for Faculty, Administrators, and Clinicians, Stylus Publishing, Richmond, Virginia, 2006.
[15]   | Ramsden, P., “Using research on student learning to enhance educational quality,” Deliberations, 2005. [Online]. Available: http://www.londonmet.ac.uk/deliberations/ocsld-publications/isltpramsden.cfm [Accessed June 18, 2013].
[16]   | Lian P, Sun Y, Ji Z., Li H., Peng J., “Moving Away from Exhaustion: How Core Self-Evaluations Influence Academic Burnout,” PLoS ONE, 9 (1), 2014.
[17]   | Moore, J.C., “A Synthesis of Sloan-C Effective Practices,” Journal of Asynchronous Learning Networks, 13 (4), 73-97, 2009.
[18]   | O’Leary, P.F., Quinlan, T.J., “Learner-Instructor Telephone Interaction: Effects on Satisfaction and Achievement of Online Students,” The American Journal of Distance Education, 21 (3), 133-143, 2007.
[19]   | Griggs, V., Blackburn, M., Smith, J., “The Educational Scorecard: The Start of our Journey,” The Electronic Journal of Business Research Methods, 10 (2), 121-131, 2012.
[20]   | Choudhury, K. A, “Model of Quality in the Education Service Sector: An Application of TOPSIS,” Journal of Strategic Management Education, 8 (2), 93-114, 2012.
[21]   | Maceli, K. M., Fogliasso, Ch. E. & Baack, D., “Differences of Students’ Satisfaction with College Professors: The Impact of Student Gender on Satisfaction,” Academy of Educational Leadership Journal, 15 (4), 35-45, 2011.
[22]   | Marsh, Sh. J., The Impact of Gender on Instructor Evaluations: A Thesis Submitted to the Graduate School in Partial Fulfillment of the Requirements for the Degree of Master of Science, Pittsburg State University, Pittsburg, Kansas, 2000.
[23]   | Kesselheim, J.C., “Balancing Education and Service in Graduate Medical Education: Data From Pediatric Trainees and Program Directors,” Academic Medicine 89 (4), 652-657, 2014.
[24]   | Kaymaz, K., “Performance Feedback: Individual Based Reflections and the Effect on Motivation,” Business and Economics Research Journal 2 (4), 115-134, 2011.
[25]   | Nair Ch. S., Bennett L., Mertova P., “Responding to the student voice: a case study of a systematic improvement strategy,” The TQM Journal 22 (5), 553-564, 2010.
[26]   | Šviráková, E. “Dynamic Iteration Method: New Approach to Cultural Events Management,” In Recent Advances in Business Administration, Marketing and Economics. (BAME 2013). Europment, 2013, 57-64.