Dear Label_Name,

Greetings! I hope you are well.

I am writing on behalf of an international publishing house, Science and Education Publishing, USA (http://www.sciepub.com).

Science and Education Publishing (SciEP) is a multidisciplinary open access publisher of Journals covering the fields of Science, Technology and Medicine. All SciEP publications are subject to high-quality peer review, editorial and production processes and are freely available online.

Welcome to the April, 2013 edition of SciEP newsletter, providing monthly summaries of articles on science & education related topics of interest to scientist and professionals. The mission of our journals is to provide an open access forum for publishing new findings on issues pertinent to scientific and academic field. We also invite you to submit original research papers, review articles, short reports, or case studies are considered for publication in our Multidisciplinary Journals.

American Journal of Public Health Research
DOI: 10.12691/ajphr-1-1-3

The Implementation of a Tele-Homecare System with Service Oriented Architecture and HL7 Message Transmission Standard
Ching-Sung Wang

American Journal of Medical and Biological Research
DOI: 10.12691/ajmbr-1-1-2

The Influence of Ethanol on Pyruvate Kinases Activity in Vivo, in Vitro, in Silico
Lelevich Sergey Vladimirovich, Khrustalev Vladislav Victorovich, Barkovsky Eugene Victorovich, Shedogubova Tatyana Aleksandrovna

American Journal of Medical Sciences and Medicine
DOI: 10.12691/ajmsm-1-1-1

Clinical and Histological Features in Wegener Granulomatosis
Li Cavoli Gioacchino, Passantino Rita, Bono Luisa, Tortorici Calogera, Ferrantelli Angelo, Giammarresi Carlo, Rotolo Ugo

American Journal of Infectious Diseases and Microbiology
DOI: 10.12691/ajidm-1-1-2

The Curse of Lymphatic Filariasis: Would the Continual Use of Diethylcarbamazine Eliminate this Scourge in Papua New Guinea?
Anne Lanham, Lillian Mwanri

We have released issues of some journals on February 2013.
American Journal of Public Health Research
American Journal of Medical Sciences and Medicine
American Journal of Medical and Biological Research
American Journal of Pharmacological Sciences
American Journal of Clinical Medicine Research
American Journal of Nursing Research
American Journal of Cancer Prevention
American Journal of Medicine Studies
American Journal of Infectious Diseases and Microbiology
American Journal of Cardiovascular Disease Research
American Journal of Biomedical Research
American Journal of Mechanical Engineering
American Journal of Microbiological Research
Journal of Preventive Medicine
American Journal of Food and Nutrition
American Journal of Hypertension Research
American Journal of Sports Science and Medicine
Journal of Cancer Research and Treatment
American Journal of Zoological Research
Journal of Biomedical Engineering and Technology
American Journal of Microbiological Research
Biomedical Science and Engineering

We are pleased to announce that submissions are now being accepted for peer review for all the journals across all our subject areas in SciEP.

Please submit your manuscript via our online submission system:
http://www.sciepub.com/author/signin

To unsubscribe from our mailing list, please click on the following link: unsubscribe

I am looking forward to hearing from you.

Kind regards,

David Rain
Email:join@sciepub.com
Science and Education Publishing (http://www.sciepub.com)

Stakeholders’ Views on Science Education - Curricular Delphi Study in Georgia

Marika Kapanadze*, Claus Bolte, Theresa Schulte, Ekaterine Slovinsky

Ilia State University, Georgia

Abstract: To compare the different stakeholders’ views about science education and contribute to bridge the gap between science communities in Georgia, a Curricular Delphi study was conducted as part of the International PROFILES Curricular Delphi Study on Science Education in the context of the PROFILES project, funded from the European Community’s Seventh Framework Programme. The aim of this curricular Delphi study was to engage different stakeholders from science or science education related areas in reflecting on contents and aims of science education as well as in identifying desirable aspects and approaches of modern science education with regard to scientific literacy. This paper presents the results of two rounds of Curricular Delphi Study in Georgia. The results from the first and the second rounds of this national curricular Delphi Study, makes clear that Georgian stakeholders stress the importance of scientific contexts, connected with everyday life in both educational and out of school settings..

Keywords: Curricular Delphi Study, educational reform, science education, stakeholders’ views

Received May 05, 2015; Revised June 16, 2015; Accepted June 17, 2015

Citation: Marika Kapanadze, Claus Bolte, Theresa Schulte, Ekaterine Slovinsky, Stakeholders’ Views on Science Education - Curricular Delphi Study in Georgia. American Journal of Educational Research, 3 (7). 897-906. January 2015. doi:10.12691/education-3-7-14

1. Introduction

The National Educational Reform in Georgia began in 2004. During these years, several versions of new national curricula for elementary and secondary schools were piloted and implemented. One of the focuses within these processes is the current situation of science education and the importance of a scientifically literate society. Much attention is paid to the development of new science curricula and the acknowledgement of more inquiry-based and student oriented approaches. In view of these developments, an important consideration is to establish a modern and contemporary understanding of desirable science education at schools of general education.

For a differentiated approach to such a discussion, it is necessary to bridge the gap between different groups of society that are involved with science and science education (referred to as “stakeholders”), taking into account their views and opinions about aspects of modern and desirable science education. The aim of the “Curricular Delphi Study on Science Education”- which the Ilia State University (ISU) conducted in the frame of PROFILES project [20] in accordance to [2-22], is to engage different stakeholders in reflecting on contents and aims of science education as well as in outlining aspects and approaches of modern science education. In this regard, the Curricular Delphi Study on Science Education in Georgia offers comprehensive insights into the set of opinions of different stakeholders in the society who are concerned with sciences and science education (such as: students, science teachers, science education researchers and scientists).

2. Main Research Question and Design of the Study

The main purpose of Curricular Delphi Study in Science Education is to collect the views and knowledge of stakeholders from different areas and classify them in a systematic and meaningful way [1,2,11-17]. The main research question of the study is: What aspects of science education do stakeholders consider advisable and pedagogically desirable for the individuals in the society of today and in the near future?

The Curricular Delphi Study on Science Education in Georgia in accordance to [2] is structured into three rounds (Figure 1). The first round offers the participants the possibility to express their ideas about aspects of contemporary and pedagogically desired science education in three open questions regarding “motives, situations and contexts”, “fields and methods” and “qualifications” [4]. The participants’ answers are classified into categories. In the second round, these categories are reported back to the participants for further assessment. They are asked to prioritize the given categories and to assess to what extent the aspects expressed by the categories are realized in practice. In the third round, the identified concepts are presented to the participants for further assessment.

This paper presents the results of the first and the second rounds of Curricular Delphi Study in Georgia.

Figure 1. Method of data collection and data analysis of the Curricular Delphi Study in Science Education [2]

2.1. Methodology

2.1.1. First Round

The participants’ statements of the first round were processed through qualitative and quantitative analyses.

Procedure and method of the qualitative data analysis

The PROFILES group from Georgia in the first round of this study has used questionnaire provided by Freie Universität Berlin (FUB) [4]. The questionnaire has been translated into Georgian language and adopted to the Georgian context. The statements received from the participants in the first round of the Curricular Delphi Study in Science Education were analyzed step-by-step as indicated in Figure 2 following [1].

Figure 2. Overview of the procedure of the data analysis

As a first step in the analysis of the results from this study, the classification system provided by the team of FUB (Freie Universität Berlin) were examined for their applicability to classifying the Georgian stakeholders’ statements [2,12]. The statements of the response sheets were prepared following the qualitative content analysis approach according to [19]. All statements from the questionnaires were paraphrased, grouped, summarized and systematized due to the classification system provided by FUB (Freie Universität Berlin). After a detailed examination of 20 questionnaires (step 2), the prior classification system was modified and completed by additional categories that appeared in the Georgian stakeholders’ statements (step 3).

The set of categories was subdivided into four different parts (I - situations, contexts and motives, II - fields and III- qualifications, IV - methodical aspects). Part II according to FUB (Freie Universität Berlin) system was subdivided into part II a (scientific concepts and topics) and part II b (scientific fields and perspectives). Part IV (methodical aspects) was established as an additional part (Table 4).

In the following course of data analysis, the participants’ statements were examined by applying the modified category system to the statements of all answer sheets in order to assign those statements to the respective category (step 4). In some cases an assignment and classification of a statement to one of the existing categories was not possible, which is why the list of categories had to be revised again by either modifying existing categories or adding new categories (step 5). After examining all statements with the revised list of categories, for an objectivity test a set of 20 questionnaires was randomly chosen and examined by two independent coders (step 6). The established classification system (Table 4) was confirmed (step 7) and maintained for final labeling and coding of all statements concerning the data transformation into SPSS (step 8). The step 9 data was then analyzed using statistical methods and the results were summarized (step 10).

For the objectivity of the qualitative analysis of the statements the method of calculating the inter-rater agreement was used according to the following formula [1,12-18]:

With N₊ being the number of cases in which the positive coding of the two different coders matches, and N₋ being the number of cases in which only one coder coded a category positively, this quotient takes only into account positive coding and is thus considered as a rather strict measure for the inter-rater agreement [12].

Procedure and method of the quantitative data analysis

As mentioned at the beginning, the first round offered participants the opportunity to express their ideas in three open questions. They had the choice to fill out up to 5 form sheets. In order to prepare the results of the qualitative analysis for quantitative statistical analyses, the data was coded in the following manner [1].

Although a category could have been referred to several times on one form sheet, a certain category reference was only counted once per form sheet. A category stated on a form sheet was coded with “1”, every category that was not mentioned was coded with “0”. When calculating the relative frequency, multiple entries of the same category of a person were not considered.

In order to get a more differentiated overview over the empirical data, descriptive statistical analyses were carried out taking into account both the total sample and the four sub-samples. In the quantitative analyses categories were considered that were mentioned rarely (≤5%) or often (≥20%). The analyses of the frequencies were guided by the questions which general statements could be derived from the participants’ responses and which distinctive features appeared after the analyses of the different sub-samples. In order to gain answers to those questions, the following characteristic values were taken into account:

Number of all form sheets filled out by the participants

•  Average number of form sheets per person

•  Number of all categories mentioned by the participants

•  Average number categories mentioned per person

•  Relative frequencies of the categories regarding

•  the total sample

•  the different four sub-samples

2.1.2. Second Round

The second round of the Curricular Delphi Study is based on questions which resulted from the first round [1,2,12,17,18]. Following the Delphi method, a two-part questionnaire was sent to the participants [4]. The participants were asked both to prioritize the given categories and to assess to what extent the aspects expressed by the categories are realized in practice.

For both cases, a six-tier rating scale was used. The coding of the answers ranged from 1 (very low priority/to a very low extent) to 6 (very high priority/to a very high extent). Figure 3 shows part of the questionnaire used in the first part of the second round.

The data in the first part of the second round was analyzed through descriptive and variance analytical methods. Priority and practice assessments were taken into account during the analyses of the results as well as priority-practice differences, which were determined by subtracting the practice values from the priority values.

Figure 3. Design of the questionnaire of the first part of the second round

For identifying empirically sound concepts regarding science education that are considered important, participants of the second round in part 2 were asked to combine from the given set of categories those categories that seem especially important to them in their combination. Figure 4 shows the design of the questionnaire of the second part of the second round with the given task:

Figure 4. Design of the questionnaire of the second part of the second round

The category combinations from the second questionnaire of the second round were analyzed by means of hierarchical cluster analyses, using the Ward method and squared Euclidian distance [8].

3. Results

3.1. Results of the First Round

3.1.1. Sample of the First Round

A total of 186 potential participants (‘experts’) in Georgia were asked via e-mail to fill out the Delphi questionnaire. 110 stakeholders took part in the first round of the Curricular Delphi Study on Science Education in Georgia (Table 1).

As it is seen from Table 2, 31% from the participants were students, 27% science teachers, 12% science educators, 25% Scientists and 5% others. The group of ’’Students’’ refers to students between the age of 15 and 17. ’’Science education students’’ at university refer to students whose major subject is primary science, biology, or physics and education respectively. ’’Trainee science teachers’’ are teachers who have just started their career as a teacher; ’’Science teachers’’ are experienced teachers in the fields of biology, chemistry or physics. The group of ’’Teacher educators’’ refers to teacher educators in the education department of universities, as well as education experts who work at the Teachers House (responsible for the teacher trainings) or at the Curriculum Department (responsible for curricula development) in the Ministry of Education and Sciences of Georgia. The group of ’’Scientists’’ consists of scientists who work in the field of biology, chemistry or physics at the universities or in different academic science institutes. The group of ’’Others’’ refers to the people who worked in science (physics, chemistry, biology), but left their profession for different reasons and have other professions at the time of the survey.

Table 1. Structure of the Sample, 1st round

3.1.2. Results and Analysis of the First Round

As mentioned in the methodology part 2.1., the method of calculating the inter-rater agreement was used to account for the issue of objectivity in qualitative analysis of the statements [1,12-18].

The results of the objectivity test are shown in Table 2.

Table 2. Results of the inter-rater agreement of two different coders after coding 20 questionnaires

The inter-rater quotients range between 75% and 84%, which shows that the procedure of the qualitative data analysis met the demands for objectivity [16].

Results of the qualitative analysis

A final classification system for the analysis of the participants’ statements was developed and established. The classification system consists of 100(+9) categories, which are listed in Table 3.

In most cases, the categories, which are approved in Georgia, agree with categories established in previous Delphi studies [2,12-18] and refer to aspects of modern science education [9,10]. In Table 3, additional categories in the Georgian Delphi study to the ones of the German system are indicated in italics. For part I, 19 categories were developed – 2 of them are different from the FUB categories. The sub-parts II a and II b consist of 21 (13 Freie Universität Berlin (FUB) and 8 additional Ilia State University (ISU) categories) and 35 (20 FUB and 15 additional ISU categories) categories respectively. The part III contains 25 (13 FUB and 12 additional ISU) categories. The additional part IV consists of 9 (3 FUB and 6 ISU) categories [13,14].

Table 3. Overview of the categories for the analysis of the experts’ statements

Discussions

The procedure of statement analysis led to a systematization of the participants’ statements [2]. On the basis of the Georgian stakeholders’ statements, it was necessary to make some modifications regarding the number of categories provided by FUB (Freie Universität Berlin). According to the requirements, the statements were differentiated as necessary and summarized as possible, the number of categories in the Georgian system was extended to total number of 100 (+9). In order to differentiate methodological aspects from part II, an additional category IV was developed by FUB (Table 3).

Results of the quantitative analysis

The participants used the opportunity to fill out up to 5 form sheets to a very different degree. The average number of different categories mentioned per participant was considered as well in order to determine the amount of differentiation of statements. For this purpose, it was only taken into account if a category was mentioned in general and not how many times it was mentioned by a person on different form sheets. The results are shown in Table 4. As it can be seen, the average number of different categories mentioned by a participant regarding the total sample was 10.00. The greatest difference between the average numbers of different categories mentioned per person regarding the sample groups can be found among the group of students (7,2 different categories per person on an average) and others (13,3 different categories person on an average).

Table 4. Number of different statements per participant – total sample and sample groups

Figure 5. Overview over the categories that were mentioned rarely (≤5%) or often (≥20%): Mean percentages regarding the whole sample

Figure 5 presents the frequencies of the categories which were mentioned by the whole sample rarely (≤5%) or particularly often (≥20%).

As Figure 5 shows, the categories ’’Everyday life’’ and ’’Students’ interest/motivation’’ were mentioned by a high percentage of participants (40% and 52%) as context to be considered in science education. Other categories, mentioned quite often by the participants were “Environmental awareness’’ (37%), “Rational thinking /analyzing / drawing conclusions” (33%), “Applying knowledge” (32%), “Experiments, practical work” (30%), “Specialized knowledge” (29%), “Civic” (28%), “Matter in everyday life” (24%), “Human biology” (22%) and “General biology” (21%). Only 5% of the participants mentioned aspects concerning the category “Metacognition” and “Universal science law”.

The distribution of the categories by groups are shown in Table 5:

Table 5. Overview the distribution of categories by groups

It is visible in Table 5 that students, science educators and the group ‘Others’ mentioned a limited number of given categories, in contrast, only 15-16% of the given categories are not mentioned by science teachers and scientists.

3.2. Second Round

3.2.1 Sample of the Second Round

As it is shown in Table 6, 110 stakeholders who participated in the first round of the Curricular Delphi Study on Science Education in Georgia were asked to participate in the second round. 83 stakeholders followed the request and took part in the second round as well.

Table 6. Structure of the sample, 2nd round

In Table 6, it is an increased number of in-service teachers and science educators in the second round can be identified. The reason of this might be some exchange between the groups – for example - scientists or trainee teacher became an in-service teacher etc.

3.2.2. Results and Analysis of the Second Round

As the second round of this curricular Delphi Study consisted of two parts, the results are accordingly be divided two parts as well. The first part describes the descriptive and variance statistical analyses, the second part the hierarchical cluster analyses.

Results of the descriptive-statistical analyses

First, selected results from the descriptive-statistical analyses with regard to the priority and practice assessment as well as to the calculated priority-practice difference are presented. These analyses were made on the data basis of the five different sample groups (students, teachers, scientific educators, scientists and the others).

The results of the priority of the total sample, practice of the total sample and priority – practice differences of the total sample are shown in Table 7. This table displays the categories that show particularly high or low mean values in the total sample, listing the top ten and low ten categories in descending order.

The highest mean value with the regard to the priorities in participants responses is “Acting reflectedly and responsibly” (mean value = 5.42). The ten highest categories listed in this table refer to aspects related to general and inquiry based science education. The lowest category in this table is “Cosmetology” (mean value = 3.54). The most of the lowest ten categories listed in Table 7 refer to aspects of specific fields of science, such as Atomic/nuclear physics, Relativistic theory or Pharmacology.

For the practice assessments by the total sample, Mathematics is assessed with the highest mean value (M=3.96) by the total sample. The lowest mean value is 2.09 and relates to Occupation.

Table 7. Top ten and low ten categories of the priority assessments, practice assessment and priority – practice differences of the total sample group

Results of the cluster analysis

To identify important concepts of science education, the Georgian participants in the second part of the second round were asked to combine from the given set of 109 categories those categories which seemed important to them in their own combinations The results of the cluster analyses are based on the form sheets which the participants of the second round were asked to fill out the second part of the questionnaire. The received clusters were interpreted as three concepts, which are given in Figure 6:

Figure 6. Three concepts – results of the cluster analysis

The labeling of these three clusters is based on the FUB concept [5] because of similarities and overlaps in terms of content. It is important to note that these three concepts are interrelated, and not mutually exclusive concepts of desirable science education. These empirically developed concepts will be analyzed in the third round of the study.

3.3. Third Round

The third round of the International PROFILES Curricular Delphi Study on Science Education is about considering and further processing the findings from the hierarchical cluster analysis of the second round [6]. In particular, the aim of the third round of the International PROFILES Curricular Delphi Study on Science Education is to identify which priority and reality assessments the participants assign to the three concepts of desirable science education derived from the hierarchical cluster analyses in round 2; in addition, to find out where, in the opinions of the participants, priority and realization in science educational practice drift apart. The results of the third round of this study will be reported separately.

4. Discussions of the Results of the First and the Second Round

In the first round of the Curricular Delphi Study on Science Education conducted by Ilia State University in Georgia, 110 participants,– different stakeholders engaged with science education, took part. Eighty three of them returned in the second round. The results of the analyses show general tendencies as well as specific focuses of the participants. The calculation of the different frequencies after the first round illustrates the emphases made in the statements of all participants. A differentiated view on the category frequencies of the different sample groups shows that the different groups feature different focuses and thus deviate in several cases from each other regarding the relative frequency of mentioning the different categories. In general, students’ interest and motivation, as well as rational thinking, analyzing, drawing conclusions and applying knowledge were pointed emphasized by all groups. Students highlighted more general science subjects – physics, chemistry and biology, while teachers pointed to more experimental work, inquiry skills and environmental awareness. Environmental awareness was highlighted also by science educators and scientists, as well as by the group of “others”.

Analyzing the results from the first and second round of this curricular Delphi study shows that Georgian stakeholders stress the importance of scientific context, connected with everyday life in both educational and out of school settings. The stakeholders also stressed the priority of scientific inquiry and development of general skills.

These findings from Georgian Curricular Delphi Study can be related to the definition of European Commission about the main goals of science education in Europe – “the key point is equipping every citizen with the skills needed to live and work in the knowledge society by giving them the opportunity to develop critical thinking and scientific reasoning that will enable them to make well informed choices” [21]. In addition, similar aspects are also initiated in the PROFILES project philosophy, which is identified as “education through science” [7].

5. Conclusion

Stakeholders’ involvement and cooperation in the PROFILES project are seen as extremely important in order to bridge the gap between the communities who are involved in school science education. To fulfill these needs PROFILES project initiated DELPHI Study on Science Education. The outcomes of this study in Georgia stressed the importance of scientific inquiry and development of general skills.

For many years in Georgia, as well as in all the Post-Soviet countries, content-based learning has been the main approach in system of education. The system was highly centralized and there were unified methodological approaches across all the Soviet Union countries.

Among the participants and other stakeholders, there is with regard to the National Educational Reform in Georgia, which has begun in 2004, great interest in the Delphi study and its results. Several versions of new curricula were in the course of the National Educational Reform in Georgia piloted and implemented during 2004-2010. In the 2011-2012 school years, the new revised version of the national curriculum was implemented on elementary level and in 2012-2013 also for all public basic and secondary schools. Ongoing reform radically changed the educational system. New requirements are suggested for science teaching within this reform as well. Inquiry-based and problem-based learning are main methods suggested in the Georgian Science Curriculum nowadays. These new requirements also correspond to the results we obtained in the course of the Curricular Delphi Study on Science Education in Georgia.

The Ministry of Education and Science of Georgia announced the beginning of National Curricula revision. Therefore, it is very important to have different views of stakeholders about science education in Georgia today. The outcomes of Curricular Delphi Study on Science Education in Georgia can be used as recommendations for improving the science curricula in the country and also for program development of pre-service and in-service teacher preparation courses. The results of this study were already used for the designing of continuous professional development programs for teachers under the PROFILES project [15]. These programs were implemented in Georgia in 2013 – 2015 in terms of the inquiry based teaching and learning as well as the needs for “education through science” [7,13,14,15].

Acknowledgements

The research leading to these results has received funding from the European Community’s Seventh Framework Programme under Grant Agreement Number 266589. The support offered by the European Commission, through the PROFILES project, is gratefully acknowledged.

Authors of this paper express their thanks to DAAD (Deutscher Akademischer Austausch Dienst) for the support of the cooperation between Ilia State University, Tbilisi, Georgia and Freie Universität Berlin, Germany.

References

[1]   | Bolte, C. (2003). Chemibezogene Bildung zwischen Wunsch und Wirklichkeit – Ausgewählte Ergebnisse aus dem yweiten Untersuchungsabschnitt der curricularen Delphi-Studie Chemie. ZfDN, 9, 27-42
[2]   | Bolte, C. (2008). A Conceptual Framework for the Enhancement of Popularity and Relevance of Science Education for Scientific Literacy, based on Stakeholders’ Views by Means of a Curricular Delphi Study in Chemistry. Science Education International, 19(3), 331-350.

[3]  
Bolte, C., & Schulte, T., (2014). Stakeholders Involvement and Interaction in PROFILES. In: C. Bolte, & F. Rauch, (Eds.). Enhancing Inquiry- based Science Education and Teachers’ Continuous Professional development in Europe: Insights and Reflections on the PROFILES Project and other PROJECTS funded by the EUROPEAN Commission. (pp. 34-39). Berlin: Freie Universität Berlin (Germany) / Klagenfurt: Alpen-Adria-Universität Klagenfurt (Austria).

[4]   | Bolte, C., & Schulte, T. (2011). PROFILES Curricular Delphi Study on Science Education. Interim Report on the First Round of the FUB Working Group. Polyscript (Status July 2011). Unpublished.
[5]   | Bolte, C., & Schulte, T. (2012). PROFILES Curricular Delphi Study on Science Education. Interim Report on the Second Round of the FUB Working Group. Polyscript (Status May 2012). Unpublished
[6]   | Bolte, C., & Schulte, T. (2013). PROFILES Curricular Delphi Study on Science Education. Interim Report on the Third Round of the FUB Working Group. Polyscript (Status February 2013). Unpublished.

[7]  
Bolte, C., Streller, S., Holbrook, J., Rannikmae, M., Hofstein, A., Mamlok Naaman, R., & Rauch, F. (2012). Introduction into PROFILESProfessional Reflection-Oriented Focus on Inquiry-based Learning and Education through Science. In: Bolte, C., Holbrook, J., & Rauch, F. (Eds.). Inquiry-based Science Education in Europe: Reflections from the PROFILES Project. (pp. 31-42). Berlin:Freie Universität Berlin. Klagenfurt, Austria: Alpen-Adria-Universität Klagenfurt

[8]   | Bortz, J. (2005). Statistik für Human- und Sozialwissenschaften. Heidelberg: Springer.
[9]   | Bybee, R.W., McCrae, B., & Laurie, R. (2009). PISA 2006: An Assessment of Scientific Literacy. Journal of Research in Science Teaching, 46(8), 865-886.
[10]   | Fensham, P. J. (2009) Real World Contexts in PISA Science: implications for the Context-Based Science Education.Journal of Research in Science Teaching, 46(8), 884-896.
[11]   | Häder, M. (2009). Delphi-Befragungen: ein Arbeitsbuch (2. Aufl.) wiesbaden: VS, Verl. Für Sozialwissenschaften.
[12]   | Häußler, P., Frez, K., Hoffman, L., Rost, J & Spada, H. (1980). Physikalische Bildung: Eine curriculare Delphi-Studie. Teil I: verfahren und Ergebnisse. IPN-Arbeitsbericht 41. Institut für Pädagogik der Naturwisswnschaften Kiel. IPN-Arbeitsbericht 41. Kiel: IPN.

[13]  
Kapanadze, M., & Slovinsky, E., (2014). Inquiry-based Science Education within the Project PROFILES in Georgia. In In: C. Bolte, & F. Rauch, (Eds.). Enhancing Inquiry-based Science Education and Teachers’ Continuous Professional development in Europe: Insights and Reflections on the PROFILES Project and other PROJECTS funded by the EUROPEAN Commission. (pp. 112-118). Berlin: Freie Universität Berlin (Germany) / Klagenfurt: Alpen-Adria-Universität Klagenfurt (Austria).

[14]  
Kapanadze, M., & Slovinsky, E., (2014). Stakeholders’ Views on Science Education in Georgia – Curricular Delphi Study. In: C. Bolte, J. Holbrook, R., Mamlok-Naaman, & F. Rauch, (Eds.). Science Teachers’ Continuous Professional Development in Europe. Case Studies from the PROFILES Project (pp. 24-30). Berlin: Freie Universität Berlin (Germany) / Klagenfurt: Alpen-Adria-Universität Klagenfurt (Austria).

[15]   | Kapanadze, M., Bolte,C., Schneider, V., Slovinsky, E., (2015) Enhencing Science Teachers’ Continious Professional Development in the Field of IBSE and other Aspects of Inovative Science Lessons. Journal of Baltic Science Education, 2015, Vol.14, (2), (pp. 254-266).
[16]   | Landis, J. R., Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics 33:159-174.
[17]   | Listone, H. A., &Turoff, M. (1975). The Delphi Method: Techniques and Applications. Reading, Mass. u.a: Addison-Wesley.
[18]   | Mayer, J. (1992). Formenvielfalt im Biologieunterricht: Ein Vorschlag zur Neubewertung der Formenkunde. Institut für die Pädagogik der Naturwisswnschaften Kiel. IPN – Schriftenreihe 132. Kiel: IPN
[19]   | Mayring, P. (1983) Qualitative Inhaltsanalyse: Grundlagen und Techniken. Weinheim. u.a: Beltz
[20]   | PROFILES. (2010). The PROFILES Project. www.profiles-project.eu.
[21]   | Rocard, M. (2007). Science Education NOW: A renewed Pedagogy for the Future of Europe, Brussels: European Commission. Retrieved from: http://ec.europa.eu/research/science-society/ document_library/pdf_06/report-rocard-onscience- education_en.pdf (2.06.2015).

[22]  
Schulte, T., & Bolte, C. (2012). European Stakeholders Views on Inquiry Based Science Education – Method of and Results from the International PROFILES Curricular Delphi Study on Science Education Round. In C. Bolte, J. Holbrook, & F. Rauch (Eds.), Inquiry based Science Education in Europe - First Examples and Reflections from the PROFILES Project (pp. 42-51). Berlin: Freie Universität Berlin, Print: University of Klagenfurt (Austria), pp. 31-42.

[23]  
Schulte, T., & Bolte, C. (2014). Case Studies on Science Education based on Stakeholders‘ Views Obtained by Means of a National/International PROFILES Curricular Delphi Study. In: C. Bolte, J. Holbrook, R., Mamlok-Naaman, & F. Rauch, (Eds.). Science Teachers’ Continuous Professional Development in Europe. Case Studies from the PROFILES Project (pp. 19-24). Berlin: Freie Universität Berlin (Germany) / Klagenfurt: Alpen-Adria-Universität Klagenfurt (Austria).