ABSTRACT
This study reports on the development of a National Educational Research Agenda. The study surveys 912 stakeholders regarding their perceptions of pertinent education issues believed to be significant in the context of the current educational reform. Through a modified Delphi Technique (DT), thirteen faculty members in a National Arab-Gulf university reduced the 2332 remarks to nine categories containing 10 Research Statement/Questions (RS/Q). The knowledge based research agenda will produce a research program aimed at improving teacher practices and students educational outcomes from which these results and findings will be used to support new educational decisions and policies.
PDF Abstract XML References Citation
How to cite this article
DOI: 10.3923/jas.2014.2359.2366
URL: https://scialert.net/abstract/?doi=jas.2014.2359.2366
INTRODUCTION
A research agenda is a structured framework developed through the systematic involvement of stakeholders (Flesher, 1996). The purpose of a study agenda is to examine a number of issues that range from legitimate, agreed upon public concerns/issues or a narrowly defined agenda that becomes a public domain and dominant discourse (Cobb and Elder, 1983). There is a fine distinction made between a developed research agenda and research agenda setting. Research agenda setting somewhat differs from research agenda because it largely involves narrowing a set of varying issues and concerns that are subsequently focused on by decision-makers. Cobb and Elder (1983) explained that during research agenda setting, public organizations or political communities that focus on issues perceived to be significant by the organizations, special interest groups and individuals in power (Cobb and Elder, 1983). Research agendas are scientifically and methodologically developed based on issues that are grounded in actuality and are agreed upon by stakeholders and specialists in the field. An educational research agenda provides a voice for stakeholders based on general knowledge then substantiated by an expert knowledge.
In a number of fields, there are networks of specialized experts who develop research agendas that are used as knowledge base to strategize for research routes. For example, the health sciences, science education and defense field have developed research agendas that address significant issues in their fields (Lagemann, 2002). In the development of a research agenda in any field there are three important elements to take in account, (1) Focus on a critical issues in the profession, (2) Identify a collaborative approach through a network of specialists that often times cross academic disciplines and (3) Conduct periodic reviews of publications and knowledge bases (Radhakarishna and Xu, 1997).
Educational reform often requires policymakers to develop research agendas that need to focus on educational improvement and development. Research agendas usually are forced to address challenges that have surfaced because of a major overhaul in a system or structure (McKinsey, 2007). Also there is a need to acquire the knowledge about significant issues pertaining to educational development. In fact, this is the case regarding the educational changes in Qatar where the reform has created areas of needed research because of a number of challenges and new knowledge that has developed. Furthermore, it is rare to find higher education institutions developing research agendas yet alone national research agendas. Faculty in higher education are disposed with the expert knowledge about the challenges in the field. The experts have a key role of carrying out the research and the implementation of these national educational priorities resulting from the research agenda. When education is compared to other fields, such as the hard sciences, in education, there is a general knowledge about teacher practices in which a large volume of research exist on teacher practices and the relation between teacher practices and student outcomes (Donovan et al., 1999; Feuer et al., 2002). Therefore, research agendas must address specific complex educational needs that is knowledge-based, provide the validity of research needing study, generates practical application and provide enough evidence to be effective in reaching the established aims and goals of the research agenda. The application and approach can be generalized to other fields.
NEED AND CONTEXT OF THE STUDY
Currently there are no studies published in the Middle East or the Arab World on the development of a research agenda. More importantly, in the Arab-gulf region, research agendas are difficult to implement given the limited human capacity and educational knowledge base available (Lagemann, 2002). In this particular situation, Qatars educational reform provided the motivation to develop a research agenda. In 2001, the Qatari government was concerned about the learning outcomes of the Qatari school system. This was prompted by Qatari students low-ranking scores on the trends in International Mathematics and Science Study and the Program for International Assessment. The government embarked on an initiative that would develop an educational system to meet Qatars changing needs and begin the process of aligning it with international standards (McKinsey, 2007).
The educational reform was designed to catapult Qatar Education into a world class educational system that would drastically change the K-12 educational landscape by providing high quality schools, the development of human capacity, extensive teacher professional development, coupled with the innovative educational and social policies (Brewer et al., 2007). Several changes quickly took place. The Supreme Education Council was established replacing the Ministry of Education. To provide direction for the nations educational policy, the State of Qatar decided to gradually transform the centralized school system that was traditionally regulated by the Ministry Of Education (MOE), to a decentralized government funded independent school system. The long-term goals of the educational reform envisioned national curriculum and school leader and teacher standards, accountability through testing and the development of teacher certification and the transformation of schools into semi-autonomous independent schools (charter-like schools) (Zellman et al., 2009).
At the onset, Qatari educational authorities relied primarily on international consulting organizations without consideration of the available knowledge among experts within Qatar. However, recently, institutions inside Qatar are becoming involved in the educational reform, resulting in stakeholders concerns, national investments in professional development initiatives for government schools as well as developing a concern about the quality of educational staff and the resulting impact on instruction (Zellman et al., 2009).
One reason for the need of a research agenda was the numerous challenges presented by the educational reform especially of student outcomes and the necessity to develop a research agenda that examines these critical issues and provides sound knowledge based responses that will guide future policy and decision making. As many educationalists have a stake hold in the educational system, faculty of education knowledge is needed to understand both learning and teaching in Qatar. Because of this role in education and research, the author took the proactive and initial steps in the development of an educational research agenda that examined educational reform successes and challenges. Once the research agenda was developed it would be used as a knowledge base and a road map of sort to determine what studies to conduct that policy makers could use in decisions about relevant educational issues.
A number of stakeholders in Qatar were invited to participate through a nationwide survey questionnaire. The respondents identified critical education issues that have evolved because of the educational reform. The author approached the development of this research agenda by analyzing the survey of stakeholders and the expert consensus through a modified Delphi Technique.
Hence, this study describes the procedure for establishing an educational research agenda for an Arab-Gulf nation. The primary purpose of this study is to identify a set of agreed upon research statement/questions that serve as objectives for a national educational research agenda. The aim of this study is to provide knowledge based, logical, structured and empirically valid development of a national research agenda. This research agenda is grounded and contextualized, where experts are living, studying and researching in the field and thus providing key priority topics needing study. The key priority areas can be utilized by policy makers and other researchers to draw a map of the research priorities. It is believed that this method has empirical validity and logical operational aspects for use by organizations, research institutes, councils, special interest groups and other similar organizations that may need to develop an agenda.
A MODIFIED DELPHI TECHNIQUE
The study used the Delphi Technique (DT) which was designed to obtain a consensus among a panel of specialists through a controlled questionnaire, administered across dispersed periods of time (Dalkey and Helmer, 1963). During this process, group attitudes and priorities are registered through consensus process (Hasson and Keeney, 2011). The DT provides validity through one or more expert that rate the issues and challenges.
Hasson and Keeney (2011) extensive review offers a new perspective regarding the validity concept by framing the worthiness of the method through the following four constructs within trust worthiness, credibility, dependability, confirmability and transferability. Credibility is achieved through numerous rounds of specialists feedback. Dependability is evident when the panel consists of a diverse number of specialists in the field. Confirmability is attained by reporting the process undertaken to establish the DT technique and the variations reported. Transferability is established in the verification of the applicability of the DT findings.
For this study, a modified DT prioritized critical education issues. Rather than the panel responding to questions, the DT was modified to address research issues and developed them into researchable questions or statements and based on the following three basic DT traditions, (1) Members of the expert panel were aware of the subject matter or had knowledge of the educational developments (Dalkey and Helmer, 1963), (2) No. of rounds that address the same topic and (3) A feedback system was in place such that at each round researchers collected information from the previous rounds, organized the information and then presented it in subsequent rounds (Tapio, 2003).
METHODOLOGY
A nationwide open ended inquiry survey solicited information from stakeholders in Qatar. Using a divergent scheme, the survey was administered to independent schools, including teachers, school principals, parents, school administrators, government officials in the Supreme Education Council, Supreme Council of Information and Communication Technology (equivalent to the Ministry of Telecommunication), Qatar Petroleum (public/private national petroleum company), Qatar Gas (public/private national gas company) and all other higher education institutions. These public and private organizations make substantial social and economic contributions to Qatars economic development and growth. Therefore, they would have significant concerns and knowledge of the challenges that the educational sector faces in Qatar.
The questionnaire asked participants to list and then rank at least 3 educational issues from most to least important. Over a one-year span, the author received 912 responses from different stakeholders. A research assistant transcribed all of the remarks of the 912 respondents.
Table 1: | Respondents characteristics of the needs assessment |
Table 2: | Stakeholder responses on research ranked from highest to lowest |
A representative sample is presented in Table 1, the responses were classified into at least nine main issues that respondents thought needed to be studied and are illustrated on Table 2.
The 2332 remarks from stakeholders were considered knowledge issues and were classified by three researchers into nine categories. This was followed by 13 education faculty which served as members of a panel. Each individual on the panel possessed three important characteristics being specialist in the field (had knowledge of the reform), worked in the educational context and were aware of the national research priorities.
When developing a panel to use in the DT, Parente and Anderson-Parente (1987) suggested there should be at least 10 carefully selected panelists on the final panel who are experts and have knowledge in the field. These experts have specific dispositions in their content knowledge, including cross-sectional experiences gained from different backgrounds and rich specialized experiences (Delp et al., 1977). Second, these faculty members had work experience as educators in the field and were experts in their chosen field of study (Delp et al., 1977; Murry and Hammons, 1995). Finally, they possessed in-depth knowledge regarding the educational challenges facing Qatar specifically the challenges facing schools and other matters relevant to Qatar culture and education.
The 2332 remarks were collapsed into the nine categories in which no more than 18 remarks fit one category. In the first round the panel validated remarks which were fit in each category in the first panel meeting, developed the RS/Qs in the second round and prioritized the RS/Qs in the third and fourth round.
FINDINGS
Results from nationwide survey: From a nationwide survey, 912 responses were received totaling 2332 remarks which were collated and collapsed into nine categories. The remarks were collated and reclassified into nine categories by a committee of three researchers.
Expert panel round one: The expert panel (Table 3 for experts backgrounds) met four times to develop, validate and prioritize the most important remarks. In the first round the nine categories were distributed to the panel, the panel determined whether each of the remarks fit into the appropriate classification.
Expert panel round two: In the second round the panel developed RS/Qs as a knowledge base of educational issues (Campbell and Cantrill, 2001). For example, if a remark or issue stated, there is a need for teacher effectiveness, then a possible RS/Q was developed by experts as follows: What are the effective teacher training practices that could improve teacher practices and student outcomes? The DT process involved the panel developing the RS/Qs to correspond to each classification. The RS/Qs were tabulated in relation to each classification. During this process, the panel expressed the need to address some revisions regarding the wording of some of the issues. The expert panel had to make an evaluation regarding the best fit of the RS/Qs to the established categories.
Expert panel round three: The third round was a validation round for the previous two rounds because they focused on prioritizing RS/Qs from the panel, so only the most important RS/Qs were considered. Particularly, during the third round, the panel was asked to rank order the RS/Qs listing the top five RS/Qs. The RS/Qs for each classification were included and all other RS/Qs were removed.
Expert panel round four: The authors reviewed the five questions that had the highest frequencies. After the list of questions was collected and organized by the authors, the list was returned to the panel in a final round (round four) where the panel ranked the most important RS/Qs. The final DT round required that the panel to select and rank the top five RS/Qs with one of the following marks 1: High, 2: Middle and 3: Low. The panel ranked all of the RS/Qs once, i.e., each RS/Q could be ranked as high, middle, or low. Table 4 illustrates this procedure. For example, research question 1 had the highest frequency of being rated as high. The authors took these results and ranked the RS/Qs by the highest frequencies. Therefore, when the frequency came to three, i.e., it was listed as the first question and therefore, was a high priority to be addressed. The highest frequency-ranked question is listed as the most important question to be addressed, followed by the second and third-ranked questions. To maintain a high response rate and to preserve the original panel, the author emailed respondents who could not attend round 3 and 4 to gather their responses.
Table 3: | Background information for each of the panel |
Table 4: | Example of the ranked procedure |
Table 5: | Final list of categories |
The four rounds were completed in a five-week period. The proposed research agenda was based on the highest rank ordered (based on frequency) RS/Q. The prioritized list shown in Table 5 presents the RS/Qs of the research agenda. The categories were ranked according to the number of issues they addressed (Classification 1: Most, challenge 9: least) as tallied from the surveys.
DISCUSSION
This study involved the survey of educational issues from the knowledge base that stakeholders believe needed to be researched and studied. The approach elicited the knowledge of stakeholders and experts in the field. The nationwide survey was distributed to schools, educational institutions, universities, businesses and the Supreme Education Council. Knowledge issues were identified, collated and addressed using the DT requiring a number of rounds resulting in a list of RS/Qs that serve as the research agenda.
A scientific research agenda generally addresses challenges and key issues in an area of knowledge which yields findings that can support policy decisions supporting best practices and could possibly result in higher-quality teachers in schools.
Generally, there are a number of studies that have developed research agendas through extensive reviews that address knowledge specifically, in the areas of science education, vocational education and technology (Moed et al., 2004; Simmons et al., 2005; Lambeth, 2008). Research agendas provide a road map and knowledge base for research and a plan for conducting research with focus and structure. Furthermore, in education research outcomes from a research agenda shall generally improve the educational practices within the education system. Agendas can be used to produce effective teachers and leadership practices; To make empirically validated knowledge-based decisions and to build human capacities and to disseminate information regarding the best practices. The challenges in any educational system going through an educational reform require a well-developed research projects that address current challenges as well as provide insight into future research that will eventually add to the research agenda.
In having expert go through the exchange of diverse perspectives increases the decision accuracy. Even when such advice come from multiple sources in general i.e., a number of experts from different areas of speciality (Forster and von der Gracht, 2014), the outcomes are generally more credible and effective especially, if it provides diverse perspectives which are core characteristics of the Delphi technique. Further as suggested by Ertmer and Glazewski (2014) that research agendas can involve a number of researchers which could verify and validate the most significant research areas needing study through consensus (Cheyne et al., 2013; Dainty et al., 2013).
At the core was the need to improve teacher effectiveness and the improvement of student learning. This emphasis on teacher improved pedagogical practices and its impact on student performance serves as a road map that will lead to a substantive set of research studies over a five year span. The results and findings will be utilized for improvement of the current educational reform where teachers will be central to the improvement of the education in their power to positively or negatively affect student development and academic achievement.
The discussion that follows the addresses each classification in light of the current Qatar educational reform and provides additional insight into the reasoning behind the final selection of these challenges. Keep in mind, the panel had the knowledge base of the current educational landscape in Qatar and their individual specialties and experiences provide valuable insight regarding the selected challenges.
Challenge 1 (Curriculum issues): Teachers, students and parents indicate a concern that the current educational reform has placed too much emphasis on English as the medium of instruction. There is a legitimate concern about losing the Arabic culture and language in the midst of educational reform, given that math and science teachers are required to instruct in English, the national university requires students to demonstrate English proficiency and the cultural climate emphasizes English. Parents stated that their children are not able to correctly express themselves in Arabic and there is a fear that the teaching of Islam and cultural traditions is hampered by the increasing emphasis on communication in English.
Challenge 2 (Teaching/learning and assessments): There is great concern about the quality of teacher practices in Qatar, including what will improve the quality of teaching and learning. School principals and subject area coordinators are engaged in action research regarding how they can encourage staff to develop critical reflection skills aimed at their own practices and abilities with the goal of identifying professional development needs that are grounded in teachers understanding of their strengths and limitations. In addition, there is a need for subject area coordinators to help teachers develop professional development plans. The problem is that many teachers in the SEC schools lack the knowledge of how to engage in a deliberate critical reflection. Therefore, they lack the ability to assess their practices.
Challenge 3 (School systems, Governance and policy): Qatar currently faces a deficit of Qatari teachers. Most of the teachers in Qatar are expatriates from Arab countries who eventually return to their countries of origin. There is a movement to Qatarize schools by having Qatari leadership personnel, as well as the Qatari teachers, take ownership of the educational establishment given that they possess the necessary understanding of the Qatari culture. There are many arguments that suggest there is not enough Qataris to replace the large expatriate work force. Even if such Qatarization occurs, it is highly contested that there will be improvements evident from current Qatari teachers effectiveness.
Challenge 4 (Student behavioral issues): Qatari society has been predominantly nomadic and in the past two decades there has been a movement to sedentarize the population due to the new wealth generated out of the oil and gas sector. Currently, the majority of the Qatari population is located near the capital of Doha. Many male students of Bedouin backgrounds maintain rebellious attitudes, in that they perceive education as authoritative and generally have difficulty dealing with the structure and organization within the schools. Such behaviors are particularly evident among male students, in male segregated schools whereas female students generally exhibit conforming behaviors. Behavioral issues remain a challenge among male students and how to deal with behavioral issues and motivating students are key concerns to stakeholders and educators in Qatar.
Challenge 5 (Student motivation): It is often the case that the teaching in the SEC schools is limited to traditional teaching approaches with a lack of interactive learning. There are also discrepancies in the quality of the social interactions between school administrators and teachers. In our interactions with teachers, it is found that only a few teachers had full command of the motivational theories needed to entice males and females to develop academic interests and achieve positive academic outcomes. Thus, understanding the socio-cultural aspects of motivation could help researchers plan and organize teacher education to improve the quality of academic work in Qatari schools.
Challenge 6 (Social issues): There is little doubt that parents have been greatly affected by the educational reform. Parents have expressed many concerns such as the need to work more with their children at home due to their childrens weaknesses in Arabic, the need to hire tutors for their students, the difficulty with new reform requirements, dealing with new ways of studying and different teaching methods among many other concerns. These are vital issues that demand further examination.
Challenge 7 (Technology and education): Information Communication Technology (ICT) has been adopted in both public and private spheres and with particular intensity in educational institutions (Ten Brummelhuis, 1995; Drent and Meelissen, 2008). This has created a climate where ICT affects educational stakeholders including teachers, students and parents. In Qatar, ICT is beginning to generally change traditional communication and knowledge sharing in Qatari classrooms. There are many schools that are advanced in their use of ICT, whereas others lag behind. This has created a have and have-not knowledge situation and needs to be examined in order to reduce the current existing discrepancies.
Challenge 8 (Educational leadership): Currently in Qatar, there are national professional standards for school leaders and a new professional licensure system that is directly linked to the leadership standards. These standards set forth an understanding of leadership and the skills required for effective leaders. The national professional standards and the licensing process were developed and are being implemented with help from outside educational consultants and has raised many concerns regarding effective leadership in the Qatari context.
Challenge 9 (Educational research): Action research is currently an established part of the teacher professional development of the SEC schools. Schools are positioning teachers to be leaders in Qatari independent schools through the creation of scientific and action research centers creating research as an integrated part of the school culture. Specifically, there is a focus on how to support teachers who are conducting action research in key scientific areas, such as science and mathematics.
CONCLUSION
The research agenda provides the academic community a knowledge-base that could engage academics to improve programs, develop research projects and support policy on perennial educational issues needing improvement. Further, it shall enable experts to build a knowledge network to tie together teacher education to practices and student improvement. Experts who are active in research and expressed interest in examining some of the RS/Qs will play a significant role in carrying out the research studies. Many of the issues in the research agenda are interrelated in that they all contribute to students learning outcomes. There is plenty of data and knowledge about how to improve teachers, professional skills and knowledge in order to improve learning outcomes. Two main organizations can make use of this knowledge, these include the Ministry of Education as well as research incentive organizations that are working to improve the educational outcomes of schools. In partnership these organizations can use the knowledge base and implement the research agenda that will yield research findings and can contribute to a better decisions regarding the educational school process in Qatar.
Through the methodological approach and measurement exercise to what Carrillo and Batra (2012) called knowledge-based development measures. The applicability of such method could extend to other fields specifically the applied sciences and social sciences. Research agendas provide direction, a road map of research and areas of future making. Many concentration and fields may require the direction of research and lacks the local and international knowledge base. It is clearly evident that there are no decisions or policies that have a significant impact without concrete knowledge-based evidence to support the rationale in arriving at the new decisions/policies. This is why a proactive meta-research approach is significant because it can produce the knowledge and evidence leading to informed policies and decisions. Ideally, a strong partnership might be established between policy makers and researchers through the operationalization of this research agenda. Because a research agenda is based on a knowledge base it can yield through research the findings for which policymakers shape and refine the educational or scientific policies.
REFERENCES
- Campbell, S.M. and J. Cantrill, 2001. Consensus methods in prescribing research. J. Clin. Pharm. Ther., 26: 5-14.
CrossRefDirect Link - Carrillo, F.J. and S. Batra, 2012. Understanding and measurement: Perspectives on the evolution of knowledge-based development. Int. J. Knowledge-Based Dev., 3: 1-16.
CrossRefDirect Link - Cheyne, H., C. McCourt and K. Semple, 2013. Mother knows best: Developing a consumer led, evidence informed, research agenda for maternity care. Midwifery, 29: 705-712.
CrossRefPubMedDirect Link - Drent, M. and M. Meelissen, 2008. Which factors obstruct or stimulate teacher educators to use ICT innovatively? Comput. Educ., 51: 187-199.
CrossRefDirect Link - Dalkey, N. and O. Helmer, 1963. An experimental application of the Delphi method to the use of experts. Manage. Sci., 9: 458-467.
CrossRefDirect Link - Ertmer, P.A. and K.D. Glazewski, 2014. Developing a research agenda: Contributing new knowledge via intent and focus. J. Comput. Higher Educ., 26: 54-68.
CrossRefDirect Link - Feuer, M.J., L. Towne and R.J. Shavelson, 2002. Scientific culture and educational research. Educ. Res., 31: 4-14.
CrossRefDirect Link - Flesher, J.W., 1996. A conceptual framework for research agenda development. J. Ind. Teacher Educ., 34: 102-105.
Direct Link - Forster, B. and H. von der Gracht, 2014. Assessing Delphi panel composition for strategic foresight-A comparison of panels based on company-internal and external participants. Technol. Forecasting Soc. Change, 84: 215-229.
CrossRefDirect Link - Hasson, F. and S. Keeney, 2011. Enhancing rigour in the Delphi technique research. Technol. Forecasting Soc. Change, 78: 1695-1704.
CrossRefDirect Link - Murry Jr., J.W. and J.O. Hammons, 1995. Delphi: A versatile methodology for conducting qualitative research. Rev. Higher Educ., 18: 423-436.
Direct Link - Radhakarishna, R.B. and W. Xu, 1997. A review of subject matter topics researched in agricultural education and extension education. J. Agric. Educ., 38: 56-69.
CrossRefDirect Link - Simmons, P.E., H. Brunkhorst, V. Lunetta, J. Penick, J. Peterson, B. Pietrucha and J. Staver, 2005. Developing a research agenda in science education. J. Sci. Educ. Technol., 14: 239-252.
CrossRefDirect Link - Tapio, P., 2003. Disaggregative policy Delphi: Using cluster analysis as a tool for systematic scenario formation. Technol. Forecasting Soc. Change, 70: 83-101.
CrossRefDirect Link - Dainty, K.N., J.L. Jensen, B.L. Bigham, I.E. Blanchard and L.H. Brown et al., 2013. Developing a Canadian emergency medical services research agenda: A baseline study of stakeholder opinions. Can. J. Emergency Med., 15: 83 -89.
PubMed