One of the methods that play an important role in the measurement and evaluation
of data structuring in science education is concept maps. Concept maps show
the relations that the students establish between the concepts in their minds.
Concept maps contribute to the students in both meaningful learning and learning
how to learn (Novak and Gowin, 1984). Concept maps were
developed in 1972 in the course of Novak research programme at Cornell University
where he sought to follow and understand changes in students knowledge of science
(Novak and Musonda, 1991). That research was based on
learning psychology of Ausubel (1978). According to Ausubel,
learning is about what the learner already knows. In that respect, concept maps
contribute to setting up relationships between the existing knowledge and new
knowledge for meaningful learning. Concept mapping activities are more effective
than reading passage, attending lectures and participating in class discussions
for attaining knowledge retention and transfer (Nesbit and
Concept maps are tools used in submission and organization of knowledge. These
tools are generally made up of concepts, which are circled or put in a box and
lines drawn for indicating the relationship between these concepts and linking
word or word groups written on them. Concepts and the relationships between
them are established in compliance to a certain hierarchical order. The combination
of two concepts and a labeled line forms a proposition. Another characteristic
of concept maps is cross-links. Cross-links are relations between concepts in
different segments (White and Gunstone, 1992).
When we survey literature, there are many studies on concept maps in the field
of science, related to practice in different fields and for different purposes.
Concept maps are considered as auxiliary tools for obtaining information related
to existing knowledge structure of students (Ruiz-Primo and
Shavelson, 1996; Zele et al., 2004), assessment
of knowledge (Rice et al., 1998), adding new knowledge
to the concepts which already exist in the minds of the students (Kinchin
et al., 2000), investigating of achievement gain and conceptual reorganization
(Carter et al., 2003), assessing conceptual understanding
(Kaya, 2008; Tekkaya, 2003), tracing
conceptual development (Novak, 2005), investigating conceptual
change (Uzuntiryaki and Geban, 2005), exploring cognitive
structure (Tsai and Huang, 2002; Thompson
and Mintzes, 2002), investigating students mental model (Chang,
2007), teaching/learning of concepts (Sket and Glazer,
2005), knowledge integration (Fuang and Diao, 2008),
investigating misconceptions (Lee, 2007) and improving
problem solving skills (Bagno et al., 2000; Lee
and Nelson, 2005).
Concept maps can be formed by the students by using pencil and paper (this
technique is known as construct-a-map-from-scratch) or can be complemented by
the students by leaving some parts empty on the previously prepared concept
maps (fill-in-the-nodes where nodes are left blank, fill-in-the- lines are left
blank) (Ruiz-Primo et al., 2001a, b;
Schau et al., 2001). Among these techniques, construct-a-map-from-scratch
technique is a low-directed one where the others are high-directed. Application
of high-directed maps is more practical since it is easier and takes less time.
However, although the construction of low-directed maps is harder and requires
more time, they reflect better knowledge structure of students (Ruiz-Primo
et al., 2001a). Since this present study was performed in order to
expose knowledge structure of the students, it was considered more suitable
to use construct-a-map technique. One of the most important components of concept
maps is the relationship between concepts. Analysis of these relationships is
very important as stressed by Novak and Gowin (1984). In
the recent studies it is suggested that these relationships can be examined
statically and dynamically (Safayeni et al., 2005;
Derbentseva et al., 2007). Static relationships
reflect organization, description and identification of information. Dynamic
relationships indicate the functionality of consistency among concepts. Concept
maps, before all else, are based on static relationships, however, dynamic relationships
are also very important.
McClure et al. (1999) categorized Novaks comments
on using concept maps in science education in four groups: (1) learning strategy,
(2) instructional strategy, (3) strategy for planning curriculum and (4) means
of assessing students understanding of science concepts. For example, Romance
and Vitale (2001) used concept maps for in-depth science concept instruction,
Hilbert and Renkl (2008) used concept maps for learning
strategy. In this study, concept maps are used in order to assess students
understanding in work-energy subjects related to group 4.
In science education, concept maps are frequently used as an assessment method
(Ruiz-Primo and Shavelson, 1996; McClure
et al., 1999; Slotte and Lonka, 1999; Zieneddine
and Abd-El-Khalick, 2001; BouJaoude and Attieh, 2003;
Stoddart et al., 2000; West
et al., 2002; Yin et al., 2005; Hollenbeck
et al., 2006; Liu et al., 2005). Different
scoring methods were used in evaluation in earlier studies of literature. One
of these methods is based on scoring by comparing students maps with criterion
or master map (Rye and Rubba, 2002). Another method is
founding scoring of concept maps on correctness of propositions (Rice
et al., 1998). Nicoll et al. (2001)
presented an alternative analysis method for concept map with large number of
concept nodes and links. This method is a three-tier (utility, stability, complexity)
system for evaluating concept map links. McClure et al.
(1999) assessed concept maps with 6 different scoring methods (1) holistic,
(2) holistic with master map, (3) relational, (4) relational with master map,
(5) structural and (6) structural with master map. Structural scoring method,
adapted from the scoring method defined by Novak and Gowin
(1984), is used in this study.
Ruiz-Primo and Shavelson (1996) reported that reliability
and validity studies were required related to different concept mapping techniques
prior to using and evaluating concept maps. McClure et
al. (1999) stated that reliability of six different scoring methods
used for the evaluation of concept maps, change between 0.23 and 0.76 according
to g-coefficients calculation. Among these methods, relational with master map
was determined as the one with highest reliability and structural with master
map as the one with lowest reliability. Rice et al.
(1998) used concept maps in evaluating the level of knowledge and understanding
in science achievement, graded maps with two researchers in his studies and
found interrater agreement as 0.98. Ruiz-Primo et al.
(2001) compared fill-in-the-map and construct-a-map-from-scratch which are
two different concept map techniques from different view-points in their studies.
Average interrater reliability for concept maps was found as 0.99 graded by
three raters with three different scoring types: proposition accuracy, convergence
Making a study on the correlation between multiple choice test scores and concept
map scores for validity of concept maps, is a very common situation in literature.
Past and recent many studies carried out for the validity of concept maps, share
the same opinion that there are middle and high correlations between concept
map scores and standardized multiple choice scores of the students, although
value ranges change (Rice et al., 1998; Schau
et al., 2001; Ruiz-Primo et al., 2001b;
Rye and Rubba, 2002).
When literature is considered, it is interesting that there are limited number
of studies related to concepts maps concerning work and energy. Goldring
and Osborne (1994) made a research on the level of students in understanding
some concepts related to energy. Research was made with 75 students. A questionnaire
consisting of 26 questions of 6 groups which contained qualitative and quantitative
knowledge and understanding was made. At the end of the research, it was determined
that only five students (7%) achieved the highest score. It was observed that
approximately 50% of the students do not have enough understanding of basic
concepts and ideas related to energy. These findings were further supported
by oral interviews. A weak correlation was found between answers of the quantitative
questions and qualitative questions. In this study, it was proposed to use concept
maps for the improvement of science understanding of students.
Cornwell (1996) explored a number of pedagogical techniques
and tool including a computer algebra system, the dynamic simulation program
working model and concept maps. Mechanical engineering students were asked on
a scale from one to five how lecture, in-class group work, homework, working
model demonstrations, concept maps and computer algebra helped the students
in five areas. The five areas were: (1) problem solving skills, (2) learning
and comprehension of dynamics material, (3) motivation and interest in dynamics,
(4) ability to visualize problems and develop intuition and (5) enjoyment of
dynamics. When the results were analyzed, in dynamics topic where the basic
principles of Newtons Laws and concepts like work-energy and impulse-momentum
were used, concept maps and lecture were determined as the most auxiliary factors
for problem solving skills and learning/comprehension.
In another study that carried out relations to concept maps in work and energy
subjects (Shymansky et al., 1997), conceptual
understanding and conceptual improvement of 10th grade students were examined.
In this study, classical mechanical concepts of the students (kinematics, Newton’s
laws of motion, work and energy) were discussed and in order to evaluate the
understanding of the students over and over again, concept map and follow-up
interview methods were used. The research continued in a period of 14 weeks
with 22 students enrolled for the physics class. Data was collected from students
four times at different stages of the study. Follow-up interviews were used
for explanation, expansion and approval of concept map information. At the end
of the research, it was seen that knowledge structure of the students remained
the same for 10 weeks and did not change for 4 weeks after the end of the training,
either. Moreover, misconceptions, that the students have about classical and
mechanical subjects obtained from students concept maps during the research,
were listed. Among these, the ones related to work and energy are as follows:
force and energy are same, force and work are equivalent, movement implies doing
work, energy and work are same, potential energy is motion dependent and energy
is not conserved.
Kucukozer (2006) aimed to better understand construction
of the meaning of physics concepts in mechanics (Newtons third law) during
a teaching sequence at the upper secondary school level. She reported that the
notions of object and the concept of gravitations were simultaneously founder
notions. The analysis of the data in the study showed that how much the use
of the different physical and conceptual resources such as several categories
of knowledge, the elements of the situation, influences the construction of
Work-energy is a very important topic in school science. Energy is especially
important because it is a basic topic not only in physics but also other science
disciplines, such as chemistry, biology, engineering etc. The basic concepts
which are on the topic of work-energy, like force, distance, velocity, gravity,
power, kinetic and potential energy, are fundamental concepts in that they underlie
some the other topics of physics. Unlike some of the concepts found in physics,
work-energy is often experienced by students in their daily lives. Therefore,
we believe that it is crucial to examine closely the knowledge structures which
they construct in their minds about the concepts related to the subject of work-energy.
In this study, concept mapping is used as an assessment tool to explore both
the concepts of work-energy, which the students have and the relationships between
these concepts. When the literature is examined, there is no in-depth research
about the knowledge structures used in the subject of work-energy. The scarcity
of qualitative studies done on this subject makes it advisable to continue this
In this study, the concepts of the 7th grade work and energy subjects
in the curriculum and the relationship between these concepts were analyzed.
Within this scope, the goals of the study can be listed as follows:
||Using concept maps as an assessment method
||Exposing concept structure of the students in work and energy subject
||Examining the relationship between the concept map scores and achievement
test scores of the students
MATERIALS AND METHODS
Participants: This research was made with 50 7th grade students
in science and technology course, 28 girls and 22 boys. Students, who
were involved in the research, were from two different classes in an elementary
school located in a socio-economically developed neighborhood of Ankara.
All students have the same teacher for science and technology class.
Procedure: Data collection was realized in spring of 2005-2006 academic
years. The research lasted for 7 weeks/21 courses and it was conducted by the
first of the writers of this study while the class teacher was also there as
an observer. In the first week, concept maps were introduced to the students
in general terms and it was explained to them how to construct concept maps.
Following five weeks the researcher trained the students in work-energy issues
and at that stage different concept maps were drawn with the students to familiarize
the students with concept maps. At the last week of the research, two lessons
were used for work-energy matters and for every student to construct his/her
own concept map. In the last course an achievement test was made. Each student
was given a paper of 50×30 cm. They were not allowed to use any other material
except for pencil, eraser and the mentioned paper. In that respect, concept
maps were used as an assessment method. Seventeen concepts related to the subject,
which are included in the curriculum, were determined at the beginning of concept
map construction process of students. These concepts were written on the board
with no predetermined order. These concepts are namely, work, energy, power,
mechanical energy, kinetic energy, potential energy, elastic potential energy,
gravitational potential energy, velocity, force, mass, weight, height, elasticity,
stretch, compress and distance. Later on, students formed their individual maps
using these concepts. These concept maps were assessed according to the criteria
suggested by Novak and Gowin (1984). These criteria can
be shown as follows:
|No. of concepts
||Two points for every concept
||Ten points for location of concepts according to their degree of
proximity on the concept map
|Correctness of proposition
||Two points for each correct proposition between the concepts
|Accuracy of proposition
||Two points for accuracy of proposition between the concepts
|Direction of proposition
||Two points for correct understanding of the relationship between
concepts when read in the direction mentioned in the proposition
||Every cross link between the concepts was scored over 10 points
according to its correctness and accuracy
|| Variance of analysis
The achievement test used in the research was work-energy achievement
test. This test consists of 28 multiple choice questions which were categorized
in four levels: knowledge, comprehension, problem solving and scientific
method process. The rationale behind choosing this test is that: the concepts
related to the questions in the achievement test and 17 concepts which
are expected to exist in the concept map overlap with each other.
Reliability and validity: Twenty five random concept maps were
evaluated by four researchers in groups of two according to the criteria
of assessment. Generalizability analysis was made in order to determine
interraters reliability. Generalizability coefficient was calculated
with the following formula:
states individual variance and
is error variance (Atilgan, 2006). Interraters generalizability
coefficient was found as 0.97.
Pearson moment multiplication correlation coefficient of interraters
was calculated with SPSS program and was found as 0.994 (p<0.01). This
coefficient indicates that there is a statistically meaningful, high and
positive relationship between raters. According to the results of the
analysis made here, it can be said that evaluation is reliable and valid.
Content of validity of achievement test was provided by taking opinions
of the experts.
Achievement test was made to 226 students in 7th grade and Cronbach Alpha
reliability coefficient was calculated as 0.79. Item analysis of the test
indicated that mean difficulty value of the test is 0.457 and mean biserial
correlation value of items is 0.496. This test was used directly since
the analysis show that this test is valid and reliable and it is convenient
for the age group subject to this research. The results of analysis of
variance are shown in Table 1.
Results obtained from concept maps: In the study concept maps
drawn by 50 students were scored by the researchers of this study according
to assessment criteria. Concept map scores of the students can be seen
in Fig. 1.
|| Distribution of concept map scores according to students
|| Scale codes of concept map scores and distribution
|| The percentage of the concept in the concept maps
Concept map scores are between 12 and 125. The average score is 61.9.
Scores were scaled in order to comment on them easily. Scale codes and
distribution of students according to these criteria is given in Table
It can be seen in Table 2 that approximately 68% of
the student scores were accumulated in average and poor scale codes. There
are only 2 students in the scale code determined as excellent. Use percentage
of 17 concepts, which were mentioned previously in procedure, in the evaluated
concept maps drawn by the students, is given in Table 3.
Almost all of the students used concepts like work, energy, kinetic energy
and potential energy in their maps.
||Minimum, maximum means and standard deviations of students
concept map link components
Although mechanical energy contains
kinetic and potential energy, the percentage of using it in the maps (88%)
is less than the percentage of using kinetic energy and potential energy
concepts. It was determined that the rate of using the concepts force
and distance are the same (84%). It is believed that since these two concepts
are mentioned together during the definition of work concept, when one
is used it recalls the other. It can be said that the concepts mentioned
here are considered as preferential concepts and that is why they are
used in the maps. The rest of the concepts are close to each other like
in Table 3 and have descending rates of use. It can
be said that these concepts are considered as secondarily preferential
by the students.
It is inevitable that the relationship between the concepts must be examined
taking into consideration the reality that the main purpose of concept
maps is to investigate the relationship between the concepts. The following
findings were reached after examining the relationship that the students
established between the concepts. Maximum, minimum and mean values related
to number of direct propositions, correct directions and open propositions
in concept maps are shown in Table 4.
Another point that reveals relationships in concept maps is cross links.
It is very interesting that only three students used one cross link each.
Since 30% of the students established a relationship between the concept
power with concepts like mechanical energy, force, weight, mass, tension
rather than the concepts work, energy; it makes us think that they use
this concept in daily life instead of muscular force. Figure
1 shows a student concept map related to using the concept power as
|| Example of a concept map where the concept power is
||A student concept map example where the concepts height
and weight are directly associated with potential energy
As the maps are in Turkish, English version of the concepts
are written next to the figure for the sake of clarity.
Students were expected to divide potential energy to two, namely elastic
potential energy and gravitational potential energy and relate it to weight
and height. However, it was seen that only 30% of the students set up
a relation between potential energy with gravitational potential energy
and elastic potential energy, height and weight concepts are directly
associated with potential energy. Figure 2 illustrates
a student concept map with this kind of relationship.
Students are expected to relate mechanical energy with kinetic energy
and potential energy and indicate in the concept maps that it is composed
by these two concepts. However, when the concept maps are analyzed, it
was seen that 20% of the students could not directly relate mechanical
energy with kinetic and potential energy.
|| A student concept map example where the concept mechanical
energy is used wrongly
|| A student concept map example that composed of small
concept maps independent from each other
They related mechanical energy,
kinetic energy and potential energy concepts at the same level as energy
concept in Fig. 3-5.
It was observed that 14% of the students drew multiple small concept
maps which are not related to others. Following concept groups existed
in these unrelated small concept maps: work-force-distance, flexion-tension-compaction,
mechanical energy-kinetic energy-potential energy-velocity-mass-weight-height.
Fourteen percent of the students reflected correctly the relationship
between mass and weight related to force which was realized before work-energy.
Ten percent of the students made a direct relation between the concept
mass and the concept velocity. The percentage of students who included
the concept power in their maps without relating it to any other concepts
Findings obtained from the achievement test: One point was given
to each correct answer while grading the test.
||Maximum, mean and standard deviations of students scores
of achievement test
|| Scale codes of achievement test scores and student
This way, the highest grade that can be scored from the test is 28. The
minimal score that students got in exam was 3. Average of students scores
and standard deviation values can be found in Table 5.
In order to comment easier on achievement test scores, they were graded
the same way as in concept. Grading criteria and distribution of students
according to these criteria are given in Table 6.
It was seen that 70% of the students score very good and good scale codes
when student distribution is examined. It is interesting that there are
very few students with poor and bad scale codes (Table 6).
Findings related to the relationship between concept map scores and
achievement test scores: Achievement test scores of the students were
compared to concept map scores. Pearson moments multiplication correlation
coefficient was calculated as 0.416 (p<0.01). This coefficient shows
that there is a significant, average and positive relationship between
concept map scores and achievement test scores of the students.
Scores of the participant students on concept maps accumulate at poor
and average levels. Only 4% of them are at very good level and 14% of
them are at invalid level, which is found to be a significant issue. Level
of usage of 17 introduced concepts in evaluated concept maps vary from
66 to 100%. Although all the 17 concepts were on the board and were instructed
to the students, the participants used some of the concepts in the concept
maps less than others. When the utilization of the concepts was inspected,
work, energy, kinetic energy and potential energy were revealed out to
be the most commonly used ones. Therefore, it can be stated that these
above concepts are primary concepts. Levels of use of the rest concepts
are close to each other and show declining percentage. Thus, it can be
stated that these are secondary concepts. The difference observed in the
use of concepts, which are determined and listed with the contribution
of students before test, may also denote that students suffer from building
interconceptual relations. Since students build concept maps by relating
one concept to another, one can state that they avoid using the ones that
are not clearly understood.
Propositions are one of the concept map components that state the relations
among concepts. Student concept maps reveal that concepts were connected
to each other with lines where propositions were scarcely used. Average
number of propositions per concept map is 4 while average number of clear
propositions decrease to 2. To illustrate this, work is related to power
is not a false proposition but also it is not a clear one. Work is proportional
to power is both a true and a clear one. Limited quantity of propositions
and students tendency to define relations with lines in the concept maps
indicate that students experienced trouble in building interconceptual
Another important issue that reflects the conceptual interrelations is
cross-links. Only 3 students built cross-links. This is also another evidence
of deficiency in constructing relations. Another fact supporting the above
statement is, some students concept maps included several independent
concept maps with one or more concepts. Although students were expected
to relate concept of power, which is a base concept of work and energy
topic, to work or energy, 30% of them related it to other concepts like
force, weight or mass. Therefore, it can be stated that students equate
this concept with the muscular strength, which is a more familiar concept
from their daily life. Another important issue about this concept is 8%
of the students accommodated it in their concept maps alone, without relating
it to another.
Thirty percent of the students stressed that both gravitational potential
energy and elastic potential energy were subordinates of potential energy.
On the other hand, most of them connected height and weight to potential
energy. This reveals that students take gravitational potential energy
into consideration when potential energy is mentioned. About 20% of the
students related mechanic energy to energy concept as a form of energy
in the same hierarchical level with kinetic energy and potential energy.
Expected behavior was displaying mechanic energy as a compound of kinetic
energy and potential energy.
Deficiencies in correctness and clearness of the propositions and definition
of them as only a directed line prove that students have problems with
propositions. Since propositions state the relations between concepts,
missing labels on the links between concepts makes us think that there
are blank spots in students knowledge structure about work and energy
topic. Therefore, it would be a wise way to compare success test results
to concept map scores. Both scores were examined in same way according
to a 5-grade-scale (very good, good, average, poor, bad). This way comparison
between two scores is made easier. Graph 2 and graph 3, which show the
distribution of students according to above criteria, show that concept
map scores accumulate at the right side of the graph (weaker) where success
test scores accumulate left side (stronger). Success test scores pose
a better result in comparison to concept map results. Since there is a
medium level correlation (r = 0.416) among these two evaluation criteria,
both methods have corresponding and non-corresponding sides. Concept map
method defines the relations between concepts while success test evaluates
information, comprehension, application of concepts and scientific methodology.
Therefore, it would be appropriate to use both methods as a complement
to each other.
Students with lower scores from the concept maps are explained with the fact
that this technique is not familiar to the group. Another reason for this is,
concept map technique does not comply with Turkish language structure as stated
in Bagci (2003). In Turkish, verbs are placed at the
end of the sentence and this makes building propositions harder. We believe
that students may overcome this handicap if concept map use is spread throughout
the whole instruction process.
One of the main properties in measurement instruments and methodology is reliability.
Reliability studies in concept maps are a prime issue as stressed by Ruiz-Primo
and Shavelson (1996). Usually, reliability of concept maps is provided by
employing different raters and comparing the raters scoring. Generalization
theory is utilized to find out the reliability of raters. In this study of 2
raters, g-coefficient was computed as 0.97. In six different scoring methods
reliability was computed with 2 raters (McClure et al.,
1999). One of six above methods was the one used in this study and reliability
of it was computed as 0.41. Although this reliability level is lower than the
level determined in the research, other studies (Rice
et al., 1998; Ruiz-Primo et al., 2001a)
found similar scores supporting the results. Researchers think, when the components
of the scoring system (number of concepts, hierarchy, proposition correctness,
proposition clearness, cross-links) proposed by Novak and Gowin
(1984) are taken into consideration, reliability is satisfactory. High reliability
factor computed at the research also supports this thesis.
Although there is no reliability problem in scoring concept maps, it is an
area of research that if using concept maps is an appropriate way to determine
student success. Inspecting correlation between concept map scores and standardized
multiple-choice test scores is a common approach in literature. Studies in this
area revealed that there exists a medium and high correlation in between (Rice
et al., 1998; Schau et al., 2001; Ruiz-Primo
et al., 2001a; Rye and Rubba, 2002). In this
study a medium level correlation is observed (0.416). Considering examination
of interconceptual relations is harder when multiple-choice tests are used,
evaluation by exploiting concept maps gain importance. We believe that instead
of using these methods alone, a combination of both would be a better approach.
Positive correlation between two methods may be accepted as evidence to present
Work and energy is one of the primary topics in physics. Basic concepts
of this topic are not only important because they construct a basis for
advanced topics of physics but also because they are common in our daily
lives. Therefore, examining the students knowledge structure and completing
their deficiencies would contribute to meaningful learning. Concept maps
are useful instruments to serve this purpose. By solidifying the relations
among concepts, information may be kept by students in their minds with
meaningful inter-connections instead of sequential ordering and serve
to retention of knowledge. Authors recommend exploiting the advances of
concept map use for this purpose as well.