A new scientometric indicator has been recently proposed by Hirsch to evaluate
the performance of individual scientists. It has been named h-index and is defined
as follows: a scientist has index h if h of his or her P papers has at least
h citations each and the other (P-h) papers have = h citations each (Hirsch,
2005). This index has several advantage of evaluating scientific output
(Batista et al., 2006; Torro-Alves
et al., 2007; Mugnaini et al., 2008;
Bornmann and Daniel, 2005; Glänzel,
2006; Ball, 2005), however his applicability to
different areas of knowledge has not been well established. In fact, Hirsch
(2005) has shown that the top ten in physics and biology have very different
h indexes. The first ranked physicist (Witten E) has h-value equal to 110, while
in the life sciences the highest h-value (Snyder SH) is 192. Although, this
is not recommended, the performance of individual scientists is frequently weighted
solely by scientometric indicators. If a scientist has four times more citations
than another, one would be wrongly inclined to assume that the results of the
former are four times more important/relevant than the results of the latter.
This problem is even more complicated when analyzing different research areas.
In context, in this study, we analysis of scientific research output of professors
of undergraduate courses of speech-language and hearing science of Faculdade
de Medicina (FM), Faculdade de Odontologia de Bauru (FOB) and Faculdade de Medicina
de Ribeirão Preto (FMRP) and we discuss the applicability of three indexes.
We observed that most speech-language and hearing science papers in Brazil are published in journal not indexed by ISI and therefore they are not considered in the calculation of the h-index. Then, the Curriculum Lattes (CL) analysis was important because showed that the lower h-index average was not related to low scientific productivity. Therefore, it would be interesting to develop new strategies for evaluating some research fields, such as speech-language and hearing science or other disciplines from the social sciences, which are generally not properly assessed in terms of their scientific contribution. Recently, the Scientific Electronic Library Online (SciELO) that is a database of scientific journals on the Internet of Latin America and the Caribbean countries had his 479 scientific journals indexed by Scopus-Elsevier. Scopus-Elsevier database encompasses 17.000 of all journals in the world while that Thompson ISI Web of Science encompasses less 7.000 journals in the world.
The analysis obtained by Curriculum Lattes have shown that several papers published by professors of USP are of local/regional journals that not are indexed no ISI. However, these journals are indexed by SciELO. Scopus provides full citation coverage from 1996 and onwards and claims to be the largest abstract and citation database of research literature and select results from the web. Scopus provides citation data only for the items indexed by it.
With the index of SciELO by Scopus, the professors of speech-language and hearing science able to assessed his novel h-index by Scopus.
MATERIALS AND METHODS
In this study, the evaluate research produced by professors linked to FM-USP,
FOB-USP and FMRP-USP; we analyze three indicators of scientific productivity
(Herculano et al., 2008). The indexes analyzed
for each professor were: 1) total number of papers (indexed in the Curriculum
Lattes database, a National Curriculum Vitae (CV) database), 2) the number of
papers indexed by the Thomson ISI Web of Science database and 3) the h-index.
The number of papers indexed by ISI and citations to professors linked of the FM, FOB and FMRP were obtained from the Web of Science data base (Thomson-ISI). All data were collected in May 2008. All professors belong to undergraduate courses of speech-language and hearing science linked to FMRP, FOB and FM were considered in this survey.
The h-index for each author was calculated from the citations of all publications
listed in the ISI-Web of Science (Hirsch, 2005).
The total number of papers of each professor linked to FM, FOB and FMRP were
calculated using information available in their CV Lattes (Torro-Alves
et al., 2007). The Lattes System is a Brazilian database that includes
CVs of researchers linked to Brazilian academic institutions (CNPq,
2008). According to the latest statistics of the Lattes Platform, until
May, 2008, there were 800,000 researchers from 4,000 different research institutions
registered in the system. The survey was conducted by typing the names of professors
linked to FM, FOB and FMRP in the tag field search for researchers (simple search)
available in the Curriculum Lattes database (CNPq, 2008).
RESULTS AND DISCUSSION
The literature is abundant in scientometric studies comparing the scientific
bibliographic productivity in countries (Torro-Alves et
al., 2007; Batista et al., 2006; Mugnaini
et al., 2008). The present article focuses on the performance of
the professors of speech-language and hearing science at USP. This University
is one of the most prestigious institutions in this area in Brazil.
Figure 1 shows the means of h-index values, number of papers indexed by the ISI and total number of papers in the Curriculum Lattes of professors linked to FM, FOB and FMRP. Notice however, a greater means of papers indexed by Curriculum Lattes of undergraduate course of speech-language and hearing science offered by FM (34.92) when compared to FOB (21) and FMRP (13.71). Also, we observed that bibliometric evaluations using papers indexed by ISI presented closely similar results: FM (2.77), FOB (2.58) and FMRP (3). As already excepted, the h-index presented also similar results: FM (0.85), FOB (0.83) and FMRP (0.71).
We have verified that citation distributions and the number of papers. The mean values of scientific output were then submitted to between-within ANOVA according to the following model: 3 schools of undergraduate studies (FM, FOB and FMRP), vs. 3 indexes of productivity (total number of papers, number of papers indexed by ISI, h-index). The differences in the h-indexes and the papers indexed by ISI between FM, FOB and FMRP are not significant. However, statistical analysis showed a significant main effect of the factor 3 schools of undergraduate studies vs. Curriculum Lattes (CL). An a posteriori comparison test (Mann-Whitney) indicated that the FOB (p<0.05) and FMRP (p<0.05) presented lower values for the index quantified (Table 1).
Means of h-index values, number of papers indexed by the Thompson
ISI Web of Science and total number of papers in the Curriculum Lattes of
professors linked to FM, FOB and FMRP
||The p-values of the hypothesis test concerning equality between
undergraduate courses of speech-language and hearing science at Universidade
de São Paulo
List of graduate programs that the professors of: (a) Faculdade
de Odontologia de Bauru (FOB) and (b) Faculdade de Medicina (FM) are linked
Low h-indexes in the humanities, speech-language and hearing science and accounting
are common. According to literature (Torro-Alves et al.,
2007), the inferior h-index value obtained for these areas is result of
this fieldÂ´s particularities. This seems to be due to the traditional mode
of communication in this area of knowledge, which makes more use of books and
proceedings of meetings when compared to natural sciences. These publications
are not covered by ISI.
Figure 2 shows that the data analysis obtained by Curriculum
Lattes database revealed, that only three professors of FMRP are linked to graduate
program of (Ophthalmology, Otolaryngology and Head and Neck Surgery-USP), while
that 12 professors of FM belong to 30 graduate programs and 14 professors of
FOB belong to 36 graduate programs. We observed low papers indexed in the Curriculum
Lattes of undergraduate course of FMRP when compared to FOB and FM. A possible
reason for this is the recent foundation of course in 2003 while the undergraduate
course offers by FOB was founded in 1990 and the offers by FM was founded in
1958. The other reason for the low papers indexed by Curriculum Lattes of professors
linked to FMRP is the low numbers professors belong to graduate programs. In
general, the major number of professors belong to graduate programs publish
more papers. According to literature (Strehl and dos Santos,
2002), this occurs due to need of the financial resources obtained by funding
We observed that the coverage by ISI Web of Science is still a great problem, mainly because only 5% of all journals published in the world are included in Thompson ISI Web of Science database. Thus, the assessment of the h-index through ISI Web of Science may be improper for evaluating areas of knowledge that publish in journals not included in the Thompson ISI Web of Science database.
According to literature (Van Raan, 2006), the h-index
should not be considered superior to other assessment forms, since it correlates
with other bibliometric indicators as well as with peer opinion. According to
literature (Herculano et al., 2008; Torro-Alves
et al., 2007), the bibliometric indicators seem to be more appropriate
for comparing graduate research programs than undergraduate courses. However,
currently, many international research funding agencies employ the h-index as
criterion for evaluation.
In summary, the present study indicated that similar results in the assessment of the h-index, the number of papers indexed by Thompson ISI Web of Science was obtained by FM, FOB and FMRP. However, we found a worse performance in the assessment of the papers indexed by Curriculum Lattes of the undergraduate course of FMRP, which can be associated with the recent foundation of course in 2003. The h-index, like other bibliometric indicators, is sensitive to the field of knowledge and must not be used as the only factor for assessing an individual's scientific research output.
We show that most speech-language and hearing science papers are published in journal not indexed by ISI and therefore the impact of papers published by professors of the speech-language and hearing science course cannot be appropriately assessed on the basis of data collected from the ISI Web of Science. In this respect, the analysis of the Curriculum Lattes was especially important since it showed that a lower h-index average was not necessarily related to low scientific productivity (number of papers), but rather to a small percentage of papers indexed by Thompson ISI Web of Science.
The three undergraduate course of Universidade de São Paulo presented lower h-index and fewer papers indexed by the ISI. However, we observed that undergraduate course linked to FM presented a high papers indexed by Curriculum Lattes when compared to the other undergraduate courses (FOB and FMRP).
We also recommend to further explore the speech-language and hearing science papers using the h-index and papers indexed by Scopus database.