Abstract: Background and Objective: Randomized controlled trials (RCTs) and non-RCTs are typically used for evaluating treatment effects in chicken research. The abstract is an important part of scientific reports that readers usually read first and then decide whether to read the entire article, so the information provided in the abstract should be adequate. The consolidated standards of reporting trials (CONSORT) for Abstracts checklist has been developed and used as a guideline to help authors prepare their manuscripts. This checklist has also been used as a tool to evaluate published abstracts. The objectives of this study were to evaluate the quality of reporting in abstracts of trials reporting randomization in their abstracts (RCT-A) and trials not reporting randomization in their abstracts (non-RCT-A) from chicken research and to identify the factors associated with reporting quality. Materials and Methods: PubMed was searched for abstracts of controlled trials involving chicken research published between 2006 and 2015. The abstracts were evaluated using the modified CONSORT for Abstracts checklist. The primary outcome was a median Overall Quality Score (OQS), which for each abstract was a sum of items recommended in the modified checklist. Some pre-specified factors were also evaluated for their association with reporting quality using simple and multiple ordinal logistic regression analyses. Results: A total of 949 abstracts (n = 262 for RCT-A and n = 687 for non-RCT-A) were included and evaluated. The OQS was significantly higher for RCT-A than non-RCT-A (median (interquartile range), 7.0 (6.0-7.0) vs 3.0 (3.0-4.0); p<0.001) but both median scores were still less than half of the maximum score of 15. The two most frequently reported items (>80%) were the objective and conclusions. Trial design, participants and interventions were adequately reported only in RCT-A. In contrast, identifying the study as a randomized trial in the title and including a clearly defined primary outcome, blinding, numbers analyzed, estimated effect size and its precision for the primary outcome, trial registration and funding in the abstracts were not reported or were reported in <5% of all abstracts. Four factors-year of publication, number of trials reported, number of experimental groups reported and sample size reported were associated with OQS. That is, abstracts with higher OQSs were published more recently, reported a single trial rather than multiple trials, reported the number of experimental groups and reported the sample size. These factors explained about 39.4% of the variance of OQS. Conclusion: The reporting qualities of both RCT-A and non-RCT-A from chicken research were suboptimal. Efforts should therefore be made to improve the transparency, completeness and detail of reporting in controlled-trial abstracts from chicken research, especially the development of specific guidelines based on the CONSORT for Abstracts checklist.
INTRODUCTION
Randomized Controlled Trials (RCTs) are considered the gold standard in health science research for evaluating the health benefits or harms of treatments or interventions because randomization can reduce bias in assigning subjects to treatments1. The RCTs must be publicly reported for further use as scientific evidence by relevant stakeholders such as industry and research communities. Clear, transparent and complete reporting of RCTs is necessary for critical appraisal by readers. The Consolidated Standards of Reporting Trials (CONSORT) statement was first developed in 1996 to improve RCT reporting2 and was then updated in 20011 and 20103. The CONSORT extension statement for reporting single-patient (N-of-1) trials (CENT 2015) was also recently developed4.
The abstract is one of the most important parts of a published RCT because it is a summary of the entire RCT and it is the easiest section to access5. The abstract is therefore read first by most readers. Unfortunately, only one item in the CONSORT statement is designed for reporting in abstracts. The CONSORT for Abstracts checklist was developed to help ensure that the abstract contained adequate information for readers5. This checklist helps authors prepare the abstracts of their manuscripts and has been used as the gold standard for evaluating the quality of reporting in an RCT abstract6-12. Findings from previous studies suggest that the reporting quality of RCT abstracts from health research is suboptimal8,10,13-16. Several factors, such as abstract word limit, abstract format, publication year and impact factor of the journal, may be associated with the reporting quality of RCT abstracts9-11.
RCTs for livestock are inherently different from RCTs with human subjects but also need clear, transparent and complete reporting. A team led by O’ Connor and Sergeant therefore developed The REFLECT statement17,18, a modified version of the CONSORT statement for reporting RCTs for livestock. Chickens are a major source of protein for humans worldwide. The consumption of poultry meat throughout the world was estimated to be 13.8 kg per capita in 2015 and is expected to be 17.2 kg per capita by 2030 (FAO, http://www.fao.org/docrep/005/y4252e/y4252e05b.htm). Most chickens sold today in markets worldwide are raised under a mass-production industrial system to meet the high demands of consumers. Research, especially controlled trials, is thus needed to reduce cost, improve production and solve health problems in commercially raised chickens. Many controlled trials for livestock are published each year and readers expect to read RCTs rather than non-RCTs but a substantial proportion of non-RCTs have unfortunately been reported for livestock research19,20.
The objectives of this study were to evaluate the reporting quality of RCT-A and non-RCT-A from chicken research using the modified CONSORT for Abstracts checklist and to identify particular factors that may be associated with quality.
MATERIALS AND METHODS
Literature search: We searched PubMed for articles published from 2006 to 2015 in July 2015 with the keywords " chicken" and " experiment" . The search details were ((" chickens" [MeSH Terms] OR " chickens" [All Fields] OR " chicken" [All Fields]) AND experiment [All Fields]) AND (" 2006/01/01"[PDAT]: " 2015/12/31"[PDAT]). The search was updated in October 2015 to add more recent abstracts from the database.
Inclusion and exclusion criteria: We categorized the controlled-trial abstracts into trials reporting randomization in their abstracts (RCT-A) and trials not reporting randomization in their abstracts (non-RCT-A). The RCT-A and non-RCT-A were included if they (1) Involved live chickens (either broiler or layer chickens) as experimental or observational units and (2) Clearly defined a treatment or intervention. Abstracts were excluded that reported trials or experiments that involved chicken sperm, fertilized eggs, or chicken embryos. The abstracts were also excluded that reported a single-group experiment, an observational study, an in vitro study, or a review.
All abstracts were selected that satisfied the inclusion criteria in each year as a sample of this study to ensure a large enough sample size of selected abstracts for drawing a clear conclusion. Except for years with more than 100 abstracts, 100 abstracts were randomly selected using a computer-generated random sequence (https://www.random.org/).
Data extraction: We used the modified CONSORT for Abstracts checklist for data extraction (Table 1). This checklist5 is widely used to assess reporting quality for abstracts of human randomized controlled trials6-12. The checklist consists of 17 items covering all important domains (title, trial design, methods, results and conclusions) that are necessary for readers. Some aspects of chicken trials are inherently different from those of human trials, so we slightly modified the checklist to adapt it to chicken trials. Some information for this modification came from the statement of the Reporting Guidelines for Randomized Controlled Trials in Livestock and Food Safety (REFLECT), also known as the modified CONSORT statement for livestock17,18.
| Table 1: | The modified CONSORT for Abstracts checklist with guidance for scoringa |
| aThe checklist Hopewell et al.5 was modified for controlled trials in chicken research, bThis item is not included because it is specific to conference abstracts, cThis item is not included because it is not applicable to controlled trials in chicken research | |
Two of the original 17 items of the CONSORT checklist (authors and recruitment) were excluded because the authors item was specific for conference abstracts and the recruitment item was not applicable to trials in chicken research. A table of checklist items with their original and modified definitions as a guideline for scoring was created (Table 1). A score of 0 was assigned if an item was not reported or was not reported clearly and a score of 1 if an item was clearly reported. Each abstract was thus assigned a score between 0 and 15.
Data was also extracted for the following characteristics of trials and abstracts: journal name, ISI impact factor (2014), year of publication, number of authors, region of publication (continent of residence of the first author), word count of the abstract (excluding title, author names and keywords), abstract format (structured vs unstructured), number of trials reported per abstract, number of experimental groups reported and number of experimental chickens (the number of experimental groups and number of chickens were extracted only for the first trial in abstracts reporting multiple trials). These characteristics were pre-specified factors and were used for simple and multiple ordinal logistic regression analyses.
Pilot study: The method of data extraction was validated using the modified CONSORT for Abstracts checklist by extracting data from 20 randomly selected abstracts by two of the authors (PS and SK) and by calculating the kappa-statistic to determine inter-rater reliability. Overall, the kappa-statistic (95% confidence interval) for scoring items was 0.81 (0.61-1.00), indicating that inter-rater agreement was good21,22. We then extracted data from all selected abstracts by these two authors. Disagreement was resolved by consensus.
Measured outcomes and statistical analysis: The primary outcome was a median Overall Quality Score (OQS) for abstract reporting. This score is a sum of the items reported in the modified checklist and ranges from 0 (minimum) to 15 (maximum). A score of 15 indicated complete reporting in the abstract. The secondary outcome was the percentage or frequency of reporting for each item of the modified checklist. Rate ratios were also calculated by comparing the rates of reporting for each item between the RCT-A and non-RCT-A.
The SPSS version 17 (SPSS Inc., Chicago, IL) was used for all statistical analyses. Descriptive statistics included frequencies, percentages, means, standard deviations, medians and interquartile ranges (IQR). Results from the Shapiro-Wilk test indicated that the OQS data were not normally distributed. A Mann-Whitney U test was used to compare OQSs of the RCT-A vs non-RCT-A, a Chi-squared test was used for the rate ratio and simple and multiple ordinal logistic regression analyses was used to identify the factors associated with OQS. Potential factors included year of publication (continuous, 2006-2015), journal impact factor (<1, 1-2, or >2), region of publication (continent of residence of the first author, including Asia, Europe, North America or other), number of authors (<4, 4-7, or >7), abstract format (structured or unstructured), trials reported (single or multiple), experimental groups (not reported, 2 groups, or >2 groups) and sample size (not reported or reported). A simple ordinal logistic regression analysis was used to determine the association between OQS and each pre-specified factor described above. A multiple ordinal logistic regression analysis was used to construct a final model by backward elimination of non-significant factors. All statistical tests were two-tailed and values were considered significant at p<0.05.
RESULTS
Literature search: The search initially identified 1896 abstracts. Of these, 838 were excluded for various reasons (Fig. 1). The remaining 1058 abstracts were either RCT-A or non-RCT-A. This number was further reduced by retaining a maximum of 100 abstracts for each publication year.
| Fig. 1: | Flow diagram of the literature search and identification of controlled trial abstracts from chicken research |
| Fig. 2: | Number and percentage of the non-RCT-A and RCT-A in each year from 2006-2015. Values within the bar are the numbers of the RCT-A and non-RCT-A. Values above the bar are the percentages of the RCT-A, (Error bars: 95% CI) |
A total of 949 abstracts were ultimately included for analysis. Only 262 of these (27.6%) were RCT-A and 687 (72.4%) were non-RCT-A. The proportion of RCT-A increased slightly from 23.7% in 2006 to 34.1% in 2015 (not significantly different, p = 0.146) (Fig. 2).
Characteristics of the abstracts: The characteristics of the 949 abstracts are presented in Table 2. Poultry Science published the majority of both RCT-A (42.3%) and non-RCT-A (42.5%). Most abstracts were unstructured (95% for RCT-A and 97.1% for non-RCT-A). The majority of the abstracts reported a single trial (86.5% for RCT-A and 66.6% for non-RCT-A) and more than two experimental groups per trial (86.5% for RCT-A and 58.3% for non-RCT-A). More than half (54.0%) of the non-RCT-A did not report a sample size.
Overall Quality Score (OQS): The median OQS (or median number of items reported in the abstracts) was 7.0 (IQR, 6.0-7.0) for RCT-A and 3.0 (IQR, 3.0-4.0) for non-RCT-A (significantly different, p<0.001). The mean and median OQS of RCT-A and non-RCT-A for each characteristic are presented in Table 3. None of the RCT-A reported more than nine items and none of the non-RCT-A reported more than seven items (Fig. 3).
Item-specific reporting: The proportions of item-specific reporting for the RCT-A and non-RCT-A using the modified checklist are shown with an associated rate ratio in Table 4.
Reporting of the title and trial design: None of the RCT-A or non-RCT-A included " randomized" (or other variations of this term) in the title and 96.9% and 3.1% of the RCT-A and non-RCT-A reported trial design, respectively.
Reporting of trial methods: The RCT-A reported descriptions of experimental chickens (participants) often more than non-RCT-A (89.2 vs 54.0%, respectively; p<0.001). The details of interventions were reported in 93.8% of the RCT-A compared with 74.2% of the non-RCT-A. Both RCT-A and non-RCT-A often reported objectives of the studies (97.7% for RCT-A and 94.5% for non-RCT-A). Both RCT-A and non-RCT-A rarely reported clearly primary (main or principal) outcomes (3.8% for RCT-A and 4.9% for non-RCT-A). Blinding of either the outcome assessors, caregivers or both was not reported in RCT-A and was reported in only one non-RCT-A.
Reporting of trial results: Except for the number randomized item of RCT-A (80% reported), the reporting of all other items of trial results in both RCT-A and non-RCT-A was suboptimal. In particular, the number analyzed item was reported in only one RCT-A and was not reported in any of the non-RCT-A.
| Table 2: | Characteristics of the included abstracts |
IQR: Interquartile range, non-RCT-A: Trials not reporting randomization in their abstracts, RCT-A: Trials reporting randomization in their abstracts, aThe continent of residence of the first author, bIf an abstract reported more than one trial, the number of chickens was determined from the first trial only |
|
| Fig. 3: | Percentages of the RCT-A and non-RCT-A reporting the indicated number of items on the 15-item scale. Less than 15% of the RCT-A reported 8 items or more; in contrast, less than 15% of the non-RCT-A reported 5 items or more |
| Table 3: | Mean and median OQS of the RCT-A and non-RCT-A for characteristics |
CI: Confidence interval, IQR: Interquartile range, non-RCT-A: Trials not reporting randomization in their abstracts, OQS: Overall quality score, RCT-A: Trials reporting randomization in their abstracts, SD: standard deviation, aThe continent of residence of the first author, bMedian OQS of RCT-A was significantly greater than that of non-RCT-A (p<0.001, Mann-Whitney U test) |
|
| Table 4: | Item-specific reporting of the RCT-A and non-RCT-A |
| CI: Confidence interval, non-RCT-A: Trials not reporting randomization in their abstracts, RCT-A: Trials reporting randomization in their abstracts | |
| Table 5: | Ordinal logistic regression analyses for identifying factors associated with the OQS |
| CI: Confidence interval, aFor multivariate analysis, Cox and Snell R2 = 0.394 and p<0.001, bThe continent of residence of the first author | |
Reporting of conclusions, trial registration and funding: Conclusions were reported in 90 and 82.7% of abstracts that reported randomization versus abstracts that did not report randomization, respectively. None of the abstracts, however, reported trial registration or funding.
Factors associated with OQS: Four factors-year of publication, number of trials reported, number of experimental groups reported and sample size reported were associated with OQS in the final model of the multiple ordinal logistic regression analysis (Table 5). That is, abstracts with a higher OQS were published more recently, reported a single trial rather than multiple trials, reported the number of experimental groups and reported the sample size. The Cox and Snell R2 for this model was 39.4%. The mean OQS for both RCT-A and non-RCT-A improved slightly over time (Fig. 4).
DISCUSSION
The reporting quality of 949 controlled-trial abstracts from chicken research published in the last 10 years (between 2006 and 2015) using the modified CONSORT for Abstracts checklist was evaluated. Of the 949 abstracts, non-RCT-A (n = 687) substantially outnumbered RCT-A (n = 262), although the proportion of RCT-A increased slightly in more recent years (Fig. 2). Results of the present study indicated that overall reporting quality was suboptimal for both RCT-A and non-RCT-A. The OQS was used to infer the overall reporting quality of the abstracts (an OQS of 15 indicated complete reporting). The median (IQR) OQS was 7.0 (6.0-7.0) for RCT-A and 3.0 (3.0-4.0) for non-RCT-A. The median OQS was significantly higher for RCT-A than non-RCT-A but both medians were less than half of the maximum score of 15. OQSs, though, should be interpreted with caution. A low OQS does not necessarily indicate a poorly conducted trial. The quality of reporting differs from the quality of the methodology (e.g., well-conducted trials may be reported poorly)23 and should be evaluated in different ways. Present study findings of suboptimal reporting were consistent with those of previous studies in other fields of health research6,10,13-16.
The reporting of specific items in the modified checklist varied greatly from item to item. Approximately two-thirds of the items were rarely or never reported; indeed, several items (title indicating the study as randomized, blinding, number analyzed, trial registration and funding) were not reported in RCT-A or non-RCT-A or both (Table 4).
| Fig. 4(a-b): | Mean OQS of the (a) non-RCT-A and (b) RCT-A from chicken research from 2006-2015 |
Our finding that neither the RCT nor non-RCT studies indicated randomization in the title was consistent with those of previous studies in livestock19,20 but differed from studies with human subjects, where more than half of the studies indicated randomization in the title9,12. These discrepancies may indicate that reporting randomization in the title improved after medical journals began to adopt the CONSORT statement as a guideline for manuscript preparation9,12. Unlike medical journals, veterinary and animal science journals have not adequately implemented this guideline, even though the REFLECT statement was developed in 2010 to improve RCT reporting for livestock. Of the five journals from which most abstracts were obtained, only J Anim Physiol Anim Nutr (Berl) currently encourages authors to adhere to animal research reporting standards. Identifying RCT studies by searching databases may thus be more difficult for studies in livestock than those in humans.
Trial registration and funding were not reported in RCT-A and non-RCT-A for chicken research. Trial registration is important to encourage the publication of negative trials, preventing publication bias24; leading medical journals therefore require trial registration as a condition for publication25. As far as we are aware, however, a formal agency for the registration of animal trials is not available and this issue is a concern, especially for animal models of human disease26. Neither RCT-A nor non-RCT-A reported funding, likely because funding is usually reported in the " Acknowledgements" section of the manuscript. The reporting of funding in the abstracts of medical journals varies from 0%7 to 80%10, indicating variable journal requirements but this reporting has improved over time9,12.
Most methodological items were better reported in RCT-A than non-RCT-A from chicken research. Randomization, one of the most important items in this domain, is an experimental-design tool used for reducing bias and for categorizing trials into RCTs and non-RCTs. The reporting quality of abstracts from the medical literature is usually assessed for RCTs only7,10,11,14,15 because non-RCTs are not widely acceptable due to the high risk of bias. A previous study of livestock abstracts20, however, found that non-RCT-A outnumbered RCT-A, so we decided to study both types of abstracts. Results of the present study for chicken research confirmed the findings of Snedeker et al.20. It is found that the majority of abstracts reported the objective, which is consistent with previous studies of human trials8,11,16. The reporting of clearly defined primary outcomes was suboptimal for both RCT-A and non-RCT-A from chicken research. The abstracts typically reported several outcomes but did not clearly specify the primary (main or principal) outcome. Blinding is also an experimental-design tool for reducing bias but blinding was not reported in any of the RCT-A and was reported in only one of the 687 non-RCT-A. The abstracts of medical journals also inadequately report blinding, ranging from <10%7,10 to <40%14 but reporting of blinding has improved over time11,12.
The numbers analyzed item in the results domain was reported in only one of the 262 RCT-A and was not reported in any of the non-RCT-A. This finding differed from human studies, which reported this item at rates ranging from >10%10 to >50%9,14. This discrepancy may be due to the different natures of animal and human trials. Reporting the number of subjects analyzed is crucial in human trials because participants may withdraw from trials at any time, leading to a difference between the number of participants analyzed and the number randomized. However, it is important to acknowledge that animals that are randomized in the study may be dropped from the analysis for any number of reasons (death, injury, loss of individual identifying number, etc.), so it is still essential for researchers in animal trials to report the numbers of animals analyzed for each intervention group. Surprisingly, reporting outcomes in abstracts for chicken trials (primary outcome, a result for each group and the estimated effect size and its precision) was rarely done (1.1% for RCT-A and 0.9% for non-RCT-A). This finding also differed from human trials12,14 because primary outcomes were not clearly defined and the precision of the estimated effect size was rarely reported in the abstracts from chicken research.
Many factors may be associated with overall reporting quality. It is observed that four factors (year of publication, number of trials reported, number of experimental groups reported and sample size reported) were associated with OQS. Overall reporting quality of the RCT-A and non-RCT-A was suboptimal but results of the present study indicated that the quality improved slightly over time (Figure 4). This finding is consistent with studies in medical journals9,12. Overall reporting quality in the medical literature clearly improved in both full-texts27,28 and abstracts9,12 after medical journals adopted the CONSORT statement and checklist. Concerns of reporting quality for animal studies have been raised for both laboratory animals and livestock. Some useful guidelines (the ARRIVE guidelines for laboratory animals29 and the REFLECT statement for livestock18) have been developed to help authors prepare their manuscripts for animal studies but implementation is still not common30. Reporting multiple trials per article was as high as 13.7% in RCT-A and 33% in non-RCT-A from chicken research (Table 2), unlike in human trials, where most articles report only one trial. Reporting quality was lower for multiple than single trials due to space constraints. A substantial number (24.6%) of the non-RCT-A did not report the number of experimental groups, resulting in low OQSs. A two-parallel-group design is common in human trials but the majority of chicken trials have more than two groups (Table 2). Many abstracts (12.8% for the RCT-A and 54.1% for the non-RCT-A) did not report sample size (number of chickens, cages, pens, or other replicates), resulting in low OQSs.
This study has several limitations. First, a comparison between the abstracts and their corresponding full-text articles was beyond the scope of our study. The reporting quality of the abstracts could therefore not be associated with or infer the reporting quality of the full-text articles. Second, the RCT-A and non-RCT-A in this study were categorized based solely on the information in the abstracts. Abstract types should be interpreted with caution and should not be confused with the real design of the trials (RCTs and non-RCTs). That is, a real study design of a particular non-RCT-A may be either a randomized controlled trial or a non-randomized controlled trial, depending on the detailed information provided in the Methods section of a full-text article. To answer this misclassification bias, we further assessed 530 available full-texts of non-RCT-A. We found that 283 (53.4%) were identified as RCTs in the Methods section. This result indicated a substantial discrepancy between the full-texts and their abstracts. Proper and reliable reporting should put important information, such as study design, both in the abstract and in the full-text to prevent miscommunication, especially when readers do not read the whole article. Third, we only used the PubMed database, so our findings may not be representative of all controlled-trial abstracts from chicken research. Inference of these findings to other databases should be carefully justified. Indeed, a preliminary search with the same keywords in SCOPUS and ProQuest Agriculture Journals found that both databases contained more initially identified abstracts than PubMed. We expected that the reporting quality of the abstracts would be more heterogeneous for SCOPUS and ProQuest Agriculture Journals than PubMed because these two databases contained more journals of chicken research. Fourth, we used the modified CONSORT for Abstracts checklist, the original version of which was primarily designed for use for human trials. Even for human trials, the criteria for scoring each item may be set or judged differently depending on author perspectives, which may produce different reporting scores from study to study. In fact, different authors define " Reporting quality score" differently, e.g., an overall quality score with a maximum score of 189 and an overall CONSORT score with a maximum score of 1611. In our study, if we changed the criteria " Number randomized" (number of animals randomized to each group) to " number assigned" (number of animals assigned to each group), the mean (SD) OQS of non-RCT-A was slightly increased from 3.3 (1.1) to 3.8 (1.3). Lastly, our multiple ordinal logistic regression analysis indicated significant associations between some predictor factors (publication year, number of trials reported, number of experimental groups reported and sample size reported) and the reporting quality of the abstracts. These four factors explained approximately 39.4% of the variance of OQS in our final multiple regression model. Other potential factors beyond the scope of the present study might be associated with OQS.
CONCLUSION
Reporting quality was significantly better in chicken research abstracts that reported randomization than in abstracts that did not but the reporting quality of both abstract types was suboptimal. The results of this study indicate the need for developing strategies to improve reporting quality in abstracts from chicken research. Specific guidelines should be developed for reporting controlled-trial abstracts from chicken research to improve the transparency, completeness and detail of reporting.
SIGNIFICANCE STATEMENT
This study discovers suboptimal quality in the reporting of controlled trial abstracts from chicken research. This result will urge chicken research communities to develop strategies for improving reporting quality.
ACKNOWLEDGMENTS
This study was supported by the Faculty of Veterinary Medicine, Khon Kaen University, Khon Kaen, Thailand. We would like to thank Paphawi Theppharin, Panomyong Sittiwong and Chitchanok Chitchuea for data verification. We would like to thank Dr. William Blackhall for language editing.