• Home
  • Other
  • Evaluation Worksheet for Critiquing Research Study Report

Evaluation Worksheet for Critiquing Research Study Report Essay Example

  • Category:
    Other
  • Document type:
    Research Paper
  • Level:
    Undergraduate
  • Page:
    7
  • Words:
    4918

HST2122_171 Assessment 2 Worksheet for Evaluating a Health Research Study Report

HST2122 Assessment 2

Worksheet for Evaluating a Health Research Study Report

Instructions: Use this template to record your evaluation of the article that you have critiqued for Assessment 2. See separate instructions for detailed guidance on completing assessment.

NOTE: Hide or remove the rows in the summary table, and pages of the detailed critique, not relating to the article chosen for critique. Your summary table should reflect the score and overall assessment given (and supported by evidence) for each component, in the detailed tables to follow. The current row width can be expanded as high as necessary to provide a complete base of evidence.

Critic’s Name

End-text citation of article critiqued:

Zeek, M., Savoie, M., Song, M., Kennemur, L., Qian, J., Jungnickel, P., Westrick, S.

(2015). Sleep Duration and Academic Performance Among Student Pharmacists. American Journal of Pharmaceutical Education, 2015; 79 (5).

Summary Table

Component of evaluation

Max score

Overall assessment of component

Part A: Evaluating the Front Matter in a Research Report

Adequate quality

Part B: Evaluating the Statement of the Problem in a Research Report

Low quality

Part C: Evaluating the Literature Review in a Research Report

Low quality

Part D: Evaluating a Purpose in a Research Report

Adequate quality

Part E1: Evaluating the Research Design in a Quantitative Research Report

Adequate quality

Part F1: Evaluating the Participants and Data Collection in a Quantitative Research Report

Part G1: Evaluating the Data Analysis and Results in a Quantitative Research Report

Part E2: Evaluating the Research Design in a Qualitative Report

Low quality

Part F2: Evaluating the Participants and Data Collection in a Qualitative Report

Low quality

Part G2: Evaluating the Data Analysis and Findings in a Qualitative Report

Adequate quality

Part H: Evaluating the Conclusion and Back Matter in a Research Report

Adequate quality

Maximum possible score / total score given

And Overall assessment

1 Quantitative OR

2 Qualitative

Summary evaluation and justification

In summary based on the final score of this study as indicated above, this study being a qualitative study I would conclude that the quality of this research study is above average (80/129), but was found to be weak in many aspects. Consequently, the findings of the research study while they could be valid to a large extent, the accuracy could not be independently established based on the findings discussed here alone.

Part A: Evaluating the Front Matter in a Research Report

Quality Criteria

Quality Rating

Your Evidence and/or Reasoning

3 = Excellent

The Key Elements

  1. The study’s authors and journal are reputable.

The research study has a total of 7 authors that were involved in the study. Out of the 7 authors, 3 of the authors have PhD qualifications while the rest 4 authors have degrees in Pharmacies. Therefore I surmise that the author of this study have competent skills in conducting research studies because they have advanced education, and also because by working as a team they have been able to complement each other in both skills and experience.

Additionally, the study is published in the journal “American Journal of Pharmaceutical Education”. This journal has a long history of publication and was found in 1937 and is therefore considered as a reputable leader in publications of pharmaceutical related studies. However, this journal has an impact factor of only 1.082 as at year 2014. Based on impact factor rating this is the lowest and for this reason the repute of this journal is not high but this is compensated by the long history of publication for this journal.

Therefore I would give a rating of 2 at this section.

  1. The title reflects the content and focus of the study.

The title of this study is “Sleep Duration and Academic Performance Among Student Pharmacists”. This title is summarizes the research study in 1 sentence, is concise and an excellent topic of the study given the focus of this study.

  1. The abstract concisely but accurately summarises the aim, methodology and main findings of the report.

The abstract used for this study is not prose but is in fact a snapshot and highlight of each section of the study namely; objective, methods, results and conclusion. By adopting this approach one is able to quickly review the study designs and findings quickly to grasp the content of the study. This abstract is therefore excellent for this study.

General Evaluation

  1. The front matter accurately reflects the content of the report and allows an evaluation of relevance.

Based on above evaluation the study research is appropriately written and relevant to the research design of the study.

Overall Quality Part A

0 — 6 = Low quality

7 — 9 = Adequate quality

10 — 12 = High quality

Total Part A
Score = 8

My Overall Assessment of Part A = Adequate quality

Part B: Evaluating the Statement of the Problem in a Research Report

Quality Criteria

Quality Rating

Your Evidence and/or Reasoning

3 = Excellent

The Key Elements

  1. The topic is interesting.

The authors of this study have chosen a topic that is not widely researched, or even published. The topic of the study is basically an evaluation of association between sleep duration and academic evaluation. This subject matter is of curiosity among students and educators since it can provide insight in an area that is not well researched. It is therefore an interesting field to study.

  1. There is a meaningful problem.

0

A research study that attempts to investigate an inference between sleep duration and academic performance among pharmacist students would not score highly as a subject of meaningful significance. First because the study is too limited and focus only among pharmacist students, which involves an insignificant number in the population. And secondly because the study in itself through this study is not attempting to solve a need or a pressing problem in the society but rather is focused on providing insight in an area not affecting significant number of the population.

  1. The importance of the problem is justified.

The justification provided by the authors in this study are majorly;

— “Suboptimal sleep is a national problem, with more

than a quarter of the US adult population not obtaining the

recommended 7 hours of sleep each night”

— “Few studies have explored sleep

habits in populations of students who are pursing health

care degrees.” (Zeek et al., 2015)

I find above justification not serious enough to justify a need for this study mainly because the focus is among only student pharmacists which are a minority in the population. But since it is an area that is rarely researched there is some form of justification.

  1. There are deficiencies in the knowledge about the problem.

Only few studies have investigated the association between sleep duration and academic performance and therefore there is indeed lack of knowledge in this topic.

  1. There are audiences who can benefit from the missing knowledge.

The findings of this study can only be generalized across a small population group, and to a less extent across the wider student population. Because of this limitation only a few can benefit from this study.

General Evaluation

  1. The passage clearly argues that the study is warranted.

0

The general justification for this study is relatively weak. No major need or problem in the society is being addressed through this study.

  1. The passage is well written.

The research study is well structured, designed and discussed. This is good.

Overall Quality Part B

0 — 10 = Low quality

11 — 16 = Adequate quality

17 — 21 = High quality

Total Part B
Score = 10

My Overall Assessment of Part B =

Low quality

Part C: Evaluating the Literature Review in a Research Report

Quality Criteria

Quality Rating

Your Evidence and/or Reasoning

3 = Excellent

The Key Elements

  1. The review includes the relevant literature.

The purpose of a literature review is to document previous studies that has been done on the field of study while at the same time exploring new areas that warrant research (Creswell, 2014). The authors of this study were not able to achieve this since this study has no literature review section and consequently only very minimal literature review was done in the introduction section of the study.

The study has cited a list of 22 sources in the references list. Out of this all except one are journal articles which demonstrates that the choice of references is superb. But the chosen articles used in the study are relevant to the topic of this study

  1. The review examines sources that are recent and of high quality.

0

Out of all the journal articles used only 1 is less than 5 years old while the rest have year of publication beyond 5 years. The implication of this is that the study have not used recent literature review which impacts on research quality

  1. The literature review is appropriately documented
    .

This study has no literature review section

The study authors have done an excellent job in documenting the sources used, but only for end-text citation but not for in-text.

  1. The literature is thoughtfully synthesized

    .

0

The review of the literature review for this study was only done in a small section in the introduction of the paper. Therefore, no adequate synthesis of the literature cited in the paper was done. The citation of existing literature is at it minimum considering there was no literature review section.

  1. The literature is critically examined

    .

0

The authors of this study were not able to demonstrate a critical examination of the sources used in the study. The theories, themes and evidence of literature of citation is hardly mentioned or discussed.

General Evaluation

  1. The study has a strong foundation in the literature.

The study has not been able to demonstrate a strong foundation of the current literature in the field of study. It also does not appear to build from the findings of other studies previously done on the same subject.

  1. The use of the literature fits the study’s overall research approach.

The literature used is relevant for the study topic, design and approach. The choice of the literature is also appropriate and relevant.


Overall Quality Part C

0 — 10 = Low quality

11 — 16 = Adequate quality

17 — 21 = High quality

Total Part C
Score = 5

My Overall Assessment of Part C = Low quality

Part D: Evaluating a Purpose in a Research Report

Quality Criteria

Quality Rating

Your Evidence and/or Reasoning

3 = Excellent

The Key Elements

  1. The study’s purpose is clearly specified.

The research study identified only 1 objective of the research without indicating the overall research goal. Usually a study requires the overall research purpose which is the goal/focus of the study, which is then broken into various areas of research study focus in form of research questions or study objectives.

For this study the authors only highlighted the study objective alone without elaborating the overall study purpose succinctly.

  1. The focus of the study is appropriate.

The authors have picked a narrow enough study area that they chose to focus their research study. Therefore, the focus of the study is excellent for this study since it does not digress throughout the study.

  1. The overall intent of the study is appropriate.

The study sought to “to identify sleep patterns among student pharmacists and the frequency of daytime sleepiness during the school week and to assess the association between sleep duration and academic performance among these students.” (Zeek et al, 2015).

Towards this end and as far as investigating association between academic performance and sleep duration, the intent of the study has been achieved.

  1. The participants and sites are appropriate.

The site of this study was carried out at the main University of Auburn University’s Harrison School of Pharmacy. This is Higher Education learning institutions that trains pharmacists’ students. Therefore in regard to the site and participants chosen for this study this is appropriate.

However, in choosing only 1 university to carry out the study the participants have opened up possible study biases that are inherent in this type of sample selection such as selection bias

  1. The purpose is narrowed through appropriate research questions and/or hypotheses.

This study did not have critical sections that are necessary in good research designs. The study did not have a research goal, research questions, or hypothesis. Rather the authors opted to narrow the research study using 1 study objective which was not able to narrow down the research focus to a specific area.

General Evaluation

  1. The purpose follows logically from the statement of the problem and literature review.

This study does not have a clear link between the problem statement of the study, to literature review and finally towards research methods. In fact, the study does not have a succinct problem statement or a literature review section. The authors only allude to the problem statement as being lack of adequate research on this area as well as lack of knowledge in how sleep impacts academic performance.

Because of the way that the study is designed the writer can only make inferences in regard to study design problem statement, justification in order to link to the study methodology.

  1. The purpose is consistent with the study’s overall approach.

The purpose of the study which is to establish if there is an association between hours of sleep and academic performance is consistent with the study design adopted by the authors to research the same. The respondents were assessed on their sleep durations vis a vis academic grades obtained through a questionnaire. This approach is appropriate to establish a causal association


Overall Quality Part D

0 — 10 = Low quality

11 — 16 = Adequate quality

17 — 21 = High quality

Total Part D
Score = 14

My Overall Assessment of Part D = Adequate Quality

Part E1: Evaluating the Research Design in a Quantitative Research Report

Quality Criteria

Quality Rating

Your Evidence and/or Reasoning

3 = Excellent

The Key Elements

  1. The choice of the research design is appropriate and justified.

The authors of this study employed a mix method research study that involved qualitative and quantitative approaches. In general, “mixed methods research represents research that

involves collecting, analyzing, and interpreting quantitative

and qualitative data in a single study or in a series of studies

that investigate the same underlying phenomenon” (Plano Clark and Creswell, 2015).

The strength of using mixed method in research is that it allows a researcher to take advantage of both approaches of designs in order to obtain a better insight of a research problem more than either approach alone can provide (Plano Clark, 2007; Cameron, R and Miller, 2011).

Mixed method research design is the most preferable approach in conducting research in the field of social sciences because it allows analysis of various categories of data (Onwuegbuzie, 2008; Baumeister and Leary, 1997). The study also utilizes use of a well-designed sampling approach which is ideal for this study.

General Evaluation

  1. The study used a rigorous research design.

This study lacks the research rigor that is expected of high quality research studies. The research study has not put in place various controls to guard against the various forms of biases that are inherent in any form a research study. Because of this, there are multiple areas that this study could be biased from sample selection bias, respondent bias to researcher bias.

  1. The use of the quantitative research design addressed the study’s purpose.

The study applies use of qualitative research design approach. Through this design the study was able to demonstrate that

— “Student pharmacists had an average

sleep duration of a little over 6 hours on a typical school

night. They had even greater sleep deficits the night prior

to an examination, with an average sleep duration of 5

— “Longer sleep duration the night prior to an

examination was associated with higher course grades and semester grade point averages (GPAs)” (Zeek et al., 2015).

The findings above are based on analyses of qualitative data obtained during the study which demonstrates that it was sufficient to make a finding in regard to the study purpose.


Overall Quality Part E

0- 4 = Low quality

5- 7 = Adequate quality

8 — 9 = High quality

Total Part E
Score = 5

My Overall Assessment of Part E = Adequate quality

Part F1 : Evaluating the Participants and Data Collection in a Quantitative Research Report

Quality Criteria

Quality Rating

Your Evidence and/or Reasoning

3 = Excellent

The Key Elements

  1. The sampling strategy is appropriate and justified.

  1. The sample size is appropriate and justified.

  1. Good quantitative procedures are used to select and assign participants

  1. High quality instruments are used to gather data.

  1. The data are gathered using ethical quantitative procedures.

  1. The data are gathered using standardized quantitative procedures.

General Evaluation:

  1. The study has a high level of internal validity.

  1. The study has a high level of external validity.

Overall Quality Part F

0 — 12 = Low quality

13 — 20 = Adequate quality

22 — 24 = High quality

Total Part F
Score =

My Overall Assessment of Part F =

Part G1: Evaluating the Data Analysis and Results in a Quantitative Research Report

Quality Criteria

Quality Rating

Your Evidence and/or Reasoning

3 = Excellent

The Key Elements

  1. The data were rigorously scored and prepared.

  1. Good descriptive analyses were conducted.

  1. Good hypothesis testing procedures were used.

  1. The results are comprehensive.

  1. The results include sufficient information.

General Evaluation:

  1. The data analysis represents a good quantitative process.

  1. The results provide a good explanation of the study’s purpose.

Overall Quality Part G

0 — 10 = Low quality

11 — 16 = Adequate quality

17 — 21 = High quality

Total Part G
Score =

My Overall Assessment of Part G =

Part E2: Evaluating the Research Design in a Qualitative Report

Quality Criteria

Quality Rating

Your Evidence and/or Reasoning

3 = Excellent

The Key Elements

  1. A research design guides the conduct of the qualitative study.

The design of the research study used for this study is appropriate for the topic that is being investigated.

Samples are necessary in helping a researcher to make inferences about the general populations. Therefore a researcher needs to determine the size and method of researcher design that is appropriate to the study in question.

Indeed, it is a rule of thumb that a “researcher needs to evaluate the most appropriate methodological approach to answer the specific research question” (Berg, 2001). This study adopted a mixed-method approach that involved both qualitative and quantitative data.

  1. The choice of the research design is appropriate and justified.

The study research design is appropriate for this study and justified since the study needed to collect data both quantitative and qualitative.

General Evaluation:

  1. The study used a rigorous research design.

0

Rigor is the degree that a research study’s pays attempts to control various forms of biases that may affect the validity and reliability of its results (Krefting, 1991). The research design applied in this case is not rigorous given the myriad of factors that potentially biased the study findings for this study. First, the selection of sample is biased since it only included ALL respondents from only 1 University, instead of taking various samples from a list of a number of universities which would have been a better approach. Secondly, the data collection for this study involved self-assessment whereby respondents were supposed to fill in the questionnaire, this by default opens up the issue of respondent bias particularly the Acquiescence bias “Also known as “yea-saying” or the friendliness bias, acquiescence bias occurs when a respondent demonstrates a tendency to agree with and be positive about whatever the moderator presents” (Krefting, 1991). Finally the study did not justify the sample size chosen since it does not provide the statistics of the general population of the subjects under study.

  1. The use of the qualitative research design addresses the study’s purpose.

The study applied minimal qualitative research study but which is clearly linked to study findings and conclusion.

Overall Quality Part E

0 – 6 = Low quality

7 – 9 = Adequate quality

10 – 12 = High quality

Total Part E
Score = 6

My Overall Assessment of Part E = Low quality

Part F2: Evaluating the Participants and Data Collection in a Qualitative Report

Quality Criteria

Quality Rating

Your Evidence and/or Reasoning

3 = Excellent

The Key Elements

  1. The sampling strategy is appropriate and justified.

The sampling of this study was not well designed, neither does it take into consideration randomization aspect that is desirable for all experimental research studies. Because there was no randomization one can argue that by using samples only from a single university, the study findings cannot be generalized across the general populations of the study subjects in this case, students pharmacists nationwide.

In addition, because the general size of the population is not mentioned in the research study it is impossible to determine whether the sample size chosen is an adequate representation of the population under study. Finally, it will appear that the authors deliberately opted for purposive sampling which unfortunately has the disadvantage of introducing bias.

Therefore it is impossible to justify the sampling procedure used in this study and there is no doubt it has also created bias in the study

  1. The sample size is appropriate and justified.

0

It is impossible to determine if the sample size is adequate or reliable since we don’t have information regarding statistics of the general population nationwide, in this case student pharmacists. In addition, the technique for deriving the sample size is not discussed by the authors.

The sample size therefore cannot be justified.

  1. The data types are appropriate.

This study uses questionnaires to collect the various types of data needed for this study that include qualitative, quantitative, discreet and continuous data. The mix of various data types is appropriate to investigate the association between sleep duration and academic performance that the authors are attempting to investigate. By using both types of data the study findings are more strengthened since data quality and comparison of data is enhanced in the study.

  1. The data are gathered using rigorous qualitative procedures.

The data for this study was gathered using questionnaires that were self-administered. The researchers also pretested the tools during a pilot study before use in the main study where feedback was used to improve the tool.

For this reason it is clear that the researchers took into consideration factors of rigor in order to ensure data quality and validity and consequently study findings. However, other forms of data validation should have been pursued.

  1. Data collection issues are handled ethically and thoughtfully.

The study participants were asked for consent before they were administered with the data questionnaire. In addition the researchers had obtained ethical approval of this study prior to carrying the study among the target respondents. Finally, the respondents were briefed on the importance of the study and it benefits prior to assessment in order to convince subjects to participate in the study.

Therefore issues of data collection and ethical for this study were adequately addressed.

General Evaluation:

  1. The selected participants are information rich.

This can be said to be true of the target subjects for this study

  1. The database provides extensive and credible information about the central phenomenon.

The database used by the researchers in this study has not been disclosed in the study

Overall Quality Part F

0 — 10 = Low quality

11 — 16 = Adequate quality

17 — 21 = High quality

Total Part F
Score = 9

My Overall Assessment of Part F = Low quality

Part G2: Evaluating the Data Analysis and Findings in a Qualitative Report

Quality Criteria

Quality Rating

Your Evidence and/or Reasoning

3 = Excellent

The Key Elements

  1. The analysis process used rigorous qualitative procedures.

The data analysis used by the researchers is rigorous enough. Various statistical techniques that measures associations such as ANOVA and CHI-square tests were used in data analysis

  1. Strategies were used to validate the findings.

The researcher took steps to validate the findings through ensuring triangulation of data quality as well as instruments of study such as the questionnaire. Where researchers were not able to validate findings this was disclosed.

  1. The findings include description of the people, places, or events in the study.

The research was able to provide a description of the respondents, their age groups, location of study, missing data and other details of the study related to the same

  1. The findings include appropriate themes about the central phenomenon.

0

The study did not explore extensively relevant themes of the study such as education, academic performance or various other factors that are known to impact on student performance in Higher education learning institutions which would be expected in this type of study

  1. The findings relate multiple themes to each other.

0

No themes were explored in this study by the researchers

General Evaluation:

  1. The data analysis represents a good qualitative process.

The data analysis overall was good

  1. The findings provide a good exploration of the central phenomenon.

The findings of this study can be replicated by other researchers applying same methodology with minimal marginal error.

Overall Quality Part G

0 — 10 = Low quality

11 — 16 = Adequate quality

17 — 21 = High quality

Total Part G
Score = 11

My Overall Assessment of Part G = Adequate quality

Part H: Evaluating the Conclusion and Back Matter in a Research Report

Quality Criteria

Quality Rating

Your Evidence and/or Reasoning

3 = Excellent

The Key Elements

  1. The major results are identified and summarized.

The study was able to summarize the findings of the research in a succinct manner by providing major outcomes of the study. In this case the authors concluded that

“A majority of student pharmacists had suboptimal durations of sleep, defined as fewer than 7hours. Adequate sleep the night prior to an examination was positively associated with student course grades and semester GPAs” (Zeek et al, 2015).

This summary is based on the data analysis that was also well summarized.

  1. The results are thoughtfully examined in relation to the literature and personal reflections.

The study findings as discussed in this study is consistent with various literature of other studies done in this field. The authors were also able to draw parallel between the findings of their studies and other similar studies done previously. For instance in discussion the study notes “This finding is

consistent with Medeiros et al’s research among medical

students that found students who reported sleeping for

longer durations obtained higher scores on examinations,

as well as Veldi et al’s study that found sleep behaviors to be associated with academic progression” (Zeek et al, 2015).

Therefore based on this, this study is able to demonstrate consistency of findings across the sector which is a plus in the overall quality of the study.

  1. Appropriate implications of the results for practice are identified and justified.

The study was able to demonstrate the implication of the research. However, because of the limited nature of the focus of the study, the generalization of this study might not be applicable in wider population which makes inference of findings not applicable in wider general area.

  1. Thoughtful critiques of the study’s limitations are provided and appropriate for the research approach.

The authors have disclosed a list of three limitations of the study, both in design and analysis of data that might impact on the validity of the results as discussed.

  1. Suitable implications of the results for future research are identified and justified.

The study lacks suitable implications of research and fails to provide a logical and clear approach or areas of future research for this topic

General Evaluation:

  1. The interpretations are consistent with the study’s results and limitations.

The study has been able to demonstrate that findings imply that an association exists between academic performance and sleep patterns. This is also justified through the study results based on data analysis.

  1. The back matter is appropriate for the study report.

The conclusion of the study is appropriate and provides a summary of the study aspects consistent with literature review, study findings and analysis.

Overall Quality Part H

0 — 10 = Low quality

11 — 16 = Adequate quality

17 — 21 = High quality

Total Part H
Score = 12

My Overall Assessment of Part H = Adequate quality

References

Baumeister, Roy F. & Leary, Mark R. (1997). Writing Narrative Literature Reviews.

Review of General Psychology, 3: 311-320.

Berg, Bruce L. (2001). Qualitative Research Methods for the Social Sciences. Allyn and

Bacon. London

Cameron, R and Miller, P, (2011). Mixed method research designs: a case study of their

adoption in a doctor of business administration program. International Journal of Multiple Research Approaches, 5: 293-308.

Creswell, J & Plano Clark, V. (2015). Designing and Conducting Mixed Methods Research.

Washington. Sage Publications

Creswell,J. W.,Plano Clark,V. L.,Guttman,M.,& Hanson,W. (2003). Advanced mixed

methods research designs. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social & behavioral research (pp. 209-240). Thousand Oaks,CA:Sage

Krefting, Laura. (1991). Rigor in Qualitative Research: The Assessment of Trustworthiness.

The American Journal of Occupational Therapy, 45: 3, 214-222

Leech N, Onwuegbuzie A, (2008) A typology of mixed methods research designs, Quality

and Quantity, 43:2, 265-275.

Zeek, M., Savoie, M., Song, M., Kennemur, L., Qian, J., Jungnickel, P., Westrick, S.

(2015). Sleep Duration and Academic Performance Among Student Pharmacists. American Journal of Pharmaceutical Education, 2015; 79 (5).

13| PageAdapted from tables in HST2122 textbook Plano Clark and Creswell, 2015.