Back in 2007, I realized the need for a critical evaluation tool based on those created initially by Tricia Greenhalgh*, and used in evidence-based medicine. Applying tools used for medical research to LIS is not always a good fit, and when I evaluated a bibliometric study for the journal Evidence Based Library and Information Practice, for their Evidence Summaries section, I realized that something more specific might be needed.
Building on the basics of Greenhalgh's 12 questions, I considered what other elements should be questioned in a bibliometric study, and used the research article I was evaluating to guide its development. Later, I created and co-taught a course on evidence-based librarianship, and offered this tool as one of the critical evaluation question sets for use by students at UNC. Mentioning the tool to the Evidence Summaries section author Lori Kloda, I was pleased when she shared the document with a colleague who had expertise in bibliometric research. His suggestions brought about several improvements, particularly in the statistical methods evaluation.
Finally - and most recently - I have been teaching a doctoral seminar on research. The course is structured around the creation of a critically-evaluated literature review, so I once again offered the tool as one of the options for use, asking students to incorporate discussion of the utility of their chosen tools in their write-ups. Once again, as I'd done at Chapel Hill and in a medical library reference course, the write-ups were based upon the evidence summary model provided by the Evidence Based Library and Information Practice journal. Students have found the tool to be useful, and have even (in an unexpected outcome) incorporated questions about the literature review section from the document into other, previously created question sets.
Considering their comments, and writing another evidence summary myself at the same time, has brought about more changes to the original tool. I'm sure it still needs work - everything always does! - but I am pleased with the process, and driven to follow this trail in developing and testing such tools that can be used with research in the social sciences. Here is the current iteration. I would sincerely welcome feedback and commentary on this tool.
* Greenhalgh, T. (2010). How to read a paper: The basics of evidence-based medicine. Chichester, West Sussex, UK: Wiley-Blackwell.
Building on the basics of Greenhalgh's 12 questions, I considered what other elements should be questioned in a bibliometric study, and used the research article I was evaluating to guide its development. Later, I created and co-taught a course on evidence-based librarianship, and offered this tool as one of the critical evaluation question sets for use by students at UNC. Mentioning the tool to the Evidence Summaries section author Lori Kloda, I was pleased when she shared the document with a colleague who had expertise in bibliometric research. His suggestions brought about several improvements, particularly in the statistical methods evaluation.
Finally - and most recently - I have been teaching a doctoral seminar on research. The course is structured around the creation of a critically-evaluated literature review, so I once again offered the tool as one of the options for use, asking students to incorporate discussion of the utility of their chosen tools in their write-ups. Once again, as I'd done at Chapel Hill and in a medical library reference course, the write-ups were based upon the evidence summary model provided by the Evidence Based Library and Information Practice journal. Students have found the tool to be useful, and have even (in an unexpected outcome) incorporated questions about the literature review section from the document into other, previously created question sets.
Considering their comments, and writing another evidence summary myself at the same time, has brought about more changes to the original tool. I'm sure it still needs work - everything always does! - but I am pleased with the process, and driven to follow this trail in developing and testing such tools that can be used with research in the social sciences. Here is the current iteration. I would sincerely welcome feedback and commentary on this tool.
* Greenhalgh, T. (2010). How to read a paper: The basics of evidence-based medicine. Chichester, West Sussex, UK: Wiley-Blackwell.
Posts
