Another update: The bibliometric study evaluation tool

Back in 2007, I realized the need for a critical evaluation tool based on those created initially by Tricia Greenhalgh*, and used in evidence-based medicine. Applying tools used for medical research to LIS is not always a good fit, and when I evaluated a bibliometric study for the journal Evidence Based Library and Information Practice, for their Evidence Summaries section, I realized that something more specific might be needed.

Building on the basics of Greenhalgh's 12 questions, I considered what other elements should be questioned in a bibliometric study, and used the research article I was evaluating to guide its development. Later, I created and co-taught a course on evidence-based librarianship, and offered this tool as one of the critical evaluation question sets for use by students at UNC. Mentioning the tool to the Evidence Summaries section author Lori Kloda, I was pleased when she shared the document with a colleague who had expertise in bibliometric research. His suggestions brought about several improvements, particularly in the statistical methods evaluation.

Finally - and most recently - I have been teaching a doctoral seminar on research. The course is structured around the creation of a critically-evaluated literature review, so I once again offered the tool as one of the options for use, asking students to incorporate discussion of the utility of their chosen tools in their write-ups. Once again, as I'd done at Chapel Hill and in a medical library reference course, the write-ups were based upon the evidence summary model provided by the Evidence Based Library and Information Practice journal. Students have found the tool to be useful, and have even (in an unexpected outcome) incorporated questions about the literature review section from the document into other, previously created question sets.

Considering their comments, and writing another evidence summary myself at the same time, has brought about more changes to the original tool. I'm sure it still needs work - everything always does! - but I am pleased with the process, and driven to follow this trail in developing and testing such tools that can be used with research in the social sciences.  Here is the current iteration. I would sincerely welcome feedback and commentary on this tool.


* Greenhalgh, T. (2010). How to read a paper: The basics of evidence-based medicine. Chichester, West Sussex, UK: Wiley-Blackwell.

The Sense-making behaviors of hospital librarians

I will be presenting some of the results of my dissertation research at the 2012 Annual MLA Meeting in Seattle, on May 22. Here's a link to the handout: http://tinyurl.com/librarian-sensemaking . I hope it makes sense (pun intended) - it's no mean feat to try to collapse a 300-page dissertation into a 15 minute presentation! 

Article of interest

from Health Information and Libraries Journal by Paolo Gardois, Nicoletta Colombi, Gaetano Grillo, Maria C. Villanacci

Abstract

Background:  Academic, medical and research libraries frequently implement Web 2.0 services for users. Several reports notwithstanding, characteristics and effectiveness of services are unclear.

Objectives:  To find out: the Web 2.0 services implemented by medical, academic and research libraries; study designs, measures and types of data used in included articles to evaluate effectiveness; whether the identified body of literature is amenable to a systematic review of results.

Methods:  Scoping review mapping the literature on the topic. Searches were performed in 19 databases. Inclusion criteria: research articles in English, Italian, German, French and Spanish (publication date ≥2006) about Web 2.0 services for final users implemented by academic, medical and research libraries. Reviewers’ agreement was measured by Cohen’s kappa. From a data set of 6461 articles, 255 (4%) were coded and analysed.

Results:  Conferencing/chat/instant messaging, blogging, podcasts, social networking, wikis and aggregators were frequently examined. Services were mainly targeted at general academic users of English-speaking countries.

Conclusions:  Data prohibit a reliable estimate of the relative frequency of implemented Web 2.0 services. Case studies were the prevalent design. Most articles evaluated different outcomes using diverse assessment methodologies. A systematic review is recommended to assess the effectiveness of such services.

What is the role of the Medical Library Association during a time of crisis?

I have read and responded to recent posts on Medlib-L feeling the desperation and anger expressed by some librarians as their libraries close, as they talk about re-educating, wondering where the magic key might be. I read it and wonder whether the answer is an acceptance of changed/changing times. 

Will we go the way of the 8-track? How much of our response is due to fear of change – and can we even know if this is so?  

Some feel compelled to persuade administrators that without our expertise, health care quality is a deteriorated thing, based upon ignorant, sloppy searches (if searches are done rather than just flat-out retrieval, a la UpToDate). While efforts are ongoing to establish value, is it too little, too late, too unpersuasive? If we had the Gettysburg address of persuasiveness –unalterably and inarguably persuasive, would this reverse the tide?

Some feel the role of associations (in this case, MLA) is to help, even to lead a fight against loss, while others are embittered over what they see as past advocacy failures by the same associations to support the needs of non-academic practitioners. It is not a new perception. Nor is it completely without merit. 

And yet: The MLA Hospital Library Section’s Vital Pathways survey, conducted as the result of an intiative of then-MLA President M.J. Tooey, captured data not elsewhere assembled (Tooey, 2010).

The MLA Education and Research Sections rewrote standards to provide guidelines for change in order to empower practitioners, pointing toward more rigor in research in an environment where best evidence is highly regarded.

The Research Policy statement expressly reinforces its “commitment to assure the vital presence and continued growth of both individuals and the profession of information research” (MLA Research Policy Executive Summary). 

Meanwhile, the Education Policy Statement could not be more clear in its call for action:

“In light of the rate of environmental change, the specific knowledge and skills required of health sciences librarians, and the broad scope of the continuum of learning, it is clear that all who have a stake in the success of the profession need to take action” (MLA Education Policy Statement: Recommendations for Action).

I post this to refute the idea that MLA, as an entity, has not supported library practitioners. But I would also like to see the kind of discussion that is happening on the Medlib-L list take a broader form. This is time for leadership in exploring, and there is a real need for facilitation of peer support networking. I have seen how often hospital librarians use the listserv for many functions, including research, news, job postings, and importantly, to create and sustain a vitally important network of peers. For some, there are no geographically close peers; librarians are truly alone. Libraries close with a whimper, and they are mourned. There is anger, but little sense of direction. There is loss but where is healing?

Maybe it is in our involvement with our own association in ways that we have not been involved before, due to the perception that MLA is not representing our interests. 

Hospital librarians still constitute around 30% of MLA members, including those active in the Hospital Library Section. First, I believe that MLA should consider free or low-fee membership to librarians who have lost their jobs, or whose jobs are otherwise affected. Secondly, MLA/HLS might consider funding a research study hard on the heels of the Vital Pathways survey. 

Let us begin to understand the work environment of medical librarians in ways we do not, now. With that understanding, let us support the retooling of the profession, set the agenda for a discourse on change, investigate educational funding options for those who are deeply affected. 

If there is fear and loss, let us recognize it and respond to our colleagues.  If there is to be change, let us find the path together.

Misleading titles for $10,000: "Surprising decline in consumers seeking health information"

In one of those all-too-frequently overhyped announcements, the Center for Studying Health System Change drew attention to their new tracking report, published November 11, and now making rounds on consumer health oriented sites and medical/consumer health library listservs. I have to wonder if the same author wrote the content and the title of the announcement, because the content is solid - and because there really IS a good story here - well worth further study - but it got missed in the hype. Takeaways from this study include the following:

• Resource media type shifts: Reported use of print, television, and radio as resources for health information has dropped between 2007-2010. Meanwhile, online source use has risen only slightly, leading the study authors to speculate about cause. However, question construction and sampling changes may have had some effect upon results (see below).
• Change in health information seeking by those over 65: Seniors' use of online health information increased from 17% to 24% during the same period.
• Without more complete transparency, comparison of responses across the 3 studies (2001, 2007, and 2010 - the basis for the authors' conclusions) is problematic.

Overall, there is an obvious downward trend of health information seeking behavior, but my own takeaway is that numbers and statistics about information sources are never the whole story. The question should also be whether what was found proved satisfactory in answering health concerns.The report may also reflect a flattening trend, as the prior study found huge leaps in the use of online resources. None of the studies asked what specific resources were used, whether information was located, or whether the information located matched their questions.

--------------------------------------------------------

Bolding is my own:

In 2010, 50 percent of American adults sought information about a personal health concern, down from 56 percent in 2007, according to a new national study from the Center for Studying Health System Change (HSC). The likelihood of people seeking information from the Internet and from friends and relatives changed little between 2007 and 2010, but their use of hardcopy books, magazines and newspapers dropped by nearly half to 18 percent.

What you need to know: the study questions permitted overlap, so that respondents could check any and all information sources (see Table 1, footnote). In other words, some of what we may be seeing is that people who used to seek their health information in hardcopy and online are now using paper less frequently.

In 2007, there were three leading sources of health information—books, magazines and newspapers in print format; friends and relatives; and the Internet—each used by roughly one-third of adults to seek health information for themselves. By 2010, the use of friends/relatives and the Internet remained relatively steady, but the use of print media fell sharply—33 percent to 18 percent—accounting for much of the drop in overall information seeking. 

The previous tracking report, published in 2008, announced a Striking Jump in Consumers Seeking Health Care Information. In fact, the earlier study (2001), against which the data is compared, used a different participant selection method. While the authors were commendably transparent in their methodologies for the all-too-brief summation made publicly available, the 2001 study used cluster sampling, while the 2007 update employed stratified random selection:

Although both surveys are nationally representative, the sample for the 2000-01 survey was largely clustered in 60 representative communities, while the 2007 survey was based on a stratified random sample of the nation (section on Data source: http://www.hschange.org/CONTENT/1006/#ib8).

Data provided in the online report is insufficient to make this call, but it is not out of the question that sampling methods used in the 2001 study were changed for the 2007 and 2010 studies because they were problematic in some way. In cluster sampling (used in 2001), it's important to know whether within-cluster responses are homogeneous, but what I don't know from reading the summative report is how this was tested in practice.


The authors speculate that differences in source usage may be due to negative or frustrating experiences in health information seeking, frustration and confusion caused by (online) information overload, or even to a decline in health visits due to the economic downturn.

The decline in health information seeking may reflect, in part, consumers’ reactions to their previous experiences with health information ('Implications,' para. 2, http://www.hschange.org/CONTENT/1260/#ib5).

The real question to ask here is whether we are seeing a flattening-out of health information seeking activities, with saturation of use from those who now regularly use online sources?  Might the flattening-out also reflect a decreasing tendency to seek information from multiple sources, as people find preferred sites and (in the case of chronic health conditions) online communities? 

Mentioning MedlinePlus.gov as a leading source of reliable health information, the report concludes, in part:

Such resources can be valuable consumer tools; however, the consumers best positioned to locate these resources and use them constructively likely are already informed, sophisticated consumers. As a result, the gap between the haves and have-nots among health information consumers is likely to continue growing.

In efforts to address this disparity, policy makers—including the Institute of Medicine and the Office of Disease Prevention and Health Promotion in the U.S. Department of Health and Human Services—have implemented initiatives both to improve health literacy in consumers and to make information accessible and actionable to consumers across a broader range of health literacy skills. 

These reports don't just highlight the gap between haves and have-nots, they also identify a rich opportunity for research about what sources are used and why there appears to be a decline in resource types. For those who are concerned with supporting the health information needs of health consumers, this is a call for our involvement.

Librarians and informaticians on the health care team

Free full text available on the journal site, along with 29 other intriguing essays in response to the editor's 2011 Question of the Year: "What improvements in medical education will lead to better health for individuals and populations?"

Teaching Physicians to Make Informed Decisions in the Face of Uncertainty: Librarians and Informaticians on the Health Care Team
Moore, Mary; for the Association of Academic Health Sciences Librarians

Academic Medicine. 86(11):1345, November 2011.

doi: 10.1097/ACM.0b013e3182308d7e

From the essay:

As medicine becomes more complex and information technologies transform decision making, physicians must learn not only how to apply new tools and technologies effectively but also new ways of decision making that foster multiple inputs, including team input. Reaching this goal will require informatics education integrated throughout the medical curriculum, faculty role models with searching minds who understand biomedical and health informatics, and continuing education to retrain those who were educated under older models.


Librarians have been envisioning this future since the 1970s, when clinical medical librarians joined health care teams that included students, residents, attending physicians, nurses, nutritionists, and others. In the 1980s, librarians embraced a vision for the future where all health information systems would be connected when at all possible,and they have been working toward that future ever since.

Defended!

I'm happy to say that I successfully defended my dissertation on August 9th. This morning I read an interesting post from a new assistant prof, talking about a bit of post-defense let-down. I think I get that. The last bit prior to defending was my version of a high-speed chase right to the edge of a steep cliff... the cliff being several deadlines that had to be met. Meanwhile, I was teaching summer school for the first time ever, 3 courses. Not advisable, folks.

Now it's prep for fall - what happened there? I seem to have crossed some Twilight line, where time vanishes. I'm not sure what's in the news. We had the absolute pleasure of visiting the library yesterday, where I found a few items of questionable literary value but hopefully good entertainment value. Plus, I hit a small motherlode of nearly my favorite kind of book: the autobiographical foodie account. MFK Fisher only wrote so many things, and it's certain that others (Colwin, Bourdain) are no Fisher, but there's something irresistible about these works to me.

As I'm thinking about getting an e-reader (once there is not more month left than paycheck) I keep wondering if this sort of indulgence can be experienced in that form. One book, about the history of women through cooking, is large and heavy... the text is pretty bad, stylistically, and I'd never buy it, but the images are great. I would not buy this as an e-text, and I am not at all sure I'd check it out from the library. Besides, what's all that mess about publishers restricting the number of items libraries can check out to each patron?  I don't think so. Part of the joy for me has been leaving the library so overloaded that my arms ache. I used to be able to check out back issues of This Old House, and you know I flipped right to the back to see the remuddled building, taking pleasure in tsk-tsking at the tackiness of 70s siding on a Queen Anne. Books are weight relished, to me. Although. I need weight and lightness, depending - but I want to make the choice myself.

Hmm. Nice to think about something besides the you-know-what. I thought my husband was going to put a sock in it, because it's all I talked about for years. I'll get back to the topic of this blog, but right now - it's nice to be doing something else.