ASIST08: Evaluating E-Reference: Transforming Digital Reference through Research and Evaluation

There are a lot of sessions at ASIS&T (and probably most conferences) with fairly impregnable titles. I’ve found myself sitting in sessions which were about something very different than I thought. But this session title is pretty straightforward: It was all about evaluating virtual reference services.

Marie Radford (Rutgers University) and Lynn Connaway (OCLC) spoke about a long-term research project currently underway in which they’re evaluating users, non-users, and librarians about their positive and negative e-reference (and non-e-reference, in the case of non-users) experiences. Some key points:

• Librarians considered relational and attitudinal aspects of the reference transaction as much more important than users did; for users, answers (content) were key.
• Librarians want to teach and users don’t always want to be taught. It seems, though, that users are more open to instruction face-to-face than virtually.
• A great suggestion to work through the “don’t teach me” barrier: Provide the requested information, and then ask, “Would you like to know how I found it?” I think this is a GREAT idea, because it sounds almost like you’re offering a secret or something.
• Oh, earlier, when I noted that relational aspects of the transaction matter less to patrons: That’s not entirely true. They just THINK they matter less, but customer services is still important. It’s just that good customer service can be invisible to users, because they’re able to focus on the answer part.
• People are willing to wait longer for a subject-specialist.

And of course, all I could think of was my Reference course, in which we pretty much entirely focused on customer service and the theoretical implications of the reference transaction, but in which I didn’t really learn to do what users actually want: provide information. Sigh.

Jeffrey Pomerantz (UNC Chapel Hill—These UNC folks are everywhere!) talked about librarians participating in online answer boards like Yahoo! Answers. He was specifically talking about this Slam the Boards project, which involves, on the 10th of every month, librarians going onto these sites and answering questions. I’m not entirely sure why we’re only supposed to do this one day each month, but it’s an interesting idea. Jeffrey tried to evaluate whether librarians’ answers were better, or more specifically, whether they were rated more highly by question askers.

His conclusion was essentially that it’s really hard to evaluate librarian participation in online answer boards. There are approximately 100,000 questions posted to Yahoo! Answers everyday, their API for culling data doesn’t allow you to pull information for more than 5,000 questions, and as a researcher, one has to rely on users self-identifying as librarians. But his research raised some great questions and thoughts:

• Are these boards places where libraries should be? Most of the questions are kind of silly, and it doesn’t really seem librarians are making a huge impact
• In the midst of all these silly questions, there are some serious questions: How do we get the askers of these serious questions to remember their local library (or its online services) as a resource?

Lorri Mon of Florida State University and the Internet Public Library talked about blogging as a reference service, and about users’ needs and use in her area. She mentioned an article I’m going to try to read on the plane tonight: Pomerantz and Stutzman’s “Collaborative reference work in the blogosphere” (2006), and she primarily talked about how students in an online course used blogs and their comment sections to post questions, answer each other, and provide a sense of community.

She pointed out that most libraries are still not using social networking sites, and that of those they are, most of them are using MySpace (ugh). Her research shows that blogs are the most widely used technology tool in libraries, followed by wikis (which are mostly used as behind-the-scenes staff tools, as they are in my library). She talked a bit about the blurring of boundaries between different types of tools (chat embedded in blogs and facebook, chat reference in Second Life). Finally, she raised an interesting question: How is eReference being taught in library schools? Well, I can answer that from my own experience: barely. Sure, I read an article or two on it. But as she noted, there was no hands on experience, or even role playing, and that probably would have been easy to set up and helpful. But I think I already mentioned that I didn’t learn a ton in my reference course.

To close up the session, Joseph Janes responded to all three presenters and brought up some really excellent points. He largely talked about the differences between Yahoo and Google: Yahoo is about community, and Google is about answers. And in that way, Google Reference services makes more sense than Yahoo reference services. He argued that we need some hook into the mass of people with information needs, whether that’s a local hook, a subject-oriented hook, a site-based hook, or something else entirely (though I lean toward a subject-oriented hook and already have started thinking about how that might work. And I think it’s all about search engine optimization.).

Lorri Mon made the comment that “people are looking for their personal librarian,” and I jumped on this one. I’ve talked before about embedded librarianship in the academic community, and I think we need to set each student up from day one of their college careers with a personal librarian. What if we assigned a librarian to students the way we assign them advisors? Subject-specialist librarians could be personal librarians for people in specific departments (for example, when you declare your major, you’re assigned a reference librarian affiliated with that department). I think just the personalization of that, the introduction to a librarian, would make it less intimidating, and would make people think about going to the library for their information needs more readily than they otherwise might.

This was one of the most inspiring sessions I attended, in that I came out of it with ideas, papers to read, things I wanted to experiment with and research. So, thanks to all these great presenters! As though they will ever see my lonely little bloggity blog. 🙂