Before Dr. Fred Heath joined the University of Texas Libraries as Vice Provost and Director, he had spent several years at Texas A&M with a team of faculty and research professionals developing and honing an assessment tool called LibQUAL+™. With a certain degree of prescience, his earlier work has given the Libraries a leg up in dealing with the economic downturn.

Dr. Heath took some time to discuss his experience and perspective on the vital role of assessment in building a library.

So, why do assessment?

Fred Heath: I think this pressure for accountability has always been part of the public sector where it’s really hard to measure bottom lines. If you’re going to succeed in advancing your program then you are going to have to have some structure that relates investments to outcomes, and there aren’t many tools to measure our “profits and losses” in the public sector. For several years we’d been searching and we found a tool called SERVQUAL that was used with great success in the private sector, and we also had a great relationship with the developers of that tool. We started noodling there, and it grew.

There were three young professors in the College of Business at Texas A&M – assistant professors at the time they started developing SERVQUAL – who needed a research protocol, and grew it into, perhaps, the most significant user satisfaction survey in that sector. It applied to everything from aircraft companies, to insurance, to restaurants. And one day we visited with the professor who was still there – the other two are now employed elsewhere around the world in higher education – and asked, “Do you think we can redirect SERVQUAL to the not-for-profit sector, specifically to education, and then even more specifically to libraries?” And we were really cautious, because it was a shameless emulation of what those three researchers were trying to do, but he was hugely supportive, open to the idea, and, in fact, all three of those developers have lent time to us over and again to help us build the tool that LibQUAL+™ became. Without them, without that beginning, I doubt we would’ve had the perspective and background ever to get it launched.

How important was it for the development of LibQUAL+™ to have faculty members as part of the development team?

FH: We could never have done this without the methodologists and the economists and the statisticians that we had on our side. And we, ourselves, were faculty in our own specialties, but not those skill sets. It was bringing all of those tools, all of that commitment, an emerging awareness of survey protocols in a nascent Web environment…we’d had no clue how we’d plan to do this on paper, and one of the methodologists said, “You know, I think we can do this on the Web pretty soon.” So, it took a village to build it; it took many different types of faculty to make it happen.

We had to re-ground the instrument for the not-for-profit sector, and one way you do that is, you go out around the country, because you want to make sure that you’re developing the tool across types of universities, types of clients – from the undergraduate to the graduate student to the faculty, and you want engineering faculty as well as humanities scholars. So we traveled and traveled and traveled. That took a year of just building the possible questions to test. And we had to test a far larger number of questions than we now use in order to come up with the twenty-two that measure the dimensions of LibQUAL+™. So, that shakedown was a terrific challenge. We begged the patience of many faculty. The year that we were trying to ground fifty-two question, it must have taken 30 minutes to complete the survey.

So clearly it has developed over the course of time.

FH: Yes. Much to my surprise. We do take some satisfaction out of the fact that around the world, more than a million people have responded to the survey. More than a thousand different universities have taken it. It’s in more than a dozen languages – those are official languages. The highest point of praise, I guess, is imitation; we have two instances from Iran where it’s been taken from the literature and basically reissued in a clandestine way…published in the literature subsequentely, so we’re pretty pleased with the reality that it is a sort of standard now.

I noticed in other interviews that you had incorporated practices from peer institutions, like the Clemson Summit (PDF link). Are there any other practices that you’ve added over the course of time that may have been picked up from others?

FH: I think so. The community is a fairly small group of practitioners…those in library assessment. There are probably a hundred to two hundred people that have come to know each other’s work, come to share each other’s work, share the protocols and develop the tools. We have now held three international conferences on library assessment where we bring our various tools together. LibQUAL+™ is still one of the drivers, still one of the major tools, but we have balanced scorecard folks, we have other protocols that also come in and share that stage, that planning with us. And so, at the same time that we’re borrowing each other’s best ideas, we’re also building a suite of tools. We recognize that LibQUAL+™ is not THE answer. There are other tools that can answer other needs. There are other tools that can advance planning. LibQUAL+™ is not really a planning tool, it’s an assessment tool. But it does fold neatly into balanced scorecard, for example. So we have that community now of about two hundred or so of us that get together periodically. We have a conference in the UK in August, and we’ll likely go, and likely take some of our team to that meeting.

I can’t point to any single issue, but I can point colloquy, that conversation among experts in the field, and I do think that LibQUAL+™ was a prime mover in building that collaboration.

We talk about tools in a toolbox, and it’s one of a half dozen tools that people can choose from depending on their own needs, depending on their own skill sets that they can bring to the problem. One of the advantages of LibQUAL+™ is probably that it is the most forgiving of all the tools. It requires the least expertise resident in your own situation. It can be employed from without and can assist you in interpreting the results, so that’s part of its appeal and part of its strength.

I want to touch on the current political/budgetary situation. Your initial experience with developing LibQUAL+™ occurred while you were still at Texas A&M, so you’ve had the time to view the successes of the assessment tool. Can you speak about some of those?

FH: Again the key and the term that’s popular right now is this term of ROI – return on investment – with pressures on the public sector to be ever more efficient, ever more accountable. T’was always thus. That’s where we started, and that’s what we’re trying to do with each iteration, here and elsewhere, to show that the dollars are going to the things on the academic campuses, in the public libraries, that are most important – to the undergraduates, the graduates, and the graduate students in the college sector, or to the taxpayer – in the public sector. So you can tell which things are less important, which things are most desirable, among those things that are most desirable, where you are doing a less effective job than you might. It can inform you to redirect resources from an area where you may be very successful, and it may be very important, but you can afford to move some of those dollars to an area where you are succeeding less well. It gives you the comfort to move a dollar from an area that’s well received to an area where the client says, “I need help here.” And you are able to turn to the people who allocate resources to you and show that you are really good stewards with their dollars.

Think about an academic campus: all the areas where services are distributed, from student counseling to parking to dorms to food services to IT to the libraries – each of those areas could be measuring their success, could be measuring the things that their clients feel are important. Right now, only one of those areas has a tool designed: the libraries. So, in that sense, we are able to tell resource allocators that we know what they want, we know what we’re doing well, we know where we need to do better. Not everyone can say that. It creates a climate of confidence that should be there all the time, but is especially important in times when pressures for accountability are huge, such as now.

So having a basis in assessment would allow for a certain degree of confidence in approaching solutions to problems, because you’ve already done the legwork.

FH: And a little bit more confident in mapping a strategic future, because I think needs evolve, but we have a pretty good sense of a pulse of the community. We know what’s important to them. We know where they’re asking us to go. So we can deliver whatever resources are available with a little more confidence toward the things we know that the client needs.

The full interview originally appeared in the University of Texas Libraries Spring 2011 Newsletter, which also contains Dr. Heath’s follow-up column, “From the Director.”