IEEE TCDL Bulletin
space space

TCDL Bulletin
Volume 3   Issue 2
Summer 2007


Pilot Testing the DigiQUAL™ Protocol: Lessons Learned

Martha Kyrillidou
Association of Research Libraries
21 Dupont Circle, Suite 800
Washington, DC 20036
(202) 296-2296
Sarah Giersch
Association of Research Libraries
312 Severin St.
Chapel Hill, NC 27516
(919) 960-8208

The Association of Research Libraries is developing the DigiQUAL™ protocol to assess the service quality provided by digital libraries. During 2005-06, focus group statements and the online survey tool that comprise the DigiQUAL™ protocol were pilot tested with the National Science Digital Library and with six NSDL partner DLs in a two-step validation process.

First, DL developers selected statements (from a possible 180) they felt were most applicable to the services and content that DLs provided and that most matched users' experiences on their sites. Second, after the surveys were posted on developers' sites, users were presented with five randomly selected statements every time the survey form was viewed online. The two-step process reflected the need to negotiate with each DL the applicability of the items to their unique user communities. Twenty statements were selected by all DL developers as being applicable to their sites. It can be inferred that the statements describe key elements that are common across all sites.

Over a nine-month period, 1,107 DigiQUAL™ surveys have been completed. However, the number of completed surveys is not distributed evenly across participating DLs. This discrepancy in response rates has made it difficult to analyze and draw conclusions from aggregate data that would otherwise allow benchmarking across sites. Factors to be controlled for include differing DL contexts (e.g., discipline or audience types) and various online survey notification and distribution methods. ARL continues to investigate promising methods for identifying and recruiting respondents and for effectively notifying users and distributing online surveys.

Thumbnail image of poster

For a larger view of Figure 1, click here.
Thumbnail image of poster

For a larger view of Figure 1, click here.

© Copyright 2007 Martha Kyrillidou and Sarah Giersch
Some or all of these materials were previously published in the Proceedings of the 6th ACM/IEEE-CS Joint Conference on Digital libraries, ACM 1-59593-354-9.

Top | Contents
Previous Article
Next Article
Home | E-mail the Editor