A Study on the Evaluation Model for University Libraries in Digital Environments

Kwak, Byeong Heui; Jun, Woochun; Gruenwald, Le; Hong, Sunwook

doi:10.1007/3-540-45747-x_15

Cited by 10 publications

(20 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…al., 2001] • Ease of understand and learn • Informativeness [Huxley, 2002; • Novelty • Potential distraction [Sumner et al, 2003] • Readability • Scalability Kenney et al, 1998; • Supportiveness of human-computer [Budhu & Coleman, 2002] & social/group interaction [Borgman & Gilliland-Swetland, 2000] • Timeliness (freshness) [Bekele, 2002;Kwak et al, 2002] • Usefulness …”

mentioning

confidence: 99%

“…However, considering the existence of diverse DLs and the necessity of integrating these systems, it is disappointing to see no evaluations on interoperability/compatibility among different DL systems, although the metrics are included in Kwak et al's evaluation model (2000) for university libraries in the digital age. Kwat et al's model is a product of their two-phased research (i.e., existing work survey + thrice-run Delphi) grounded on opinions from DL experts and researchers.Similarly, auxiliary functionality (e.g., privacy protection, firework) receives little attention.19 Table 2.3: Existing Criteria for DL Evaluation -Technology Hardware• Accessibility [Bishop, 1998;Kwak et al, 2002;Meyyappan et al 2000;Wilson et al, 2002] • Appropriateness for digital information • Comfort for use [Wilson et al, 2002] • Cost [Thebridge et al, 2002] • Display quality [Wilson et al, 2002] • Efficiency (number of node utilization [Xi et al, 2002]; response time Kengeri et al, 1999;; network related response time [Kapidakis et al, 1998;Kwak et al, 2002]) • Robustness for digital information [Marchall & Ruotolo, 2002;Wilson et al, 2002] Software• Accessibility [Bishop, 1998;Kwak et al, 2002;Meyyappan et al 2000;Wilson et al, 2002] • Auxiliary functionality (e.g. privacy protection, firework • Complexity in query support and response Meyyappan et al 2000] • Cost [Thebridge et al, 2002] • Efficiency (e.g.…”

mentioning

confidence: 99%

“…privacy protection, firework • Complexity in query support and response Meyyappan et al 2000] • Cost [Thebridge et al, 2002] • Efficiency (e.g. number of node utilization [Xi et al, 2002]; response time ; network related response time [Kapidakis et al, 1998;Kwak et al, 2002]) • Interoperability / compatibility (among different IR and DL system [Kwak et al, 2002] • Relevance-based effectiveness (e.g. number of relevant document retrieved Khoo et al, 1998], P/R Hee 1999;Khoo et al, 1998;Sanderson & Crestani, 1998], Actual vs. perceived P/R ; accuracy of surrogate extraction ; relative distance relevance • Reliability ; stability of system One might find that some system criteria listed in Table 2.3 (e.g., display quality, robustness for digital information) tends to evaluate electronic and communication devices rather than DLs.…”

mentioning

confidence: 99%

“…Table 2.6: Existing Criteria for DL Evaluation -Service • Accessibility-access barrier Cullen, 2001] • Accuracy • Cost Lankes et al, 2003] • Cost-benefit [Kantor & Saracevic, 1999] • Control capability [White, 2001] • Courtesy • Difference before and after service intervention [Bertot, 2003] • Empathy [Cullen, 2001] • Functionality [White, 2001] • Gaps between expectations of users and providers [Bertot, 2003] • Personnel support (e.g. number of staff engaged) • Positive feedback [Carter & Jones, 2000];• Reliability [Cullen, 2001] • Responsiveness [Cullen, 2001;Lankes et al, 2003;White, 2001] • Satisfaction (material/emotional) [Cullen, 2001] • Use Carter & Janes, 2000;Cullen, 2001;Hauptmann et al, 2001;Kwak et al, 2002;Lankes et al, 2003;Shim, 2000] For this reason, service evaluation is purposely separated from other evaluation levels for identification and discussion. (Bishop, 1999;Saracevic, 2000) in the domain of information science, DL effects are not only restricted in the ways in which people find information, but also the ways in which they live in society.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Developing a holistic model for digital library evaluation

Zhang

2009

J. Am. Soc. Inf. Sci.

View full text Add to dashboard Cite

The objective of the research is to develop a holistic model for digital library (DL) evaluation. To develop such a model, a three-stage research approach was applied: exploration, confirmation, and verification. During the exploration stage, a literature review was conducted, and then an interview along with card sorting technique was employed to collect perceptions from DL experts with emphasis on determining what criteria should be used in DL evaluation. Then, the criteria identified from the exploration were used for developing an online survey during the confirmation stage. Heterogeneous DL stakeholders were asked to rate the importance of each criterion to DL evaluation.The holistic model was constructed by utilizing descriptive and inference statistical techniques. Its holistic nature was ensured through: (1) incorporation of various DL stakeholders' perspectives in light of Marchionini's multifaceted evaluation approach, and (2) inclusion of all digital library levels suggested by Saracevic's stratified information retrieval model. Eventually, in the verification stage, selected criteria from the model were tested in real DL use setting. Some significant findings include: (1) consistently perceived important criteria for DL evaluation. DL stakeholders care more about premise (e.g., accessibility and iii iii sustainability of a DL), process (e.g., ease of use, technology reliability, and service responsiveness), and direct performance (e.g., usefulness of information, successfulness and efficiency of task completion), whereas less concerned about indirect factors (e.g., personalization, behavior change, service courtesy, and extended social effects); (2) inter-group divergence in importance perception for some evaluation criteria. The divergence primarily exists between the user and other DL stakeholder groups; (3) some promising criteria (e.g., comprehensiveness of collection, integrity of information, integration of service to information seeking path, collaboration/sharing) augment the existing DL evaluations whereby important criteria have essentially been covered; and (4) most importantly, the core dissertation objective is fulfilled, that is the construction of the holistic evaluation model, in which heterogeneous stakeholders' perspectives at all DL levels are presented. Moreover, some prestigious journals also have devoted special issues to this subject (e.g., 2 Science 1993Science , 2000 Information Processing and Management, 1999; Library Trends, 2000; Communication of ACM, 1995. IEEE Computer, 1995. Journal of American Society for InformationHowever, behind this fast-growing scene are some weaknesses that might hinder the progress of DL innovation. One of the remarkable weaknesses is reflected in the two competing visions of what is a DL, which are pinpointed by Borgman in 1999. One vision, with the research domain (e.g., Fox et al., 1993;Lesk, 1997) Unsurprisingly, the divergent visions of the definition of the DL cannot but yield to different foci in DL development. The view ...

show abstract

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Developing a holistic model for digital library evaluation

Zhang

2009

J. Am. Soc. Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…The objective of most Delphi applications is the reliable and creative exploration of ideas for decision making. The Delphi method is based on a structured process for collecting and distilling knowledge from a group of experts by means of a series of questionnaires interspersed with controlled opinion feedback (Kwak, Jun, Gruenwald & Hong, 2002). For the application of the Delphi method, first, for a test, tentative 31 indicators and scales were created based upon statistics on university libraries and upon the above-mentioned survey results.…”

Section: Delphi Methodsmentioning

confidence: 99%

An Evaluation Model for the National Consortium of Institutional Repositories of Korean Universities

Kim

2006

Proc of Assoc for Info

View full text Add to dashboard Cite

This study, funded by KERIS (The Korean Education and Research InformationService) aims at building an evaluation model for diagnosing and solving problems of the "dCollection" consortium, composed of institutional repositories at more than 40 universities. The evaluation model, as tools not only for a member library's self-evaluation of its own system but also for assessment of the dCollection systems as a whole, is set up in two steps: First, based upon literature review from digital library evaluation and institutional repositories, and case studies on university repositories, the conceptual framework was composed of four categories: content; system and network; uses, users and submitters; and management and policy. And, second, the evaluation framework was modified through three methods, such as a Delphi method, an expert forum and a simulation test, leading to the evaluation model with 4 categories and 34 indicators. IntroductionAccess to academic journals has become more difficult as the prices of academic journals have increased rapidly. Furthermore, the advent of Web-based journals has changed the way institutions subscribe to journals, from choosing individual publications to contracting with publishing companies. These changes have led to a decrease in materials at libraries, resulting in two problems: it becomes more difficult to obtain information, and more difficult to circulate it. Under these circumstances the open access movement started. The movement aims to provide researchers with more access to resources, and authors with wider dissemination of their works, resulting in greater research impact. To realize open access as an alternative to the existing scholarly communications, distributing journals through institutional repositories was proposed.Motivated by the open access movement, academic institutions are building institutional repositories (IRs) for a nationwide knowledge distribution infrastructure. KERIS, the governmental agency which provides various support for university information systems, proposed to organize institutional repositories into a consortium, called "dCollection (Digital Collection)," composed of more than 40 universities since 2003, to give users more access to resources, and to give authors more readers. For this consortium, KERIS also built the repository software platform called "dCollection system," in cooperation with member universities, to freely distribute the platform to the member universities. For the federated searches, KERIS's Research Information Service System, called "RISS," harvests metadata from the dCollection systems operated by the member libraries.The dCollection Promotion Project, funded by KERIS, aims at building an IR evaluation model. In this model the procedural assessment is more emphasized than the actual performance, in order to pinpoint procedural weakness of institutional repositories of each member library and to find their customized solutions, where procedural assessment evaluates current operating and financial processes, while perf...

show abstract