IEEE TCDL Bulletin
 
space space

TCDL Bulletin
Volume 3   Issue 2
Summer 2007

 

Analysis of the Bid Behavior of the 2005 JCDL Program Committee

Marko A. Rodriguez
Knowledge & Information Systems Science Team
Los Alamos National Laboratory
Los Alamos, NM 97544
marko@lanl.gov
Johan Bollen, Herbert Van de Sompel
Digital Library Rsearch & Prototyping Team
Los Alamos National Laboratory
Los Alamos, NM 97544
{jbollen, herbertv@lanl.gov
 

This poster will present an analysis of the bid behavior of the 2005 JCDL program committee. For each paper, poster, and tutorial submitted for review to the 2005 JCDL, each program committee member was asked to state their expertise with regards to the subject domain of the submission. Therefore, for each submission, each referee selected one of the four categories:

  1. expert in the domain of the submission and wants to review
  2. expert in the domain of the submission
  3. non-expert in the domain of the submission
  4. conflict of interest between referee and the submission

When ignoring conflict of interest situations, the hypothesis of this study is that referee bidding is based solely on the subject domain of the submission and the area of expertise of the referee. In order to validate or falsify this hypothesis, submission similarity is determined according to a TFIDF and cosine similarity calculation of the terms in their abstracts and then compared against the similarity of submissions as determined by referee bidding. The correlation between these two calculations is positive, though not strong. Therefore, it is concluded that other aspects besides submission subject domain are influencing referee bidding. Next, if referees are bidding with respect to submission subject domain, then referees of similar domain expertise should be bidding in a similar manner. An analysis of the relative expertise of program committee members is computed using a relative-rank algorithm within a co-authorship network. The correlation between referee similarity as determined by their bidding behavior and referee similarity as determined by the relative-rank algorithm is positive, but not strong. Therefore, the hypothesis is again falsified because submission subject domain is not the only factor influencing referee bidding.

This poster presents the various statistical techniques used throughout this study with sufficient diagrams to ensure that the audience understands the methods and conclusions drawn from the study.

Thumbnail image of poster

For a larger view of Figure 1, click here.

 

© Copyright 2007 Marko A. Rodriguez, Johan Bollen and Herbert Van de Sompel
Some or all of these materials were previously published in the Proceedings of the 6th ACM/IEEE-CS Joint Conference on Digital libraries, ACM 1-59593-354-9.

Top | Contents
Previous Article
Next Article
Home | E-mail the Editor