Analysis of the Bid Behavior of the 2005 JCDL Program Committee
This poster will present an analysis of the bid behavior of the 2005 JCDL program committee. For each paper, poster, and tutorial submitted for review to the 2005 JCDL, each program committee member was asked to state their expertise with regards to the subject domain of the submission. Therefore, for each submission, each referee selected one of the four categories:
When ignoring conflict of interest situations, the hypothesis of this study is that referee bidding is based solely on the subject domain of the submission and the area of expertise of the referee. In order to validate or falsify this hypothesis, submission similarity is determined according to a TFIDF and cosine similarity calculation of the terms in their abstracts and then compared against the similarity of submissions as determined by referee bidding. The correlation between these two calculations is positive, though not strong. Therefore, it is concluded that other aspects besides submission subject domain are influencing referee bidding. Next, if referees are bidding with respect to submission subject domain, then referees of similar domain expertise should be bidding in a similar manner. An analysis of the relative expertise of program committee members is computed using a relative-rank algorithm within a co-authorship network. The correlation between referee similarity as determined by their bidding behavior and referee similarity as determined by the relative-rank algorithm is positive, but not strong. Therefore, the hypothesis is again falsified because submission subject domain is not the only factor influencing referee bidding.
This poster presents the various statistical techniques used throughout this study with sufficient diagrams to ensure that the audience understands the methods and conclusions drawn from the study.
© Copyright 2007 Marko A. Rodriguez, Johan Bollen and Herbert Van de Sompel