TCA Scoring Guide
As a critical evaluation tool for Web-based electronic resources, The Charleston Advisor will use a rating system which will score each product based on four elements: content, searchability, price and contract options/ features. A Composite Score averaging these elements will provide an "at a glance" rating which will be prominently displayed near the top of each review.
The rating system will be based on a five-star model (with one star being the lowest and five being the highest). As each of the four elements are rated they should be assigned a ranking with granularity no finer than one-half star (e.g. 3.5 stars). These will then be equally weighted and averaged for an overall Composite Score. It is realized that any ranking system is open to personal opinion and interpretation; however, it is recommended that brief evaluative comments be made following each ranking, especially in cases where a product or service receives less than the highest value in a category.
TCA is meant to be a critical evaluation tool—not just a descriptive review of a product. Our reviewers are tough! They take great care in the ranking section of the review since this will be one of the most visible yet controversial components of TCA. The following brief guidelines will help in understanding the ranking process.
Content
Special attention is paid to the intended audience for which the product is targeted—
does the product meet the intended user needs? The content being compared with competitive
products on the marketplace and any major omissions or special strengths will be factored
into the rating.
Searchability
The user interface and search engine are evaluated in terms of meeting the intended
purposes. Is the product intuitive and easy-to-use? Are advanced searching features
available if the product warrants it? Are graphics and other screen design features in
keeping with the intent of the product and its audience? Is the search engine reliable and
does it provide consistent results? Are there special features, installation requirements,
plug-ins or other special software requirements? If so, is the product easy to use or more
of a nuisance?
Price
The value of the product in relation to its cost must be assessed. A high price alone does
not necessarily mean a low ranking, but the product is evaluated in terms of content, user
interface and value added features. However, vendors who resell duplicative content in
different "packages," with enough difference to force libraries into acquiring
these different packages, may be marked down. Vendors who are flexible (or inflexible) in
their pricing options will be noted.
Contract Options/Features
The contract provisions that accompany a service will be viewed in terms of accepted
national guidelines (e.g. those adopted by major organizations such as the International
Coalition of Library Consortia, Association of Research Libraries, ALA). Factors which
might be considered include: definition of acceptable users, archiving provisions (when
appropriate), lease/ ownership of data, interlibrary loan provisions, redistribution of
information provisions, or other peculiar or interesting issues.
Composite Score
This overall score will be calculated based on an averaging of the above four elements
with equal weighting for each element. The Composite Score has been rounded up to the next
highest quarter star when necessary (e.g. 3.625 stars).
In the case of Opposing Opinions, an Opposing Composite Score will be calculated, yielding a Composite Composite Score. Not all reviews will have an Opposing Opinion.
