evaluation: preliminary results from the server side
Post on 06-Jan-2016
27 Views
Preview:
DESCRIPTION
TRANSCRIPT
Evaluation: Preliminary Results from the Server Side
Frank A. SettleElizabeth Blackmer
Thomas Whaley
The Alsos Digital Library for Nuclear IssuesWashington & Lee University
Lexington, Va
User
ServerServer log data
Who? What?How? Why?
Evaluation from the Server Side
Preliminary Observations from the Server Side
• Robots, crawlers, and other automated searches produce over 50% of the visits. While they inflate the number of visitors, they do produce human visits indirectly.
• Search engines (Google and Yahoo) provide the majority of referrals.
• Sites that have embedded, directed links to specific bibliographical information provide over 10% of the referrals. • Referrals from the NSDL portal as well as other sites that include Alsos in their list constitute a small percentage of referrals. However, users may be using a search engine to follow up on an Alsos link they saw in a list on another site.
Month Total visitors
(including robots, etc.)
Visitors from top referrers
per month
Prints
per month
Estimated human visitors per day
Visitors
from NSDL portal
Per month
September 2002
1696 na 410 29 na
March 2003
2619 na 364 44 na
May 2003 2008 na 312 33 na
June 2003 1455 na 260 23 na
August
2003
2489 914 505 43 5
September
2003
4810 1474 1500 89 18
Overall Usage - September 2002 – September 2003
Top Referral Sites – September 2003
Visitors •www.google.com 687• search.yahoo.com 289• chemcases.org 227• search.msn.com 138• www.google.ca 54• aolsearch.aol.com 39• www.atomicarchive.com 38
Embedded Link from Atomic Archive to Alsos
Preliminary Observations from the Server Side
• Printing (hardcopy or files) indicates real engagement of the user. However there are also serious users who do not print so this criteria for estimating engaged users is low.
• Almost all of the visitors listed as referrals from other sites are human users, not robots, crawlers, etc.
• Based on a typical day (Tuesday, September 30, 2003) a large number of users (~72%) who are referred directly to an annotation by another site, print the annotation and exit Alsos immediately.
• User access to the six general browse areas (Issues, Warfare, etc.) indicates an equal distribution of interests. This validates the selection of these areas and shows the broad interests of users.
No 1 6 36 43
72%
Yes 4 6
7 17
28%
Total 5
8%
12
20%
43
72%
60
Did user move beyond printing the annotation? Homepage Search Annotation
Entry Point
Engaged Users’ BehaviorTuesday, Sept. 30, 2003
Distribution of Browse Topics for September 2003(human visitors)
Home Page
Browse Issues Area
Results for Nuclear Waste
Complete Citation with Annotation
Processed Information from Server
• Tabular session displays – summary information for each session
• Individual session display – tracks the user through a session
• Suffix statistics – provides information on types and also locations of international users
• Hit counts for individual references – skewed by robots and multiple hits during a single session
Processed Server Log Data
Problem – hits are skewed by robots, crawlers, etc.
Types of Users
• Robots, crawlers, etc.
• Humans who scan topics, site information, etc., but don’t access specific annotations
• Single reference users referred to Alsos from a search engine or website to a specific annotation. User gets information and exits. Often prints.
• Human browsers who go to a browse area, obtain a list of references, and then examine specific annotations. May or may not print.
• Human searchers who enter keywords(s), title, or creator to trigger a search. Most users in this category do not use the advanced search feature.
Single reference user from Google
Searcher enters keyword “Iran”
Browser in People Area
Casual Visitor – Scans Information
Summary
• Massive amounts of raw server log data must be transformed into smaller, organized sets of information.
• This information can provide useful information on who is using the collection and how they use it.
• This information is useful in improving the utility of the collection and the effectiveness of the user interface.
• This information is critical in seeking funds to sustain and expand the collection
top related