wikipedia. lih
TRANSCRIPT
Wikipedia asParticipatory
Journalism: Reliable Sources?
Metrics for evaluating collaborative media as a news source
Andrew Lih, Assistant ProfessorInternational Symposium on Online Journalism
April 17, 2004
Overview
Participatory journalismWikipedia introductionGrowthMethodologyResults
Participatory Journalism
We Media, Bowman and Willis
The act of a citizen, or group of citizens, playing an active role in the process of collecting, reporting,
analyzing and disseminating news and information. The intent of this participation is to provide independent, reliable, accurate, wide-
ranging and relevant information that a democracy requires.
Forms
Web logs – one to manyWikis – many to many
Wikis
Based on Hawaiian word for quick wikiwikiWeb site where any page can be edited by users (Edit this page button)Registered or anonymous usersWikipedia encyclopedia formatHow could it possibly work?
How could it work?
Neutral point of view (NPOV)“The neutral point of view attempts to present ideas and facts in such a fashion that both supporters and opponents can agree.”
Journalistic decisions
Avoiding the use of word “terrorism,” very much like ReutersIntense debate on naming accuser in Kobe Bryant case
Wikipedia Traffic Rank vs. Other Encyclopedias (2003-2004)
Traf
ficRa
nk
Wikipedia.org
Britannica.com
Encyclopedia.com
500
2000
1000
4000
6000
Apr-03 Jul-03 Oct-03 Jan-04
10,000
12,000
Source: www.alexa.com
How could it work?
Neutral point of view (NPOV)“The neutral point of view attempts to present ideas and facts in such a fashion that both supporters and opponents can agree.”
Research questions
What quantitative metrics for quality?Analyze articles cited in the press
Any patterns or quality level of articles?Effects of citation on article?
Metrics for each article
Rigor - number of editsDiversity - number of unique editorsTaken from the edit history for each pageMetadata only, can be used for any language
Methodology
Establish benchmark by analyzing representative general topicsAssumption that these would be exemplarsMeasure median for those topics in Wikipedia
Edits: 61Unique editors: 36.5
Methodology
Determine metrics for cited articlesExamine movement before citation, after citation
Reputation of Major Encyclopedia Topics in English Wikipedia (Diversity vs. Rigor)
Median
SportBuddhism
Law
History_of_China
Hinduism
World_War_II
Astronomy
Islam
ReligionCanada
PhilosophyMathematics
World_War_I
EarthNative_American
0
100
200
300
400
500
600
700
800
0 50 100 150 200 250 300 350
Diversity (number of unique editors)
Rig
or (n
umbe
r of e
dits
)
Reputation of Major Encyclopedia Topics in Wikipedia(Diversity vs. Rigor) using Logarithmic Scale
Colonial_America
History_of_the_Middle_East
World_War_IIAstronomy
IslamHinduism
History_of_China
Median
1
10
100
1000
1 10 100 1000
Diversity (number of unique editors)
Rig
or (n
umbe
r of e
dits
)
News outlets citing WikipediaAljazeerah.infoArizona RepublicAssociated PressCBC NewsworldDaily Telegraph onlineGreenville NewsGroklawGuardianHouston ChronicleJapanToday.comKansas City StarKnoxville News Sentinel (Tennessee)Loudon Times-MirrorLowell Sun OnlineMetropolisNew Straits TimesNewScientist.comOC WeeklySacramento Bee
Salt Lake TribuneSan Francisco ChronicleSan Jose Mercury NewsSlashdotStar-TelegramSydney Morning HeraldTechTVThe AgeThe Dallas Morning NewsThe Globe and MailThe New Zealand HeraldThe Philadelphia InquirerThe Pocono RecordThe Straits TimesThe Sydney Morning HeraldTime.comVenezuelan News and AnalysisWOAI-TV San Antonio
Wikipedia Articles Cited in Press Jan 2003-March 2004 (Before Citation)Highlighted Articles
X_Window_SystemWho_Wants_to_Be_a_Millionaire
Vietnam_WarList_of_recipes
11_March_2004_Madrid_attacks
Saddam_Hussein
Fuck
Chen_Shui-bianCommunist_Party_of_China
0
100
200
300
400
500
600
700
800
900
1000
0 20 40 60 80 100 120 140 160 180
Diversity (number of unique editors)
Rig
or (n
umbe
r of e
dits
)
Trends
Madrid attacks subject of constant updatesAsian topics have high edits, but low number of contributorsScience and history have high showing
Wikipedia Articles Cited in Press Jan 2003-March 2004 (Before Citation)
1
10
100
1000
10000
1 10 100 1000
Diversity (number of unique editors)
Rig
or (n
umbe
r of e
dits
)
Wikipedia Articles Cited in Press Jan 2003-March 2004 (After Citation)
1
10
100
1000
10000
1 10 100 1000
Diversity (number of unique editors)
Rig
or (n
umbe
r of e
dits
)
Articles Moving Across Benchmark Point
62274522British_honours_system
63362811Suez_Crisis
54383528Leap_year
69385930Explosive_material
65404628Palestinian_Authority
70405330Execution_by_firing_squad
76412712Iain_Duncan_Smith
86413930Alexander_Graham_Bell
60434131Breakdancing
83435423Form_of_government
105446032Australian_dollar
95474716History_of_Iraq
98472717Lyndon_LaRouche
95482015Provisional_Irish_Republican_Army
102515022Weapons_of_mass_destruction
106545229Juliana_of_the_Netherlands
84624133Aramaic_language
2261102413Ahmed_Yassin
EUEEUE
AfterBeforeArticle name
Articles improved, without direct link to current events
Alexander_Graham_BellAustralian_dollarBreakdancingExecution_by_firing_squadExplosive_materialForm_of_governmentLyndon_LaRoucheSuez_Crisis
Conclusions
Simple metrics for rigor and diversity can help identify trendsCitations in press can lead to improvement of articlesIf news is the first draft of history, Wikipedia is a “working draft of history”
Future Work
Content analysis in conjunction with metadata
Detect edit wars, vandalism, which skew resultsAnalyze internal and external hyperlinks
Measure more limited window of timeMore broad benchmark topics
