digging into the deep web

Dig into the Deep Dig into the Deep WebWeb

Going on a treasure hunt…Going on a treasure hunt…Wendy DeGroatWendy DeGroat

MW Library, November 2009MW Library, November 2009B

y

cam

ero

np

ark

ins

How do sources get buried?How do sources get buried?

► Requires log-onRequires log-on► Accessed by queryAccessed by query► Protected from robotsProtected from robots► Public access expiredPublic access expired► Can’t be bookmarked Can’t be bookmarked

(scripted, dynamic, (scripted, dynamic, etc.)etc.)

► More than 3 clicks More than 3 clicks deepdeep

(He, et al., 2007)

How big is the deep web?How big is the deep web?

► Conflicting answers…Conflicting answers… 91,000 terabytes 91,000 terabytes (Deep Web, n.d.)(Deep Web, n.d.)

500x larger than surface 500x larger than surface (Sullivan as cited in He, et al., (Sullivan as cited in He, et al., 2007)2007)

63% 63% (He, et al., 2007)(He, et al., 2007) - 95% of web - 95% of web(D

ash

, 20

07)

What’s in the treasure chest?What’s in the treasure chest?

► Images and multimedia files, including the Images and multimedia files, including the content of many online exhibitscontent of many online exhibits

►Diaries, letters, historic mapsDiaries, letters, historic maps► Patents, laws, bills, and treatiesPatents, laws, bills, and treaties► Articles licensed/protected by copyrightArticles licensed/protected by copyright► ““Old” newspaper articles (esp. pre-1980’s)Old” newspaper articles (esp. pre-1980’s)► Collections of papers, preprints (conference, Collections of papers, preprints (conference,

etc.)etc.)►Declassified documents, Congressional Declassified documents, Congressional

testimonytestimony►Government reports, data setsGovernment reports, data sets► And much more…And much more…

Strategies for finding an Strategies for finding an X, and then digging X, and then digging

for…for…

By merfam

Why search for it?Why search for it?

By brittanyg

Treasure HuntTreasure Hunt

►MappingMapping►PlanningPlanning►Selecting the proper toolsSelecting the proper tools►DiggingDigging

(Concept) mapping(Concept) mapping

►Browse for broader conceptsBrowse for broader concepts Categories of web directoriesCategories of web directories Sections of archivesSections of archives Subject headings in databasesSubject headings in databases

►Note related/narrower conceptsNote related/narrower concepts►Amend your map as you searchAmend your map as you search

Preparing for the journeyPreparing for the journey

►Reference book articlesReference book articles PowerSearch – Books tabPowerSearch – Books tab U.S. and World History RC’s – Reference U.S. and World History RC’s – Reference

tabtab Biography, Science, and Literature RC’sBiography, Science, and Literature RC’s Google Book/WorldcatGoogle Book/Worldcat NetLibrary NetLibrary (note: not all reference, some (note: not all reference, some

biased)biased)

Preparing for the journeyPreparing for the journey

►Finding reference sources, overviews, etc.Finding reference sources, overviews, etc. Recommended sources from Wikipedia/EBRecommended sources from Wikipedia/EB Literature reviewsLiterature reviews Reference listsReference lists Further/recommended reading listsFurther/recommended reading lists Briefs/reports from Congressional Research Briefs/reports from Congressional Research

Service, State Department, think tanks, etc.Service, State Department, think tanks, etc.►Finding compilations of sourcesFinding compilations of sources

Search subject + Search subject + bibliography, research guide, bibliography, research guide, resource guide, study guide, handbook…resource guide, study guide, handbook…

Google Book

Tools for the digTools for the dig

Digging tipsDigging tips

►GaleGale Reference sources Reference sources

►Search by broad conceptSearch by broad concept►See Reference / Books result tabsSee Reference / Books result tabs►Refine by document typeRefine by document type►Go beyond PowerSearchGo beyond PowerSearch

Follow leads within and between databasesFollow leads within and between databases Journal/mag/news articles & primary sourcesJournal/mag/news articles & primary sources

►Start broad, then narrowStart broad, then narrow►Use Advanced search to combine keywords and Use Advanced search to combine keywords and

subject headingssubject headings


► JSTOR and Project MUSEJSTOR and Project MUSE Use Advanced SearchUse Advanced Search In general, limit to articlesIn general, limit to articles If looking for books, limit to reviewsIf looking for books, limit to reviews JSTORJSTOR

►Refine by limiting by discipline and/or date rangeRefine by limiting by discipline and/or date range

Project MUSEProject MUSE►Refine by combining key word and subject Refine by combining key word and subject

searchingsearching►May also refine by limiting to May also refine by limiting to All Except TextAll Except Text


►LexisNexisLexisNexis Supreme court decisionsSupreme court decisions LawsLaws NewsNews

►GeneralGeneral►RegionalRegional►Foreign news sourcesForeign news sources►WiresWires►TranscriptsTranscripts


► VCU databasesVCU databases Browse by topicBrowse by topic Search by journal title or article citationSearch by journal title or article citation Follow to find in other databasesFollow to find in other databases Not in a database? Check online catalog for Not in a database? Check online catalog for

copies in print, and on microfilm or copies in print, and on microfilm or microfichemicrofiche

► VCU is also a government repositoryVCU is also a government repository► VCU library’s resource guidesVCU library’s resource guides


►Federated search toolsFederated search tools e.g. Google Scholar, USAsearch.gove.g. Google Scholar, USAsearch.gov Finding tools = Finding tools = portalsportals in library’s bookmarks in library’s bookmarks

►DirectoriesDirectories e.g. Infomine, Intutee.g. Infomine, Intute Finding tools = Finding tools = directoriesdirectories in library’s bookmarks in library’s bookmarks

►Metasearch enginesMetasearch engines e.g. Clusty, Dogpilee.g. Clusty, Dogpile Finding tools = Finding tools = metasearchmetasearch in library’s in library’s

bookmarksbookmarks


►Search for search toolsSearch for search tools Finding tools bundle in MW Library’s Finding tools bundle in MW Library’s

bookmarks = bookmarks = portals, directories, portals, directories, metasearchmetasearch

Search subject + Search subject + directory, portaldirectory, portal, or , or searchsearch

►Social bookmarking sitesSocial bookmarking sites del.icio.usdel.icio.us diggdigg stumbleuponstumbleupon

Choose the right GoogleChoose the right Google

►Google BookGoogle Book►Google ScholarGoogle Scholar►Google News & Google News & News ArchiveNews Archive

Save customizationsSave customizations Specify news source locationSpecify news source location Available for other countriesAvailable for other countries Set up news alertsSet up news alerts

►Google by countryGoogle by country►Google custom search Google custom search

engineengine

By shawnbot

Surveying the landscapeSurveying the landscape

►Who cares about this topic?Who cares about this topic?►Where do these people live/gather?Where do these people live/gather?

Research centersResearch centers Regulating/monitoring agenciesRegulating/monitoring agencies Professional/stakeholder organizationsProfessional/stakeholder organizations

►Where would their work be collected?Where would their work be collected? Key publications, conferences, databasesKey publications, conferences, databases Historical societies, museums, galleriesHistorical societies, museums, galleries

Staying on trackStaying on track

►Stop periodicallyStop periodically See where you areSee where you are Reflect on what you’ve found/learnedReflect on what you’ve found/learned Decide where to look nextDecide where to look next

►Follow leadsFollow leads►Amend your (concept) mapAmend your (concept) map►Take notes (interacting vs. pasting)Take notes (interacting vs. pasting)►Document your sourcesDocument your sources

By ninjaneil902

What questions do you have?What questions do you have?

►MW Library web siteMW Library web site http://mwlibrary.wordpress.com (or select http://mwlibrary.wordpress.com (or select

Media Center on school’s home page)Media Center on school’s home page) Navigation menus at top and on leftNavigation menus at top and on left

►MW Library bookmark accountMW Library bookmark account http://del.icio.us/dragonlibrary/http://del.icio.us/dragonlibrary/

wilkes_koreanwarwilkes_koreanwar

►MW Library research wikiMW Library research wiki http://mwlibrary.wetpaint.comhttp://mwlibrary.wetpaint.com See See Dig into the Deep WebDig into the Deep Web

By brittanyg

ReferencesReferences

Dash, R. (2007, April 24). Exposing the invisible web to search engines. Dash, R. (2007, April 24). Exposing the invisible web to search engines. Search Search

Engine JournalEngine Journal. Retrieved March 8, 2008, from . Retrieved March 8, 2008, from

http://www.searchenginejournal.com/exposing-the-invisible-web-to-search-http://www.searchenginejournal.com/exposing-the-invisible-web-to-search-

engines/4771/engines/4771/

Deep web. (n.d.). Deep web. (n.d.). WikipediaWikipedia. Retrieved March 8, 2008, from . Retrieved March 8, 2008, from

http://en.wikipedia.org/wiki/Deep_webhttp://en.wikipedia.org/wiki/Deep_web

He, B., Patel, M., Zhang, Z., & Chang, K. (2007). Accessing the deep web. He, B., Patel, M., Zhang, Z., & Chang, K. (2007). Accessing the deep web.

Communications of the ACMCommunications of the ACM 50 50 (5),(5), 94-101. Retrieved March 8, 2008, from ACM 94-101. Retrieved March 8, 2008, from ACM

Digital Library.Digital Library.

digging into the deep web

Technology