rss feeds using millennium data
DESCRIPTION
I presented this talk on creating RSS feeds at the European Innovative Users Group (EIUG) 2010 conference held at Aston University, 15-16 June 2010. I describe a method of exporting and reusing metadata held in the Innovative Millennium LMS that enables reuse of the data and presentation as an RSS feed - in this case new books lists. This is achieved using Free / Open Sources software. I explain how the process can be generalised to export of other bibliographic data, for example to export reading lists information to a VLE (BlackBoard) as XML, or presenting lists of e-resources on a Web site using a PHP front-end.TRANSCRIPT
![Page 1: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/1.jpg)
www.london.ac.uk
RSS feeds using Millennium data
Andrew Preater, University of London Research Library Services
Presented at EIUG 2010, 15th June 2010
![Page 2: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/2.jpg)
A short break in County Durham
I work for University of London Research Library Services, at Senate House
But I will talk about my previous development work at Durham University Library
![Page 3: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/3.jpg)
Introduction
The problem is the new books list
We use these to list new items for readers as a current awareness tool
Various ways to do this...
![Page 4: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/4.jpg)
A traditional approach
![Page 5: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/5.jpg)
Durham’s new books featured list
![Page 6: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/6.jpg)
Problems
High maintenance
Not split by subject; not easily ‘mashable’
Usage next to nothing by 2007-08
10 hits!
![Page 7: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/7.jpg)
Puts our metadata where the reader is
Much less work for library staff
Standards-based XML data, can be reused elsewhere or mashed up
RSS feed improvements
RSS feed icon from www.feedicons.com
![Page 8: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/8.jpg)
Low-risk pilot for automated export and processing of Millennium data
Demonstrates this utility for future projects
Quickest and easiest example using this approach
Project as proof of concept
![Page 9: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/9.jpg)
Automated as much as possible
Minimal effort by non-systems staff to maintain
No special software – no budget!
Stable and reliable, ‘just works’
Desired outcomes
![Page 10: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/10.jpg)
Other than Millennium...
1. Linux server with Perl installed
2. MySQL database
3. Web server running PHP
Software used
![Page 11: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/11.jpg)
A featured list was created each week based on changing book item status to ‘d’
So a ‘new books’ review file was being made...
Basic idea
New step added: export the contents of the review file and reuse it
![Page 12: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/12.jpg)
BIB MARC 245 $aBIB MARC 245BIB AUTHORBIB IMPRINTBIB SUBJECTBIB RECORD #ITEMFUND CODEITEMSHELFMARKITEMLOCATION
Export these fields
![Page 13: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/13.jpg)
"Dead white men and other important people :"~"Dead white men and other important people : sociology's big ideas / Ralph Fevre and Angus Bancroft."~"Fevre, Ralph, 1955-"~"Basingstoke : Palgrave Macmillan, 2010."~"Social sciences -- Philosophy.";"Sociology."~"b25978974"~"bgsoc"~"300.1 FEV"~"main4"
Example single item
![Page 14: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/14.jpg)
Perl script run every 15 minutes by cron:
1. Checks if there is a new file
2. Processes the data
3. Loads it into a MySQL database
4. Cleans up
Processing this list
![Page 15: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/15.jpg)
1. Replace & with &
2. Insert RFC822-compliant date
3. Strip quotation marks around fields
4. Strip trailing non-alphanumeric character in 245 $a
5. Lowercase fund codes
Step 2: tidying up the data
![Page 16: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/16.jpg)
Step 2: example single item
|Dead white men and other important people|Dead white men and other important people : sociology's big ideas / Ralph Fevre and Angus Bancroft.|Fevre, Ralph, 1955-|Basingstoke : Palgrave Macmillan, 2010.|Social sciences -- Philosophy.";"Sociology.|b25978974|bgsoc|300.1 FEV|main4|Mon, 07 Jun 10 12:31:01 BST|Mon, 07 Jun 10 12:31:01 BST|
![Page 17: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/17.jpg)
Step 2: example single itemDead white men and other important people 245$aDead white men and other important people : sociology's big ideas / Ralph Fevre and Angus245 Bancroft.Fevre, Ralph, 1955- AuthorBasingstoke : Palgrave Macmillan, 2010. ImprintSocial sciences -- Philosophy.";"Sociology. Subjectb25978974 Record #bgsoc Fund code300.1 FEVShelfmarkmain4 LocationMon, 07 Jun 10 12:31:01 BST Date
![Page 18: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/18.jpg)
Two tables are used:
items is refresh weekly: contains our books information
fundmap maps Millennium fund codes to subjects. Export is automated but doesn’t need to run weekly
Database
![Page 19: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/19.jpg)
foobar
fundmap exampledeptcode
fundcode deptname site
ECON bcecoEconomics & Finance
DURHAM
HIST bchis History DURHAM
MEIS bbcmeGovt & Intl Affairs/IMEIS
DURHAM
MEIS bxabcGovt & Intl Affairs/IMEIS
DURHAM
CTV ctvl1Trevelyan College Library
DURHAM
![Page 20: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/20.jpg)
Web front end
PHP script hosted on IT Service Web server will serve the feeds
http://www.dur.ac.uk/reading.list/newitems.php?dept=HIST
Parameter is ‘all’ or a subject code
![Page 21: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/21.jpg)
1. Select items from database
2. Writes beautiful, valid RSS
3. Serves it up to the browser
What it does
A bit more detail...
![Page 22: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/22.jpg)
Generating RSS feed XMLWrite <title>, <description> <link>, <image> once
Item <description> author, shelfmark and subjects hyperlinked to catalogue search.
Item <title> is 245 $a and links to catalogue bib record
Item <description> contains full item data. Can include encoded HTML
For each database line, write one RSS news <item> ...
![Page 23: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/23.jpg)
Finished product - I
Shown in Akregator feed reader
Running happily since August 2007
![Page 24: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/24.jpg)
Finished product - II
HTML version of RSS feeds on Library Web site
Also: in-house PC screensavers, plasma displays...
![Page 25: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/25.jpg)
SummaryMillennium review file
Exported flat file
Display with Web front end
Process and load into database
![Page 26: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/26.jpg)
Easiest to use Unicode everywhere
Write valid RSS 2.0 or Atom, use http://feedvalidator.org for hints
Few complaints; change uncovered a tiny hard core of featured lists fans
Lessons learned
That said...
![Page 27: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/27.jpg)
You can automate much of it with Expect or AutoIt
Recommend Marc Dahl’s presentation on Expect for Innopac: http://bit.ly/dahl-expect
“Couldn’t you automate this?”
![Page 28: RSS feeds using Millennium data](https://reader038.vdocuments.mx/reader038/viewer/2022102814/549350acb47959604d8b47d9/html5/thumbnails/28.jpg)
Automated export and processing used for:
Exporting Course Reserves to Blackboard
Display of e-resources data in CMS
Sending fines data to Oracle Financials
Following on from this...