kqed and the npr api

KQED and the NPR API

Hacks & HackersMay 22, 2010

What is the NPR API?

API = Application Programming Interface A structured way for (web) applications to

get NPR stories in a predictable, flexible and powerful way

Includes audio from most NPR programs dating to 1995, and text and images

250,000 stories in 5,000 different aggregations

source – http://www.npr.org/api/

History of the NPR API

Launched on July 16, 2008 Over 250,000 stories from NPR.org Over 400,000 audio files, totaling over 25,000 hours of

programming Features Added in December, 2008

Mix Your Own Podcasts Station Finder API More content: Fresh Air, StoryCorps and Most Emailed stories

Features Added in July, 2009 Transcript API, including over 80,000 transcripts Added over 180,000 more MP3 files dating back to 2001

Features Added in March 2010 API Ingest including OPB, N3, KQED, WBUR and WXPN First blogs added to the API

Who can use the NPR API?

Anyone who registers for an API Key Personal, non-commercial use, or

non-commercial use by a 501(c)(3) See the terms of service for more

details at: http://www.npr.org/api/apiterms.php

Target Audiences

NPR Supports the entire infrastructure of our new site Improvements to CMS to enable custom feeds by Editorial and

Design

Stations Enables us to serve content to our member stations more easily Enables our stations to serve their communities better

Partners Creates new opportunities because - easy to implement with very

little integration cost Makes existing opportunities easier to maintain and grow

Public Engages the community – part of NPR’s Public Service Mission Lots of great widgets, tools, sites built using the API

The NPR API Suite

APIs Offered Full Story API Station Finder and Information API Transcript API

Content Virtually everything on NPR.org is available in the API (rights restricted) Over 6500 categories Content from dozens of NPR stations, including KQED

Content Types Full text Full transcripts Audio (mp3, m3u, aac, wma, rm) Images Virtually all other assets and fields on NPR.org

Output Formats Eight different output formats

NPRML, our custom, full content XML format RSS, MediaRSS and PodcastRSS JSON Atom HTML and JavaScript Widgets

How does it work?

Send a query to a URL, it returns data in a structured format

Default format is NPRML, a custom XML specification

Also supports RSS, MediaRSS, JSON, Atom

Easiest – HTML and Javascript widgets

Example:

http://api.npr.org/query?id=1149&apiKey=demo

Returns an NPRML document with the latest stories on Afghanistan

See all topics: http://api.npr.org/list?id=3002

Can also query by genre, program, reporter, music artist, series, blogs, columns, categories & tags

Query generator (demo?)

NPR API Demo – query generator

How KQED uses the NPR API

We ingest all top news stories into our MySQL database and publish them automatically to our home page and news section: http://www.kqed.org and http://www.kqed.org/news/

Alternative workflow – editors choose which movie reviews to publish in our Arts section: http://www.kqed.org/arts/movies/

More coming soon

KQED implementation – tech details

Java class that uses Jaxb to de-serialize NPRML data

Persists to our database using Hibernate (Object-relational mapping layer)

Per-feed workflow changes (ie., editors can choose to auto-publish, or publish exactly what they wish)

All stories fully editable in our CMS (“SimpleCMS”)

KQED example output

New stuff – API ingest

For member stations, the API is two-way, allowing us to publish our data to NPR’s database

Current feeds that are live (all wordpress blogs, via MediaRSS): Climate Watch Shifting Gears Capital Notes

NPR API Growth

NPR API : Monthly Growth of API Requests

0

10

20

30

40

50

60

Nov-08

Dec-08

Jan-09 Feb-09

Mar-09

Apr-09 May-09

Jun-09 Jul-09 Aug-09

Sep-09

Oct-09 Nov-09

Dec-09

Jan-10 Feb-10

Mar-10

Months

Re

qu

es

ts i

n M

illi

on

s

Output format distribution

NPR API : Distribution of Output Formats

PodcastRSS

JSON

Atom

MediaRSS

RSS

HTML

JavaScript

NPRML

Q&A / Demo of CMS

Ken Murphy is the primary Java developer – kmurphy@kqed.org

Daniel Jacobson – NPR tech lead: djacobson@npr.org

Me: kcooke@kqed.org or @kic00 on twitter