all data big and small - by stephen o'grady

Post on 18-Nov-2014

711 Views

Category:

Technology

1 Downloads

Preview:

Click to see full reader

DESCRIPTION

See conference video - http://www.lucidimagination.com/devzone/events/conferences/revolution/2011 In 2009 The Guardian launched The Open Platform, a suite of services and tools that enable content partners and developers to build applications with The Guardian’s rich content. The content API, hosted on Solr instances on EC2, contains JSON representations of all Guardian articles back to 1999 - over 1 million articles, and is an increasingly complete representation of the output of the organization. The DataStore contains curated data sets for use in applications and virtualizations. This talk will cover how The Guardian opened up their business, enriched it, and reached new markets with its Open Platform strategy. Stephen will cover the technical architecture, implementation of Solr (the key technology powering the platform), and how The Guardian has used it to embrace disruption in the media space, while finding new sources of revenue and innovation.

TRANSCRIPT

10.20.2005

All Data Big and Small

May 2011

2

http://redmonk.com/public/lucene.pdf

3

In the beginning, there was the database...

4

1979 1983 1989

5

When you have a hammerand so on

6

Source: http://www.flickr.com/photos/pagedooley/2234031789/

7

December 29, 2004

8

The Cambrian Non-relationalExplosion

9

Source: http://www.pnas.org/content/97/9/4426/F1.expansion.html

10

11

Why?

12

Different tools for different jobs

13

Or, rather, different data

14

A lot of different data

15

16

Most of the attention goes to Big Data

17

In spite of the fact that comparatively few have it

18

Less heralded is unstructured data

19

20

Between the size and (un)structure, it's amazing anything gets found

21Source: http://www.flickr.com/photos/28705377@N04/4142872268/

22

It's hard to ask the right question

23

To make matters worse, you may only get one chance

24

The most important answeris the next question

25

Some questions

26

27

28

29

30

31

OTHER QUESTIONS

top related