JIVING WITH JENKINSMaking the CI server dance to your tune

JOHN SHEPHERDSON,

ASSOCIATE DIRECTOR, TECHNICAL SERVICES

UK DATA ARCHIVE

17 April 2014 V1.0

DANCE CARD

2

Introductions

Basic steps Hoofing with Hudson Stepping with Selenium Tangoing with TV

Fancy footwork Dream Ballet Sequence Dancing Jiving with Jenkins

UK Data Archive

3

• based at the University of Essex since 1967

• curator of the UK’s largest collection of digital data in the social sciences

• currently holds nearly 6,000 data collections for research and teaching, both quantitative and qualitative

• certified to ISO 27001, the international information security standard

• makes these available via the new UK Data Service

Website: www.data-archive.ac.uk

UK Data Service

4

• the UK Data Service indexes all data collections in the Archive – all catalogued at thematic level

– many indexed at variable level

• also harvests metadata from other sources

• all are available for download via Discover search-and-browse catalogue:discover.ukdataservice.ac.uk

UK Data Service

5

• also provides access to UK Census data (1971 to 2011)• source of guidance, training, and support for data users in

UK and around the world• currently serve approx. 24,000 registered users• newly funded to coordinate the Administrative Data

Research Network, part of UK’s Big Data strategy

Websites: ukdataservice.ac.uk, census.ukdataservice.ac.uk

• led by experts at University of Essex along with colleagues at Manchester, Leeds, Southampton, Edinburgh and UCL

6

BASIC STEPS

HOOFING WITH HUDSON

7

Basic CI stuff• Instrumentation• Unit testing• Style checking• Documentation

Instrumentation

8

Job Status

Build History

Instrumentation

9

Build artefacts

Revision history

Audit trail

Audit trail

Unit Testing

10

Style Checking

11

StylecopCheckStyle

Code QA - Sonar

12

Code QA - Sonar

13

Database docs (Red Gate SQL Doc)

14

Database docs (Red Gate SQL Doc)

15

Class docs (dOxygen)

16

Class docs (dOxygen)

17

STEPPING WITH SELENIUM

18

User Journey assurance

Plugin to Firefox

Allows User clicks to be captured

convert to test scripts

run on demand

Selenium Grid

test multiple browser types and variants

See ‘Browser Watch’ presentation for more details

STEPPING WITH SELENIUM

20

User Journey assurance

TANGOING WITH TV

21

Other QA/Validation: Link checking, Spelling, HTML standards compliance, Accessibility

FANCY FOOTWORK

DREAM BALLET

23

Deployment

• Copy builds, configuration, data from server to server

• Keep test instance data in step with production

How? Wrap a tool (such as RoboCopy or Red Gate deployment manager) in a script (such as Ant) and call it from Jenkins.

SEQUENCE DANCE

24

Upstream/downstream builds

• Build dependency chain

– Harvest data via web service API

– Import harvested data in to XML database

– Run quality checks against database contents

– Extract data etc.

JIVING WITH JENKINS

25

From Continuous Integration to Orchestration

• Minimal coding

– Mostly configuration

– Some scripting (Ant, Shell, DOS etc.)

– Chain internally and/or call externally (REST API)

– Rich instrumentation

– Notification on success and/or failure

Make your CI server dance to your tune!

26

Jira http://www.atlassian.com/software/jira

Jira Agile https://www.atlassian.com/software/jira/agile

SVN http://subversion.apache.org/

SVN Edge http://www.collab.net/products/subversion

Jenkins http://jenkins-ci.org/

Selenium http://docs.seleniumhq.org/

Stylecop http://stylecop.codeplex.com/

Resharper http://www.jetbrains.com/resharper/

Red Gate http://www.red-gate.com/products/

Doxygen http://www.stack.nl/~dimitri/doxygen/

Sonar http://www.sonarsource.org/

Find Out More

27

Find out more Data Archive – data-archive.ac.uk UK Data Service – ukdataservice.ac.uk

Contact Information John Shepherdson

UK Data ArchiveUniversity of Essex

Wivenhoe ParkColchester CO4 3SQE-mail: jwshep@essex.ac.uk