stabiexit: approaching microservices and user …8bit-inferno.de/stabiexit.pdfstabiexit: approaching...

StabiEXIT: Approaching Microservices and User-Centered Agile Development in Libraries

Copenhagen, June 7th 2016!!Dr.-Ing. David Zellhöfer!Head of Unit "Applications and Data Management of the Library Core Processes" at Berlin State Library (Staatsbibliothek zu Berlin)!

17.08.2011 Titel der Präsentation Seite 2

Located in two architectural landmarks��11 million prints�1,1 million maps, atlases, globes�480.000 manuscripts and autographs�450.000 digital objects��100.000 media acquisitions/year�50.000 subscriptions (printed and online)�

The Berlin State Library /Staatsbibliothek zu Berlin!


The Berlin State Library /Staatsbibliothek zu Berlin!

50.000 registered users

1,7 million loans per year/ 5.000 per day

>800 employees

Major Cataloging and Research Systems since 1997!

2015

1997

1998

1999

2000

20012002

20032004

20052006

20072008

20092010

20112012

2013

IKARAltkartendatenbank

Einband-

datenbank

Kallisto

Kalliope Drucke des

16. Jhdts.

Berlin-

ansichten

Bibliographia

Cartographica

Orient-Digital

Gesamtkatalog

der Wiegendrucke

Typenrepertorium

Manuscripta

Mediaevalia

Kalliope OPAC

Ostasienkatalog (Erfassung)

LeihverkehrsdatenbankTopoListe

Lieferantendatenbank

2016Muscat

VD Lied

Druckermarken-

Wiki

GBVDrucke des

17. Jhdts.

Musik-IPACs

ZEFYS

GBV-Katalog.

stabikat

(OPAC)

First Appearance of Google!

stabikat+

(Discovery)

GBVGBV

Nationallizenzen…

2014

Development has been driven by librarians and researchers alone because qualified IT staff was not available due to different reasons��Development of IT-supported library services was often funded with the help of project funding provided by research funding agencies��Functional additions were added constantly if funding or staff was available to satisfy current project needs��Sustainability and maintainability of the created monolithic software systems was often neglected� �

The Typical Genesis of !IT-Supported Library Services!

!The Result!

complex applications, which are hardly maintainable and very complex to extend

needed innovations for the supported scientific services can no longer be implemented in a short period of time

Getty Images; http://www.spiegel.de/fotostrecke/vom-alten-rom-bis-heute-die-macht-des-wetters-fotostrecke-63529-6.html

The resulting innovation bottleneck yields disappointed researchers, librarians, and IT staff.


Berlin, 17.08.2011

HIER STEHT EIN EIN- BIS!ZWEIZEILIGER PRÄSENTATIONSTITEL.!EIN MÖGLICHER UNTERTITEL.!

The Department of Information and Data Management (IDM)

Human Resources Development Acquisition Office IT Security IT User Helpdesk

Typical Service Development Interactions!

Sekretariat Stab

IDM 1 Infrastructures of Hard- and

Software

IDM 2"Applications and Data Management of the

Library Core Processes

IDM 3"Innovation Management and

Online Services

„Traditional“ Library Departments

Manuscript Department

Acquisition Department

East Asia �Department

Newspaper Department ...

Existent Services Evolution of Services Third-Party Projects

!Key Challenges!

1. Cooperation with „Traditional“ Library Departments

2. Creation of a Sustainable Software Architecture

„Because the library has become software, it is no longer viable for our services to exist separately from our software.“��„Most importantly, all library staff must understand that our software is our library, and is everyone’s responsibility.“��Opinion: Libraries are Software - Cody Hanson (CC BY 3.0); http://codyhanson.com/writing/software.html

!Why to Improve Cooperation?!

!How to Improve Cooperation?!

https://ww

w.maxxor.com

/software-development-process

Why to Create a Sustainable Software Architecture?!

•  To leverage innovation•  To limit the complexity of (hidden) software dependencies•  To shift maintenance effort from developers to

administrators/DevOps•  To support varying resource demand•  To enable infrastructure maintenance without service

downtime•  To allow a standardized, centralized logging and

monitoring•  To support the co-existence of different programming

languages and software architectures

How to Create a Sustainable Software Architecture?!

•  Evaluation of the microservice pattern•  Migration of a legacy application•  Proof of concept of a microservice platform infrastructure

•  A microservice is “an approach to developing a single application as a suite of small service, each running in its own process […]“ (Martin Fowler, thoughtworks)

•  Evolution of modular programming and software as a service•  Lightweight communication via HTTP•  Programming language-agnostic

!Microservice Architecture!

!Microservice Architecture!

Team A

Team B Team C

Team D

Continuous Deployment /�Continuous Integration Server

Microservice Team

Build automation,Testing, and Deployment

!Platform Services!

•  Service Registry allows the dynamic discovery of microservices to...

•  Provide failsafe operation•  Facilitate load balancing•  Abstract from the infrastructure

•  Platform services also provide…•  Load balancing•  Monitoring•  Logging•  Configuration management

Load Balancing – !A Sample Scenario!

DMS

DMS

DMS

Sidecar JMeter

Operations: PUT,

UPDATE, GET, DELETE

Load Balancing – !A Sample Scenario!

Monitoring and!Centralized Logging!

Nagios, Turbine, Hystrix

But is the grass greener on a field of microservices?

Some Challenges Introduced by Microservices!

•  Programming teams have to be trained•  Network-based communication increases programming

overhead and potential latency problems•  Increased infrastructure demands due to CI/CD servers,

platform services, service registry…

•  Problem to determine the set of microservices•  “as small as possible but as big as necessary to

represent the domain concept they own”•  development teams have to have the big picture of the

whole library in mind

!Advantages of Microservices!

•  Less complexity per microservice•  Limitation of architectural erosion risk•  Enables automation of development, test, and

deployment tasks•  Standardization of maintenance tasks•  Legacy applications can be included into the Platform

Service Framework

•  Subsequent re-use of extended microservices!

The Department of Information and Data Management (IDM)

Human Resources Development Acquisition Office IT Security IT User Helpdesk

Typical Service Development Interactions!

Sekretariat Stab

IDM 1 Infrastructures of Hard- and

Software

IDM 2"Applications and Data Management of the

Library Core Processes

IDM 3"Innovation Management and

Online Services

„Traditional“ Library Departments

Manuscript Department

Acquisition Department

East Asia �Department

Newspaper Department ...

Existent Services Evolution of Services Third-Party Projects

Microservices as a Benefit for the Organization!

Existent Services

Evolution of

Services

Third-Party

Projects

http://www.pngall.com/grass-png

!Where Are We Heading Next?!

•  Evaluation of Frontend Integrations Concepts•  Typo3•  JavaScript

•  Microservice Infrastructure Introduction Concepts•  Service Registry•  Monitoring•  Centralized Logging

•  Standards for Continuous Integration and Deployment

My thanks go to our team at the library!! Malgorzata Asch, Constantĳn Blondel, Piotr Czarnecki, Konrad Eichstädt, Thomas Fries, Konstantin Görlitz, Steffi Gräber, Gerrit Gragert, Tim Jabs, Jens Ludwig, Astrid Meyer, Marco Remy, Marco Scheidhuber, Oliver Schöner and many more...

Thank you.! Do not hesitate to contact me. David Zellhöfer [email protected] @dzellhf Technical information in German: http://blog.sbb.berlin/author/ke/

stabiexit: approaching microservices and user …8bit-inferno.de/stabiexit.pdfstabiexit: approaching...

Documents