stored procedure as a service
TRANSCRIPT
Stored Procedureas a
Service (SPaaS)
t @abhishektiwari w https://abhishek-tiwari.com
• solutions manager apac @ Isentia• leading media intelligence company• data and product engineering• data processing with NLP & search focus
t @abhishektiwari w https://abhishek-tiwari.com
Bit ofcontext
Wolf in sheep's clothing
• a data pipeline built using legacy technologies• database as queue, ETL using StoredProc• trigger calling StoredProc calling trigger• stateful, strongly coupled, monolith databases
t @abhishektiwari w https://abhishek-tiwari.com
• thin applications fat StoredProc VLDB• more than 5000 StoredProc written over 10Y• deliver new products/services in a short span• need to move fast - continuous delivery
Ourchallenge
t @abhishektiwari w https://abhishek-tiwari.com
subroutine stored in the database data dictionary
StoredProc
Application(VC) DatabaseClientUIInBrowser
StoredProc&
UDFs
t @abhishektiwari w https://abhishek-tiwari.com
StoredProc
good idea that quickly turned into massive technical debt
t @abhishektiwari w https://abhishek-tiwari.com
Moving data is harder than moving logic
StoredProc
t @abhishektiwari w https://abhishek-tiwari.com
Good Bad Ugly
StoredProcConcerns
• improved performance• security and access control
• highly procedural• limited constructs• can't pass objects• vendor lock-in/specific• maintainability
• zillion parameters• business logic• versioning & testing• debugging & logging• continuous delivery• cross-database StoredProc
t @abhishektiwari w https://abhishek-tiwari.com
Phased Approach
t @abhishektiwari w https://abhishek-tiwari.com
Ecosystem of MicroservicesDecouple UI and Backend Ecosystem of APIs
Separate UI layer from the backend layer
t @abhishektiwari w https://abhishek-tiwari.com
ClientUIInBrowser
APIs
Static Stateless UI(Angular/React)
Delivered via CDN
w https://example.com
w https://api.example.com
MobileApps
Convert legacy backend layer into ecosystem of APIs
t @abhishektiwari w https://abhishek-tiwari.com
Ecosystem of APIson top of Databases
Instant CRUD APIs without code
LoopBack Script Logic
DreamFactory Script Logic
Roll your own APIs
FaaS
Authentication & Authorization Setup
Script Logic
SPaaS Authentication & Authorization Setup
decision tree
SPaaS
t @abhishektiwari w https://abhishek-tiwari.com
APIGatewayClientUI APIs
(Node.js)w https://api.example.com
DatabaseStoredProc
&UDFs
MSSQL
StoredProc
Node.js
Express.js mssql
Middleware RouteHandler
connect
execute
TDSdriver
TDSProtocol
UDFs
HTTPRequest
/v1/coverage/today/50
HTTPResponse
t @abhishektiwari w https://abhishek-tiwari.com
Promises, StreamsorCallbacks
ConnectionPooling
Wiring up StoredProc
Execute
SQL2JSON
Variation of FaaS
• self-contained piece of reusable functionality• can be executed by events or API endpoints• read and write to database backend• can be written in multiple languages
t @abhishektiwari w https://abhishek-tiwari.com
SimilarConcerns
• vendor lock-in or specific implementations• tooling for continuous integration & delivery• monitoring, logging and debugging• testing, discovery, security and latency
t @abhishektiwari w https://abhishek-tiwari.com
SPaaS API Delivery
DevelopAPIs
Changecommitted
Build&Test
PromotetoUAT
PromotetoProd
UATServers
ProdServers
Artifacts
t @abhishektiwari w https://abhishek-tiwari.com
SPaaS SQL Delivery
StoredProcChanges
Changecommitted
Build&Test
PromotetoUAT
SchemaChanges
Changecommitted
PromotetoProd
DatabaseUAT
DatabaseProd
Artifacts
SQLServer
Managem
entStudio
t @abhishektiwari w https://abhishek-tiwari.com
Redgate SQL Compare
DatabaseINT
DatabaseDEV
Redgate DLM Automation
• mock API request & response using Swagger• mock environments as first-class citizen• reusable API tests – contract, load, security• tooling interoperability using Swagger
Mockfirst
t @abhishektiwari w https://abhishek-tiwari.com
Breaking Monolith
• strict rule - no new StoredProc • separate business logic & data logic• build microservices – one at a time• each microservice owns it’s own data store
t @abhishektiwari w https://abhishek-tiwari.com
APIGatewayClientUI APIs
w https://api.example.comDatabase
StoredProc&
UDFs
ServiceA DB-A
ServiceB DB-B
ServiceX DB-X
CDC
Transition
t @abhishektiwari w https://abhishek-tiwari.com
APIGatewayClientUI APIs
w https://api.example.com
ServiceD DB-D
ServiceE DB-E
ServiceX DB-X
CDC
Future
t @abhishektiwari w https://abhishek-tiwari.com
ServiceC DB-C
ServiceB DB-B
ServiceA DB-A
DrivingOutcomes
• more than 200 APIs in less than 6 months• two new native mobile products• a new responsive web product • seamless opportunities – API economy
t @abhishektiwari w https://abhishek-tiwari.com
• team maturity drives API maturity model• experience design influences API reusability• developer experience defines overall success• use Swagger as epicentre of you API program
LessonsLearned
t @abhishektiwari w https://abhishek-tiwari.com
t @abhishektiwari w https://abhishek-tiwari.com
Q&A