33degree krzysztof debski - let's build a solid base for a scale

@DebskiChris

#33rdDegree

Krzysztof Dębski Allegro Group

Let’s build a solid base for a scale

#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY

Agenda

The Allegro Situation How to build a new World? The way to improve What’s in it for you?

Allegro The New World Improvement WIIFY #33rdDegree @DebskiChris

The Allegro Situation

Allegro

6 million LOC

400 people in IT

Allegro

A New Hope

A New Hope?

Service

Oriented

Ambiguity

Domain Driven Design

(Micro)Services

Business needs

Transaction

(Micro)Services

Independent

Transaction

(Micro)Services

Transaction

(Micro)Services

Polyglot

Cassandra

MongoDB

Oracle

Transaction

(Micro)Services

Smart Endpoints

Cassandra

MongoDB

Oracle

Transaction

The first approach

Service

Auto deployable Auto monitored Auto scalable Auto healable Auto discoverable … and many more features

How to build a new World?

The first project

Service

Jetty Server

War file

Spring

Gradle

Let’s REST

JAX-RS / JSR Compliant @Path("/users") @Consumes(CONTENT_TYPE_JSON) @Produces(CONTENT_TYPE_JSON) public class UsersEndpoint {

// [...] @GET public UserCollectionResponse findAllUsers() { //[...] }

Let’s expose our methods

Swagger @Path("/users") @Api(value = "/users") @Consumes(CONTENT_TYPE_JSON) @Produces(CONTENT_TYPE_JSON) public class UsersEndpoint {

// [...] @ApiOperation(value=“Get all users”, response=UserCollectionResponse.class) @GET public UserCollectionResponse findAllUsers() { //[...] }

Swagger @Path("/users") @Api(value = "/users") @Consumes(CONTENT_TYPE_JSON) @Produces(CONTENT_TYPE_JSON) public class UsersEndpoint {

Swagger

And make them discoverable

Discovery ZooKeeper

Register

Discovery ZooKeeper

Get User

Discovery ZooKeeper

Get User

Discovery ZooKeeper

Get User Get offer for user

Multiple API versions

Content negotiation @Path("/users") @Consumes(”application/json”) @Produces(”application/json”) public class UsersEndpoint {

Content negotiation @Path("/users") @Consumes(”application/vnd.allegro.user.v1+json”) @Produces(”application/vnd.allegro.user.v1+json”) public class UsersEndpoint {

curl --dump-header - -H ”Accept: application/vnd.allegro.user.v1+json" -X GET http://localhost:8080/users

Content negotiation @Path("/users") @Consumes(”application/vnd.allegro.user.v1+json”) @Produces(”application/vnd.allegro.user.v1+json”) public class UsersEndpoint {

// [...] @GET @Produces(“application/vnd.allegro.user.v2+json”) public UserCollectionResponse findAllUsers() { //[...] }

curl --dump-header - -H ”Accept: application/vnd.allegro.user.v2+json" -X GET http://localhost:8080/users

And it became slow…

Lower startup time

External Jetty

Service

Jetty Server

War file

Lower startup time

Embedded Jetty

Service

Jetty Server

Jar file

Lower startup time

Embedded Jetty

Service

Jetty Server

Jar file

Lower startup time

Embedded UnderTow

Service

Jetty Server Undertow

Jar file

Lower startup time

Immutable Images

Service

Undertow

Zip file

Server

Immutable image

How to configure it?

Service

java –jar service.jar

Service ZooKeeper

java –jar service.jar –Dzookeeper.address=zookeper.service

Service ZooKeeper

java –jar service.jar –Dzookeeper.address=zookeper.service –Denvironment=prod

prod.properties

Service ZooKeeper

java –jar service.jar –Dzookeeper.address=zookeper.service –Denvironment=prod -Dservice.name=testService

prod.properties

Do you test your tests?

Pitest

Integration tests

Test without mocks Run from IDE

public class UsersIntegrationTest {

private static Map<String, String> overrideConfiguration = Maps.newHashMap();

static { overrideConfiguration.put("property.name", "This was overwritten value"); }

@ClassRule public static final RestServiceStarted DEPLOYED_SERVICE = new RestServiceStarted(overrideConfiguration);

private WebTarget getUsersResourceWebTarget() { return DEPLOYED_SERVICE.getWebTarget().path("users"); }

Monitoring

Let’s see what’s happening

NxLog LogStash Kibana

Let’s see what’s happening

Graphite # Metrics metrics.reporters.graphite.enabled=true metrics.reporters.graphite.host=graphite.service metrics.reporters.graphite.port=2003 metrics.reporters.graphite.prefix=stats.Prod.service metrics.reporters.interval=30

Monolith Alert

Multi module project support

Monolith Alert

Multi module project support

The way to improve

Authentication

OAuth2

Patch Support

Standardized by IETF https://tools.ietf.org/html/rfc6902

PATCH /user/1 HTTP/1.1 Host: user.service.local Content-Length: 312

Content-Type: application/json-patch+json [

{”op”: ”test”, ”path”: ”/firstname”, ”value”: ”Jane”}, {”op”: ”add”, ”path”: ”/maidenname”, ”value”: ”Smith”}, {”op”: ”replace”, ”path”: ”/lastname”, ”value”: ”Doe”},

{”op”: ”remove", ”path”: ”/meetings”}, {”op”: ”move”, ”from”: ”/balance”, ”path”: ”sharedbalance”}, {”op”: ”copy”, ”from”: ”/a”, ”path”: ”/b”}

Simplify annotations

Swagger? @Path("/users") @Api(value = "/users") @Consumes(CONTENT_TYPE_JSON) @Produces(CONTENT_TYPE_JSON) public class UsersEndpoint {

Deployment

Docker support

What’s in it for you?

Use the latest tools

GradleW in CI ENV and parent POM issues. Don’t do DDOS yourself and your partners.

Change is the only constant

Deployment tools Service API provider Dependency injection HTTP Server

Cargo Docker

Ecosystem

Swagger Jersey addon

Guice Spring

Jetty Undertow

Focus on right metrics

How to have 100% code coverage? public class UserMediaType {

public static final String V1_JSON = "application/vnd.allegro.user.v1+json” public static final String V2_JSON = "application/vnd.allegro.user.v2+json”

Focus on right metrics

import org.junit.Test; import static net.trajano… public class MediaTypesTest { @Test public void mediaTypesShouldBeValidUtilityClasses()

throws Throwable { assertUtilityClassWellDefined(UserMediaType.class); } }

You don’t need 100% code coverage!

Tools also lie

Metrics tend to lie

Configuration: default PHP metrics

Tested file: 4535 CLOC Imperative programming

Tools also lie

How many violations are there?

Tools also lie

How many violations are there?

4 At least according to Sonar.

Learn to REST

REST is not so obvious

/api/bi/XYZ123/1

/recommendations/1/items

/offer/100?output=ESI/JSON

Community

Involve all developers in building the Bootstrap. Or they will build their own tools.

Łukasz Drumiński Mateusz Gajewski @wendigo

33degree krzysztof debski - let's build a solid base for a scale

debskichris allegro

allegro situation

new hope

debskichris microservices

json x

deployable auto

service oriented ambiguity

jaxrs jsr compliant

Technology

audyt_energetyczny krzysztof fryżewski

krzysztof kieślowski

krzysztof pulkiewicz kpi

krzysztof musiał

mjr krzysztof gaczyński

krzysztof penderecki

krzysztof osiak, krzysztof wrzeszcz - gemalto, infostream -...

krzysztof szanecki przelewy24

title: polsko- krzysztof nowak nowak krzysztof. (2017

krzysztof kmieć

lukasz debski: kačky a žvýkačky

howell times greg debski 100th donation

krzysztof kieslowski

krzysztof wieczorek

j. nutr.-1987-debski-1091-7

geecon.cz 2015 debski krzysztof

internet tasks and projects in ethnic schools robert debski...

groovy ast demyistified - 33degree

debski rafal - zoroaster gwiazdy umieraja w milczeniu.pdf

krzysztof bojarczuk_2005