making your app see with mobile vision api

RAMON RABELLO CEO @ VISYO

MAKING YOUR APP SEE WITH MOBILE VISION API

RAMON RABELLOabout.me/ramonrabello

Agile Community Active Member

GDG Brasília Organizer

Senior Developer ~10 years

HOW DO WE SEE?

HOW DO MACHINES SEE?

WHAT IS COMPUTER VISION?

HOW MACHINES UNDERSTAND THE WORLD

COMPUTER VISION IS EVERYWHERE!

ON ROBOTICS.

ON SECURITY.

http://www.eecs.qmul.ac.uk/assets/img/uploads/page-image/behaviour.png

ON SOCIAL NETWORKS.

ON MEDICINE.

ON GAMING.

ON CARS.

AND ALSO ON MOBILE.

MOBILE VISION API OVERVIEW

OVERVIEW COMMON FUNCTIONALITY FACE API BARCODE API TEXT API (OCR)

COMMON FUNCTIONALITY EVENT-DRIVEN PIPELINE FACE DETECTOR BARCODE DETECTOR TEXT DETECTOR

FACE API WHAT CAN WE DO WITH FACES?

FACE TRACKING

LANDMARKS DETECTION

FACE CLASSIFICATIONS EYES OPENED

0.0 1.0

FACE CLASSIFICATIONS SMILLING

0.0 1.0

FACE ORIENTATION POSE ANGLE ESTIMATION

Y

ZX

BARCODE API WHAT TO DO WITH BARCODES?

BARCODE TRACKING

FORMATS SUPPORTED

TEXT API READING TEXT FROM IMAGES

READING TEXT FROM IMAGES

18+ LANGUAGES SUPPORTED

TEXT STRUCTURE

MOBILE VISION API USAGE 1. ADD DEPENDENCIES 2. ADD META-DATA ON MANIFEST 3. CREATE DETECTOR 4. CHECK IF DETECTOR IS OPERATIONAL 5. CREATE FRAME 6. DETECT FACES

ADD DEPENDENCIES build.gradle

dependencies {

…

compile 'com.google.android.gms:play-services-vision:{version}'

…

}

ADD META-DATA ON MANIFEST

AndroidManifest.xml

<application…> <meta-data android:name="com.google.android.gms.vision.DEPENDENCIES" android:value="face" />

</application>

CREATE DETECTOR onCreate()

FaceDetector faceDetector = new FaceDetector.Builder(context) .setTrackingEnabled(false) .setProminentFaceOnly(false) .setMode(FaceDetector.FAST_MODE) .setClassificationType(FaceDetector.ALL_CLASSIFICATIONS) .setLandmarkType(FaceDetector.NO_LANDMARKS) .build();

CHECK IF DETECTOR IS OPERATIONAL

onCreate()

faceDetector.isOperational(){

// code to detect faces here

}

downloads a native library, when run for the first time.

CREATE FRAME

Bitmap bitmap = …; // read Bitmap

Frame frame = new Frame.Builder() .setBitmap(bitmap) .build();

onCreate()

DETECT FACES

SparseArray<Face> faces = detector.detect(frame);

// face(s) detected if (faces.size() > 0){

}

onCreate()

DEMO THE FACEMINATOR

/visyo/faceminator/

WHAT’S NEXT? MOBILE VISION GUIDE https://developers.google.com/vision/

WHAT’S NEXT? VISION SAMPLES https://github.com/googlesamples/android-vision

WHAT’S NEXT? DEVBYTES https://www.youtube.com/androiddevelopers

THANKS!

goo.gl/v5D0mg

@visyosolutions

/visyo

goo.gl/VBJTIh

@gdgbrasilia

/gdgbrasilia