building an analytics engine on mongodb to revolutionize advertising

Building an Analytics Engine on MongoDB to Revolutionize Advertising

@ Yallzi

/crowd-Fun the Internet

“Hello (MongoDB) World!”

Steve and ALL@YALLZI{“title”: ”yallzi@yallzi”,“email”: ”[email protected]”,“linkedin”: ”https://www.linkedin.com/in/stevebond”,“twitter”: [“company”: “@YALLZI”, “personal”: “@TheOneTechie”],“web”: “yallzi.com”,"location": "garage/Austin_TX"}

mailto:[email protected]

Agenda

/ About/ Challenges, requirements/ Journey/ Outcome, next

/crowd-Fun the audience

About

/crowd-Fun the moderator

Who are you?

/about

Crowd-Fun the internet. Started *not* that long ago... in a galaxy that involved moving from NYC to Austin

1. Lost all local NYC hangouts over two short nights 2. Is there a guide to living like an Austin local and get to know business owners? 3. No excuses not to unpack boxes.

We asked ourselves:

Why there is no app for this?

/about

Dreaming of..

- Not Unpacking!- Cold Beer- Movie, - Local Museum - BBQ! I am in Texas...

Can’t a local just share the best places to hangout and let me chat vendors up ??!

/about

I want to Share my City: “My Stay-cation is someone’s perfect Vacation”: I’m searching for an easier way to help others experience the town I love. Family and friends constantly ask for recommendations for their visits. They share these with their friends and families… and then I get calls from people they’ve shared it with who I don’t even know. AND at the worst times of the day. There’s gotta be an easier way to share the town I love…

I want Advertising that works for me: “Invite vendors to your dialogue” I am not sure which venues to select. I wish I could have a dialogue with multiple vendors to help me pick the best spots for my group and needs… I wish my vendors fought for my business, maybe through hand-picked discounts and freebies? I’m sure they would like this personal connection too.

I want to Explore New Cities: “Someone’s Stay-cation is my perfect Vacation” I am planning a vacation. We love traveling like locals. Hours of scouring the web for perfect places are turning to weeks… This is becoming a second job! We just want to find a local’s perfect staycation. Isn’t there an easier way for me to experience Places others love?

/jɑːlzi:/

YALLZI/about

YALLZI helps you share all your favorite places with friends or the whole world.

YALLZI helps you explore new worlds that other Yallziers have shared.

YALLZI breaks the barrier between Vendor and Consumer.

/about

Vision

/Challenges, Requirements

/challenges, requirements

Strength-Skills● C, C++, Java

● Product Development

● Content Management

● SQL - Any RDBMS since mid 1990s

● NoSQL - 5 years of dev and prod

● Data Warehousing

● Ad Servers and Analytics

● Proven ability to squeeze 4 weeks of work out of 7 days of free demo version

● Mean Extra Dry Martini’s


Needs● Data set of POI covering major verticals in US and International

● Ensure parallel development of web, iOS and Android apps

● APP that does not need a lot of design work initially but will tell a story and please investors

● Multi-language support

● Save Time and Money on Everything

● Lighting fast speed :○ vision → prototype → alpha/beta → GA○ iterate○ scale ○ data and geo data retrieval○ DB is NOT a bottleneck for frequent app builds

● Close to real time analytics collection and processing to power apps


● Data set of POI covering major verticals in US and International -- FACTUAL, US Based POI

● Ensure parallel development of web, iOS and Android apps






Solutions


Solutions● Data set of POI covering major verticals in US and International -- FACTUAL, US Based POI

● Ensure parallel development of web, iOS and Android apps RESTful dev









● APP that does not need a lot of design … Hold Expensive Design - go simple





Solutions





● Multi-language support Pick A DB!




Solutions






● Save Time and Money on Everything Go Open Source



Solutions







● Lighting fast speed : Pick A DB!○ vision → prototype → alpha/beta → GA○ iterate○ scale ○ data and geo data retrieval○ DB is NOT a bottleneck for frequent app builds


Solutions






● Lighting fast speed : Pick A DB!○ vision → prototype → alpha/beta → GA○ iterate○ scale ○ data and geo data retrieval○ DB is NOT a bottleneck for frequent app builds

● Close to real time analytics collection and processing to power apps Pick A DB!


Solutions

- expressive query language- indexing- GeoIndexing- performance- scalability- flexibility - no need to learn new languages - easy to use management tools- a vendor behind the product- opensource / strong community- run well on Cloud (deployment architecture

flexibility)


Pick a DB...

SQL noSQL

Select * from actors a, activities ac, history h where a.actorID=...

Selecting MongoDB● Multi-language support

● Save Time and Money on Everything - Open Source/Supported by Vendor

● Lightning fast speed to ○ vision → prototype → alpha/beta → GA○ iterate○ scale ○ data and geo data retrieval○ DB is NOT a bottleneck for frequent app builds

● Close to real-time analytics collection and processing to power apps


/Journey

● Database Selection - MongoDB 2.6

● Design○ DB’s: POI, Actors, Actor Activities(Analytics) ○ Access ○ Indexing○ Availability ○ Scale ○ Flexibility

● Schema○ Actors○ Analytics ○ Advertising Secret Potion○ Personalize This!○ Deployment

/journey

Building BackEnd: Data Meets The App while both constantly Evolve


● Design○ DB’s: POI, Actors, Actor Activities(Analytics) ○ Access : API ○ Indexing○ Availability ○ Scale ○ Flexibility


/journey


All calls are made as HTTP requests to one common backend residing on one URL and serviced by one application, so all calls will have a common prefix: http://<server>:<port>/api


● Design○ DB’s: POI, Actors, Actor Activities(Analytics) ○ Access : API ○ Indexing: Frugal, don’t over-index to avoid degradation ○ Availability ○ Scale ○ Flexibility


/journey



● Design○ DB’s: POI, Actors, Actor Activities(Analytics) ○ Access : API ○ Indexing: Frugal, don’t over-index to avoid degradation ○ Availability: 24/7 365 - revisit at deployment meeting ○ Scale ○ Flexibility


/journey



● Design○ DB’s: POI, Actors, Actor Activities(Analytics) ○ Access : API ○ Indexing: Frugal, don’t over-index to avoid degradation ○ Availability: 24/7 365 - revisit at deployment meeting ○ Scale: targeting 100,000 active users in 3-6 months ○ Flexibility


/journey



● Design○ DB’s: POI, Actors, Actor Activities(Analytics) ○ Access : API ○ Indexing: Frugal, don’t over-index to avoid degradation ○ Availability: 24/7 365 - revisit at deployment meeting ○ Scale: targeting 100,000 active users in 3-6 months ○ Flexibility : Don’t throw kitchen sink in DB. Keep DB’s collections as slim as possible


/journey




● Schema○ Actors○ Analytics○ Advertising Secret Potion○ Personalize This!○ Deployment

/journey


1. Location (Dynamic)a. geo : latitude, longitude (geoLocation)

2. Preferences (Historical & “Acquired”)a. Existing preference

- Bob loves bicycling followed by a beer and lives on UES... A fellow Yallzier just biked East River Greenway, followed by a beer at Pony Bar. Yallzi renders this plan to Bob.

- A fellow Yallzier Has Shared Interests and fellow Yalzier is a referrer

a. Shared - Bob and Sue both like

Mexican, Sue loves Rosa Mexicana. Yallzi asks, Will Bob?)

- Sue has similar interests and is a referer

a. Adjacent and could be “Acquired” - Bob likes Ballets and Opera.

Sue loves this + Broadway Show. Yallzi suggest Broadway show)

- Sue has similar interests and is a referrer

1. Availability (Dynamic)a. time

{ "message": "", "status": "OK", "auth": “156D66FE8B9097A336DF325589837E674B”, “data”: {

“user”: { “name”: “Yallzier”, “email”: “[email protected]”, “phone”: “+1-2121212121”, “socialLogin”: “FACEBOOK_Yallzier”, “deviceToken”: [

“appleToken123”, “appleToken234”, ...

]}

} }

/journey

/api/v2/user/getDetails

Actors(Consumers)Parameters String auth: authentication token previously received double lat: latitude of users current position

long: longitude of users current position















]}

} }

/journey

Actors(Consumers)

/api/v2/user/getDetails Parameters String auth: authentication token previously received double lat: latitude of users current position













/journey




]}

} }

Parameters String auth: authentication token previously received double lat: latitude of users current positionlong: longitude of users current position


Actors(Consumers)Parameters String auth: authentication token previously received double lat: latitude of users current position













/journey




]}

} }

Parameters String auth: authentication token previously received double lat: latitude of users current positionlong: longitude of users current position

Actors(Consumers)


1. Location (Static)2. Vertical (Static)

a. Food & Diningb. Recreationc. Entertainmentd. Arts & Culturee. Sports

3. Categorya. Mexican

4. Offerings-Deals (dynamic)5. Hours

/journey

{ "message": "", "status": "OK", "data": {

"date": "10/26/14",! "deals": [

{ "dealId": “DINE_184a2409-7d4c-4854-ad1c-e7de099064ee", "name": "Porfirio’s Tacos",! "imageUrl": "http://api.yallzi.com/api/resources/rest.png", "address": "1512 Holly Street", "locality": "Austin", "postcode": "78703", "country": "us", "type": "Restaurant", "locationId": "184a2409-7d4c-4854-ad1c-e7de099064ee", "latitude": 30.255314, "longitude": -97.730564, "details": "", "amount": 0, "distance": 0.8936700034260777, "categoryLabels": [ [

"Mexicanl", “Tex Mex” ], ... ],

"hours": { "monday": [ ]}....

Parameters:String auth: authentication tokendouble lat: latitude of users current positiondouble long: longitude of users current position"String verticalIdint numberOfPeople: number of participants"long date: time stamp of event start"double distance: search radiusdouble searchLat, if different from users current position double searchLong, if different from users current position

Actors(Vendors)/api/v2/deal/list





/journey


"date": "10/26/14",! "deals": [



"hours": { "monday": [ ]}....

Parameters: String auth: authentication tokendouble lat: latitude of users current positiondouble long: longitude of users current position"String verticalIdint numberOfPeople: number of participants"long date: time stamp of event start"double distance: search radiusdouble searchLat, if different from users current position double searchLong, if different from users current position

/api/v2/deal/list

Actors(Vendors)





/journey


"date": "10/26/14",! "deals": [



"hours": { "monday": [ ]}....

Parameters:String auth: authentication tokendouble lat: latitude of users current positiondouble long: longitude of users current position"String verticalIdint numberOfPeople: number of participants"long date: time stamp of event start"double distance: search radiusdouble searchLat, if different from users current position double searchLong, if different from users current position

Actors(Vendors)/api/v2/deal/list

● Database Selection - MongoDB 2.6->3.0. DB size decreased by 70%



/journey





/journey


Analyticsan·a·lyt·ics

/ˌanl’itiks /noun

1. the systematic computational analysis of data or statistics.o information resulting from the systematic analysis of data or statistics.

We are Analyzing the ACTOR ACTIVITIES through Referrers and Referrals and Following…

We analyze

● intent( following)● checkIn ( confirmed at the place)● Scoop( reviews, after the visit)

With emphasis on people voting with their “feet”

/journey

User Activities include ● intent to do stuff and/or share it● intent to “borrow” or reuse of these

plans ● Checked in Activities of Referrers● Checked in Activities of Followers

{ "message": "", "status": "OK", "auth": “156D66FE8B9097A336DF325589837E674B”,! “data”: [

{ “id”: “abcdef1234”,

“referral”: “abcdefabcdef”

...}, ...] }

/journey

Referrer/Referral new Concept

/journey


{ “id”: “abcdef1234”,


...}, ...] }




Analytics

... And ... Secret Potion:

Yallzi enables:

● suggestions in your current or desired location that match your interests● suggestions for “acquired” interests (delta between having common and different preferences)● identification of emerging trends ● preparation for personal recommendations based on your and people with common interests behavior● identification of new ratings model (people vote with their feet)

/journey

● Activities collection is sharable

● Initial activity owner is a Referrer. Referrals are ‘followed”

● Vote signifies acceptance

● Ratings and Reviews=People Vote with their feet + Comments- is mined and analyzed real time

/journey


“id”: “xxx”, “date”: 1445901561064,! “dealId”: “yyy”, “YallziID”: [“xxx1”, “xxx2”, ...], “groupId”: “541f149be4b0c4c655ad5721”, “referral”: “12345677”, “votes”: [ “bead43122” : 0,

“edba43edf” : 1 ],

“comments”: [ { “user” : “ab12345678901234567”, “timestamp” : 14455667788, “text” : “This is a comment” }, ...

]}}

api/v2/yallzi/get

Activity LogPARAMs: String auth: authentication token previously received Double lat: latitude of users current positionDouble long: longitude of users current position"String yallziId: event to update"

/journey



“edba43edf” : 1 ],


]}}





api/v2/yallzi/get

Activity LogPARAMs: String auth: authentication token previously received Double lat: latitude of users current positionDouble long: longitude of users current position"String yallziId: event to update"

/journey

Actor Activity{ "message": "", "status": "OK", "auth": “156D66FE8B9097A336DF325589837E674B”, “data”: {


“edba43edf” : 1 ],


]}}

PARAMs: String auth: authentication token previously received double lat: latitude of users current positiondouble long: longitude of users current position"String yallziId: event to update"





api/v2/yallzi/get




/journey


{{ "message": "", "status": "OK", "auth": “156D66FE8B9097A336DF325589837E674B”, “data”: {

“id”: “xxx”, “date”: 1445901561064,! “dealId”: “yyy”,! “children”: [“xxx1”, “xxx2”, ...] “groupId”: “541f149be4b0c4c655ad5721”, “referral”: “12345677”, “votes”: [ “bead43122” : 0, “edba43edf” : 1 ],

“comments”: [ { “user” : “ab12345678901234567”, “timestamp” : 14455667788, “text” : “Looking for a table for 1” }, ...

] }

/journey

Vendor Direct Messaging● Message - Direct Interaction with the vendor (online) ● Consumer initiates bidirectional conversation with

business/brand● Analyzed to provide service● Can be private or public and become a hospitality message

that all consumers can view by interacting with a link.

/journey





“id”: “xxx”, “date”: 1445901561064,! “dealId”: “yyy”,! “children”: [“xxx1”, “xxx2”, ...], “groupId”: “541f149be4b0c4c655ad5721”, “referral”: “12345677”, “votes”: [ “bead43122” : 0, “edba43edf” : 1 ],

“comments”: [ { “user” : “ab12345678901234567”, “timestamp” : 14455667788, “text” : “table for 1,quiet place?” }, ...

] }

/journey





“id”: “xxx”, “date”: 1445901561064,! “dealId”: “yyy”,! “children”: [“xxx1”, “xxx2”, ...], “groupId”: “541f149be4b0c4c655ad5721”, “referral”: “12345677”, “votes”: [ “bead43122” : 0, “edba43edf” : 1 ],

“comments”: [ { “user” : “ab12345678901234567”, “timestamp” : 14455667788, “text” : “table for 1,quiet place?” }, ...

] }

/journey


business/brand● Analyzed to provide service● Can be private or public and become a hospitality

message that all consumers can view by interacting with a link.




/journey


Personal “funBox” Collection

● Your most frequent referrers can be pre-processed

○ Yallzi personal “funBox”

/journey

Grouping and Deltas{WIP}

Will include: - Plans of most frequently used

referrers - Does not require any action on

user part - Analyses Yallzier’s activity log



● Schema○ Actors○ Analytics○ Advertising Secret Potion○ Personalize This!○ Deployment : AWS, MMS

/journey


Outcome, Next and Recap

Outcome- Dream to Prototype : 2.5 weeks

- Prototype to Alpha: 1 month

- iOS App development: ~5 months

- Android App development: ~ 2 months

- Suspended

- Rebranded and Pivoted 5 months into Dev. Adjust within 2 weeks

- Beta Releases: twice a week

- AWS hosting $36 a month

- MongoDB : rock solid

/outcome, next

Recap (our lessons)- MongoDB is great fit for bootstrapped startUps like Yallzi or for an innovative project with aggressive goals

- Take risks, but stick to the following:- Simplify everything from storage to aggregation downstream- Understand what you are storing and where (vis a vis static and dynamic data)- Determine what data will be made public/private- Don’t go Index happy- Flexibility is not throwing the kitchen sink in DB. Flexibility is for growth and dynamics of your business.- Overuse of database strengths can create a big SPOF- Simplify access patterns/abstract

- If it is too easy to do, you might not have thought of something

- Have fun! It will work!

/outcome, next

What’s Next- Development:

- Completing within next few weeks- Beta launch:

- Follow us on Twitter @Yallzi or @TheOneTechie for the date to install the app and Crowd-Fun the internet

- We are looking for partners and developers- Funding :

- We are going strong for the first prize in crowdfunding drives- We are in conversations with TX based accelerators- We love talking to investors even those who want to say NO!

- Community:- We are looking for advise and happy to share our thoughts

- Future:- See y’all on Yallzi and at MongoDB World next year

- Philosophy:- Do one thing, do it right, test, repeat!

/outcome, next

Hit is on Twitter- @YALLZI- @TheOneTechie

Yes, We’re Hiring:- http://yallzi.com/internships/

Eureka Moments:- http://yallzi.com/internships/our-eureka-moments/

Angel Profile:- https://angel.co/yallzi

On Social Choice Data:- https://www.linkedin.com/pulse/article/social-choice-data-steve-bond- https://www.linkedin.com/pulse/how-do-groups-decide-social-choice-d

ata-part-2-steve-bond

On App Adoption:- https://www.linkedin.com/pulse/article/first-date-app-steve-bond- https://www.linkedin.com/pulse/article/userapp-dating-steve-bond

Holly Street Garage, Austin, Texas----------------------------------------------Also Found on: KickStarter, LaunchLeader, CrunchBase.. and other fine ‘net sites

/outcome, next

https://twitter.com/yallzi

https://twitter.com/theonetechie

https://yallzi.com/internships/

http://yallzi.com/internships/our-eureka-moments/

https://angel.co/yallzi

https://www.linkedin.com/pulse/article/social-choice-data-steve-bond

https://www.linkedin.com/pulse/how-do-groups-decide-social-choice-data-part-2-steve-bond

https://www.linkedin.com/pulse/how-do-groups-decide-social-choice-data-part-2-steve-bond

https://www.linkedin.com/pulse/article/first-date-app-steve-bond

https://www.linkedin.com/pulse/article/userapp-dating-steve-bond

Thank you

/Tricks

Actually, there are no tricks, whatsoever… except- be “lazy” developers making the best of the tools we build- provide users, be they sharers, benefactors, or business owners with simple tools:

- map and ‘save’ their most memorable moments and share it with y’all- see others best memorable moments- reach out to vendors to invite them into the conversation

For this we needed a thought through, light-weight and flexible model that allows us to build/add additional artifacts and store them.

We also decided to bank of reverse of Similar Interests and see if people might want to discover stuff they never experiences before. We are still working on it

We are looking forward to learn from reputable companies and scrappy startUps!

What challenges do you solve when you don’t get paid for it?

● A revolution in combining social choice data with behavioral data that results in Kick Ass User Experience

● New Dimension of Advertising

● We don’t reinvent the wheel, even though we tried 17 times and came up with helium inflatable wheels supporting a beach chair with 2 beers and my puppy than annoys me when I work

/What We Do

#startUp Recent SCRUM NOTES

● “Man, where did all those competitors come from?”

● VC Breaking News: said they Loved the idea and we definitely should.. call them after we have 1 million DL’s.

● Alerts: Looks like we hacked our own bank account but due to error in the code we left 23 cents. Whose code is it, i don’t care whose money..

● Battle of API versus App developer is won by API. After all: iPhone processing power is probably greater than our server!

building an analytics engine on mongodb to revolutionize advertising

Technology

yallzi crowdfun

perfect places

someones perfect vacation

austin local

easier way

best places

scale data

favorite places