Band of Brothers, building scalable social web apps on Windows Azure
with ASP.NET MVC3, MongoDB, RabbitMQ
Marjan NikolovskiCo-Owner, Senior Software Engineer at EmitKnowledgeSenior Software Engineer at Seavus
Contact:emitknowledge.com/@[email protected]
Agenda
• Social web architecture• Intro to MongoDB• Data modeling for MongoDB• Intro to RabbitMQ• Preparing your guns for pub/sub with RabbitMQ• Data painting with MVC3• Hosting and scaling strategies on Windows Azure
Social web architecture
• By definition a social web must provide connectivity between the users for a common good
• From functional aspect a social web must provide:• Connectivity (Relationships)• Privacy• Private messaging• Notifications and events (both real-time and offline)• Recommendations
Social web architecture
• Connectivity, how users are socializing• Directional or bidirectional relations ?• How are users going to socialize ?
• Post Sharing• Groups• Private messaging• Gamification
Social web architecture
• Privacy• Beware of details• Who, When and What
UserA User B User C
Who can access my data ? Y N
When can my data be accessed ? After publish Delay with 1 day
What kind of data could be accessed ? Posts and events Posts
Social web architecture
• Private messaging aka Conversations• Define messaging types:
• One to One• Group messaging
• Define messaging strategy• When a new conversation starts ?• Until when does it last ?
• Spam detection and prevention• Mark as spam strategy• Spam filtering strategy• Both ?
Social web architecture
• Notifications and events (both real-time and offline)• When to notify ?• Frequency of notifications• To stream or to aggregate?• To filter or display all ?
Social web architecture
• Recommendations• Define recommendation method:
• Friends method• Recommends the friends of my friends• Ease to implement• Big miss factor• Recommends people that we possibly know• Not very practical
• Sharing interests method• Recommend me a friend that we have a starting point to talk about• Efficient for connectivity• Needs initial user input for building the analytics
Social web architecture
• From technical aspect must respect:• Stateless• Pluggability• Async execution• Load distribution• Intensive logging and auditing• Content delivery strategy
Social web architecture
• Stateless• Azure instance load balancing is your enemy
• We don’t know on which instance the request will end (cloud load balancing)
• So, forget about using server side session or tempdata• Yes we can store it in DB, but performance per request ?
Social web architecture
mysocialapp.cloudapp.net
mysocialapp.cloudapp.net
Social web architecture
• Pluggability• Enable the platform to go down partially• Develop your functionalities as plugins• Enable real-time plugin load/unload (hotplug)• Partial deployment
Social web architecture
Social web architecture
• Async execution• Sequential processing only at the minimum, everything else
delegate to background services• User registration use case• User reset password use case• User notification use case
• Time and data intensive operations must execute as background services jobs
Social web architecture
Web server Messaging infrastructure
Send create user request
Worker services
User created response
Send confirmation email message
Notify for new user registration message
Social web architecture
• Load distribution• We hit the processing limit per machine. Now what ?
• Develop your data and time intensive functionalities with job distribution in mind
• Event based Producer – consumer jobs• Each functionality requires
• Job orchestrator• Job processor
Social web architecture
MESSAGING INFRASTRUCTURENew user postNotify all of my friends Notify from 1 – 100
Notify from 101 – 123
Notify all of my friends
Notify from 1 – 100
Notify from 101 – 123
Social web architecture
• Intensive logging and auditing• You will always want to know what your users are up to• Analytics can’t help !• Put your intensive logging and auditing so you can playback later• Log and audit everything you can get• Build your own analytics system
Social web architecture
{ "Username" : "some_username", "IsAuthenticated" : false, "RawUri" : "/Account/Login", "BaseUri" : "/Account/Login", "HttpMethod" : "POST", "IpAddress" : "127.0.0.1", "Refferer" : "http://localhost:42378/", "UserAgent" : "mozilla/5.0 (windows nt 6.1) applewebkit/535.12 (khtml, like gecko) maxthon/3.3.4.4000 chrome/18.0.966.0 safari/535.12", "IsCrawler" : false, "IsSecureResource" : false, "Action" : "Emit.Knowledge.Controllers.RequestLogging.UserRequestLoggingController.VerifyUserCredentials(String username, String password, User& verifiedUser)", "Data" : { "username" : "test333" }, "IsFaulted" : false, "CreatedOn" : { "UtcDateTime" : new Date("3/10/2012 01:40:52") }}
Social web architecture
• Content delivery strategy• Reference your scripts with build number
• .js?ver=1.0.1 to propagate script changes with ease• Minify as many js scripts into one• Group page images into sprites and add build number
• .png?ver=1.0.1• Reference user images to the CDN • Analyze the average change of user profile image to determine
the image caching period
Intro to MongoDB
• Document store• Fast, scalable and available• JSon – used for data hydratation• Dynamic• Stores data structured as documents instead of row as seen in
RDBMS• Uses Query, Insert, Update and Remove for data manipulation• Used for adding data at high rates without “choking” the system
Intro to MongoDB
• Documents in MongoDB• Each document can be of size max to 16mb• Each document you insert into MongoDB will be assigned an id
consisted of: • This is a 12-byte value consisting of 4 parts:
timestamp (4 bytes)• machine identifier (3 bytes)• process id (2 bytes)• increment (3 bytes)
• in MongoDB, each document will contain not only the values, but the key names (~"column names" in relational db-speak) too. So if you have keys: “Username" and “Password", then those alone will add 16 bytes to each document.
• MongoDB automatically adds some padding to the documents to allow for documents to grow in size to try and reduce the need for documents to be moved around if they do grow
Data modeling for MongoDB
• Forget what you’ve learned at school and shift your mind• Want to be fast ? – Denormalize data that will be static
(IAuditable – ring any bells ?)• Common pitfalls• To render a post we need the post content and the username• Modeling by the book will get you to model the Content entity to
have Title, Content, UserId • Now we need to fire two queries to display the username
Data modeling for MongoDB
• Beware of your queries• Analyze all of your queries• Set index per query
• But don’t forget the order by in the index • Mongodb has limitation for “OR” queries and sorts (indexing will
not help you here)• Think in map/reduce way maybe ?
Intro to RabbitMQ
• Robust and reliable messaging for apps• Supports many messaging patterns• Publish-subscribe• Topic based PubSub• Point-to-point• Request-reply• Store and forward• …
Publish-subscribe{
Publisher
Topic
Subscriber Subscriber
Consumes the event Consumes the event
Publish an event
Topic based PubSub
Publisher
Topic
Subscriber Subscriber
Consumes the event Consumes the event
Publish an event
SubtopicSubscriber
Consumes the event
{
Point-to-point
Publisher SubscriberQueue
{
Request-reply
Publisher/Subscriber SubscriberTopic
Subscriber
Subscriber
{
Store and forward
Publisher/Subscriber SubscriberTopic
Subscriber
Subscriber
Data
{
Preparing your guns for pub/sub with RabbitMQ
• Establishing pub/sub architecture• Who will listen on what ?• Do we need to intercept messages ?• Do we need to deliver one message to many subscribers ?• Do we need to distribute the message processing ?
• Two levels of messaging• Job coordination and distribution• Server instancing coordination
• Messaging patterns of interests• Pub/Sub• Topic based Pub/Sub
Preparing your guns for pub/sub pub/sub with RabbitMQ
• Persist messages that must be delivered, everything else just flush it through the wire
• Topic subscription is your enemy• Don’t expand your topics tree in depth• Minimize topic root subscriptions
Data painting with MVC3
• Delegate data render to the client with dynamic data instead on server side
• Prepare your static content and the templates in the views (this will help you to reduce the traffic. Dom will be created on fly.)
• Knockout your dynamic data
Data painting with MVC3
<script id="notes-template" type="text/html"> {{each notes }} <div class="note-wrap"> <div class="note-title"> <div class="title-text">${GetTitle}</div> <div class="title-date">${LastEditedDate}</div> </div> </div> </div> {{/each}}</script>
Data painting with MVC3
<div class="note-wrap"> <div class="note-title"> <div class="title-text">Title 1</div> <div class="title-date">2012-01-01</div> </div> <div class="note-title"> <div class="title-text">Title 2</div> <div class="title-date">2012-01-02</div> </div> <div class="note-title"> <div class="title-text">Title 3</div> <div class="title-date">2012-01-03</div> </div> </div></div>
Data painting with MVC3
Template + Ajax Server render
638 bytes - JSON /
2246 bytes - HTML template 22460 bytes - HTML
TOTAL 2884 bytes 22460 bytes – HTML
Hosting and scaling strategies on Windows Azure
• Design deployment and scaling strategy per component:• Web server• Data server• Messaging server• Worker server
Hosting and scaling strategies on Windows Azure
• When to scale ?• MongoDB instances
• Indexes are too large to fit in memory• First scale vertically with RAM then shard
• RabbitMQ instances• Message delivery slows down• Scale vertically with CPU then shard
• ASP.NET MVC instances• Number of users goes large• Check what is eating the machine throughtput
• Usually problem with the notifications long pooling• Scale horizontally with more extra small or small instances
• Worker instances• Job processing time goes “large”• Scale horizontally with more extra small or small instances• Round robin processing strategy will help you to relax the job processing
Hosting and scaling strategies on Windows Azure
• Two instances per type to meet with the SLA requirements• Use CentOS for MongoDB, Linux has better memory
fragmentation than Windows• For RabbitMQ choose the OS that you are most comfortable
with.• Windows 2K8 OS for the ASP.NET MVC
Hosting and scaling strategies on Windows Azure
• Open SSH and MongoDB ports only• Don’t forget to disable anonymous login on the MongoDB and
set u/p combinations for both server and database
Hosting and scaling strategies on Windows Azure
• Open SSH/Remote Desktop ports and RabbitMQ communication port
• Disable anonymous connections to the instance and add u/p combinations for connection to the RabbitMQ server
?