User Log Analyzing Algorithm Simulator

491 May15-11

TEAM and Client

• TEAM CLIENT & ADVISOR

• Wenqin Yu Professor Daji Qiao

• Shi yiqun Brian Peck

• Junjie Fan

Client Requirement

• A simulation platform for algorithms

• Serving a real application

• Provide a effective way to study users’ behavior pattern.

• Multi-platform (GAEs and Amazon EC2s)

Platform Requirement

• Google App Engine

• Automatic scaling (auto allocates more resources for the web apps to handle additional demand.)

• Free charge up to certain level.

• Amazon EC2

• Allows user to obtain and configure capacity with minimal friction

Problem Statement

• Need to provide a real application for users (File Editor App)

• Need for log generating and storing

• Accessible for algorithm to analyze

• Need an admin control panel for analysis output review

Conceptual Sketch

• GAE (1): Application Server

• Amazon EC2 (1): User Database Server

• Amazon EC2 (2): Log Process Server

• GAE (2): Admin Control Panel Server

Functional Requirements

• User authentication (User)

• File editing (User)

• Log generating / storing

• Log available to Review (Admin)

• Provide API for Log retrieving (Algorithm)

Non-Functional Requirements

Application Requirements

• Accessibility (Application can be access on most popular browsers)

Non-functional Requirements

Log Requirements

• Log accuracy

• Log data accessibility (Good format and structure)

Technical Consideration 1

Multi-platform:

In this project, we need use 2 GAE and 2 Amazon EC2 as client required. Data and files will be frequently transmitting among these platforms.

Solution: Among these servers, using HTTP protocol, build API on each platform for file and data transmitting.(File size will be small)

Technical Consideration 2

Log Accuracy

During log transmitting among platforms, log package will under risk to be lost.

Solution:We will generate log data from both User Database server and App Server, and make comparison in Log server before finalized into log database

Project Milestones & Schedule

• End of Fall semester:

• Finished Design Document

• Finished Project Plan

• Project Website

• Setup Google App Engine / Amazon EC2

• End of Winter break:

• File Editor Application (App Server)

• User Management System

• Setup user database

• End of Spring semester:

• Log database setup

• Finished all API and request sender

• Build up algorithm container

• Admin Control panel

System Design

Functional Decomposition

Detail DesignApplication Server

User Database Server

Log Process Server

Functional Decomposition-Application Server

Application Server

Functional Decomposition-User Database Server & Log Server

Application Server

User Database Server

Log Process Server

File editor

Functional Decomposition- Admin Control Panel Server

Log Process Server

Tech Specification

• Language used:

• Java/Python: Algorithm container (In Amazon EC2)

• JavaScript: For front end design

• MySQL

• PHP:

• 1. API

• 2. User control penal

• 3. Generating Log

Platform Used

•Amazon AWS (Linux)

•Google cloud platform (google app engine)

Test Plan

• Unit testing

• External Unit Test

• Real user testing

• Front-end Testing Request

Metrics

• Network throughput

• API format

• Log format

CONCLUSION

Project Status

• User management has been setup

• Setting up the GAE and EC2

• Working on file editing app (locally)

Task Responsibility

• Wenqing Yu

• Log generating system

• Log transmit and fetch

• Algorism container

• Shi yiqun

• User manage system

• Analyze output retrieving

• Junjie Fan

• File editing application

• Admin control panel

Plan for Next Semester

• Build java algorithm container

• HTTP connection APIs

• Admin control panel

• Testing

• Debug

• Lunching simulator

• Finished project