OPETUS DATABASE DESIGN PHILOSOPHY

Most primitive database design of Opetus supported Multi Company / Location / Year. When data across year needs to be archived in a single database without swapping backups, design considerations have to segregate metadata, masters and transactions, indexes separately by partitioning storage into various file groups. This helps manage storage and retrieval effectively without performance penalty over age of database. Management of data such as re-indexing, backups, defragmentation can be planned and scheduled to keep it healthy

Database design isolates types of data segregating based of usage frequency, operations on data, change in size and search needed for reporting and analysis. Metadata which just has read operations can be cached and secured, masters which has frequent reads and rare updates, and transactions which has most read and writes are partitioned by design during creation of database object. Most retrievals are via views being called through stored procedures. If views need computed logic they are preceded by custom functions which are either scalars or table valued

Opetus archives binary LOB data such as pictures, movies, audio content, etc… by embedding objects rather than linking paths. This eases backup management and does not depend on Active directory settings or network topology to be taking care of scattered linked entities. A separate partitioned file group manages all binary large object data. Upon insert of any such binary objects a unique GUID gets assigned to them. Extraction / retrieval or update of such objects is via Opetus Restful web services

All textual attachments (non binary) are archived separately in a full text file group. Each text LOB is also assigned a unique GUID and has a separate search, retrieval and update web service. Most assignments, presentations are generally PPTs, PDFs, DOCs. Such textual content have a separate storage layer and SQL Server is deployed / enabled with full text features

OPETUS DATABASE PARTITIONING SCHEME

Opetus default schema has following groups as shown in database schematic. Every table has auto identity clustered primary keys and non clustered unique key persisted in INDEX file group. Partition names are self explanatory. If ERP integration needs various devices to be interfaced such as ARM / IOT devices and scheduled CRON pulls data from these devices, it is saved in a separate partition. The reason for this is as soon as analytics or aggregations from readings are derived retention of such data is useless and they need to be purge and partition shrunk

When primary database needs data to be collected from web, exposing critical file groups to abuse is a risk. Let’s take the example of online application form. Parents and sceptic users would enter perfectly valid junk data for trial purpose just to know the steps in interface or to be sure about themselves. This guarantees massive collection of digitized “shit” which needs to be filtered, sanitized via ETL and loaded for further workflow. Hence for the sake of database sanity this file group can be used as scratch pad for data aggregation.

If reporting needs require large sets of temporary tables to be created and deleted, a separate file group needs to be assigned to takeoff the load off “tempdb”. For some implementations we maintain a separate “ERROR” file group with triggers on tables to relay notification emails via queues or message brokers. This use case is very specific for heterogeneous system integrations where 24x7 online SFT needs to be maintained for HA/HS operations

If clients have a high speed NAS / SAN or servers have IO accelerators or SSD hard drives, the paths of these file groups can be scattered to achieve maximum I/O parallelism. In case of online operations where a replicated DB is maintained separately for reporting / analytics, partitioning of database to suite RAID 1-0, RAID 5, or RAID 0 can be customized

OPETUS DATA SECURITY

Based on customer’s security requirements Opetus uses TDE, Symmetric / Asymmetric keys, certificates if any. Certain key metadata tables and columns are encrypted by default and sensitive data can be secured as needed. Opetus has a obfuscated custom ISQL.dll which is shipped optionally for extra security. This dll has a common security interface for web services to encrypt / decrypt data on the fly. ISQL registered assembly functions and procedures help SQL server manipulate data when performance becomes critical

Opetus default database design has audit trails of DDL changes. This helps in DB revision management as well as track abuse of system by unauthorized system administrators. Critical data such as mark sheet records have row level CRC generated. These features are available only in enterprise edition of Opetus

Many application forms which admin has access and are prone to abuse have separate provision for encrypted log trails. Opetus generates logs for all web service requests, responses, user behavior, communication such as SMTP / POP / SMS / Chat, errors and exceptions, CRON tasks etc…

Change of date triggers automated compression and FTP of day to day log files. When Opetus is configured in master-slave mode for distributed networks with poor internet connectivity, store and forward mechanism ships log files of differential writes for data centralization either via email / ftp or web services

Jobs for automated backups, database health monitoring, full text indexing are part of Opetus metadata

OPETUS MASTER DESIGN PHILOSOPHY

Name: Defines the name of the object. Ex: Subject Name can be “English”.

Short Name: This is generally a short form or acronym of the object name. It primary purpose is to aid columnar reports. Try to define the acronym so that users can co-relate the name to the defined entity. Ex: Short form of English can be “ENG”.

Alias Name: Often objects have more than one addressable names. One which is global or as per standards and other the regional or what the local dialect understands it popularly. Also in many cases the printed name and screen name differ. Use alias name to address these implementation issues.

Prefix/Suffix: Many workflow processes need object specific key generation for a record. This helps index/recognize record isolation faster. Ex: In education institutes location prefix/suffix (BBY/NSK) or Academic Year (2009/2010) is appended or prepended in application forms or enrollment numbers to search records. This applies to all business processes wherein many document/asset tags are generated using series of object prefix/suffix.

ID: In specific use cases objects/records are identified using numbers rather than strings. It is easier to sort numbers and they occupy lesser byte area. Hence whenever and object to number translation is needed ID column can be used. Ex: Many universities when publishing results will specify subject number rather than name and provide a legend which translates to a name. Even grades/credits follow simple translation or string to number or number to string using this basic design principle.

Order: Often records need to be sorted as per user sequence which may not be alphanumeric. Ex: Report cards often publish language subjects first, then Science, Math's, Social Studies and lastly gradable subjects. The sequence of names and order to publish and print varies. Order column specifies user definable chronological order rather than alphanumeric.

Most master records in OPETUS will have these inputs:

Type: Object sets of object have a collective behavior or properties. Software logic is bound to behave as per functional nature of types of objects. Hence if objects can be categorized or classified the resulting definition segregation yields several types. Ex: Admission category/type (Open/SC/ST/OBC) specifies how student admission process has been categorized into various types. Subject type specifies whether the subject is compulsory or optional …

Fore/Back color: Many a times in sheet/grid or matrix representation of data, a pattern needs to be presented to user to enable him/her visualize a snapshot of density/scatter of frequency of repetitions / similarity. Color enables to highlight or segregate patterns.

Attributes: These are Boolean entities or properties of objects which deviate behavior of objects under specific conditions. Software logical tweaks are full of these if/else’s. Most common method to capture this information is using checkboxes. Opetus design philosophy has isolated all these attributesand cumulatively captured them in matrix format. Many future revisions or ECR (engineering change requests) can be resolved just by changing some data rather than using programming logic to version enhancements. Ex: Student can have several attributes such as whether is a staff or alumni child, whether he uses school transport (bus child) or not. All such Boolean entities are flagged cumulatively to impart logical behavior for the student.

Level: This column is more related to hierarchical objects whenever there exists a parent child relationship in records and hierarchical level abstracts depth of tree data. Ex: Math's being a parent subject and Algebra / Geometry as child subjects. Same applies to Science (physics/chemistry/biology as child subjects) and Social Science wherein history/geography/civics are child subjects.

LOBID: Many records of masters have associated content (unstructured data such as images/documents which needs to be archived with the master record. Opetus design supports data capturing of centralized indexed Large Binary Objects along with structured transactional data.

Parent: Some master data need to be linked in parent – child relationship. Parents are combo’s populating same object data to facilitate hierarchical associativity.

OPETUS MASTER DESIGN PHILOSOPHY

Path: If application needs unstructured data of a record (content such as images/docs/audios/videos) to be associated with physical storage path so that it can be translated to a URI/URL later for universal web access and yet maintain a methodical directory structure for mapping physical, logical and achieved entities user can specify a path pointing to a storage device (generally NAS/SAN). Opetus I-Server uses this path data to create virtual directories and isolate storage for sets of data associativity. Ex: If user want all Biology content to be in a specific path as per section/standard or a division, all uploads/downloads will get associated to path specified. Care needs to be exercised during change of storage path as volume of content can create operational nightmares.

Tasks: Any event outside the scope of software needs to be associated real time for workflow purpose. External or internal influences needs to be translated into events and associated workforce needs to be notified of these events. Some of these events needs manual interventions so has to translate to tasks/reminders. Each object can be associated with set of tasks as per Opetus design philosophy. Events can be synchronous i.e. have a periodicity associated or can be adhoc/disruptive based on certain conditions of data.

Taxonomy: Often vendors cannot customize perpetually for specific requirements and yet the user needs these data of the object for statutory / reporting purpose. These data fields do not have any bearing on functionality yet sometimes certain information needs to be captured mandatory. Opetus design philosophy allows end user to define his own object taxonomy. End user can define tags and capture data without customization. Ex: while defining a trust or institute some customers may need to capture the historical data to publish on web site or to integrate ERP with their CMS. Taxonomy helps centralize data capturing and retrieval. Heterogeneous data sources can be avoided and information sanctity can be enhanced. Also lot of objects which are non textual (images/audio/video) needs tagging and cataloguing especially digital library. Opetus helps capture structured data with unstructured (content) and map them into relational entities.

OPETUS MASTER DESIGN PHILOSOPHY

State Machine: Object life cycle may involve many states and common used practice defines states of object using CHAR(1) data type, using characters such as ‘A’ and ‘D’ for active and inactive states. Proposed design case defines state machines as BIGINT with zero being active and non-zero values can indicate series of conditions under which object got inactive. For workflow design bit masked states can be used to check state of progress of a transaction based on stages completed or stages through which the object had to undergo transitions from its creation till completion

FOR TRIAL / DEMO CONTACT

India Offices:

MUMBAI

Mr. Bhavesh Gandhi

Contact (+91) 9819365466

bhavesh.gandhi@opetus.co.in

SURAT

Mr. Gitesh Modi

Contact (+91) 9898883050

gitesh.modi@opetus.co.in