cs317 file and database systemsmercury.pr.erau.edu/~siewerts/cs317/documents/lectures/...[c++...
TRANSCRIPT
![Page 1: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/1.jpg)
August 27, 2018 Sam Siewert
CS317File and Database Systems
Lecture 1 - Introduction
![Page 2: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/2.jpg)
Dr. Sam SiewertJohnson Space Center– Shuttle Ascent and Entry GN&C, Mission Control Center
JPL AI Group, CU Space Grant and Ball Aerospace
CU Boulder Adjunct Professor, CTO, Architect, Developer/Engineer in Local Start-ups and Intel Architecture / Labs
U. of Alaska, Assistant Professor, Computer Systems Engineering, Alaska Space Grant, DHS ADAC
Embry Riddle Prescott, Assistant Professor, Computer, Electrical and Software Engineering, ARI Drone Net
Sam Siewert 2
https://en.wikipedia.org/wiki/Intel
![Page 3: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/3.jpg)
Preparation for CS 317 - SE’sCS118 (Python), CS125 (C), CS315 (Data Structures)Goes well with OS (Linux)C++ not required, but OO will be discussed
Sam Siewert 3
File systems - Linux
Physical DBMS - C
Logical DBMS - SQL
Python MySQLConnectors (Apps)
C Connectors
[C++ - optional]
[JavaScript/PHP]
![Page 4: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/4.jpg)
Preparation for CS 317 - CybersecuritySpring CS 317 Section with Cybersecurity Emphasis
CS118 (Python), CS125 (C), CS315 (Data Structures)Goes well with OS (Linux)C++ not required, but OO will be discussed
Database Security Notes
Sam Siewert 4
Fall - Emphasis on Design for Security
Secure Connector Apps (SQL Injection)Secure Design - Stored Programs, Views,DBA Access Control (Grants, Priveleges)
![Page 5: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/5.jpg)
Sam Siewert 5
Related Industry BackgroundGeneral Experience (~25 Years in Embedded and Scalable Systems)– 12 Years NASA JSC, NASA JPL, CU, Ball
Aerospace– 12+ Years Commercial Telecomm,
Storage/Networks, Embedded, Digital Video
Software Engineering– NASA Johnson and JPL (Shuttle Ascent/Entry
Guidance, Deep Space)– Intel, Emulex, Start-ups
CTO– RAID, HPC– iSCSI, Fiber Channel, Infiniband– Large Boxes of Disk Drives - JBODs– Large Boxes of SSDs - JBOF?
Consulting– Graphics, Storage and Networking– Advanced RAID and Erasure Codes
RAID
SAS
![Page 6: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/6.jpg)
File and DB Systems Experience2002-2006 – Emulex (Intel), Chip-down Fiber Channel
2006-2010 – Atrato Inc. (Start-up), Scalable HDD/SSD Hybrid RAID Systems (10GE, 4/8G Fiber Channel)
2010-12 – Intel Corporation – Beyond Software RAID Research
Low-Level! - Block Layer Below Oracle, MS SQL, MySQL
Self-Taught on MySQL (Easy to Learn, Hard to Master)
Sam Siewert 6
![Page 7: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/7.jpg)
Learning ObjectivesIntroduction - Block Storage, Structured and Unstructured Data
– Block Storage Devices [Disk Drives, SSD, Persistent Memory]– Split between Files and Databases [Integration of Both]
DML - Data Manipulation using SQL (ISO Standard)
DDL - Data Definition Language, Database Design with SQL– Logical Design [Schema]– Physical Design [Hosting DB]
Theory of Databases– Relational, – OODBMS (SQL Extensions, C++ Alternatives)– NoSQL
DBA - Database Administration and Security
File and Database Systems– Physical Implementation and Scaling [Block Partition, File], – Indexing [ISAM, B Tree, B+ Tree, R-Tree], – Network Access [Connectors], – Web Front Ends and Connectors (C, Python, C++, JavaScript, PHP)
Sam Siewert 7
![Page 8: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/8.jpg)
Current Research - Data AnalyticsBlockchain for UAS, Machine Learning for UASERAU ICARUS Drone Net [2016 – present]5th Big Data Silicon Valley, Drone Net, NASA UTM
Sam Siewert 8
Machine Vision
Machine Learning
Big Data Analytics
GP-GPUs, FPGA
Google TPU
NAS Storage
Structured & Unstructured Data
![Page 9: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/9.jpg)
Course Goals and OutlineTextbook - Database Systems: A Practical Approach to Design, Implementation, and Management, 6th Edition, Thomas Connolly and Carolyn Begg, ISBN 978-0-13-294326-0 (Amazon, Publisher)
… introduces the theory behind databases in a concise yet comprehensive manner, providing database design methodology that can be used by both technical and non-technical readers. The methodology for relational Database Management Systems is presented in simple, step-by-step instructions in conjunction with a realistic worked example using three explicit phases—conceptual, logical, and physical database design.
http://mercury.pr.erau.edu/~siewerts/cs317/
Current Syllabus
Sam Siewert 9
New Textbook!Reference Only forSecurity Considerations
Primary TextbookProfessional IT Referencefor Oracle Enterprise Systems
![Page 10: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/10.jpg)
Why SQL and MySQLWide Use, Partially Open Source (Alternate – MariaDB)RDBMS Majority Market Share – DB-Engines RankingAlternatives to R-DBMS1. NoSQL - MongoDB, Redis, Cassandra2. Distributed Data - Blockchain (Linux Foundation - Hyperledger)3. OODBMS (C++ Connectors)
Sam Siewert 10
![Page 11: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/11.jpg)
File System and DBMS CodeTake Poll on Who has had CS125 [C programming], CS225 [C++ programming], CS315 [Data structures]
ALL must Learn SQL and Use PRCLab1 Web Interface and Command Line Interface
Assignment #5 Options [Teams]– Dig Deeper into DBA SQL Security [Non C/C++ Programmer
Option]– DBE Design to Deploy a DDL Schema and Populate with DML Test
Data – Build a Front End [C, Python, JavaScript, PHP]– Alternative DB - Blockchain, NoSQL (SE Workstation), Mobile
(SQLite)
ALL must Learn R-DBMS and Logical Design Concepts
Sam Siewert 11
![Page 12: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/12.jpg)
PRClab1 – Linux for SQL and COption #1 – Use PRClab1, prclab1.erau.edu via SSH– Unix Account LDAP - Ernie login– Recommend MobaXterm access– SQL and C Code Development (GCC, Make, etc.) - Linux Help – MySQL Workbench (on STEM Room 110 and Bldg #16 Lab)– http://prclab1.erau.edu/phpmyadmin/ (PHPAdmin login)– IT Support [email protected] – For Help with PRClab1
login, etc.
Option #2 – Use X2Go and SE Workstation– Must Have Windows, Macintosh or Linux Personal PC and install
X2Go– SE Workstation Access Instructions - Ryan Sutton– Maria DB instead of MySQL (possible)– Final alternative project with Maria, NoSQL, or Blockchain
Sam Siewert 12
![Page 13: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/13.jpg)
Sam Siewert 13
Why Work with Linux and Virtual Box…From Mobiles to Super-Computing to Datacenters
From Android Mobiles to GIS and Digital Video Services
Huge Value in Open Source Drivers, Tools, and Applications – Speeds Up Time to MarketOracle Virtual Box – Great Cross OS Test Environment
http://www.top500.org/
Tianhe – 33+ PflopsGoogleEmbedded/Mobile
![Page 14: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/14.jpg)
How We’ll Do It1/3 Knowledge, Concepts, Theory – Lectures/Reading– Lectures related to Connolly-Begg Textbook and Instructor’s
Experience– Logical and Physical DBMS– Review of File systems and Key Differences with DBMS– LAMP – Linux, Apache, MySQL, PHP– Discussions
1/3 Practice – PRClab1 or SE Workstation LAMP– SQL, C code, Python, [JavaScript and PHP if you wish]– Building, Modifying, Querying and Using MySQL (MariaDB)– Logical RDBMS Design, Implementation and Management– Physical RDBMS Data structures (B-tree), Physical Ext-n I-node
structures and debugfs
1/3 Project [DBA, DBE]– Group Project to Build a Significant Database on PRClab– Final Assignment
Sam Siewert 14
![Page 15: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/15.jpg)
Sam Siewert 15
AdministriviaIntroductions– Instructor (Office Hours) – Office
Hours– Students (Introductions) – Please do
Collaborate, but cite well!– Policies
Mercury and CANVAS– CANVAS Assignment Management– Access via ERNIE -
https://ernie.erau.edu– CS 317 Mercury Website
Course Information– Attendance & E-mail list (please sign
up on sheet being passed around)– Mercury Lecture Notes– Will post Assignments on CANVAS
Must have PRClab1 account -prclab1.erau.edu, http://prclab1.erau.edu/phpmyadmin/
PHP WebAccess
SQL CLIAccess
![Page 16: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/16.jpg)
25 August 2014 Sam Siewert
MySQL on LinuxLAMP vs. Connector App
onFile system / Block Storage
Introduction Session(File System vs. DBMS)
Linux OS
SAN, NAS
MySQL
PHP
Apache
Web Client
C, Python App
![Page 17: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/17.jpg)
4-tier PRClab with MyPHP LAMPFirst tier– Your client Web browser
Second tier– Apache Web server
Third tier– PHP interface between Web
server and MySQL
Fourth tier– MySQL / Linux
Sam Siewert 17
ApacheWeb Server
AdminerPHP
MySQL
Storage
![Page 18: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/18.jpg)
3-tier PRClab with Connector AppFirst tier– Your App - GUI, Mobile, CLI
Second tier– Connector tunnel/path
between App and MySQL
Third tier– MySQL / Linux
Sam Siewert 18
PRClab1App
Connector(C, C++, Python)
MySQL
Storage
![Page 19: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/19.jpg)
File System Concepts – Review?Name SpaceMeta-DataFile-DataI-nodesBlocks with Byte Offsets Belong to Name Space Files
Sam Siewert 19
![Page 20: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/20.jpg)
Volume Use by File and Folder
Sam Siewert 20
![Page 21: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/21.jpg)
Make a File Systemmkfs –t ext3 /dev/sdb1Install Name Space and Metadata on Volume
Sam Siewert 21
![Page 22: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/22.jpg)
Use a File SystemBlock Access via dd in Linux – e.g. Clone VolumeName Space Access via Shell or Browser
Sam Siewert 22
![Page 23: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/23.jpg)
Sam Siewert 23
MySQL Command Line and PHPAdminWhy and How is a RDBMS Different?
Login to MySQL via Webhttp://prclab.pr.erau.edu/adminer.php
Sign up for Unix account today!!!ExploreRead Tutorial Docs -http://dev.mysql.com/doc/index-topic.html
Login to MySQL via SSH Shell (MobaXterm) - TA/DBA Account Creation E-mail coming …
In Case You Want More (NOT REQUIRED)Install Oracle Virtual BoxInstall Centos or Ubuntu LTSInstall MySQLInstall Apache and PHPSetup PHPAdmin on localhost
![Page 24: CS317 File and Database Systemsmercury.pr.erau.edu/~siewerts/cs317/documents/Lectures/...[C++ programming], CS315 [Data structures] ALL must Learn SQL and Use PRCLab1 Web Interface](https://reader033.vdocuments.mx/reader033/viewer/2022042306/5ed2187e328bb40b05486300/html5/thumbnails/24.jpg)
Next Time …DBMS vs. File System (read DBMS Chapter #1) – Come to Class Prepared to Discuss– Why is DBMS Better then File System– Why do we Use File Systems still?– What is NoSQL? (Not in Book, http://en.wikipedia.org/wiki/Nosql)– Structured vs. Unstructured Data? – For Discussion
Assignment #1 Discussion– I will Post Every Other Tuesday, We’ll Discuss, Due Following
Week on Friday– Late Assignments – 10% Penalty for Monday Turn-in, After
Monday, only with Instructor Permission
Sam Siewert 24