Firebird 3.0Initial Review

Development priorities● Primary goals

SMP and multi-core friendly multi-threaded engine with the shared cache

Renewed architecture for the next versions

● Secondary goals Improved performance and scalability More security options Better query optimization Extended monitoring abilities SQL language enhancements

Shared cache● Must have

Shared page cache with scalable synchronization Support for both internal (cross-thread) and external

(cross-process) locking

● Nice to have Plan A: shared metadata cache,

shared compiled requests cache Plan B: no shared metadata,

but reduced memory usage(release cached requests as soon as desirable)

Cache Architecture

Database

Thread 1

Thread 2

Thread N

Shared Cache

Process 1

Thread 1

Thread 2

Thread N

Shared Cache

Process 2

Thread 1

Thread 2

Thread N

Shared Cache

Process N

Locking Locking

Server Architecture● Unified but flexible

Engine library supporting single major ODS plus server (aka network listener) executableloading the engine library

Shared vs exclusive database access:configurable (per database) via firebird.conf

Execution models available for the shared access and configurable at the server level (using a command-line switch):– process based (former Classic)– thread based (former SuperClassic)

Embedded access may also be shared or exclusive

Server Architecture

Legacy engineprovider

ODS11database

V3.0 engineprovider

Remote redirectorprovider

Dispatcher(Y-valve)

ODS12database

Server(network listener)

Network

Network

Architecture in Older Terms● SuperServer

Exclusive database access, multi-threaded mode Maximum performance

● SuperClassic Shared database access, multi-threaded mode Allow other applications to access the database

● Classic Shared database access, multi-process mode Maximum reliability (crash tolerance)

ODS 12● Physical structure

Flag to mark data pages as not having any garbage(improves sweep performance)

Inventory of the page ↔ SCN relationships(improves physical backup performance)

RLE compression runs are extended from 128 bytes to 32K (less storage for declared long but de-facto short or empty strings)

● Logical structure New system and monitoring tables

Optimization● Algorithms

Cost based approach wherever possible Efficient joining for streams containing aggregates,

unions and procedures INNER JOIN for dependent (via input parameters)

procedures now works properly Hash joins, outer merge joins, indexed full outer

joins● Cached invariant subqueries Advanced (much more detailed) plan output

Advanced plan output

SELECT statement -> First [10] -> Sort [SUM, O_ORDERDATE] -> Aggregate -> Sort [L_ORDERKEY, O_ORDERDATE, O_SHIPPRIORITY] -> Inner Loop Join -> Filter -> Table «ORDERS» Access By ID -> Bitmap -> Index «ORDERS_ORDERDATE» Range Scan -> Filter -> Table «CUSTOMER» Access By ID -> Bitmap -> Index «CUSTOMER_PK» Unique Scan -> Filter -> Table «LINEITEM» Access By ID -> Bitmap -> Index «LINEITEM_PK» Unique Scan

Optimization● Statistics

Table level data: page count, row count, average compression ratio, etc

Index level data: tree depth, page count, average compression ratio, clustering factor, etc

Value distribution histograms Refresh detalization: complete (column level) or

partial (table level) Manual or automatic (background) refreshing

SQL features● Already implemented

External procedures, functions and triggers PSQL scalar functions PSQL packages (similar to Oracle) DDL triggers Identity columns Window functions Scrollable (bi-directional) cursors in PSQL Parameterized user-defined exceptions

SQL features● Planned

Bi-directional cursors in API Enhanced (fixed) long numeric arithmetics

in Dialect 3 Basic optimizer hints The «unstable cursor» issue to be fixed Timeouts for queries, transactions, connections Extended length limit for SQL queries, plans and

result sets

Security● Network level

Pluggable authentication Over-the-wire encryption

● Database level Optional per database authentication

(users stored inside the database) Page level encryption GRANT ROLE TO ROLE Implicit roles (aka user groups) Permissions for DDL operations

Release Schedule● Maintenance

2.0.7 — Q2-Q3 2011,likely to be the last one in the v2.0.x series

2.1.4 — Q1 2011,2.1.5 to appear the next year

● New versions 2.5.0 has been released,

2.5.1 to follow Q2 2011 3.0 Alpha 1 — Q3 2011

Questions?