tips and tricks to build high database applications · •large binary or text objects may be...

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Tips and Tricks to Build High Performance, Highly Available .NET Database Applications

Christian Shay Product Manager - .NET Technologies Oracle May 10, 2017

Confidential – Oracle Internal/Restricted/Highly Restricted


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Confidential – Oracle Internal/Restricted/Highly Restricted 3


In today’s session, we will cover….

• Key performance trouble spots and what to do about them

• “Low hanging fruit” – easy ways to improve performance

• What does it mean for a .NET database app to be “highly available”?

• What can I do to make my database application highly available?



Database Connections

• Connections are expensive!

– Can be on the order of seconds to establish

• Applications typically connect and disconnect repeatedly

• Easy Solution = Use a connection pool

– …and monitor that pool under load to learn ideal settings!




• ODP.NET connection pooling

– Min Pool Size = # connections at steady state or average load

– Max Pool Size = # connections at maximum capacity • Min and Max Pool Size always obeyed over other CP attributes

– See documentation for connection pooling parameter options

• ODP.NET performance counters

– Monitor with Windows Performance Monitor

– See documentation for details on how to enable

Example: ODP.NET Connection Pool



Code Example ODP.NET Connection Pooling

7

static void Main(string[] args) { string constr = "User Id=hr;Password=hr;Data Source=localhost:9999/PDBORCL.gse99.oraclecloud.internal;" + "Pooling=true;Min Pool Size=20; Max Pool Size=100;"; OracleConnection con = new OracleConnection(constr); con.Open();


Database Connections ODP.NET Connection Pool – Monitoring with Performance Counters




• Connection pool tagging

– Identify connections with specific settings using tags • i.e. American users or French users

• i.e. gold customers or platinum customers

– ODP.NET searches pool for matching tag and retrieves that connection to be opened

– No round trips to set session settings for every new connection

ODP.NET Connection Pool




• Run-time connection load balancing

– With RAC, Data Guard, GoldenGate, and GDS

– Automated load balancing at connection dispense time • Better performance because dispense based on real-time load

– Set “Load Balancing = true” in connection string

ODP.NET Connection Pool

10



• Connection Pool is on database server

• Connections can be shared across processes, across machines

• Oracle Database Resident Connection Pooling (DRCP)

Server Side Connection Pooling

11



• Watch out for lingering references to connections

– Garbage collector may not implicitly dispose objects when under heavy load • Increasing memory usage

• Excessive connections idling

• Close and Dispose objects to be safe

– Can use “Using” statement instead

12

References to connections


Code Example Disposing Objects Explicitly

13

OracleDataReader dr = cmd.ExecuteReader(); if (dr.Read()) { Console.WriteLine("Country Name: {0}", dr.GetOracleString(0)); } dr.Dispose(); p_country_id.Dispose(); cmd.Dispose(); con.Dispose();


Bind Variables

• Prevents re-parsing of frequently executed statements

– SQL, Stored Procedures

• Improves subsequent command executions

– Literal value changes forces a re-parse and re-optimization

– Literal values should become bind variables

• Executed statements stored in database

14


Code Example Bind Variables

15

StringBuilder sbSQL = new StringBuilder(); sbSQL.Append("select country_name "); sbSQL.Append("from countries "); sbSQL.Append("where country_id = :country_id"); OracleCommand cmd = new OracleCommand(); cmd.Connection = con; cmd.CommandText = sbSQL.ToString(); OracleParameter p_country_id = new OracleParameter(); p_country_id.OracleDbType = OracleDbType.Varchar2; p_country_id.Value = "UK"; cmd.Parameters.Add(p_country_id); OracleDataReader dr = cmd.ExecuteReader();


SQL Statement Caching

• Database retains frequently parsed statements up to a limit

– Prevents repeated parsing in server

• Best with bind variables

• ODP.NET: Self-tuned cache size – on by default

– No code changes needed

16


Data Retrieval

• Control how much data is retrieved per DB roundtrip

– Too much data retrieved – excessive client-side memory or processing

– Too little data retrieved – additional round trips

17


Data Retrieval

• Use OracleCommand.RowSize and OracleDataReader.FetchSize to control result size

– RowSize populated after statement execution • Can be set dynamically at run-time

– FetchSize can be set as multiple of RowSize

ODP.NET Fetchsize

18


Code Example ODP.NET Fetchsize

19

//Fetchsize = 1 row per fetch OracleDataReader reader = cmd.ExecuteReader(); reader.FetchSize = cmd.RowSize * 100; while (reader.Read()); reader.Dispose(); //Fetchsize = 100 rows per fetch OracleDataReader reader = cmd.ExecuteReader(); reader.FetchSize = cmd.RowSize * 100; while (reader.Read()); reader.Dispose();


Data Retrieval

• Large binary or text objects may be costly to download

• Options:

– Retrieve pointer to the LOB (no data) when retrieving a row

– Download a small portion of the LOB with the row if needed • Image file header

• Document heading

– Download entire LOB with every row

– LOB manipulation via SQL or Stored Procedures • Instead of bringing data down and then sending it back up again

– LOB Random Access, if available

Large Objects (LOBs)

20


Data Retrieval

• OracleCommand.InitialLOBFetchSize

– Default is zero, no LOB data returned when returning row (just pointer)

– Retrieve a chunk using OracleClob and OracleBlob classes Read method

– Value of -1 returns all LOB data with every row

• Update/Insert/Delete SQL statements acting on LOBs

– Modify LOB without retrieving the data to the client side • Uses LOB locator

ODP.NET and LOBs

21


Data Updates

• Statement batching

– Use DataAdapter.UpdateBatchSize to batch updates from DataSet

– ODP.NET uses an anonymous PL/SQL block to do it

22


Data Retrieval and Updates

• Use performant arrays to pass mass data to stored procedures/functions

– For Oracle, use PL/SQL associative arrays. DON’T user VARRAYs or NESTED TABLE

• Use parameter array binding

– Useful if executing the same statement multiple times

– Bind variables are the same, variable values can be different

– One execution for each element in the bound array

Mass Data Movement with Arrays

23


Code Example ODP.NET Associative Arrays

24

cmd = new OracleCommand("MYPACK.MYSP", con); cmd.CommandType = CommandType.StoredProcedure; OracleParameter param1 = cmd.Parameters.Add("param1", OracleDbType.Varchar2); OracleParameter param2 = cmd.Parameters.Add("param2", OracleDbType.Varchar2); param1.CollectionType = OracleCollectionType.PLSQLAssociativeArray; param2.CollectionType = OracleCollectionType.PLSQLAssociativeArray; param1.Direction = ParameterDirection.Input; param2.Direction = ParameterDirection.Output;


Code Example ODP.NET Associative Arrays

25

param1.Value = new string[3] { "Oracle", "Code", "2017" }; param2.Value = null; param1.Size = 3; param2.Size = 3; param1.ArrayBindSize = new int[3] { 20, 20, 20 }; param2.ArrayBindSize = new int[3] { 20, 20, 20 }; cmd.ExecuteNonQuery(); for (int i = 0; i < 3; i++) { Console.Write((param2.Value as OracleString[])[i]); }


Data Retrieval and Updates

• Offload work to the database

– Reduce data that needs to be shipped back and forth

– Utilize array binding when available

• Database Cursors – A pointer to the data instead of the data itself

– Fetch the data only when you need it

Stored Procedures and Functions

26


Oracle REF Cursors

• OracleRefCursor class

• Defers result set retrieval until needed

• Retrieve data as needed

– Control data retrieved via FetchSize

– Fill a DataSet with just a portion of the REF Cursor result

• Usage

– Can create REF Cursors as part of an anonymous PL/SQL block

– Can return REF Cursors from stored procedures

– Can pass REF Cursors to database as input stored procedure parameters

27


Caching

• Good use cases

– Few and mostly read-only tables

– Few rows updated for each table

– Replace app logic that checks for updates to ensure user has most up to date data

• Bad use case

– Do not use it for constantly changing data

When should you use a cache?

28


Better Caching Solutions

• DB responsible for change notification

– DB server = the one source of truth

• Overcomes typical disadvantages

– Network traffic – more throughput • Only generates traffic if database change occurs

– Higher scalability • Only updates if change occurs and data is read

– Timing – greater accuracy • As soon as change occurs, clients receive acknowledgement

29


Oracle .NET Caching Solutions

• Oracle .NET client-side DB caches

– Client Result Cache

– Continuous Query Notification (CQN) – customizable cache

– TimesTen In-Memory Database

• Automatically updates/alerts client cache upon server changes

30


Oracle Client Result Cache

• Automatically updating query result set cache

– Invalidation notifications piggyback on existing client round trips

• Easy to use

– Use a SQL hint if it is not turned on for all queries

– Example: select /*+ result_cache */ first_name, last_name, salary from employees

• Snapshot consistent

– Cache refreshes without user intervention

31


Client Result Cache

• To use cache results, the following must match

– SQL text

– Bind values

– Session settings

• Different sessions with same user settings share cache results

– Must exist in same client process

• Different users do not share the same result sets

How It Caches

32


Code Example ODP.NET Client Result Cache

33

con2.ConnectionString = conString; con2.Open(); OracleCommand cmd2 = con2.CreateCommand(); cmd2.CommandText = "select /*+ result_cache */ first_name, last_name, salary from employees "; OracleDataReader reader2 = cmd2.ExecuteReader();


Oracle Continuous Query Notification (CQN)

• Programmatic control over cache notifications and updates

• Also known as Database Change Notification

• Benefits over Client Result Cache

– More control over how cache behavior • What if only a subset of the cached data is required?

• How long should a query be cached?

• Do I want additional logic executed when the cache is refreshed?

• Requires significant customization

– CQN provides cache infrastructure

More Control Over Cache Behavior

34


Notification

Queue

Application

Listener

OracleCommand

OracleDependency

Data Dictionary

OnChange

Add Dependency

Execute()

Notification Request

TABLE

Data Change


Oracle TimesTen In-Memory Database

• Fully featured relational database

• Oracle compatible SQL and PL/SQL

• Persistent and durable

– Transactions with ACID properties

– Flexible options for durability

• Exceptional performance

– Instantaneous response time, high throughput, embeddable

• ODP.NET TimesTen provider for TimesTen DB

– Same ODP.NET APIs

Memory-Optimized Relational Database

36


Application High Availability

•Applications hang on TCP/IP timeouts

•Connection requests hang when services are down

•Not connecting when services resume

•Receiving errors during planned maintenance

•Attempting work at slow, hung, or dead nodes

What if my application does not address high availability?

37


Application High Availability

• What is application HA?

– Applications handle DB and network failures transparently

– Developers add code to handle failures gracefully specific for their use case

• Benefits – Improved end user service levels

– Less administrative time spent handling failover

38

.NET


Fast Application Notification (FAN)

39


FAN in a Nut Shell

• FAN events specify what changed, where, when

– Server initiated messages

– ODP.NET apps receive these messages

– Adjusts connection behavior automatically


FAN • Rapid notification about DB and service state changes

• Event types

– Down • Received quickly to invoke failover

– Planned Down • Drains sessions for planned maintenance with no user interruption

– Up • Re-allocates sessions when services resume

– Load % • Advice to balance sessions for RAC locally and GDS globally

– Affinity • Advice when to keep conversation locality


• FAN alerts ODP.NET of impending downtime or service relocation

– ODP.NET stops allocating new connections to that DB node

– Idle connections closed

– Connections returned to the pool are closed

• Pool draining feature – Part of Fast Connection Failover (FCF)

• Shutdown commences when all connections are closed – Optionally set a maximum timeout upon which shutdown occurs

• End user experience: Little to no disruption

Planned Maintenance Outage Connection Pool Scenarios


Pool Draining

• Set “Pooling=true” (default)

• Set ODP.NET connection string attribute “HA Events = true”

– In ODP.NET 12.2, set to true by default

• Recommend using Oracle Database 11.2.0.4+ and ODP.NET 11.2.0.4+

43

ODP.NET settings


Pool Draining

• Data Guard 12.2 adds support for pool draining

– Ex. SWITCHOVER TO <database name> [WAIT <timeout in seconds> ];

• Switchovers require primary to shut down before standby comes up

– End users may request a connection during that time

– With no available DB to serve the request, users receive timeout errors

– But this can be prevented by “pausing” ODP.NET requests

Data Guard Switchovers

44


Pool Draining

• ODP.NET ServiceRelocationConnectionTimeout setting blocks incoming connection requests until standby is up (or timeout is hit)

– Eliminates connection timeout errors

– Set in the settings section of the .NET config file

– Default = 90 seconds

• Oracle TNSNAMES settings complementary to blocking connection requests

– RETRY_COUNT • Number of times ADDRESS list traversed before connection attempt terminates

– RETRY_DELAY • Delay in seconds between subsequent retries for a connection

Data Guard Switchovers

45


RETRY_COUNT and RETRY_DELAY TNSNAMES.ORA Sample File

alias =(DESCRIPTION_LIST =

(DESCRIPTION=

(RETRY_COUNT=10)(RETRY_DELAY=5) (ADDRESS_LIST=(ADDRESS = . . .)(ADDRESS= . . .)) (CONNECT_DATA=(SERVICE_NAME=hr_svc))) .

(DESCRIPTION=

(RETRY_COUNT=10)(RETRY_DELAY=5) (ADDRESS_LIST=(ADDRESS = . . .)(ADDRESS=. . .)) (CONNECT_DATA=(SERVICE_NAME=hr_svc2))))

Retry while service is unavailable

46


Unplanned Outages

47


Fast Connection Failover

• FAN alerts ODP.NET of immediate DB node, service, etc. failure

– ODP.NET stops allocating new connections to failed entity

– Idle connections closed

– Active connections are closed

• Benefits (assuming another DB instance is available)

– New users do not pick up invalid connections

– No need to clear pools explicitly

Unplanned Outage

48


Transparent Application Failover

• Upon failure, automatically reconnect to an available DB instance (if available)

– .NET application can receive callback notifications if desired

• Reposition cursors to position they were on prior to failure

• Restore session state – new in 12.2

– i.e. NLS, timezone, action , module, etc.

• Replay initial transaction statement – new in 12.2 – Any DML or stored procedure

Unplanned Outage

49


Application Continuity (AC)

• Recover active queries and transactions seamlessly after unplanned DB outages from a recoverable error

• Masks hardware, software, network, storage errors, and timeouts

– Available with RAC, RAC One Node, and Active Data Guard

• Benefits

– AC replays in-flight work on recoverable errors

– No additional ODP.NET coding required

– Transparent to end users (except for a slight delay)

Unplanned Outage


AC

• Set “Pooling=true” (default)

• “Application Continuity=true” (default) in the connection string

– New ODP.NET 12.2 attribute

• Requires ODP.NET 12.2 and Oracle DB 12.2

ODP.NET Settings


Transaction Guard

• ODP.NET can determine whether a transaction committed even upon a DB failure without custom coding

• Benefit

– Ensures accurate knowledge of transaction outcome

• App can query transaction outcome

– OracleConnection properties return transaction status

– OracleLogicalTransaction class

• Requires Oracle Database 12c and ODP.NET 12c


Transaction Guard Scenario

1. ODP.NET receives FAN down event or error

2. IsRecoverable=false roll back IsRecoverable=true re-submit

3. To re-submit, retrieve OracleConnection.OracleLogicalTransaction

4. Retrieve transaction status with OracleLogicalTransaction.GetOutcome.

5. If committed and completed, done. If not committed nor completed, re-submit.


Transaction Guard ODP.NET Catch Block Code Sample

54

catch (Exception ex) { if (ex is OracleException) { // It's safe to re-submit the work if the error is // recoverable and the transaction has not been committed if (ex.IsRecoverable && ex.OracleLogicalTransaction != null && !ex.OracleLogicalTransaction.Committed) { // safe to re-submit work } else { // do not re-submit work } } }


Oracle Compute Cloud Service

• Windows 2012 R2 and Windows 2008 R2 VMs available from Oracle Cloud Marketplace to

– Deploy to Oracle Compute

• Deploy and configure IIS, .NET, and ODP.NET apps to Oracle Compute

– How To White Paper: Deploying Microsoft Web Application Server on Oracle Compute Cloud Service

– http://bit.ly/2l9PJox

Deploy .NET applications to Oracle IaaS on Windows

https://cloud.oracle.com/marketplace/en_US/listing/7055818

https://cloud.oracle.com/marketplace/en_US/listing/7630342

http://www-content.oracle.com/technetwork/topics/dotnet/tech-info/oracle-iaas-dotnet-iis-twp-3112591.pdf

http://www-content.oracle.com/technetwork/topics/dotnet/tech-info/oracle-iaas-dotnet-iis-twp-3112591.pdf


Oracle Database Exadata Express Cloud Service

• Use ODP.NET and ODT 12.1 for Oracle Public Cloud or higher

– ODT for VS 2015 and VS 2013

– Managed and unmanaged ODP.NET

• How to connect: – Developing .NET Applications for Oracle Database Exadata Express

Cloud Service

– http://bit.ly/2mr0kM3

– Uses Oracle Net Services with Oracle Wallet to secure connection

Connect from On-premises

http://www.oracle.com/technetwork/topics/dotnet/tech-info/dotnetcloudexaexpress-3112654.html






Oracle Database Cloud Service (non-Exadata Express)

• Use ODP.NET and ODT 12.1 for Oracle Public Cloud or higher

– ODT for VS 2015 and VS 2013

– Managed and unmanaged ODP.NET

• How to connect: – Developing .NET Applications for Oracle Database as a Service

– http://bit.ly/2lRvSsZ

– Secure Shell (SSH) required to secure connection • Use SSH client to create tunnel, such as PuTTY

• PuTTY can also generate private and public SSH key pair

Connect from On-premises

http://www.oracle.com/technetwork/topics/dotnet/dotnetdbaas-3208838.html


Additional Oracle .NET Resources

OTN otn.oracle.com/dotnet

Twitter twitter.com/OracleDOTNET

YouTube youtube.com/OracleDOTNETTeam


Safe Harbor Statement

The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


tips and tricks to build high database applications · •large binary or text objects may be...

Documents