Download - Logging Last Resource Optimization for Distributed Transactions in Oracle Weblogic Server
Logging Last Resource Optimization for Distributed Transactions in
Oracle WebLogic Server
T. Barnes, A. Messinger, P. Parkinson, A. Ganesh, G. Shegalov, S. Narayan, S. Kareenhalli
OLTP: Online Transaction Processing
Transaction is an ACID contract● Atomic – all or nothing
● Consistent – from the application perspective
● Isolated – masked concurrency through locking or snapshots
● Durable – once committed changes survive subsequent failures
Checking = 2000Savings = 8000
Checking = 1000Savings = 9000
time
beginc -= 1000s += 1000commit
OLTP: Single Resource● A and D are typically implemented using Write-Ahead Logging
● Transaction recovery is “simple”: REDO phase, UNDO phase.BEGIN TRANSACTION
/* LSN = 1: log for undo and redo in MM buffer*/
UPDATE Accounts SET balance = balance – 1000 WHERE Number = 1
/* LSN = 2: log for undo and redo in MM buffer*/
UPDATE Accounts SET balance = balance + 1000 WHERE Number = 2
/* LSN = 3: log commit and force (5-6 orders slower)*/
COMMIT TRANSACTION
Accounts LSN=1
1 1000
2 8000
Accounts LSN=2
1 1000
2 9000
Accounts LSN=0
1 2000
2 8000
OLTP: Distributed / Two-Phase Commit
Transaction Coordinator
Resource 1 Resource 1
prepare --> force-log prepare force-log prepare
<-- OK <-- OK
commit --> force-log commit force-log commit
<-- ACK <-- ACK
Like a wedding ceremony
Coordinator: Will you ...? (prepare)
Resource: I will (OK)
Coordinator: I pronounce you … (commit)
2PC is A CI D
● 2PC is not about Concurrency Control.
● 2PC transaction is therefore
○ Globally Atomic
○ Locally Isolated
○ Locally Consistent
○ Globally Durable
OLTP: Queued Transactionsclient` app server database
begin transaction
req_q.enqueue(req1)
commit transaction
begin transaction
creq = req_q.dequeue()
resp = creq.execute()
res_q.enqueue(resp)
commit transaction
begin transaction
resp = res_q.dequeue()
process(resp)
commit transaction
WebLogic Server: Java EE++
● App containers: Web (Servlets, WS), EJB, app clients
● Services: Messaging (JMS),
Transactions (JTA), Database (JDBC), …
Example: Queued Transactions (JEE)
@MessageDriven(
mappedName="jms/inboundQueue“
activationConfig = {@ActivationConfigProperty(
propertyName = “connectionFactoryJndiName",
propertyValue = “jms/inboundConnectionFactory"
)}))
@TransactionAttribute(TransactionAttributeType.REQUIRED)public class OrderMDB implements javax.jms.MessageListener {
@Resource javax.jms.Queue outboundQueue; @Resource javax.jms.ConnectionFactory outboundCf; @Resource javax.sql.DataSource orderDataSource; public void onMessage(Message message) { java.sql.Connection jdbc = orderDataSource.getConnection();
javax.jms.Connection jms = outboundCf.createConnection();
// update SQL via JDBC, notify via JMS connections …
}
}
“School” Presumed Abort 2PC TM Resources
Timeline
prepare
yes
force-log prepared
commit
force-log commit
all-prepared: force-log commit
all-commit: log end
ack
2n+1 writes, 4n messages
PA2PC (1): TM (Coordinator)
PA2PC (2): Resource (Participant)
“Real Life” XA 2PC
TM Resources
Timeline xa_prepare
ack_preparedforce-log prepared
xa_commitforce-log commit
all-prepared: force-log commit
all-commit: log end
ack_committed
xa_startack_started
ack_endedxa_end
2n+1 writes, 8n messages
Standard 2PC Optimizations
● 1PC: if only one resource enlisted, prepare skipped
● Read-Only: if voted read-only, commit skipped
● XA ceremony of xa_(start|end) is always present
Nested 2PC: Coordinator Role Transfer
TC Res2 Res3
prepare commit
commitcommit
● Last Resource is committed in one phase
● 2n messages/ 2n-1 forced writes
● Known topology: linked Databases
[Gray’78]
p
c
c
WebLogic Design Constraints and Goals
● No control over foreign XAResource, TM and topology
● Broadband: minimize blocking RPC, not messages
● Unneeded XA on Res3: save xa_start, xa_end
Typical WLS Deployment
● JMS and TM share the same FileStore
● Collocated JMS connection cost is negligible
● JDBC Datasource is remote: blocking RPC
● DB internal resources (locks, latches, etc.) are more
expensive and JEE is not a single client
● Outbound JMS notifies about a JDBC update
● Ideally: JDBC updates visible before JMS updates
JDBC as Logging Last Resource
● User enables a non-XA JDBC Datasource as LLR
○ LLR table WL_LLR_<server> in the DS schema
○ No XA overhead for the LLR
● TM log is local log UNION LLR table log
○ WLS does not boot if any LLR table is unavailable
● Restriction: 1 LLR datasource / Transaction
● No coordinator transfers as in Nested 2PC
XA 2PC Commit with LL Resource
1. Prepare concurrently all non-LLR XAResources
2. Insert XID into the LLR table
3. Commit the LLR-Resource
4. If 3 is successful, commit non-LLR XAResources
5. Lazy garbage-collection of 2PC records of completed
transactions is piggybacked on future LLR transactions
LLR Failure Recovery
● Failure before LLR.commit() => global abort
● Failure during LLR.commit() => similar to media failure
○ Wait until LLR Datasource / table is available for read
○ Presence of the LLR commit log decides the global outcome
○ If unavailable for AbandonTimeoutSeconds log abandoned
● JVM/OS crash: TM scan local log UNION LLR
○ Usual transaction outcome resolution
● 2PC recovery guarantees are not compromised
LLR Savings
Back-of-the-envelope for the single-threaded case with Jeff Dean’s numbers [Google key notes]: ● xa_start (RPC),
● xa_end (RPC),
● xa_prepare (RPC + force-log)
● Insert into LLR table + commit done via single RPC
------------------------------------------------4xRTT + 1xDiskSeek = 4x500,000ns + 10,000,000ns = 12 milliseconds
LLR in DS Wizard: Non-XA Driver
LLR in DS Wizard: Safe unlike Emulate
Research Workload EAStress2004 [SPECjAppServer’04]
EAStress2004 Benchmark HW/Setup
Driver 1 (3x Dealer, 3x Manufacturing)
Driver 2 (3x Dealer, 3x Manufacturing)
External Supplier Emulator
2x Quad Core 3.7Ghz x86_64, 2MB L2, 8GB RAM
2x Quad Core 2.7Ghz x86_64, 2MB L2, 16GB RAM
2x Quad Core 2.7Ghz x86_64, 2MB L2, 16GB RAM
System Under Test
Oracle WebLogic Server 11gR1 (Middle Tier)
Oracle RDBMS 11gR1 EE(Database Tier)
2x Quad Core 2.83Ghz x86_64, 6MB L2, 16 GB RAM
4x Quad Core 2.7Ghz x86_64, 2MB L2, 64GB RAM
Performance Evaluation (Utilization)
MDB scenarioMT = WLS on JrockItDB = Oracle Database
EAStess2004 v1.08, IR = 700 (not reviewed by SPEC)
Performance Evaluation (Response Time)EAStess2004 v1.08, IR = 700 (not reviewed by SPEC)
Purchase Manage Browse Manufacturing
XA 4.20 2.40 5.40 3.75
LLR 1.50 1.20 1.90 3.00
improvement 2.8x 2x 2.8x 1.25x
Future Improvements (probably in 12c)
● LLR does not detect Read-Only
● Transaction GUID instead of LLR table for Oracle
Thank You. Questions?
oracle.com/weblogicoracle.com/benchmarks
WebLogic FileStore
● XA-capable KV store on local file system
● Mime design: allocate under write-head
○ fast writes
○ slow recovery
○ works well up to a couple of GiB
● Transactional use: for JMS messages and JTA logs
● Non-transactional use: Diagnostics and Config