limitations of concurrency in transaction processing

28
Limitations of Concurrency in Transaction Processing PETER FRANASZEK and JOHN T. ROBINSON IBM Thomas J. Watson Research Center Given the pairwise probability of conflict p among transactions in a transaction processing system, together with the total number of concurrent transactions n, the effective level of concurrency E(n,p) is defined as the expected number of the n transactions that can run concurrently and actually do useful work. Using a random graph model of concurrency, we show for three general classes of concurrency control methods, examples of which are (1) standard locking, (2) strict priority sched- uling, and (3) optimistic methods, that (1) E(n, p) 5 n(1 - p/2)“-‘, (2) E(n, p) 5 (1 - (1 - p)“)/p, and (3) 1 + ((1 - p)/p)ln(p(n - 1) + 1) 5 E(n, p) 5 1 + (l/p)ln(p(n - 1) + 1). Thus, for fixed p, as n + Q), (1) E + 0 for standard locking methods, (2) E 5 l/p for strict priority scheduling methods, and (3) E + co for optimistic methods. Also found are bounds on E in the case where conflicts are analyzed so as to maximize E. The predictions of the random graph model are confirmed by simulations of an abstract transaction processing system. In practice, though, there is a price to pay for the increased effective level of concurrency of methods (2) and (3): using these methods there is more wasted work (i.e., more steps executed by transactions that are later aborted). In response to this problem, three new concurrency control methods suggested by the random graph model analysis are developed. Two of these, called (a) running priority and (b) older or running priority, are shown by the simulation results to perform better than the previously known methods (l)-(3) for relatively large n or large p, in terms of achieving a high effective level of concurrency at a comparatively small cost in wasted work. Categories and Subject Descriptors: F.2.2 [Analysis of Algorithms and Problem Complexity]: Nonnumerical Algorithms and Problems; H.2.4 [Database Management]: Systems General Terms: Algorithms, Theory Additional Key Words and Phrases: Concurrency control, multiprocessors, parallel processing, transaction processing 1. INTRODUCTION As is well known, the problem of supporting increased transaction rates against a common shared database can be approached by developing systems with increased numbers of processors. The details of such systems are not of direct concern here, except we assume that in order to utilize effectively a system consisting of many processors it will be necessary to run many transactions Authors’ address: IBM Thomas J. Watson Research Center, P.O. Box 218, Yorktown Heights, NY 10598 Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. 0 1985 ACM 0730-0301/85/0300-0001$00.75 ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985, Pages 1-28.

Upload: khangminh22

Post on 21-Feb-2023

1 views

Category:

Documents


0 download

TRANSCRIPT

Limitations of Concurrency in Transaction Processing PETER FRANASZEK and JOHN T. ROBINSON IBM Thomas J. Watson Research Center

Given the pairwise probability of conflict p among transactions in a transaction processing system, together with the total number of concurrent transactions n, the effective level of concurrency E(n,p) is defined as the expected number of the n transactions that can run concurrently and actually do useful work. Using a random graph model of concurrency, we show for three general classes of concurrency control methods, examples of which are (1) standard locking, (2) strict priority sched- uling, and (3) optimistic methods, that (1) E(n, p) 5 n(1 - p/2)“-‘, (2) E(n, p) 5 (1 - (1 - p)“)/p,

and (3) 1 + ((1 - p)/p)ln(p(n - 1) + 1) 5 E(n, p) 5 1 + (l/p)ln(p(n - 1) + 1). Thus, for fixed p, as n + Q), (1) E + 0 for standard locking methods, (2) E 5 l/p for strict priority scheduling methods, and (3) E + co for optimistic methods. Also found are bounds on E in the case where conflicts are analyzed so as to maximize E.

The predictions of the random graph model are confirmed by simulations of an abstract transaction processing system. In practice, though, there is a price to pay for the increased effective level of concurrency of methods (2) and (3): using these methods there is more wasted work (i.e., more steps executed by transactions that are later aborted). In response to this problem, three new concurrency control methods suggested by the random graph model analysis are developed. Two of these, called (a) running priority and (b) older or running priority, are shown by the simulation results to perform better than the previously known methods (l)-(3) for relatively large n or large p, in terms of achieving a high effective level of concurrency at a comparatively small cost in wasted work.

Categories and Subject Descriptors: F.2.2 [Analysis of Algorithms and Problem Complexity]: Nonnumerical Algorithms and Problems; H.2.4 [Database Management]: Systems

General Terms: Algorithms, Theory

Additional Key Words and Phrases: Concurrency control, multiprocessors, parallel processing, transaction processing

1. INTRODUCTION

As is well known, the problem of supporting increased transaction rates against a common shared database can be approached by developing systems with increased numbers of processors. The details of such systems are not of direct concern here, except we assume that in order to utilize effectively a system consisting of many processors it will be necessary to run many transactions

Authors’ address: IBM Thomas J. Watson Research Center, P.O. Box 218, Yorktown Heights, NY 10598 Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. 0 1985 ACM 0730-0301/85/0300-0001$00.75

ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985, Pages 1-28.

2 ’ P. Franaszek and J. T. Robinson

concurrently, with more concurrent transactions required as the number of processors inceases.

However, two or more transactions can conflict in a variety of ways: they can require common resources l;hat must be allocated exclusively, or they can access common data items in incompatible modes. In such a case it will generally be necessary to have some transactions wait, or backup, or restart certain transac- tions, until the transactions they conflict with have run to completion. Intuit- tively, if the probability of ‘conflict is high, then only a few transactions can run concurrently so that all run to completion. In such a case a limit to increased transaction rates will soon be encountered, and this limit is determined by the nature of the transactions.

Our primary goal here is to analyze this fundamental limitation of concurrency for various general classes of scheduling policies independently of any given transaction processing system. In contrast, previous work has concentrated mainly on analysis and simulation of relatively detailed models of particular kinds of systems, and on experiments with actual systems (e.g., [l-4, 8, 9, 11, 121). In such cases it is difficult to separate the effect of probability of conflict from other system characteristics, particularly in the case in which concurrency is increased to an arbitrari1.y high degree.

A conflict between two transactions is defined as a condition existing between them that results in the restriction that both cannot concurrently do useful work (i.e., work leading to their completion). The work done by a transaction that is currently executing but later aborted is not considered useful since in general it will be necessary to repeat this work in order that the transaction run to completion (in some systems this is only approximately true, since it may be possible to backup a transaction to a previously established checkpoint). We call the steps executed by a transaction that is later aborted wasted work. A trans- action may also not be doing useful work because it is waiting. If the total number of concurrent transactions is n, and any pair of transactions conflict with prob- ability p, we define the effective level of concurrency E(n, p) as the expected number of the n transactions that will be doing useful work. The expected remaining n - E(n, p) transactions will either be doing wasted work or else be waiting.

In Section 2, using a random graph model of transaction processing systems, we bound the effective level of concurrency for three general classes of concur- rency control policies, examples of which are (1) the usual locking method, which we call standard locking; (2) strict priority scheduling, with priorities assigned independently of conflicts; and (3) optimistic methods. These classes are shown to be fundamentally different. For example, as p is held fixed and n + m, (1) E + 0 for the first class, (2) E 5 l/p for the second class, and (3) E +- m for the third class. Also found are bounds on E in the case where priorities are assigned based on an analysis of the conflicts so as to maximize E.

The results of the random graph model analysis are confirmed by simulations of an abstract transaction processing system, as described in Section 3, using the three aforementioned concurrency control policies. Although the latter two policies do indeed increase the effective level of concurrency, they do so only with large costs in increased wasted work. However, the random graph model analysis suggests that policies that take into account the waiting or executing ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

Limitations of Concurrency in Transaction Processing l 3

state of conflicting transactions could be useful for increasing the effective level of concurrency. Using this information only, or combining it with the priorities used in the strict priority scheduler, leads to three new concurrency control policies. Two of these methods are shown to offer substantial performance improvements for relatively large n or large p, in terms of achieving a high effective level of concurrency at a comparatively small cost in wasted work.

Section 4 contains a summary of the main results, their implications, and a discussion of further research.

2. ANALYSIS OF A RANDOM GRAPH MODEL OF CONCURRENCY

In this section we consider the effective level of concurrency that can be achieved under various basic types of scheduling policies, using a simple random graph model of a concurrent transaction processing system.

Let S = (sl, s2, s3, . . .) be the sequence of transactions to be processed by the system, and let T = (Z’i, T2, T3, . . . T,,) be the concurrent set, consisting of those transactions that have entered the system but whose processing has not yet completed. Each Y’i should be thought of as a variable that can assume the values Sl, s2, s3, * - *. The cardinality of the concurrent set, n, is called the total level of concurrency. It is assumed that the concurrent set is ordered by a priority relation that is a total ordering of the transactions in S. In an actual system transaction priority could be determined by the time of entry, for example, possibly combined with transaction class (such as interactive or batch).

Given T, we define a conflict graph C as a graph whose nodes are members of T and whose edges, Cij, i < j, represent conflicts between transaction pairs Ti and Tj. We also define a scheduling function as a function V: T + (0, l), where V(Ti) = 1 represents the case in which the transaction Ti is doing useful work (as discussed in the Introduction), and V(Ti) = 0 otherwise. If V(Ti) = 1 we say Ti is active, otherwise Ti is said to be inactive.

We investigate the expected number of active transactions (i.e., the effective level of concurrency as defined in the Introduction) as a function of the total level of concurrency and the probability of conflict, using the following model of a transaction processing system.

(1) Initially, the concurrent set consists of the first n transactions from the transaction sequence S, and each conflict Cij occurs with probability p, inde- pendently of any other conflicts.

(2) At the end of each unit of time, all active transactions complete and are removed from the concurrent set. Each such transaction is replaced with a new transaction taken from the transaction sequence S in sequence order, so that the concurrent set always contains n transactions.

(3) After all active transactions have been removed and replaced, a new cycle begins. Given a pair of transactions in the concurrent set of the new cycle, Ti and Tj, i <j, if both transactions were in the concurrent set of the previous cycle, the conflict Cij occurs in the new cycle if and only if a conflict occurred between these two transactions in the previous cycle. Otherwise, one or both of Ti and Tj are new transactions, and Cij occurs with probability p, independently of any other conficts.

ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

4 - P. Franaszek and J. T. Robinson

Note that it is assumed that sufficient processing power is available to handle any effective level of concurrency up to n, the total level of concurrency. What is being studied is the effective level of concurrency permitted by the probability of conflict among transactions and the scheduling function.

The above is a highly simplified picture of transaction processing, and questions arise concerning whether results obtained using this model are applicable to real systems.What concerns us here, though, is not so much the problem of detailed performance prediction as that of achieving a fundamental understanding of what is to be expected for differing policies as the total level of concurrency is increased. For example, the amount of useful work may, for increasing n, go to zero, be bounded by a constant, or increase indefinitely. Furthermore, the outcome of the analysis of the above model has been confirmed by simulations, as described in Section 3. The simulation results indicate that some of the analytic expressions derived in this section can be surprisingly accurate.

Four scheduling functions will be studied here. In defining these scheduling functions, we are motivated by a consideration of basic types of policies that can be used in actual systems. If some transaction l’i cannot do useful work because of a conflict with some transaction Tj, we say that Ti is blocked (from doing useful work) by Tj. The first two scheduling functions we study correspond to policies in which a transaction Ti can block other transactions even though Ti is itself blocked by another transaction. In contrast, the latter two scheduling functions correspond to policies in which a transaction can block other transac- tions only if it is doing useful work. We call this property essential blocking. In general, the essential blocking property cannot be obtained using incremental locking techniques, in which transactions acquire locks on required resources and data items as they run, since it may be impossible to determine whether a transaction is actually doing useful work: a currently running transaction may later be aborted due to deadlock, or in some locking policies it may later be aborted due to a conflict with a higher priority transaction. However, essential blocking can be realized by locking in the case where each transaction requests all required locks at initiation time (see below). Essential blocking is also a property of optimistic methods [6], since in these methods access requests are always granted, and transactions are aborted only when they conflict with transactions that are guaranteed to run to completion (i.e., guaranteed to be doing useful work).

The four types of policies that are the motivation for the scheduling functions we define differ according to whether blocking is nonessential or essential, whether priorities are used, and whether the conflicts that occur are analyzed globally. These four basic types are as follows:

(1) Priority-less. In this type of policy, no priorities are used in determining which transaction blocks another; instead, any transaction Ti may be blocked by any other conflicting transaction Tja Examples of this type of policy are those we call standard locking policies, in which locks are requested incrementally, a lock is always granted to the first transaction that requests it, and subsequent transactions that request the lock in a conflicting mode are made to wait until the earlier transaction completes or is aborted. Variants result from the lock modes that are available, the way deadlock is detected and resolved, and so on. ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

Limitations of Concurrency in Transaction Processing l 5

A more detailed description of the use of this type of method in actual systems can be found in [5]. Most of today’s transaction processing systems use a concurrency control method of this type.

(2) Strict priority. In a strict priority type of policy, given a conflict between two transactions, the higher priority transaction always blocks the other. It is assumed that priorities are assigned independently of the conflicts that occur, as is typically the case in actual systems. Note that this means that blocking is nonessential, since the decision to block is made only on the basis of priorities, and not on any available information on which transactions are doing useful work. An example of this type of policy is the wound-wait method of [13] in which, given a conflict between two transactions, the transaction with a larger transaction number (i.e., lower priority) is made to wait if possible, and is otherwise aborted.

(3) Essential blocking with priority. Under this type of policy, a transaction is blocked if and only if it conflicts with some higher priority transaction that is doing useful work. Again, it is assumed that priorities are assigned independently of conflicts. In the case that transactions request all required locks at initiation time, an example of this type of policy is the method in which transactions wait until all of their requested locks can be granted, with either all or none of the locks being granted atomically. If the concurrency control examines the waiting transactions in FIFO order, then priorities are determined by starting times. As discussed above, other examples of this type of policy are optimistic methods, in which access requests are always granted. Under optimistic methods, priorities are determined by read-phase completion times, where the read-phase completion of a transaction is defined as that point at which the transaction is known to require no further reads to the database.

(4) Conflict dependent. Here it is assumed that the system is capable of examining all conflicts, either in advance, or (in systems using an optimistic method) before completion time. This information is then used to select active transactions with the objective of maximizing throughput. We are not aware of any current system that uses a concurrency control method of this type. Our purpose in considering this type of policy is to investigate the improvement that could be achieved by a concurrency control method that makes use of such an approach.

2.1 Priority-Less

Since under standard locking policies a conflict is handled (in the absence of deadlock) without the use of priorities, and without regard to any other conflicts, we assume that given a conflict between two transactions, either transaction is determined as inactive as a result of the conflict with probability l/2. We therefore define VPL, the priority-less scheduling function, algorithmically as follows: (1) let I be a set of transactions, initially empty (I will become the set of inactive transactions); (2) for each transaction Ti in the concurrent set, for each j such that the conflict Cij or Cji occurs, add Ti to I with probability l/2; (3) VpL(Ti) = 0 if Ti E 1, otherwise VpL(Ti) = 1.

This scheduling function is reasonable only in a probabilistic sense, since given two conflicting transactions, outcomes are permitted in which neither or both

ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

6 - P. Franaszek and J. 7’. Robinson

/

P-O

/ P-0.u

P-O.1

10 20 SO 40 50 60 70 60 90 100 MJWER OF WNCiBftMf TRAN!3CTIOtS

Fig. 1. Upper bounds on expected active transactions for priority-less policies,

are active. A possibly more! serious problem is that in an actual system, if two transactions conflict, it is in general more likely that the transaction that began most recently will be waiting on the other “older” transaction, since the latter transaction will have had more time to acquire locks on needed resources and data items. Nevertheless, this simple model appears to yield accurate predictions on the overall behavior of t:he effective level of concurrency for standard locking policies, as will be seen in Election 3.

For this scheduling function, the computation of the expected number of active transactions for the first cycle is straightforward: each transaction !Z’i is active if and only if for each transaction Tj, j # i, it is not the case that (1) Ti conflicts with Tj and (2) Ti is added to I as a result of the conflict. Since there are n - 1 transactions Tj with j # i, Ti is active with probability (1 - ~/2)(~-l). The expected number of active transactions for the first cycle, A&(n, p), is thus n(1 -p/2)‘“-“. Note that A&(n, p) + 0 for fixed p as n + 00.

The concurrent set during the rth cycle, r > 1, may contain transactions for which this is the 9th cycle, q = 1, 2, 3, . . . , r (i.e., transactions that have been inactive for q - 1 cycles). For a transaction with q = 1, the probability of being active is (1 - ~/2)(~-l), as in the analysis of the first cycle. Next consider a transaction Ti with q = 2’. Let Xi be the subset of the concurrent set of the (r - 1)st cycle consisting of those transactions that conflicted with Tie Note that the conditional probability of a transaction Tj completing at the end of the (r - 1)st cycle, given that 7) E Xi, is less than the unconditional probability of Tj completing, since, if Tj E .Xi, it is known that Tj has at least one conflict. Thus the expected number of conflicts for Ti during its second cycle is greater than during its first, and, similarly, greater during its qth cycle than during its (q - 1)st. We have the following result. ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

Limitations of Concurrency in Transaction Processing l 7

PROPOSITION 1. The steady state expected number of active transactions under the priority-less scheduling function, ApL(n, p), satisfies

&A P) < &L(n, P) = n 1 - E ( )

(n-1) .

AbL(n, p) is shown for various values of n and p in Figure 1.

2.2 Strict Priority

We assume that the concurrent set is ordered by the priority relation on the transactions in S, with Ti having higher priority than Tj if i < j. With this definition of priority, the scheduling function corresponding to the class of strict priority policies, Vsp, is as follows: for each transaction Tip if for some j < i there exists a conflict Cji, then Vsp(Ti) = 0, otherwise Vsp(Ti) = 1.

Consider the operation of the system during the first cycle. Each transaction Ti has i - 1 higher priority transactions, and since for this class of policies we assume that the priority relation is independent of conflicts, Ti is active with probability (1 - p)‘-l. Therefore, the expected number of active transactions during the first cycle, A&(n, p), is

A,&(n, p) = i (1 - p)i-l = ‘11 yl I”d; = 1 - (i-P)“. i=l

Note that A&(n, p) is bounded above by l/p. Next we examine the probability of Ti completing in the rth cycle, r > 1. If Ti

is a new transaction in the concurrent set for this cycle, then its probability of completion is again (1 - P)~-‘. Otherwise it was present and inactive in the previous cycle as, say, Tip. Note that the conditional probability of Ti not conflicting with a higher priority transaction, given that there was at least one such conflict in the previous cycle, is no greater than (1 - P)~-‘, since transactions in the (r - 1)st cycle with higher priority than Ti* completed independently of conflicts with Tie. This yields the following result.

PROPOSITION 2. The steady state expected number of active transactions under the strict priority scheduling function, AsP(n, p), satisfies

&dn, PI 5 &h PI = 1 - (1 - p)”

P *

A& (n, p) is shown for various values of n and p in Figure 2.

2.3 Essential Blocking with Priority

As in the case of strict priority policies, we order the concurrent set using the priority relation. The class of essential blocking with priority policies is repre- sented in the random graph model by defining a scheduling function VEB that satisfies the property that Ti can block Tj only if i < j and VEB(Ti) = 1. Such a scheduling function can be defined recursively as follows: (1) V,,( Tl) = 1, and (2) for each Tiy if for some j < i with VE,(Tj) = 1 there exists a conflict Cji, then VEB (Ti) = 0, otherwise VEB (Ti) = 1.

Consider the operation of the system under V Es during the first cycle. Let ei be the probability that Ti is active. Since we are assuming that the priority

ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

a. P. Franaszek and J. T. Robinson

10 20 30 40 50 60 70 60 90 100 h’UWER OF WMCURRENT TRAHUCTIM

Fig. 2. Upper bounds on expected active transactions for strict priority policies.

relation is independent of conflicts, oi can be computed recursively as follows: al = 1, and

ai+l = (1 - paA - pad . . . (1 - pai)

= Ui(l - pai)*

Next consider the opera.tion of the system during the rth cycle, r > 1. If a transaction Ti in the concurrent set of the rth cycle was in the concurrent set of the (r - l)st cycle, then it is known that Ti conflicted with one or more higher priority active transactions. However, all such transactions completed and were removed from the concurrent set at the end of that cycle. Note that under essential blocking the conflicts that occur between inactive transactions in a given cycle are independent of the set of transactions to complete in that cycle, since an inactive transaction does not block any other transaction. Furthermore, conflicts between transactions in S occur independently, and thus Ti is equivalent to a new transaction in terms of its probability of conflict with other transactions in the rth cycle. It follows that oi is the steady state probability that Ti completes in a given cycle.

LEMMA. ai satisfies

1-P 1 p(i---= p(i - 1) + 1.

PROOF. For the upper bound, let

u; = 1, ui

Ui+l = ~ 1 + PUi’

ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

Limitations of Concurrency in Transaction Processing l 9

The solution to this is

Ui= . 1

p(2 - 1) + 1’

and it is easy to prove by induction that ai 5 Ui, using

Ui+l = *i (1 - (Pai)2)9

and the fact that 0 -E pai 5 1. For the lower bound, let

L1 = 1, Li+l = Li 1 - i

’ p(i - 1) + 1 ) -

The solution to this is

Li = 1-P p(i - 2) + 1’

and it is easy to prove by induction that Li 5 ai, using (from the upper bound)

l- P p(i - 1) + 1

5 1 - PCZi. q

PROPOSITION 3. The steady state expected number of active transactions under the essential blocking with priority scheduling function, AEB(n, p), satisfies

1 + (1 -P)

P In(p(n - 1) + 1) 5 jI p(i ti)‘+ 1.

5i 1 5 1 + ln(p(n - 1) + 1). is1 p(i - 1) + 1

k

PROOF. The summations follow directly from the lemma. For the lower bound, let [x] represent the greatest integer not exceeding X. Then n n+l

i=l p L - 2) + 1 c (’ 1-P =1+ S ‘-’ dx

2 P([Xl - 2) + 1

S n+l

rl+ 1-P 2 p(x - 2) + 1 dx

= 1 + (’ - ‘) ln(p(n - 1) + 1). P

ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

Limitations of Concurrency in Transaction Processing 11

A0 A-0 A0

c'b c'b l't,

A 0 A-0 : 0 C-D C-Q C-D

A 0 A-0 A 0

c-b c-b LA

A 0 A-0 A 0

ci; ci; ;r;

:-” c D

A 0

C’D

tie c D

A 0

CXD

A-0 A 0

:: b CYi

:i: c D

t-0

C-D

:sO C-D

t-t: C-D

:2 C-D

A 0

CXb

A\0 C-D

AX0 C-D

A 0

cx

A 0

2%

A-0

C'D

A-0

cXb

‘iO C-D

A-0

csl

“i’: C-D

A-0

2%

Fig. 4. All conflict graphs for n = 4.

Consider a scheduler that has complete knowledge of all conflicts that will occur between transaction pairs, and uses this information so as to maximize the expected steady state effective level of concurrency. The selection of active transactions in a given cycle has two goals: (a) to maximize the number of active transactions in the current cycle, and (b) to minimize the conflicts in subsequent cycles. Figure 4 shows all conflict graphs for n = 4. A brute force analysis of these graphs (omitted here) leads to the observation that the above goals are on the average contradictory: removing a maximal set of nonconflicting transactions leads to a greater expected number of conflicts for the next cycle. Although this may be true in general, our purpose in considering conflict-dependent policies is only to estimate the improvement that could be achieved by such an approach. In the following we ignore goal (b), and consider only conflict-dependent sched- uling that meets goal (a). Also, it is assumed that conflicts occur independently with probability p, as is the case in the first cycle of operation.

Let VcD be a scheduling function that selects a set of active transactions Y of maximal size, subject to the constraint that for all i, j, i < i, if Ti E Y and Tj E Y, then Cij does not occur. Note that VcD is not determined uniquely, since there may be more than one subset Y of maximal size satisfying the above constraint; however, the following applies to any such scheduling function.

Let F(k) be the probability that k or less transactions are active and that n - k or more transactions are inactive under V cn. Note that F(k) is the discrete c.d.f. of the random variable that is the number of active transactions, and that F(0) = 0 and F(n) = 1.

ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

12 l P. Franaszek and J. T. Robinson

LEMMA. Let

G(k) = (1 - (1 -p) W+W) (k:l).

Then F(k) 2 G(k), 0 5 k < n.

PROOF. F(k) (k < n) is the probability that all subsets of transactions of size k + 1 contain at least one conflict. Given a subset of transactions 2, let c(Z) be the event in which there exists at least one conflict between transactions in 2. Given two subsets, Z1 and ZZ,

probM&) n c&J) = probM&) I cG)hobM.&)).

If Z1 and Z2 do not have two or more transactions in common, ~(2,) and ~(2,) are independent, and so prob(c(Z1) 1 ~(2,)) = prob(c(&)). Otherwise, ranging over the probability space restricted to c(Z,), a conflict appearing in 22 may also appear in &, in which case c(Z,), and so prob(c(ZJ lc(&)) > prob(c(2,)). Repeating, it can be shown that given subsets &, 22, 23, . . . ,

prob(c(&) n ~(2,) n c(&) . . .) L prob(c(Zi))prob(c(Z2))prob(c(Z3)) . . . .

Next, given a subset of size k + 1, there are k(k + 1)/2 transaction pairs in the subset, and so there is at least one conflict with probability 1 - (1 - P)~(~+‘)“. The lemma follows from the fact that there are (kr’+l) subsets of size k + 1.

PROPOSITION 4. If all conflicts occur independently, each with probability p, the expected number of active transactions under the conflict-dependent scheduling function Vco, Aco(n, p), satisfies

n-1

AEB(n, p) 5 AcD(~, P) 5 n - c (1 - (1 - P) k(k+lW)(k:l)~ k=l

PROOF. The lower bound follows from the fact that we can define VcD as the essential blocking scheduling function with priorities assigned based on an analysis of C so as to maximize the number of active transactions: we simply assign every Ti in Y (where Y is as in the definition of Vco) a higher priority than all Tj not in Y. Therefore, under Vco we must have an equal number or more active transactions than under VEB with priorities assigned independently of conflicts. For the upper bound, evaluating Ace using F(k):

AcD(n, p) = (F(1) - F(0)) + 2(F(2) - F(1)) + 3(F(3) - F(2)) + . . .

+ n(F(n) - F(n - 1))

= n - F(1) - F(2) - . . . - F(n - 1)

5 n - G(1) - G(2) - . . . - G(n - 1). 0

The lower bound AEB (computed exactly, using the recurrence of Section 2.3) and the upper bound given above are shown in Figure 5 for various values of n andp. Examining this figure, we conclude that although some performance gains may be possible using conflict-dependent scheduling, it appears that no dramatic improvements such as an order of magnitude increase in the effective level of concurrency can be expected. ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

Limitations of Concurrency in Transaction Processing

P.0 P-O.001

l 13

P-O.01

P-O.03

10 20 30 40 so 60 70 60 90 loo N’JWER OF UNCURRENT TRANSACTIOt6

Fig. 5. Bounds on expected active transactions for conflict-dependent policies.

3. NEW POLICIES, SIMULATIONS, AND COMPARISONS

The simplifying assumption of the random graph model that conflicts occur independently between pairs of transactions uniformly with probability p makes it seem difficult to apply the analytic results directly to actual transaction processing systems. First, in an actual system, transactions do varying amounts of work: a transaction that accesses many data objects and/or takes a long time to execute will clearly have a higher probability of conflict than a “small quick” transaction. In other words, the pairwise probability of conflict is not uniformly p. Second, scheduling policies in which transactions are made to wait affect the set of concurrently executing transactions, so that if we were to examine succes- sive states of an actual system, conflicts would not be seen to occur independently. For example, imagine that a “snapshot” was taken in an actual system using a locking-based scheduling policy by saving the current wait-graph at the comple- tion of each transaction. Then, if by chance it happened at some point that m transactions T,, T2, . . . , T,,, were waiting each for the next in a chain, with T2 waiting on T1, T3 waiting on T2, and so on, then T,,, would be waiting on T,,-, for at least m - 1 successive snapshots, assuming no transaction in the chain was aborted. That is, conflicts are not regenerated pairwise independently in succes- sive system states in locking-based policies, since some transactions that conflict are “held back” by the scheduler. This is essentially the effect that led to the analytic expressions of Propositions 1 and 2 in the previous section being upper bounds rather than exact results. Third, regarding the limiting values of the analytic expressions of Section 2, in any real system the database will consist of some maximum number of indivisible granules, and clearly there can never be more concurrent transactions doing useful work than this number. There are many similar considerations, such as the effect of deadlock resolution, the effect

ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

14 l P. Franaszek and J. T. Robinson

of data structures used by the transaction processing system, the effect of interleaving of transactions, and so on, that are not addressed in the random graph model.

However, as explained earlier, our goal is not to analyze in detail the perform- ance of actual systems, but to investigate system-independent limitations on the effective levels of concurrency that result from various basic types of scheduling policies. One can hypothesize that the random graph model captures enough of the behavior of dynamic transaction processing systems to give valid predictions on the nature of the performance limits of these systems as the total level of concurrency is increased. Another consideration is the followfng: the “real-life” scheduling policies that were the motivation for the three scheduling functions of Section 2 can achieve an increase in the effective level of concurrency only by aborting more transactions, but the distinction between a transaction that is waiting and a transaction that will later be aborted is not made in the random graph model. For optimistic methods this distinction is not needed: since no transactions wait, all inactive transactions are doing wasted work. For the “wound-wait” method, though, the cost in wasted work of achieving an increase in the effective level of concurrency is of interest. Finally, the definition of essential blocking suggests new locking-based scheduling policies, described below in Section 3.2.

For these reasons-to try to validate the random graph model, to determine the costs of achieving increases in the effective level of concurrency in wasted work, and to investigate new scheduling policies suggested by the concept of essential blocking-simulations were done of an abstract transaction processing system. Before discussing the scheduling policies that were used, and the vali- dation procedure, it is necessary to discuss in more detail the simulation model.

3.1 The Simulation Model

In the simulation model there are d distinct objects, where d is called the database size, n processors, and initially n concurrent transactions. A transaction exists through one or more incarnations, where each incarnation consists of a number of steps, each of which is processed in one unit of time: an initialization step, followed by one or more access steps, and, finally, if the transaction is not aborted, a completion step. If a transaction is aborted, a new incarnation begins, as described below. Otherwise the transaction completes, and a new transaction enters the system on the next step. That is, there are always exactly n concurrent transactions in the system.

When a transaction enters the system, and its first incarnation begins, the number of access steps it executes, along with the object accessed in each step, is randomly generated, and the starting “time” of the transaction (generated by incrementing a counter) is recorded. At each access step, the transaction requests access from a scheduler. Each access request is interpreted as a request for exclusive access to the object from the beginning of the step when the access request is issued until the end of the step when the transaction is aborted or completed. Based on the decision of the scheduler, the transaction either begins waiting, is aborted, or executes its access step. In the case where the transaction begins waiting, it continues to wait either until all the transactions it conflicts ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

Limitations of Concurrency in Transaction Processing l 15

with are aborted or completed, or until it is aborted due to some other conflict that occurs after it begins waiting. The scheduling policies that were used are described in the following section.

All transactions that are not waiting due to an access request execute in parallel, so that in one unit of time up to n steps can be executed concurrently. Note that a transaction can be delayed only due to conflicts with other transac- tions. Our motivation in using an infinite server model is similar to our choice of the random graph model in the previous section: although a model with CPU and I/O queueing delays would be more immediately applicable to certain types of actual systems, it would offer less insight concerning inherent limitations of concurrency due to conflict among transactions.

At the beginning of each unit of time the access requests of all transactions beginning an access step are processed in random order, any order equally likely. In the case in which a transaction is aborted, a new incarnation is begun on the next step, with the same number of accesses and the same object accesses in each step as in the previous incarnation.

All the steps of a transaction were classified as one of three types: (1) a waiting step is a step in which an incarnation of the transaction is waiting; (2) a wasted step is an initialization or access step executed by an incarnation that was later aborted; and (3) a useful step is an initialization, access, or completion step of an incarnation that completed.

3.2 Scheduling Policies

Six scheduling policies were simulated, three of which were the motivation for the definition of the three scheduling functions of Section 2. These three were as follows, where in each case the actions taken in response to a request that causes a conflict are described (any access request that did not cause a conflict was of course granted).

(1) Standard locking. The requesting transaction is made to wait on all transactions whose current incarnation has previously requested access to the object causing the conflict (henceforth we call this set of transactions the conflicting transactions). If this results in a deadlock, the transaction participating in the deadlock with the greatest starting time (the most “recent” transaction) is aborted.

(2) Older priority. First, define the priority of a transaction as the starting time of the transaction, with lesser starting times giving higher priority (that is, “older” transactions are given higher priority). The requesting transaction is made to wait on all conflicting transactions with higher priority, and all conflict- ing transactions with lesser priority are aborted. Note that this is the “wound- wait” method of [ 131, with starting times used for transaction numbering.

(3) Optimistic. Access requests of transactions are always granted. Whenever a transaction reaches the completion step, all transactions whose current incar- nation has previously been granted access to any object accessed by the complet- ing transaction are aborted. In this policy, conflicting transactions are aborted at the time of validation of the completing transaction (as in [12]), which results in less wasted work than optimistic methods in which conflicting transactions are aborted when they request validation.

ACM Transactions on Database Systems, Vol. 10, NO. 1, March 1985.

16 l P. Franaszek and J. T. Robinson

The older priority policy is a strict priority policy, while the optimistic policy is an example of a policy that satisfies the essential blocking property. The results of Section 2 suggest that essential blocking should yield substantially greater effective levels of concurrency for large n. However, as explained earlier, essential blocking has the disadvantage that, if it is obtained using an optimistic policy, no transactions wait: all transactions not doing useful work are doing wasted work. Since the number of transactions doing wasted work is expected to be n - O(log(n)), this is a potentially serious problem.

The amount of wasted work under an optimistic policy might be reduced by determining at some point that a given transaction Tl is unlikely to “win” the race to the completion step with a conflicting transaction T2. In such a case, Tl could be made to wait on Tz if the conflict resulted from its current request, otherwise Tl could be aborted immediately. Alternatively, essential blocking might be approximated using incremental locking techniques by not permitting a transaction to wait on other transactions that are known not to be doing useful work. The simplest such criterion when incremental locking techniques are used is that a waiting transaction is not doing useful work. Expressing this as a form of priority leads to the following policy:

(4) Running priority (RP). Define the priority of a transaction as 1 if the transaction is not waiting, otherwise its priority is 3, with 1 as highest priority. Process an access request that causes a conflict by first evaluating the priority of all conflicting transactions, with the requesting transaction being given priority 2, and then follow the older priority method above, using this new definition of priority. This results in the requesting transaction waiting on all conflicting transactions that are running, and all conflicting transactions that were waiting at the time of the request being aborted.

There are two ways in which waiting/running priorities can be combined with starting time priorities: we can either take the disjunction or the conjunction of the two priority relations. This results in the following two scheduling policies:

(5) Older or running priority (OOR). Combine the older priority and running priority policies by having the requesting transaction wait on all conflicting transactions that are older or are running, and by aborting all conflicting transactions that are not older and are waiting.

(6) Older and runningpriority (OAR). Combine the older priority and running priority policies by having t.he requesting transaction wait on all conflicting transactions that are older and are running, and by aborting all conflicting transactions that are not older or are waiting.

In addition to standard locking, deadlock can occur under the older or running priority (OOR) policy. For this policy, deadlock was also resolved by aborting the most recent transaction participating in the deadlock, as in standard locking. It is easy to show that deadlock cannot occur in any of the remaining four policies.

The simulation results presented below indicate that the running/waiting status of a transaction is indeed useful: ranges of n and p exist for which of the two new policies yield better t.rade-offs between useful and wasted work than the earlier policies. ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

Limitations of Concurrency in Transaction Processing l 17

MJWER OF CUdCURRENT TrUNW3IM

Fig. 6. Useful steps for d = 1000.

3.3 Simulation Results

Two sets of simulations were done, one set with a database size d = 1000 and another set with d = 2000, for n = 10, 20, 30,. . . , 100, 120, 140,. . . , 200, 250, and 300, and for each of the six scheduling policies. In all runs the number of access steps of a new transaction was generated by selecting one of 8, 9, 10, 11, or 12 access steps, any equally likely, and the object accessed in each access step was generated by randomly selecting one of the d objects, any equally likely. Note that it was possible that more than one access step of a transaction could be for the same object. If a transaction requested access from the scheduler for an object to which it had previously been granted access, it was simply granted access again.

In each run the model was simulated for 3000 steps. Statistics on the total number of waiting, wasted, and useful steps were saved and reset at the end of each 1000 steps. Thus, statistics for the first lOOO-step segment could have included some start-up transient, but it will be seen that the statistics for a given d, IL, and scheduling policy were close for the three segments in all cases.

All statistics were normalized by dividing by 1000, resulting in the average number of transactions waiting, executing a wasted step, or executing a useful step, at each step of the segment. The results are shown in Figures 6 through 11. Note that there are three curves for each policy: one for each lOOO-step segment.

Several results are of interest. First, note that the curves representing useful steps for the standard locking, older priority, and optimistic methods generally correspond to the predictions of the random graph model: under the standard locking method, the number of useful steps reaches a maximum and then is monotonically decreasing; under the older priority method, the number of useful

ACM Transactions on Database Systems, Vol. 10, No. 1. March 1985.

18 l P. Franaszek and J. T. Robinson

50 100 150 200 250 300 NuleER OF -ENl 1lUNSKTlONS

Fig. 7. Wasted steps for d = 1000.

STANIMRD LOCXlffi

am PRIORITY

am AND Rwmo PRICRIM

0 50 100 1M 200 250 500 NUBER OF COfKMftN TR~~NsACTIONS

Fig. 8. Waiting steps for d = 1000.

steps is monotonically increasing but seems to be approaching a constant; and under the optimistic method, the number of useful steps is monotonically increas- ing but does not seem to be approaching any limiting value. This correspondence is analyzed more exactly in the following section. ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

Limitations of Concurrency in Transaction Processing . 19

OLDER PRICRITY

OLDER AND RWING ~I~lfY

Fig. 9. Useful steps for d = 2000.

in r

0 0 50 100 150 20

Fig. 10. Wasted steps for d = 2000.

The difference between the older priority and OAR policies is that, in the latter policy, there is an additional condition in which a conflicting transaction is aborted: a conflicting transaction is aborted if it is currently waiting, even if it is older than the requesting transaction. The simulation results show that,

ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

20 - P. Franaszek and J. T. Robinson

MJWER OF C.WCMRENT TRAMACTIM

Fig. 11. Waiting steps for d = 2000.

although this leads to significantly more wasted work, it does not affect the effective level of concurrency, since the useful steps for these two policies are uniformly close. Thus, using OAR does not offer any advantages over older priority.

The peak in useful steps under standard locking occurs at about n = 20 for d = 1000 and at about n = 40 for d = 2000. “Blow-ups” of Figures 6 and 9, showing this peak more clearly, are shown in Figures 12 and 13. Up to this peak, all policies yield roughly similar performance in terms of useful steps; however, standard locking has the advantage of having the lowest level of wasted work. We believe that most current systems either have a small enough degree of concurrency or sufficiently low probability of conflict that they operate in the region well to the left of this peak.

To the right of the standard-locking peak, the number of useful steps under standard locking falls sharply, eventually reaching levels that could have been achieved with very small n. Higher throughput requires other forms of concur- rency control, independently of the particular system. Although the optimistic method gives the most useful steps, it does so only at a very high cost in wasted work. For example, for d = 1000 and n = 300, there are 6.1 wasted steps for every useful step under the optimistic method, and for d = 2000, n = 300, there are 3.4 wasted steps per useful step. The next two best methods in this region, in terms of useful steps, are the RP and OOR policies. These methods also result in a peak and subsequent drop in the effective level of concurrency, but these peaks occur well to the right of that for standard locking. Prior to this second set of peaks, since these two methods are also best in terms of doing the least wasted work (with the exception of standard locking), these are probably the best choices for a system operating in this region. Subsequent to these peaks, increasing the ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

Limitations of Concurrency in Transaction Processing l 21

I! : am?mRuwNc~ 15 3 .

t; 3 * AllD RUHING

i! - . . . ..--.--....-.. kj ‘O-

; . STANrnRD LmmJG

z

$ 5: c

b OY I I I J

0 10 20 30 40 NJNBER Of CONCURRENT TRANSACTIONS

Fig. 12. Useful steps for d = 1000, n 5 40.

20 40 60 80 NLMER OF CoNciJwENT 7RANsACTIcen

Fig. 13. Useful steps for d = 2000, n I 80.

PRICRITY

effective level of concurrency appears to require a closer approach to essential blocking.

There is a range of total levels of concurrency lying between the peaks for standard locking and those for the RP and OOR policies where the latter policies

ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

22 l P. Franaszek and J. T. Robinson

yield improved trade-offs between the effective level of concurrency and the amount of wasted work. For example, consider the simulation data for d = 1000 shown in Figures 6 and 7. The peak for standard locking occurs at about n = 20 (as noted above), but the peak for the RP policy is at approximately n = 80. At n = 80, the optimistic policy yields an average of 27.6 percent more useful work than the RP policy, but at a penalty of 2.7 times as much wasted work. For n = 40, the useful steps for the optimistic and RP policies are essentially the same, about 50 percent more than the maximum for standard locking. Here the ratio of useful to wasted work produced by the RP policy is approximately a factor of three better than that resulting from the optimistic policy. This is a significant improvement, since the optimistic policy is at this total level of concurrency yielding less than one useful step for every two steps executed.

The exact point at which the OOR or RP policies would perform better than standard locking in a real system is highly system-dependent. First of all, we expect that these policies will be useful primarily in systems that are “contention bound” under standing locking, that is, systems in which increased throughput is limited by conflicts among transactions and not by CPU, I/O, or communica- tion bottlenecks. Given a contention-bound system, the cost of wasted work could depend on several factors. For example, in a multiprocessor system in which data objects are cached in memories local to processors executing trans- actions, a transaction that is later restarted, and thus doing wasted work, can actually be doing some useful work in caching copies of data objects [ 121. A more detailed model of a system of interest would be required to determine whether the OOR or RP policies would provide more throughput than standard locking at a given level of concurrency in that particular system.

3.4 Comparisons to Random Graph Model Analysis

In order to compare the analysis of Section 2 with the simulation results in more detail, we first need a value for p. As explained earlier, there are various complex effects that can influence the probability of conflict between transactions, but, as a first approximation, we simply assume that two concurrent transactions conflict if they access any common object. In the simulations, each transaction accessed 8, 9, 10, 11, or 12 objects, any number equally likely, and each object access was to one of d objects, any equally likely, independently of all other accesses. Given two random selections of d objects, with any object selected equally likely in each selection, the probability that the selections are to different objects is 1 - l/d. Next, by our assumption, two transactions accessing i and i objects respectively do not conflict only if all IJ pairs of selections are to different objects. In the case where i andj are small compared to d, the probability that two transactions conflict can be closely approximated by assuming that i distinct objects and j distinct objects are selected (for an exact analysis see [7]), and is given by 1 - (1 - l/d)‘j. Finally, there are 25 possible pairs (i, j). Therefore, under these assumptions, p as a function of d is

Evaluating this for d = 1000 and d = 2000 gives ~(1000) = 0.095 and ~(2000) = 0.049. ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

Limitations of Concurrency in Transaction Processing l 23

1 STAHWRO LOCKING MPRESSION 2 STRICT PRIORITY EXPRESSICN 3 ESSENTIAL BLw(INC EXPRESSION

NUBEN OF COKURffENT TR4NSACTIo3s

Fig. 14. p computed for standard-locking simulations, d = 1000.

The simulation results are compared to the analytic expressions derived in Section 2 by inverting these expressions to findp, as follows: for each lOOO-step segment of each simulation run, given the n used in that run, a value of p is found for each of the three expressions (1) n(1 - p/2)“-l, (2) (1 - (1 - p)“)/p, and (3) 1 + (l/p)ln(p(n - 1) + l), so that the resulting value is the average number of transactions executing useful steps per simulation step for that run. The resulting values are shown in Figures 14-19 for the standard locking, older priority, and optimistic methods, and for d = 1000, 2000. Note that there are three curves for each of the expressions (l)-(3): one for each lOOO-step simulation segment. The interpretation of these figures is that a constant value of p over n for one of the expressions (l)-(3) for a particular method indicates a fit of the analytic expression to the results of the simulations of that method.

The older priority and optimistic methods do indeed appear to have an associated constant value of p, and, in fact, plots of the expressions (2) (for the older priority method) and (3) (for the optimistic method), using values of p estimated from the data of Figures 15, 16, 18, and 19, fit the useful step results of the simulations quite closely, lying uniformly within the “noise” of the statistics for each of the three segments of each run. However, the values of p estimated from this data (0.079 for Figure 15, 0.076 for Figure 16, 0.041 for Figure 18, and 0.036 for Figure 19) fall below ~(1000) and ~(2000) computed above. Although the expressions above were derived in Section 2 only as upper bounds on the effective level of concurrency, some of the additional factors that could result in this discrepancy are: (1) two concurrent transactions that access a common object will not conflict in the simulations if they overlap so that one completes before the second requests access to the common object, in contrast to the assumption made in derivingp(d) above; (2) for the older priority method, in the case where

ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

USIffi 1 SlANmRD LoowJc DPRESSlar 2 STRICT PRIUWY EXPRESSICN 3 ESSENTIAL BLOCKING EXPRESSION

24 l P. Franaszek and J. T. Robinson

o.oo-’ ’ B a ’ * ’ n ’ ’ * ’ ’ ’ ’ * ’ ’ ’ 0 50 100 150

MJMBER OF CONCURRENT TRANUCTIM

Fig. 15. p computed for older priority simulations, d = 1000.

USING 1 STANmRu LOCWNC ExPREsIaN 2 STRICT PRICRITY EXPRESSICN 3 ESSENTIAL ELOCWW EXPRESSICN

MJWER OF CONCUtRENl TRWSACTIONS

Fig. 16. p computed for optimistic method simulations, d = 1000.

a transaction waits, it never has to wait for the entire time the transaction it conflicts with is running; and (3) for the optimistic method, short transactions have a greater chance of running to completion than longer transactions, so that the expression forp(d) above should actually be weighted for smaller transactions. ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

Limitations of Concurrency in Transaction Processing l 25

%iNaRo Loo(ING EXPRESI~ 2 STRICT PRIORITY EXPRESSION 3 ESsENTIAL BLOCKING EXPRESSION

~...“...I,...I....I....l....J

50 100 150 200 250 300 WEROF CONURRENT TRAN'SACTICN!S

Fig. 17. p computed for standard-locking simulations, d = 2000.

0.20?

- USING . I STANMRO LOCKING EXPRESSION

f 2 STRICT PRIORITY EXPRESSION

- 3 mumAL e~oamo EXPRESSI~M i5 0.15-

0.00L”““““‘,,,‘*““~.‘.l...,J 0 50 100 150 200 250 300

HJYBER OF CUUXJRRDJT TRAMACTIM

Fig. 18. p computed for older priority simulations, d = 2000.

Returning to the standard locking method, although the values of p derived from expression (1) are not constant, they are “relatively” constant, lying within the range 0.04 5 p 5 0.09 for Figure 14 and 0.02 5 p I 0.06 for Figure 17. We conjecture that the initial increase in p as n increases is partly due to the “holding

ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

26 l P. Franaszek and J. T. Robinson

USING 0.15- 1 STANMRD LOWING EXPRESSION

5 2 STRICT PRIORITY EXPRESSION 3 ESSENTIAL &OCKINc EXPRESSIaJ

5

2 ;

z

o-10-

Ia.

E

Fig. 19. p computed for optimistic method simulations, d = 2000.

back” effect of locking described earlier, and that the decrease in p starting at n = 50 in Figure 14 and n = 90 in Figure 17 is partly due to the deadlock resolution procedure: since, given a deadlock, the most recent transaction par- ticipating in the deadlock is aborted, as the number of deadlocks increases one would expect more of an older priority kind of behavior, leading to a higher effective level of concurrency. Another effect is the following: in the case where many transactions are currently waiting, each waiting transaction will have requested access to some number of objects less than all of its accesses, so that a new transaction entering the system will have a probability of conflict lower than that computed above. In summary, although the analytic expression for standard locking does not fit the simulation results exactly, it is possible to vary p over a small range so that the expression does coincide with the simulation results. This expression does capture the essential behavior of this method, consisting of a peak and subsequent fall-off of the effective level of concurrency.

4. SUMMARY AND CONCLUSIONS

We have shown that three basic classes of concurrency control methods have fundamentally different limiting properties, independent of any particular sys- tem: as the total level of concurrency n is increased and the pairwise probability of conflict p remains fixed, the effective level of concurrency (1) reaches a maximum and then goes to zero under priority-less policies such as standard locking methods, (2) increases but is bounded by l/p for strict priority policies such as the older priority method, and (3) increases indefinitely under policies satisfying the essential blocking property such as optimistic methods, but only at a rate of O(log (n)). Although an increase in the effective level of concurrency ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

Limitations of Concurrency in Transaction Processing l 27

may be possible using conflict-dependent scheduling, it appears that no dramatic increase can be expected in the case where conflicts occur independently.

In terms of current systems, these results are of practical importance only for fairly large n or p. However, recent work on parallel program structuring, and on operating system design for distributed systems consisting of large numbers of processors, has focused on the use of the database transaction concept as a fundamental construct for system development (e.g., see [lo, 141). In systems in which a transaction mechanism is used at the lowest level, one might expect greater numbers of concurrent transactions than in today’s transaction process- ing systems. Even in the case of traditional transaction processing systems, throughput requirements are in many cases growing rapidly, while response times are limited by such factors as disk accesses. This suggests that substantial increases can be expected in multiprogramming levels.

As explained earlier, essential blocking has the disadvantage that it results in very high costs in wasted work if it is obtained using optimistic methods. However, it has been shown that essential blocking can be approximated by using infor- mation on the running/waiting states of conflicting transactions in the scheduler. Two such methods, the RP and OOR policies, offer significant performance advantages over standard locking for relatively large n or p, in terms of yielding high effective levels of concurrency at comparatively small costs in wasted work. Although the effective level of concurrency also peaks and then drops under these policies, the peaks occur for much larger values of n than with standard locking. Thus, these techniques have the potential to significantly extend the usefulness of incremental locking in contention-bound systems.

The essential blocking property can also be obtained when transactions request all required locks at initiation time. This could be the case, for example, in the proposed distributed operating systems mentioned above that use transactions as a fundamental construct, since one might expect typical operating system primitives to be “simpler” than the application programs of current transaction processing systems. This would avoid the wasted work problem of optimistic methods, but the problem that the effective level of concurrency increases at a rate of only O(log (n)) still remains. If we assume that the utilization of every processor is held fixed, the conclusion is that a linear increase in the number of processors requires an exponential increase in the total number of concurrent transactions in the case where conflicts occur independently with fixed probabil- ity. The implication is that distributed systems consisting of extremely large numbers of processors can be effectively utilized only if they can be designed so that conflicts do not occur independently (and this is used to advantage), or else designed so that the probability of conflict decreases with increasing system size. A more precise analysis requires more detailed system assumptions, and is a subject for future work.

REFERENCES

1. AGRAWAL, R., AND DEWITT, D. J. Integrated concurrency control and recovery mechanism: Design and performance evaluation. Rep. 497, Computer Sciences Dept., Univ. of Wisconsin- Madison, Feb. 1983.

2. CAREY, M. J. Modelling and evaluation of database concurrency control algorithms. Ph.D.

ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.

28 l P. Franaszek and J. T. Robinson

dissertation (UCB/ERL 83/56), Electronics Research Lab., Univ. of California, Berkeley, Sept. 1983.

3. GALLER. B. I. Concurrency control performance issues. Ph.D. dissertation, Dept. of Computer Science, Univ. of Toronto, Sept. 1982.

4. GARCIA-M• LINA, H. Performance of update algorithms for replicated data in a distributed database. Ph.D. dissertation (STAN-CS-79-744), Dept. of Computer Science, Stanford Univ., June 1979.

5. GRAY, J. Notes on database operating systems. In Notes in Computer Science 60: Operating Systems, R. Bayer, R. M. Graham, and G. Seegmuller, Eds., Springer-Verlag, Berlin, 1978, 393- 481.

6. KUNG, H. T., AND ROBINSON, J. T. On optimistic methods for concurrency control. ACM Trans. Database Syst. 6,2 (June 1981), 213-226.

7. LANGER, A. M., AND SHUM, A. W. The distribution of granule accesses made by database transactions. Commun. ACM 25, 11 (Nov. 1982), 831-832.

8. LEE, C. H. Queueing analysis of global locking synchronization schemes for multicopy databases. IEEE Trans. Comput. C-29,5 (May 1980), 371-384.

9. LIN, W. K. Performance evaluation of two concurrency control mechanisms in a distributed database system. In Proc. ACM-SIGMOD 1981 International Conference on Management of Data, ACM, New York, 84-92.

10. LOMET, D. B. Process structuring, synchronization, and recovery using atomic actions, ACM SIGPLAN Not. 12,3 (Mar. 1977).

11. POTIER, D., AND LEBLANC, P. Analysis of locking policies in database management systems. Commun. ACM 23,10 (Oct. 1980), 584-593.

12. ROBINSON, J. T. Experiments with transaction processing on a multimicroprocessor. Rep. RC 9725, IBM T. J. Watson Research Center, Yorktown Heights, N.Y., Dec. 1982.

13. ROSENKRANTZ, D. J., STEARNS, R. E., AND LEWIS, P. M., II. System level concurrency control for distributed database systems. ACM Trans. Database Syst. 3, 2 (June 19781, 178-198.

14. SPECTOR, A. Z., AND SCHWARZ, P. M. Transactions: A construct for reliable, distributed computing. Oper. Syst. Reu. 17, 2 (Apr. 1983), 18-35.

Received September 1983; revised <July 1984; accepted August 1984

ACM Transactions on Database Systems, Vol. 10, No. 1, March 1985.