Journal of Automated Reasoning 6: 287-297, 1990. 287 �9 1990 Kluwer Academic Publishers. Printed in the Netherlands.

Short Proofs of the Pigeonhole Formulas Based on the Connection Method

W. BIBEL University of British Columbia and Canadian Institute for Advanced Research*

(Received: 1 July 1988; accepted: 26 July 1989)

Al~tract. Quadratic proofs of pigeonhole formulas are presented using connection method proof techniques. The interest of this result derives from the fact that for this class of formulas exponential lower bounds are known for the length of resolution refutations.

Key words. Automated theorem proving; Connection method; Resolution; Proof complexity.

I. Introduction

Comparing the performance of different proof systems has generally been acknowl- edged as a very hard research problem. In fact, concrete mathematical results have been achieved only under drastic idealizations. One such idealization focuses on the length of proofs (in propositional logic) as a measure of comparison thus totally ignoring the cost of search for obtaining them (see [2], Section IV.3, for other approaches). It has also been driven by its relevance to the P-NP-problem.

It was long conjectured that no short resolution refutations exist of the pigeon- hole formulas Pn. These encode the principle that n + 1 pigeons cannot fit exactly into n holes. Haken [10] has finally settled this conjecture by providing a lower bound on the number of distinct steps needed for the shortest resolution refutation of Pn that is exponential in n, while Pn has length O(n 3). Recent papers [ 17, 7] have provided similar results for more classes of formulas.

This is certainly not good news for automated theorem proving (ATP), which to a large extent relies on resolution as its main proof technique. What is the value of a proof technique that takes an exponential amount of steps just to write down the proof for an intuitively simple problem, let alone one that has so many more steps usually wasted in the search for the proof. There exists a variant of resolution, called extended resolution [16], which offers short proofs of the pigeonhole formu- las, as Cook [8] has shown. Hence resolution may be regarded as computationally

inadequate in the sense that it fails to provide a proof in a short (polynomial)

* Present address: Technische Hochschule Darmstadt, B.R.D.

288 w. BIBEL

amount of time, at least for some formulas that are inherently simple (i.e. they permit polynomial proofs in other systems). On the other hand, to the best of this author's knowledge, none of the operative theorem proving systems uses extended resolution, most likely because it seems difficult to be used efficiently from a search point of view.

The connection method [2, 3], like resolution, has a state-dependent, finitely branching search space. Further it can simulate (the search for) resolution proofs [9], so it is at least as computationally adequate as resolution. The question is whether this method has the potential to do significantly better than resolution. The present paper provides a positive answer to this question simply by providing short (quadratic) proofs of the pigeonhole formulas. Similar results were obtained in [6] for a Frege proof system, and, based on an earlier version of the present paper, in [14] for a close relative of the connection method.

When referring to the connection method, we have always been using the word 'method' in its original sense of a more general approach: thus we are referring to a family of calculi that is based on the characterization of validity (or inconsistency) of a formula in terms of 'spanning sets of connections', rather than to any particular calculus in it like those described in [2, 9]. In order to establish the result of this note, we specify (in the subsequent section) the particular features that any such calculus must have in order to allow the kind of proofs presented in this paper. In other words, we prove our result for the subclass of connection calculi, that have these features. Among the features are two rules, MONOTONE and RENAME, that were not used explicitly in previous connection calculi. The important thing is that their introduction is not destructive with respect to the property of computa- tional adequacy.

The sitution for ATP might not be that bad after all. First, existing theorem provers do quite well on a wide range of problems of importance. Second, there is still a considerable potential for their improvement on the basis of techniques such as those just mentioned.

The following section summarizes the terminology needed for the presentation. Section 3 presents the connection proofs for the pigeonhole formulas and an analysis of their lengths. The final section evaluates the significance of the result.

2. Preliminaries

This paper deals with propositional logic represented in the set-theoretic form commonly used in ATP [2]. That is, we have variables x and negated variables both of which are called literals. Clauses are sets of literals, and matrices are sets of clauses. This construction of matrices may be iterated, i.e. clauses may also contain matrices, resulting in non-normal form matrices.

Matrices are usefully displayed in a 2-dimensional way as illustrated by the examples in the next section. Thereby an appropriate visual grouping is applied in the case of non-normal form matrices. The way of displaying them in this paper has

SHORT PROOFS OF THE PIGEONHOLE FORMULAS 289

been chosen with affirmative rather than refutational proofs in mind, so that clauses become columns rather than rows in the matrix. However, since the matrix representation is actually independent of its interpretation as logical formulas (apart from the minor difference just noted), the reader may read them in any of the two possible ways. In the affirmative view clauses, for instance, would be read as conjunctions, while from the refutational viewpoint taken in resolution, they are interpreted as disjunctions.

Paths (through a matrix) are sets of literals collected while traversing the matrix from left to right thereby paying attention to the nested grouping just mentioned. A path is complementary if it contains the subset {x, $}, called a connection, for some variable x. A matrix is complementary if all its paths are complementary. Depending on which way of interpretation as logical formulas the reader prefers, complementarity means validity or inconsistency of the represented propositional formula (see [2] for details).

The connection method performs theorem proving by checking the paths in some systematic way for complementarity. Various such ways have been developed. Section 11.3 in [2] describes one using linear chaining with an operation called extension. In the more advanced versions, advantage is taken of the fact that often the complementarity of one path implies the complementarity of another. Such an implication may be true for not just one but for sets of paths. A different way of applying such results is by substituting the given matrix by a reduced one (as we will do in the subsequent section) which, however, is just a different way of looking at the path-checking process. Any combination of these techniques may be used in a particular calculus from the connection-method family. A number of results of the kind just described are known.

One such result is Prawitz' reduction [15] which states the following. If the matrix contains two clauses of the form {x}uc and {~}ud then all paths containing literals from both c and d may safely be ignored, or, in other words, if the paths other than these ignored ones are complementary, then all are. As we just said, there are different ways of viewing such a result. This one may, for instance, be viewed as a clause substitution rule applicable to non-normal form matrices, in which form it reads

c {{d, {x}}, {c,

In words, if the matrix contains two clauses of the form shown on the left side of the rule then its paths all are complementary if this is true for the matrix obtained by substituting these two clauses by the single one on the right side of the rule. Note that this single clause consists of two items (which are matrices rather than just literals), hence any path will go through exactly one of them. In particular, no path will therefore contain elements from both c and d at the same time, since c is contained in the one and d in the other item. This shows that such paths are indeed ignored as stipulated above. Further illustration of this rule is given in the subsequent section.

290 w. BIBEL

Other results of this kind are the following well-known reduction rules.

PURE. Any clause containing a literal not contained in any connection may be deleted.

SUBSUMPTION. Any clause containing another one may be deleted.

UNIT. If a matrix contains a unit clause (with one literal only), then, after resolving

upon all connections containing this literal, all clauses naming (i.e. containing a literal with) its variable may be deleted.

ISOL. If the literals in a connection are not contained in any other connection, then, after resolving upon this connection, the parent clauses may be deleted.

FACTOR. If a number of clauses share a common variable, they may be replaced by a single clause with one occurrence of this literal obtained by application of the associativity and distributivity law.

MONOTONE. A path containing another one may be ignored. Consequently, if a matrix is a superset of another one, its complementarity is implied by that of the smaller one.

RENAME. Consistent renaming of variables does not affect complementarity.

Each of these reductions is illustrated in the subsequent section. Note that they can be executed fast and nondeterministically so that their costs are negligible with respect to proof-lengths (a few remarks concerning the costs involved in the search for a proof are found in the Conclusions). In what follows, we are dealing with an infinite sequence Pn, n/> 1, of sets of clauses that is defined in the following way.

e,, = . . . . , ' - - ' U x , , . \ i f f i l i 1 l<~j<k<~n+l

The set of the first n + l (purely negative) clauses is abbreviated by P~-, that of the remaining (n 3 + n2)/2 ones by P + . Then P~ contains the n �9 (n + l) variables from

the set

X n = { x v l i = 1 , . . . , n , j = 1 . . . . ,n + l}

which again may be regarded as an n x (n + l) matrix. P~ may be constructed from X~ in the following way. With each column of Xn associate a clause consisting of all negated variables from the column. With each row associate the set of 2-literal clauses obtained by assembling any two distinct variables in the row. Then obviously Pn is the set of all clauses associated with X~ in this way. We will use this association in the subsequent section for illustrating more complicated matrices Q ~ constructed from P~.

The matrices for PI and P2 are depicted in the next section. P~ encodes the statement that n + 1 'pigeons' can fit exactly into n 'holes' which obviously expresses an inconsistency. For this encoding and in the affirmative view, x o. may be read as 'hole i is occupied by pigeon j ' , whereby we use the standard format

SHORT PROOFS OF THE PIGEONHOLE FORMULAS 291

axioms ~ theorem with no theorem taken into consideration here since the axioms

are already inconsistent.

3. Short Connection Proofs for Pn

In the present section we prove the following result.

LEMMA. There is a short proof for Pn in the connection method, the length o f which

is quadratic in the number o f clauses in Pn, n >>. 1.

The proof is by induction on n. There are two extensions needed for P~ as illustrated by the two connections in

its matrix representation.

~ X l l Xll 2 1 2 7 12

There are ten extensions needed for P2 as illustrated again by the twelve resulting connections in its matrix representation where two clauses are listed twice for an easier depiction.

f ' X , l Xll J-~12 X22 -~23 _X13~ X21 3~22 X12 "~13 ~X23 /_/ f_71,..3

\ X21 XI2 X22 X23 XI3 Xll X22 XI2 X13 X23 ....~X21

So, as the induction hypothesis, let us assume that there is a proof for P , _ 1 of appropriate length. We will reduce P, to Pn _ ~ by a number of reduction operations. Although this will, of course, be done for arbitrary n, each of these operations will at the same time be illustrated for the generic case n = 3. For this special case the matrix P3 is the following one.

-~11 "~12 X13 ~14 XII XII XI1 XI2 X12 X13 Y21 X21 X21 X22 X22 X23 X31 X31 X31 X32 X32 X33

X21 "~22 "~23 -~24 Y12 X13 XI4 XI3 XI4 X14 X22 X23 X24 X23 X24 Y24 X32 X33 X34 X33 X34 X34.

-~3I 3C32 X33 -~34

The reduction of P~ to P~_ ~ will in turn be proved by induction. For that purpose we consider the following sequence of matrices Qnm, m = 1 . . . . , n + 1.

( 0 t= , {{2~1 . . . . . ,Xm}})k-)(tn~m~ = 1 {{'~] . . . . . . )~n-- 1 i} ' )

U \ l = l i~J<k~n+l

By comparison with the definition of Pn, given in Section 2 it is obvious from this representation that Q~(~+, = P~. Apart from such a comparison the reader may gain a better understanding of the matrices Q~,, by way of the following nice

292 w. BIBEL

illustration 1 in terms of the association of clauses with the matrix X, defined in the previous section. For this purpose we generalize this association to matrices resulting from X," by deleting (or forbidding) some of its elements. Consider the following matrix.

Xll XI2 X13 Xl4

X21 X22 X23 X24"

X31 X32 �9 �9

It represents X3 from which x33 and x34 are deleted in this way. The set of clauses associated with it is exactly Q32. In general, Q,~ is the set of clauses associated with X," from which x,'i, i = m + l , . . . , n + l, is deleted. Figure 1 shows the reduction steps from P3 to P2 illustrated by way of this representation. A display of the full set of clauses of Q,m for the special case n = 3 will be given shortly.

The base case for the induction on m will be handled further below. So we now turn our attention to the induction step in which Q,m will be reduced to Q,(,._ 1) using the reduction rules from the previous section. For that purpose we focus our attention on those clauses in Q,',~ that contain the variable x,,. which are the following ones.

Xlm

: X,'l Xnm-- 1

2,,_ 1,, X,'m X,'m"

,'m

Factoring reduces this to the following submatrix consisting of two clauses.

Xlm

: X,'l "'" X . m - 1"

X . _ lm X .m

Xnm

Prawitz reduction allows us to restrict our attention to the set of paths through the entire matrix that are identified by the following two (left and right) submatrices.

Xnl �9 X n m - 1

Xnm

Xlm

2n - lm Xnm"

Notice that superimposing the left submatrix on the right one results in the previous submatrix. In terms of the notation used in the Prawitz rule given in the previous section, these two submatrices are the two items in the clause on the right side of the rule. If we wished to be in accordance with our two-dimensional representation,

i Suggested by one of the referees.

SHORT PROOFS OF THE PIGEONHOLE FORMULAS

2:11 212 213 214

221 222 223 ~24

X31 3732 233 Z'34

293

211 XI2 213 �9 211 -T12 213 214

"~21 X22 223 �9 :r21 ~22 ~23 224

�9 �9 �9 �9 ~,31 ~ ~,33 �9

Xl l X12 �9 214 211 ,212 Z13 ,214

X21 ~22 �9 .,'Le 24 .T21 ,.T 2"2 .,~23 .,T24 q,3"2

�9 �9 �9 �9 ~31 ~ �9 �9

X l l �9 -~13 214 ~11 "12 X13 ~14

X21 �9 X23 224 221 222 X23 224 Q31

�9 �9 �9 �9 ~31 �9 �9 �9

Fig. I. The reduction of P3 to P2.

these two items actually had better been presented one on top o f the other (as all

clauses are presented this way). The c in the second i tem o f the rule here consists o f one element x , ~ . . , x ,m_ ~ being itself a matr ix with m - 1 clauses, while d here is

a clause containing the literals $~ . . . . . , :~,_ ~,,.

Fo r each of these two submatr ices a long with the remaining clauses o f Q,,~

complementa r i ty has to be established which m a y be done independent ly (unless

there is good reason for combining the two tasks). Hence we will talk o f the first, corresponding to the left submatr ix , and the second case. In the special case o f

Q34 = P3 the first case is given by the following matr ix.

XII XI2 XI3 - Xll X12 X13 X21 X22 X23 X31 X31 X32 p f . X34 X31 X32 X33 X21 X22 X23

g14 XI4 XI4 X24 X24 X24 X32 X33 X33 X31 X32 X33

X,r~ is pure, so this unit clause m a y be deleted. Similarly, x;m, i = 1 . . . . , n - I, are all pure as well, hence all clauses containing them m a y be deleted. Fur ther , all

clauses subsumed by the unit clauses x. , , i = 1 . . . . . m - 1, will be deleted as well.

The same unit clauses m a y also be used to remove themselves and the literals ft,, f rom the negative clauses {xt . . . . . . x,i} by ISOL-reduct ion , i = 1 . . . . . m - 1. In the resulting matr ix , i l lustrated by the left subnodes in Figure 1, we rename the variables by exchanging the names x;~,+~) and x,,, wherever they occur, i = 1 . . . . . n - 1. The result is P , _ 1 to which the induct ion hypothesis ( for n) can be applied.

294 w. BIBEL

Having solved the first case we now turn our attention to the second one. Again, as an aid to the reader, we depict the matrix for the special case of Q34 = P3.

Xll xl2 xl3 3~21 )~22 3~23 XI4 2~34 XII X12 x13 x21 X22 X23 X31 X31 X32 p~-.

X31 x32 x33 X24 X14 x14 x14 X24 x24 X24 X32 x33 X33

The variable X,,m is pure, hence the unit clause may be deleted. The remaining matrix, by construction, is just Qn(m-~ as defined further above and which is illustrated as the right subnodes in Figure 1. Hence in this second case the induction

hypothesis (on m) can be applied again. For illustration of the base case for the m-induction, which has still to be proved,

let us present the matrix of Q31, which is the following one:

X12 X13 X14 Xll Xll Xll X12 X12 XI3 X21 X21 X21 X22 X22 X23

X22 X23 .X24 X12 XI3 X14 X13 X14 XI4 X22 X23 X24 X23 X24 X24. )~31

Qn~ contains Pn-~ as a submatrix up to renaming the variables. Indeed, after exchanging the variable names x;m and x,.~+~ wherever they occur, for i = 1 . . . . . n - 1, we explicitly have P~_ l c Q~I. The monotonicity property thus settles the base case by way of the induction hypothesis, which is the last piece

needed for the full proof. Let us now turn our attention to the length of the connection proof obtained for

P~. For the reduction of Pn to P~_ 1 we had to carry out n steps. Each of these involved identifying and processing the first and the second case. The identification is of length O(n). O(n 2) steps are needed for performing each of these first cases which adds up to O(n 3) steps for this first-case part. This then turns out to be the overall complexity, since it also majorizes the constant amount needed to carry out each of the second cases and the linear time needed for the renaming in the base case. Since n such reduction steps have to be carried out and since the number I of clauses is O(n3), the overall complexity is O(n 4) or 0 ( l �9 n). With a little more effort invested in handling the first case (since it is basically the same each time) and on the actual calculation of the complexity we could have obtained a linear proof, but for the purposes of this paper it would not have made any difference anyway.

4. Conclusions

In the previous section we presented short proofs of pigeonhole formulas based on proof techniques provided by the connection method. In a sense this amounts to the presentation of a simple exercise. But the significance of this contribution lies not in these proofs themselves, rather in the consequences of their existence for the evaluation of the connection method, in particular in view of Haken's result. It also points to ways of further improvements of actual implementations.

The crucial feature that avoids the exponential explosion in our proofs lies in their capability to take advantage of renaming and of subsumption on a global

SHORT PROOFS OF THE PIGEONHOLE FORMULAS 295

level. If a subproblem is subsumed by another one (by monotonicity), it can be ignored. Put in a more general (and technical) way, if a path is subsumed by another one in the same problem, it can be ignored during the proof process. Resolution alone (but see below) has no such global capabilities. To understand this correctly one has to keep in mind that the resolvent is not a subproblem in this sense. Rather one has to add the resolvent: to the given set of clauses in order to obtain the resulting subproblem [4].

From a mathematician's viewpoint this feature in essence may be regarded as the capability of recognizing lemmas and applying them more than once. Because of its significance this capability to some extent was built into the theorem prover SETHEO [1, 12] which is based on the connection method. It does not, however, currently fully cover the features RENAME and MONOTONE as presented in this paper. Just as an aside we mention that even without these crucial features SETHEO proves P4 in 1.4 seconds on a Sun 4/260 performing 393 inferences.

The connection proofs of the previous section are presented in a way that does not exactly follow the connection calculus in [2]. For the purpose of this paper, using the Prawitz matrix reduction rule made it easier for the reader. Since this rule is incorporated in the more advanced forms of the connection method (see [2], Section IV.6), it is obvious that we could as well have stuck strictly to the techniques provided by the connection method, as already indicated in Section 2.

With this paper's observation of a crucial difference between resolution and the connection method, a belief expressed in [13] (p. 379) turns out at least to be questionable, depending on what exactly was meant by it. The belief was that "resolution strategies may be able to mimic the matrix reduction strategies of importance". What we know at present is the following.

Resolution can mimic derivations in the connection calculus presented in Chapter III of [2] even in a step-by-step fashion, as shown in Proposition 3.1 of [9]. But the converse is true as well for a generalized connection calculus, as shown in Proposition 8.2 of the same reference. The generalization allows for the reuse of connection structures more than once in a proof for which reason Eder calls it the connection structure calculus. Both results are obtained for first-order logic. Further the present paper has shown that with the integration of several features such as renaming, subsumption, and monotonicity, any connection-method calculus does exponentially better on the pigeonhole formulas than resolution. However, extended forms of resolution - like extended resolution, but also symmetric resolution [ 11] - do provide short proofs for the pigeonhole formulas, but they seem not to be used in actual implementations.

All these results focus on the length of derivations so that one might question how relevant they actually are for realistic theorem proving. Eder's results certainly carry over to the case where search is included since they provide step-by-step simulations (with constant costs per step execution) so that any search strategy easily translates from one calculus to the other. But what are the search costs involved in usefully applying the reduction operations such as renaming and (local

296 w. BIBEL

and global) subsumption? This paper cannot be expected to answer this question in any detail. A simple answer is that their applicability can be tested in polynomial time so that from a complexity point of view even testing each of them after every step (however 'step' is defined) in the proof search would result in a polynomial behavior for the pigeonhole formulas. Obviously this is not a satisfactory answer, since the behavior most likely would still be infeasible from a practical point of view. These questions have to be attacked in the broader context of the method involved. Because of its global nature, the connection method continues to offer an attractive potential in particular for a more problem sensitive guidance of the proof search.

At any rate our result underlines once again the importance of the simplification operations. In particular, it emphasizes the renaming and the global subsumption operations that seem to have been neglected in actual implementations. If a feasible proof mechanism exists at all, it might necessarily consist of a combination of a basic logical operation (such as the one in the connection structure calculus) with a (hopefully small) number of special reduction and preprocessing operations. But even if it does not exist, the same might be true for a computationally adequate mechanism (recall this concept from the Introduction) that fails for intrinsically hard problems only. Under this aspect one might question the practical relevance of possible future work on the complexity of any proof method unless it takes these operations into account (if they make a difference).

Of course, one would now like to see further natural questions being settled. The most obvious one is how does the connection method perform on other hard examples for resolution such as those in [17]? 2 Also, which are the hard examples for the connection method (unless P = NP)? Further it seems that the technique of factoring also used in our proofs above has a similar power in the context of the connection method as the generalized matrix reduction method [5]. To see the reason for such a belief, the interested reader might have a look at Example 3.6 there. Factoring the variables x5 and x6 in that example and in the same way as in the proofs of the previous section, result in the same sort of short proof that is demonstrated there with the generalized matrix reduction method. In fact, factoring along with the possibility of deleting paths containing the empty set may substitute subsumption which leads to the question whether there are more general and fewer principles that permit the substitution of some of the reduction rules mentioned in this paper.

Acknowledgements

I appreciate inspiring discussions with D. Kirkpatrick, H. Levesque, and A. Mackworth, as well as helpful comments on a previous version of this note by A. Urquhart and L. Wallen. Furthermore I owe a number of improvements to suggestions from three anonymous referees.

2 Urquhart (personal communication) has shown that the addition of a form of RENAME (comple- menting) provides linear resolution proofs for the Tseitin graph-based clauses which seems to carry over to the connection proofs as well.

SHORT PROOFS OF THE PIGEONHOLE FORMULAS 297

References

1. Bayerl, S., Eder, E., Kurfess, F., Letz, R. and Schumann, J., 'An implementation ofa PROLOG-like theorem prover based on the connection method', in Ph. Jorrand and V. Sgurev (editors), AIMSA '86, Artificial Intelligence II - Methodology, Systems, Applications, pp. 29-36, North-Hol- land, Amsterdam, 1987.

2. Bibel, W., Automated Theorem Proving. Vieweg Verlag, Braunschweig, Second edition, 1987. 3. Bibel, W., 'Matings in matrices', Comm. ACM, 26 (1983) 844-852. 4. Bibel, W., 'On matrices with connections', JACM, 28 (1981) 633-645. 5. Bibel, W., 'Tautology testing with a generalized matrix reduction method', Theor. Comput. Sci., 8

(1979) 31-44. 6. Buss, S. R., 'Polynomial size proofs of the propositional pigeonhole principle', The Journal of

Symbolic Logic, 52(4) (1987) 916-927. 7. Chv~tal, V. and Szemer&ii, E., 'Many hard examples for resolution', JACM, to appear. 8. Cook, S. A., 'A short proof of the pigeon hole principle using extended resolution', ACM SIGACT

News, 8 (Oct.-Dec. 1976) 28-32. 9. Eder, E., 'A comparison of the resolution calculus and the connection method, and a new calculus

generalizing both methods'. To appear in Computer Science Logic, Lecture Notes in Computer Science, Springer, Berlin.

10. Haken, A., 'The intractability of resolution', Theor. Comput. Sci., 39 (1985) 297-308. 11. Krishnamurthy, B., 'Short proofs for tricky formulas', Acta Informatica, 22(3) (1985) 253-275. 12. Letz, R., Schumann, J. and Bayed, S., 'SETHEO - A sequential theorem prover for first-order

logic', Technische Universit/it M/inchen, 1988. 13. Loveland, D. W., 'A unifying view of some linear Herbrand procedures', JACM, 19 (1972) 366-384. 14. Murray, N. V. and Rosenthal, E., Short Proofs of Pigeonhole Formulas Using Path Dissolution and

Semantic Graphs, Technical Report, Dept. of Computer Science, SUNY at Albany, 1988. 15. Prawitz, D., 'A proof procedure with matrix reduction', in M. Laudet et at. (editors), Symposium on

Automatic Demonstration, Lecture Notes in Math. 125, pp. 207-214, Springer, Berlin, 1970. 16. Tseitin, G. S., 'On the complexity of derivation in propositional calculus', in A. O. Slisenko (editor),

Studies in Constructive Mathematics and Mathematical Logic, Part II, pp. 234-259, Seminars in Mathematics, V. A. Steklov Mathematical Institute, vol. 8, Leningrad, 1968. English translation: Consultants Bureau, New York, 1970, pp. 115-125.

17. Urquhart, A., 'Hard examples for resolution', JACM, 34 (1987) 209-219.