THE MINIMAL BASE SIZE OF PRIMITIVE SOLVABLEPERMUTATION GROUPS

AKOS SERESS

ABSTRACT

A base of a permutation group G is a sequence B of points from the permutation domain such that onlythe identity of G fixes B pointwise. Answering a question of Pyber, we prove that all primitive solvablepermutation groups have a base of size at most four.

1. Introduction

In his seminal work [18], Sims introduced the notion of a base as one of thefundamental data structures for computing with permutation groups. A base for apermutation group G ^ Sym (Q) of degree n is a sequence B = (/?l9 /?2, . . . , fiM) from Clwith the property that only the identity of G fixes each point of B. A base naturallydefines a chain of subgroups

G = G[1] ^ G[2] ^ ... ^ G[M+1] = 1, (1)

where G[i] = G{Pi p } is the subgroup of G which fixes pointwise the set {filt... ,&_!},1 ^ / ^ M + l . The base is called non-redundant if there is a strict inclusionQW > Qd+i) for ajj i <£. i ^. M. Sims' method computes a base and (right) transversalsRt for GmmodG[<+1]. Each element of G can be factored uniquely in the formg = rM...r2r15 rtei?4. This factorization can be done algorithmically, and it isessentially the only way to test membership in G.

Not surprisingly, this factorization process (and, consequently, any morecomplicated algorithm which uses it as a subroutine) is faster when the base is small.Current computational interest centres around groups with small (size at mostpolylogarithmic as a function of n) bases; all permutation representations of finitesimple groups except the alternating ones and most primitive groups fall into thiscategory. By a result of Liebeck [9], a primitive G ^ Sn has a base of size at most

(m\ ®91og« unless n = I and G is a subgroup of Sm\,Sr containing (Am)r, where Am

acts on the /c-element subsets. Recently, a significant library of algorithms [1, 2,3,10,16, 17] was developed which works especially well in the case of input groups withsmall base.

If B is a non-redundant base, then 2 ^ \GW: Gc<+1]| ^ n for all i ^ M, and so wehave

2|B| ^ \G\ ^ «|B|. (2)

Received 20 June 1994.

1991 Mathematics Subject Classification 20B99.

Research partially supported by NSF grants CCR-9201303 and CCR-9503430, and NSA grantMDA904-92-H-3046.

J. London Math. Soc. (2) 53 (1996) 243-255

244 AKOS SERESS

Hence \G\ determines the minimal base size up to a factor logn. It is easy to constructexamples such that the minimal base size is close to either bound given in (2).However, it is observed in practice that primitive groups have rather small bases,within a constant factor of log|G|/logn. In his excellent survey [14, p. 207], L. Pyberasks the question whether there is an absolute constant c such that all primitivegroups of degree n have minimal base size at most clog|G|/log/i. The purpose of thispaper is to answer Pyber's question in the first non-trivial case, for primitive solvablegroups.

THEOREM 1.1. All primitive solvable permutation groups have a base of size at mostfour.

Theorem 1.1 is best possible: by a result of Palfy [11] and Wolf [21], if G ^ Sn isprimitive solvable, then |G| ^ 24 - 1 / V with c = 3.243..., and equality is attained forinfinitely many values of n. Hence, by (2), there are arbitrarily large primitive solvablegroups with minimal base size 4.

Our method is mostly combinatorial. It is well known that if G ^ Sn is primitivesolvable, then n = pd for some prime p, and the stabilizer H of a point is isomorphicto an irreducible subgroup of GL(d,p). First, using an extension of a counting methodof Gluck and Manz [7], we handle the case of primitive (as a linear group) H. Gluckand Manz showed that if a certain parameter e, defined in Section 2, has value at least132, then H acts regularly on one of its orbits and so has a base of size 1. For mostsmaller values, a variant of the argument shows that H has a base of size at most 3.When this counting argument bottoms out, we use some group theory (mostly,information about maximal solvable subgroups in certain low dimensional symplecticgroups). For some small examples, even this method is not sufficient; in these cases,we construct the groups using the GAP system [15] and check that they have a smallbase.

While handling the case of imprimitive H, we prove the following fact which maybe of independent interest.

THEOREM 1.2. Let G ^ Sym (Q) be an arbitrary solvable permutation group. Thenthere is a partition PofCl into at most five parts such that only the identity element ofG fixes P.

The number 5 in Theorem 1.2 is best possible. For example, S4 X C2 in its naturalimprimitive action on 8 points requires five parts, since if the first four points do notfall into different parts, then an element of S4 x 1 fixes the partition. Similarly, thesecond four points must fall into different parts. However, if there are only four partsaltogether, some element of S4 X C2 fixes the partition. Slightly more generally, thesame argument shows that for any transitive H^ Sk, S4\,H acting on 4k pointsrequires five parts.

For primitive groups of odd order, Theorem 1.1 can be strengthened.

THEOREM 1.3. All primitive permutation groups of odd order have a base of size atmost three.

Palfy [12] showed that if G ^ Sn is primitive of odd order, then \G\ ^ 3"1/2«c withc = 2.278..., and equality is attained for infinitely many values of/?. Hence there are

MINIMAL BASE SIZE 245

arbitrarily large primitive groups of odd order with minimal base size 3. The analogueof Theorem 1.2 for odd order groups was proven by Gluck [6]: there is always apartition into two parts whose stabilizer in G is the identity.

Although the problem investigated in this paper was motivated by algorithmicconsiderations, our methods provide no easy way to construct a minimal base. By aresult of Blaha [4], finding the minimal base size in arbitrary permutation groups isan NP-hard problem. In view of Theorem 1.1, the minimal base size in primitivesolvable groups can be found in polynomial time (just try all subsets of size at most4), but for practical applications we need something faster. Blaha also proved that thegreedy base construction (that is, given an initial segment D of a base, the next basepoint is always chosen from the largest orbit of the pointwise stabilizer of D) yieldsa base within a factor of 0(loglog«) of the minimal base size. In the case of primitivelinear H, for most values of e it can be shown that the greedy base of H has size atmost 3. We conjecture that the greedy base has size at most 4 for all primitive solvablepermutation groups.

Any G ^ Sym (Q) acts naturally on Qfc := Q x ... x Q (k copies), by the rule(a15..., OLk)

g := (a?,. . . , a£). Wielandt [20] defined the k-closure G(k) of G as the largestsubgroup of Sym (Q) respecting the G-orbits on Qfc. Clearly,

... ^ G{*-X) ^ Gik) ^ . . ^ G

In [13], Praeger and Saxl ask for a description of ^-closures of primitive groups ofaffine type. Wielandt proved [20, 5.12] that if G has a base of size at most k— 1, thenG{k) = G. Combining this result with Theorem 1.1, we obtain that for a primitivesolvable group, G(b) = G.

2. Primitive linear groups

Let G ^ Sym (Q) be a primitive solvable permutation group. We identify Q withthe vector space V = GF(/?)d, p prime, and denote Go, the stabilizer of the 0 elementof V, by H. In order to prove Theorem 1.1, we have to show that H has a base ofsize at most three. In this section, we handle the case when H ^ GL{d,p) is a primitivelinear group. We say that the bases A = (a15..., <xM), B = (/?15... ,/?M) for H are non-equivalent if there is no geH satisfying a? = (3t for 1 ^ / ^ M.

THEOREM 2.1. Let H ^ Gh{d,p) be a primitive solvable linear group, acting onV = GF(p)d. Then H has a base of size at most 3. Moreover, when the minimal base sizeof H is 3, H has at least five non-equivalent minimal bases.

The technical condition about non-equivalent bases will be used in the handlingof imprimitive linear groups.

It is enough to prove Theorem 2.1 in the case when H is a maximal primitivesolvable linear group (with respect to inclusion). The following lemma collects therelevant properties of such groups from [19, §§19-20], [12, Proposition 2.1] and [21,Corollary 2.4].

LEMMA 2.2. Let H ^ GL(d,p) be a maximal solvable primitive group. Then H hasa unique maximal abelian normal subgroup A. Let C = CH(A) and B = Fit(C), theFitting subgroup of C. Then we have the following.

(a) A is cyclic of order pa—\ for some integer a.(b) The linear span of A in Mat(d,/7) is the field GF(/?a).

246 AKOS SERESS

(c) H/C is isomorphic to a subgroup of Aut(GF(pa)).(d) d = ae for an integer e.(e) C ̂ GL(e,pa).(f) Let e = Y\ $ be the prime decomposition ofe. Then each qi divides \A\ = pa — 1.(g) \B/A\ = e2 and B/A is the direct product of elementary abelian groups.(h) There is a C-invariant non-degenerate symplectic form on each Sylow subgroup

of B/A.(i) C/B is isomorphic to a completely reducible subgroup of the direct product of

symplectic groups Sp (2et, qt).(j) For each q{, the q{-Sylow subgroup of B can be written in the form EQ( TQi, where

EQi is an extraspecial group of order q26i+1, TQ( is the qt-Sylow subgroup of A, andE9( n TQ( = Z{EQ). Ifqt>2, then Et has exponent qt.

(k) The degree of the irreducible components of EQ( over GF(pa) is q*1.

Throughout this section, we shall use the notation a,e,A,B,C introduced inLemma 2.2.

For any geH, the fixed points of g comprise a subspace of V = G¥(p)d. Gluckand Manz [7] observed that this subspace is of size o(pd). More exactly, the followingholds. For geH, let Cv(g) denote the set of fixed points of g.

LEMMA 2.3. Let H ^ GL(d,p) be a maximal solvable primitive group. Then(a) ifgeA\\,then\Cv(g)\ = \;(b) ifgeB\A, then\Cv(g)\^(pd)m;(c) ifgeC\B, thenlCMl^ipT*;(d) ifgeH\C,then\Cv(g)\^{pT2.

Proof. See the first part of the proof of [7, Proposition 4].

Gluck and Manz used Lemma 2.3 to prove that if e ^ 132, then H has a regularorbit, by showing that the union of the sets Cv(g) cannot cover V. A similar argumentshows that for most values of e, the minimal base size of H is at most 3.

LEMMA 2.4. Let H ̂ GL(d,p) be a maximal solvable primitive group, d = ae, andsuppose that e ^ 3. Then B has a regular orbit unless the pair (p°, e) is one of {A, 3), (3,4)and (5,4).

Proof Let us first suppose that e ^ 5. Since the elements of A have no non-zero fixed points and \B\A\ = (pa-1)(^2 — 1),

L (\Cv(g)\-\) ^ (pa-l)(e*geB

Therefore, if( ^ l ) ( 2 l ) ( ^ 2 - l ) < J p - - l , (3)

then some v e V\{0] is not covered by any of the fixed point sets Cv(g), for g # 1. Thismeans that v is in a regular orbit of B.

It is easy to see that the inequality

e2 < (paye-2»2 (4)

implies (3). Ifpa ^ 9, then (4) holds for all e ^ 5. If/?0 = 7, then (4) holds for e ̂ 6.However, e = 5 is impossible, since the prime divisors of e must divide pa — 1.Similarly, if pa = 5, then (4) holds for e ̂ 7. Now e must be a power of 2, so e ^ 8.

MINIMAL BASE SIZE 247

In the cases e = 3,4, we have to refine the argument above. Suppose that e = 4and let E be the extraspecial 2-group involved in B, \E\ = 32. The elements of B\A canbe written in the form gx for some geE\A and xeA. If the order of gx is not 2, then(gx)2eA\l and so |CV(gx)| = 1. For fixed geE\A, there are at most two xeA suchthat (gx)2 = 1 since the square of their ratio must be 1. Hence

geB

Now I V\ — pia, sop" ^ 8 implies that B has a regular orbit. In the case/;" = 7, we needthe additional observation that if geE\A has order 4, then g2 is an order 2 elementof ,4 and so there is no xeA with the property that (gx)2 — 1. Since the number ofelements of E\A of order 2 is at most 18 [8, 5.3],

geB

and so B has a regular orbit.Finally, let e = 3 and let is be the extraspecial 3-group involved in B, \E\ = 27. The

elements of B\A can be written in the form gx for some ge E\A and xeA. If the orderof gx is not 3, then (gx)3eA\\ and so |CV(gx)| = 1. For fixed geE\A, there are atmost three xeA such that (gx)3 = 1, and so there are at most 72 elements of order 3in B\A. Moreover, if (gx)3 = 1, then gx and (gx)2 have the same fixed point set, so weneed veV\{0}, which avoids at most 36 sets of size at most pZal2 — \.36(p3al2-\) <p3a-l holds for pa ^ 11 so, since 31 (pa-1), the only subcase left ispa = 7. If pa = 7, then the fixed point spaces of elements of B\A are at most one-dimensional, hence we need veV\{0}, avoiding at most 36 sets of size 6. Since216 < 73— 1, such v exists.

We remark that Lemma 2.4 holds for the pairs (pa,e) = (4,3), (5,4) as well, butwe shall not use this fact in the rest of the proof.

LEMMA 2.5. Let H ^ GL(d,p) be a maximal solvable primitive group, d= ae,

/T, (5)

then H satisfies the conclusion of Theorem 2.1.

Proof. By Lemma 2.4, there exists v1eV\{0} such that HV (]B = 1. ThereforeHv is isomorphic to a subgroup of H/B, and Hv 0 C is isomorphic to a subgroup ofC/B. Since the fixed point spaces of elements of C are subspaces of a vector space overGF(pa), Lemma 2.3 implies

E (\Cv(g)\-\)^\C/B\(pa^-\).ge(Hv nC)\l

Therefore there exists v2 e F\{0} which is covered by at most

^ - \ ) \C/B\(pae-\) p*e/i]

248 AKOS SERESS

of the sets Cv(g). This implies

Applying Lemma 2.3 again, we obtain

If this quantity is less than \V\ — 4\HV v |, then Hv v has at least five regular orbits.Choosing v3 from such a regular orbit, the sequence (y15 i>2, v3) is a base of//. Choosingi>3 from different regular orbits of Hv v, we obtain inequivalent bases. Finally, weobserve that using (6) and \H \ ̂ o\H n C|, (5) implies

COROLLARY 2.6. Le? / / ^ GL(d,p) be a maximal solvable primitive group, andd= ae. If e~^ 16, then H satisfies the conclusion of Theorem 2.1 unless pa = 3 and

If |C/£| ^/7a(e/4), then it is easy to check that (5) holds. Otherwise, weuse the fact that C/B acts completely reducibly on a module of size e2 so, bythe Palfy-Wolf bound [11, 21], \C/B\ ^ 24 - 1 / V ) 2 2 4 3 ' " < e*-&/2. Hence(eib/pae/i)(p3ae/i + (a-\)pael* + 4a) is an upper estimate for the left-hand side of (5),and it is enough to prove that

f ^ ( p ^ + ia_l)pae/2 + 4a) <pae ( ? )

For fixed pa, if (7) holds for e = 16 then it holds for all greater values of e as well:increasing e by 1, the left-hand side increases by a factor at most (17/16)45/Ja/2 whilethe right-hand side increases by a factorpa. (17/16)45 < 1.32, so (17/16)4>O/2 <pa

for all pa ^ 3. In the case e = 16, it is straightforward to check that (7) holds forpa ^ 5. If pa = 4 then e must be a power of 3, so e ^ 27. Again, it is easy to check that(7) holds. Finally, if pa = 3 then e must be a power of 2 and so e ^ 32 and (7) holds.

For the values e ^ 15, we use sharper estimates for \C/B\ in (5). The followinglemma is a simple corollary of the classification of maximal irreducible solvablesubgroups of SL(2,/>) = Sp(2,/?) in [19, §21].

LEMMA 2.7. Le? 3 ̂ p ^ 13, /> prime, and let K be a completely reducible solvablesubgroup ofSp(2,p). Then \K\ ^ 24 for p = 3,5,11, \K\ ^ 28 for p = 13, and \K\ ^ 48for p = 1.

Using Lemma 2.7 and the fact that solvable subgroups of Sp(4,2) = S6 have orderat most 72, we obtain that Lemma 2.5 is applicable for most values 3 ^ e < 15.

LEMMA 2.8. Let H ^ GL(d,p) be a maximal solvable primitive group, and d = ae.If '3 t^e ^ 7 or 10 ̂ e < 15, then H satisfies the conclusion of Theorem 2.1 unless thepair (pa,e) is one of (4,3), (3,4), (5,4) and (1,4).

MINIMAL BASE SIZE 249

Proof. If e=15 , then C/B ^ Sp(2,3) x Sp(2,5) and, by Lemma 2.7,\C/B\ ^ 242 = 576. Also, 3 and 5 must divide pa-1, so /?a ̂ 16. Substituting theseestimates into (5), it is easy to see that (5) holds. Similarly:if e = 14, then C/B ^ Sp(2,2) x Sp(2,7), \C/B\ ^ 6 • 48 = 288, pa>29;if e = 13, then C/B < Sp(2,13), \C/B\ ^ 28, pa ^ 27;if <? = 12, then C/B ^ Sp(2,3) x Sp(4,2), | C/£| ^ 24 • 72 = 1728, pa ^ 7;if e = 11, then C/5 < Sp(2,11), \C/B\ ^ 24, pa ^ 23;if e = 10, then C/5 ^ Sp(2,2) x Sp(2,5), \C/B\ ^ 6 • 24 = 144, pa^U;if e = 7, then C/5 < Sp(2,7), \C/B\ ^ 48, pa ^ 8;ife = 6, then C/5 ^ Sp(2,2) x Sp(2,3), |C/5| ^ 6-24 = 144, pa ^ 7;if e = 5, then C/£ ^ Sp(2,5), \C/B\ ^24,pa^\\;if e = 4, then C/5 ^ Sp(4,2), |C/,S| ^ 72, pa is odd;if e = 3, then C/£ ^ Sp(2,3), |C/5| ^ 24, pa = 4 or pa ^ 7.In all cases, with the exceptions mentioned in the statement of the lemma, it isstraightforward to check that (5) holds.

In some of the remaining cases, we shall use the following variant of Lemma 2.5.In all applications, a will be 1, so we do not state the most general version.

LEMMA 2.9. Let H ^ GL(e,p) be a maximal solvable primitive group, e ̂ 4, and(p,e) # (3,4), (5,4). Moreover, let x denote the number of elements of prime order inH/B. If

[\ (8)then H satisfies the conclusion of Theorem 2.1.

Proof. By Lemma 2.4, there exists vx e K\{0} such that Hv OB = 1, and so Hv isisomorphic to a subgroup of H/B. By Lemma 2.3, each element of Hv of prime orderhas a fixed point space of size at most /?3e/4, which implies that some v2e V\{0} iscovered by at most |x//?e/4] of them. Hence Hv v contains at most [x/pe/4\ elementsof prime order. The union of fixed points spaces of these elements covers at most[x/pel4\piel4 elements of V. Choosing v3 outside this union, Hv v v contains no elementsof prime order, that is,Hvvv = 1. In addition, if (8) is satisfied, then Hv v has at leastfive regular orbits, and so H has at least five non-equivalent bases.

Let D and Q denote the dihedral group of order 8 and the quaternion group,respectively. In the cases when e is a power of 2, B = EA for some extraspecial 2-group E, \E\ = 2e2. If pa = 3(mod4) then E is the unique 2-Sylow subgroup of B; inparticular, B determines E. The symplectic form/: B/A x B/A -> GF(2) provided bythe commutator mapping of E is symmetric. Also, the squaring map defines aquadratic form on E, compatible with /. Hence the factor group C/B is isomorphicto a subgroup of the orthogonal group O+(2m, 2) or O~(2m,2), for e = 2m. Ifpa = l(mod 4), then E is not determined uniquely. In fact, since Q * C4 and D * C4 areisomorphic, both types of extraspecial groups of order 2e2 occur in B.

Now we handle the remaining cases for the value of e. For e = 3, the only subcaseleft ispa = 4. We construct the primitive group Hin the GAP system. The extraspecial3-group B = E, \E\ = 27, has exponent 3, so it is unique up to isomorphism. AlthoughE has two irreducible 3-dimensional representations over GF(4), these repre-sentations, considered as subgroups of GL(6,2), are conjugate. Hence it is enough to

250 AKOS SERESS

construct any irreducible representation of E, and take the normalizer N in GL(6,2).It turns out that N has minimal base size 3. This group N = (31+2SL(2,3))C2

^ GL(6,2) is one of the two primitive linear groups we are aware of which do nothave a base of size at most 2.

If e = 4 and pa = 3(mod4), then E^D*Q or E^D*D. UE^D*Q, then C/Bis isomorphic to a solvable subgroup of O"(4,2) = Sb, so \C/B\ ^ 2 4 . If E ̂ D*D,then C/B is isomorphic to a subgroup of O+(4,2) = 5 3 \ . S 2 .

Now let us consider the subcase pa = 7.HE = D*Q, then \C/B\ ^ 24 and Lemma2.5 is applicable. If E = D*D then, using the GAP system, we find thatO+(4,2) = 5"3 X S2 has 21 elements of order 2, and 8 elements of order 3. Thus Lemma2.9 is applicable.

For pae{3,5}, we construct the primitive groups H in the GAP system. Bothextraspecial 2-groups E of order 32 have a unique irreducible representation ofdimension 4 over GF(3) and GF(5). In the case pa = 5, we obtain B by adding thescalar matrices to the generators of E. (We obtain the same B no matter whichextraspecial group we start with.) After that, we take the normalizer N of B inGL(4,/?a). For pa = 5, it turns out that even the non-solvable group N = 2?Sp(4,2)has a base of size 2. ¥or pa = 3, ifis = D*Q, then A^has a base of size 2. If E = D*D,then N has a base of size 3. This group N = (D*D) O+(4,2) < GL(4,3) is the otherprimitive linear group we know of which does not have a base of size at most 2.

LEMMA 2.10. Let G < Sp(4,3) be solvable and act completely reducibly on GF(3)4.Then \G\ ̂ 1152.

Proof. Any irreducible subspace of G must be totally isotropic or nonsingular,since the radical would be invariant. In particular, G has no irreducible subspace ofdimension 3.

Suppose first that G acts reducibly. If G has a one-dimensional invariant subspace,then |G|^|GL(1,3)|2|GL(2,3)| = 192. if G has two 2-dimensional irreduciblesubspaces and at least one of them is nonsingular, then \G\ ̂ |Sp(2,3)||GL(2,3)|= 1152. If G has two 2-dimensional totally isotropic irreducible subspaces U, V, andwx, u2 is a basis for U, then we can choose vx, v2 e V such that u0 v{ is a hyperbolic pairfor i = 1,2. Then the pointwise stabilizer of U must fix vvv% as well, so

Suppose that G acts irreducibly. If G is imprimitive with one-dimensional blocks,then \G\ ^ |GL(1,3)'VSI

4| = 384. If G is imprimitive with 2-dimensional nonsingularblocks, then |G| ^ |Sp(2,3) X S2\ = 1152. If G is imprimitive with 2-dimensionaltotally isotropic blocks U, V, then we can choose hyperbolic pairs w(, vt as in thereducible case. The pointwise stabilizer of U fixes V, so |G| ^ |GL(2,3)||52| = 96.

If G is primitive, then it embeds into a maximal primitive solvable group ofGL(4,3) for which the structural description of Lemma 2.2 can be applied. If e = 1,then |G| < |TL(1,34)| = 320. If e = 2, pa = 32, then

|G| ^ (^- l )e2 |Sp(2,2) | |Aut(GF(^)) | = 384.

If e = 4, B^D*Q, then \C/B\ < 24 and so |G| ^ 768. If e = 4, B^D*D,then G ^ (D*D)O+(4,2). Since D*D contains elements of determinant — 1,

Lemma 2.10 implies that in the case e = 9, Lemma 2.5 is applicable.

MINIMAL BASE SIZE 251

LEMMA 2.11. (a) Let G ̂ Sp(6,2) be solvable and act completely reducibly onGF(2)6. Then \G\ ^ 1296. Moreover, if\G\ > 432, then G is isomorphic to a subgroupof S3 X S3 or to a subgroup of (31+2 SL(2,3)) C2.

(b) Let G ̂ Sp(8,2) be solvable and act completely reducibly on GF(2)8. Then\G\ ^ 31104. Moreover, if\G\ > 5184, then G is isomorphic to a subgroup of S3 X 54 orto a subgroup o/(31+2SL(2,3)) C2 x S3.

Proof. We start with some general remarks pertinent to both cases. Anyirreducible subspace of G must be totally isotropic or nonsingular. We shall also usethat maximal irreducible solvable subgroups of GL(3,2) are isomorphic to FL(1,23),and maximal irreducible solvable subgroups of GL(4,2) are isomorphic either torL(l,24)ortoGL(2,2)%52.

(a) Suppose first that G ̂ Sp(6,2) acts reducibly. If the largest irreduciblesubspace is 4-dimensional, then G < Sp(4,2) x GL(2,2) and so \G\ < 72 • 6 = 432. If Ghas one 3-dimensional irreducible subspace, then G ̂ GL(3,2) xGL(2,2) and\G\ ^ 21 -6 = 126. If G has two 3-dimensional irreducible subspaces U, V, then U, Vare totally isotropic; moreover, if w15 w2, u3 is a basis for U, then we can choose vlt i>2, v3

such that Ui,vt is a hyperbolic pair for / = 1,2,3. Then the pointwise stabilizer of Ufixes V, so \G\ ^ 21. If the maximal irreducible subspaces for G are at most2-dimensional, then \G\ ^ |GL(2,2)|3 = 216.

Suppose that G acts irreducibly. If G is imprimitive, then the blocks havedimension 2 or 3 (over GF(2), one-dimensional blocks lead to a reducible group). Ifthe blocks are 2-dimensional, then G ̂ GL(2,2) "V S3, which is one of the exceptionalcases mentioned in the statement of the lemma. If there are two 3-dimensional blocksU, V, then U, Fare totally isotropic, since <rad (U), rad (F)> is an invariant subspace.As in the reducible case, it is easy to see that the pointwise stabilizer of U fixes V,implying \G\ < 21-2 = 42.

If G is primitive, then it embeds into a maximal primitive solvable group ofGL(6,2). For this maximal group, e = 1 or e = 3. If e = 1, then \G\ ^|FL(1,26)|= 378. The case e = 3 leads to the second exception in the statement of the lemma.

(b) Suppose first that G ^ Sp(8,2) acts reducibly. If the largest irreduciblesubspace is at most 4-dimensional, then |G| ^ 722 = 5184. It is impossible to have 5-or 7-dimensional irreducible subspaces. If there is a 6-dimensional irreduciblesubspace, then, by the analysis in part (a), either \G\ < 378 • |GL(2,2)| = 2268 or Gbelongs to the two exceptional cases mentioned in the statement of the lemma.

If G acts irreducibly but imprimitively, then the minimal blocks have dimension2 or 4. If the minimal blocks have dimension 2, then G < GL(2 ,2) \S ' 4 ^S ' 3 \S 4 .If there are two 4-dimensional minimal blocks U,V, then G ^ rL(l,24)X52.The subspaces U, V are both totally isotropic or nonsingular (since the radicalsgenerate an invariant subspace). If they are totally isotropic, then the pointwise stabil-izer of U fixes V, so |G|^|FL(1,24)||S2| = 120. If U,V are nonsingular, thenG<(FL(l,24)nSp(4,2))'VS2. Since Sp(4,2)^56 has no element of order 15,|FL(1,24) n Sp(4,2)| ^ 20 and |G| ^ 202-2 = 800.

Finally, if G is primitive, then it embeds into a maximal solvable primitivesubgroup of GL(8,2) with e = 1. Hence \G\ ^ |FL(1,28)| = 2040.

If e = 8 then, by Lemma 2.1 l(a), \C/B\ ^ 1296. Hence Lemma 2.5 is applicable forpa > 7. If pa = 5 and \C/B\ ^ 432, then we can again apply Lemma 2.5. If pa = 5 and\C/B\ > 432, then we use GAP to determine that S3\,S3 has 135 elements of order

252 AKOS SERESS

2 and 98 elements of order 3, and (31+2SL(2,3)) C2 has 117 elements of order 2 and98 elements of order 3. Hence we can apply Lemma 2.9. Finally, if pa = 3, then weconstruct H. There are two possibilities for the extraspecial 2-group B = E, namelyD*D*D and D*D*Q. Both of them have a unique irreducible 8-dimensionalrepresentation over GF(3). It turns out that even the entire normalizer TV of E inGL(8,3), isomorphic to (D * D * D) O+(6,2) and (D * D * Q) CT(6,2), respectively, hasa base of size 3. Moreover, even N has at least 5 non-equivalent bases of size 3.

For e = 16, the only subcase left ispa = 3. H\C/B\ < 5184, then we apply Lemma2.5; otherwise, we use that S3^VSi has 939 elements of order 2 and 944 elements oforder 3, (31+2SL(2,3)) C2 x S3 has 471 elements of order 2 and 296 elements of order3, and apply Lemma 2.9.

Finally, we consider the cases e = 1,2. If e = 2, then C ^ GL(2,pa) so, for a basis(i^i;,) of GL(2,pa), ffVinC=l. Hence I/Z^J ^ a and, since (a- \)pa <p*a-4afor a ^ 2, Hv has at least five regular orbits. Therefore if the minimal base size ofH is 3, then H has five non-equivalent bases. Similarly, if e = 1 then C ^ GL(\,pa)and \HVi\ < a for any vx e V\{0}. Since (a- \)pa/2 < pa for a ^ 2, H has a base of sizeat most 2. This finishes the proof of Theorem 2.1.

The analogue of Theorem 2.1 for odd order groups is much easier to prove.Espuelas [5] and Palfy (personal communication) independently proved the following.Let VL(\,pd) denote the group of semilinear transformations of GF(pd).

THEOREM 2.12. Let p be an odd prime, and let H ^ GL(d,p) be primitive linear ofodd order. Then H has at least two regular orbits unless H ^ rL(l,/?d).

It is easy to check that in the case H < FL(l,/)d), / /has at least two non-equivalentbases of size 2.

3. Imprimitive linear groups

The main goal of this section is to finish the proof of Theorem 1.1. However, westart with the following.

Proof of Theorem 1.2. Gluck [6] showed that if G ^ Sym(Q) is a primitivesolvable permutation group then, with finitely many exceptions, there is a subsetA c f i such that the setwise stabilizer of A in G is the identity. In other words, thereis a partition of Q into two parts such that only the identity element fixes the partition.

The exceptions in Gluck's theorem are: Frobenius groups acting on 3, 5 or 7points; S4, A4 acting on 4 points; AFL(1,23) acting on 8 points; and some large ordersubgroups of AGL(2,3) acting on 9 points. It is easy to check (for example, using thelibrary of primitive groups in the GAP system) that in all cases there is a partition ofthe permutation domain into at most four parts which is fixed only by the identityelement of the primitive group. In fact, with the exception of 54, three parts suffice.

A partition of a permutation domain Q can be considered as a colouring of Q. Foreach (isomorphism type of a) primitive solvable permutation group G, we fix acolouring of its permutation domain with the minimal possible number of colourssuch that the only colour-preserving permutation in G is the identity.

Clearly, it is enough to prove Theorem 1.2 in the case when G ^ Sym(Q) istransitive, since the required partition can be constructed orbitwise. So suppose thatG is transitive. We define a structure tree of G as a rooted tree T with levels

MINIMAL BASE SIZE 253

To, Tj , . . . , Tm such that the root is Q (that is, 7̂ = {Q}), the leaves are the elements ofQ. (that is, Tm = Q), and internal nodes of T correspond to certain blocks ofimprimitivity in Q. The blocks on a fixed level T{ define a partition of Q into a blocksystem. We also require that for each non-leaf node x, the children of x are a partitionof x, and the stabilizer of x acts primitively on the children of JC. Note that the actionof G can be extended naturally to the structure tree, and G acts transitively on eachlevel.

We define a colouring F: T-* GF(5). Let the colour of the root be 0. Recursively,suppose that the colouring of 7J,,7j,...,7J is already defined and let xeTt. Thestabilizer of x acts as a primitive group G(x) on the set Q(x) of children of x, and thereis a fixed colouring for the isomorphism type of G(x) with k(x) ^ 4 colours. Then wecolour £l(x) with colours F(x)+\,F(x) + 2,...,F(x) + k(x), as prescribed for theisomorphism type of G(x).

We claim that the only colour-preserving permutation geG of T is the identity.By induction on / = 0,1,2, . . . , m, the definition of F implies that a colour-preservingpermutation of To U 7i U ... U Tt must fix To U Tx U ... U Tt pointwise. In particular, gmust fix Q = Tm pointwise.

Also, we claim that the only colour-preserving permutation g e G of Q = Tm is theidentity. To see this, we first observe that from the F-colouring of Tm, it is possible toreconstruct the ^-colouring of the entire tree T. Then, by induction on/ = m, m— 1,... , 0, it is clear that g must preserve the colouring of Tt U Ti+1 U ... U Tm.In particular, g preserves the colouring of T, so g is the identity.

Thus the F-colouring of Q = Tm defines a partition P of Q into at most five partssuch that only the identity element of G fixes P.

THEOREM 3.1. Let H^GL(d,p) be an irreducible imprimitive solvable lineargroup, acting on V = GF(/?)d. Then H has a base of size at most 3.

Proof. Since H is imprimitive, there exists a non-refinable decomposition

into /-dimensional subspaces with d = A/, a primitive linear group L ^ GL(J^), anda transitive group T^ Sk such that H permutes the V{ and H ^ L\,T. It is enoughto prove that L % T has a base of size at most 3. We fix a subgroup To^ L*\ T whichis isomorphic to T. Then 7̂ provides an identification between the V(.

By Theorem 2.1, L < GL(FX) has a base B of size at most 3. Also, by Theorem 1.2,there exists a partition {1,2,.. . , k) = P1 U P2 U ... U P5 which is fixed only by the identityelement of To. We allow that some of the Pt, 3 ^ / < 5, are empty.

We distinguish three cases, according to the minimal base size v(L) of L. Ifv(L) = 1, then let vne Vx be from a regular orbit of L and let vae V{ be the ro-imageof vn in Vt. Define

» i = E vn> y 2 = E vn a n d vz= E vn-

We claim that (vlyv2,v3) is a base of L \ T. Suppose that ge(L\,T)v v v. Let iePx

and V\ = Vy Since g fixes vlt jeP1 U P2 U P3. Also, since g fixes y2, jePl U P4. Fromthese, we conclude that y e / i and, since / was an arbitrary element of JP15 we see that

254 AKOS SERESS

g fixes the set Px. Similarly, we can easily see that g fixes Pm, for 2 ^ m ̂ 5. Since onlythe identity element of To fixes the partition Px U P2 U ... U Pb, we obtain that g€Lk.From this, it follows easily that g = 1.

Next, suppose that v(L) = 2, let (i>n,y12) be a base of L, and let (vn,vi2) be the 7 -̂

image of (vxx, v12) in ^. Define

i\i 2 Z-J i2 3 L-J i2 ' ^ j <1*

We claim that ( f j , ^ ,^ ) is a base of L\,T. To show this, suppose thatge{L\,T)v v v. Let / e / ^ and V\ = ^. Since g fixes vx,j$Pb. Similarly, since g fixesv2 and y3,y^ P3 U P4. Finally,7 £ P2 since u§ = u2 would imply v9

i2 = vj2 and v\ = u3 wouldimply v\2 = yn, a contradiction. So g fixes the set Px and then it follows easily that gfixes Pm, for 2 ̂ w ̂ 5. Thus geLk, and so g = 1.

Finally, suppose that v(L) = 3, and let Bx,B2,...,Bh be five inequivalent bases ofsize 3 for L. For iePm, define (viX,vi2,vi3) as the 7^-image of Bm. Also, define

k k kVl = E y<l» y2 = E yi2 a i l d y3 = E y(3#

(=1 i=l (=1

We claim that (vx, v2, v3) is a base of L X J1. Suppose that ge(L\.T)v v v. Let / ̂ A;arbitrary, and let Vf = V}. Then (vix,vi2,vi3)

9 = (vjX,vj2,vj3) and so (viX,vi2,vi3) and(v}X, vn, vj3) cannot be the images of non-equivalent bases. This means that g fixes thepartition Px U P2 U ... U P5, geLk, and so g = 1.

We finish this section with the following.

Proof of Theorem 1.3. It is enough to show that if p is an odd prime,H < GL(d,p), \H\ is odd, and H acts irreducibly on V = GF(p)d, then H has a baseof size at most 2. If H is a primitive linear group, then this follows from Theorem 2.12and the remark following that theorem. If H is imprimitive then, using the notationintroduced in the previous proof, d = k£, V = 0f=1 V{ and H^ L I T for someprimitive linear L ^ G L ^ ) and transitive T ̂ Sk. As above, we fix 7̂ ̂ L X T whichis isomorphic to T. By a result of Gluck [6], there exists a partition{1,2,..., k} = Px U P2 which is fixed only by the identity element of T.

If L^TL(\,pl), then let vx,v2eVx be in different regular orbits of L. For1 ^ / ̂ k, let viX, vi2 e Vt denote the 7^-images of vx and v2, respectively. Then, using themethod from the proof of Theorem 3.1, it is easy to see that

L-t 11 ' L-i t2iePl ieP2

is in a regular orbit of L X T.If L < rL(l,/?'), then let 5 l 5 5 2 t>e t w o non-equivalent bases of size 2 for L. For

iePm, define (y(1,yi2) as the 7^-image of Bm. Also, define

»1 = E ytl a n d 2̂ = E Vi2-

Then, by the usual argument, (vvv2) is a base for L\,T.

ACKNOWLEDGEMENT. I am indebted to Thomas Breuer, Peter Palfy, LaszloPyber and Ron Solomon for stimulating conversations on this subject.

MINIMAL BASE SIZE 255

References

1. L. BABAI, G. COOPERMAN, L. FINKELSTEIN and A. SERESS, 'Nearly linear time algorithms forpermutation groups with a small base', Proc. ISSAC '91, Bonn (1991) 200-209.

2. R. BEALS, ' Computing blocks of imprimitivity for small-base groups in nearly linear time', Groups andcomputation, DIMACS Ser. Discrete Math. Theoret. Comp. Sci. 11 (ed. L. Finkelstein andW. Kantor, Amer. Math. Soc, Providence, RI, 1993) 17-26.

3. R. BEALS and A. SERESS, 'Structure forest and composition factors for small base groups in nearlylinear time', Proc. 24th ACM STOC (1992) 116-125.

4. K. BLAHA, 'Minimum bases for permutation groups: the greedy approximation', J. Algorithms 13(1992) 297-306.

5. A. ESPUELAS, 'Large character degrees of groups of odd order', Illinois J. Math. 35 (1991) 499-505.6. D. GLUCK, 'Trivial set-stabilizers in finite permutation groups', Canad. J. Math. 35 (1983) 59-67.7. D. GLUCK and O. MANZ, 'Prime factors of character degrees of solvable groups', Bull. London Math.

Soc. 19(1987)431-437.8. D. GORENSTEIN, Finite groups (Harper & Row, New York, 1968).9. M. W. LIEBECK, 'On minimal degrees and base sizes of primitive permutation groups', Arch. Math.

43(1984)11-15.10. E. LUKS and A. SERESS, 'Computing the Fitting subgroup and solvable radical of small-base

permutation groups in nearly linear time', Groups and computation II, DIMACS Ser. DiscreteMath. Theoret. Comp. Sci. (ed. L. Finkelstein and W. Kantor, Amer. Math. Soc, Providence,RI), to appear.

11. P. P. PALFY, 'A polynomial bound for the orders of primitive solvable groups', J. Algebra 77 (1982)127-137.

12. P. P. PALFY, 'Bounds for linear groups of odd order', Proc. Second Internat. Group Theory Conf.,Bressanone/Brixen 1989, Suppl. Rend. Circ. Mat. Palermo 23 (1990) 253-263.

13. C. E. PRAEGER and J. SAXL, 'Closures of finite primitive permutation groups', Bull. London Math.Soc. 24(1992)251-258.

14. L. PYBER, 'Asymptotic results for permutation groups', Groups and computation, DIMACS Ser.Discrete Math. Theoret. Comp. Sci. 11 (ed. L. Finkelstein and W. Kantor, Amer. Math. Soc,Providence, RI, 1993) 197-219.

15. M. SCHONERT et al. 'GAP: Groups, Algorithms, and Programming', Lehrstuhl D fur Mathematik,RWTH Aachen (1992).

16. M. SCHONERT and A. SERESS, 'Finding blocks of imprimitivity in small-base groups in nearly lineartime', Proc ISSAC '94, Oxford (1994) 154-157.

17. A. SERESS, 'Nearly linear time algorithms for permutation groups: an interplay between theory andpractice', Ada Appl. Math., to appear.

18. C. C. SIMS, ' Computation with permutation groups', Proc. Second Sympos. on Symbolic and AlgebraicManipulation (ed. S. R. Petrick, ACM, 1971).

19. D. A. SUPRUNENKO, Matrix groups (Amer. Math. Soc, Providence, RI, 1976).20. H. W. WIELANDT, 'Permutation groups through invariant relations and invariant functions', Lecture

Notes, The Ohio State University (1969).21. T. R. WOLF, 'Solvable and nilpotent subgroups of GL{n,qm)', Canad. J. Math. 34 (1982) 1097-1111.

Department of MathematicsThe Ohio State UniversityColumbus, OH 43210USA