The normal core
For a group G, the normal core of a subgroup H is the largest normal subgroup of G that is contained in H (or equivalently, the intersection of the conjugates of H). More generally, the core of H with respect to a subset S⊆G is the intersection of the conjugates of H under S, i.e.
Under this more general definition, the normal core is the core with respect to S=G. The normal core of any normal subgroup is the subgroup itself.
Normal cores are important in the context of group actions on sets, where the normal core of the isotropy subgroup of any point acts as the identity on its entire orbit. Thus, in case the action is transitive, the normal core of any isotropy subgroup is precisely the kernel of the action.
A core-free subgroup is a subgroup whose normal core is the trivial subgroup. Equivalently, it is a subgroup that occurs as the isotropy subgroup of a transitive, faithful group action.
For a prime p, the p-core of a finite group is defined to be its largest normal p-subgroup. It is the normal core of every Sylow p-subgroup of the group. The p-core of G is often denoted , and in particular appears in one of the definitions of the Fitting subgroup of a finite group. Similarly, the p′-core is the largest normal subgroup of G whose order is coprime to p and is denoted . In the area of finite insoluble groups, including the classification of finite simple groups, the 2′-core is often called simply the core and denoted . This causes only a small amount of confusion, because one can usually distinguish between the core of a group and the core of a subgroup within a group. The p′,p-core, denoted is defined by . For a finite group, the p′,p-core is the unique largest normal p-nilpotent subgroup.
The p-core can also be defined as the unique largest subnormal p-subgroup; the p′-core as the unique largest subnormal p′-subgroup; and the p′,p-core as the unique largest subnormal p-nilpotent subgroup.
The p′ and p′,p-core begin the upper p-series. For sets π1, π2, ..., πn+1 of primes, one defines subgroups Oπ1, π2, ..., πn+1(G) by:
The upper p-series is formed by taking π2i−1 = p′ and π2i = p; there is also a lower p-series. A finite group is said to be p-nilpotent if and only if it is equal to its own p′,p-core. A finite group is said to be p-soluble if and only if it is equal to some term of its upper p-series; its p-length is the length of its upper p-series. A finite group G is said to be p-constrained for a prime p if .
Every nilpotent group is p-nilpotent, and every p-nilpotent group is p-soluble. Every soluble group is p-soluble, and every p-soluble group is p-constrained. A group is p-nilpotent if and only if it has a normal p-complement, which is just its p′-core.
Just as normal cores are important for group actions on sets, p-cores and p′-cores are important in modular representation theory, which studies the actions of groups on vector spaces. The p-core of a finite group is the intersection of the kernels of the irreducible representations over any field of characteristic p. For a finite group, the p′-core is the intersection of the kernels of the ordinary (complex) irreducible representations that lie in the principal p-block. For a finite group, the p′,p-core is the intersection of the kernels of the irreducible representations in the principal p-block over any field of characteristic p. Also, for a finite group, the p′,p-core is the intersection of the centralizers of the abelian chief factors whose order is divisible by p (all of which are irreducible representations over a field of size p lying in the principal block). For a finite, p-constrained group, an irreducible module over a field of characteristic p lies in the principal block if and only if the p′-core of the group is contained in the kernel of the representation.
A related subgroup in concept and notation is the solvable radical. The solvable radical is defined to be the largest solvable normal subgroup, and is denoted . There is some variance in the literature in defining the p′-core of G. A few authors in only a few papers (for instance Thompson's N-group papers, but not his later work) define the p′-core of an insoluble group G as the p′-core of its solvable radical in order to better mimic properties of the 2′-core.