Collective atom-cavity coupling and nonlinear dynamics with atoms with multilevel ground states

We investigate experimentally and theoretically the collective coupling between atoms with multilevel ground-state manifolds and an optical cavity mode. In our setup the cavity field optically pumps populations among the ground states. The ensuing dynamics can be conveniently described by means of an effective dynamical atom-cavity coupling strength that depends on the occupation of the individual states and their coupling strengths with the cavity mode. This leads to a dynamical backaction of the atomic populations on the atom-cavity coupling strength which results in a nonexponential relaxation dynamics. We experimentally observe this effect with laser-cooled 87Rb atoms, for which we monitor the collective normal-mode splitting in real time. Our results show that the multilevel structure of electronic ground states can significantly alter the relaxation behavior in atom-cavity settings as compared to ensembles of two-level atoms.