The Prym map on divisors, and the slope of $\A_5$ ( appendix by K. Hulek)

The birational geometry of the moduli spaces of curves $\calM_g$ and of principally polarized abelian varieties (ppav) $\calA_g$ has been studied extensively, with results for large enough $g$ starting with Harris and Mumford and Eisenbud and Harris' \cite{hamu,harris,eiha} proof that $\calM_g$ is of general type for $g>23$ (followed by Farkas' proofs that the Kodaira dimension of $\calM_{23}$ is at least 2 \cite{fathesis} and that $\calM_{22}$ is of general type \cite{fa22}), and Tai's \cite{tai} and Mumford's \cite{mumforddimag} proofs that $\calA_g$ is of general type for $g\ge 9$ and $g\ge 7$ respectively. A more precise question is to describe the effective cone of $\calM_g$ ($\calM_g$ is of general type if the canonical class is in its interior). The slope of the effective cone o $\calM_g$ for small genus was described by Harris and Morrison \cite{hamo}, who, led by their results, conjectured that the Brill-Noether divisor has minimal slope. The minimal slope of $\calA_g$ (and its relation to $\calM_g$) for $g\le 4$ was described by the second author in \cite{sm}. For the first non-classical cases, it turns out that the minimal slope of $\calA_4$ is given by the Schottky form --- the divisor of the locus of Jacobians --- of slope 8 (see also \cite{hahu}), while recently the effective slope of $\calM_g$ has attracted a lot of attention, after G.~Farkas and Popa \cite{fapo} disproved the slope conjecture of Harris and Morrison \cite{hamo}. Despite the further work \cite{farkassyzygies,khosla}, the slope of the effective cone of $\oMg$ is not known for $g\ge 11$. It is known by the work of Tai \cite{tai} (see also \cite{grAgsurvey}) that the slope of the effective cone of $\overline{\calA_g}$ approaches zero as $g\to\infty$. However, no examples of effective divisors of slope less than 6, on either $\calM_g$ or $\calA_g$, are known for any $g$, while it is not known if 6 is a lower bound for the slope of effective divisors on $\calM_g$. (We also note that the nef cones have been studied for $\oMg$ --- see \cite{gikemo,fagi,gibney} and for $\overline{\calA_g}$ --- see \cite{hulek,husaA4,shepherdbarron}). In the last few years G.~Farkas and Verra, and also Ludwig, have studied the geometry of various covers and fibrations over the moduli spaces of curves \cite{farkasevenspin,farkasverraoddspin,farkasverrainterm}. In this paper we concentrate on the moduli space $\calR_g$ of Prym curves, the subject of a recent survey \cite{faprymsurvey}: this is the moduli space of pairs consisting of a smooth genus $g$ algebraic curve $X$ together with a line bundle $\eta$ on $X$ such that $\eta^2=\calO_X$ while $\eta\ne\calO_X$. Such a data defines an \'etale double cover of $X$, and the associated Prym variety is an element of $\calA_{g-1}$ --- thus we have a morphism $p:\calR_g \to\calA_{g-1}$. One then wants to extend this morphism to a suitable compactification. Indeed, if one takes a partial compactification $\calR_g^{part}$ of $\calR_g$ obtained by adding stable curves with one node, and partial compactification $\calA_{g-1}^{part}$ of $\calA_{g-1}$ obtained by adding semiabelic varieties of torus rank 1 (i.e.~for which the normalization is a $\PP^1$ bundle over an abelian variety, see \cite{mumforddimag}) the Prym map can naturally be extended to a morphism $\calR_g^{part}\to\calA_{g-1}^{part}$. Taking an actual compactification is trickier, as the structure depends on which toroidal compactification of $\calA_g$ is taken. Alexeev, Birkenhake, and Hulek \cite{albihu} studied the extension to a map to the second Voronoi toroidal compactification. Note, however, that no matter what toroidal compactification $\overline{\calA_{g-1}}$ of $\calA_{g-1}$, the morphism on the partial compactification (without defining it on $\overline{\calR_g}\setminus\calR_g^{part}$ gives a rational map $p:\overline{\calR_g}\dashrightarrow \overline{\calA_{g-1}}$. In general the Picard group of an arbitrary toroidal compactification $\overline{\calA_g}$ may be very large, and is not known (see \cite{husaA4} for the discussion in genus 4). However, for the perfect cone compactification the Picard group (over $\QQ$) has rank two, and is generated by the class $L$ of the Hodge bundle, and the class $D$ of the (irreducible in this case) boundary divisor. We note also that the perfect cone compactification is $\QQ$-Cartier, as modular forms of weight $ k $ are a line bundle with divisor class $kL$, and cusp forms of weight $k$ are a line bundle of class $kL -D$, and thus any class $aL-bD$ is their linear combination. {\em From now on we denote by $\overline{\calA_g}$ the perfect cone toroidal compactification of the moduli space of principally polarized abelian varieties, and consider the rational map $p:\overline{\calR_g}\dashrightarrow\overline{\calA_{g-1}}$.} In this paper we use the Schottky-Jung relations to compute the pullback under $p$ of the theta-null divisor and thus compute the pullback map $p^*$ on divisors, and use this to bound the slope of the effective cone of $\overline{\calA_5}$. Our main result is theorem \ref{thm:pull} %and \ref{thm:push} computing the map $p^*$ %and $p_*$ on divisors, %respectively, and the following bound for the slope: \begin{thm}\label{thm:slope} The minimal slope $s$ of effective divisors on $\overline{\calA_5}$ satisfies $$ 7+\frac{5}{7}\ge s\ge 7+\frac{4198}{6269}\qquad ({\rm i.e.\ } 7.7142\ldots\ge s\ge 7.6696\ldots ). $$ \end{thm} Recall that the slope of a divisor $aL-bD$ is defined to be $a/b$. The new statement in the theorem is the lower bound for the slope. The upper bound is provided by the Andreotti-Mayer divisor $N_0'$ (see remark \ref{rem:AM}). Our result is thus the lower bound for the slope of effective divisors (or of Siegel modular forms) in genus 5, where no such bound was known a priori.