Globular cluster system erosion in elliptical galaxies