Studies on the upgrade of the muon system in the forward region of the CMS experiment at LHC with GEMs