There shall be no voting except within the time specified by the referendum agent as stated in the notice of the referendum.