Any permit issued pursuant to this part may be revoked at any time within the discretion of the Secretary. Agricultural and grazing leases dealt with in Sec. 173.17 shall be subject to cancellation as provided for in the respective parts 162 and 166 of this chapter, and the conditions of the instruments executed pursuant thereto.