Upon the institution of the proceeding authorized in § 58.1-3168 the Governor may suspend such officer from collecting the revenues of the Commonwealth and of the county or city and from performing any of the other duties of his office. The Governor may also appoint a person to act in the place of such suspended officer in the performance of the duties of the office. Such appointee, after having qualified and given bond according to law, shall discharge all the duties of the office to which he is appointed during the time of the suspension of his predecessor, shall be entitled to the compensation provided for such officer and shall be amenable to all the rules, regulations, requirements and responsibilities declared by the laws of this Commonwealth pertaining to the collection and disposition of, and accounting for, the public revenue. The suspension of the officer shall continue, unless sooner removed by the Governor, until the ouster proceedings so instituted have been finally determined.