The substitute officer, appointed pursuant to § 58.1-3169, shall, upon the ouster of his predecessor, unless sooner removed by the Governor or under the provisions of §§ 24.2-230 to 24.2-238, continue to serve in such capacity during the remainder of the term of his predecessor and until his successor be elected or appointed and qualified.