(b) The commissioner shall be removable for cause by the appellate division of the supreme court in the judicial department embracing the county, upon written charges and opportunity to be heard after due notice thereof. If such commissioner is the county clerk, the appellate division may remove him from the office of commissioner in the same manner.