does Azir Sovereign allow moving exhausted "Sand Soldiers" on my base along with the Sovereign?

Yes. Azir, Sovereign's ability moves "any number of your token units," with no restriction on their exhausted state or their source location.

Exhaustion only matters as a cost for the Standard Move and for attacking/defending — it does not prevent a unit from being moved by an ability or spell (Rules 420.1–420.2, 144.2). Sand Soldiers in your base are your token units, and FAQ rulings confirm the ability pulls tokens from anywhere (base or other battlefields) to the battlefield Azir attacked. They will still arrive exhausted, since nothing readies them.