Does the units Azir moves with him still go to his attacking battlefield if he gets pushed back to base by Overzealous Fan?

No, the token units will not move to the battlefield.

According to RiftJudge FAQ #2127, Azir, Sovereign's ability requires him to be at a valid battlefield for the effect to resolve. If Azir is moved to his base by an effect like Overzealous Fan before his attack trigger resolves, the ability "whiffs" because "this battlefield" no longer refers to a valid location.

Sequence of Events:

1. Attack Trigger: When Azir attacks, his "When I attack" ability is placed on the chain. You choose which token units to move at this time.
2. Reaction: The defending player uses Overzealous Fan's ability (a Defend Trigger) to move Azir to his base.
3. Resolution: When the chain resolves, the game checks for "this battlefield" to move the tokens to. Because Azir is now at his base, the battlefield he was attacking is no longer his location, and the ability fails to resolve (whiffs).

As noted in RiftJudge FAQ #253, this interaction applies specifically to effects that move Azir away from the battlefield before his trigger resolves.