If Kayn moved at least once then got banished via Portal Rescue, when it gets replayed, does he retain the amount of times he moved this turn for the sake of his ability?

Ruling: No, Kayn does not retain the amount of times he moved. When he is banished and replayed, he becomes a new game object without any memory of previous events.

Nuances:

• Whenever a Game Object changes zones to or from a Non-Board Zone, all Temporary Modifications of all kinds cease to be tracked on it in all capacities.