If you get Switcheroo and have Guardian Angel, when you recall do you go back to normal might or stay at the swapped amount but -1 with the loss of GA?

You stay at the swapped amount, minus the 1 Might from losing Guardian Angel.

Here's why. Switcheroo calculates the difference between both units' current Might (including all gear bonuses) and applies a fixed +/- modifier for the turn (FAQ #9335). When Guardian Angel triggers, it's killed as part of its replacement effect, so its +1 Might bonus drops off. But the swap modifier from Switcheroo is a layer alteration that persists through the recall — Recalls do not reset Might or remove applied layer effects (Rule 453, FAQ #8433).

Concrete example from FAQ #9335: If your unit had 5 Might (base 4 + GA's +1) and Switcheroo swapped it with a 2-Might unit (applying a -3 modifier), your unit sits at 2 after the swap. GA then dies → your unit becomes 1 (2 − 1). After the recall to base, it stays at 1 until the swap effect expires at end of turn.