can ruin rex choose a deflect target then refuse to pay deflect to fizzle ability?

Yes. Ruined Rex's Deathknell must choose a valid enemy unit if one exists (targeting is mandatory, Rule 402.4.b), but once it targets a unit with Deflect, you may decline to pay the Deflect cost. The ability then fails to finalize and is removed from the Chain — no damage is dealt.

This isn't "fizzling" — Riftbound doesn't use that term. The trigger simply never becomes a Finalized Chain Item, and this is not treated as countering it (Rule 404.2, FAQ #11469). Note this only works for triggered abilities like Deathknell; Deflect on a spell you play from hand is a mandatory cost you can't refuse.