Can you Rebuke a Rengar reaction coming into the battlefield trying to conquer?

No — not as a reaction to Rengar being played. Rebuke is [Action], not [Reaction], so it can't be added to a chain or played during the Closed State while Rengar's play is resolving (Rules 338.1.a, 813; FAQ #3425).

You do get a window after he resolves: once the chain empties, the showdown reopens and Focus passes to you, and Action spells are playable in showdowns on any turn (Rule 347.1.b; FAQ #8659). So you can play Rebuke then to bounce Rengar back to hand before the showdown closes.

But that alone doesn't stop the Conquer — Rengar's player had to have a unit at that battlefield to be attacking, and if that unit is still there when the showdown closes, they establish control and Conquer anyway (Rule 348.2.a). Rebuke only prevents the Conquer if it leaves them with no units at the battlefield.