When I activate Rek'Sai's Void Burrower when I conquer, do I have to pay the cost to play one of the cards I revealed?

Yes, you must pay the full cost of the card if you choose to play it using the ability of Rek'Sai, Void Burrower.

According to RiftJudge FAQ #10100 (and supported by FAQs #9823, #9215, and #284), you are required to pay the card's full Energy and Power costs as normal when playing it with this ability. Because the ability does not contain the phrase "ignoring its cost," no cost reduction is applied.