Can I play Rengar Trophy Hunter as a reaction to an open Battlefield?

No. Rengar, Trophy Hunter cannot be played to an open battlefield.

An open battlefield is unoccupied and uncontrolled — it has no enemy units (Rule 169.10.c). Rengar's card text lets him Ambush to "a battlefield where there are enemy units," and an open battlefield has none. The standard Ambush keyword also requires "a battlefield where you control Units" (Rule 822.1.b), which you don't at an open battlefield. Neither condition is met.

Even outside of Ambush, units can only be played to your Base or a Battlefield you Control (Rule 813.3.a). An open battlefield is uncontrolled, so standard play rules don't permit it either.