Can I choose to assign non-lethal damage? If I have a 4 might unit and I'm fighting two enemy 3 might units, could I choose to assign 2 damage to each or am I forced to kill one before assigning damage to another?

No, you cannot choose to assign non-lethal damage in that manner. You are forced to assign lethal damage to one unit before you can assign any remaining damage to another.

According to Rule 443.1.d.3:

"Units must have lethal damage assigned to them in full before damage is assigned to a different Unit. Example: If a player has 5 damage to distribute among four 3 Might units, they may not choose to assign 2 damage to one of the units and 1 damage to each of the remaining 3. They must assign at least 3 damage to one, and the remaining 2 to another."

Additionally, the FAQ in your retrieved context confirms this:

"Ruling: You can choose any unit to assign damage to first, but once you choose a unit, you must assign lethal damage to it before you can assign damage to another unit. You cannot overkill units (assign excess damage beyond lethal) during combat resolution."

Your Scenario

If you have a unit with 4 Might and are attacking two enemy units that each have 3 Might:

  1. Assigning Damage: You must assign 3 damage to the first chosen enemy unit (this constitutes lethal damage).
  2. Remaining Damage: You have 1 Might remaining. You may then assign that 1 remaining damage to the second enemy unit.
  3. Result: The first unit receives 3 damage (lethal) and is killed. The second unit receives 1 damage (non-lethal).

You cannot choose to split the 4 damage as 2 and 2, because you would not have assigned lethal damage (3) to the first unit before moving to the second.