Make physical heli controls rotation based not position based.
Currently to pitch and roll you have to physically move your hand and to yaw you have to rotate your hand. My concern comes from the inherent rotation of the controller that comes from moving your hand across the plane, as it is tough to keep your arm in line without moving your head or other arm. Games such as VRChat and VTOL use rotation based input, with yaw rotation around the "up" axis of the controller, allowing significantly more precision and overall more intuitive controls for VR.

If for whatever reason this does not work, my only other thought would be a complete break from VR norm and have yaw on the left hand (elevation), as moving the arm up and down without rotation is much easier than left or right.