Ok let's start by saying in the code example you made a small mistake.
- set the Primary and Secondary Cameras to the Cardboard left and right cameras:
VuforiaBehaviour.Instance.PrimaryCamera = cardboardCameraLeft;
VuforiaBehaviour.Instance.PrimaryCamera = cardboardCameraRight;
^uh, we don't want to set the primarycamera twice. So change it to SecondaryCamera
Now that that is out of the way, when I use your example, I can only switch to mono once and stereo once. After that it doesn't work, nothing happens. So what's the problem?