Is the sample rate changed? Why do you change the clock settings after opening a session? What ecact change are you making in the clock settings?
Some interfaces need an engine reset when the clock source changed, this also depends on the driver and the host. Changing the buffers works like a reset, that is why you can continue after that.