In my experience, try to minimize plugging and unplugging anything but rugged stuff like guitars and kbds.
Many reasons for this.
If it was me, I would always shut everything down before unplugging. And then, always plug everything back in while shut down, then turn on. Don't have Fireface and have never used one, but sometimes these I/Os etc don't like being unplugged/repluggedwhen on.
Of course, you're using a laptop and it probably moves around. Embarrassing, but I've never used a laptop for audio stuff!
You WILL figure it out. Have faith!
R