Re: Question
I agree with Marcel regarding the memory card. I do know of cases where the removal of a memory card while it is being accessed (read or write) can cause damage to the data that is on the card. This could result in the loss of all photos that are on the card. I don't believe it will physically damages any hardware. The card can be reformatted and reused.
Changing lenses with power on, however, can cause a different set of problems.
Your camera has sensitive electronic circuits. When you mate the lens to the camera, you complete the circuits to those in the lenses. The contacts never touch "Instantly", there is always contact bounce to some degree. (Bounce is where the contacts touch, they then bounces away then touch again). This bounce takes only milliseconds to happen but is does happen. If power is on, then this bounce creates surges in the current that can cause damage to the camera circuitry.