How hard do I have to pull on the catheter to get it to come out?

If the balloon is deflated, it should just about fall out when you stand up. The weight of the empty bag will pull the catheter out.