For the most part, you should not feel like you are going to fall out of the side of the seat. Body types differ, obviously, but there should be enough side bolstering to keep you in the seat. Sometimes driver posture can effect how stable you are in the seat as well. I often find my butt sliding forward due to the flat bottom seat I use. Ideally you want to put some forward pressure on the steering wheel to push your torso back into the seat. This serves two purposes. One, it keeps your upper body weight working with the chassis to hike the inside rear wheel and Two, keeps the rear end loaded under braking (single rear brake). When using good posture, you will find yourself steering more with your Lateral Back Muscles than your Biceps.
Another point to make about keeping you planted in the seat is using the Heel Stops. Those pegs your heels pivot on when working the pedals. They should be sort of a brace for your legs to help push you back into the seat and hold your butt/back into the seat. The more you are pushed back into the seat, the less upper body strength will be required to keep you planted. That will also help with control and handling of the kart.
See this article: Posture, and head movement