Physically preparing for a fight is just as important as mental preparedness and, in some cases, much more important than gear and gun selection. Owning a firearm is not the end-all-be-all answer. Firearms can be a great equalizer, but if you are able to improve your physical condition, you owe it to yourself and your loved ones to do so.
Proper defensive firearms training should include a fair amount of artificial stress, such as time limits, movement, competition, etc. If you are shooting knotholes during slow fire, but crater during the stress shoots due to poor physical condition, what do you think will happen when faced with an armed attacker? Not all gunfights will start with you and the attacker face-to-face at 7 yards. It might start with you being jumped from behind, pulled from your car or being chased through a parking lot.
Also, you try to avoid getting into a gunfight in the first place, since there is no guarantee that you will win (or that the bad guy won't injure a friendly in the process). In many situations, the better option may be to turn and sprint to your car or back into your house, instead of being caught in the open, out of breath and with a pounding heart. I have seen guys in wheelchairs and senior citizens with oxygen tanks on the range, training with firearms, and that's awesome and inspirational, but the rest of us need to do what we can to prepare physically for the fight.
I strongly recommend performing some form of combative training. This will not only provide the needed physical exercise, but will also contribute to the mindset we need to have to keep ourselves alive. You may not be up for full speed MMA classes, but at least try judo, ju-jitsu or even boxing or tae kwon do. Anything that involves getting the heart rate up while facing an opponent is a good choice for this vital activity.