Tai chi can be best described as "mediation in motion." It is more slow-moving than tae kwon do and focuses more on internal energy development whereas tae kwon do is more external. However, that does not mean that tai chi itself will not provide a workout for you. It may not be as strenuous or physically demanding as tae kwon do, but you will see some physical benefits out of doing it. In general, the movements will work your core, back, legs, and arms. Though it looks slow from an outside perspective, some of what you may be doing might not come so easy.
Tae kwon do on the other hand is more "hard" and external with its phsyical movements. You will be doing various punches, kicks, techniques, and forms as you go through the curriculum. Sparring, I believe, is also another component of it. Doing that for a while will have an effect on strengthening and conditioning your body for learning and adapting to the material.
It really depends on what you want in terms of "fitness and exercise" so it might be better to clarify that to get a better answer. I started doing both kung fu and tai chi at the same time at my school, but many people choose one or the other. Some people who do only tai chi do it just to move around (who tend to be older in age as well). I have seen people who choose that route improve their posture, joint movements, and some flexibility. People at my school who jump right into kung fu will also develop more physical stamina along with overall body strength.
Considering the difference between those two styles, what I would do is try to visit each class and observe it or try it out if you can. Stay with each one for some time to get a better feel of what you will be doing and how your body will respond. However, I don't think you should be averse to choosing one over the other (maybe not now, but perhaps later) since neither one should impair what you do in the other art. You may find them to be complementarily beneficial (over time once you develop a deeper understanding of your physical movements and what each art entails).