I talk to a lot of beginner cyclists who are afraid to ride on the road. Some try to conquer their fears by riding anyway, only to find that they end up being more afraid, not less. The proximity of the passing cars frightens them so much, that their bike handling suffers. This in turn makes them more likely to experience close calls and drivers honking at them, terrifying them even more. It becomes a vicious cycle.
Fear is a nasty, crippling emotion. I don't mean the philosophical kind, where you are thinking "Gee I don't think I can handle this." What I mean is the visceral kind: that sinking feeling in the pit of your stomach, adrenaline, heart racing, trembling, weak at the knees. It is debilitating and difficult to control. But it can be avoided.
If an aspect of cycling scares you at a strong visceral level, my suggestion would be to take it down a notch - to a point where you do not experience the fear - and work from there. If cycling on busy roads gives you panic attacks, try riding on quiet side streets exclusively, until that feels so comfortable that you are ready for the next step. If even that is too much, stick to bike paths, parks and empty lots until you are ready for side streets. If your neighbourhood has none of these things, try riding in the middle of the night or very early morning (with good lights of course). Even in a busy city, the roads will be nearly empty. Whatever it takes, find a way to ride so that you are relaxed and not in panic mode. I would apply this to every aspect of cycling - from riding for transportation, to learning new skills, to trying drop bars, clipless pedals, and riding off road.
It's also important to understand that when we ride with friends and spouses, their level of comfort may be very different from ours. Without meaning any harm, they might coax or pressure us to do things we cannot handle. When we are ready for it, being encouraged to push ourselves can be a good thing. I've certainly received more than a couple of nudges that were helpful. But overcoming nervousness or timidity is one thing. Intense, limbs-atremble fear is not good and can affect our behaviour in unpredictable ways. It is our responsibility to know the difference.
Conquering one's fears is a worthwhile endeavor. But it takes time and there is more than one way to approach it. In my experience, those who find cycling more frightening than enjoyable, stop cycling. Therefore, I suggest sticking to riding in a way that feels fun and avoiding riding in a way that feels terrifying. Cycling should be a positive experience.