How do I build my confidence while driving?

I'm 28 and am just learning to drive. I have zero confidence and every time I make a mistake and my husband points in out it just gets worse!!! I know all the rules of the road by heart but as soon as I get behind the wheel I draw a blank. I'm extremely discouraged, frustrated and tired of trying and just wanna quit. Unfortunately, my husband won't let me and all I wanna do is punch him in the face (not literally, of course) I just keep telling myself how much I suck at driving. I really am trying but I blow chunks and don't want to do it anymore. How do I build more confidence if I hate driving?