I was freaked about College Algebra at first, 'cause I'd been out of high school five years before I went back to college, and those last two years were spent not having a frickin' clue what I was doing in Pre-Cal and Calculus. Thankfully, I had a VERY cool teacher who explained things well (and didn't get hung up on us memorizing formulas), and I was actually comfortable enough with math when I left his class (with an A), I took (and got an A in) Applied Physics last semester.

Good luck!