What happens when an independent contractor is really a de facto employee?

I am independent contractor with a company for long time. They can't seem to find and evidence of our (old) contract. I've no question that, contract or not, I'm not operating as a contractor. My question is if the CA EDD were to get involved with this, what would it mean to me and my "employer". I personally take no real deductions as an IC, and pay both sides of my social security, so even if I had to refigure back taxes, I'd not really lose much. Or would I? Would I get back all my past social security taxes? What would it mean to the employer?