Assuming everything you have said here is true, the agreement you came to with your boss supercedes any previous agreement you had with him regarding IP, remuneration etc. So it doesn't particularly matter what your written contract of employment says.
The difficulty is in proving that your agreement actually existed. Was it in writing? If so, that's a start. If as sounds likely, it was a verbal agreement, it's worthless unless you have witnesses or some other persuasive evidence that what you say happened, really happened.
However, even if you can prove that your agreement really existed, you don't seem to have much of a case.
Consider, there are basically two IP components involved here.
The first part, the original code, can clearly be used by your boss. I don't think you can withdraw that right from him if you GPLd the code.
- The stuff you developed in your own time and released under the GPL
- The stuff you developed in your bosses time "for hire" in the understanding that it would be relased under the GPL.
Similarly, he has the right to modify that code, and keep those modifications secret and proprietary PROVIDED he doesn't distribute the changed code. So if he only plans to use it directly, there's no way to force him to distribute it under the GPL.
Now, turning to the "in bosses time" modifications that you carried out. I think since the work was done "for hire" he does own the IP - this would be true even if it was to be GPL distributed. The IP owner always gets to choose how (or if) the property is to be licensed and distributed.
Now, if you can prove that you had an agreement that the code was to be distributed under the GPL, you might have a case against him for breach of contract. But winning such a case wouldn't force him to release the code, it would just force him to compensate you for your financial loss under the contract. But, uhm, since you were being paid by him to write the code, I don't see what financial loss you could claim to have suffered.
Sorry, but the civil legal system sees things in terms of dollars and cents. That doesn't fit too well with the world of free software.
IAAPBL (I Am A Part-trained British Lawyer) and you should certainly not rely on my analysis as regards US contract law.