You'll experience some real limitations to HTTP trying to do this. There is a multipart/x-mixed-replace mime-type that can be used (I think just with Mozilla based browsers) for things like constantly pushing image data to a browser for something like a webcam. However the support for this isn't consistent, so you'll have headaches trying to implement it. This is why so often a java applet/application or something "thicker" like a real IM application is used for chat rather than just HTTP/HTML.

I wrote a stand-alone Perl-CGI "web-chat" utility that uses periodic reloading to get the job done. It ain't pretty, but it might give you some ideas. It works pretty well, and I still use it occasionally where I work. It sounds much like what you have done.