Trihedralguy,
Grandfather is wise. So is Your Mother. They are trying to teach, not disrespect you. I had to actually look at your web site to understand the problem. After some quick troubleshooting:

The problem does not appear to be server side or network latency. I get pings well under 100ms to your domain. Using Firefox with Live HTTP Headers plugin, you can see an almost instantaneous request-response cycle followed by a few seconds before the page update renders.

Looking at the HTML source, you are rendering around 300 <div> tags, each having an anchor tag with empty href (generated on line 70 of your Perl code). First, I would remove the unused anchors. You may also want to try building the image with a table instead of divs and see if that renders any faster. Finally the javascript library you are using may also be a factor. Check if they have a mailing list or forum to help you with it.