You can use ssh-agent and ssh-add to allow you to manually enter the passphrase once, then have the keys available to all shells/programs you run under the agent, including your ssh-to-multiple-servers app. (Most Linux distros run ssh-agent by default when you log in under X, at least; if it's not active, ssh-agent bash will open a new shell with an active agent.)
So the worst-case process would be:
Log in
Run ssh-agent bash
Run ssh-add, which prompts for your passphrase
Enter passphrase
Run the multiple-ssh program
Step 2 may not be necessary if you're running under an ssh-agent by default. If you completely trust the system you're running on, steps 3 and 4 can be skipped by using an empty passphrase on the private key, but I wouldn't recommend doing so unless you need it to be able to run unattended (e.g., from cron), since that does go back to putting the complete login credentials into a file.