How much is too much for a database is sensitive to a number of factors, including the database engine, the O/S, and the hardware it's running on, so there's not an answer to "is it bad to have one database store that much info?" without considerably more information. I would not consider a table with 5,000,000 entries to be unacceptably large, but the bulk of my DB experience is as a user, querying a DB2 database via an API (the database stored, among other things, the drawing and manufacturing document data for a very complex manufactured artifact).

If you're seriously worried, get a DBA to help you layout the DB system.