The bottom line is that the data and indexes reside on a disk somewhere. Your process can either:

• Load them itself, follow the pointers and extract the required data.
• Or:
  1. Compete for attention from a DB process.
  2. Wait for connection.
  3. Negotiate authorisation.
  4. Wait while the SQL parser parses your query.
  5. Wait while the DBM checks your query against the schema.
  6. Wait while the DBM finds the relevant data and indexes.
  7. Wait while the DBM optimises your query.
  8. Wait while it checks/frees cache space for your indexes.
  9. Wait while it actually runs the query and builds a results table.
  10. Wait while the communications negotiate the transfer of the results.
  11. Wait while your process allocates space to hold the results.
  12. Wait while it structures the results into a perlish data structure.
  13. Finally, do something with the results if you haven't run out of memory in the meantime.

There are many good reasons for using an RDBMS, not least of which is that someone else does a lot of the work and maintenance, but performance is not one of them!