The bottleneck here is not fetching the data from the DB, but sending the resulting HTML to the browser and rendering it there. Limiting the number of rows you fetch will reduce the size of the HTML and make it a lot faster. Accessing the DB will also be faster, but if that takes 0.2 secs instead of 0.3 secs is not relevant anyway.

So if you can live with the page displaying only a part of the data (at least at once, you can have many pages) go for it.