I'd split up the different phases ( like

• reading/filtering directory,
• reading data from file,
• operating on data structure,
• ...

) into different subs and unit test them against typical cases.

And after testing all small building steps you test some big one combining easier ones and so on.

Of course you have to mock data, but with this approach you only need to simulate the simpler building cases that convince you.

The trick is to add new tests as soon that you discover a bug caused by an edge case you forgot to cover.

With this incremental approach the probability of bugs shrinks very fast in linear time.

UPDATE

I think a good test suite combines the qualities of a spider web and a pyramid.