I think you certainly should do the refactoring. Whether the factored modules should be packaged separately is a matter of what namespace they belong in.

For example, if your logging module is distinct and useful enough that it could take a place in the Log:: namespace, it probably should. If not, perhaps your private logging should be rewritten to use something already in Log::*. That would add simplicity.

I'd say the whole object of refactoring is to add simplicity. Doesn't mean it's not a complex task, of course.