there are several kinds of tree: red/black, ordered, fibonacci, heap

i think if you use an external library (gdbm or what other) this is not in you power, and this does not need your attention. maybe, if you notice performance falling after use, you can consider to use a different kind of implementation.

defragmentation, on the other hand, is something related to take fragments of a "file" around a storage and try to rejoin together in the natural order: you can see it's not related to btree but with fs and it's, i think, off the topic of your question.

(i hope my english is at least understandable)