Jack B. Nymbol,
Here is a greedy approach that I think would produce good results (though I haven't even attempted to test it) and should be relatively fast.

1. Divide the list into two ordered sublists (above and below the target average)
2. Take the largest item off the above list
3. Take as many items off the below list (smallest first) until you are within the target average
4. Take the smallest item off the below list
5. Take as many items off the above list (smallest first) until you are within the target average
6. If either steps 2/3 or 4/5 fail to balance, drop the item from step 2/4 and repeat the process
7. Repeat until one of the two lists is empty
8. Of the list that remains, see if any of the remaining items closest to the target can be added to the pile and maintain the target average