My goal is to process ~130000 bibliographic records (with tree-like structure) with interactive editing of perl code which is applied to each record with interactive response (under three seconds for edit/run/see result cycle).

Right now, execution of code on each database record isn't a bottleneck (code is simple, and even if it isn't I would just boot a few more machines and distribute smaller shards of data to each of them).

However, when nodes start to produce more than ~20000 results merge on master becomes a problem. This is where merge to central Redis from nodes comes into play, removing need for merge step all-together at expense of network traffic. My fingers are itching to try that out, but this will have to wait at least tomorrow morning...