work around starvation issue where multiple indexing threads flush segments faster than the single thread can publish them