Multi data path config can cause a shard to be perceived as corrupted

Multi data path config support writes a file to a data location based on the available size (by default). There is a Lucene file called segments.gen that has the same name, and only in that case, we need to make sure we alway write it to the same data location, otherwise, the index will have multiple segments.gen files, and the shard can seem to be corrupted.

The message if this case happens is that segments_xxx file was not found, in which case, a find for segments.gen can yield multiple files. Deleting the segments.gen files will cause the shard to recover properly (as its an extra protection layer to resolve the segments header by Lucene)

Make sure the segments.gen file is writtne to the same directory every time
fixes elastic#4674

Author: kimchy

src/main/java/org/elasticsearch/index/store/Store.java

@@ -22,6 +22,7 @@
 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableMap;
 import com.google.common.collect.Maps;
+import org.apache.lucene.index.IndexFileNames;
 import org.apache.lucene.store.*;
 import org.apache.lucene.util.IOUtils;
 import org.elasticsearch.common.Nullable;
@@ -426,7 +427,9 @@ public IndexOutput createOutput(String name, IOContext context) throws IOExcepti
         public IndexOutput createOutput(String name, IOContext context, boolean raw) throws IOException {
             ensureOpen();
             Directory directory;
-            if (isChecksum(name)) {
+            // we want to write the segments gen file to the same directory *all* the time
+            // to make sure we don't create multiple copies of it
+            if (isChecksum(name) || IndexFileNames.SEGMENTS_GEN.equals(name)) {
                 directory = distributor.primary();
             } else {
                 directory = distributor.any();
@@ -441,7 +444,7 @@ public IndexOutput createOutput(String name, IOContext context, boolean raw) thr
                     boolean computeChecksum = !raw;
                     if (computeChecksum) {
                         // don't compute checksum for segment based files
-                        if ("segments.gen".equals(name) || name.startsWith("segments")) {
+                        if (IndexFileNames.SEGMENTS_GEN.equals(name) || name.startsWith(IndexFileNames.SEGMENTS)) {
                             computeChecksum = false;
                         }
                     }
@@ -562,7 +565,7 @@ public void sync(Collection<String> names) throws IOException {
             }
             for (String name : names) {
                 // write the checksums file when we sync on the segments file (committed)
-                if (!name.equals("segments.gen") && name.startsWith("segments")) {
+                if (!name.equals(IndexFileNames.SEGMENTS_GEN) && name.startsWith(IndexFileNames.SEGMENTS)) {
                     writeChecksums();
                     break;
                 }