Enhance nbtree tuple scan key optimizations.

Postgres 17 commit e0b1ee1 added two closely related nbtree
optimizations: the "prechecked" and "firstpage" optimizations.  Both
optimizations avoided needlessly evaluating keys that are guaranteed to
be satisfied by applying page-level context.  These optimizations were
adapted to work with the nbtree ScalarArrayOp execution patch a few
months later, which became commit 5bf748b.

The "prechecked" design had a number of notable weak points.  It didn't
account for the fact that an = array scan key's sk_argument field might
need to advance at the point of the page precheck (it didn't check the
precheck tuple against the key's array, only the key's sk_argument,
which needlessly made it ineffective in corner cases involving stepping
to a page having advanced the scan's arrays using a truncated high key).
It was also an "all or nothing" optimization: either it was completely
effective (skipping all required-in-scan-direction keys against all
attributes) for the whole page, or it didn't work at all.  This also
implied that it couldn't be used on pages where the scan had to
terminate before reaching the end of the page due to an unsatisfied
low-order key setting continuescan=false.

Replace both optimizations with a new optimization without any of these
weak points.  This works by giving affected _bt_readpage calls a scankey
offset that its _bt_checkkeys calls start at (an offset to the first key
that might not be satisfied by every non-pivot tuple from the page).
The new optimization is activated at the same point as the previous
"prechecked" optimization (at the start of a _bt_readpage of any page
after the scan's first).

The old "prechecked" optimization worked off of the highest non-pivot
tuple on the page (or the lowest, when scanning backwards), but the new
"startikey" optimization always works off of a pair of non-pivot tuples
(the lowest and the highest, taken together).  This approach allows the
"startikey" optimization to bypass = array key comparisons whenever
they're satisfied by _some_ element (not necessarily the current one).
This is useful for SAOP array keys (it fixes the issue with truncated
high keys), and is needed to get the most out of range skip array keys
(we expect to be able to bypass range skip array = keys when a range of
values on the page all satisfy the key, even when there are multiple
values, provided they all "satisfy some range skip array element").

Although this is independently useful work, the main motivation is to
fix regressions in index scans that are nominally eligible to use skip
scan, but can never actually benefit from skipping.  These are cases
where a leading prefix column contains many distinct values, especially
when the number of values approaches the total number of index tuples,
where skipping can never be profitable.  The CPU costs of skip array
maintenance is by far the main cost that must be kept under control.

Skip scan's approach of adding skip arrays during preprocessing and then
fixing (or significantly ameliorating) the resulting regressions seen in
unsympathetic cases is enabled by the optimization added by this commit
(and by the "look ahead" optimization introduced by commit 5bf748b).
This allows the planner to avoid generating distinct, competing index
paths (one path for skip scan, another for an equivalent traditional
full index scan).  The overall effect is to make scan runtime close to
optimal, even when the planner works off an incorrect cardinality
estimate.  Scans will also perform well given a skipped column with data
skew: individual groups of pages with many distinct values in respect of
a skipped column can be read about as efficiently as before, without
having to give up on skipping over other provably-irrelevant leaf pages.

Author: Peter Geoghegan <[email protected]>
Reviewed-By: Heikki Linnakangas <[email protected]>
Reviewed-By: Masahiro Ikeda <[email protected]>
Reviewed-By: Matthias van de Meent <[email protected]>
Discussion: https://postgr.es/m/CAH2-Wz=Y93jf5WjoOsN=xvqpMjRy-bxCE037bVFi-EasrpeUJA@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-WznWDK45JfNPNvDxh6RQy-TaCwULaM5u5ALMXbjLBMcugQ@mail.gmail.com

Author: petergeoghegan

src/backend/access/nbtree/nbtpreprocesskeys.c

@@ -1389,6 +1389,7 @@ _bt_preprocess_array_keys(IndexScanDesc scan, int *new_numberOfKeys)
 	arrayKeyData = (ScanKey) palloc(numArrayKeyData * sizeof(ScanKeyData));
 
 	/* Allocate space for per-array data in the workspace context */
+	so->skipScan = (numSkipArrayKeys > 0);
 	so->arrayKeys = (BTArrayKeyInfo *) palloc(numArrayKeys * sizeof(BTArrayKeyInfo));
 
 	/* Allocate space for ORDER procs used to help _bt_checkkeys */

src/backend/access/nbtree/nbtree.c

@@ -349,6 +349,7 @@ btbeginscan(Relation rel, int nkeys, int norderbys)
 	else
 		so->keyData = NULL;
 
+	so->skipScan = false;
 	so->needPrimScan = false;
 	so->scanBehind = false;
 	so->oppositeDirCheck = false;

src/backend/access/nbtree/nbtsearch.c

@@ -1648,47 +1648,14 @@ _bt_readpage(IndexScanDesc scan, ScanDirection dir, OffsetNumber offnum,
 	pstate.finaltup = NULL;
 	pstate.page = page;
 	pstate.firstpage = firstpage;
+	pstate.forcenonrequired = false;
+	pstate.startikey = 0;
 	pstate.offnum = InvalidOffsetNumber;
 	pstate.skip = InvalidOffsetNumber;
 	pstate.continuescan = true; /* default assumption */
-	pstate.prechecked = false;
-	pstate.firstmatch = false;
 	pstate.rechecks = 0;
 	pstate.targetdistance = 0;
 
-	/*
-	 * Prechecking the value of the continuescan flag for the last item on the
-	 * page (for backwards scan it will be the first item on a page).  If we
-	 * observe it to be true, then it should be true for all other items. This
-	 * allows us to do significant optimizations in the _bt_checkkeys()
-	 * function for all the items on the page.
-	 *
-	 * With the forward scan, we do this check for the last item on the page
-	 * instead of the high key.  It's relatively likely that the most
-	 * significant column in the high key will be different from the
-	 * corresponding value from the last item on the page.  So checking with
-	 * the last item on the page would give a more precise answer.
-	 *
-	 * We skip this for the first page read by each (primitive) scan, to avoid
-	 * slowing down point queries.  They typically don't stand to gain much
-	 * when the optimization can be applied, and are more likely to notice the
-	 * overhead of the precheck.  Also avoid it during scans with array keys,
-	 * which might be using skip scan (XXX fixed in next commit).
-	 */
-	if (!pstate.firstpage && !arrayKeys && minoff < maxoff)
-	{
-		ItemId		iid;
-		IndexTuple	itup;
-
-		iid = PageGetItemId(page, ScanDirectionIsForward(dir) ? maxoff : minoff);
-		itup = (IndexTuple) PageGetItem(page, iid);
-
-		/* Call with arrayKeys=false to avoid undesirable side-effects */
-		_bt_checkkeys(scan, &pstate, false, itup, indnatts);
-		pstate.prechecked = pstate.continuescan;
-		pstate.continuescan = true; /* reset */
-	}
-
 	if (ScanDirectionIsForward(dir))
 	{
 		/* SK_SEARCHARRAY forward scans must provide high key up front */
@@ -1716,6 +1683,13 @@ _bt_readpage(IndexScanDesc scan, ScanDirection dir, OffsetNumber offnum,
 			so->scanBehind = so->oppositeDirCheck = false;	/* reset */
 		}
 
+		/*
+		 * Consider pstate.startikey optimization once the ongoing primitive
+		 * index scan has already read at least one page
+		 */
+		if (!pstate.firstpage && minoff < maxoff)
+			_bt_set_startikey(scan, &pstate);
+
 		/* load items[] in ascending order */
 		itemIndex = 0;
 
@@ -1752,6 +1726,7 @@ _bt_readpage(IndexScanDesc scan, ScanDirection dir, OffsetNumber offnum,
 			{
 				Assert(!passes_quals && pstate.continuescan);
 				Assert(offnum < pstate.skip);
+				Assert(!pstate.forcenonrequired);
 
 				offnum = pstate.skip;
 				pstate.skip = InvalidOffsetNumber;
@@ -1761,7 +1736,6 @@ _bt_readpage(IndexScanDesc scan, ScanDirection dir, OffsetNumber offnum,
 			if (passes_quals)
 			{
 				/* tuple passes all scan key conditions */
-				pstate.firstmatch = true;
 				if (!BTreeTupleIsPosting(itup))
 				{
 					/* Remember it */
@@ -1816,7 +1790,8 @@ _bt_readpage(IndexScanDesc scan, ScanDirection dir, OffsetNumber offnum,
 			int			truncatt;
 
 			truncatt = BTreeTupleGetNAtts(itup, rel);
-			pstate.prechecked = false;	/* precheck didn't cover HIKEY */
+			pstate.forcenonrequired = false;
+			pstate.startikey = 0;
 			_bt_checkkeys(scan, &pstate, arrayKeys, itup, truncatt);
 		}
 
@@ -1855,6 +1830,13 @@ _bt_readpage(IndexScanDesc scan, ScanDirection dir, OffsetNumber offnum,
 			so->scanBehind = so->oppositeDirCheck = false;	/* reset */
 		}
 
+		/*
+		 * Consider pstate.startikey optimization once the ongoing primitive
+		 * index scan has already read at least one page
+		 */
+		if (!pstate.firstpage && minoff < maxoff)
+			_bt_set_startikey(scan, &pstate);
+
 		/* load items[] in descending order */
 		itemIndex = MaxTIDsPerBTreePage;
 
@@ -1894,6 +1876,11 @@ _bt_readpage(IndexScanDesc scan, ScanDirection dir, OffsetNumber offnum,
 			Assert(!BTreeTupleIsPivot(itup));
 
 			pstate.offnum = offnum;
+			if (arrayKeys && offnum == minoff && pstate.forcenonrequired)
+			{
+				pstate.forcenonrequired = false;
+				pstate.startikey = 0;
+			}
 			passes_quals = _bt_checkkeys(scan, &pstate, arrayKeys,
 										 itup, indnatts);
 
@@ -1905,6 +1892,7 @@ _bt_readpage(IndexScanDesc scan, ScanDirection dir, OffsetNumber offnum,
 			{
 				Assert(!passes_quals && pstate.continuescan);
 				Assert(offnum > pstate.skip);
+				Assert(!pstate.forcenonrequired);
 
 				offnum = pstate.skip;
 				pstate.skip = InvalidOffsetNumber;
@@ -1914,7 +1902,6 @@ _bt_readpage(IndexScanDesc scan, ScanDirection dir, OffsetNumber offnum,
 			if (passes_quals && tuple_alive)
 			{
 				/* tuple passes all scan key conditions */
-				pstate.firstmatch = true;
 				if (!BTreeTupleIsPosting(itup))
 				{
 					/* Remember it */
@@ -1970,6 +1957,20 @@ _bt_readpage(IndexScanDesc scan, ScanDirection dir, OffsetNumber offnum,
 		so->currPos.itemIndex = MaxTIDsPerBTreePage - 1;
 	}
 
+	/*
+	 * If _bt_set_startikey told us to temporarily treat the scan's keys as
+	 * nonrequired (possible only during scans with array keys), there must be
+	 * no lasting consequences for the scan's array keys.  The scan's arrays
+	 * should now have exactly the same elements as they would have had if the
+	 * nonrequired behavior had never been used.  (In general, a scan's arrays
+	 * are expected to track its progress through the index's key space.)
+	 *
+	 * We are required (by _bt_set_startikey) to call _bt_checkkeys against
+	 * pstate.finaltup with pstate.forcenonrequired=false to allow the scan's
+	 * arrays to recover.  Assert that that step hasn't been missed.
+	 */
+	Assert(!pstate.forcenonrequired);
+
 	return (so->currPos.firstItem <= so->currPos.lastItem);
 }