Skip to content

Commit 5466811

Browse files
Darwin Huangmibrunin
authored andcommitted
[Backport] Security bug 1161048
Partial manual backport (library update) of patch originally reviewed on https://chromium-review.googlesource.com/c/chromium/src/+/2600334: Roll src/third_party/sqlite/src/ 0324bd3ef..d9581878f (171 commits) https://chromium.googlesource.com/chromium/deps/sqlite.git/+log/0324bd3ef1af..d9581878fcf8 $ git log 0324bd3ef..d9581878f --date=short --no-merges --format='%ad %ae %s' 2020-12-22 huangdarwin Fix SQLITE_OMIT_WINDOWFUNC builds by moving declaration of sqlite3ExpandSubquery out of "ifndef SQLITE_OMIT_WINDOWFUNC" block. 2020-12-22 huangdarwin Amalgamations for release 3.34.0 2020-12-01 drh Version 3.34.0 2020-11-30 Dan Kennedy Update the tea/win/makefile.vc file that ships as part of the autoconf package to use "sqlite3" instead of "tclsqlite3" for the installed dll filename. 2020-11-30 drh Do not reuse the obsolete SQLITE_TESTCTRL_PRNG_RESET value for SQLITE_TESTCTRL_SEEK_COUNT. Give SEEK_COUNT its own unique value. This avoids incompatibility with legacy test code. 2020-11-27 Dan Kennedy Fix a compiler warning in fts5_aux.c. 2020-11-27 Dan Kennedy Add missing cast to fts5 bm25() code. 2020-11-27 Dan Kennedy Remove a redundant "if( rc==SQLITE_OK )" from the fts5 bm25() code. 2020-11-27 Dan Kennedy Move an "if( rc==SQLITE_OK )" to outside a loop body in the fts5 bm25() code. 2020-11-27 Dan Kennedy Fix a case in the FTS5 integrity check where a corrupt database could cause a buffer overread. 2020-11-26 Dan Kennedy Update mkunicode.tcl to match the change erroneously made to machine generated file fts5_unicode2.c in [b7b7bde9]. 2020-11-25 drh Update a requirement mark in the e_expr.test script. 2020-11-25 drh Fix harmless compiler warnings about unused function parameters. 2020-11-25 drh Fix a potential NULL pointer dereference in the geopoly_overlap() routine of the GeoPoly extension. 2020-11-25 drh Fix the shell1.test test script so that it works on windows. 2020-11-25 drh Change the -init option to the command-line shell to honor the -bail option. Also change it to report an error if the filename mentioned cannot be opened. 2020-11-24 drh Fix a typo in the sqlite3_txn_state() documentation. 2020-11-24 drh Add the new sqlite3_txn_state() interface to the set of routines accessible by run-time loadables extensions. 2020-11-24 drh Fix a harmless typo in a comment. 2020-11-24 dan Fix a bug in OOM handling code introduced by [6a28713d]. 2020-11-24 dan Ensure that super-journal and other journal filenames passed by SQLite to an sqlite3_vfs.xOpen() implementation may be safely passed to sqlite3_uri_parameter() and similar functions. 2020-11-24 drh Remove the VERSION_NUMBER macro from configure.ac as it has been unused since 2009 check-in [7f4810747b086498]. See also [forum:/forumpost/bb2c634fcd|forum post bb2c634fcd]. 2020-11-23 drh Syntactic changes to work around a bug in gcov 9.3.0. No changes to the generated code for release builds. 2020-11-23 drh In os_unix.c, put ALWAYS() on unreachable branches associated with pathname normalization. 2020-11-23 dan Fix a test script problem causing crash8.test to fail when run with leak-sanitizer. 2020-11-23 drh Allow "substring()" as an alias for "substr()" for compatibility with other SQL engines. 2020-11-20 dan Add file doc/vdbesort-memory.md, containing notes on the way vdbesort.c uses heap memory. 2020-11-19 drh More test case changes so that shared-cache mode tests all work with normalized database names. 2020-11-19 drh Fix the tkt3793 test case so that work even when filenames are normalized. 2020-11-18 drh Improvements to the mechanism that attempts to report SQLITE_CORRUPT if pread() says that the underlying filesystem is corrupt. 2020-11-18 dan Update cksumvfs.c so that if SQLITE_AMALGAMATION is defined, SQLITE_CKSUMVFS_STATIC is implied. 2020-11-18 dan Add test infrastructure for cksumvfs. And update cksumvfs so that it works in concert with version 2 VFSs. No changes to core SQLite. 2020-11-18 dan Fix a problem causing sqlite3_carray_bind() to return an undefined value. 2020-11-18 dan Update test code to fix a problem with SQLITE_OMIT_VIRTUAL_TABLE builds. 2020-11-18 drh On unix, for certain error codes of read()/pread() return SQLITE_IOERR_CORRUPTFS instead of SQLITE_IOERR_READ. And then convert this error into SQLITE_CORRUPT prior to returning back to the application. 2020-11-17 drh Claw back most of the performance lost in the previous commit. 2020-11-17 dan Fix trivial memory leaks in the shell and sqldiff programs. 2020-11-17 dan Fix test file busy2.test so that it works with the "inmemory_journal" permutation. 2020-11-17 drh Add a single-argument form to the CARRAY table-valued function, with content bound using the sqlite3_carray_bind() interface that is included with the extension. 2020-11-16 drh Enhance the unix VFS so that it removes extra "/", "/./" and "/../" from the database filename. 2020-11-14 drh Modify UPDATE so that two-pass updates on a rowid table use an ephemeral table to store rowids rather than a RowSet. This uses less memory, though it is slower. 2020-11-12 drh Improved diagnostics output with ".wheretrace 0x800". No changes to non-debug builds. 2020-11-11 drh Fix a typo in a comment. No changes to code. 2020-11-08 drh Use NEVER and ALWAYS macros to confirm that the return value from sqlite3ExprSkipCollateAndLikely() is never NULL in some of its use cases. 2020-11-06 drh Add ALWAYS() to conditionals associated with SHM locking that are always true. 2020-11-05 drh Suppress errors associated with TEMP triggers that reference objects in non-TEMP databases. This is a continuation of the fix for ticket #3810 shown in check-in [ba1afc040171810d] from [/timeline?c=trunk:200908061743|2009-08-06], based on a bug report in [forum:/forumpost/157dc791df|forum post 157dc791df] 2020-11-05 drh Remove unused variable from speedtest1.c 2020-11-02 drh Fix an another OSS-Fuzz discovered assertion fault due to ALTER TABLE and strange triggers. 2020-10-31 drh Add the -tabs command-line option to the CLI. 2020-10-31 dan Update the ALTER TABLE command to correctly handle UPDATE ... FROM statements within trigger programs. (...) 2020-09-15 drh Do not invoke usleep() for more than 999999 microseconds. 2020-09-11 dan Catch fts5 index corruption caused by issuing 'delete' commands with incorrect data earlier in some cases. Also fix a couple of test script problems. 2020-09-10 drh Try again to fix the typo in the sqlite3_txn_state() documentation. 2020-09-10 drh Fix typo in the documentation for the new sqlite3_txn_state() interface. 2020-09-10 drh If an amalgamation is created using SQLITE_ENABLE_UPDATE_DELETE_LIMIT, ensure that it can still be compiled without that option by setting the SQLITE_UDL_CAPABLE_PARSER flag in the preamble. 2020-09-07 dan Fix another (harmless in practice) tsan error in shared-cache mode. 2020-09-06 drh Remove an artifical limitation on the length of columns in the "box" output mode of the command-line shell. 2020-09-05 drh Fix a bug in Fossil that might cause it to crash if there is a multi-terminal token with a space following the "|" separator. This does not affect SQLite. 2020-09-04 dan Fix a tsan error that could occur when using shared-cache mode. 2020-09-04 dan Ensure that the busy-handler count is reset at the end of each sqlite3_file_control() and sqlite3_prepare() (and _v2() and _v3()). 2020-09-01 mistachkin Fix harmless compiler warning. 2020-09-01 drh In the Lemon output, add a prefix comment that explains that the output file is automatically generated and shows the name of the source file. 2020-09-01 drh Lemon updates: (1) include the #defines for all tokens in the generated C file, so that the C-file can be stand-alone. (2) If the grammar begins with a %include {...} directive on line one, make that directive the header for the generated C file. (3) Enhance the lemon.html documentation. 2020-09-01 drh Improvements to the IN-early-out optimization so that it works more efficiently when there are two or more indexed IN clauses on a single table. 2020-09-01 drh Fix a harmless compiler warning. 2020-08-31 drh New test-control that returns the number of calls to sqlite3BtreeMovetoUnpacked() on the main database and then resets the counter. This only works for SQLITE_DEBUG builds. 2020-08-31 drh Fix the documentation for the OP_IdxGT family of opcodes to show that the P5 operand is not used. 2020-08-29 dan Fix a sanitizer warning in zipfile.c. 2020-08-28 dan Fix handling of an xShmLock(SHARED, UNLOCK) call when the caller does not hold any lock on the specified slot, but another connection in the same process holds an EXCLUSIVE. 2020-08-28 dan Modify the unixShmLock() function to avoid iterating through the (possibly large) set of connections to the same database file. 2020-08-28 mistachkin Add the miscellaneous 'series' extension to the shell. 2020-08-28 drh Update Lemon documentation. Patches from sgbeal. 2020-08-28 drh Fix a couple of unreachable branches. 2020-08-28 dan Add fts5 test to confirm that for a table with columns a, b, c and d, "{a b}" and "-{c d}" are handled similarly. 2020-08-27 drh Remove a (harmless) redundant variable from the CLI implementation. 2020-08-27 drh Include the original text of the CHECK constraint in the error message for anonymous CHECK constraints. 2020-08-27 drh Apple the same fix for ticket [9eda2697f5cc1aba] to text-to-integer conversions that was done for floating point conversions by check-in [1c76f1d8ec0937a2]. 2020-08-27 dan Do not attempt to run test script analyze3.test as part of the "prepare" permutation. 2020-08-26 drh Enhance the ".databases" command in the CLI so that it shows the result of sqlite3_db_readonly() and sqlite3_txn_state() for each database file. 2020-08-26 drh If the argument to the ".read" command in the CLI begins with "|" then run the remainder of the argument as a command and read input from the output of that command. 2020-08-25 drh Add support for the sqlite3_txn_state() interface. 2020-08-24 drh Reduce the N in the logN term for the sorting cost associated with computing DISTINCT by B-Tree by half, under the assumption that the DISTINCT will eliminate about half the rows of output. This is really a wild guess. But we do not have any better way of estimating what the row-count reduction due to DISTINCT will actually be. 2020-08-24 dan Correctly handle expressions like "x IS (not) true/false" within the rhs of IN() expressions. Fix for [f3ff1472]. 2020-08-20 dan Fix a crash that could occur in SQLITE_MAX_EXPR_DEPTH=0 builds when processing SQL containing syntax errors. 2020-08-20 dan Avoid a buffer overrun in test code that could occur if certain test functions were passed a hex-string containing an odd number of digits. 2020-08-19 drh Try to make SQLite easier to compiler for Mac Catalyst. See [https://sqlite.org/forum/forumpost/803387a1c5|forum post 803387a1c5]. 2020-08-19 drh Do not skip over TK_IF_NULL_ROW operators when bypassing TK_COLLATE operators. Fix to check-in [ac31edd3eeafcef4] which was itself a fix for ticket [45f4bf4eb4ffd788]. 2020-08-19 dan Update releasetest_data.tcl so that the "Fast-One" configuration runs bigmmap.test as part of release testing. 2020-08-19 dan Fix a problem in test file bigmmap.test. 2020-08-17 drh Fix the -quote option on the CLI to set the correct column separator. 2020-08-16 drh Optimization to sqlite3BeginWriteOperation(). 2020-08-15 drh Optimization of the sqlite3SrcListDelete() routine. 2020-08-15 drh Remove more unnecessary sqlite3GetVdbe() calls, replacing them with assert()s. 2020-08-15 drh Avoid unnecessary calls to the sqlite3GetVdbe() routine. Add assert() statements to prove each call is unnecessary. 2020-08-15 drh Omit the unnecessary not-NULL check on the upper bound of a forward index range scan. The subsequent OP_IdxGT will always catch the NULL. Similarly, omit the not-NULL check on the lower bound of a reverse index range scan, as the SeekLE opcode will disallow the NULL. 2020-08-14 drh Update the version number to 3.34.0 for the next development cycle. 2020-08-14 drh Optimizations to the logic that converts main table accesses into equivalent index accesses. Code is now slightly smaller and faster than trunk. 2020-08-14 drh For UPDATE and DELETE, use OP_DeferredSeek always. If the seek must later be resolved, add the OP_FinishSeek opcode after all WHERE clause terms have been processed. This obviates the need for the WHERE_SEEK_TABLE and WHERE_SEEK_UNIQ_TABLE flags to sqlite3WhereBegin() and the ensuing complication, and it allows the covering index optimization to be used further into WHERE clause processing. 2020-08-14 drh Experimental change to try to get some DELETE operations to access values using the index rather than the main table, so as to avoid unnecessary main table seeks. 2020-08-14 dan Fix an inconsequential memory leak in sqldiff. Update the "Sanitize" case in wapptest.tcl to use -fsanitize=address,undefined, and to run more tests. Created with: roll-dep src/third_party/sqlite/src Bug: 1161048 Change-Id: Ieb8f0154930d6cc0597b5aee9444678cf021035b Reviewed-by: Chris Mumford <[email protected]> Commit-Queue: Darwin Huang <[email protected]> Cr-Commit-Position: refs/heads/master@{#839609} Reviewed-by: Jüri Valdmann <[email protected]>
1 parent 7535d5d commit 5466811

File tree

7 files changed

+13126
-11253
lines changed

7 files changed

+13126
-11253
lines changed

chromium/third_party/sqlite/OWNERS

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,3 @@ [email protected]
44
# Secondary:
55
66
7-
8-
9-
# COMPONENT: Internals>Storage
Lines changed: 7 additions & 327 deletions
Original file line numberDiff line numberDiff line change
@@ -1,334 +1,14 @@
11
Name: sqlite
22
URL: https://sqlite.org/
3-
Version: 3.31.1
3+
Version: 3.34.0
4+
CPEPrefix: cpe:/a:sqlite:sqlite:3.34.0
45
Included In Release: Yes
56
Security Critical: Yes
67
License: Public domain
78

8-
1) Managing differences between SQLite core and Chromium's version.
9-
2) Making changes to Chromium SQLite.
10-
3) Import new release of SQLite.
11-
4) Running SQLite's test suite within the Chromium checkout.
9+
Chromium maintains a mirror of the official SQLite Git repository at
10+
https://chromium.googlesource.com/chromium/deps/sqlite. Generally the stock
11+
upstream release branch is used, but occasionally unreleased upstream bug
12+
fixes are backported to the release branch.
1213

13-
---
14-
15-
1) Managing differences between SQLite core and Chromium's version.
16-
17-
Chromium maintains some differences WRT SQLite, for reasons beyond this
18-
document's remit. Some differences are bugs we have found and fixed (and
19-
hopefully upstreamed), some are fixes we've backported from a later version of
20-
SQLite, and some our local changes unlikely to ever be upstreamed. New versions
21-
of SQLite are imported every year or two, at which point the changes need to be
22-
reviewed for continued applicability, and sometimes adjusted to reflect upstream
23-
code changes.
24-
25-
To this end, the repository contains a reference copy of the SQLite source code
26-
as of the last import, plus a series of patches which can be applied to
27-
re-create the current trunk code. These patches are generated and processed by
28-
git, with the intention of re-creating a commit series so that importers can use
29-
their regular revision-control knowledge to manage import merges.
30-
31-
The directory structure is as follows. Files common to all third_party projects
32-
(BUILD.GN, OWNERS, LICENSE) are omitted.
33-
34-
* sqlite-src-*/ - Upstream source code, without any modifications. The number
35-
after src- is a release version or a snapshot number (which is
36-
a commit time). See https://www.sqlite.org/cgi/src/timeline
37-
* patches/ - Our patches to the currently used release, formatted by git using
38-
the UNIX mailbox format. The patches can be applied with git am,
39-
and created with git format-patch.
40-
* patched/ - The currently used source code, with our patches applied.
41-
* amalgamation/ - The supported method of using SQLite is via an amalgamation
42-
build, which merges all the code in one .c file and one .h
43-
file. See https://www.sqlite.org/amalgamation.html
44-
* amalgamation/config.h - Linux build configuration
45-
* scripts/ - Scripts that generate the files in the amalgamation
46-
* sqlite.h - The header used by the rest of Chromium to include SQLite. This
47-
forwards to amalgamation/sqlite3.h
48-
* fuzz/ - Google OSS-Fuzz (ClusterFuzz) testing for Chromium's SQLite build
49-
50-
---
51-
52-
2) Making changes to Chromium SQLite.
53-
54-
third_party/sqlite/patched is the patched source from SQLite. This is used to
55-
generate the amalgamation, a concatenation of all of the files into a giant
56-
sqlite3.c. To prototype, edit in patched/, then call
57-
./scripts/generate_amalgamation.sh
58-
to regenerate sqlite3.c. The code in patched/ is much easier to edit, and the
59-
SQLite test framework can easily be run. During development it may be
60-
convenient to modify BUILD.gn based on patched/main.mk to just pull in the
61-
patched/ files rather than sqlite3.c.
62-
63-
Once your patch is complete, squash it down into a reasonable CL, then
64-
re-generate the patches. This is a truncated version of the import flow. The
65-
following is written like a shell script to allow copy/paste to a shell, ignore
66-
comments and change the obvious lines. These instructions should work on Linux
67-
or OSX. They may assume a modern version of git.
68-
69-
# The steps below are easier if done in the SQLite directory.
70-
cd third_party/sqlite
71-
72-
# Must match the version in //third_party/sqlite/sqlite-src-xxxxxxx.
73-
# This is SQLite's version number, and uses upstream's convention.
74-
export BASE=3310100
75-
export GNU_SED=sed # OSX: "brew install gnu-sed", then use "gsed" here.
76-
77-
#### Create a reference branch.
78-
git new-branch sqlite-base
79-
git rm -rf patched
80-
cp -r sqlite-src-${BASE}/ patched
81-
# Clean up trailing whitespace and CRLF so any patches look clean.
82-
find patched/ -type f -not -iname "*.db" -not -iname "*.eps" \
83-
-not -iname "*.ico" -not -iname "*.jpg" -not -iname "*.pfx" \
84-
-not -iname "*.png" -not -iname "*.tiff" -not -iname "*.vsix" \
85-
-exec ${GNU_SED} --in-place 's/[[:space:]]\+$//' {} \+
86-
git add patched/
87-
git clean -i -d -x patched # Make sure no file is git-ignored.
88-
git commit -m "Squash: Reset SQLite patched/ to sqlite-src-${BASE}."
89-
# This branch will not build. It will be used for rebasing, then deleted.
90-
91-
#### Create a reference branch with patches applied.
92-
git new-branch --upstream-current sqlite-dev
93-
git am --keep-non-patch --ignore-space-change patches/*.patch
94-
git diff origin/master patched/
95-
# This branch should be identical to master.
96-
97-
#### Develop and validate the change, or cherry-pick it from a dev branch.
98-
# The goal is to have a set of reasonably-independent CLs which can be
99-
# understood separately, so that future importers can sensibly determine how to
100-
# handle conflicts. So use git-rebase and slipstream fixups back into existing
101-
# patches, or add a new patch.
102-
./scripts/generate_amalgamation.sh
103-
git cl format amalgamation/rename_exports.h
104-
cd ../..
105-
ninja -C out/Default
106-
# Check that extract_sqlite_api.py added chrome_ to all exported symbols.
107-
# Only "_fini" and "_init" should be unprefixed.
108-
nm -B out/Default/libchromium_sqlite3.so | cut -c 18- | sort | grep '^T'
109-
out/Default/sql_unittests
110-
third_party/blink/tools/run_web_tests.py -t Default storage/websql/
111-
cd third_party/sqlite
112-
113-
#### Create the review.
114-
# Rebuild the patch set.
115-
git rm patches/*
116-
git format-patch --output-directory=patches --zero-commit \
117-
sqlite-base..sqlite-dev
118-
119-
### Document and link any backported patches to the upstream repository URL
120-
### and crbug.
121-
# Under the *.patch's added by this change, the below information should be
122-
# added between the "Subject: *" line and the first "---" line. For an example,
123-
# please reference https://crrev.com/c/1480822/2/third_party/sqlite/patches/0008-Adjustments-to-the-page-cache-to-try-to-avoid-harmle.patch.
124-
#
125-
# This backports https://www.sqlite.org/src/info/XXX
126-
#
127-
# Bug: XXX
128-
129-
### Commit and create CL
130-
git add amalgamation/
131-
git add patches/
132-
git commit -m "Squash: regenerate amalgamation and patches."
133-
git branch --set-upstream-to=origin/master
134-
git cl upload --squash
135-
136-
---
137-
138-
3) Import a new SQLite release.
139-
140-
Importing a new SQLite involves merging our local changes with SQLite's changes.
141-
Like any other merge, this may involve modifying some commits and dropping
142-
others. The basic idea below is to generate git branches to work with:
143-
144-
* sqlite-new-upstream - new release code archived in a separate directory
145-
* sqlite-old-base - current release without patches
146-
* sqlite-old - current release with patches mapped to git commits
147-
* sqlite-new-base - new release without patches
148-
* sqlite-new - new release with patches mapped to git commits
149-
* sqlite-new-cl - new release in one git commit, for git cl upload
150-
151-
We will upload sqlite-new-upstream as a massive (800k LOC+) CL that cannot
152-
possibly be reviewed, but is generated in an automated fashion. We will then
153-
squash sqlite-new to sqlite-new-upstream and obtain one CL that only contains
154-
diffs. The second CL is still large, but it's a fraction of the first
155-
(automated) CL.
156-
157-
# The steps below are easier if done in the SQLite directory.
158-
cd third_party/sqlite
159-
160-
# The numbers below are SQLite version numbers, and use upstream's convention
161-
# for tagging release binaries and source zipballs.
162-
export OLD=3310100
163-
export NEW=3320000
164-
export GNU_SED=sed # OSX: "brew install gnu-sed", then use "gsed" here.
165-
export YEAR=2020
166-
167-
#### Download and unpack the new SQLite release.
168-
git new-branch sqlite-new-upstream
169-
# URL from "Alternative Source Code Formats" at https://sqlite.org/download.html
170-
curl https://sqlite.org/${YEAR}/sqlite-src-${NEW}.zip > upstream.zip
171-
mkdir sqlite-src-${NEW}
172-
unzip ./upstream.zip -d sqlite-src-${NEW}
173-
rm ./upstream.zip
174-
mv sqlite-src-${NEW}/sqlite-*/* sqlite-src-${NEW}/
175-
rm -r sqlite-src-${NEW}/sqlite-*/.fossil-settings
176-
rmdir sqlite-src-${NEW}/sqlite-*/
177-
rm -r sqlite-src-${NEW}/compat
178-
xdg-open sqlite-src-${NEW} # Make sure everything looks right.
179-
180-
#### Add the new release code in a separate CL, for code review sanity.
181-
git add sqlite-src-${NEW} # Committing the code as downloaded, on purpose.
182-
git clean -i -d -x sqlite-src-${NEW} # Make sure no file is git-ignored.
183-
git commit -m "sqlite: Add code for release ${NEW}"
184-
git cl upload # Have the new code in a separate CL.
185-
186-
#### Create a branch for the old SQLite release's upstream version.
187-
git new-branch sqlite-old-base
188-
git rm -rf patched
189-
cp -r sqlite-src-${OLD}/ patched
190-
# Clean up trailing whitespace and CRLF so any patches look clean.
191-
find patched/ -type f -not -iname "*.db" -not -iname "*.eps" \
192-
-not -iname "*.ico" -not -iname "*.jpg" -not -iname "*.pfx" \
193-
-not -iname "*.png" -not -iname "*.tiff" -not -iname "*.vsix" \
194-
-exec ${GNU_SED} --in-place 's/[[:space:]]\+$//' {} \+
195-
git add patched/
196-
git clean -i -d -x patched # Make sure no file is git-ignored.
197-
git commit -m "Squash: Reset SQLite patched/ to sqlite-src-${OLD}."
198-
# This branch will not build. It will be used for rebasing, then deleted.
199-
200-
#### Create a branch for our old SQLite code, with patches mapped to commits.
201-
git new-branch --upstream-current sqlite-old
202-
git am --keep-non-patch --ignore-space-change patches/*.patch
203-
git diff origin/master patched/
204-
# This branch should be identical to master.
205-
206-
#### Create a branch for the new SQLite release's upstream version.
207-
git checkout sqlite-old-base
208-
git new-branch --upstream-current sqlite-new-base
209-
git rm -rf patched
210-
git checkout sqlite-new-upstream -- sqlite-src-${NEW}/
211-
git mv sqlite-src-${NEW}/ patched
212-
# Clean up trailing whitespace and CRLF so any patches look clean.
213-
find patched/ -type f -not -iname "*.db" -not -iname "*.eps" \
214-
-not -iname "*.ico" -not -iname "*.jpg" -not -iname "*.pfx" \
215-
-not -iname "*.png" -not -iname "*.tiff" -not -iname "*.vsix" \
216-
-exec ${GNU_SED} --in-place 's/[[:space:]]\+$//' {} \+
217-
git add patched/
218-
git clean -i -d -x patched # Make sure no file is git-ignored.
219-
git commit -m "Squash: Reset SQLite patched/ to sqlite-src-${NEW}."
220-
# This branch will not build. It will be used for rebasing, then deleted.
221-
222-
#### Create a branch for updating our patches.
223-
git checkout sqlite-old
224-
git new-branch --upstream-current sqlite-new
225-
# Rebase our patches, which are mapped to separate commits, onto the new
226-
# release. There will be merge conflicts that must be fixed. This is the
227-
# interesting part of the work.
228-
git rebase sqlite-new-base
229-
230-
#### Finally, create the branch that we'll upload.
231-
git new-branch --upstream-current sqlite-new-cl
232-
./scripts/generate_amalgamation.sh
233-
git cl format amalgamation/rename_exports.h
234-
235-
#### Copy any new entries to the seed-corpus for dbfuzz2.
236-
# We use a set of seed databases for fuzzing SQLite's resilience to database
237-
# corruption. Sometimes, new seed databases are added upstream.
238-
# Find any files of the pattern patched/test/dbfuzz2-seed*.db, and copy them to
239-
# the fuzz/db_corpus directory.
240-
cp --no-clobber patched/test/dbfuzz2-seed*.db fuzz/db_corpus
241-
242-
#### Validate the upgrade.
243-
# The goal is to have a set of reasonably-independent CLs which can be
244-
# understood separately, so that future importers can sensibly determine how to
245-
# handle conflicts. So use git-rebase and slipstream fixups back into their
246-
# original CL until everything builds and works.
247-
cd ../..
248-
autoninja -C out/Default
249-
# Check that extract_sqlite_api.py added chrome_ to all exported symbols.
250-
# Only "_fini" and "_init" should be unprefixed.
251-
nm -B out/Default/libchromium_sqlite3.so | cut -c 18- | sort | grep '^T'
252-
out/Default/sql_unittests
253-
third_party/blink/tools/run_web_tests.py -t Default storage/websql/
254-
cd third_party/sqlite
255-
256-
#### Create the review.
257-
# Rebuild the patch set.
258-
git rm patches/*
259-
git format-patch --output-directory=patches --zero-commit \
260-
sqlite-new-base..sqlite-new
261-
git add amalgamation/
262-
git add patches/
263-
git commit -m "Squash: regenerate amalgamation and patches."
264-
git branch --set-upstream-to=origin/master
265-
git cl upload --squash
266-
# Example commit message (https://crrev.com/c/1699400):
267-
# sqlite: Upgrade from 3.28.0 to 3.29.0
268-
#
269-
# sqlite 3.29.0 update changes:
270-
# https://www.sqlite.org/releaselog/3_29_0.html
271-
#
272-
# Bug: 983713
273-
274-
#### Drop the old version of SQLite.
275-
git new-branch sqlite-rm-old
276-
git rm -r sqlite-src-${OLD}
277-
git commit -m "sqlite: Remove source code for old release ${OLD}."
278-
git cl upload
279-
280-
### Update README.chromium to reference the new sqlite version number.
281-
# Update:
282-
# (1) the header's "Version"
283-
# (2) `export OLD` and `export NEW` values.
284-
285-
--------------------------------------------
286-
287-
4) Running SQLite's test suite within the Chromium checkout.
288-
289-
TODO(pwnall): This hasn't been tried out for at least a year.
290-
291-
Prerequisites: The test suite requires tcl-dev and libicu-dev. Install those on
292-
Ubuntu like:
293-
sudo apt-get install tcl8.6-dev libicu-dev
294-
On macOS, I use Homebrew:
295-
brew install icu4c tcl-tk
296-
export PATH="$(brew --prefix icu4c)/bin:$(brew --prefix tcl-tk)/bin:$PATH"
297-
298-
Run the commands in scripts/generate_amalgamation.sh, but replace the "make"
299-
command with "make test".
300-
301-
make test > /tmp/test.log
302-
egrep 'errors out of' /tmp/test.log
303-
# Show broken tests:
304-
egrep 'Failures on these tests:' /tmp/test.log
305-
# Broken tests will also show lines ending in "..." instead of "... Ok".
306-
307-
In version 3.10.2 on OSX 10.11.2, I see:
308-
6 errors out of 139819 tests
309-
The failed tests are:
310-
pager4-1.3 pager4-1.4 pager4-1.5 pager4-1.9 pager4-1.10 pager4-1.11
311-
This is due to the change in os_unix.c fileHasMoved() to support WebDatabase.
312-
Commenting out the early return allows them to succeed.
313-
314-
In version 3.10.2 on Ubuntu 14.04.3 I see:
315-
9 errors out of 140309 tests
316-
The failed tests are:
317-
oserror-1.1.1 oserror-1.1.2 oserror-1.1.3
318-
pager4-1.3 pager4-1.4 pager4-1.5 pager4-1.9 pager4-1.10 pager4-1.11
319-
The oserror tests fail because there are too many fds available, and can be
320-
fixed by running "ulimit -n 1024" before the test. The pager4 tests are failing
321-
for the same reason as for OSX.
322-
323-
--
324-
325-
NOTE(pwnall): On Ubuntu it is possible to run the tests in a tmpfs something
326-
like:
327-
328-
TMPFS=/dev/shm/sqlite_build
329-
BUILD=$PWD
330-
mkdir $TMPFS
331-
(cd $TMPFS ; $BUILD/testfixture $BUILD/../test/veryquick.test >/tmp/test.log)
332-
333-
This is faster, but it is plausible that different things are being tested than
334-
real-world use.
14+
The official upstream SQLite Git mirror is at https://github.com/sqlite/sqlite.

0 commit comments

Comments
 (0)