Skip to content

Не завещается бекап с реплики Wait for LSN in archived WAL segment #486

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
PavelSorokin opened this issue Apr 27, 2022 · 3 comments · Fixed by #493

Comments

@PavelSorokin
Copy link

PavelSorokin commented Apr 27, 2022

/bin/pg_probackup-13 backup -B /u01/backup --instance=myinst     -b FULL  --progress -j 10 
/bin/pg_probackup-13 --version
pg_probackup-13 2.5.5 (PostgreSQL 13.5)
pgconf
wal_sender_timeout = '5min'
min_wal_size = '35GB'
archive_timeout = '35min'
postgres=# show archive_mode ;
 archive_mode
--------------
 always
pg_probackup-13 show-config -B /u01/backup/ --instance=myinst     
# Backup instance information
pgdata = /var/lib/pgsql/13/data
system-identifier = 6941686244701855037
xlog-seg-size = 16777216
# Connection parameters
pgdatabase = postgres
pghost = myinst     
pgport = 5432
pguser = backup
# Replica parameters
replica-timeout = 65min
# Archive parameters
archive-timeout = 65min
# Logging parameters
log-level-console = INFO
log-level-file = INFO
log-filename = myinst/%Y-%m-%d_%H%M%S.log
error-log-filename = myinst/%Y-%m-%d_%H%M%S.error
log-directory = /u01/backup/log
log-rotation-size = 0TB
log-rotation-age = 0d
# Retention parameters
retention-redundancy = 2
retention-window = 14
wal-depth = 0
# Compression parameters
compress-algorithm = zlib
compress-level = 4
# Remote access parameters
remote-proto = ssh
remote-host = myinst     
remote-user = postgres

Бекап выполняю с реплики, после копирование файлов и выполнения pg_stop_backup() функции, зависает на ожидание LSN

LOG
2022-04-26 23:10:01 MSK [10490]: INFO: Backup start, pg_probackup version: 2.5.5, instance: myinst     , backup ID: RAYQOP, backup mode: FULL, wal mode: ARCHIVE, remote: true, compress-algorithm: zlib, compress-level: 4
2022-04-26 23:10:01 MSK [10490]: INFO: Backup RAYQOP is going to be taken from standby
2022-04-26 23:10:01 MSK [10490]: INFO: wait for pg_start_backup()
2022-04-26 23:10:04 MSK [10490]: INFO: Wait for WAL segment /u01/backup/wal/myinst/0000000100012E3B00000050 to be archived
2022-04-26 23:10:33 MSK [10490]: WARNING: By default pg_probackup assume WAL delivery method to be ARCHIVE. If continuous archiving is not set up, use '--stream' option to make autonomous backup. Otherwise check that continuous archiving works correctly.
2022-04-26 23:13:39 MSK [10490]: INFO: PGDATA size: 11TB
2022-04-26 23:13:39 MSK [10490]: INFO: Start transferring data files
******
2022-04-27 09:36:54 MSK [10490]: INFO: Progress: (45336/45336). Process file "base/16443/1001991216"
2022-04-27 09:37:24 MSK [10490]: INFO: Data files are transferred, time elapsed: 10h:23m
2022-04-27 09:37:25 MSK [10490]: INFO: wait for pg_stop_backup()
2022-04-27 09:37:25 MSK [10490]: INFO: pg_stop backup() successfully executed
2022-04-27 09:37:26 MSK [10490]: INFO: Wait for LSN 12EA6/45FFFFE0 in archived WAL segment /u01/backup/wal/myinst /0000000100012EA600000045

Дальше бекап, останавливается и не идет.

В логе PostgreSQL я вижу что данный сегмент в архив попал


2022-04-27 09:37:24 MSK [23341]: [1-1]: INFO: pg_probackup archive-push WAL file: 0000000100012EA600000042, threads: 4/10, batch: 4/10, compression: zlib
2022-04-27 09:37:25 MSK [23356]: [1-1]: INFO: pg_probackup archive-push WAL file: 0000000100012EA600000046, threads: 3/10, batch: 3/10, compression: zlib
2022-04-27 09:37:26 MSK [23372]: [1-1]: INFO: pg_probackup archive-push WAL file: 0000000100012EA600000049, threads: 3/10, batch: 3/10, compression: zlib

База 11 TB, после обновления до 2.5.5 FULL не можем сделать
В чем может быть проблема?

@funny-falcon
Copy link
Collaborator

А с какой версии обновлялись?
Что меняли в конфиге при обновлении?

Меня напрягает, что в выводе show-config параметры pghost и remote-host выведены с пробелами в конце: myinst
Или это вы так замену сделали для оформления тикета?

@PavelSorokin
Copy link
Author

PavelSorokin commented Apr 27, 2022

Была версия 2.4.15
В конфиге ничего не менял, уже после такого сообщения увеличил

Replica parameters
replica-timeout = 65min

Archive parameters
archive-timeout = 65min

В оригинале пробелов в конце нет.

Данная проблема была и на 2.4.15 но после перезапуска FULL бекап завершался. А сейчас я начиная с пятницы пытаюсь сделать бекап, но все время натыкаюсь на это сообщение.
Сейчас пока запустил с параметром stream

@funny-falcon
Copy link
Collaborator

А у вас случайно в конфиге ssh не прописано ControlMaster=yes или auto ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants