Barman - Backup and Recovery Manager for PostgreSQL

EDB

Last update: Dec 30, 2022

Related tags

Storage postgres backup postgresql recovery disaster-recovery

Overview

Barman, Backup and Recovery Manager for PostgreSQL

Barman (Backup and Recovery Manager) is an open-source administration tool for disaster recovery of PostgreSQL servers written in Python. It allows your organisation to perform remote backups of multiple servers in business critical environments to reduce risk and help DBAs during the recovery phase.

Barman is distributed under GNU GPL 3 and maintained by 2ndQuadrant.

For further information, look at the "Web resources" section below.

Source content

Here you can find a description of files and directory distributed with Barman:

AUTHORS : development team of Barman
NEWS : release notes
ChangeLog : log of changes
LICENSE : GNU GPL3 details
TODO : our wishlist for Barman
barman : sources in Python
doc : tutorial and man pages
scripts : auxiliary scripts
tests : unit tests

Web resources

Website : http://www.pgbarman.org/
Download : http://sourceforge.net/projects/pgbarman/files/
Documentation : http://www.pgbarman.org/documentation/
Man page, section 1 : http://docs.pgbarman.org/barman.1.html
Man page, section 5 : http://docs.pgbarman.org/barman.5.html
Community support : http://www.pgbarman.org/support/
Professional support : https://www.2ndquadrant.com/
pgespresso extension : https://github.com/2ndquadrant-it/pgespresso

Licence

Barman is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

Barman is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with Barman. If not, see http://www.gnu.org/licenses/.

Comments

Some imporvements on barman documentation

Hello there, I am trying to set up barman for the first time and its a little bit confusing, the documentation. I have created a user called barman on hostname called 'pg' as the doc says, and then at this point:

Make sure you test the following command before proceeding: barman@backup$ psql -c 'SELECT version()' -U barman -h pg postgres

I got lost, barman@backup, I thought I should test that on pg instance not on the backup as the documentation says, in addition to that the user must be postgres to run the above command. Actually, I tested this as postgres@pg and it seems working like that. Here is an output in my case.

PostgreSQL 13.4 on x86_64-redhat-linux-gnu, compiled by gcc (GCC) 11.2.1 2021120 3 (Red Hat 11.2.1-7), 64-bit (1 row)

I just wanna suggest a correction if my correction is right, and a little bit of improvement on the documentation, to grasp the concepts easily. Thank you.
bug

opened by elhananjair 27
Replace argh with argparse

This is an initial attempt to replace argh with the stdlib argparse library to resolve #398

Should be compatible across all python's supported by barman (at least locally it was). The tests for the cli pass as well. However there are still some issues to resolve, like the help command showing properly the positional arguments.

Let me know what you think of that approach.

opened by stratakis 24
Incremental backups

Hello. I would like to discuss page-level incremental backups. I’ve created proof-of-concept fork of barman here There is no docs and unit-tests right now, but this will be fixed in near future.

Motivation: We have large number of databases with pgdata size about 3 terabytes and changes about 1% of data per 24h. Unfortunately barman backups with hardlinks gives us about 45% deduplication ratio (there are small changes in many data-files, so many data-files changes between backups, but page changed ratio is about 2%)

Solution to this problem seems simple: take only changed pages to backup. I’ve created simple script named barman-incr (it is in bin dir of source code). It handles backup and restore operations. Barman runs it on database host and passes LSN, timestamp and list of files from previous backup. Then we just open each datafile and read every page in it (if it turns out that file we opened is not datafile, we’ll take it all). If page is lsn >= provided lsn we take this page to backup.

Some tests: Database with pgdata size 2.7T, 120G wals per 24h. Full backup size is 537G (compressed with gzip -3), time to take backup - 7h. Incremental backup size is 14G (also compressed with gzip -3), time to take backup - 30m.

I’ve also tested restore consistency (restored database to some point of time and compared pg_dump result with paused replica).

Block change tracking (Oracle DBAs should be familiar with this, here is white paper about this) implementation will require some changes in wal archiving process. I’ll present some thoughts and test results on this in Q1 2016.
enhancement

opened by secwall 23

Streaming-only backups/WAL archiving?

I try to do streaming-only Barman backup scenario. My barman.conf is

      [barman]
      barman_user = barman
      configuration_files_directory = /etc/barman.d
      barman_home = ${barmanHome}
      log_level = INFO
      compression = gzip

      [live]
      description =  "Streaming copy of live"
      conninfo = host=${liveHostname} user=barman dbname=postgres
      streaming_conninfo = host=${liveHostname} user=streaming_barman
      backup_method = postgres
      archiver = off
      streaming_archiver = on
      slot_name = barman

My postgresql.conf is

synchronous_standby_names = 'barman_receive_wal'
max_wal_senders = 10
max_replication_slots = 10
wal_level = 'hot_standby'

I do start wal receive process:

# barman receive-wal live
Starting receive-wal for server live
2017-03-06 15:45:59,311 [4456] barman.server INFO: Starting receive-wal for server live
2017-03-06 15:45:59,324 [4456] barman.wal_archiver INFO: Synchronous WAL streaming for barman_receive_wal: True
2017-03-06 15:45:59,325 [4456] barman.wal_archiver INFO: Activating WAL archiving through streaming protocol
live: pg_receivexlog: starting log streaming at 0/2000000 (timeline 1)
2017-03-06 15:45:59,330 [4456] barman.command_wrappers INFO: live: pg_receivexlog: starting log streaming at 0/2000000 (timeline 1)

But barman check live complains:

# barman check live
Server live:
2017-03-06 21:41:54,942 [6044] barman.server ERROR: Check 'WAL archive' failed for server 'live'
        WAL archive: FAILED (please make sure WAL shipping is setup)
2017-03-06 21:41:54,957 [6044] barman.wal_archiver INFO: Synchronous WAL streaming for barman_receive_wal: True
        PostgreSQL: OK
        superuser: OK
        PostgreSQL streaming: OK
        wal_level: OK
        replication slot: OK
        directories: OK
        retention policy settings: OK
        backup maximum age: OK (no last_backup_maximum_age provided)
        compression settings: OK
        failed backups: OK (there are 0 failed backups)
        minimum redundancy requirements: OK (have 0 backups, expected at least 0)
        pg_basebackup: OK
        pg_basebackup compatible: OK
        pg_basebackup supports tablespaces mapping: OK
        pg_receivexlog: OK
        pg_receivexlog compatible: OK
        receive-wal running: OK
        archiver errors: OK

Is it mandatory to rsync WAL files from DB server for Barman to be able to do streaming backups?

(barman 2.1)

question

opened by danbst 17

Backup fails using postgresql method

I've updated the barman version from 2.3 to 2.4 (debian strech package from the pgdg repository ) and now I've problems using the postgresql basebackup method. The basebackup process seems to completing the data transfer but the backup ends with an error:

cat /var/log/barman/barman.log | grep 32434
2018-05-28 11:45:46,894 [32434] barman.config DEBUG: Including configuration file: pgcluster.conf
2018-05-28 11:45:46,899 [32434] barman.cli DEBUG: Initialised Barman version 2.4 (config: /etc/barman.conf, args: {'reuse_backup': None, 'jobs': None, 'server_name': ['pgcluster'], 'format': 'console', 'quiet': False, 'retry_times': None, 'command': 'backup', 'retry_sleep': None, 'debug': False})
2018-05-28 11:45:46,960 [32434] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup
2018-05-28 11:45:46,960 [32434] barman.server DEBUG: Retention policy for server pgcluster: RECOVERY WINDOW OF 7 DAYS
2018-05-28 11:45:46,960 [32434] barman.server DEBUG: WAL retention policy for server pgcluster: MAIN
2018-05-28 11:45:46,960 [32434] barman.server DEBUG: Starting check: WAL archive
2018-05-28 11:45:46,961 [32434] barman.server DEBUG: Starting check: empty incoming directory
2018-05-28 11:45:46,961 [32434] barman.server DEBUG: Starting check: empty streaming directory
2018-05-28 11:45:46,961 [32434] barman.server DEBUG: Starting check: PostgreSQL
2018-05-28 11:45:47,005 [32434] barman.command_wrappers DEBUG: Command: ['/usr/bin/pg_receivewal', '--version']
2018-05-28 11:45:47,056 [32434] barman.command_wrappers DEBUG: Command return code: 0
2018-05-28 11:45:47,057 [32434] barman.command_wrappers DEBUG: Command stdout: pg_receivewal (PostgreSQL) 10.4 (Debian 10.4-2.pgdg90+1)
2018-05-28 11:45:47,057 [32434] barman.command_wrappers DEBUG: Command stderr: could not change directory to "/root": Keine Berechtigung
2018-05-28 11:45:47,060 [32434] barman.wal_archiver DEBUG: Look for 'barman_receive_wal' in 'synchronous_standby_names': ['']
2018-05-28 11:45:47,061 [32434] barman.wal_archiver DEBUG: Synchronous WAL streaming for barman_receive_wal: False
2018-05-28 11:45:47,061 [32434] barman.command_wrappers DEBUG: Command: ['/usr/bin/pg_basebackup', '--version']
2018-05-28 11:45:47,123 [32434] barman.command_wrappers DEBUG: Command return code: 0
2018-05-28 11:45:47,123 [32434] barman.command_wrappers DEBUG: Command stdout: pg_basebackup (PostgreSQL) 10.4 (Debian 10.4-2.pgdg90+1)
2018-05-28 11:45:47,123 [32434] barman.command_wrappers DEBUG: Command stderr: could not change directory to "/root": Keine Berechtigung
2018-05-28 11:45:47,124 [32434] barman.server DEBUG: Check 'PostgreSQL' succeeded for server 'pgcluster'
2018-05-28 11:45:47,125 [32434] barman.server DEBUG: Starting check: is_superuser
2018-05-28 11:45:47,125 [32434] barman.server DEBUG: Check 'is_superuser' succeeded for server 'pgcluster'
2018-05-28 11:45:47,125 [32434] barman.server DEBUG: Starting check: PostgreSQL streaming
2018-05-28 11:45:47,125 [32434] barman.server DEBUG: Check 'PostgreSQL streaming' succeeded for server 'pgcluster'
2018-05-28 11:45:47,125 [32434] barman.server DEBUG: Starting check: wal_level
2018-05-28 11:45:47,125 [32434] barman.server DEBUG: Check 'wal_level' succeeded for server 'pgcluster'
2018-05-28 11:45:47,125 [32434] barman.server DEBUG: Starting check: replication slot
2018-05-28 11:45:47,126 [32434] barman.server DEBUG: Check 'replication slot' succeeded for server 'pgcluster'
2018-05-28 11:45:47,126 [32434] barman.server DEBUG: Starting check: directories
2018-05-28 11:45:47,126 [32434] barman.server DEBUG: Check 'directories' succeeded for server 'pgcluster'
2018-05-28 11:45:47,126 [32434] barman.server DEBUG: Starting check: retention policy settings
2018-05-28 11:45:47,126 [32434] barman.server DEBUG: Check 'retention policy settings' succeeded for server 'pgcluster'
2018-05-28 11:45:47,126 [32434] barman.server DEBUG: Starting check: backup maximum age
2018-05-28 11:45:47,127 [32434] barman.server DEBUG: Check 'backup maximum age' succeeded for server 'pgcluster'
2018-05-28 11:45:47,127 [32434] barman.server DEBUG: Starting check: compression settings
2018-05-28 11:45:47,127 [32434] barman.server DEBUG: Check 'compression settings' succeeded for server 'pgcluster'
2018-05-28 11:45:47,127 [32434] barman.server DEBUG: Starting check: failed backups
2018-05-28 11:45:47,130 [32434] barman.server INFO: Ignoring failed check 'failed backups' for server 'pgcluster'
2018-05-28 11:45:47,130 [32434] barman.server DEBUG: Starting check: minimum redundancy requirements
2018-05-28 11:45:47,130 [32434] barman.server DEBUG: Check 'minimum redundancy requirements' succeeded for server 'pgcluster'
2018-05-28 11:45:47,130 [32434] barman.server DEBUG: Starting check: pg_basebackup
2018-05-28 11:45:47,130 [32434] barman.server DEBUG: Check 'pg_basebackup' succeeded for server 'pgcluster'
2018-05-28 11:45:47,130 [32434] barman.server DEBUG: Starting check: pg_basebackup compatible
2018-05-28 11:45:47,130 [32434] barman.server DEBUG: Check 'pg_basebackup compatible' succeeded for server 'pgcluster'
2018-05-28 11:45:47,131 [32434] barman.server DEBUG: Starting check: pg_basebackup supports tablespaces mapping
2018-05-28 11:45:47,133 [32434] barman.server DEBUG: Check 'pg_basebackup supports tablespaces mapping' succeeded for server 'pgcluster'
2018-05-28 11:45:47,134 [32434] barman.server DEBUG: Starting check: configuration
2018-05-28 11:45:47,134 [32434] barman.server DEBUG: Starting check: pg_receivexlog
2018-05-28 11:45:47,134 [32434] barman.server DEBUG: Check 'pg_receivexlog' succeeded for server 'pgcluster'
2018-05-28 11:45:47,134 [32434] barman.server DEBUG: Starting check: pg_receivexlog compatible
2018-05-28 11:45:47,134 [32434] barman.server DEBUG: Check 'pg_receivexlog compatible' succeeded for server 'pgcluster'
2018-05-28 11:45:47,134 [32434] barman.server DEBUG: Starting check: receive-wal running
2018-05-28 11:45:47,134 [32434] barman.server DEBUG: Check 'receive-wal running' succeeded for server 'pgcluster'
2018-05-28 11:45:47,134 [32434] barman.server DEBUG: Starting check: archiver errors
2018-05-28 11:45:47,135 [32434] barman.server DEBUG: Check 'archiver errors' succeeded for server 'pgcluster'
2018-05-28 11:45:47,135 [32434] barman.backup DEBUG: initialising backup information
2018-05-28 11:45:47,139 [32434] barman.backup INFO: Starting backup using postgres method for server pgcluster in /var/lib/barman/pgcluster/base/20180528T114547
2018-05-28 11:45:47,141 [32434] barman.backup_executor DEBUG: detecting data directory
2018-05-28 11:45:47,146 [32434] barman.backup_executor DEBUG: detecting tablespaces
2018-05-28 11:45:47,147 [32434] barman.backup_executor DEBUG: issuing start backup command
2018-05-28 11:45:47,150 [32434] barman.backup_executor INFO: Backup start at LSN: 17/50000140 (000000010000001700000050, 00000140)
2018-05-28 11:45:47,152 [32434] barman.backup_executor INFO: Starting backup copy via pg_basebackup for 20180528T114547
2018-05-28 11:45:47,153 [32434] barman.command_wrappers DEBUG: Command: ['/usr/bin/pg_basebackup', '--dbname=dbname=replication host=pgcluster options=-cdatestyle=iso replication=true user=postgres application_name=barman_streaming_backup', '-v', '--no-password', '--pgdata=/var/lib/barman/pgcluster/base/20180528T114547/data', '--no-slot', '--wal-method=none']
2018-05-28 11:48:33,543 [32434] barman.command_wrappers DEBUG: Command return code: 0
2018-05-28 11:48:33,545 [32434] barman.backup ERROR: Backup failed copying files.
DETAILS: 'ascii' codec can't encode character u'\xf6' in position 216: ordinal not in range(128)

cat /var/lib/barman/bareos/base/20180528T114547/backup.info 
backup_label=None
begin_offset=320
begin_time=2018-05-28 11:45:47.144933+02:00
begin_wal=000000010000001700000050
begin_xlog=17/50000140
config_file=/etc/postgresql/10/main/postgresql.conf
copy_stats=None
deduplicated_size=None
end_offset=None
end_time=None
end_wal=None
end_xlog=None
error=failure copying files ('ascii' codec can't encode character u'\xf6' in position 216: ordinal not in range(128))
hba_file=/etc/postgresql/10/main/pg_hba.conf
ident_file=/etc/postgresql/10/main/pg_ident.conf
included_files=None
mode=postgres
pgdata=/var/lib/postgresql/10/main
server_name=bareos
size=None
status=FAILED
tablespaces=None
timeline=1
version=100004
xlog_segment_size=16777216

 barman diagnose
{
    "global": {
        "config": {
            "barman_home": "/var/lib/barman", 
            "barman_user": "barman", 
            "configuration_files_directory": "/etc/barman.d", 
            "errors_list": [], 
            "log_file": "/var/log/barman/barman.log", 
            "log_level": "DEBUG"
        }, 
        "system_info": {
            "barman_ver": "2.4", 
            "kernel_ver": "Linux barman 4.16.0-0.bpo.1-amd64 #1 SMP Debian 4.16.5-1~bpo9+1 (2018-05-06) x86_64 GNU/Linux", 
            "python_ver": "", 
            "release": "Distributor ID:\tDebian\nDescription:\tDebian GNU/Linux 9.4 (stretch)\nRelease:\t9.4\nCodename:\tstretch", 
            "rsync_ver": "rsync  version 3.1.2  protocol version 31", 
            "ssh_ver": ""
        }
    }, 
    "servers": {
        "pgcluster": {
            "backups": {
                "20180526T225302": {
                    "backup_id": "20180526T225302", 
                    "backup_label": null, 
                    "begin_offset": 40, 
                    "begin_time": "Sat May 26 22:53:03 2018", 
                    "begin_wal": "0000000100000016000000B9", 
                    "begin_xlog": "16/B9000028", 
                    "config_file": "/etc/postgresql/10/main/postgresql.conf", 
                    "copy_stats": {
                        "copy_time": 163.11997, 
                        "total_time": 163.11997
                    }, 
                    "deduplicated_size": 18417589383, 
                    "end_offset": 0, 
                    "end_time": "Sat May 26 22:55:45 2018", 
                    "end_wal": "0000000100000016000000B9", 
                    "end_xlog": "16/BA000000", 
                    "error": null, 
                    "hba_file": "/etc/postgresql/10/main/pg_hba.conf", 
                    "ident_file": "/etc/postgresql/10/main/pg_ident.conf", 
                    "included_files": null, 
                    "mode": "postgres", 
                    "pgdata": "/var/lib/postgresql/10/main", 
                    "server_name": "pgcluster", 
                    "size": 18417589383, 
                    "status": "DONE", 
                    "tablespaces": null, 
                    "timeline": 1, 
                    "version": 100004, 
                    "xlog_segment_size": 16777216
                }, 
                "20180528T112307": {
                    "backup_id": "20180528T112307", 
                    "backup_label": null, 
                    "begin_offset": 10348264, 
                    "begin_time": "Mon May 28 11:23:07 2018", 
                    "begin_wal": "00000001000000170000004E", 
                    "begin_xlog": "17/4E9DE6E8", 
                    "config_file": "/etc/postgresql/10/main/postgresql.conf", 
                    "copy_stats": null, 
                    "deduplicated_size": null, 
                    "end_offset": null, 
                    "end_time": null, 
                    "end_wal": null, 
                    "end_xlog": null, 
                    "error": "failure copying files ('ascii' codec can't encode character u'\\xf6' in position 216: ordinal not in range(128))", 
                    "hba_file": "/etc/postgresql/10/main/pg_hba.conf", 
                    "ident_file": "/etc/postgresql/10/main/pg_ident.conf", 
                    "included_files": null, 
                    "mode": "postgres", 
                    "pgdata": "/var/lib/postgresql/10/main", 
                    "server_name": "pgcluster", 
                    "size": null, 
                    "status": "FAILED", 
                    "tablespaces": null, 
                    "timeline": 1, 
                    "version": 100004, 
                    "xlog_segment_size": 16777216
                }, 
                "20180528T114547": {
                    "backup_id": "20180528T114547", 
                    "backup_label": null, 
                    "begin_offset": 320, 
                    "begin_time": "Mon May 28 11:45:47 2018", 
                    "begin_wal": "000000010000001700000050", 
                    "begin_xlog": "17/50000140", 
                    "config_file": "/etc/postgresql/10/main/postgresql.conf", 
                    "copy_stats": null, 
                    "deduplicated_size": null, 
                    "end_offset": null, 
                    "end_time": null, 
                    "end_wal": null, 
                    "end_xlog": null, 
                    "error": "failure copying files ('ascii' codec can't encode character u'\\xf6' in position 216: ordinal not in range(128))", 
                    "hba_file": "/etc/postgresql/10/main/pg_hba.conf", 
                    "ident_file": "/etc/postgresql/10/main/pg_ident.conf", 
                    "included_files": null, 
                    "mode": "postgres", 
                    "pgdata": "/var/lib/postgresql/10/main", 
                    "server_name": "pgcluster", 
                    "size": null, 
                    "status": "FAILED", 
                    "tablespaces": null, 
                    "timeline": 1, 
                    "version": 100004, 
                    "xlog_segment_size": 16777216
                }
            }, 
            "config": {
                "active": true, 
                "archiver": false, 
                "archiver_batch_size": 0, 
                "backup_directory": "/var/lib/barman/pgcluster", 
                "backup_method": "postgres", 
                "backup_options": "concurrent_backup", 
                "bandwidth_limit": null, 
                "barman_home": "/var/lib/barman", 
                "barman_lock_directory": "/var/lib/barman", 
                "basebackup_retry_sleep": 30, 
                "basebackup_retry_times": 0, 
                "basebackups_directory": "/var/lib/barman/pgcluster/base", 
                "check_timeout": 30, 
                "compression": null, 
                "conninfo": "host=pgcluster user=postgres dbname=postgres", 
                "custom_compression_filter": null, 
                "custom_decompression_filter": null, 
                "description": "PostgreSQL Database on pgcluster", 
                "disabled": false, 
                "errors_directory": "/var/lib/barman/pgcluster/errors", 
                "immediate_checkpoint": false, 
                "incoming_wals_directory": "/var/lib/barman/pgcluster/incoming", 
                "last_backup_maximum_age": null, 
                "max_incoming_wals_queue": null, 
                "minimum_redundancy": 0, 
                "msg_list": [], 
                "name": "pgcluster", 
                "network_compression": false, 
                "parallel_jobs": 1, 
                "path_prefix": null, 
                "post_archive_retry_script": null, 
                "post_archive_script": null, 
                "post_backup_retry_script": null, 
                "post_backup_script": null, 
                "post_delete_retry_script": null, 
                "post_delete_script": null, 
                "post_recovery_retry_script": null, 
                "post_recovery_script": null, 
                "post_wal_delete_retry_script": null, 
                "post_wal_delete_script": null, 
                "pre_archive_retry_script": null, 
                "pre_archive_script": null, 
                "pre_backup_retry_script": null, 
                "pre_backup_script": null, 
                "pre_delete_retry_script": null, 
                "pre_delete_script": null, 
                "pre_recovery_retry_script": null, 
                "pre_recovery_script": null, 
                "pre_wal_delete_retry_script": null, 
                "pre_wal_delete_script": null, 
                "recovery_options": "", 
                "retention_policy": "window 7 d", 
                "retention_policy_mode": "auto", 
                "reuse_backup": null, 
                "slot_name": "barman", 
                "ssh_command": "ssh postgres@pgcluster", 
                "streaming_archiver": true, 
                "streaming_archiver_batch_size": 0, 
                "streaming_archiver_name": "barman_receive_wal", 
                "streaming_backup_name": "barman_streaming_backup", 
                "streaming_conninfo": "host=pgcluster user=postgres dbname=postgres", 
                "streaming_wals_directory": "/var/lib/barman/pgcluster/streaming", 
                "tablespace_bandwidth_limit": null, 
                "wal_retention_policy": "simple-wal 7 d", 
                "wals_directory": "/var/lib/barman/pgcluster/wals"
            }, 
            "status": {
                "config_file": "/etc/postgresql/10/main/postgresql.conf", 
                "connection_error": null, 
                "current_size": 18418710804.0, 
                "current_xlog": "000000010000001700000052", 
                "data_directory": "/var/lib/postgresql/10/main", 
                "hba_file": "/etc/postgresql/10/main/pg_hba.conf", 
                "ident_file": "/etc/postgresql/10/main/pg_ident.conf", 
                "is_in_recovery": false, 
                "is_superuser": true, 
                "pg_basebackup_bwlimit": true, 
                "pg_basebackup_compatible": true, 
                "pg_basebackup_installed": true, 
                "pg_basebackup_path": "/usr/bin/pg_basebackup", 
                "pg_basebackup_tbls_mapping": true, 
                "pg_basebackup_version": "10.4-2.pgdg90+1)", 
                "pg_receivexlog_compatible": true, 
                "pg_receivexlog_installed": true, 
                "pg_receivexlog_path": "/usr/bin/pg_receivewal", 
                "pg_receivexlog_supports_slots": true, 
                "pg_receivexlog_synchronous": false, 
                "pg_receivexlog_version": "10.4-2.pgdg90+1)", 
                "pgespresso_installed": false, 
                "replication_slot": [
                    "barman", 
                    true, 
                    "17/52000000"
                ], 
                "replication_slot_support": true, 
                "server_txt_version": "10.4", 
                "streaming": true, 
                "streaming_supported": true, 
                "synchronous_standby_names": [
                    ""
                ], 
                "systemid": "6536089255217622081", 
                "timeline": 1, 
                "wal_level": "replica", 
                "xlogpos": "17/52000140"
            }, 
            "wals": {
                "last_archived_wal_per_timeline": {
                    "00000001": {
                        "compression": null, 
                        "name": "000000010000001700000051", 
                        "size": 16777216, 
                        "time": 1527500912.2586727
                    }
                }
            }
        },

and:

barman check pgcluster
Server pgcluster:
	PostgreSQL: OK
	is_superuser: OK
	PostgreSQL streaming: OK
	wal_level: OK
	replication slot: OK
	directories: OK
	retention policy settings: OK
	backup maximum age: OK (no last_backup_maximum_age provided)
	compression settings: OK
	failed backups: FAILED (there are 2 failed backups)
	minimum redundancy requirements: OK (have 1 backups, expected at least 0)
	pg_basebackup: OK
	pg_basebackup compatible: OK
	pg_basebackup supports tablespaces mapping: OK
	pg_receivexlog: OK
	pg_receivexlog compatible: OK
	receive-wal running: OK
	archiver errors: OK

Any idea what is wrong? Or any hints? Thanks & Greetings, Michael

bug

opened by mwuttke 15

wal_keep_segments breaking change to wal_keep_size in PG 13

Hi,

i gave a shot to postgres 13 with barman 2.11, and got

2020-09-26 01:24:13.343 CEST [604692] barman@postgres ERROR:  unrecognized configuration parameter "wal_keep_segments"
2020-09-26 01:24:13.343 CEST [604692] barman@postgres STATEMENT:  SHOW "wal_keep_segments"
2020-09-26 01:24:13.344 CEST [604692] barman@postgres ERROR:  current transaction is aborted, commands ignored until end of transaction block
2020-09-26 01:24:13.344 CEST [604692] barman@postgres STATEMENT:  SELECT 1

Cheers !

opened by kapouer 14

barman recover Failure copying base backup: data transfer failure

Hello. I have problem when i use barman recover command. I delete all files and folders from recovery target area. In de target area; DATA Directory is empty or not existing. However during recover process tablespaces and directories was created in target, and they're tried to recreate again. Then gives errror.

In target area, I did: -bash-4.2$ cd data/ -bash-4.2$ ls tablespace1_tbl tablespace2_tbl tablespace3_tbl PG_13_202205021 tablespace4_tbl -bash-4.2$ rm -rf * -bash-4.2$ ls -alh total 0 drwx------. 2 postgres postgres 6 May 26 09:35 . drwxr-xr-x. 4 postgres postgres 38 May 26 09:34 ..

then my command barman recover is started to work in barman server: -bash-4.2$ barman recover --target-time="2022-05-23 16:01:00" mydb 20220519T023001 /data/ --remote-ssh-command "ssh [email protected]" Starting remote restore for server mydb using backup 20220519T023001 Destination directory: /data/ Remote command: ssh [email protected] Doing PITR. Recovery target time: '2022-05-23 16:01:00+03:00' 12375, tablespace1_tbl, /data/tablespace1_tbl 12376, tablespace2_tbl, /data/tablespace2_tbl 12377, tablespace3_tbl, /data/tablespace3_tbl Copying the base backup. ERROR: Failure copying base backup: data transfer failure rsync error: rsync: mkdir "/data/tablespace2_tbl" failed: File exists (17) rsync error: error in file IO (code 11) at main.c(657) [Receiver=3.1.2]

To bypass this error, during the recovery, when i see 'Copying the base backup.' result, i delete tablespace1_tbl from recovery area and bypassed, continue to recover. However, i can not understand why i should delete it during recover. Moreover, after recover, i can not see any links on pg_tblspc folder in DATA. I have to link them by myself... Is there any way to declare like 'append' or 'replace' and link tablespaces in pg_tblspc automatically? Thanks
triage

opened by ahmetmelihbasbug 12

Backup fails using rsync method

I am having problems using the rsync method. The copy is completing but the backup does not finalise.

-bash-4.2$ barman backup db-16
Starting backup using rsync-concurrent method for server db-16 in /backup/barman/db-16/base/20180425T154747
Backup start at LSN: 62/3E408028 (00000001000000620000003E, 00408028)
This is the first backup for server db-16
Starting backup copy via rsync/SSH for 20180425T154747
Copy done (time: 2 hours, 58 minutes, 37 seconds)
This is the first backup for server db-16
Asking PostgreSQL server to finalize the backup.
ERROR: Backup failed issuing start backup command.
DETAILS: Connection lost, reconnection not allowed

Processing xlog segments from streaming for db-16

Our database server runs on FreeBSD and barman server on CentOS. This particular scenario only occurs when backing up a database server in a different data centre - we have been running Barman backups successfully when the backup server is in the same DC as the database, and we are able to complete backups between sites using the postgres method.

Any advice/suggestions would be appreciated.

Thanks,

Mark

bug

opened by markferrari 12

PostgreSQL Data directory: /var/lib/postgresql/data but postgresql in a container

Hello,

I have two machines pg-host and barman-host. My postgres is running in a docker container. Everything seems to work except the WAL shipping.

Barman version 3.2.0.

When running, $barman-host: barman check pghost everything is working properly except WAL archive: FAILED (please make sure WAL shipping is setup).

My postgres database runs in a docker container, and the following command works fine on the host machine: $pg-host: barman-wal-archive IP hostname mountpath/pg_wal/000000000000000EXAMPLE> ✅

My archive_command is : archive_command = 'barman-wal-archive barman-wal-archive IP hostname mountpath/%p' However when I run $barman-host: barman switch-wal --archive --archive-timeout 60 pghost I obtain : ERROR: The WAL file 000000000000000EXAMPLE has not been received in 60 seconds

The only strange line that I have found so far is PostgreSQL Data directory: /var/lib/postgresql/data when running : $barman-host: barman status pghost. This path works only in the container. Do you have idea if this path is used ? can I modify it such that I can use the mounted path ?

Many thanks,
triage

opened by BoltMaud 10
Building python3 based packages for RHEL7 fails

Hi, Looks like something is going wrong when trying to build RPMS with python 3 on RHEL7. I'm running 'python3 setup.py bdest_rpm'. Stuff seems to be going well, but it goes wrong when handling the man pages. It copies them to the right directory, but then it runs /usr/lib/rpm/redhat/brp-compress which compresses them. During the next step it fails, because the files are now called for example barman.1.gz instead of barman.1. So then I get a bunch of error messages like this: error: File not found: /home/vagrant/barman-2.12/build/bdist.linux-x86_64/rpm/BUILDROOT/barman-2.12-1.x86_64/usr/share/man/man1/barman.1 (and the same for the other man pages) And then it stops: error: command 'rpmbuild' failed with exit status 1 So no rpms :( Is there a way I can fix this? Or is this a bug in the package? Thanks!

Cheers, Guus
packaging

opened by GuusHoutzager 10

ERROR: Unexpected output from "barman list-files"

Ubuntu 16.04
Kernel 4.4.0-75-generic #96-Ubuntu SMP Thu Apr 20 09:56:33 UTC 2017 x86_64
Python 2.7.12
Barman:

$ apt-cache show barman
Package: barman
Version: 2.1-1.pgdg16.04+1
Architecture: all
Maintainer: Marco Nenciarini <[email protected]>
Installed-Size: 660
Depends: python-argcomplete, python-argh, python-dateutil, python-psycopg2, python:any (<< 2.8), python:any (>= 2.7.5-5~), adduser, rsync (>= 3.0.4~)
Recommends: openssh-server, openssh-client, postgresql-client | postgresql-client-9.6 | postgresql-client-9.5 | postgresql-client-9.4 | postgresql-client-9.3 | postgresql-client-9.2
Suggests: barman-cli, repmgr (>= 3.2.0)
Homepage: http://www.pgbarman.org
Priority: extra
Section: database
Filename: pool/main/b/barman/barman_2.1-1.pgdg16.04+1_all.deb
Size: 118930
SHA256: 16a0c0ee70762415c4bd9d8e1208b899116ba9bfd7d71e8330c7484324cd344f
SHA1: 6aa2ea835649ab56c972d9111534f5c7964d012b
MD5sum: 85a3c6ee9f94751bcb29dd76fec187c3
Description: Backup and Recovery Manager for PostgreSQL
 Barman (Backup and Recovery Manager) is an open-source
 administration tool for disaster recovery of PostgreSQL
 servers written in Python.
 .
 It allows your organisation to perform remote backups of
 multiple servers in business critical environments to
 reduce risk and help DBAs during the recovery phase.
 .
 Barman is distributed under GNU GPL 3 and maintained
 by 2ndQuadrant.
Description-md5: 3a5b57842a831067f70d5ac6c7b3c460

Command used: repmgr -verbose -h <postgres_master_ip> -U repmgr -d repmgr -D /var/lib/postgresql/9.6/main standby clone

Output:

NOTICE: looking for configuration file in current directory
NOTICE: looking for configuration file in /etc
NOTICE: configuration file found at: /etc/repmgr.conf
NOTICE: destination directory '/var/lib/postgresql/9.6/main' provided
NOTICE: getting backup from Barman...
ERROR: Unexpected output from "barman list-files": /mnt/extra/barman/review/basebackups/20170530T160208/backup.info

postgres@pg-cluster-1:~/9.6/main$ EXCEPTION: [Errno 32] Broken pipe
See log file for more details.

Error in barman.log:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/barman/cli.py", line 1041, in main
    p.dispatch(pre_call=global_config)
  File "/usr/lib/python2.7/dist-packages/argh/helpers.py", line 55, in dispatch
    return dispatch(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/argh/dispatching.py", line 174, in dispatch
    for line in lines:
  File "/usr/lib/python2.7/dist-packages/argh/dispatching.py", line 277, in _execute_command
    for line in result:
  File "/usr/lib/python2.7/dist-packages/argh/dispatching.py", line 231, in _call
    result = function(namespace_obj)
  File "/usr/lib/python2.7/dist-packages/barman/cli.py", line 580, in list_files
    output.info(line, log=False)
  File "/usr/lib/python2.7/dist-packages/barman/output.py", line 166, in info
    _put('info', message, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/barman/output.py", line 97, in _put
    getattr(_writer, level)(message, *args)
  File "/usr/lib/python2.7/dist-packages/barman/output.py", line 368, in info
    self._out(message, args)
  File "/usr/lib/python2.7/dist-packages/barman/output.py", line 332, in _out
    print(_format_message(message, args), file=sys.stdout)

Content of /mnt/extra/barman/review/basebackups/20170530T160208/backup.info

backup_label=None
begin_offset=40
begin_time=2017-05-30 16:02:08+00:00
begin_wal=0000000200000037000000EC
begin_xlog=37/EC000028
config_file=/etc/postgresql/9.6/main/postgresql.conf
deduplicated_size=58211135370
end_offset=96
end_time=2017-05-30 16:02:08.352762+00:00
end_wal=0000000200000037000000F2
end_xlog=37/F2000060
error=None
hba_file=/etc/postgresql/9.6/main/pg_hba.conf
ident_file=/etc/postgresql/9.6/main/pg_ident.conf
included_files=['/etc/postgresql/9.6/main/postgresql.effilab.conf']
mode=postgres
pgdata=/var/lib/postgresql/9.6/main
server_name=review
size=58211135370
status=DONE
tablespaces=None
timeline=2
version=90602

I fixed the error by updating manually with pip all my python modules.

Before:

appdirs (1.4.3)
argcomplete (1.0.0)
argh (0.26.1)
barman (2.1)
barman-cli (1.2)
packaging (16.8)
pip (9.0.1)
psycopg2 (2.6.1)
pyparsing (2.2.0)
python-apt (1.1.0b1)
python-dateutil (2.4.2)
setuptools (35.0.2)
six (1.10.0)
wheel (0.29.0)

After:

appdirs (1.4.3)
argcomplete (1.8.2)
argh (0.26.2)
barman (2.1)
barman-cli (1.2)
packaging (16.8)
pip (9.0.1)
psycopg2 (2.7.1)
pyparsing (2.2.0)
python-apt (1.1.0b1)
python-dateutil (2.6.0)
setuptools (35.0.2)
six (1.10.0)
wheel (0.29.0)

I didn't take time to exactly point out which module needed to be updated sorry, but I think there might be a missing python module dependency ?

moreinfo

opened by MaesterZ 10

Improve `barman-cloud` Google Storage support with the `kms_key_name` to be used to encrypt objects

It would be great if both barman-cloud-wal-archive and barman-cloud-backup would have the option to specify also the kms_key_name value that we would like to use to encrypt the objects uploaded into the Google Storage buckets. This would allow using the same Google Storage bucket to save backups and WALs belonging to multiple Postgres instances, each one using a different KMS CMEK.

This would be useful for security reasons where we want to share a Google Storage bucket with multiple Postgres instances, being able to manage encryption keys on a per Postgres instance basis, without impacting multiple instances at the same time.

The same can currently be achieved on Azure using Storage Accounts and the already provided --encryption-scope option, which allows to use different CMKs on different Postgres instances saving backups and WALs in the same Storage Account.

This is somewhat related to what has been asked also for AWS S3 with #714

opened by valeriodelsarto 0
Check WAL shipping setup only in archiver mode

Setups with only streaming_archiver enabled will fail this check. This blocks "backup" operation, while receive-wal works.

This appears to be previously reported issue, for example see https://github.com/EnterpriseDB/barman/issues/298

This bug may be explained by the fact previously enabling both streaming_archiver and archiver (WAL shipping) was the recommended configuration, per documentation. In such configuration this check would pass.

opened by andrey-utkin 2
barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down.

Hi, My barman test environment, version 3.3.0, has these recurring messages in the logfile: barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down.

Seems to be same problem as reported in #652, but that should have been fixed in version 3.1.0 ?

Log with INFO level: 2022-12-20 13:28:02,002 [14899] barman.cli INFO: Skipping inactive server 'trinidad_14' 2022-12-20 13:28:02,061 [14899] barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down. 2022-12-20 13:28:02,317 [15002] barman.wal_archiver INFO: No xlog segments found from streaming for gibraltar_14. 2022-12-20 13:28:02,317 [15002] barman.wal_archiver INFO: No xlog segments found from file archival for gibraltar_14. 2022-12-20 13:28:02,419 [15003] barman.wal_archiver INFO: No xlog segments found from streaming for trinidad_15. 2022-12-20 13:28:02,419 [15003] barman.wal_archiver INFO: No xlog segments found from file archival for trinidad_15. 2022-12-20 13:28:03,182 [15014] barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down.

Log with DEBUG level: 2022-12-20 13:00:02,114 [13399] barman.config DEBUG: Including configuration file: gibraltar_14.conf 2022-12-20 13:00:02,115 [13399] barman.config DEBUG: Including configuration file: trinidad_14.conf 2022-12-20 13:00:02,115 [13399] barman.config DEBUG: Including configuration file: trinidad_15.conf 2022-12-20 13:00:02,116 [13399] barman.cli DEBUG: Initialised Barman version 3.3.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': False, 'debug': False, 'format': 'console', 'command': 'list-servers', 'minimal': False, 'func ': <function list_servers at 0x7fcfe0cdb950>}) 2022-12-20 13:00:02,129 [13399] barman.server DEBUG: Retention policy for server gibraltar_14: RECOVERY WINDOW OF 5 DAYS 2022-12-20 13:00:02,129 [13399] barman.server DEBUG: WAL retention policy for server gibraltar_14: MAIN 2022-12-20 13:00:02,141 [13399] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup 2022-12-20 13:00:02,141 [13399] barman.server DEBUG: Retention policy for server trinidad_14: RECOVERY WINDOW OF 3 DAYS 2022-12-20 13:00:02,142 [13399] barman.server DEBUG: WAL retention policy for server trinidad_14: MAIN 2022-12-20 13:00:02,142 [13398] barman.config DEBUG: Including configuration file: gibraltar_14.conf 2022-12-20 13:00:02,142 [13399] root DEBUG: Created a UnixLocalCommand 2022-12-20 13:00:02,142 [13398] barman.config DEBUG: Including configuration file: trinidad_14.conf 2022-12-20 13:00:02,143 [13399] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup 2022-12-20 13:00:02,143 [13398] barman.config DEBUG: Including configuration file: trinidad_15.conf 2022-12-20 13:00:02,143 [13399] barman.command_wrappers DEBUG: Command: ['/usr/pgsql-15/bin/pg_basebackup', '--version'] 2022-12-20 13:00:02,143 [13398] barman.cli DEBUG: Initialised Barman version 3.3.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': False, 'debug': False, 'format': 'console', 'command': 'cron', 'keep_descriptors': False, 'fun c': <function cron at 0x7f284f567a60>}) 2022-12-20 13:00:02,149 [13399] Command DEBUG: pg_basebackup (PostgreSQL) 15.1 2022-12-20 13:00:02,150 [13399] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:02,150 [13399] barman.command_wrappers DEBUG: Command stdout: pg_basebackup (PostgreSQL) 15.1

2022-12-20 13:00:02,150 [13399] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:02,156 [13398] barman.server DEBUG: Retention policy for server gibraltar_14: RECOVERY WINDOW OF 5 DAYS 2022-12-20 13:00:02,156 [13398] barman.server DEBUG: WAL retention policy for server gibraltar_14: MAIN 2022-12-20 13:00:02,169 [13398] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup 2022-12-20 13:00:02,169 [13398] barman.server DEBUG: Retention policy for server trinidad_14: RECOVERY WINDOW OF 3 DAYS 2022-12-20 13:00:02,170 [13398] barman.server DEBUG: WAL retention policy for server trinidad_14: MAIN 2022-12-20 13:00:02,170 [13398] barman.cli INFO: Skipping inactive server 'trinidad_14' 2022-12-20 13:00:02,171 [13398] root DEBUG: Created a UnixLocalCommand 2022-12-20 13:00:02,172 [13398] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup 2022-12-20 13:00:02,172 [13398] barman.command_wrappers DEBUG: Command: ['/usr/pgsql-15/bin/pg_basebackup', '--version'] 2022-12-20 13:00:02,179 [13398] Command DEBUG: pg_basebackup (PostgreSQL) 15.1 2022-12-20 13:00:02,179 [13398] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:02,180 [13398] barman.command_wrappers DEBUG: Command stdout: pg_basebackup (PostgreSQL) 15.1

2022-12-20 13:00:02,180 [13398] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:02,195 [13399] barman.server DEBUG: Retention policy for server trinidad_15: RECOVERY WINDOW OF 3 DAYS 2022-12-20 13:00:02,195 [13399] barman.server DEBUG: WAL retention policy for server trinidad_15: MAIN 2022-12-20 13:00:02,196 [13399] barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down. 2022-12-20 13:00:02,228 [13398] barman.server DEBUG: Retention policy for server trinidad_15: RECOVERY WINDOW OF 3 DAYS 2022-12-20 13:00:02,228 [13398] barman.server DEBUG: WAL retention policy for server trinidad_15: MAIN 2022-12-20 13:00:02,228 [13398] barman.command_wrappers DEBUG: BarmanSubProcess: ['/usr/bin/python3', '/usr/bin/barman', '-c', '/etc/barman.conf', '-q', 'archive-wal', 'gibraltar_14'] 2022-12-20 13:00:02,231 [13398] barman.command_wrappers DEBUG: BarmanSubProcess: subprocess started. pid: 13404 2022-12-20 13:00:02,231 [13398] barman.server DEBUG: Another STREAMING ARCHIVER process is running for server gibraltar_14 2022-12-20 13:00:02,237 [13398] barman.retention_policies DEBUG: Reporting backup 20221219T200002 for server gibraltar_14 as VALID (newer than 2022-12-15 13:00:02.237609+01:00) 2022-12-20 13:00:02,240 [13398] barman.retention_policies DEBUG: Reporting backup 20221218T200002 for server gibraltar_14 as VALID (newer than 2022-12-15 13:00:02.240689+01:00) 2022-12-20 13:00:02,241 [13398] barman.retention_policies DEBUG: Reporting backup 20221217T200002 for server gibraltar_14 as VALID (newer than 2022-12-15 13:00:02.240993+01:00) 2022-12-20 13:00:02,241 [13398] barman.retention_policies DEBUG: Reporting backup 20221216T200003 for server gibraltar_14 as VALID (newer than 2022-12-15 13:00:02.241202+01:00) 2022-12-20 13:00:02,241 [13398] barman.retention_policies DEBUG: Reporting backup 20221215T200002 for server gibraltar_14 as VALID (newer than 2022-12-15 13:00:02.241405+01:00) 2022-12-20 13:00:02,241 [13398] barman.retention_policies DEBUG: Reporting backup 20221214T200002 for server gibraltar_14 as VALID (newer than 2022-12-15 13:00:02.241622+01:00) 2022-12-20 13:00:02,242 [13398] barman.command_wrappers DEBUG: BarmanSubProcess: ['/usr/bin/python3', '/usr/bin/barman', '-c', '/etc/barman.conf', '-q', 'archive-wal', 'trinidad_15'] 2022-12-20 13:00:02,252 [13398] barman.command_wrappers DEBUG: BarmanSubProcess: subprocess started. pid: 13405 2022-12-20 13:00:02,253 [13398] barman.server DEBUG: Another STREAMING ARCHIVER process is running for server trinidad_15 2022-12-20 13:00:02,256 [13398] barman.retention_policies DEBUG: Reporting backup 20221220T104329 for server trinidad_15 as VALID (newer than 2022-12-17 13:00:02.256645+01:00) 2022-12-20 13:00:02,257 [13398] barman.retention_policies DEBUG: Reporting backup 20221219T200017 for server trinidad_15 as VALID (newer than 2022-12-17 13:00:02.256973+01:00) 2022-12-20 13:00:02,257 [13398] barman.retention_policies DEBUG: Reporting backup 20221218T200017 for server trinidad_15 as VALID (newer than 2022-12-17 13:00:02.257159+01:00) 2022-12-20 13:00:02,257 [13398] barman.retention_policies DEBUG: Reporting backup 20221217T200017 for server trinidad_15 as VALID (newer than 2022-12-17 13:00:02.257331+01:00) 2022-12-20 13:00:02,257 [13398] barman.retention_policies DEBUG: Reporting backup 20221216T200018 for server trinidad_15 as VALID (newer than 2022-12-17 13:00:02.257532+01:00) 2022-12-20 13:00:02,257 [13398] barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down. 2022-12-20 13:00:02,783 [13405] barman.config DEBUG: Including configuration file: gibraltar_14.conf 2022-12-20 13:00:02,789 [13405] barman.config DEBUG: Including configuration file: trinidad_14.conf 2022-12-20 13:00:02,789 [13405] barman.config DEBUG: Including configuration file: trinidad_15.conf 2022-12-20 13:00:02,790 [13405] barman.cli DEBUG: Initialised Barman version 3.3.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': True, 'debug': False, 'format': 'console', 'command': 'archive-wal', 'config': '/etc/barman.co nf', 'server_name': 'trinidad_15', 'func': <function archive_wal at 0x7f0271a290d0>}) 2022-12-20 13:00:02,806 [13404] barman.config DEBUG: Including configuration file: gibraltar_14.conf 2022-12-20 13:00:02,812 [13404] barman.config DEBUG: Including configuration file: trinidad_14.conf 2022-12-20 13:00:02,813 [13404] barman.config DEBUG: Including configuration file: trinidad_15.conf 2022-12-20 13:00:02,813 [13404] barman.cli DEBUG: Initialised Barman version 3.3.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': True, 'debug': False, 'format': 'console', 'command': 'archive-wal', 'config': '/etc/barman.co nf', 'server_name': 'gibraltar_14', 'func': <function archive_wal at 0x7fdeb5d180d0>}) 2022-12-20 13:00:02,814 [13405] root DEBUG: Created a UnixLocalCommand 2022-12-20 13:00:02,832 [13404] barman.server DEBUG: Retention policy for server gibraltar_14: RECOVERY WINDOW OF 5 DAYS 2022-12-20 13:00:02,833 [13404] barman.server DEBUG: WAL retention policy for server gibraltar_14: MAIN 2022-12-20 13:00:02,835 [13405] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup 2022-12-20 13:00:02,840 [13405] barman.command_wrappers DEBUG: Command: ['/usr/pgsql-15/bin/pg_basebackup', '--version'] 2022-12-20 13:00:02,853 [13405] Command DEBUG: pg_basebackup (PostgreSQL) 15.1 2022-12-20 13:00:02,854 [13404] barman.server DEBUG: Starting archive-wal for server gibraltar_14 2022-12-20 13:00:02,856 [13404] barman.wal_archiver INFO: No xlog segments found from streaming for gibraltar_14. 2022-12-20 13:00:02,857 [13404] barman.wal_archiver INFO: No xlog segments found from file archival for gibraltar_14. 2022-12-20 13:00:02,857 [13405] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:02,858 [13405] barman.command_wrappers DEBUG: Command stdout: pg_basebackup (PostgreSQL) 15.1

2022-12-20 13:00:02,858 [13405] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:02,899 [13405] barman.server DEBUG: Retention policy for server trinidad_15: RECOVERY WINDOW OF 3 DAYS 2022-12-20 13:00:02,900 [13405] barman.server DEBUG: WAL retention policy for server trinidad_15: MAIN 2022-12-20 13:00:02,900 [13405] barman.server DEBUG: Starting archive-wal for server trinidad_15 2022-12-20 13:00:02,901 [13405] barman.wal_archiver INFO: No xlog segments found from streaming for trinidad_15. 2022-12-20 13:00:02,902 [13405] barman.wal_archiver INFO: No xlog segments found from file archival for trinidad_15. 2022-12-20 13:00:02,914 [13425] barman.config DEBUG: Including configuration file: gibraltar_14.conf 2022-12-20 13:00:02,915 [13425] barman.config DEBUG: Including configuration file: trinidad_14.conf 2022-12-20 13:00:02,915 [13425] barman.config DEBUG: Including configuration file: trinidad_15.conf 2022-12-20 13:00:02,915 [13425] barman.cli DEBUG: Initialised Barman version 3.3.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': False, 'debug': False, 'format': 'console', 'command': 'check', 'server_name': ['gibraltar_14' ], 'nagios': False, 'func': <function check at 0x7f5b68be6510>}) 2022-12-20 13:00:02,926 [13422] barman.config DEBUG: Including configuration file: gibraltar_14.conf 2022-12-20 13:00:02,926 [13425] barman.server DEBUG: Retention policy for server gibraltar_14: RECOVERY WINDOW OF 5 DAYS 2022-12-20 13:00:02,927 [13425] barman.server DEBUG: WAL retention policy for server gibraltar_14: MAIN 2022-12-20 13:00:02,927 [13425] barman.server DEBUG: Starting check: 'WAL archive' 2022-12-20 13:00:02,927 [13425] barman.server DEBUG: Starting check: 'empty incoming directory' 2022-12-20 13:00:02,927 [13425] barman.server DEBUG: Starting check: 'empty streaming directory' 2022-12-20 13:00:02,928 [13425] barman.server DEBUG: Starting check: 'PostgreSQL' 2022-12-20 13:00:02,936 [13422] barman.config DEBUG: Including configuration file: trinidad_14.conf 2022-12-20 13:00:02,937 [13422] barman.config DEBUG: Including configuration file: trinidad_15.conf 2022-12-20 13:00:02,937 [13422] barman.cli DEBUG: Initialised Barman version 3.3.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': False, 'debug': False, 'format': 'console', 'command': 'sync-info', 'server_name': 'gibraltar_ 14', 'last_wal': '000000010000000F000000BA', 'last_position': 1015, 'func': <function sync_info at 0x7f3999e5b7b8>}) 2022-12-20 13:00:02,949 [13422] barman.server DEBUG: Retention policy for server gibraltar_14: RECOVERY WINDOW OF 5 DAYS 2022-12-20 13:00:02,949 [13422] barman.server DEBUG: WAL retention policy for server gibraltar_14: MAIN 2022-12-20 13:00:03,004 [13425] barman.command_wrappers DEBUG: Command: ['/usr/pgsql-14/bin/pg_receivewal', '--version'] 2022-12-20 13:00:03,013 [13425] Command DEBUG: pg_receivewal (PostgreSQL) 14.6 2022-12-20 13:00:03,013 [13425] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:03,014 [13425] barman.command_wrappers DEBUG: Command stdout: pg_receivewal (PostgreSQL) 14.6

2022-12-20 13:00:03,014 [13425] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:03,016 [13425] barman.wal_archiver DEBUG: Look for 'barman_receive_wal' in 'synchronous_standby_names': [''] 2022-12-20 13:00:03,016 [13425] barman.wal_archiver DEBUG: Synchronous WAL streaming for barman_receive_wal: False 2022-12-20 13:00:03,029 [13425] barman.server DEBUG: Check 'PostgreSQL' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,029 [13425] barman.server DEBUG: Starting check: 'superuser or standard user with backup privileges' 2022-12-20 13:00:03,029 [13425] barman.server DEBUG: Check 'superuser or standard user with backup privileges' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,029 [13425] barman.server DEBUG: Starting check: 'PostgreSQL streaming' 2022-12-20 13:00:03,029 [13425] barman.server DEBUG: Check 'PostgreSQL streaming' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Starting check: 'wal_level' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Check 'wal_level' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Starting check: 'replication slot' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Check 'replication slot' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Starting check: 'directories' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Check 'directories' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Starting check: 'retention policy settings' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Check 'retention policy settings' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Starting check: 'backup maximum age' 2022-12-20 13:00:03,038 [13425] barman.server DEBUG: Check 'backup maximum age' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,038 [13425] barman.server DEBUG: Starting check: 'backup minimum size' 2022-12-20 13:00:03,039 [13425] barman.server DEBUG: Check 'backup minimum size' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,039 [13425] barman.server DEBUG: Starting check: 'wal maximum age' 2022-12-20 13:00:03,040 [13425] barman.server DEBUG: Check 'wal maximum age' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Starting check: 'wal size' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Check 'wal size' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Starting check: 'compression settings' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Check 'compression settings' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Starting check: 'failed backups' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Check 'failed backups' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Starting check: 'minimum redundancy requirements' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Check 'minimum redundancy requirements' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Starting check: 'ssh' 2022-12-20 13:00:03,041 [13425] barman.command_wrappers DEBUG: Command: "ssh '[email protected]' '-o' 'BatchMode=yes' '-o' 'StrictHostKeyChecking=no' 'true'" 2022-12-20 13:00:03,287 [13425] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:03,288 [13425] barman.command_wrappers DEBUG: Command stdout: 2022-12-20 13:00:03,288 [13425] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:03,288 [13425] barman.server DEBUG: Check 'ssh' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,288 [13425] barman.server DEBUG: Starting check: 'postgres minimal version' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Starting check: 'configuration' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Starting check: 'systemid coherence' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Check 'systemid coherence' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Starting check: 'pg_receivexlog' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Check 'pg_receivexlog' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Starting check: 'pg_receivexlog compatible' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Check 'pg_receivexlog compatible' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Starting check: 'receive-wal running' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Check 'receive-wal running' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,290 [13425] barman.server DEBUG: Starting check: 'archive_mode' 2022-12-20 13:00:03,290 [13425] barman.server DEBUG: Check 'archive_mode' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,290 [13425] barman.server DEBUG: Starting check: 'archive_command' 2022-12-20 13:00:03,290 [13425] barman.server DEBUG: Check 'archive_command' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,290 [13425] barman.server DEBUG: Check 'continuous archiving' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,290 [13425] barman.server DEBUG: Starting check: 'archiver errors' 2022-12-20 13:00:03,290 [13425] barman.server DEBUG: Check 'archiver errors' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,428 [13444] barman.config DEBUG: Including configuration file: gibraltar_14.conf 2022-12-20 13:00:03,428 [13444] barman.config DEBUG: Including configuration file: trinidad_14.conf 2022-12-20 13:00:03,429 [13444] barman.config DEBUG: Including configuration file: trinidad_15.conf 2022-12-20 13:00:03,429 [13444] barman.cli DEBUG: Initialised Barman version 3.3.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': False, 'debug': False, 'format': 'console', 'command': 'sync-info', 'server_name': 'trinidad_1 5', 'last_wal': '000000010000001C00000063', 'last_position': 999, 'func': <function sync_info at 0x7ff230d267b8>}) 2022-12-20 13:00:03,441 [13444] root DEBUG: Created a UnixLocalCommand 2022-12-20 13:00:03,453 [13444] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup 2022-12-20 13:00:03,453 [13444] barman.command_wrappers DEBUG: Command: ['/usr/pgsql-15/bin/pg_basebackup', '--version'] 2022-12-20 13:00:03,460 [13444] Command DEBUG: pg_basebackup (PostgreSQL) 15.1 2022-12-20 13:00:03,461 [13444] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:03,461 [13444] barman.command_wrappers DEBUG: Command stdout: pg_basebackup (PostgreSQL) 15.1

2022-12-20 13:00:03,461 [13444] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:03,497 [13444] barman.server DEBUG: Retention policy for server trinidad_15: RECOVERY WINDOW OF 3 DAYS 2022-12-20 13:00:03,497 [13444] barman.server DEBUG: WAL retention policy for server trinidad_15: MAIN 2022-12-20 13:00:03,503 [13444] barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down. 2022-12-20 13:00:03,592 [13454] barman.config DEBUG: Including configuration file: gibraltar_14.conf 2022-12-20 13:00:03,592 [13454] barman.config DEBUG: Including configuration file: trinidad_14.conf 2022-12-20 13:00:03,593 [13454] barman.config DEBUG: Including configuration file: trinidad_15.conf 2022-12-20 13:00:03,593 [13454] barman.cli DEBUG: Initialised Barman version 3.3.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': False, 'debug': False, 'format': 'console', 'command': 'check', 'server_name': ['trinidad_15'] , 'nagios': False, 'func': <function check at 0x7f9edb492510>}) 2022-12-20 13:00:03,605 [13454] root DEBUG: Created a UnixLocalCommand 2022-12-20 13:00:03,616 [13454] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup 2022-12-20 13:00:03,616 [13454] barman.command_wrappers DEBUG: Command: ['/usr/pgsql-15/bin/pg_basebackup', '--version'] 2022-12-20 13:00:03,623 [13454] Command DEBUG: pg_basebackup (PostgreSQL) 15.1 2022-12-20 13:00:03,623 [13454] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:03,624 [13454] barman.command_wrappers DEBUG: Command stdout: pg_basebackup (PostgreSQL) 15.1

2022-12-20 13:00:03,624 [13454] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:03,659 [13454] barman.server DEBUG: Retention policy for server trinidad_15: RECOVERY WINDOW OF 3 DAYS 2022-12-20 13:00:03,660 [13454] barman.server DEBUG: WAL retention policy for server trinidad_15: MAIN 2022-12-20 13:00:03,660 [13454] barman.server DEBUG: Starting check: 'WAL archive' 2022-12-20 13:00:03,661 [13454] barman.server DEBUG: Starting check: 'empty incoming directory' 2022-12-20 13:00:03,661 [13454] barman.server DEBUG: Starting check: 'empty streaming directory' 2022-12-20 13:00:03,661 [13454] barman.server DEBUG: Starting check: 'PostgreSQL' 2022-12-20 13:00:03,703 [13454] barman.command_wrappers DEBUG: Command: ['/usr/pgsql-15/bin/pg_receivewal', '--version'] 2022-12-20 13:00:03,710 [13454] Command DEBUG: pg_receivewal (PostgreSQL) 15.1 2022-12-20 13:00:03,710 [13454] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:03,710 [13454] barman.command_wrappers DEBUG: Command stdout: pg_receivewal (PostgreSQL) 15.1

2022-12-20 13:00:03,710 [13454] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:03,711 [13454] barman.wal_archiver DEBUG: Look for 'barman_receive_wal' in 'synchronous_standby_names': [''] 2022-12-20 13:00:03,712 [13454] barman.wal_archiver DEBUG: Synchronous WAL streaming for barman_receive_wal: False 2022-12-20 13:00:03,714 [13454] barman.command_wrappers DEBUG: Command: ['/usr/pgsql-15/bin/pg_basebackup', '--version'] 2022-12-20 13:00:03,720 [13454] Command DEBUG: pg_basebackup (PostgreSQL) 15.1 2022-12-20 13:00:03,720 [13454] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:03,720 [13454] barman.command_wrappers DEBUG: Command stdout: pg_basebackup (PostgreSQL) 15.1

2022-12-20 13:00:03,720 [13454] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:03,721 [13454] barman.server DEBUG: Check 'PostgreSQL' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,721 [13454] barman.server DEBUG: Starting check: 'superuser or standard user with backup privileges' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Check 'superuser or standard user with backup privileges' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Starting check: 'PostgreSQL streaming' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Check 'PostgreSQL streaming' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Starting check: 'wal_level' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Check 'wal_level' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Starting check: 'replication slot' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Check 'replication slot' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Starting check: 'directories' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Check 'directories' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,723 [13454] barman.server DEBUG: Starting check: 'retention policy settings' 2022-12-20 13:00:03,723 [13454] barman.server DEBUG: Check 'retention policy settings' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,723 [13454] barman.server DEBUG: Starting check: 'backup maximum age' 2022-12-20 13:00:03,729 [13454] barman.server DEBUG: Check 'backup maximum age' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,730 [13454] barman.server DEBUG: Starting check: 'backup minimum size' 2022-12-20 13:00:03,731 [13454] barman.server DEBUG: Check 'backup minimum size' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,731 [13454] barman.server DEBUG: Starting check: 'wal maximum age' 2022-12-20 13:00:03,732 [13454] barman.server DEBUG: Check 'wal maximum age' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,733 [13454] barman.server DEBUG: Starting check: 'wal size' 2022-12-20 13:00:03,733 [13454] barman.server DEBUG: Check 'wal size' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,733 [13454] barman.server DEBUG: Starting check: 'compression settings' 2022-12-20 13:00:03,733 [13454] barman.server DEBUG: Check 'compression settings' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,733 [13454] barman.server DEBUG: Starting check: 'failed backups' 2022-12-20 13:00:03,733 [13454] barman.server DEBUG: Check 'failed backups' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,734 [13454] barman.server DEBUG: Starting check: 'minimum redundancy requirements' 2022-12-20 13:00:03,734 [13454] barman.server DEBUG: Check 'minimum redundancy requirements' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,734 [13454] barman.server DEBUG: Starting check: 'pg_basebackup' 2022-12-20 13:00:03,734 [13454] barman.server DEBUG: Check 'pg_basebackup' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,734 [13454] barman.server DEBUG: Starting check: 'pg_basebackup compatible' 2022-12-20 13:00:03,734 [13454] barman.server DEBUG: Check 'pg_basebackup compatible' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,735 [13454] barman.server DEBUG: Starting check: 'pg_basebackup supports tablespaces mapping' 2022-12-20 13:00:03,735 [13454] barman.server DEBUG: Check 'pg_basebackup supports tablespaces mapping' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,736 [13454] barman.server DEBUG: Starting check: 'configuration' 2022-12-20 13:00:03,736 [13454] barman.server DEBUG: Starting check: 'systemid coherence' 2022-12-20 13:00:03,736 [13454] barman.server DEBUG: Check 'systemid coherence' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,736 [13454] barman.server DEBUG: Starting check: 'pg_receivexlog' 2022-12-20 13:00:03,736 [13454] barman.server DEBUG: Check 'pg_receivexlog' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,736 [13454] barman.server DEBUG: Starting check: 'pg_receivexlog compatible' 2022-12-20 13:00:03,736 [13454] barman.server DEBUG: Check 'pg_receivexlog compatible' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,737 [13454] barman.server DEBUG: Starting check: 'receive-wal running' 2022-12-20 13:00:03,737 [13454] barman.server DEBUG: Check 'receive-wal running' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,737 [13454] barman.server DEBUG: Starting check: 'archive_mode' 2022-12-20 13:00:03,737 [13454] barman.server DEBUG: Check 'archive_mode' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,737 [13454] barman.server DEBUG: Starting check: 'archive_command' 2022-12-20 13:00:03,737 [13454] barman.server DEBUG: Check 'archive_command' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,737 [13454] barman.server DEBUG: Check 'continuous archiving' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,737 [13454] barman.server DEBUG: Starting check: 'archiver errors' 2022-12-20 13:00:03,738 [13454] barman.server DEBUG: Check 'archiver errors' succeeded for server 'trinidad_15' 2022-12-20 13:01:02,085 [13491] barman.cli INFO: Skipping inactive server 'trinidad_14' 2022-12-20 13:01:02,143 [13491] barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down. 2022-12-20 13:01:02,531 [13501] barman.wal_archiver INFO: No xlog segments found from streaming for trinidad_15. 2022-12-20 13:01:02,531 [13501] barman.wal_archiver INFO: No xlog segments found from file archival for trinidad_15. 2022-12-20 13:01:02,561 [13500] barman.wal_archiver INFO: No xlog segments found from streaming for gibraltar_14. 2022-12-20 13:01:02,561 [13500] barman.wal_archiver INFO: No xlog segments found from file archival for gibraltar_14. 2022-12-20 13:01:03,077 [13508] barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down.
bug

opened by hzetters 2
Configurable recovery options filename

This is https://github.com/EnterpriseDB/barman/pull/707 with additional commits based on further thinking about the right way to handle the recovery options when a different destination file is requested.

opened by mikewallace1979 1

Backup from standby still waits when there is NO traffic on database

Hello, according to documentation

It is especially important that primary_conninfo is set if the standby is to be backed up when there is little or no write traffic on the primary. If primary_conninfo is not set then the backup will still run however it will wait at the stop backup stage until the current WAL semgent on the primary is newer than the latest WAL required by the backup.

This actually works when there is at least little traffic. Backup waits endlessly when there is NO traffic at all. I have configured backup on standby and it works great for instance with normal traffic but on one cluster we have long periods of time with zero changes on database which causes pg_switch_wal() to do absolutely nothing.

According to psql documentation:

However, if there has been no activity which generates WAL since the last WAL file switch, a switch will not be carried out and the start location of the current WAL file will be returned.

also with archive_timeout set to non-zero value

 When this parameter is greater than zero, the server will switch to a new segment file whenever this amount of time has elapsed since the last segment file switch, and there has been any database activity

backup hangs in STARTED state

when I force change on a database then backup completes

executed command

barman backup standby_host --reuse-backup=link

configuration file

[standby_host]
active = "True"
description = "Backup of sql prod"
ssh_command = ssh postgres@standby_host
conninfo = "host=standby_host user=barman dbname=postgres"
primary_conninfo = "host=primary_host user=barman dbname=postgres"
backup_method = "rsync"
reuse_backup = "link"
backup_options = "concurrent_backup"
archiver = on
minimum_redundancy = "1"
retention_policy = "REDUNDANCY 7"

psql configuration on standby

archive_command = 'barman-wal-archive barman standby_host %p'
archive_mode = always
wal_level = replica

opened by pru-anixe 7

Releases(release/3.3.0)

release/3.3.0(Dec 14, 2022)
Version 3.3.0 - 14 December 2022

Features

A backup can now be given a name at backup time using the new --name option supported by the barman backup and barman-cloud-backup commands. The backup name can then be used in place of the backup ID when running commands to interact with backups. Additionally, the commands to list and show backups have been been updated to include the backup name in the plain text and JSON output formats.

Stricter checking of PostgreSQL version to verify that Barman is running against a supported version of PostgreSQL.

Bug fixes

Fix inconsistencies between the barman cloud command docs and the help output for those commands.

Use a new PostgreSQL connection when switching WALs on the primary during the backup of a standby to avoid undefined behaviour such as SSL error messages and failed connections.

Reduce log volume by changing the default log level of stdout for commands executed in child processes to DEBUG (with the exception of pg_basebackup which is deliberately logged at INFO level due to it being a long-running process where it is frequently useful to see the output during the execution of the command).

Source code(tar.gz)
Source code(zip)
barman-3.3.0-manual.pdf(1.29 MB)
barman-3.3.0-py2.py3-none-any.whl(343.39 KB)
barman-3.3.0.tar.gz(1.23 MB)
barman-3.3.0.tar.gz.md5(54 bytes)
release/3.2.0(Oct 20, 2022)
Version 3.2.0 - 20 October 2022

Features

barman-cloud-backup-delete now accepts a --batch-size option which determines the maximum number of objects deleted in a single request.

All barman-cloud-* commands now accept a --read-timeout option which, when used with the aws-s3 cloud provider, determines the read timeout used by the boto3 library when making requests to S3.

Bug fixes

Fix the failure of barman recover in cases where backup_compression is set in the Barman configuration but the PostgreSQL server is unavailable.

Source code(tar.gz)
Source code(zip)
barman-3.2.0-manual.pdf(1.29 MB)
barman-3.2.0-py2.py3-none-any.whl(340.74 KB)
barman-3.2.0.tar.gz(1.23 MB)
barman-3.2.0.tar.gz.md5(54 bytes)
release/3.1.0(Sep 14, 2022)
Version 3.1.0 - 14 September 2022

Features

Backups taken with backup_method = postgres can now be compressed using lz4 and zstd compression by setting backup_compression = lz4 or backup_compression = zstd respectively. These options are only supported with PostgreSQL 15 (beta) or later.

A new option backup_compression_workers is available which sets the number of threads used for parallel compression. This is currently only available with backup_method = postgres and backup_compression = zstd.

A new option primary_conninfo can be set to avoid the need for backups of standbys to wait for a WAL switch to occur on the primary when finalizing the backup. Barman will use the connection string in primary_conninfo to perform WAL switches on the primary when stopping the backup.

Support for certain Rsync versions patched for CVE-2022-29154 which require a trailing newline in the --files-from argument.

Allow barman receive-wal maintenance options (--stop, --reset, --drop-slot and --create-slot) to run against inactive servers.

Add --port option to barman-wal-archive and barman-wal-restore commands so that a custom SSH port can be used without requiring any SSH configuration.

Various documentation improvements.

Python 3.5 is no longer supported.

Bug fixes

Ensure PostgreSQL connections are closed cleanly during the execution of barman cron.

barman generate-manifest now treats pre-existing backup_manifest files as an error condition.

backup_manifest files are renamed by appending the backup ID during recovery operations to prevent future backups including an old backup_manifest file.

Fix epoch timestamps in json output which were not timezone-aware.

The output of pg_basebackup is now written to the Barman log file while the backup is in progress.

Acknowledgements

We thank the following who contributed to this release:

barthisrael

elhananjair

kraynopp

lucianobotti

mxey

Source code(tar.gz)
Source code(zip)
barman-3.1.0-manual.pdf(1.29 MB)
barman-3.1.0-py2.py3-none-any.whl(339.06 KB)
barman-3.1.0.tar.gz(1.22 MB)
barman-3.1.0.tar.gz.md5(54 bytes)
release/3.0.1(Jun 27, 2022)
Version 3.0.1 - 27 June 2022

Bug fixes

Fix package signing issue in PyPI (same sources as 3.0.0)

Source code(tar.gz)
Source code(zip)
barman-3.0.1-py2.py3-none-any.whl(334.28 KB)
barman-3.0.1.tar.gz(1.22 MB)
barman-3.0.1.tar.gz.md5(54 bytes)
manual-3.0.1-manual.pdf(1.29 MB)
release/3.0.0(Jun 23, 2022)
Version 3.0.0 - 23 June 2022

Features

BREAKING CHANGE: PostgreSQL versions 9.6 and earlier are no longer supported. If you are using one of these versions you will need to use an earlier version of Barman.

BREAKING CHANGE: The default backup mode for Rsync backups is now concurrent rather than exclusive. Exclusive backups have been deprecated since PostgreSQL 9.6 and have been removed in PostgreSQL

If you are running Barman against PostgreSQL versions earlier than 15 and want to use exclusive backups you will now need to set exclusive_backup in backup_options.

BREAKING CHANGE: The backup metadata stored in the backup.info file for each backup has an extra field. This means that earlier versions of Barman will not work in the presence of any backups taken with 3.0.0. Additionally, users of pg-backup-api will need to upgrade it to version 0.2.0 so that pg-backup-api can work with the updated metadata.

Backups taken with backup_method = postgres can now be compressed by pg_basebackup by setting the backup_compression config option. Additional options are provided to control the compression level, the backup format and whether the pg_basebackup client or the PostgreSQL server applies the compression. NOTE: Recovery of these backups requires Barman to stage the compressed files on the recovery server in a location specified by the recovery_staging_path option.

Add support for PostgreSQL 15. Exclusive backups are not supported by PostgreSQL 15 therefore Barman configurations for PostgreSQL 15 servers are not allowed to specify exclusive_backup in backup_options.

Various documentation improvements.

Use custom_compression_magic, if set, when identifying compressed WAL files. This allows Barman to correctly identify uncompressed WALs (such as *.partial files in the streaming directory) and return them instead of attempting to decompress them.

Bug fixes

Fix an ordering bug which caused Barman to log the message "Backup failed issuing start backup command." while handling a failure in the stop backup command.

Fix a bug which prevented recovery using --target-tli when timelines greater than 9 were present, due to hexadecimal values from WAL segment names being parsed as base 10 integers.

Fix an import error which occurs when using barman cloud with certain python2 installations due to issues with the enum34 dependency.

Fix a bug where Barman would not read more than three bytes from a compressed WAL when attempting to identify the magic bytes. This means that any custom compressed WALs using magic longer than three bytes are now decompressed correctly.

Fix a bug which caused the --immediate-checkpoint flag to be ignored during backups with backup_method = rsync.

Source code(tar.gz)
Source code(zip)
barman-3.0.0-manual.pdf(1.29 MB)
barman-3.0.0-py2.py3-none-any.whl(334.25 KB)
barman-3.0.0.tar.gz(1.22 MB)
barman-3.0.0.tar.gz.md5(54 bytes)
release/2.19(Mar 10, 2022)
Version 2.19 - 9 March 2022

Features

Change barman diagnose output date format to ISO8601.

Add Google Cloud Storage (GCS) support to barman cloud.

Support current and latest recovery targets for the --target-tli option of barman recover.

Add documentation for installation on SLES.

Bug fixes

barman-wal-archive --test now returns a non-zero exit code when an error occurs.

Fix barman-cloud-check-wal-archive behaviour when -t option is used so that it exits after connectivity test.

barman recover now continues when --no-get-wal is used and "get-wal" is not set in recovery_options.

Fix barman show-servers --format=json ${server} output for inactive server.

Check for presence of barman_home in configuration file.

Passive barman servers will no longer store two copies of the tablespace data when syncing backups taken with backup_method = postgres.

Acknowledgements

We thank richyen for his contributions to this release.
Source code(tar.gz)
Source code(zip)
barman-2.19-manual.pdf(1.28 MB)
barman-2.19-py2.py3-none-any.whl(327.20 KB)
barman-2.19.tar.gz(1.21 MB)
barman-2.19.tar.gz.md5(53 bytes)
release/2.18(Jan 24, 2022)
Version 2.18 - 21 January 2021

Features

Add snappy compression algorithm support in barman cloud (requires the optional python-snappy dependency).

Allow Azure client concurrency parameters to be set when uploading WALs with barman-cloud-wal-archive.

Add --tags option in barman cloud so that backup files and archived WALs can be tagged in cloud storage (aws and azure).

Update the barman cloud exit status codes so that there is a dedicated code (2) for connectivity errors.

Add the commands barman verify-backup and barman generate-manifest to check if a backup is valid.

Add support for Azure Managed Identity auth in barman cloud which can be enabled with the --credential option.

Bug fixes

Change barman-cloud-check-wal-archive behavior when bucket does not exist.

Ensure list-files output is always sorted regardless of the underlying filesystem.

Man pages for barman-cloud-backup-keep, barman-cloud-backup-delete and barman-cloud-check-wal-archive added to Python packaging.

Acknowledgements

Thanks to the following for their contributions:

richyen

stratakis

Source code(tar.gz)
Source code(zip)
barman-2.18-manual.pdf(1.28 MB)
barman-2.18-py2.py3-none-any.whl(320.15 KB)
barman-2.18.tar.gz(1.20 MB)
barman-2.18.tar.gz.md5(53 bytes)
release/2.17(Dec 1, 2021)
Version 2.17 - 1 December 2021

Bug fixes

Resolves a performance regression introduced in version 2.14 which increased copy times for barman backup or barman recover commands when using the --jobs flag.

Ignore rsync partial transfer errors for sender processes so that such errors do not cause the backup to fail (thanks to barthisrael).

Source code(tar.gz)
Source code(zip)
barman-2.17-manual.pdf(1.27 MB)
barman-2.17-py2.py3-none-any.whl(300.54 KB)
barman-2.17.tar.gz(1.19 MB)
barman-2.17.tar.gz.md5(53 bytes)
release/2.16(Nov 30, 2021)
Version 2.16 - 17 November 2021

Features

Add the commands barman-check-wal-archive and barman-cloud-check-wal-archive to validate if a proposed archive location is safe to use for a new PostgreSQL server.

Allow Barman to identify WAL that's already compressed using a custom compression scheme to avoid compressing it again.

Add last_backup_minimum_size and last_wal_maximum_age options to barman check.

Bug fixes

Use argparse for command line parsing instead of the unmaintained argh module.

Make timezones consistent for begin_time and end_time.

Thanks for their contributions:

chtitux

George Hansper

stratakis

Thoro

vrms

Source code(tar.gz)
Source code(zip)
barman-2.16-manual.pdf(1.27 MB)
barman-2.16-py2.py3-none-any.whl(304.73 KB)
barman-2.16.tar.gz(1.19 MB)
barman-2.16.tar.gz.md5(53 bytes)
release/2.15(Oct 13, 2021)
Version 2.15 - 12 October 2021

Features

Add plural forms for the list-backup, list-server and show-server commands which are now list-backups, list-servers and show-servers. The singular forms are retained for backward compatibility.

Add the last-failed backup shortcut which references the newest failed backup in the catalog so that you can do: barman delete <SERVER> last-failed

Bug fixes

Tablespaces will no longer be omitted from backups of EPAS versions 9.6 and 10 due to an issue detecting the correct version string on older versions of EPAS.

Source code(tar.gz)
Source code(zip)
barman-2.15-manual.pdf(1.27 MB)
barman-2.15-py2.py3-none-any.whl(294.84 KB)
barman-2.15.tar.gz(1.18 MB)
barman-2.15.tar.gz.md5(53 bytes)
release/2.14(Sep 23, 2021)
Version 2.14 - 22 September 2021

Features

Add the barman-cloud-backup-delete command which allows backups in cloud storage to be deleted by specifiying either a backup ID or a retention policy.

Allow backups to be retained beyond any retention policies in force by introducing the ability to tag existing backups as archival backups using barman keep and barman-cloud-backup-keep.

Allow the use of SAS authentication tokens created at the restricted blob container level (instead of the wider storage account level) for Azure blob storage

Significantly speed up barman restore into an empty directory for backups that contain hundreds of thousands of files

Bug fixes

The backup privileges check will no longer fail if the user lacks "userepl" permissions and will return better error messages if any required permissions are missing (#318 and #319)

Source code(tar.gz)
Source code(zip)
barman-2.14-manual.pdf(1.27 MB)
barman-2.14-py2.py3-none-any.whl(293.77 KB)
barman-2.14.tar.gz(1.18 MB)
barman-2.14.tar.gz.md5(53 bytes)
release/2.13(Jul 26, 2021)
Version 2.13 - 26 July 2021

Add Azure blob storage support to barman-cloud

Support tablespace remapping in barman-cloud-restore via --tablespace name:location

Allow barman-cloud-backup and barman-cloud-wal-archive to run as Barman hook scripts, to allow data to be relayed to cloud storage from the Barman server

Bug fixes:

Stop backups failing due to idle_in_transaction_session_timeout (https://github.com/EnterpriseDB/barman/issues/333)

Fix a race condition between backup and archive-wal in updating xlog.db entries (#328)

Handle PGDATA being a symlink in barman-cloud-backup, which led to "seeking backwards is not allowed" errors on restore (#351)

Recreate pg_wal on restore if the original was a symlink (#327)

Recreate pg_tblspc symlinks for tablespaces on restore (#343)

Make barman-cloud-backup-list skip backups it cannot read, e.g., because they are in Glacier storage (#332)

Add -d database option to barman-cloud-backup to specify which database to connect to initially (#307)

Fix "Backup failed uploading data" errors from barman-cloud-backup on Python 3.8 and above, caused by attempting to pickle the boto3 client (#361)

Correctly enable server-side encryption in S3 for buckets that do not have encryption enabled by default.

In Barman 2.12, barman-cloud-backup's --encryption option did not correctly enable encryption for the contents of the backup if the backup was stored in an S3 bucket that did not have encryption enabled. If this is the case for you, please consider deleting your old backups and taking new backups with Barman 2.13.

If your S3 buckets already have encryption enabled by default (which we recommend), this does not affect you.

Source code(tar.gz)
Source code(zip)
barman-2.13-manual.pdf(1.27 MB)
barman-2.13-py2.py3-none-any.whl(277.50 KB)
barman-2.13.tar.gz(1.22 MB)
barman-2.13.tar.gz.md5(53 bytes)