Barman - Backup and Recovery Manager for PostgreSQL

Overview

Barman, Backup and Recovery Manager for PostgreSQL

Barman (Backup and Recovery Manager) is an open-source administration tool for disaster recovery of PostgreSQL servers written in Python. It allows your organisation to perform remote backups of multiple servers in business critical environments to reduce risk and help DBAs during the recovery phase.

Barman is distributed under GNU GPL 3 and maintained by 2ndQuadrant.

For further information, look at the "Web resources" section below.

Source content

Here you can find a description of files and directory distributed with Barman:

  • AUTHORS : development team of Barman
  • NEWS : release notes
  • ChangeLog : log of changes
  • LICENSE : GNU GPL3 details
  • TODO : our wishlist for Barman
  • barman : sources in Python
  • doc : tutorial and man pages
  • scripts : auxiliary scripts
  • tests : unit tests

Web resources

Licence

Copyright (C) 2011-2020 2ndQuadrant Limited

Barman is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

Barman is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with Barman. If not, see http://www.gnu.org/licenses/.

Comments
  • Some imporvements on barman documentation

    Some imporvements on barman documentation

    Hello there, I am trying to set up barman for the first time and its a little bit confusing, the documentation. I have created a user called barman on hostname called 'pg' as the doc says, and then at this point:

    Make sure you test the following command before proceeding: barman@backup$ psql -c 'SELECT version()' -U barman -h pg postgres

    I got lost, barman@backup, I thought I should test that on pg instance not on the backup as the documentation says, in addition to that the user must be postgres to run the above command. Actually, I tested this as postgres@pg and it seems working like that. Here is an output in my case.

    PostgreSQL 13.4 on x86_64-redhat-linux-gnu, compiled by gcc (GCC) 11.2.1 2021120 3 (Red Hat 11.2.1-7), 64-bit (1 row)

    I just wanna suggest a correction if my correction is right, and a little bit of improvement on the documentation, to grasp the concepts easily. Thank you.

    bug 
    opened by elhananjair 27
  • Replace argh with argparse

    Replace argh with argparse

    This is an initial attempt to replace argh with the stdlib argparse library to resolve #398

    Should be compatible across all python's supported by barman (at least locally it was). The tests for the cli pass as well. However there are still some issues to resolve, like the help command showing properly the positional arguments.

    Let me know what you think of that approach.

    opened by stratakis 24
  • Incremental backups

    Incremental backups

    Hello. I would like to discuss page-level incremental backups. I’ve created proof-of-concept fork of barman here There is no docs and unit-tests right now, but this will be fixed in near future.

    Motivation: We have large number of databases with pgdata size about 3 terabytes and changes about 1% of data per 24h. Unfortunately barman backups with hardlinks gives us about 45% deduplication ratio (there are small changes in many data-files, so many data-files changes between backups, but page changed ratio is about 2%)

    Solution to this problem seems simple: take only changed pages to backup. I’ve created simple script named barman-incr (it is in bin dir of source code). It handles backup and restore operations. Barman runs it on database host and passes LSN, timestamp and list of files from previous backup. Then we just open each datafile and read every page in it (if it turns out that file we opened is not datafile, we’ll take it all). If page is lsn >= provided lsn we take this page to backup.

    Some tests: Database with pgdata size 2.7T, 120G wals per 24h. Full backup size is 537G (compressed with gzip -3), time to take backup - 7h. Incremental backup size is 14G (also compressed with gzip -3), time to take backup - 30m.

    I’ve also tested restore consistency (restored database to some point of time and compared pg_dump result with paused replica).

    Block change tracking (Oracle DBAs should be familiar with this, here is white paper about this) implementation will require some changes in wal archiving process. I’ll present some thoughts and test results on this in Q1 2016.

    enhancement 
    opened by secwall 23
  • Streaming-only backups/WAL archiving?

    Streaming-only backups/WAL archiving?

    I try to do streaming-only Barman backup scenario. My barman.conf is

          [barman]
          barman_user = barman
          configuration_files_directory = /etc/barman.d
          barman_home = ${barmanHome}
          log_level = INFO
          compression = gzip
    
          [live]
          description =  "Streaming copy of live"
          conninfo = host=${liveHostname} user=barman dbname=postgres
          streaming_conninfo = host=${liveHostname} user=streaming_barman
          backup_method = postgres
          archiver = off
          streaming_archiver = on
          slot_name = barman
    

    My postgresql.conf is

    synchronous_standby_names = 'barman_receive_wal'
    max_wal_senders = 10
    max_replication_slots = 10
    wal_level = 'hot_standby'
    

    I do start wal receive process:

    # barman receive-wal live
    Starting receive-wal for server live
    2017-03-06 15:45:59,311 [4456] barman.server INFO: Starting receive-wal for server live
    2017-03-06 15:45:59,324 [4456] barman.wal_archiver INFO: Synchronous WAL streaming for barman_receive_wal: True
    2017-03-06 15:45:59,325 [4456] barman.wal_archiver INFO: Activating WAL archiving through streaming protocol
    live: pg_receivexlog: starting log streaming at 0/2000000 (timeline 1)
    2017-03-06 15:45:59,330 [4456] barman.command_wrappers INFO: live: pg_receivexlog: starting log streaming at 0/2000000 (timeline 1)
    

    But barman check live complains:

    # barman check live
    Server live:
    2017-03-06 21:41:54,942 [6044] barman.server ERROR: Check 'WAL archive' failed for server 'live'
            WAL archive: FAILED (please make sure WAL shipping is setup)
    2017-03-06 21:41:54,957 [6044] barman.wal_archiver INFO: Synchronous WAL streaming for barman_receive_wal: True
            PostgreSQL: OK
            superuser: OK
            PostgreSQL streaming: OK
            wal_level: OK
            replication slot: OK
            directories: OK
            retention policy settings: OK
            backup maximum age: OK (no last_backup_maximum_age provided)
            compression settings: OK
            failed backups: OK (there are 0 failed backups)
            minimum redundancy requirements: OK (have 0 backups, expected at least 0)
            pg_basebackup: OK
            pg_basebackup compatible: OK
            pg_basebackup supports tablespaces mapping: OK
            pg_receivexlog: OK
            pg_receivexlog compatible: OK
            receive-wal running: OK
            archiver errors: OK
    

    Is it mandatory to rsync WAL files from DB server for Barman to be able to do streaming backups?

    (barman 2.1)

    question 
    opened by danbst 17
  • Backup fails using postgresql method

    Backup fails using postgresql method

    I've updated the barman version from 2.3 to 2.4 (debian strech package from the pgdg repository ) and now I've problems using the postgresql basebackup method. The basebackup process seems to completing the data transfer but the backup ends with an error:

    cat /var/log/barman/barman.log | grep 32434
    2018-05-28 11:45:46,894 [32434] barman.config DEBUG: Including configuration file: pgcluster.conf
    2018-05-28 11:45:46,899 [32434] barman.cli DEBUG: Initialised Barman version 2.4 (config: /etc/barman.conf, args: {'reuse_backup': None, 'jobs': None, 'server_name': ['pgcluster'], 'format': 'console', 'quiet': False, 'retry_times': None, 'command': 'backup', 'retry_sleep': None, 'debug': False})
    2018-05-28 11:45:46,960 [32434] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup
    2018-05-28 11:45:46,960 [32434] barman.server DEBUG: Retention policy for server pgcluster: RECOVERY WINDOW OF 7 DAYS
    2018-05-28 11:45:46,960 [32434] barman.server DEBUG: WAL retention policy for server pgcluster: MAIN
    2018-05-28 11:45:46,960 [32434] barman.server DEBUG: Starting check: WAL archive
    2018-05-28 11:45:46,961 [32434] barman.server DEBUG: Starting check: empty incoming directory
    2018-05-28 11:45:46,961 [32434] barman.server DEBUG: Starting check: empty streaming directory
    2018-05-28 11:45:46,961 [32434] barman.server DEBUG: Starting check: PostgreSQL
    2018-05-28 11:45:47,005 [32434] barman.command_wrappers DEBUG: Command: ['/usr/bin/pg_receivewal', '--version']
    2018-05-28 11:45:47,056 [32434] barman.command_wrappers DEBUG: Command return code: 0
    2018-05-28 11:45:47,057 [32434] barman.command_wrappers DEBUG: Command stdout: pg_receivewal (PostgreSQL) 10.4 (Debian 10.4-2.pgdg90+1)
    2018-05-28 11:45:47,057 [32434] barman.command_wrappers DEBUG: Command stderr: could not change directory to "/root": Keine Berechtigung
    2018-05-28 11:45:47,060 [32434] barman.wal_archiver DEBUG: Look for 'barman_receive_wal' in 'synchronous_standby_names': ['']
    2018-05-28 11:45:47,061 [32434] barman.wal_archiver DEBUG: Synchronous WAL streaming for barman_receive_wal: False
    2018-05-28 11:45:47,061 [32434] barman.command_wrappers DEBUG: Command: ['/usr/bin/pg_basebackup', '--version']
    2018-05-28 11:45:47,123 [32434] barman.command_wrappers DEBUG: Command return code: 0
    2018-05-28 11:45:47,123 [32434] barman.command_wrappers DEBUG: Command stdout: pg_basebackup (PostgreSQL) 10.4 (Debian 10.4-2.pgdg90+1)
    2018-05-28 11:45:47,123 [32434] barman.command_wrappers DEBUG: Command stderr: could not change directory to "/root": Keine Berechtigung
    2018-05-28 11:45:47,124 [32434] barman.server DEBUG: Check 'PostgreSQL' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,125 [32434] barman.server DEBUG: Starting check: is_superuser
    2018-05-28 11:45:47,125 [32434] barman.server DEBUG: Check 'is_superuser' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,125 [32434] barman.server DEBUG: Starting check: PostgreSQL streaming
    2018-05-28 11:45:47,125 [32434] barman.server DEBUG: Check 'PostgreSQL streaming' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,125 [32434] barman.server DEBUG: Starting check: wal_level
    2018-05-28 11:45:47,125 [32434] barman.server DEBUG: Check 'wal_level' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,125 [32434] barman.server DEBUG: Starting check: replication slot
    2018-05-28 11:45:47,126 [32434] barman.server DEBUG: Check 'replication slot' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,126 [32434] barman.server DEBUG: Starting check: directories
    2018-05-28 11:45:47,126 [32434] barman.server DEBUG: Check 'directories' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,126 [32434] barman.server DEBUG: Starting check: retention policy settings
    2018-05-28 11:45:47,126 [32434] barman.server DEBUG: Check 'retention policy settings' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,126 [32434] barman.server DEBUG: Starting check: backup maximum age
    2018-05-28 11:45:47,127 [32434] barman.server DEBUG: Check 'backup maximum age' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,127 [32434] barman.server DEBUG: Starting check: compression settings
    2018-05-28 11:45:47,127 [32434] barman.server DEBUG: Check 'compression settings' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,127 [32434] barman.server DEBUG: Starting check: failed backups
    2018-05-28 11:45:47,130 [32434] barman.server INFO: Ignoring failed check 'failed backups' for server 'pgcluster'
    2018-05-28 11:45:47,130 [32434] barman.server DEBUG: Starting check: minimum redundancy requirements
    2018-05-28 11:45:47,130 [32434] barman.server DEBUG: Check 'minimum redundancy requirements' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,130 [32434] barman.server DEBUG: Starting check: pg_basebackup
    2018-05-28 11:45:47,130 [32434] barman.server DEBUG: Check 'pg_basebackup' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,130 [32434] barman.server DEBUG: Starting check: pg_basebackup compatible
    2018-05-28 11:45:47,130 [32434] barman.server DEBUG: Check 'pg_basebackup compatible' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,131 [32434] barman.server DEBUG: Starting check: pg_basebackup supports tablespaces mapping
    2018-05-28 11:45:47,133 [32434] barman.server DEBUG: Check 'pg_basebackup supports tablespaces mapping' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,134 [32434] barman.server DEBUG: Starting check: configuration
    2018-05-28 11:45:47,134 [32434] barman.server DEBUG: Starting check: pg_receivexlog
    2018-05-28 11:45:47,134 [32434] barman.server DEBUG: Check 'pg_receivexlog' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,134 [32434] barman.server DEBUG: Starting check: pg_receivexlog compatible
    2018-05-28 11:45:47,134 [32434] barman.server DEBUG: Check 'pg_receivexlog compatible' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,134 [32434] barman.server DEBUG: Starting check: receive-wal running
    2018-05-28 11:45:47,134 [32434] barman.server DEBUG: Check 'receive-wal running' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,134 [32434] barman.server DEBUG: Starting check: archiver errors
    2018-05-28 11:45:47,135 [32434] barman.server DEBUG: Check 'archiver errors' succeeded for server 'pgcluster'
    2018-05-28 11:45:47,135 [32434] barman.backup DEBUG: initialising backup information
    2018-05-28 11:45:47,139 [32434] barman.backup INFO: Starting backup using postgres method for server pgcluster in /var/lib/barman/pgcluster/base/20180528T114547
    2018-05-28 11:45:47,141 [32434] barman.backup_executor DEBUG: detecting data directory
    2018-05-28 11:45:47,146 [32434] barman.backup_executor DEBUG: detecting tablespaces
    2018-05-28 11:45:47,147 [32434] barman.backup_executor DEBUG: issuing start backup command
    2018-05-28 11:45:47,150 [32434] barman.backup_executor INFO: Backup start at LSN: 17/50000140 (000000010000001700000050, 00000140)
    2018-05-28 11:45:47,152 [32434] barman.backup_executor INFO: Starting backup copy via pg_basebackup for 20180528T114547
    2018-05-28 11:45:47,153 [32434] barman.command_wrappers DEBUG: Command: ['/usr/bin/pg_basebackup', '--dbname=dbname=replication host=pgcluster options=-cdatestyle=iso replication=true user=postgres application_name=barman_streaming_backup', '-v', '--no-password', '--pgdata=/var/lib/barman/pgcluster/base/20180528T114547/data', '--no-slot', '--wal-method=none']
    2018-05-28 11:48:33,543 [32434] barman.command_wrappers DEBUG: Command return code: 0
    2018-05-28 11:48:33,545 [32434] barman.backup ERROR: Backup failed copying files.
    DETAILS: 'ascii' codec can't encode character u'\xf6' in position 216: ordinal not in range(128)
    
    cat /var/lib/barman/bareos/base/20180528T114547/backup.info 
    backup_label=None
    begin_offset=320
    begin_time=2018-05-28 11:45:47.144933+02:00
    begin_wal=000000010000001700000050
    begin_xlog=17/50000140
    config_file=/etc/postgresql/10/main/postgresql.conf
    copy_stats=None
    deduplicated_size=None
    end_offset=None
    end_time=None
    end_wal=None
    end_xlog=None
    error=failure copying files ('ascii' codec can't encode character u'\xf6' in position 216: ordinal not in range(128))
    hba_file=/etc/postgresql/10/main/pg_hba.conf
    ident_file=/etc/postgresql/10/main/pg_ident.conf
    included_files=None
    mode=postgres
    pgdata=/var/lib/postgresql/10/main
    server_name=bareos
    size=None
    status=FAILED
    tablespaces=None
    timeline=1
    version=100004
    xlog_segment_size=16777216
    
     barman diagnose
    {
        "global": {
            "config": {
                "barman_home": "/var/lib/barman", 
                "barman_user": "barman", 
                "configuration_files_directory": "/etc/barman.d", 
                "errors_list": [], 
                "log_file": "/var/log/barman/barman.log", 
                "log_level": "DEBUG"
            }, 
            "system_info": {
                "barman_ver": "2.4", 
                "kernel_ver": "Linux barman 4.16.0-0.bpo.1-amd64 #1 SMP Debian 4.16.5-1~bpo9+1 (2018-05-06) x86_64 GNU/Linux", 
                "python_ver": "", 
                "release": "Distributor ID:\tDebian\nDescription:\tDebian GNU/Linux 9.4 (stretch)\nRelease:\t9.4\nCodename:\tstretch", 
                "rsync_ver": "rsync  version 3.1.2  protocol version 31", 
                "ssh_ver": ""
            }
        }, 
        "servers": {
            "pgcluster": {
                "backups": {
                    "20180526T225302": {
                        "backup_id": "20180526T225302", 
                        "backup_label": null, 
                        "begin_offset": 40, 
                        "begin_time": "Sat May 26 22:53:03 2018", 
                        "begin_wal": "0000000100000016000000B9", 
                        "begin_xlog": "16/B9000028", 
                        "config_file": "/etc/postgresql/10/main/postgresql.conf", 
                        "copy_stats": {
                            "copy_time": 163.11997, 
                            "total_time": 163.11997
                        }, 
                        "deduplicated_size": 18417589383, 
                        "end_offset": 0, 
                        "end_time": "Sat May 26 22:55:45 2018", 
                        "end_wal": "0000000100000016000000B9", 
                        "end_xlog": "16/BA000000", 
                        "error": null, 
                        "hba_file": "/etc/postgresql/10/main/pg_hba.conf", 
                        "ident_file": "/etc/postgresql/10/main/pg_ident.conf", 
                        "included_files": null, 
                        "mode": "postgres", 
                        "pgdata": "/var/lib/postgresql/10/main", 
                        "server_name": "pgcluster", 
                        "size": 18417589383, 
                        "status": "DONE", 
                        "tablespaces": null, 
                        "timeline": 1, 
                        "version": 100004, 
                        "xlog_segment_size": 16777216
                    }, 
                    "20180528T112307": {
                        "backup_id": "20180528T112307", 
                        "backup_label": null, 
                        "begin_offset": 10348264, 
                        "begin_time": "Mon May 28 11:23:07 2018", 
                        "begin_wal": "00000001000000170000004E", 
                        "begin_xlog": "17/4E9DE6E8", 
                        "config_file": "/etc/postgresql/10/main/postgresql.conf", 
                        "copy_stats": null, 
                        "deduplicated_size": null, 
                        "end_offset": null, 
                        "end_time": null, 
                        "end_wal": null, 
                        "end_xlog": null, 
                        "error": "failure copying files ('ascii' codec can't encode character u'\\xf6' in position 216: ordinal not in range(128))", 
                        "hba_file": "/etc/postgresql/10/main/pg_hba.conf", 
                        "ident_file": "/etc/postgresql/10/main/pg_ident.conf", 
                        "included_files": null, 
                        "mode": "postgres", 
                        "pgdata": "/var/lib/postgresql/10/main", 
                        "server_name": "pgcluster", 
                        "size": null, 
                        "status": "FAILED", 
                        "tablespaces": null, 
                        "timeline": 1, 
                        "version": 100004, 
                        "xlog_segment_size": 16777216
                    }, 
                    "20180528T114547": {
                        "backup_id": "20180528T114547", 
                        "backup_label": null, 
                        "begin_offset": 320, 
                        "begin_time": "Mon May 28 11:45:47 2018", 
                        "begin_wal": "000000010000001700000050", 
                        "begin_xlog": "17/50000140", 
                        "config_file": "/etc/postgresql/10/main/postgresql.conf", 
                        "copy_stats": null, 
                        "deduplicated_size": null, 
                        "end_offset": null, 
                        "end_time": null, 
                        "end_wal": null, 
                        "end_xlog": null, 
                        "error": "failure copying files ('ascii' codec can't encode character u'\\xf6' in position 216: ordinal not in range(128))", 
                        "hba_file": "/etc/postgresql/10/main/pg_hba.conf", 
                        "ident_file": "/etc/postgresql/10/main/pg_ident.conf", 
                        "included_files": null, 
                        "mode": "postgres", 
                        "pgdata": "/var/lib/postgresql/10/main", 
                        "server_name": "pgcluster", 
                        "size": null, 
                        "status": "FAILED", 
                        "tablespaces": null, 
                        "timeline": 1, 
                        "version": 100004, 
                        "xlog_segment_size": 16777216
                    }
                }, 
                "config": {
                    "active": true, 
                    "archiver": false, 
                    "archiver_batch_size": 0, 
                    "backup_directory": "/var/lib/barman/pgcluster", 
                    "backup_method": "postgres", 
                    "backup_options": "concurrent_backup", 
                    "bandwidth_limit": null, 
                    "barman_home": "/var/lib/barman", 
                    "barman_lock_directory": "/var/lib/barman", 
                    "basebackup_retry_sleep": 30, 
                    "basebackup_retry_times": 0, 
                    "basebackups_directory": "/var/lib/barman/pgcluster/base", 
                    "check_timeout": 30, 
                    "compression": null, 
                    "conninfo": "host=pgcluster user=postgres dbname=postgres", 
                    "custom_compression_filter": null, 
                    "custom_decompression_filter": null, 
                    "description": "PostgreSQL Database on pgcluster", 
                    "disabled": false, 
                    "errors_directory": "/var/lib/barman/pgcluster/errors", 
                    "immediate_checkpoint": false, 
                    "incoming_wals_directory": "/var/lib/barman/pgcluster/incoming", 
                    "last_backup_maximum_age": null, 
                    "max_incoming_wals_queue": null, 
                    "minimum_redundancy": 0, 
                    "msg_list": [], 
                    "name": "pgcluster", 
                    "network_compression": false, 
                    "parallel_jobs": 1, 
                    "path_prefix": null, 
                    "post_archive_retry_script": null, 
                    "post_archive_script": null, 
                    "post_backup_retry_script": null, 
                    "post_backup_script": null, 
                    "post_delete_retry_script": null, 
                    "post_delete_script": null, 
                    "post_recovery_retry_script": null, 
                    "post_recovery_script": null, 
                    "post_wal_delete_retry_script": null, 
                    "post_wal_delete_script": null, 
                    "pre_archive_retry_script": null, 
                    "pre_archive_script": null, 
                    "pre_backup_retry_script": null, 
                    "pre_backup_script": null, 
                    "pre_delete_retry_script": null, 
                    "pre_delete_script": null, 
                    "pre_recovery_retry_script": null, 
                    "pre_recovery_script": null, 
                    "pre_wal_delete_retry_script": null, 
                    "pre_wal_delete_script": null, 
                    "recovery_options": "", 
                    "retention_policy": "window 7 d", 
                    "retention_policy_mode": "auto", 
                    "reuse_backup": null, 
                    "slot_name": "barman", 
                    "ssh_command": "ssh postgres@pgcluster", 
                    "streaming_archiver": true, 
                    "streaming_archiver_batch_size": 0, 
                    "streaming_archiver_name": "barman_receive_wal", 
                    "streaming_backup_name": "barman_streaming_backup", 
                    "streaming_conninfo": "host=pgcluster user=postgres dbname=postgres", 
                    "streaming_wals_directory": "/var/lib/barman/pgcluster/streaming", 
                    "tablespace_bandwidth_limit": null, 
                    "wal_retention_policy": "simple-wal 7 d", 
                    "wals_directory": "/var/lib/barman/pgcluster/wals"
                }, 
                "status": {
                    "config_file": "/etc/postgresql/10/main/postgresql.conf", 
                    "connection_error": null, 
                    "current_size": 18418710804.0, 
                    "current_xlog": "000000010000001700000052", 
                    "data_directory": "/var/lib/postgresql/10/main", 
                    "hba_file": "/etc/postgresql/10/main/pg_hba.conf", 
                    "ident_file": "/etc/postgresql/10/main/pg_ident.conf", 
                    "is_in_recovery": false, 
                    "is_superuser": true, 
                    "pg_basebackup_bwlimit": true, 
                    "pg_basebackup_compatible": true, 
                    "pg_basebackup_installed": true, 
                    "pg_basebackup_path": "/usr/bin/pg_basebackup", 
                    "pg_basebackup_tbls_mapping": true, 
                    "pg_basebackup_version": "10.4-2.pgdg90+1)", 
                    "pg_receivexlog_compatible": true, 
                    "pg_receivexlog_installed": true, 
                    "pg_receivexlog_path": "/usr/bin/pg_receivewal", 
                    "pg_receivexlog_supports_slots": true, 
                    "pg_receivexlog_synchronous": false, 
                    "pg_receivexlog_version": "10.4-2.pgdg90+1)", 
                    "pgespresso_installed": false, 
                    "replication_slot": [
                        "barman", 
                        true, 
                        "17/52000000"
                    ], 
                    "replication_slot_support": true, 
                    "server_txt_version": "10.4", 
                    "streaming": true, 
                    "streaming_supported": true, 
                    "synchronous_standby_names": [
                        ""
                    ], 
                    "systemid": "6536089255217622081", 
                    "timeline": 1, 
                    "wal_level": "replica", 
                    "xlogpos": "17/52000140"
                }, 
                "wals": {
                    "last_archived_wal_per_timeline": {
                        "00000001": {
                            "compression": null, 
                            "name": "000000010000001700000051", 
                            "size": 16777216, 
                            "time": 1527500912.2586727
                        }
                    }
                }
            }, 
    

    and:

    barman check pgcluster
    Server pgcluster:
    	PostgreSQL: OK
    	is_superuser: OK
    	PostgreSQL streaming: OK
    	wal_level: OK
    	replication slot: OK
    	directories: OK
    	retention policy settings: OK
    	backup maximum age: OK (no last_backup_maximum_age provided)
    	compression settings: OK
    	failed backups: FAILED (there are 2 failed backups)
    	minimum redundancy requirements: OK (have 1 backups, expected at least 0)
    	pg_basebackup: OK
    	pg_basebackup compatible: OK
    	pg_basebackup supports tablespaces mapping: OK
    	pg_receivexlog: OK
    	pg_receivexlog compatible: OK
    	receive-wal running: OK
    	archiver errors: OK
    

    Any idea what is wrong? Or any hints? Thanks & Greetings, Michael

    bug 
    opened by mwuttke 15
  • wal_keep_segments breaking change to wal_keep_size in PG 13

    wal_keep_segments breaking change to wal_keep_size in PG 13

    Hi,

    i gave a shot to postgres 13 with barman 2.11, and got

    2020-09-26 01:24:13.343 CEST [604692] barman@postgres ERROR:  unrecognized configuration parameter "wal_keep_segments"
    2020-09-26 01:24:13.343 CEST [604692] barman@postgres STATEMENT:  SHOW "wal_keep_segments"
    2020-09-26 01:24:13.344 CEST [604692] barman@postgres ERROR:  current transaction is aborted, commands ignored until end of transaction block
    2020-09-26 01:24:13.344 CEST [604692] barman@postgres STATEMENT:  SELECT 1
    

    Cheers !

    opened by kapouer 14
  • barman recover Failure copying base backup: data transfer failure

    barman recover Failure copying base backup: data transfer failure

    Hello. I have problem when i use barman recover command. I delete all files and folders from recovery target area. In de target area; DATA Directory is empty or not existing. However during recover process tablespaces and directories was created in target, and they're tried to recreate again. Then gives errror.

    In target area, I did: -bash-4.2$ cd data/ -bash-4.2$ ls tablespace1_tbl tablespace2_tbl tablespace3_tbl PG_13_202205021 tablespace4_tbl -bash-4.2$ rm -rf * -bash-4.2$ ls -alh total 0 drwx------. 2 postgres postgres 6 May 26 09:35 . drwxr-xr-x. 4 postgres postgres 38 May 26 09:34 ..

    then my command barman recover is started to work in barman server: -bash-4.2$ barman recover --target-time="2022-05-23 16:01:00" mydb 20220519T023001 /data/ --remote-ssh-command "ssh [email protected]" Starting remote restore for server mydb using backup 20220519T023001 Destination directory: /data/ Remote command: ssh [email protected] Doing PITR. Recovery target time: '2022-05-23 16:01:00+03:00' 12375, tablespace1_tbl, /data/tablespace1_tbl 12376, tablespace2_tbl, /data/tablespace2_tbl 12377, tablespace3_tbl, /data/tablespace3_tbl Copying the base backup. ERROR: Failure copying base backup: data transfer failure rsync error: rsync: mkdir "/data/tablespace2_tbl" failed: File exists (17) rsync error: error in file IO (code 11) at main.c(657) [Receiver=3.1.2]

    To bypass this error, during the recovery, when i see 'Copying the base backup.' result, i delete tablespace1_tbl from recovery area and bypassed, continue to recover. However, i can not understand why i should delete it during recover. Moreover, after recover, i can not see any links on pg_tblspc folder in DATA. I have to link them by myself... Is there any way to declare like 'append' or 'replace' and link tablespaces in pg_tblspc automatically? Thanks

    triage 
    opened by ahmetmelihbasbug 12
  • Backup fails using rsync method

    Backup fails using rsync method

    I am having problems using the rsync method. The copy is completing but the backup does not finalise.

    -bash-4.2$ barman backup db-16
    Starting backup using rsync-concurrent method for server db-16 in /backup/barman/db-16/base/20180425T154747
    Backup start at LSN: 62/3E408028 (00000001000000620000003E, 00408028)
    This is the first backup for server db-16
    Starting backup copy via rsync/SSH for 20180425T154747
    Copy done (time: 2 hours, 58 minutes, 37 seconds)
    This is the first backup for server db-16
    Asking PostgreSQL server to finalize the backup.
    ERROR: Backup failed issuing start backup command.
    DETAILS: Connection lost, reconnection not allowed
    
    Processing xlog segments from streaming for db-16
    

    Our database server runs on FreeBSD and barman server on CentOS. This particular scenario only occurs when backing up a database server in a different data centre - we have been running Barman backups successfully when the backup server is in the same DC as the database, and we are able to complete backups between sites using the postgres method.

    Any advice/suggestions would be appreciated.

    Thanks,

    Mark

    bug 
    opened by markferrari 12
  • PostgreSQL Data directory: /var/lib/postgresql/data but postgresql in a container

    PostgreSQL Data directory: /var/lib/postgresql/data but postgresql in a container

    Hello,

    I have two machines pg-host and barman-host. My postgres is running in a docker container. Everything seems to work except the WAL shipping.

    Barman version 3.2.0.

    When running, $barman-host: barman check pghost everything is working properly except WAL archive: FAILED (please make sure WAL shipping is setup).

    My postgres database runs in a docker container, and the following command works fine on the host machine: $pg-host: barman-wal-archive IP hostname mountpath/pg_wal/000000000000000EXAMPLE> ✅

    My archive_command is : archive_command = 'barman-wal-archive barman-wal-archive IP hostname mountpath/%p' However when I run $barman-host: barman switch-wal --archive --archive-timeout 60 pghost I obtain : ERROR: The WAL file 000000000000000EXAMPLE has not been received in 60 seconds

    The only strange line that I have found so far is PostgreSQL Data directory: /var/lib/postgresql/data when running : $barman-host: barman status pghost. This path works only in the container. Do you have idea if this path is used ? can I modify it such that I can use the mounted path ?

    Many thanks,

    triage 
    opened by BoltMaud 10
  • Building python3 based packages for RHEL7 fails

    Building python3 based packages for RHEL7 fails

    Hi, Looks like something is going wrong when trying to build RPMS with python 3 on RHEL7. I'm running 'python3 setup.py bdest_rpm'. Stuff seems to be going well, but it goes wrong when handling the man pages. It copies them to the right directory, but then it runs /usr/lib/rpm/redhat/brp-compress which compresses them. During the next step it fails, because the files are now called for example barman.1.gz instead of barman.1. So then I get a bunch of error messages like this: error: File not found: /home/vagrant/barman-2.12/build/bdist.linux-x86_64/rpm/BUILDROOT/barman-2.12-1.x86_64/usr/share/man/man1/barman.1 (and the same for the other man pages) And then it stops: error: command 'rpmbuild' failed with exit status 1 So no rpms :( Is there a way I can fix this? Or is this a bug in the package? Thanks!

    Cheers, Guus

    packaging 
    opened by GuusHoutzager 10
  • ERROR: Unexpected output from

    ERROR: Unexpected output from "barman list-files"

    • Ubuntu 16.04

    • Kernel 4.4.0-75-generic #96-Ubuntu SMP Thu Apr 20 09:56:33 UTC 2017 x86_64

    • Python 2.7.12

    • Barman:

    $ apt-cache show barman
    Package: barman
    Version: 2.1-1.pgdg16.04+1
    Architecture: all
    Maintainer: Marco Nenciarini <[email protected]>
    Installed-Size: 660
    Depends: python-argcomplete, python-argh, python-dateutil, python-psycopg2, python:any (<< 2.8), python:any (>= 2.7.5-5~), adduser, rsync (>= 3.0.4~)
    Recommends: openssh-server, openssh-client, postgresql-client | postgresql-client-9.6 | postgresql-client-9.5 | postgresql-client-9.4 | postgresql-client-9.3 | postgresql-client-9.2
    Suggests: barman-cli, repmgr (>= 3.2.0)
    Homepage: http://www.pgbarman.org
    Priority: extra
    Section: database
    Filename: pool/main/b/barman/barman_2.1-1.pgdg16.04+1_all.deb
    Size: 118930
    SHA256: 16a0c0ee70762415c4bd9d8e1208b899116ba9bfd7d71e8330c7484324cd344f
    SHA1: 6aa2ea835649ab56c972d9111534f5c7964d012b
    MD5sum: 85a3c6ee9f94751bcb29dd76fec187c3
    Description: Backup and Recovery Manager for PostgreSQL
     Barman (Backup and Recovery Manager) is an open-source
     administration tool for disaster recovery of PostgreSQL
     servers written in Python.
     .
     It allows your organisation to perform remote backups of
     multiple servers in business critical environments to
     reduce risk and help DBAs during the recovery phase.
     .
     Barman is distributed under GNU GPL 3 and maintained
     by 2ndQuadrant.
    Description-md5: 3a5b57842a831067f70d5ac6c7b3c460
    

    Command used: repmgr -verbose -h <postgres_master_ip> -U repmgr -d repmgr -D /var/lib/postgresql/9.6/main standby clone

    Output:

    NOTICE: looking for configuration file in current directory
    NOTICE: looking for configuration file in /etc
    NOTICE: configuration file found at: /etc/repmgr.conf
    NOTICE: destination directory '/var/lib/postgresql/9.6/main' provided
    NOTICE: getting backup from Barman...
    ERROR: Unexpected output from "barman list-files": /mnt/extra/barman/review/basebackups/20170530T160208/backup.info
    
    postgres@pg-cluster-1:~/9.6/main$ EXCEPTION: [Errno 32] Broken pipe
    See log file for more details.
    

    Error in barman.log:

    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/barman/cli.py", line 1041, in main
        p.dispatch(pre_call=global_config)
      File "/usr/lib/python2.7/dist-packages/argh/helpers.py", line 55, in dispatch
        return dispatch(self, *args, **kwargs)
      File "/usr/lib/python2.7/dist-packages/argh/dispatching.py", line 174, in dispatch
        for line in lines:
      File "/usr/lib/python2.7/dist-packages/argh/dispatching.py", line 277, in _execute_command
        for line in result:
      File "/usr/lib/python2.7/dist-packages/argh/dispatching.py", line 231, in _call
        result = function(namespace_obj)
      File "/usr/lib/python2.7/dist-packages/barman/cli.py", line 580, in list_files
        output.info(line, log=False)
      File "/usr/lib/python2.7/dist-packages/barman/output.py", line 166, in info
        _put('info', message, *args, **kwargs)
      File "/usr/lib/python2.7/dist-packages/barman/output.py", line 97, in _put
        getattr(_writer, level)(message, *args)
      File "/usr/lib/python2.7/dist-packages/barman/output.py", line 368, in info
        self._out(message, args)
      File "/usr/lib/python2.7/dist-packages/barman/output.py", line 332, in _out
        print(_format_message(message, args), file=sys.stdout)
    
    

    Content of /mnt/extra/barman/review/basebackups/20170530T160208/backup.info

    backup_label=None
    begin_offset=40
    begin_time=2017-05-30 16:02:08+00:00
    begin_wal=0000000200000037000000EC
    begin_xlog=37/EC000028
    config_file=/etc/postgresql/9.6/main/postgresql.conf
    deduplicated_size=58211135370
    end_offset=96
    end_time=2017-05-30 16:02:08.352762+00:00
    end_wal=0000000200000037000000F2
    end_xlog=37/F2000060
    error=None
    hba_file=/etc/postgresql/9.6/main/pg_hba.conf
    ident_file=/etc/postgresql/9.6/main/pg_ident.conf
    included_files=['/etc/postgresql/9.6/main/postgresql.effilab.conf']
    mode=postgres
    pgdata=/var/lib/postgresql/9.6/main
    server_name=review
    size=58211135370
    status=DONE
    tablespaces=None
    timeline=2
    version=90602
    

    I fixed the error by updating manually with pip all my python modules.

    Before:

    appdirs (1.4.3)
    argcomplete (1.0.0)
    argh (0.26.1)
    barman (2.1)
    barman-cli (1.2)
    packaging (16.8)
    pip (9.0.1)
    psycopg2 (2.6.1)
    pyparsing (2.2.0)
    python-apt (1.1.0b1)
    python-dateutil (2.4.2)
    setuptools (35.0.2)
    six (1.10.0)
    wheel (0.29.0)
    

    After:

    appdirs (1.4.3)
    argcomplete (1.8.2)
    argh (0.26.2)
    barman (2.1)
    barman-cli (1.2)
    packaging (16.8)
    pip (9.0.1)
    psycopg2 (2.7.1)
    pyparsing (2.2.0)
    python-apt (1.1.0b1)
    python-dateutil (2.6.0)
    setuptools (35.0.2)
    six (1.10.0)
    wheel (0.29.0)
    

    I didn't take time to exactly point out which module needed to be updated sorry, but I think there might be a missing python module dependency ?

    moreinfo 
    opened by MaesterZ 10
  • Improve `barman-cloud` Google Storage support with the `kms_key_name` to be used to encrypt objects

    Improve `barman-cloud` Google Storage support with the `kms_key_name` to be used to encrypt objects

    It would be great if both barman-cloud-wal-archive and barman-cloud-backup would have the option to specify also the kms_key_name value that we would like to use to encrypt the objects uploaded into the Google Storage buckets. This would allow using the same Google Storage bucket to save backups and WALs belonging to multiple Postgres instances, each one using a different KMS CMEK.

    This would be useful for security reasons where we want to share a Google Storage bucket with multiple Postgres instances, being able to manage encryption keys on a per Postgres instance basis, without impacting multiple instances at the same time.

    The same can currently be achieved on Azure using Storage Accounts and the already provided --encryption-scope option, which allows to use different CMKs on different Postgres instances saving backups and WALs in the same Storage Account.

    This is somewhat related to what has been asked also for AWS S3 with #714

    opened by valeriodelsarto 0
  • Check WAL shipping setup only in archiver mode

    Check WAL shipping setup only in archiver mode

    Setups with only streaming_archiver enabled will fail this check. This blocks "backup" operation, while receive-wal works.

    This appears to be previously reported issue, for example see https://github.com/EnterpriseDB/barman/issues/298

    This bug may be explained by the fact previously enabling both streaming_archiver and archiver (WAL shipping) was the recommended configuration, per documentation. In such configuration this check would pass.

    opened by andrey-utkin 2
  •  barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down.

    barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down.

    Hi, My barman test environment, version 3.3.0, has these recurring messages in the logfile: barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down.

    Seems to be same problem as reported in #652, but that should have been fixed in version 3.1.0 ?

    Log with INFO level: 2022-12-20 13:28:02,002 [14899] barman.cli INFO: Skipping inactive server 'trinidad_14' 2022-12-20 13:28:02,061 [14899] barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down. 2022-12-20 13:28:02,317 [15002] barman.wal_archiver INFO: No xlog segments found from streaming for gibraltar_14. 2022-12-20 13:28:02,317 [15002] barman.wal_archiver INFO: No xlog segments found from file archival for gibraltar_14. 2022-12-20 13:28:02,419 [15003] barman.wal_archiver INFO: No xlog segments found from streaming for trinidad_15. 2022-12-20 13:28:02,419 [15003] barman.wal_archiver INFO: No xlog segments found from file archival for trinidad_15. 2022-12-20 13:28:03,182 [15014] barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down.

    Log with DEBUG level: 2022-12-20 13:00:02,114 [13399] barman.config DEBUG: Including configuration file: gibraltar_14.conf 2022-12-20 13:00:02,115 [13399] barman.config DEBUG: Including configuration file: trinidad_14.conf 2022-12-20 13:00:02,115 [13399] barman.config DEBUG: Including configuration file: trinidad_15.conf 2022-12-20 13:00:02,116 [13399] barman.cli DEBUG: Initialised Barman version 3.3.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': False, 'debug': False, 'format': 'console', 'command': 'list-servers', 'minimal': False, 'func ': <function list_servers at 0x7fcfe0cdb950>}) 2022-12-20 13:00:02,129 [13399] barman.server DEBUG: Retention policy for server gibraltar_14: RECOVERY WINDOW OF 5 DAYS 2022-12-20 13:00:02,129 [13399] barman.server DEBUG: WAL retention policy for server gibraltar_14: MAIN 2022-12-20 13:00:02,141 [13399] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup 2022-12-20 13:00:02,141 [13399] barman.server DEBUG: Retention policy for server trinidad_14: RECOVERY WINDOW OF 3 DAYS 2022-12-20 13:00:02,142 [13399] barman.server DEBUG: WAL retention policy for server trinidad_14: MAIN 2022-12-20 13:00:02,142 [13398] barman.config DEBUG: Including configuration file: gibraltar_14.conf 2022-12-20 13:00:02,142 [13399] root DEBUG: Created a UnixLocalCommand 2022-12-20 13:00:02,142 [13398] barman.config DEBUG: Including configuration file: trinidad_14.conf 2022-12-20 13:00:02,143 [13399] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup 2022-12-20 13:00:02,143 [13398] barman.config DEBUG: Including configuration file: trinidad_15.conf 2022-12-20 13:00:02,143 [13399] barman.command_wrappers DEBUG: Command: ['/usr/pgsql-15/bin/pg_basebackup', '--version'] 2022-12-20 13:00:02,143 [13398] barman.cli DEBUG: Initialised Barman version 3.3.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': False, 'debug': False, 'format': 'console', 'command': 'cron', 'keep_descriptors': False, 'fun c': <function cron at 0x7f284f567a60>}) 2022-12-20 13:00:02,149 [13399] Command DEBUG: pg_basebackup (PostgreSQL) 15.1 2022-12-20 13:00:02,150 [13399] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:02,150 [13399] barman.command_wrappers DEBUG: Command stdout: pg_basebackup (PostgreSQL) 15.1

    2022-12-20 13:00:02,150 [13399] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:02,156 [13398] barman.server DEBUG: Retention policy for server gibraltar_14: RECOVERY WINDOW OF 5 DAYS 2022-12-20 13:00:02,156 [13398] barman.server DEBUG: WAL retention policy for server gibraltar_14: MAIN 2022-12-20 13:00:02,169 [13398] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup 2022-12-20 13:00:02,169 [13398] barman.server DEBUG: Retention policy for server trinidad_14: RECOVERY WINDOW OF 3 DAYS 2022-12-20 13:00:02,170 [13398] barman.server DEBUG: WAL retention policy for server trinidad_14: MAIN 2022-12-20 13:00:02,170 [13398] barman.cli INFO: Skipping inactive server 'trinidad_14' 2022-12-20 13:00:02,171 [13398] root DEBUG: Created a UnixLocalCommand 2022-12-20 13:00:02,172 [13398] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup 2022-12-20 13:00:02,172 [13398] barman.command_wrappers DEBUG: Command: ['/usr/pgsql-15/bin/pg_basebackup', '--version'] 2022-12-20 13:00:02,179 [13398] Command DEBUG: pg_basebackup (PostgreSQL) 15.1 2022-12-20 13:00:02,179 [13398] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:02,180 [13398] barman.command_wrappers DEBUG: Command stdout: pg_basebackup (PostgreSQL) 15.1

    2022-12-20 13:00:02,180 [13398] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:02,195 [13399] barman.server DEBUG: Retention policy for server trinidad_15: RECOVERY WINDOW OF 3 DAYS 2022-12-20 13:00:02,195 [13399] barman.server DEBUG: WAL retention policy for server trinidad_15: MAIN 2022-12-20 13:00:02,196 [13399] barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down. 2022-12-20 13:00:02,228 [13398] barman.server DEBUG: Retention policy for server trinidad_15: RECOVERY WINDOW OF 3 DAYS 2022-12-20 13:00:02,228 [13398] barman.server DEBUG: WAL retention policy for server trinidad_15: MAIN 2022-12-20 13:00:02,228 [13398] barman.command_wrappers DEBUG: BarmanSubProcess: ['/usr/bin/python3', '/usr/bin/barman', '-c', '/etc/barman.conf', '-q', 'archive-wal', 'gibraltar_14'] 2022-12-20 13:00:02,231 [13398] barman.command_wrappers DEBUG: BarmanSubProcess: subprocess started. pid: 13404 2022-12-20 13:00:02,231 [13398] barman.server DEBUG: Another STREAMING ARCHIVER process is running for server gibraltar_14 2022-12-20 13:00:02,237 [13398] barman.retention_policies DEBUG: Reporting backup 20221219T200002 for server gibraltar_14 as VALID (newer than 2022-12-15 13:00:02.237609+01:00) 2022-12-20 13:00:02,240 [13398] barman.retention_policies DEBUG: Reporting backup 20221218T200002 for server gibraltar_14 as VALID (newer than 2022-12-15 13:00:02.240689+01:00) 2022-12-20 13:00:02,241 [13398] barman.retention_policies DEBUG: Reporting backup 20221217T200002 for server gibraltar_14 as VALID (newer than 2022-12-15 13:00:02.240993+01:00) 2022-12-20 13:00:02,241 [13398] barman.retention_policies DEBUG: Reporting backup 20221216T200003 for server gibraltar_14 as VALID (newer than 2022-12-15 13:00:02.241202+01:00) 2022-12-20 13:00:02,241 [13398] barman.retention_policies DEBUG: Reporting backup 20221215T200002 for server gibraltar_14 as VALID (newer than 2022-12-15 13:00:02.241405+01:00) 2022-12-20 13:00:02,241 [13398] barman.retention_policies DEBUG: Reporting backup 20221214T200002 for server gibraltar_14 as VALID (newer than 2022-12-15 13:00:02.241622+01:00) 2022-12-20 13:00:02,242 [13398] barman.command_wrappers DEBUG: BarmanSubProcess: ['/usr/bin/python3', '/usr/bin/barman', '-c', '/etc/barman.conf', '-q', 'archive-wal', 'trinidad_15'] 2022-12-20 13:00:02,252 [13398] barman.command_wrappers DEBUG: BarmanSubProcess: subprocess started. pid: 13405 2022-12-20 13:00:02,253 [13398] barman.server DEBUG: Another STREAMING ARCHIVER process is running for server trinidad_15 2022-12-20 13:00:02,256 [13398] barman.retention_policies DEBUG: Reporting backup 20221220T104329 for server trinidad_15 as VALID (newer than 2022-12-17 13:00:02.256645+01:00) 2022-12-20 13:00:02,257 [13398] barman.retention_policies DEBUG: Reporting backup 20221219T200017 for server trinidad_15 as VALID (newer than 2022-12-17 13:00:02.256973+01:00) 2022-12-20 13:00:02,257 [13398] barman.retention_policies DEBUG: Reporting backup 20221218T200017 for server trinidad_15 as VALID (newer than 2022-12-17 13:00:02.257159+01:00) 2022-12-20 13:00:02,257 [13398] barman.retention_policies DEBUG: Reporting backup 20221217T200017 for server trinidad_15 as VALID (newer than 2022-12-17 13:00:02.257331+01:00) 2022-12-20 13:00:02,257 [13398] barman.retention_policies DEBUG: Reporting backup 20221216T200018 for server trinidad_15 as VALID (newer than 2022-12-17 13:00:02.257532+01:00) 2022-12-20 13:00:02,257 [13398] barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down. 2022-12-20 13:00:02,783 [13405] barman.config DEBUG: Including configuration file: gibraltar_14.conf 2022-12-20 13:00:02,789 [13405] barman.config DEBUG: Including configuration file: trinidad_14.conf 2022-12-20 13:00:02,789 [13405] barman.config DEBUG: Including configuration file: trinidad_15.conf 2022-12-20 13:00:02,790 [13405] barman.cli DEBUG: Initialised Barman version 3.3.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': True, 'debug': False, 'format': 'console', 'command': 'archive-wal', 'config': '/etc/barman.co nf', 'server_name': 'trinidad_15', 'func': <function archive_wal at 0x7f0271a290d0>}) 2022-12-20 13:00:02,806 [13404] barman.config DEBUG: Including configuration file: gibraltar_14.conf 2022-12-20 13:00:02,812 [13404] barman.config DEBUG: Including configuration file: trinidad_14.conf 2022-12-20 13:00:02,813 [13404] barman.config DEBUG: Including configuration file: trinidad_15.conf 2022-12-20 13:00:02,813 [13404] barman.cli DEBUG: Initialised Barman version 3.3.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': True, 'debug': False, 'format': 'console', 'command': 'archive-wal', 'config': '/etc/barman.co nf', 'server_name': 'gibraltar_14', 'func': <function archive_wal at 0x7fdeb5d180d0>}) 2022-12-20 13:00:02,814 [13405] root DEBUG: Created a UnixLocalCommand 2022-12-20 13:00:02,832 [13404] barman.server DEBUG: Retention policy for server gibraltar_14: RECOVERY WINDOW OF 5 DAYS 2022-12-20 13:00:02,833 [13404] barman.server DEBUG: WAL retention policy for server gibraltar_14: MAIN 2022-12-20 13:00:02,835 [13405] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup 2022-12-20 13:00:02,840 [13405] barman.command_wrappers DEBUG: Command: ['/usr/pgsql-15/bin/pg_basebackup', '--version'] 2022-12-20 13:00:02,853 [13405] Command DEBUG: pg_basebackup (PostgreSQL) 15.1 2022-12-20 13:00:02,854 [13404] barman.server DEBUG: Starting archive-wal for server gibraltar_14 2022-12-20 13:00:02,856 [13404] barman.wal_archiver INFO: No xlog segments found from streaming for gibraltar_14. 2022-12-20 13:00:02,857 [13404] barman.wal_archiver INFO: No xlog segments found from file archival for gibraltar_14. 2022-12-20 13:00:02,857 [13405] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:02,858 [13405] barman.command_wrappers DEBUG: Command stdout: pg_basebackup (PostgreSQL) 15.1

    2022-12-20 13:00:02,858 [13405] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:02,899 [13405] barman.server DEBUG: Retention policy for server trinidad_15: RECOVERY WINDOW OF 3 DAYS 2022-12-20 13:00:02,900 [13405] barman.server DEBUG: WAL retention policy for server trinidad_15: MAIN 2022-12-20 13:00:02,900 [13405] barman.server DEBUG: Starting archive-wal for server trinidad_15 2022-12-20 13:00:02,901 [13405] barman.wal_archiver INFO: No xlog segments found from streaming for trinidad_15. 2022-12-20 13:00:02,902 [13405] barman.wal_archiver INFO: No xlog segments found from file archival for trinidad_15. 2022-12-20 13:00:02,914 [13425] barman.config DEBUG: Including configuration file: gibraltar_14.conf 2022-12-20 13:00:02,915 [13425] barman.config DEBUG: Including configuration file: trinidad_14.conf 2022-12-20 13:00:02,915 [13425] barman.config DEBUG: Including configuration file: trinidad_15.conf 2022-12-20 13:00:02,915 [13425] barman.cli DEBUG: Initialised Barman version 3.3.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': False, 'debug': False, 'format': 'console', 'command': 'check', 'server_name': ['gibraltar_14' ], 'nagios': False, 'func': <function check at 0x7f5b68be6510>}) 2022-12-20 13:00:02,926 [13422] barman.config DEBUG: Including configuration file: gibraltar_14.conf 2022-12-20 13:00:02,926 [13425] barman.server DEBUG: Retention policy for server gibraltar_14: RECOVERY WINDOW OF 5 DAYS 2022-12-20 13:00:02,927 [13425] barman.server DEBUG: WAL retention policy for server gibraltar_14: MAIN 2022-12-20 13:00:02,927 [13425] barman.server DEBUG: Starting check: 'WAL archive' 2022-12-20 13:00:02,927 [13425] barman.server DEBUG: Starting check: 'empty incoming directory' 2022-12-20 13:00:02,927 [13425] barman.server DEBUG: Starting check: 'empty streaming directory' 2022-12-20 13:00:02,928 [13425] barman.server DEBUG: Starting check: 'PostgreSQL' 2022-12-20 13:00:02,936 [13422] barman.config DEBUG: Including configuration file: trinidad_14.conf 2022-12-20 13:00:02,937 [13422] barman.config DEBUG: Including configuration file: trinidad_15.conf 2022-12-20 13:00:02,937 [13422] barman.cli DEBUG: Initialised Barman version 3.3.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': False, 'debug': False, 'format': 'console', 'command': 'sync-info', 'server_name': 'gibraltar_ 14', 'last_wal': '000000010000000F000000BA', 'last_position': 1015, 'func': <function sync_info at 0x7f3999e5b7b8>}) 2022-12-20 13:00:02,949 [13422] barman.server DEBUG: Retention policy for server gibraltar_14: RECOVERY WINDOW OF 5 DAYS 2022-12-20 13:00:02,949 [13422] barman.server DEBUG: WAL retention policy for server gibraltar_14: MAIN 2022-12-20 13:00:03,004 [13425] barman.command_wrappers DEBUG: Command: ['/usr/pgsql-14/bin/pg_receivewal', '--version'] 2022-12-20 13:00:03,013 [13425] Command DEBUG: pg_receivewal (PostgreSQL) 14.6 2022-12-20 13:00:03,013 [13425] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:03,014 [13425] barman.command_wrappers DEBUG: Command stdout: pg_receivewal (PostgreSQL) 14.6

    2022-12-20 13:00:03,014 [13425] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:03,016 [13425] barman.wal_archiver DEBUG: Look for 'barman_receive_wal' in 'synchronous_standby_names': [''] 2022-12-20 13:00:03,016 [13425] barman.wal_archiver DEBUG: Synchronous WAL streaming for barman_receive_wal: False 2022-12-20 13:00:03,029 [13425] barman.server DEBUG: Check 'PostgreSQL' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,029 [13425] barman.server DEBUG: Starting check: 'superuser or standard user with backup privileges' 2022-12-20 13:00:03,029 [13425] barman.server DEBUG: Check 'superuser or standard user with backup privileges' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,029 [13425] barman.server DEBUG: Starting check: 'PostgreSQL streaming' 2022-12-20 13:00:03,029 [13425] barman.server DEBUG: Check 'PostgreSQL streaming' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Starting check: 'wal_level' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Check 'wal_level' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Starting check: 'replication slot' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Check 'replication slot' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Starting check: 'directories' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Check 'directories' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Starting check: 'retention policy settings' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Check 'retention policy settings' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,030 [13425] barman.server DEBUG: Starting check: 'backup maximum age' 2022-12-20 13:00:03,038 [13425] barman.server DEBUG: Check 'backup maximum age' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,038 [13425] barman.server DEBUG: Starting check: 'backup minimum size' 2022-12-20 13:00:03,039 [13425] barman.server DEBUG: Check 'backup minimum size' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,039 [13425] barman.server DEBUG: Starting check: 'wal maximum age' 2022-12-20 13:00:03,040 [13425] barman.server DEBUG: Check 'wal maximum age' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Starting check: 'wal size' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Check 'wal size' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Starting check: 'compression settings' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Check 'compression settings' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Starting check: 'failed backups' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Check 'failed backups' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Starting check: 'minimum redundancy requirements' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Check 'minimum redundancy requirements' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,041 [13425] barman.server DEBUG: Starting check: 'ssh' 2022-12-20 13:00:03,041 [13425] barman.command_wrappers DEBUG: Command: "ssh '[email protected]' '-o' 'BatchMode=yes' '-o' 'StrictHostKeyChecking=no' 'true'" 2022-12-20 13:00:03,287 [13425] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:03,288 [13425] barman.command_wrappers DEBUG: Command stdout: 2022-12-20 13:00:03,288 [13425] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:03,288 [13425] barman.server DEBUG: Check 'ssh' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,288 [13425] barman.server DEBUG: Starting check: 'postgres minimal version' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Starting check: 'configuration' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Starting check: 'systemid coherence' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Check 'systemid coherence' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Starting check: 'pg_receivexlog' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Check 'pg_receivexlog' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Starting check: 'pg_receivexlog compatible' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Check 'pg_receivexlog compatible' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Starting check: 'receive-wal running' 2022-12-20 13:00:03,289 [13425] barman.server DEBUG: Check 'receive-wal running' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,290 [13425] barman.server DEBUG: Starting check: 'archive_mode' 2022-12-20 13:00:03,290 [13425] barman.server DEBUG: Check 'archive_mode' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,290 [13425] barman.server DEBUG: Starting check: 'archive_command' 2022-12-20 13:00:03,290 [13425] barman.server DEBUG: Check 'archive_command' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,290 [13425] barman.server DEBUG: Check 'continuous archiving' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,290 [13425] barman.server DEBUG: Starting check: 'archiver errors' 2022-12-20 13:00:03,290 [13425] barman.server DEBUG: Check 'archiver errors' succeeded for server 'gibraltar_14' 2022-12-20 13:00:03,428 [13444] barman.config DEBUG: Including configuration file: gibraltar_14.conf 2022-12-20 13:00:03,428 [13444] barman.config DEBUG: Including configuration file: trinidad_14.conf 2022-12-20 13:00:03,429 [13444] barman.config DEBUG: Including configuration file: trinidad_15.conf 2022-12-20 13:00:03,429 [13444] barman.cli DEBUG: Initialised Barman version 3.3.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': False, 'debug': False, 'format': 'console', 'command': 'sync-info', 'server_name': 'trinidad_1 5', 'last_wal': '000000010000001C00000063', 'last_position': 999, 'func': <function sync_info at 0x7ff230d267b8>}) 2022-12-20 13:00:03,441 [13444] root DEBUG: Created a UnixLocalCommand 2022-12-20 13:00:03,453 [13444] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup 2022-12-20 13:00:03,453 [13444] barman.command_wrappers DEBUG: Command: ['/usr/pgsql-15/bin/pg_basebackup', '--version'] 2022-12-20 13:00:03,460 [13444] Command DEBUG: pg_basebackup (PostgreSQL) 15.1 2022-12-20 13:00:03,461 [13444] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:03,461 [13444] barman.command_wrappers DEBUG: Command stdout: pg_basebackup (PostgreSQL) 15.1

    2022-12-20 13:00:03,461 [13444] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:03,497 [13444] barman.server DEBUG: Retention policy for server trinidad_15: RECOVERY WINDOW OF 3 DAYS 2022-12-20 13:00:03,497 [13444] barman.server DEBUG: WAL retention policy for server trinidad_15: MAIN 2022-12-20 13:00:03,503 [13444] barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down. 2022-12-20 13:00:03,592 [13454] barman.config DEBUG: Including configuration file: gibraltar_14.conf 2022-12-20 13:00:03,592 [13454] barman.config DEBUG: Including configuration file: trinidad_14.conf 2022-12-20 13:00:03,593 [13454] barman.config DEBUG: Including configuration file: trinidad_15.conf 2022-12-20 13:00:03,593 [13454] barman.cli DEBUG: Initialised Barman version 3.3.0 (config: /etc/barman.conf, args: {'color': 'auto', 'quiet': False, 'debug': False, 'format': 'console', 'command': 'check', 'server_name': ['trinidad_15'] , 'nagios': False, 'func': <function check at 0x7f9edb492510>}) 2022-12-20 13:00:03,605 [13454] root DEBUG: Created a UnixLocalCommand 2022-12-20 13:00:03,616 [13454] barman.backup_executor DEBUG: The default backup strategy for postgres backup_method is: concurrent_backup 2022-12-20 13:00:03,616 [13454] barman.command_wrappers DEBUG: Command: ['/usr/pgsql-15/bin/pg_basebackup', '--version'] 2022-12-20 13:00:03,623 [13454] Command DEBUG: pg_basebackup (PostgreSQL) 15.1 2022-12-20 13:00:03,623 [13454] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:03,624 [13454] barman.command_wrappers DEBUG: Command stdout: pg_basebackup (PostgreSQL) 15.1

    2022-12-20 13:00:03,624 [13454] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:03,659 [13454] barman.server DEBUG: Retention policy for server trinidad_15: RECOVERY WINDOW OF 3 DAYS 2022-12-20 13:00:03,660 [13454] barman.server DEBUG: WAL retention policy for server trinidad_15: MAIN 2022-12-20 13:00:03,660 [13454] barman.server DEBUG: Starting check: 'WAL archive' 2022-12-20 13:00:03,661 [13454] barman.server DEBUG: Starting check: 'empty incoming directory' 2022-12-20 13:00:03,661 [13454] barman.server DEBUG: Starting check: 'empty streaming directory' 2022-12-20 13:00:03,661 [13454] barman.server DEBUG: Starting check: 'PostgreSQL' 2022-12-20 13:00:03,703 [13454] barman.command_wrappers DEBUG: Command: ['/usr/pgsql-15/bin/pg_receivewal', '--version'] 2022-12-20 13:00:03,710 [13454] Command DEBUG: pg_receivewal (PostgreSQL) 15.1 2022-12-20 13:00:03,710 [13454] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:03,710 [13454] barman.command_wrappers DEBUG: Command stdout: pg_receivewal (PostgreSQL) 15.1

    2022-12-20 13:00:03,710 [13454] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:03,711 [13454] barman.wal_archiver DEBUG: Look for 'barman_receive_wal' in 'synchronous_standby_names': [''] 2022-12-20 13:00:03,712 [13454] barman.wal_archiver DEBUG: Synchronous WAL streaming for barman_receive_wal: False 2022-12-20 13:00:03,714 [13454] barman.command_wrappers DEBUG: Command: ['/usr/pgsql-15/bin/pg_basebackup', '--version'] 2022-12-20 13:00:03,720 [13454] Command DEBUG: pg_basebackup (PostgreSQL) 15.1 2022-12-20 13:00:03,720 [13454] barman.command_wrappers DEBUG: Command return code: 0 2022-12-20 13:00:03,720 [13454] barman.command_wrappers DEBUG: Command stdout: pg_basebackup (PostgreSQL) 15.1

    2022-12-20 13:00:03,720 [13454] barman.command_wrappers DEBUG: Command stderr: 2022-12-20 13:00:03,721 [13454] barman.server DEBUG: Check 'PostgreSQL' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,721 [13454] barman.server DEBUG: Starting check: 'superuser or standard user with backup privileges' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Check 'superuser or standard user with backup privileges' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Starting check: 'PostgreSQL streaming' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Check 'PostgreSQL streaming' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Starting check: 'wal_level' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Check 'wal_level' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Starting check: 'replication slot' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Check 'replication slot' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Starting check: 'directories' 2022-12-20 13:00:03,722 [13454] barman.server DEBUG: Check 'directories' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,723 [13454] barman.server DEBUG: Starting check: 'retention policy settings' 2022-12-20 13:00:03,723 [13454] barman.server DEBUG: Check 'retention policy settings' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,723 [13454] barman.server DEBUG: Starting check: 'backup maximum age' 2022-12-20 13:00:03,729 [13454] barman.server DEBUG: Check 'backup maximum age' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,730 [13454] barman.server DEBUG: Starting check: 'backup minimum size' 2022-12-20 13:00:03,731 [13454] barman.server DEBUG: Check 'backup minimum size' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,731 [13454] barman.server DEBUG: Starting check: 'wal maximum age' 2022-12-20 13:00:03,732 [13454] barman.server DEBUG: Check 'wal maximum age' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,733 [13454] barman.server DEBUG: Starting check: 'wal size' 2022-12-20 13:00:03,733 [13454] barman.server DEBUG: Check 'wal size' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,733 [13454] barman.server DEBUG: Starting check: 'compression settings' 2022-12-20 13:00:03,733 [13454] barman.server DEBUG: Check 'compression settings' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,733 [13454] barman.server DEBUG: Starting check: 'failed backups' 2022-12-20 13:00:03,733 [13454] barman.server DEBUG: Check 'failed backups' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,734 [13454] barman.server DEBUG: Starting check: 'minimum redundancy requirements' 2022-12-20 13:00:03,734 [13454] barman.server DEBUG: Check 'minimum redundancy requirements' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,734 [13454] barman.server DEBUG: Starting check: 'pg_basebackup' 2022-12-20 13:00:03,734 [13454] barman.server DEBUG: Check 'pg_basebackup' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,734 [13454] barman.server DEBUG: Starting check: 'pg_basebackup compatible' 2022-12-20 13:00:03,734 [13454] barman.server DEBUG: Check 'pg_basebackup compatible' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,735 [13454] barman.server DEBUG: Starting check: 'pg_basebackup supports tablespaces mapping' 2022-12-20 13:00:03,735 [13454] barman.server DEBUG: Check 'pg_basebackup supports tablespaces mapping' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,736 [13454] barman.server DEBUG: Starting check: 'configuration' 2022-12-20 13:00:03,736 [13454] barman.server DEBUG: Starting check: 'systemid coherence' 2022-12-20 13:00:03,736 [13454] barman.server DEBUG: Check 'systemid coherence' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,736 [13454] barman.server DEBUG: Starting check: 'pg_receivexlog' 2022-12-20 13:00:03,736 [13454] barman.server DEBUG: Check 'pg_receivexlog' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,736 [13454] barman.server DEBUG: Starting check: 'pg_receivexlog compatible' 2022-12-20 13:00:03,736 [13454] barman.server DEBUG: Check 'pg_receivexlog compatible' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,737 [13454] barman.server DEBUG: Starting check: 'receive-wal running' 2022-12-20 13:00:03,737 [13454] barman.server DEBUG: Check 'receive-wal running' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,737 [13454] barman.server DEBUG: Starting check: 'archive_mode' 2022-12-20 13:00:03,737 [13454] barman.server DEBUG: Check 'archive_mode' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,737 [13454] barman.server DEBUG: Starting check: 'archive_command' 2022-12-20 13:00:03,737 [13454] barman.server DEBUG: Check 'archive_command' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,737 [13454] barman.server DEBUG: Check 'continuous archiving' succeeded for server 'trinidad_15' 2022-12-20 13:00:03,737 [13454] barman.server DEBUG: Starting check: 'archiver errors' 2022-12-20 13:00:03,738 [13454] barman.server DEBUG: Check 'archiver errors' succeeded for server 'trinidad_15' 2022-12-20 13:01:02,085 [13491] barman.cli INFO: Skipping inactive server 'trinidad_14' 2022-12-20 13:01:02,143 [13491] barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down. 2022-12-20 13:01:02,531 [13501] barman.wal_archiver INFO: No xlog segments found from streaming for trinidad_15. 2022-12-20 13:01:02,531 [13501] barman.wal_archiver INFO: No xlog segments found from file archival for trinidad_15. 2022-12-20 13:01:02,561 [13500] barman.wal_archiver INFO: No xlog segments found from streaming for gibraltar_14. 2022-12-20 13:01:02,561 [13500] barman.wal_archiver INFO: No xlog segments found from file archival for gibraltar_14. 2022-12-20 13:01:03,077 [13508] barman.postgres WARNING: Forcing PostgreSQLConnection cleanup during process shut down.

    bug 
    opened by hzetters 2
  • Configurable recovery options filename

    Configurable recovery options filename

    This is https://github.com/EnterpriseDB/barman/pull/707 with additional commits based on further thinking about the right way to handle the recovery options when a different destination file is requested.

    opened by mikewallace1979 1
  • Backup from standby still waits when there is NO traffic on database

    Backup from standby still waits when there is NO traffic on database

    Hello, according to documentation

    It is especially important that primary_conninfo is set if the standby is to be backed up when there is little or no write traffic on the primary. If primary_conninfo is not set then the backup will still run however it will wait at the stop backup stage until the current WAL semgent on the primary is newer than the latest WAL required by the backup.
    

    This actually works when there is at least little traffic. Backup waits endlessly when there is NO traffic at all. I have configured backup on standby and it works great for instance with normal traffic but on one cluster we have long periods of time with zero changes on database which causes pg_switch_wal() to do absolutely nothing.

    According to psql documentation:

    However, if there has been no activity which generates WAL since the last WAL file switch, a switch will not be carried out and the start location of the current WAL file will be returned.
    

    also with archive_timeout set to non-zero value

     When this parameter is greater than zero, the server will switch to a new segment file whenever this amount of time has elapsed since the last segment file switch, and there has been any database activity
    

    backup hangs in STARTED state

    when I force change on a database then backup completes

    executed command

    barman backup standby_host --reuse-backup=link
    

    configuration file

    [standby_host]
    active = "True"
    description = "Backup of sql prod"
    ssh_command = ssh postgres@standby_host
    conninfo = "host=standby_host user=barman dbname=postgres"
    primary_conninfo = "host=primary_host user=barman dbname=postgres"
    backup_method = "rsync"
    reuse_backup = "link"
    backup_options = "concurrent_backup"
    archiver = on
    minimum_redundancy = "1"
    retention_policy = "REDUNDANCY 7"
    

    psql configuration on standby

    archive_command = 'barman-wal-archive barman standby_host %p'
    archive_mode = always
    wal_level = replica
    
    opened by pru-anixe 7
Releases(release/3.3.0)
  • release/3.3.0(Dec 14, 2022)

    Version 3.3.0 - 14 December 2022

    Features

    • A backup can now be given a name at backup time using the new --name option supported by the barman backup and barman-cloud-backup commands. The backup name can then be used in place of the backup ID when running commands to interact with backups. Additionally, the commands to list and show backups have been been updated to include the backup name in the plain text and JSON output formats.

    • Stricter checking of PostgreSQL version to verify that Barman is running against a supported version of PostgreSQL.

    Bug fixes

    • Fix inconsistencies between the barman cloud command docs and the help output for those commands.

    • Use a new PostgreSQL connection when switching WALs on the primary during the backup of a standby to avoid undefined behaviour such as SSL error messages and failed connections.

    • Reduce log volume by changing the default log level of stdout for commands executed in child processes to DEBUG (with the exception of pg_basebackup which is deliberately logged at INFO level due to it being a long-running process where it is frequently useful to see the output during the execution of the command).

    Source code(tar.gz)
    Source code(zip)
    barman-3.3.0-manual.pdf(1.29 MB)
    barman-3.3.0-py2.py3-none-any.whl(343.39 KB)
    barman-3.3.0.tar.gz(1.23 MB)
    barman-3.3.0.tar.gz.md5(54 bytes)
  • release/3.2.0(Oct 20, 2022)

    Version 3.2.0 - 20 October 2022

    Features

    • barman-cloud-backup-delete now accepts a --batch-size option which determines the maximum number of objects deleted in a single request.
    • All barman-cloud-* commands now accept a --read-timeout option which, when used with the aws-s3 cloud provider, determines the read timeout used by the boto3 library when making requests to S3.

    Bug fixes

    • Fix the failure of barman recover in cases where backup_compression is set in the Barman configuration but the PostgreSQL server is unavailable.
    Source code(tar.gz)
    Source code(zip)
    barman-3.2.0-manual.pdf(1.29 MB)
    barman-3.2.0-py2.py3-none-any.whl(340.74 KB)
    barman-3.2.0.tar.gz(1.23 MB)
    barman-3.2.0.tar.gz.md5(54 bytes)
  • release/3.1.0(Sep 14, 2022)

    Version 3.1.0 - 14 September 2022

    Features

    • Backups taken with backup_method = postgres can now be compressed using lz4 and zstd compression by setting backup_compression = lz4 or backup_compression = zstd respectively. These options are only supported with PostgreSQL 15 (beta) or later.

    • A new option backup_compression_workers is available which sets the number of threads used for parallel compression. This is currently only available with backup_method = postgres and backup_compression = zstd.

    • A new option primary_conninfo can be set to avoid the need for backups of standbys to wait for a WAL switch to occur on the primary when finalizing the backup. Barman will use the connection string in primary_conninfo to perform WAL switches on the primary when stopping the backup.

    • Support for certain Rsync versions patched for CVE-2022-29154 which require a trailing newline in the --files-from argument.

    • Allow barman receive-wal maintenance options (--stop, --reset, --drop-slot and --create-slot) to run against inactive servers.

    • Add --port option to barman-wal-archive and barman-wal-restore commands so that a custom SSH port can be used without requiring any SSH configuration.

    • Various documentation improvements.

    • Python 3.5 is no longer supported.

    Bug fixes

    • Ensure PostgreSQL connections are closed cleanly during the execution of barman cron.
    • barman generate-manifest now treats pre-existing backup_manifest files as an error condition.
    • backup_manifest files are renamed by appending the backup ID during recovery operations to prevent future backups including an old backup_manifest file.
    • Fix epoch timestamps in json output which were not timezone-aware.
    • The output of pg_basebackup is now written to the Barman log file while the backup is in progress.

    Acknowledgements

    We thank the following who contributed to this release:

    • barthisrael
    • elhananjair
    • kraynopp
    • lucianobotti
    • mxey
    Source code(tar.gz)
    Source code(zip)
    barman-3.1.0-manual.pdf(1.29 MB)
    barman-3.1.0-py2.py3-none-any.whl(339.06 KB)
    barman-3.1.0.tar.gz(1.22 MB)
    barman-3.1.0.tar.gz.md5(54 bytes)
  • release/3.0.1(Jun 27, 2022)

  • release/3.0.0(Jun 23, 2022)

    Version 3.0.0 - 23 June 2022

    Features

    • BREAKING CHANGE: PostgreSQL versions 9.6 and earlier are no longer supported. If you are using one of these versions you will need to use an earlier version of Barman.

    • BREAKING CHANGE: The default backup mode for Rsync backups is now concurrent rather than exclusive. Exclusive backups have been deprecated since PostgreSQL 9.6 and have been removed in PostgreSQL

      1. If you are running Barman against PostgreSQL versions earlier than 15 and want to use exclusive backups you will now need to set exclusive_backup in backup_options.
    • BREAKING CHANGE: The backup metadata stored in the backup.info file for each backup has an extra field. This means that earlier versions of Barman will not work in the presence of any backups taken with 3.0.0. Additionally, users of pg-backup-api will need to upgrade it to version 0.2.0 so that pg-backup-api can work with the updated metadata.

    • Backups taken with backup_method = postgres can now be compressed by pg_basebackup by setting the backup_compression config option. Additional options are provided to control the compression level, the backup format and whether the pg_basebackup client or the PostgreSQL server applies the compression. NOTE: Recovery of these backups requires Barman to stage the compressed files on the recovery server in a location specified by the recovery_staging_path option.

    • Add support for PostgreSQL 15. Exclusive backups are not supported by PostgreSQL 15 therefore Barman configurations for PostgreSQL 15 servers are not allowed to specify exclusive_backup in backup_options.

    • Various documentation improvements.

    • Use custom_compression_magic, if set, when identifying compressed WAL files. This allows Barman to correctly identify uncompressed WALs (such as *.partial files in the streaming directory) and return them instead of attempting to decompress them.

    Bug fixes

    • Fix an ordering bug which caused Barman to log the message "Backup failed issuing start backup command." while handling a failure in the stop backup command.

    • Fix a bug which prevented recovery using --target-tli when timelines greater than 9 were present, due to hexadecimal values from WAL segment names being parsed as base 10 integers.

    • Fix an import error which occurs when using barman cloud with certain python2 installations due to issues with the enum34 dependency.

    • Fix a bug where Barman would not read more than three bytes from a compressed WAL when attempting to identify the magic bytes. This means that any custom compressed WALs using magic longer than three bytes are now decompressed correctly.

    • Fix a bug which caused the --immediate-checkpoint flag to be ignored during backups with backup_method = rsync.

    Source code(tar.gz)
    Source code(zip)
    barman-3.0.0-manual.pdf(1.29 MB)
    barman-3.0.0-py2.py3-none-any.whl(334.25 KB)
    barman-3.0.0.tar.gz(1.22 MB)
    barman-3.0.0.tar.gz.md5(54 bytes)
  • release/2.19(Mar 10, 2022)

    Version 2.19 - 9 March 2022

    Features

    • Change barman diagnose output date format to ISO8601.

    • Add Google Cloud Storage (GCS) support to barman cloud.

    • Support current and latest recovery targets for the --target-tli option of barman recover.

    • Add documentation for installation on SLES.

    Bug fixes

    • barman-wal-archive --test now returns a non-zero exit code when an error occurs.

    • Fix barman-cloud-check-wal-archive behaviour when -t option is used so that it exits after connectivity test.

    • barman recover now continues when --no-get-wal is used and "get-wal" is not set in recovery_options.

    • Fix barman show-servers --format=json ${server} output for inactive server.

    • Check for presence of barman_home in configuration file.

    • Passive barman servers will no longer store two copies of the tablespace data when syncing backups taken with backup_method = postgres.

    Acknowledgements

    We thank richyen for his contributions to this release.

    Source code(tar.gz)
    Source code(zip)
    barman-2.19-manual.pdf(1.28 MB)
    barman-2.19-py2.py3-none-any.whl(327.20 KB)
    barman-2.19.tar.gz(1.21 MB)
    barman-2.19.tar.gz.md5(53 bytes)
  • release/2.18(Jan 24, 2022)

    Version 2.18 - 21 January 2021

    Features

    • Add snappy compression algorithm support in barman cloud (requires the optional python-snappy dependency).
    • Allow Azure client concurrency parameters to be set when uploading WALs with barman-cloud-wal-archive.
    • Add --tags option in barman cloud so that backup files and archived WALs can be tagged in cloud storage (aws and azure).
    • Update the barman cloud exit status codes so that there is a dedicated code (2) for connectivity errors.
    • Add the commands barman verify-backup and barman generate-manifest to check if a backup is valid.
    • Add support for Azure Managed Identity auth in barman cloud which can be enabled with the --credential option.

    Bug fixes

    • Change barman-cloud-check-wal-archive behavior when bucket does not exist.
    • Ensure list-files output is always sorted regardless of the underlying filesystem.
    • Man pages for barman-cloud-backup-keep, barman-cloud-backup-delete and barman-cloud-check-wal-archive added to Python packaging.

    Acknowledgements

    Thanks to the following for their contributions:

    • richyen
    • stratakis
    Source code(tar.gz)
    Source code(zip)
    barman-2.18-manual.pdf(1.28 MB)
    barman-2.18-py2.py3-none-any.whl(320.15 KB)
    barman-2.18.tar.gz(1.20 MB)
    barman-2.18.tar.gz.md5(53 bytes)
  • release/2.17(Dec 1, 2021)

    Version 2.17 - 1 December 2021

    Bug fixes

    • Resolves a performance regression introduced in version 2.14 which increased copy times for barman backup or barman recover commands when using the --jobs flag.
    • Ignore rsync partial transfer errors for sender processes so that such errors do not cause the backup to fail (thanks to barthisrael).
    Source code(tar.gz)
    Source code(zip)
    barman-2.17-manual.pdf(1.27 MB)
    barman-2.17-py2.py3-none-any.whl(300.54 KB)
    barman-2.17.tar.gz(1.19 MB)
    barman-2.17.tar.gz.md5(53 bytes)
  • release/2.16(Nov 30, 2021)

    Version 2.16 - 17 November 2021

    Features

    • Add the commands barman-check-wal-archive and barman-cloud-check-wal-archive to validate if a proposed archive location is safe to use for a new PostgreSQL server.
    • Allow Barman to identify WAL that's already compressed using a custom compression scheme to avoid compressing it again.
    • Add last_backup_minimum_size and last_wal_maximum_age options to barman check.

    Bug fixes

    • Use argparse for command line parsing instead of the unmaintained argh module.
    • Make timezones consistent for begin_time and end_time.

    Thanks for their contributions:

    • chtitux
    • George Hansper
    • stratakis
    • Thoro
    • vrms
    Source code(tar.gz)
    Source code(zip)
    barman-2.16-manual.pdf(1.27 MB)
    barman-2.16-py2.py3-none-any.whl(304.73 KB)
    barman-2.16.tar.gz(1.19 MB)
    barman-2.16.tar.gz.md5(53 bytes)
  • release/2.15(Oct 13, 2021)

    Version 2.15 - 12 October 2021

    Features

    • Add plural forms for the list-backup, list-server and show-server commands which are now list-backups, list-servers and show-servers. The singular forms are retained for backward compatibility.

    • Add the last-failed backup shortcut which references the newest failed backup in the catalog so that you can do: barman delete <SERVER> last-failed

    Bug fixes

    • Tablespaces will no longer be omitted from backups of EPAS versions 9.6 and 10 due to an issue detecting the correct version string on older versions of EPAS.
    Source code(tar.gz)
    Source code(zip)
    barman-2.15-manual.pdf(1.27 MB)
    barman-2.15-py2.py3-none-any.whl(294.84 KB)
    barman-2.15.tar.gz(1.18 MB)
    barman-2.15.tar.gz.md5(53 bytes)
  • release/2.14(Sep 23, 2021)

    Version 2.14 - 22 September 2021

    Features

    • Add the barman-cloud-backup-delete command which allows backups in cloud storage to be deleted by specifiying either a backup ID or a retention policy.

    • Allow backups to be retained beyond any retention policies in force by introducing the ability to tag existing backups as archival backups using barman keep and barman-cloud-backup-keep.

    • Allow the use of SAS authentication tokens created at the restricted blob container level (instead of the wider storage account level) for Azure blob storage

    • Significantly speed up barman restore into an empty directory for backups that contain hundreds of thousands of files

    Bug fixes

    • The backup privileges check will no longer fail if the user lacks "userepl" permissions and will return better error messages if any required permissions are missing (#318 and #319)
    Source code(tar.gz)
    Source code(zip)
    barman-2.14-manual.pdf(1.27 MB)
    barman-2.14-py2.py3-none-any.whl(293.77 KB)
    barman-2.14.tar.gz(1.18 MB)
    barman-2.14.tar.gz.md5(53 bytes)
  • release/2.13(Jul 26, 2021)

    Version 2.13 - 26 July 2021

    • Add Azure blob storage support to barman-cloud

    • Support tablespace remapping in barman-cloud-restore via --tablespace name:location

    • Allow barman-cloud-backup and barman-cloud-wal-archive to run as Barman hook scripts, to allow data to be relayed to cloud storage from the Barman server

    Bug fixes:

    • Stop backups failing due to idle_in_transaction_session_timeout (https://github.com/EnterpriseDB/barman/issues/333)

    • Fix a race condition between backup and archive-wal in updating xlog.db entries (#328)

    • Handle PGDATA being a symlink in barman-cloud-backup, which led to "seeking backwards is not allowed" errors on restore (#351)

    • Recreate pg_wal on restore if the original was a symlink (#327)

    • Recreate pg_tblspc symlinks for tablespaces on restore (#343)

    • Make barman-cloud-backup-list skip backups it cannot read, e.g., because they are in Glacier storage (#332)

    • Add -d database option to barman-cloud-backup to specify which database to connect to initially (#307)

    • Fix "Backup failed uploading data" errors from barman-cloud-backup on Python 3.8 and above, caused by attempting to pickle the boto3 client (#361)

    • Correctly enable server-side encryption in S3 for buckets that do not have encryption enabled by default.

      In Barman 2.12, barman-cloud-backup's --encryption option did not correctly enable encryption for the contents of the backup if the backup was stored in an S3 bucket that did not have encryption enabled. If this is the case for you, please consider deleting your old backups and taking new backups with Barman 2.13.

      If your S3 buckets already have encryption enabled by default (which we recommend), this does not affect you.

    Source code(tar.gz)
    Source code(zip)
    barman-2.13-manual.pdf(1.27 MB)
    barman-2.13-py2.py3-none-any.whl(277.50 KB)
    barman-2.13.tar.gz(1.22 MB)
    barman-2.13.tar.gz.md5(53 bytes)
Owner
EDB
Power To Postgres
EDB
A generic JSON document store with sharing and synchronisation capabilities.

Kinto Kinto is a minimalist JSON storage service with synchronisation and sharing abilities. Online documentation Tutorial Issue tracker Contributing

Kinto 4.2k Dec 26, 2022
A Terminal Client for MySQL with AutoCompletion and Syntax Highlighting.

mycli A command line client for MySQL that can do auto-completion and syntax highlighting. HomePage: http://mycli.net Documentation: http://mycli.net/

dbcli 10.7k Jan 7, 2023
Postgres CLI with autocompletion and syntax highlighting

A REPL for Postgres This is a postgres client that does auto-completion and syntax highlighting. Home Page: http://pgcli.com MySQL Equivalent: http://

dbcli 10.8k Dec 30, 2022
Automatic SQL injection and database takeover tool

sqlmap sqlmap is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of

sqlmapproject 25.7k Jan 2, 2023
Implement backup and recovery with AWS Backup across your AWS Organizations using a CI/CD pipeline (AWS CodePipeline).

Backup and Recovery with AWS Backup This repository provides you with a management and deployment solution for implementing Backup and Recovery with A

AWS Samples 8 Nov 22, 2022
Prophet is a tool to discover resources detailed for cloud migration, cloud backup and disaster recovery

Prophet is a tool to discover resources detailed for cloud migration, cloud backup and disaster recovery

null 22 May 31, 2022
Backup dc registry - A simple POC that abuses Backup Operator privileges to remote dump SAM, SYSTEM, and SECURITY

Backup Operator Registry Backup to Domain Compromise A simple POC that abuses Ba

Horizon 3 AI Inc 57 Dec 18, 2022
A CLI Password Manager made using Python and Postgresql database.

ManageMyPasswords (CLI Edition) A CLI Password Manager made using Python and Postgresql database. Quick Start Guide First Clone The Project git clone

Imira Randeniya 1 Sep 11, 2022
Installer, package manager, build wrapper and version manager for Piccolo

Piccl Installer, package manager, build wrapper and version manager for Piccolo

null 1 Dec 19, 2021
A simple and easy to use Python's PIP configuration manager, similar to the Arch Linux's Java manager.

PIPCONF - The PIP configuration manager If you need to manage multiple configurations containing indexes and trusted hosts for PIP, this project was m

João Paulo Carvalho 11 Nov 30, 2022
ArinjoyTheDev 1 Jul 17, 2022
File-manager - A basic file manager, written in Python

File Manager A basic file manager, written in Python. Installation Install Pytho

Samuel Ko 1 Feb 5, 2022
Expense-manager - Expense manager with python

Expense_manager TO-DO Source extractor: Credit Card, Wallet Destination extracto

null 1 Feb 13, 2022
DeFMO: Deblurring and Shape Recovery of Fast Moving Objects (CVPR 2021)

Evaluation, Training, Demo, and Inference of DeFMO DeFMO: Deblurring and Shape Recovery of Fast Moving Objects (CVPR 2021) Denys Rozumnyi, Martin R. O

Denys Rozumnyi 139 Dec 26, 2022
smartpassgen - A cross-platform package of modules for generating, secure storage and recovery of complex, cryptographic, smart passwords on the fly.

smartpassgen - A cross-platform package of modules for generating, secure storage and recovery of complex, cryptographic, smart passwords on the fly.

null 4 Sep 4, 2021
HandTailor: Towards High-Precision Monocular 3D Hand Recovery

HandTailor This repository is the implementation code and model of the paper "HandTailor: Towards High-Precision Monocular 3D Hand Recovery" (arXiv) G

Lv Jun 113 Jan 6, 2023
Perspective recovery of text using transformed ellipses

unproject_text Perspective recovery of text using transformed ellipses. See full writeup at https://mzucker.github.io/2016/10/11/unprojecting-text-wit

Matt Zucker 111 Nov 13, 2022
Data Recovery from your broken Android phone

Broken Phone Recovery a guide how to backup data from your locked android phone if you broke your screen (and more) you can skip some steps depending

v1nc 25 Sep 23, 2022
Learning Energy-Based Models by Diffusion Recovery Likelihood

Learning Energy-Based Models by Diffusion Recovery Likelihood Ruiqi Gao, Yang Song, Ben Poole, Ying Nian Wu, Diederik P. Kingma Paper: https://arxiv.o

Ruiqi Gao 41 Nov 22, 2022
[ICCV'21] PlaneTR: Structure-Guided Transformers for 3D Plane Recovery

PlaneTR: Structure-Guided Transformers for 3D Plane Recovery This is the official implementation of our ICCV 2021 paper News There maybe some bugs in

null 73 Nov 30, 2022