Dual center disaster recovery

When the primary center is not available, the secondary center will fully take over as the new primary center. There are two ways to upgrade the backup center to the primary center: zone failover and zone switchover, which are used in different scenarios. But before executing the following commands, you need to shut down the doctor on mgr. The commands are as follows:

stop doctor；

zone failover：When the primary center node is down and cannot be recovered, which makes the primary center unavailable, execute this command, then the secondary center will be upgraded to the primary center, and the nodes in the original primary center will not continue to serve.

zone switchover：It is used to switch the primary and secondary centers temporarily, that is, the primary center is downgraded to the backup center, and the backup center is upgraded to the primary center, and all nodes are still running in the original cluster.

Standby ADBMgr to master

 mgr_ctl promote -D /home/shan2/smgr

Master-standby center failover

Executed on the standby mgr:

zone failover zone2;

The nodes of the original master center will stay in the cluster as slave after failover is complete, but the status is uninitialized and not in the cluster.

The new master center is as follows, and the corresponding nodes have been upgraded to master:

postgres=# list node zone zone2;
   name   |    host    |        type        | mastername | port  | sync_state |               path                | initialized | incluster | readonly | zone   
----------+------------+--------------------+------------+-------+------------+-----------------------------------+-------------+-----------+----------+-------
 gszone2  | localhost2 | gtmcoord master    |            |  6775 |            | /home/shan2/pgdata_xc/sgc        | t           | t         | f        | zone2
 sc0zone2 | localhost2 | coordinator master |            |  4338 |            | /home/shan2/pgdata_xc/coord/s0    | t           | t         | f        | zone2
 sc1zone2 | localhost1 | coordinator master |            |  4338 |            | /home/shan2/pgdata_xc/coord/s1    | t           | t         | f        | zone2
 ds1zone2 | localhost4 | datanode master    |            | 24335 |            | /home/shan2/pgdata_xc/datanode/s1 | t           | t         | f        | zone2
 ds0zone2 | localhost3 | datanode master    |            | 14335 |            | /home/shan2/pgdata_xc/datanode/s0 | t           | t         | f        | zone2
(5 rows)

Master-standby center switchover

Executed on the standby mgr (which has currently been upgraded to master). Note that to execute zone switchover, you must first shut down the doctor, i.e., execute the stop doctor command, otherwise there is a possibility that the switchover will fail. When the gtmcoord or coordinator has active connections, the execution of zone switchover may fail, and the switchover can be forced by adding the force parameter. The details are as follows:

zone switchover zone2;If there is an active connection, default wait 10s for the connection to end, if after 10s, there is still an active connection, the switch fails;
zone switchover zone2 30;If there is an active connection, wait 30s for the active connection to end, if there is still an active connection after 30s, the switch fails;
zone switchover zone2 force；If there is an active connection, default wait 10s, wait for the connection to end, if after 10s, there is still an active connection, force switch;
zone zone switchover force 30;If there is an active connection, wait for 30s, wait for the connection to end, if after 30s, there is still an active connection, then force the switch;