[转帖]Centos 7 部署 openGauss 3.1.0 级联备库集群及集群缩容删除节点-运维指南

一、安装环境设置

1.1 硬件环境

名称	最低配置	建议配置	测试配置
服务器数量	3	略	略
硬盘	* 至少1GB用于安装openGauss的应用程序。 * 每个主机需大约300MB用于元数据存储。 * 预留70%以上的磁盘剩余空间用于数据存储。	略	略
内存	>=2G	略	略
CPU	功能调试最小1×8核，2.0GHz	略	略
网络	300兆以上以太网，生产建议采用bond	略	略

1.2 软件环境

软件类型	信息描述
操作系统	Centos 7.6
Linux文件系统	剩余inode个数 > 15亿（推荐）
工具	bzip2
Python	* oepnEuler：支持Python 3.7.x * Centos 7.6: 支持Python 3.6.x
数据库版本	opengauss 3.1.0 企业版
数据库软件包名称	openGauss-3.1.0-CentOS-64bit-all.tar.gz

1.2.1 安装python

-- root用户【所有节点】
# 本次选择安装python 3.6.5版本
mkdir /usr/local/python3
cd /usr/local/python3
wget https://www.python.org/ftp/python/3.6.5/Python-3.6.5.tgz
tar -zxf Python-3.6.5.tgz
cd /usr/local/python3/Python-3.6.5
./configure --prefix=/usr/local/python3 --enable-shared CFLAGS=-fPIC  
make && make install
ln -s /usr/local/python3/bin/python3 /usr/bin/python3
ln -s /usr/local/python3/bin/pip3 /usr/bin/pip3

-- 设置环境变量
cat >>/etc/profile<<EOF
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/python3/bin
EOF
source /etc/profile
echo "/usr/local/python3/lib/" >> /etc/ld.so.conf
ldconfig

-- 如遇如下报错
python3: error while loading shared libraries: libpython3.6m.so.1.0: cannot open shared object file: No such file or directory
可执行 cp /usr/local/python3/Python-3.6.5/libpython3.6m.so.1.0 /usr/lib64/

1.2.2 安装软件依赖包

1.2.2.1 软件依赖要求

所需软件	建议版本
libaio-devel	建议版本：0.3.109-13
flex	要求版本：2.5.31 以上
bison	建议版本：2.7-4
ncurses-devel	建议版本：5.9-13.20130511
glibc-devel	建议版本：2.17-111
patch	建议版本：2.7.1-10
redhat-lsb-core	建议版本：4.1
readline-devel	建议版本 ：7.0-13
libnsl（openeuler+x86环境中）	建议版本 ：2.28-36

1.2.2.2 安装软件依赖包

-- root用户 【所有节点】
-- 安装依赖包
yum install -y libaio-devel flex bison ncurses-devel glibc-devel patch redhat-lsb-core readline-devel zlib readline gcc perl-ExtUtils-Embed readline-devel zlib-devel expect

-- 检查是否已安装
rpm -qa --queryformat "%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n" | grep -E "libaio-devel|flex|bison|ncurses-devel|glibc-devel|patch|redhat-lsb-core|readline-devel|zlib|readline|gcc|python|python-devel|perl-ExtUtils-Embed|readline-devel|zlib-devel|expect"

1.2.3 操作系统配置

1.2.3.1 修改操作系统参数

-- root用户 【所有节点】
-- 添加如下参数：
cat>>/etc/sysctl.conf <<EOF
net.ipv4.tcp_fin_timeout=60
net.ipv4.tcp_retries1=5
net.ipv4.tcp_syn_retries=5
net.sctp.path_max_retrans=10
net.sctp.max_init_retransmits=10
EOF

-- 生效参数
sysctl -p

## 执行sysctl -p 时有如下报错，此处选择忽略
sysctl: cannot stat /proc/sys/net/sctp/path_max_retrans: No such file or directory
sysctl: cannot stat /proc/sys/net/sctp/max_init_retransmits: No such file or directory

# 预安装过程根据系统参数配置会有相应提示，请按照提示对应系统参数

1.2.3.2 修改资源系统限制

-- root用户 【所有节点】
echo "* soft stack 3072" >> /etc/security/limits.conf
echo "* hard stack 3072" >> /etc/security/limits.conf
echo "* soft nofile 1000000" >> /etc/security/limits.conf
echo "* hard nofile 1000000" >> /etc/security/limits.conf
echo "* soft nproc unlimited" >> /etc/security/limits.d/90-nproc.conf
tail -n 4 /etc/security/limits.conf
tail -n 1 /etc/security/limits.d/90-nproc.conf

1.2.3.3 关闭透明大页

-- root用户 【所有节点】
-- openGauss默认关闭使用transparent_hugepage服务，并将关闭命令写入操作系统启动文件

cat >> /etc/rc.d/rc.local<<EOF
if test -f /sys/kernel/mm/transparent_hugepage/enabled;
	then
		echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag;
	then
		echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
EOF

-- 查看是否关闭：
cat /sys/kernel/mm/transparent_hugepage/enabled
cat /sys/kernel/mm/transparent_hugepage/defrag

1.2.3.4 关闭防火墙

-- root用户 【所有节点】
systemctl disable firewalld.service
systemctl stop firewalld.service

1.2.3.5 关闭selinux

-- root用户 【所有节点】
sed -i  "s/SELINUX=enforcing/SELINUX=disabled/"  /etc/selinux/config
-- 检查
cat /etc/selinux/config | grep disabled

1.2.3.6 修改字符集

-- root用户 【所有节点】
cat>> /etc/profile<<EOF
export LANG=en_US.UTF-8
EOF
source /etc/profile
# 检查
cat /etc/profile | grep LANG
env |grep -i lang

1.2.3.7 修改时区

-- root用户 【所有节点】
cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
-- 查看
date 

1.2.3.8 关闭SWAP

-- root用户 【所有节点】
-- 临时关闭
swapoff -a

-- 永久关闭
注释 swap 行
vim /etc/fstab

-- 查看
free -h 查看 swap一行全是 0 表示swap关闭

1.2.3.9 设置网卡MTU

-- root用户 【所有节点】
-- 设置网卡为1500
ifconfig 网卡名 mtu 1500

-- 查看
ifconfig -a |grep -i mtu

1.2.3.10 设置root用户远程登陆

-- root用户 【所有节点】
cat >>/etc/ssh/sshd_config<<EOF
PermitRootLogin yes
EOF
-- 检查
cat /etc/ssh/sshd_config |grep PermitRootLogin

1.2.3.11 配置SSH

-- root用户 【所有节点】
sed -i '/Banner/s/^/#/' /etc/ssh/sshd_config
sed -i '/PermitRootLogin/s/^/#/' /etc/ssh/sshd_config
echo -e "\n" >> /etc/ssh/sshd_config
echo "Banner none " >> /etc/ssh/sshd_config
# 修改Banner配置，去掉连接到系统时，系统提示的欢迎信息。欢迎信息会干扰安装时远程操作的返回结果，影响安装正常执行
echo "PermitRootLogin yes" >> /etc/ssh/sshd_config
cat /etc/ssh/sshd_config |grep -v ^#|grep -E 'PermitRootLogin|Banner'

-- 重启生效
systemctl restart sshd.service

1.2.3.12 修改DNS配置

-- root用户 【所有节点】
-- 节点一
cat >>/etc/resolv.conf<<EOF
nameserver 192.168.17.112
EOF

-- 节点二
cat >>/etc/resolv.conf<<EOF
nameserver 192.168.17.113
EOF

-- 节点三
cat >>/etc/resolv.conf<<EOF
nameserver 192.168.17.139
EOF

-- 节点四
cat >>/etc/resolv.conf<<EOF
nameserver 192.168.17.159
EOF

1.2.3.13 添加hosts解析

-- root用户 【所有节点】
cat >> /etc/hosts<<EOF
192.168.17.112  opengauss-db1
192.168.17.113  opengauss-db2
192.168.17.139  opengauss-db3
192.168.17.159  opengauss-db4
EOF

二、集群规划

2.1 主机名-IP地址-端口号规划

项目名称	描述说明	备注
主机名	opengauss-db1、opengauss-db2、opengauss-db3、opengauss-db4	主备主机名
IP地址	192.168.17.112、192.168.17.113、192.168.17.139、192.168.17.159	主备主机名及IP
端口号	5000、26000	cm监听端口、DBnode监听端口

2.2 用户名规划

项目名称	名称	所属类型	规划建议
用户名	omm	操作系统	参照官网
组名	dbgrp	操作系统	参照官网

2.3 软件目录规划

目录名称	对应名称	目录作用
/opt/software/openGauss	software	安装软件存放目录
/opt/gaussdb/install/app	gaussdbAppPath	数据库安装目录
/opt/gaussdb/log	gaussdbLogPath	日志目录
/opt/gaussdb/install/data/db1	dataNode1	主备节点数据存放目录
/opt/gaussdb/tmp	tmpMppdbPath	临时文件目录
/opt/gaussdb/gausstools/om	gaussdbToolPath	数据库工具目录
/opt/gaussdb/corefile	corePath	数据库core文件目录
/opt/gaussdb/install/data/cm	cmDir	CM数据目录

三、软件安装

3.1 下载软件包

打开浏览器登陆 https://www.opengauss.org/zh/download/ 下载页面下载 openGauss_3.1.0 Centos_x86_64 对应企业版

3.2 上传及解压软件包

-- 节点一 root 用户
-- 创建存放安装包目录
mkdir -p /opt/software/openGauss
chmod 755 -R /opt/software

-- 上传软件包
# 上传openGauss-3.1.0-CentOS-64bit-all.tar.gz软件至/opt/software/openGauss目录
# 解压软件包,解压openGauss-3.1.0-CentOS-64bit-all.tar.gz后需保留 .sha256文件，否则执行预检查时会报[GAUSS-50201] : The /opt/software/openGauss/openGauss-2.0.0-RedHat-64bit.sha256 does not exist.

tar -zxf openGauss-3.1.0-CentOS-64bit-all.tar.gz

tar -zxf openGauss-3.1.0-CentOS-64bit-om.tar.gz

3.3 创建XML配置文件

3.3.1 配置XML文件

-- 节点一 root 用户
-- 在/opt/software/openGauss 目录下创建 clusterconfig.xml配置文件
-- 执行操作如下
cat > clusterconfig.xml<<EOF
<?xml version="1.0" encoding="UTF-8"?>
<ROOT>
    <!-- openGauss整体信息 -->
    <CLUSTER>
    <!-- 数据库名称 -->
        <PARAM name="clusterName" value="gsCluster" />
    <!-- 数据库节点名称(hostname) -->
        <PARAM name="nodeNames" value="opengauss-db1,opengauss-db2,opengauss-db3,opengauss-db4" />
    <!-- 数据库安装目录-->
        <PARAM name="gaussdbAppPath" value="/opt/gaussdb/install/app" />
    <!-- 日志目录-->
        <PARAM name="gaussdbLogPath" value="/opt/gaussdb/log" />
    <!-- 临时文件目录-->
        <PARAM name="tmpMppdbPath" value="/opt/gaussdb/tmp"/>
    <!--数据库工具目录-->
        <PARAM name="gaussdbToolPath" value="/opt/gaussdb/gausstools/om" />
    <!--数据库core文件目录-->
        <PARAM name="corePath" value="/opt/gaussdb/corefile"/>
	<!-- 节点IP，与nodeNames一一对应 -->
        <PARAM name="backIp1s" value="192.168.17.112,192.168.17.113,192.168.17.139,192.168.17.159"/>
    </CLUSTER>
    <!-- 每台服务器上的节点部署信息 -->
    <DEVICELIST>
        <!-- opengauss-db1上的节点部署信息 -->
        <DEVICE sn="opengauss-db1">
        <!-- opengauss-db1的hostname -->
            <PARAM name="name" value="opengauss-db1"/>
        <!-- opengauss-db1所在的AZ及AZ优先级 -->
            <PARAM name="azName" value="AZ1"/>
            <PARAM name="azPriority" value="1"/>
        <!-- 如果服务器只有一个网卡可用，将backIP1和sshIP1配置成同一个IP -->
            <PARAM name="backIp1" value="192.168.17.112"/>
            <PARAM name="sshIp1" value="192.168.17.112"/>
            
        <!--CM-->
     <!--CM数据目录-->
            <PARAM name="cmDir" value="/opt/gaussdb/install/data/cm" />
            <PARAM name="cmsNum" value="1" />
      <!--CM监听端口-->
            <PARAM name="cmServerPortBase" value="5000" />
            <PARAM name="cmServerlevel" value="1" />
      <!-- CM所有实例所在节点名及监听ip -->
      <!-- 添加新增节点IP -->
            <PARAM name="cmServerListenIp1" value="192.168.17.112,192.168.17.113,192.168.17.139,192.168.17.159" />
      <!-- 节点主机名 -->      
            <PARAM name="cmServerRelation" value="opengauss-db1,opengauss-db2,opengauss-db3,opengauss-db4" />
            
      <!-- dbnode -->
        <PARAM name="dataNum" value="1"/>
      <!--DBnode端口号-->
        <PARAM name="dataPortBase" value="26000"/>
      <!-- DBnode主节点上数据目录，及备机数据目录 -->
      <!-- 添加新增节点主机名及数据目录 -->
        <PARAM name="dataNode1" value="/opt/gaussdb/install/data/db1,opengauss-db2,/opt/gaussdb/install/data/db1,opengauss-db3,/opt/gaussdb/install/data/db1,opengauss-db4,/opt/gaussdb/install/data/db1"/>
      <!-- DBnode节点上设定同步模式的节点数 -->
        <PARAM name="dataNode1_syncNum" value="0"/>
        </DEVICE>

        <!-- opengauss-db2上的节点部署信息，其中"name"的值配置为主机名称(hostname) -->
        <DEVICE sn="opengauss-db2">
            <PARAM name="name" value="opengauss-db2"/>
            <PARAM name="azName" value="AZ1"/>
            <PARAM name="azPriority" value="1"/>
            <!-- 如果服务器只有一个网卡可用，将backIP1和sshIP1配置成同一个IP -->
            <PARAM name="backIp1" value="192.168.17.113"/>
            <PARAM name="sshIp1" value="192.168.17.113"/>
            <PARAM name="cmDir" value="/opt/gaussdb/install/data/cm" />
        </DEVICE>

        <!-- opengauss-db3上的节点部署信息，其中"name"的值配置为主机名称(hostname) -->
        <DEVICE sn="opengauss-db3">
            <PARAM name="name" value="opengauss-db3"/>
            <PARAM name="azName" value="AZ1"/>
            <PARAM name="azPriority" value="1"/>
            <!-- 如果服务器只有一个网卡可用，将backIP1和sshIP1配置成同一个IP -->
            <PARAM name="backIp1" value="192.168.17.139"/>
            <PARAM name="sshIp1" value="192.168.17.139"/>
            <PARAM name="cmDir" value="/opt/gaussdb/install/data/cm" />
        </DEVICE>
        
        <!-- 级联备库节点信息 -->
		<!-- opengauss-db4上的节点部署信息，其中"name"的值配置为主机名称(hostname) -->
        <DEVICE sn="opengauss-db4">
            <PARAM name="name" value="opengauss-db4"/>
            <PARAM name="azName" value="AZ1"/>
            <PARAM name="azPriority" value="1"/>
            <!-- 如果服务器只有一个网卡可用，将backIP1和sshIP1配置成同一个IP -->
            <PARAM name="backIp1" value="192.168.17.159"/>
            <PARAM name="sshIp1" value="192.168.17.159"/>
            <PARAM name="cmDir" value="/opt/gaussdb/install/data/cm" />
			<!-- opengauss-db4 增加了cascadeRole标识，on 标识该实例为级联备库 -->
			<PARAM name="cascadeRole" value="on"/>
        </DEVICE>
    </DEVICELIST>
</ROOT>
EOF

3.3.2 参数简介

实例类型	参数	说明
整体信息	name	主机名称
azName	指定azName（Available Zone Name），字符串（不能含有特殊字符），例如AZ1、AZ2、AZ3。
azPriority	指定azPriority的优先级。
backIp1	主机在后端存储网络中的IP地址（内网IP）。所有openGauss主机使用后端存储网络通讯。
sshIp1	设置SSH可信通道IP地址（外网IP）。若无外网，则可以不设置该选项或者同backIp1设置相同IP。

3.4 初始化安装环境

# root 用户 节点一主节点操作

-- 设置lib库
[root@opengauss-db1 ~]# export LD_LIBRARY_PATH=/opt/software/openGauss/script/gspylib/clib:$LD_LIBRARY_PATH

-- 执行预执行
[root@opengauss-db1 ~]# python3 /opt/software/openGauss/script/gs_preinstall -U omm -G dbgrp -X /opt/software/openGauss/clusterconfig.xml

## 执行结果如下
Parsing the configuration file.
Successfully parsed the configuration file.
Installing the tools on the local node.
Successfully installed the tools on the local node.
Are you sure you want to create trust for root (yes/no)?yes  -- 输入yes
Please enter password for root
Password:     -- 设置 root 口令
Successfully created SSH trust for the root permission user.
Setting host ip env
Successfully set host ip env.
Distributing package.
Begin to distribute package to tool path.
Successfully distribute package to tool path.
Begin to distribute package to package path.
Successfully distribute package to package path.
Successfully distributed package.
Are you sure you want to create the user[omm] and create trust for it (yes/no)? yes -- 输入yes
Please enter password for cluster user.
Password:    -- 设置 omm 用户口令
Please enter password for cluster user again.
Password:    -- 重复输入 omm 用户口令
Generate cluster user password files successfully.

Successfully created [omm] user on all nodes.
Preparing SSH service.
Successfully prepared SSH service.
Installing the tools in the cluster.
Successfully installed the tools in the cluster.
Checking hostname mapping.
Successfully checked hostname mapping.
Creating SSH trust for [omm] user.
Please enter password for current user[omm].
Password:   -- 输入 omm 用户口令
Checking network information.
All nodes in the network are Normal.
Successfully checked network information.
Creating SSH trust.
Creating the local key file.
Successfully created the local key files.
Appending local ID to authorized_keys.
Successfully appended local ID to authorized_keys.
Updating the known_hosts file.
Successfully updated the known_hosts file.
Appending authorized_key on the remote node.
Successfully appended authorized_key on all remote node.
Checking common authentication file content.
Successfully checked common authentication content.
Distributing SSH trust file to all node.
Distributing trust keys file to all node successfully.
Successfully distributed SSH trust file to all node.
Verifying SSH trust on all hosts.
Successfully verified SSH trust on all hosts.
Successfully created SSH trust.
Successfully created SSH trust for [omm] user.
Checking OS software.
Successfully check os software.
Checking OS version.
Successfully checked OS version.
Creating cluster's path.
Successfully created cluster's path.
Set and check OS parameter.
Setting OS parameters.
Successfully set OS parameters.
Warning: Installation environment contains some warning messages.
Please get more details by "/opt/software/openGauss/script/gs_checkos -i A -h opengauss-db1,opengauss-db2,opengauss-db3,opengauss-db4 --detail".
Set and check OS parameter completed.
Preparing CRON service.
Successfully prepared CRON service.
Setting user environmental variables.
Successfully set user environmental variables.
Setting the dynamic link library.
Successfully set the dynamic link library.
Setting Core file
Successfully set core path.
Setting pssh path
Successfully set pssh path.
Setting Cgroup.
Successfully set Cgroup.
Set ARM Optimization.
No need to set ARM Optimization.
Fixing server package owner.
Setting finish flag.
Successfully set finish flag.
Preinstallation succeeded.

-- 查看预安装结果信息，并根据预检查调整参数
-- root 用户
[root@opengauss-db1 ~]# /opt/software/openGauss/script/gs_checkos -i A -h opengauss-db1,opengauss-db2,opengauss-db3,opengauss-db4 --detail
## 执行结果如下
Checking items:
    A1. [ OS version status ]                                   : Normal     
        [opengauss-db2]
        centos_7.9.2009_64bit
        [opengauss-db1]
        centos_7.9.2009_64bit
        [opengauss-db3]
        centos_7.9.2009_64bit
        [opengauss-db4]
        centos_7.9.2009_64bit

    A2. [ Kernel version status ]                               : Normal     
        The names about all kernel versions are same. The value is "3.10.0-693.el7.x86_64".
    A3. [ Unicode status ]                                      : Normal     
        The values of all unicode are same. The value is "LANG=en_US.UTF-8".
    A4. [ Time zone status ]                                    : Normal     
        The informations about all timezones are same. The value is "+0800".
    A5. [ Swap memory status ]                                  : Normal     
        The value about swap memory is correct.            
    A6. [ System control parameters status ]                    : Normal     
        All values about system control  parameters are correct.
    A7. [ File system configuration status ]                    : Normal     
        Both soft nofile and hard nofile are correct.      
    A8. [ Disk configuration status ]                           : Normal     
        The value about XFS mount parameters is correct.   
    A9. [ Pre-read block size status ]                          : Normal     
        The value about Logical block size is correct.     
    A10.[ IO scheduler status ]                                 : Normal     
        The value of IO scheduler is correct.              
    A11.[ Network card configuration status ]                   : Warning    
        [opengauss-db1]
BondMode Null
        Warning reason: network 'ens33' 'mtu' RealValue '1500' ExpectedValue '8192' -- 可忽略

        [opengauss-db2]
BondMode Null
        Warning reason: network 'ens33' 'mtu' RealValue '1500' ExpectedValue '8192' -- 可忽略

        [opengauss-db3]
BondMode Null
        Warning reason: network 'ens33' 'mtu' RealValue '1500' ExpectedValue '8192' -- 可忽略

        [opengauss-db4]
BondMode Null
        Warning reason: network 'ens33' 'mtu' RealValue '1500' ExpectedValue '8192' -- 可忽略


    A12.[ Time consistency status ]                             : Warning    
        [opengauss-db2]
        The NTPD not detected on machine and local time is "2022-11-14 17:08:56".
        [opengauss-db1]
        The NTPD not detected on machine and local time is "2022-11-14 17:08:56".
        [opengauss-db3]
        The NTPD not detected on machine and local time is "2022-11-14 17:08:56".
        [opengauss-db4]
        The NTPD not detected on machine and local time is "2022-11-14 17:08:56".

    A13.[ Firewall service status ]                             : Normal     
        The firewall service is stopped.                   
    A14.[ THP service status ]                                  : Normal     
        The THP service is stopped.                        
Total numbers:14. Abnormal numbers:0. Warning numbers:2.
-- 根据执行detail命令结果对预检查告警进行调整

3.5 执行集群安装

3.5.1 修改属主

-- 务必确保已在节点一执行预检查
-- root 用户 节点一操作
[root@opengauss-db1 ~]# chmod -R 755 /opt/software/openGauss/script/
[root@opengauss-db1 ~]# chown -R omm:dbgrp /opt/software/openGauss/script/

3.5.2 执行集群安装

-- 节点一切换到omm用户执行
[root@opengauss-db1 ~]# su - omm
-- 执行如下命令
[omm@opengauss-db1 ~]$ gs_install -X /opt/software/openGauss/clusterconfig.xml
## 执行结果如下
Parsing the configuration file.
Check preinstall on every node.
Successfully checked preinstall on every node.
Creating the backup directory.
Successfully created the backup directory.
begin deploy..
Installing the cluster.
begin prepare Install Cluster..
Checking the installation environment on all nodes.
begin install Cluster..
Installing applications on all nodes.
Successfully installed APP.
begin init Instance..
encrypt cipher and rand files for database.
Please enter password for database:    -- 设置数据库 口令 如 Passw0rd@1234
Please repeat for database:            -- 再次输入相同口令
begin to create CA cert files
The sslcert will be generated in /opt/gaussdb/install/app/share/sslcert/om
Create CA files for cm beginning.
Create CA files on directory [/opt/gaussdb/install/app_4e931f9a/share/sslcert/cm]. file list: ['client.key.rand', 'server.crt', 'cacert.pem', 'client.crt', 'server.key.rand', 'client.key', 'client.key.cipher', 'server.key', 'server.key.cipher']
Cluster installation is completed.
Configuring.
Deleting instances from all nodes.
Successfully deleted instances from all nodes.
Checking node configuration on all nodes.
Initializing instances on all nodes.
Updating instance configuration on all nodes.
Check consistence of memCheck and coresCheck on database nodes.
Successful check consistence of memCheck and coresCheck on all nodes.
Configuring pg_hba on all nodes.
Configuration is completed.
Starting cluster.
======================================================================
Successfully started primary instance. Wait for standby instance.
======================================================================
.
Successfully started cluster.
======================================================================
cluster_state      : Normal
redistributing     : No
node_count         : 4
Datanode State
    primary           : 1
    standby           : 2
    secondary         : 0
    cascade_standby   : 1
    building          : 0
    abnormal          : 0
    down              : 0

Successfully installed application.
end deploy..

3.5.3 数据库状态检查

-- omm 用户
[omm@opengauss-db1 ~]$ gs_om -t status --detail
[  CMServer State   ]

node             node_ip         instance                                    state
------------------------------------------------------------------------------------
1  opengauss-db1 192.168.17.112  1    /opt/gaussdb/install/data/cm/cm_server Primary
2  opengauss-db2 192.168.17.113  2    /opt/gaussdb/install/data/cm/cm_server Standby
3  opengauss-db3 192.168.17.139  3    /opt/gaussdb/install/data/cm/cm_server Standby
4  opengauss-db4 192.168.17.159  4    /opt/gaussdb/install/data/cm/cm_server Standby

[   Cluster State   ]

cluster_state   : Normal
redistributing  : No
balanced        : Yes
current_az      : AZ_ALL

[  Datanode State   ]

node             node_ip         instance                           state            
-------------------------------------------------------------------------------------
1  opengauss-db1 192.168.17.112  6001 /opt/gaussdb/install/data/db1 P Primary Normal
2  opengauss-db2 192.168.17.113  6002 /opt/gaussdb/install/data/db1 S Standby Normal
3  opengauss-db3 192.168.17.139  6003 /opt/gaussdb/install/data/db1 S Standby Normal
4  opengauss-db4 192.168.17.159  6004 /opt/gaussdb/install/data/db1 C Cascade Standby Normal

3.5.4 集群状态检查

-- omm 用户
[omm@opengauss-db1 ~]$ cm_ctl query -v -C -i -d
[  CMServer State   ]

node             node_ip         instance                                    state
------------------------------------------------------------------------------------
1  opengauss-db1 192.168.17.112  1    /opt/gaussdb/install/data/cm/cm_server Primary
2  opengauss-db2 192.168.17.113  2    /opt/gaussdb/install/data/cm/cm_server Standby
3  opengauss-db3 192.168.17.139  3    /opt/gaussdb/install/data/cm/cm_server Standby
4  opengauss-db4 192.168.17.159  4    /opt/gaussdb/install/data/cm/cm_server Standby

[   Cluster State   ]

cluster_state   : Normal
redistributing  : No
balanced        : Yes
current_az      : AZ_ALL

[  Datanode State   ]

node             node_ip         instance                           state            
-------------------------------------------------------------------------------------
1  opengauss-db1 192.168.17.112  6001 /opt/gaussdb/install/data/db1 P Primary Normal 
2  opengauss-db2 192.168.17.113  6002 /opt/gaussdb/install/data/db1 S Standby Normal
3  opengauss-db3 192.168.17.139  6003 /opt/gaussdb/install/data/db1 S Standby Normal      
4  opengauss-db4 192.168.17.159  6004 /opt/gaussdb/install/data/db1 C Cascade Standby Normal

3.6 查询级联备库日志接收状态

-- omm 用户 级联备库操作
[root@opengauss-db4 ~]# su - omm
Last login: Mon Nov 14 17:05:30 CST 2022
[omm@opengauss-db4 ~]$ gsql -d postgres -p 26000
gsql ((openGauss 3.1.0 build 4e931f9a) compiled at 2022-09-29 14:19:24 commit 0 last mr  )
Non-SSL connection (SSL connection is recommended when requiring high-security)
Type "help" for help.
openGauss=# \x
Expanded display is on.
openGauss=# select * from pg_stat_get_wal_receiver();

四、集群缩容

4.1 查询集群状态

-- omm 用户
[omm@opengauss-db1 ~]$ gs_om -t status --detail
[  CMServer State   ]

node             node_ip         instance                                    state
------------------------------------------------------------------------------------
1  opengauss-db1 192.168.17.112  1    /opt/gaussdb/install/data/cm/cm_server Primary
2  opengauss-db2 192.168.17.113  2    /opt/gaussdb/install/data/cm/cm_server Standby
3  opengauss-db3 192.168.17.139  3    /opt/gaussdb/install/data/cm/cm_server Standby
4  opengauss-db4 192.168.17.159  4    /opt/gaussdb/install/data/cm/cm_server Standby

[   Cluster State   ]

cluster_state   : Normal
redistributing  : No
balanced        : Yes
current_az      : AZ_ALL

[  Datanode State   ]

node             node_ip         instance                           state            
-------------------------------------------------------------------------------------
1  opengauss-db1 192.168.17.112  6001 /opt/gaussdb/install/data/db1 P Primary Normal
2  opengauss-db2 192.168.17.113  6002 /opt/gaussdb/install/data/db1 S Standby Normal
3  opengauss-db3 192.168.17.139  6003 /opt/gaussdb/install/data/db1 S Standby Normal
4  opengauss-db4 192.168.17.159  6004 /opt/gaussdb/install/data/db1 C Cascade Standby Normal

4.2 缩容节点

-- omm 用户 主节点操作
[omm@opengauss-db1 ~]$ gs_dropnode -U omm -G dbgrp -h 192.168.17.159
-- 执行结果如下
The target node to be dropped is (['opengauss-db4']) 
Do you want to continue to drop the target node (yes/no)?yes   -- 输入 yes
Drop node start with CM node.
Drop node with CM node is running.
[gs_dropnode]Start to drop nodes of the cluster.
[gs_dropnode]Start to stop the target node opengauss-db4.
[gs_dropnode]End of stop the target node opengauss-db4.
[gs_dropnode]Start to backup parameter config file on opengauss-db1.
[gs_dropnode]End to backup parameter config file on opengauss-db1.
[gs_dropnode]The backup file of opengauss-db1 is /opt/gaussdb/tmp/gs_dropnode_backup20221114182028/parameter_opengauss-db1.tar
[gs_dropnode]Start to parse parameter config file on opengauss-db1.
[gs_dropnode]End to parse parameter config file on opengauss-db1.
[gs_dropnode]Start to parse backup parameter config file on opengauss-db1.
[gs_dropnode]End to parse backup parameter config file opengauss-db1.
[gs_dropnode]Start to set openGauss config file on opengauss-db1.
[gs_dropnode]End of set openGauss config file on opengauss-db1.
[gs_dropnode]Start to backup parameter config file on opengauss-db2.
[gs_dropnode]End to backup parameter config file on opengauss-db2.
[gs_dropnode]The backup file of opengauss-db2 is /opt/gaussdb/tmp/gs_dropnode_backup20221114182030/parameter_opengauss-db2.tar
[gs_dropnode]Start to parse parameter config file on opengauss-db2.
[gs_dropnode]End to parse parameter config file on opengauss-db2.
[gs_dropnode]Start to parse backup parameter config file on opengauss-db2.
[gs_dropnode]End to parse backup parameter config file opengauss-db2.
[gs_dropnode]Start to set openGauss config file on opengauss-db2.
[gs_dropnode]End of set openGauss config file on opengauss-db2.
[gs_dropnode]Start to backup parameter config file on opengauss-db3.
[gs_dropnode]End to backup parameter config file on opengauss-db3.
[gs_dropnode]The backup file of opengauss-db3 is /opt/gaussdb/tmp/gs_dropnode_backup20221114182036/parameter_opengauss-db3.tar
[gs_dropnode]Start to parse parameter config file on opengauss-db3.
[gs_dropnode]End to parse parameter config file on opengauss-db3.
[gs_dropnode]Start to parse backup parameter config file on opengauss-db3.
[gs_dropnode]End to parse backup parameter config file opengauss-db3.
[gs_dropnode]Start to set openGauss config file on opengauss-db3.
[gs_dropnode]End of set openGauss config file on opengauss-db3.
[gs_dropnode]Start of set pg_hba config file on opengauss-db1.
[gs_dropnode]End of set pg_hba config file on opengauss-db1.
[gs_dropnode]Start of set pg_hba config file on opengauss-db2.
[gs_dropnode]End of set pg_hba config file on opengauss-db2.
[gs_dropnode]Start of set pg_hba config file on opengauss-db3.
[gs_dropnode]End of set pg_hba config file on opengauss-db3.
[gs_dropnode]Start to set repl slot on opengauss-db1.
[gs_dropnode]Start to get repl slot on opengauss-db1.
[gs_dropnode]End of set repl slot on opengauss-db1.
Stopping node.
=========================================
Successfully stopped node.
=========================================
End stop node.
Generate drop flag file on drop node opengauss-db4 successfully.
[gs_dropnode]Start to modify the cluster static conf.
[gs_dropnode]End of modify the cluster static conf.
Restart cluster ...
Stopping cluster.
=========================================
Successfully stopped cluster.
=========================================
End stop cluster.
Remove dynamic_config_file and CM metadata directory on all nodes.
Starting cluster.
======================================================================
Successfully started primary instance. Wait for standby instance.
======================================================================
.
Successfully started cluster.
======================================================================
cluster_state      : Normal
redistributing     : No
node_count         : 3
Datanode State
    primary           : 1
    standby           : 2
    secondary         : 0
    cascade_standby   : 0
    building          : 0
    abnormal          : 0
    down              : 0

Failed to reset switchover the cluster. Command: "source /home/omm/.bashrc ; cm_ctl switchover -a -t 0".
Output: "cm_ctl: create ssl connection failed.
cm_ctl: socket is [4], 462 : create ssl failed: 
cm_ctl: send switchover msg to cm_server, connect fail node_id:0, data_path:.".
[gs_dropnode] Success to drop the target nodes.

4.3 再次查询集群状态

-- omm 用户
[omm@opengauss-db1 ~]$ gs_om -t status --detail
[  CMServer State   ]

node             node_ip         instance                                    state
------------------------------------------------------------------------------------
1  opengauss-db1 192.168.17.112  1    /opt/gaussdb/install/data/cm/cm_server Primary
2  opengauss-db2 192.168.17.113  2    /opt/gaussdb/install/data/cm/cm_server Standby
3  opengauss-db3 192.168.17.139  3    /opt/gaussdb/install/data/cm/cm_server Standby

[   Cluster State   ]

cluster_state   : Normal
redistributing  : No
balanced        : Yes
current_az      : AZ_ALL

[  Datanode State   ]

node             node_ip         instance                           state            
-------------------------------------------------------------------------------------
1  opengauss-db1 192.168.17.112  6001 /opt/gaussdb/install/data/db1 P Primary Normal
2  opengauss-db2 192.168.17.113  6002 /opt/gaussdb/install/data/db1 S Standby Normal
3  opengauss-db3 192.168.17.139  6003 /opt/gaussdb/install/data/db1 S Standby Normal

4.4 删除SSH互信

-- omm 用户，被缩容节点操作
[root@opengauss-db4 ~]# mv .ssh .ssh_back
[root@opengauss-db4 ~]# su - omm
Last login: Mon Nov 14 17:33:17 CST 2022 on pts/0
[omm@opengauss-db4 ~]$ mv .ssh .ssh_back

4.5 清理备库数据

-- omm 用户 在被清理备库服务器操作
[root@opengauss-db4 ~]# su - omm
Last login: Mon Nov 14 18:28:56 CST 2022 on pts/3
[omm@opengauss-db4 ~]$ gs_uninstall --delete-data -L
This is a node where the gs_dropnode command has been executed. Uninstall a single node instead of the gs_dropnode command.
Checking uninstallation.
Successfully checked uninstallation.
Stopping the cluster.
Successfully stopped local node.
Successfully deleted instances.
Uninstalling application.
Successfully uninstalled application.
Uninstallation succeeded.

4.6 清理备库软件

-- 拷贝主库 /opt/software/openGauss/script 目录至被清理备库
[root@opengauss-db1 ~]# cd /opt/software/openGauss/script/
[root@opengauss-db1 script]# scp * root@opengauss-db4:/opt/software/openGauss/script/

-- root 用户 在被清理备库操作执行清理备库软件及目录
[root@opengauss-db4 ~]# cd /opt/software/openGauss/script/
[root@opengauss-db4 script]# ./gs_postuninstall -U omm -X /opt/software/openGauss/clusterconfig.xml --delete-user --delete-group -L
Parsing the configuration file.
Successfully parsed the configuration file.
Check log file path.
Successfully checked log file path.
Checking unpreinstallation.
Successfully checked unpreinstallation.
Deleting the instance's directory.
Successfully deleted the instance's directory.
Deleting the temporary directory.
Successfully deleted the temporary directory.
Deleting software packages and environmental variables of the local node.
Successfully deleted software packages and environmental variables of the local nodes.
Deleting local OS user.
Successfully deleted local OS user.
Deleting local node's logs.
Successfully deleted local node's logs.
Successfully cleaned environment.

4.7 清理备库目录

-- 根据clusterconfig.xml文件，清理备库相应目录
[root@opengauss-db4 opt]# rm -rf /opt/gaussdb/
[root@opengauss-db4 ~]# rm -rf /root/gauss*