原 Greenplum 6和Greenplum 7基于华为欧拉openEuler22.03的编译安装和二进制安装及相关错误处理
Tags： 原创GreenPlumopenEulerGPDB 7安装配置欧拉源码编译GPDB 6华为
👉 本文共约2383个字，系统预计阅读时间或需9分钟。
目录
- 前言
- 二进制安装（推荐）
- 安装过程
- 二进制安装GreenPlum会搞坏openEuler系统的yum、rpm、dnf等命令
- 编译安装(不推荐)
- gpcc在欧拉系统上的问题
- gpcc报错rsync: symbol lookup error: rsync: undefined symbol: ZSTD_compressStream2
- gpcc 启动报错
- GP7.2在欧拉22.03上安装的一些报错
- ModuleNotFoundError: No module named 'psycopg2'
- Error: unable to import module: No module named 'psutil'
- Error: unable to import module: No module named 'pkg_resources'
- 总结
- 参考
前言
特别注意：
1、据GP中文社区资料，openEuler 20.03 LTS SP1 & SP2 软件仓库中引入 Greenplum 6.17.0 版本，参考：https://cn.greenplum.org/greenplum_openeuler_whitepaper/ ，参考：https://www.dbaup.com/greenplum-6-jiyuopeneuler22-03bianyianzhuangbuzhichi.html
GreenPlum中文社区发布称Greenplum完全兼容欧拉开源操作系统 ，参考：https://cn.greenplum.org/greenplum_openeuler_whitepaper/ ，下载：https://cn.greenplum.org/wp-content/uploads/2022/02/%E5%AE%8C%E5%85%A8%E5%85%BC%E5%AE%B9%E6%AC%A7%E6%8B%89%E5%BC%80%E6%BA%90%E6%93%8D%E4%BD%9C%E7%B3%BB%E7%BB%9F%E7%9A%84-HTAP-%E6%95%B0%E6%8D%AE%E5%B9%B3%E5%8F%B0-Greenplum.pdf
但是仓库：https://gitee.com/src-openeuler/gpdb 为空，感觉该项目可能黄了。。。
二进制安装（推荐）
使用二进制安装可以成功！！！
请参考：
https://www.dbaup.com/greenplum-6-danjianzhuang.html
在前边的安装步骤都没问题，到最后执行gpinitsystem的时候就会报错，命令/usr/local/greenplum-db-6.25.1/bin/postgres -V会报错，主要是缺失如下4个库文件：
- /usr/lib64/libssl.so.10
- /usr/lib64/libcrypto.so.10
- /usr/lib64/libldap-2.4.so.2
- /usr/lib64/libnsl.so.1
配置不对的话，会报如下的错误：
1 2 3 4 | [gpadmin@mdw ~]$ postgres -V postgres: /lib64/libcrypto.so.10: version `OPENSSL_1.0.1_EC' not found (required by postgres) postgres: /lib64/libcrypto.so.10: version `libcrypto.so.10' not found (required by postgres) postgres: /lib64/libssl.so.10: version `libssl.so.10' not found (required by postgres) |
解决办法：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | -- 二进制安装包使用RHEL8和RHEL7都可以（推荐RHEL8） open-source-greenplum-db-6.25.1-rhel8-x86_64.rpm -- 或者 open-source-greenplum-db-6.25.1-rhel7-x86_64.rpm -- 需要提前安装的包 dnf install -y libnsl apr-devel apr-util xerces-c xerces-c-devel libcgroup dnf install -y apr-util iproute krb5-devel less net-tools \ openssh openssh-clients openssh-server openssl \ perl rsync zip libcgroup-tools libnsl \ apr-devel xerces-c xerces-c-devel libcgroup \ iputils hostname findutils systemd glibc-all-langpacks \ libbpf libmnl screen sudo \ --downloadonly --destdir=/soft/ rpm -ivh apr-*.rpm --nodeps rpm -ivh apr-util*.rpm --nodeps rpm -ivh libcgroup*.rpm --nodeps rpm -ivh libnsl*.rpm --nodeps rpm -ivh compat-openssl10-1.0.2o-4.el8.x86_64.rpm --nodeps rpm -ivh findutils*.rpm hostname*.rpm iputils*.rpm rpm -ivh iproute*.rpm --nodeps rpm -ivh less*.rpm --nodeps rpm -ivh net-tools*.rpm --nodeps rpm -ivh openssl*.rpm --nodeps rpm -ivh perl*.rpm --nodeps rpm -ivh rsync*.rpm --nodeps rpm -ivh zip*.rpm --nodeps rpm -ivh krb5-devel*.rpm --nodeps # rpm -ivh systemd-2*.rpm acl*.rpm kmod-libs*.rpm libseccomp*.rpm xkeyboard-config*.rpm diffutils*.rpm libxkbcommon*.rpm --nodeps # rpm -ivh openssh*.rpm --nodeps # rpm -ivh openssh-clients*.rpm --nodeps # rpm -ivh openssh-server*.rpm --nodeps rpm -ivh libpq*.rpm --nodeps rpm -ivh python3-ps*.rpm --nodeps ln -s /usr/lib64/libldap.so.2.0.200 /usr/lib64/libldap-2.4.so.2 ln -s /usr/lib64/libldap.so.2.0.200 /usr/lib64/libldap_r-2.4.so.2 ln -s /usr/lib64/libreadline.so.8.1 /usr/lib64/libreadline.so.6 ln -s /usr/lib64/libreadline.so.8.1 /usr/lib64/libreadline.so.7 -- 若是安装rhel7版本，则还需要安装如下的包 wget https://rpmfind.net/linux/centos/8-stream/AppStream/x86_64/os/Packages/compat-openssl10-1.0.2o-4.el8.x86_64.rpm rpm -ivh compat-openssl10-1.0.2o-4.el8.x86_64.rpm |
安装过程
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 | docker rm -f openeuler2203gpdb6 docker run -itd --name openeuler2203gpdb6 -h openeuler2203gpdb6 \ --privileged=true \ -v /sys/fs/cgroup:/sys/fs/cgroup \ lhrbest/openeuler22:5.0 \ init docker exec -it openeuler2203gpdb6 bash sed -i s/SELINUX=enforcing/SELINUX=disabled/g /etc/selinux/config setenforce 0 systemctl status firewalld.service systemctl disable firewalld.service systemctl start firewalld firewall-cmd --add-port=0-65535/tcp --permanent firewall-cmd --add-port=0-65535/udp --permanent firewall-cmd --reload firewall-cmd --list-ports systemctl stop firewalld.service cat >> /etc/security/limits.conf <<"EOF" * soft nofile 655350 * hard nofile 655350 * soft nproc 655350 * hard nproc 655350 EOF cat >> /etc/sysctl.conf <<"EOF" kernel.shmmax = 4398046511104 kernel.shmmni = 4096 kernel.shmall = 4000000000 kernel.sem = 32000 1024000000 500 32000 vm.overcommit_memory=1 vm.overcommit_ratio=95 EOF sysctl -p groupadd -g 530 gpadmin useradd -g 530 -u 530 -m -d /home/gpadmin -s /bin/bash gpadmin chown -R gpadmin:gpadmin /home/gpadmin echo "gpadmin:lhr" | chpasswd -- rpm -ivh /soft/open-source-greenplum-db-6.23.1-rhel7-x86_64.rpm chown -R gpadmin:gpadmin /usr/local/greenplum-db echo ". /usr/local/greenplum-db/greenplum_path.sh" >> /home/gpadmin/.bashrc echo "export MASTER_DATA_DIRECTORY=/opt/greenplum/data/master/gpseg-1" >> /home/gpadmin/.bashrc echo "export PGDATABASE=postgres" >> /home/gpadmin/.bashrc echo "export PGPORT=5432" >> /home/gpadmin/.bashrc ./sshUserSetup.sh -user root -hosts "mdw mdw" -advanced exverify –confirm ./sshUserSetup.sh -user gpadmin -hosts "mdw mdw" -advanced exverify –confirm chmod 600 /home/gpadmin/.ssh/config mkdir -p /opt/greenplum/data/master mkdir -p /opt/greenplum/data/primary mkdir -p /opt/greenplum/data/mirror mkdir -p /opt/greenplum/data/master_standby mkdir -p /opt/greenplum/data/ chown -R gpadmin:gpadmin /opt/greenplum su - gpadmin mkdir -p /home/gpadmin/conf/ cat > /home/gpadmin/conf/all_hosts <<"EOF" openeuler2203gpdb6 EOF cat > /home/gpadmin/conf/seg_hosts <<"EOF" openeuler2203gpdb6 EOF ssh-keygen -t rsa ssh-keygen -t dsa cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys ssh openeuler2203gpdb6 date su - gpadmin cat > /home/gpadmin/conf/initgp_config <<"EOF" declare -a DATA_DIRECTORY=(/opt/greenplum/data/primary /opt/greenplum/data/primary) declare -a MIRROR_DATA_DIRECTORY=(/opt/greenplum/data/mirror /opt/greenplum/data/mirror) ARRAY_NAME="lhrgp" SEG_PREFIX=gpseg PORT_BASE=6000 MIRROR_PORT_BASE=7000 MASTER_PORT=5432 MASTER_HOSTNAME=openeuler2203gpdb6 MASTER_DIRECTORY=/opt/greenplum/data/master DATABASE_NAME=lhrgpdb MACHINE_LIST_FILE=/home/gpadmin/conf/seg_hosts EOF gpinitsystem -c /home/gpadmin/conf/initgp_config -e=lhr -s openeuler2203gpdb6 -P 5433 -S /opt/greenplum/data/master_standby/gpseg-1 |
二进制安装GreenPlum会搞坏openEuler系统的yum、rpm、dnf等命令
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | [root@mdw ~]# yum install -y python Python path configuration: PYTHONHOME = '/usr/local/greenplum-db-6.25.1/ext/python' PYTHONPATH = '/usr/local/greenplum-db-6.25.1/lib/python' program name = '/usr/bin/python3' isolated = 0 environment = 1 user site = 1 import site = 1 sys._base_executable = '/usr/bin/python3' sys.base_prefix = '/usr/local/greenplum-db-6.25.1/ext/python' sys.base_exec_prefix = '/usr/local/greenplum-db-6.25.1/ext/python' sys.platlibdir = 'lib64' sys.executable = '/usr/bin/python3' sys.prefix = '/usr/local/greenplum-db-6.25.1/ext/python' sys.exec_prefix = '/usr/local/greenplum-db-6.25.1/ext/python' sys.path = [ '/usr/local/greenplum-db-6.25.1/lib/python', '/usr/local/greenplum-db-6.25.1/ext/python/lib64/python39.zip', '/usr/local/greenplum-db-6.25.1/ext/python/lib64/python3.9', '/usr/local/greenplum-db-6.25.1/ext/python/lib64/python3.9/lib-dynload', ] Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding Python runtime state: core initialized ModuleNotFoundError: No module named 'encodings' Current thread 0x00007fcb52646000 (most recent call first): <no Python frame> [root@mdw soft]# rpm -ivh openssl-libs-1.1.1m-18.oe2203.x86_64.rpm rpm: symbol lookup error: /usr/lib64/librpmio.so.9: undefined symbol: ZSTD_compressStream2 [root@mdw lib64]# ldd /usr/lib64/librpmio.so.9|grep zstd libzstd.so.1 => /usr/local/greenplum-db-6.25.1/lib/libzstd.so.1 (0x00007f54e3988000) |
解决：环境变量问题
1 2 3 | . /usr/local/greenplum-db/greenplum_path.sh export LD_LIBRARY_PATH='' export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin |
编译安装(不推荐)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 | docker rm -f openeuler2203gpdb6 docker run -itd --name openeuler2203gpdb6 -h openeuler2203gpdb6 \ --privileged=true \ -v /sys/fs/cgroup:/sys/fs/cgroup \ lhrbest/openeuler22:2.0 \ init docker exec -it openeuler2203gpdb6 bash wget https://ghproxy.com/https://github.com/greenplum-db/gpdb/releases/download/6.25.1/6.25.1-src-full.tar.gz wget https://rpmfind.net/linux/centos/8-stream/AppStream/x86_64/os/Packages/compat-openssl10-1.0.2o-4.el8.x86_64.rpm rpm -ivh compat-openssl10-1.0.2o-4.el8.x86_64.rpm dnf install -y libnsl apr-devel xerces-c xerces-c-devel ln -s /usr/lib64/libldap-2.4.so.2 /usr/lib64/libldap_r-2.4.so.2 ln -s /usr/lib64/libldap.so.2.0.200 /usr/lib64/libldap-2.4.so.2 ln -s /usr/lib64/libreadline.so.8.1 /usr/lib64/libreadline.so.6 dnf groupinstall -y 'Development Tools' # GCC, libtools etc dnf install -y curl-devel bzip2-devel python-devel openssl-devel readline-devel libzstd-devel dnf install -y perl-ExtUtils-Embed # If enable perl dnf install -y libxml2-devel # If enable XML support dnf install -y openldap-devel # If enable LDAP dnf install -y pam pam-devel # If enable PAM dnf install -y perl-devel # If need installcheck-world dnf install -y sudo python mkdir -p ~/.pip cat > ~/.pip/pip.conf << EOF [global] timeout = 6000 index-url = http://mirrors.aliyun.com/pypi/simple/ index-index-url = http://pypi.douban.com/simple/ [install] trusted-host = mirrors.aliyun.com pypi.douban.com EOF -- 需要python2.7.18 wget https://www.python.org/ftp/python/2.7.18/Python-2.7.18.tgz tar zxf Python-2.7.18.tgz cd Python-2.7.18/ ./configure prefix=/usr/local/python2.7.18 make -j8 && make altinstall -j8 ln -s /usr/local/python2.7.18/bin/python2.7 /usr/local/python2.7.18/bin/python ln -s /usr/local/python2.7.18/bin/python2.7 /usr/local/python2.7.18/bin/python2 ln -s /usr/local/python2.7.18/bin/python2.7 /usr/bin/python cat >> ~/.bashrc <<"EOF" export LD_LIBRARY_PATH=/usr/local/python2.7.18/lib:$LD_LIBRARY_PATH export PATH=/usr/local/python2.7.18/bin:$PATH EOF source ~/.bashrc cat >> /etc/profile <<"EOF" export LD_LIBRARY_PATH=/usr/local/python2.7.18/lib:$LD_LIBRARY_PATH export PATH=/usr/local/python2.7.18/bin:$PATH EOF source /etc/profile wget https://bootstrap.pypa.io/pip/2.7/get-pip.py /usr/local/python2.7.18/bin/python2.7 get-pip.py ll /usr/local/python2.7.18/bin/pip /usr/local/python2.7.18/bin/pip install psutil lockfile paramiko setuptools epydoc /* export PATH=/soft/gpdb_src/src/bin/pg_config:$PATH /usr/local/python2.7.18/bin/pip install PyGreSQL */ tar -zxvf 6.25.1-src-full.tar.gz cd gpdb_src/ ./configure --prefix=/usr/local/gpdb --with-perl --with-python --disable-gpcloud --disable-orca make -j16 ##若成功会提示：All of Greenplum Database successfully made. Ready to install. make install ##若成功会提示：Greenplum Database installation complete. -- 测试：使用gpadmin用户 groupadd -g 530 gpadmin useradd -g 530 -u 530 -m -d /home/gpadmin -s /bin/bash gpadmin chown -R gpadmin:gpadmin /home/gpadmin echo "gpadmin:lhr" | chpasswd chown gpadmin.gpadmin -R /soft/gpdb_src/gpAux/gpdemo/ su - gpadmin ssh-keygen -t rsa ssh-keygen -t dsa cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys cd gpdb_src/gpAux/gpdemo source /usr/local/gpdb/greenplum_path.sh export PGHOST=`hostname` make source gpdemo-env.sh sed -i s/SELINUX=enforcing/SELINUX=disabled/g /etc/selinux/config setenforce 0 cat >> /etc/security/limits.conf <<"EOF" * soft nofile 655350 * hard nofile 655350 * soft nproc 655350 * hard nproc 655350 EOF cat >> /etc/sysctl.conf <<"EOF" kernel.shmmax = 4398046511104 kernel.shmmni = 4096 kernel.shmall = 4000000000 kernel.sem = 32000 1024000000 500 32000 vm.overcommit_memory=1 vm.overcommit_ratio=95 EOF sysctl -p groupadd -g 530 gpadmin useradd -g 530 -u 530 -m -d /home/gpadmin -s /bin/bash gpadmin chown -R gpadmin:gpadmin /home/gpadmin echo "gpadmin:lhr" | chpasswd -- rpm -ivh /soft/open-source-greenplum-db-6.23.1-rhel7-x86_64.rpm chown -R gpadmin:gpadmin /usr/local/gpdb echo ". /usr/local/gpdb/greenplum_path.sh" >> /home/gpadmin/.bashrc echo "export MASTER_DATA_DIRECTORY=/opt/greenplum/data/master/gpseg-1" >> /home/gpadmin/.bashrc echo "export PGDATABASE=postgres" >> /home/gpadmin/.bashrc echo "export PGPORT=5432" >> /home/gpadmin/.bashrc ssh-keygen -t rsa ssh-keygen -t dsa cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys ssh openeuler2203gpdb6 date ./sshUserSetup.sh -user root -hosts "mdw mdw" -advanced exverify –confirm ./sshUserSetup.sh -user gpadmin -hosts "mdw mdw" -advanced exverify –confirm chmod 600 /home/gpadmin/.ssh/config mkdir -p /opt/greenplum/data/master mkdir -p /opt/greenplum/data/primary mkdir -p /opt/greenplum/data/mirror mkdir -p /opt/greenplum/data/master_standby mkdir -p /opt/greenplum/data/ chown -R gpadmin:gpadmin /opt/greenplum su - gpadmin mkdir -p /home/gpadmin/conf/ cat > /home/gpadmin/conf/all_hosts <<"EOF" openeuler2203gpdb6 EOF cat > /home/gpadmin/conf/seg_hosts <<"EOF" openeuler2203gpdb6 EOF su - gpadmin cat > /home/gpadmin/conf/initgp_config <<"EOF" declare -a DATA_DIRECTORY=(/opt/greenplum/data/primary /opt/greenplum/data/primary) declare -a MIRROR_DATA_DIRECTORY=(/opt/greenplum/data/mirror /opt/greenplum/data/mirror) ARRAY_NAME="lhrgp" SEG_PREFIX=gpseg PORT_BASE=6000 MIRROR_PORT_BASE=7000 MASTER_PORT=5432 MASTER_HOSTNAME=openeuler2203gpdb6 MASTER_DIRECTORY=/opt/greenplum/data/master DATABASE_NAME=lhrgpdb MACHINE_LIST_FILE=/home/gpadmin/conf/seg_hosts EOF gpinitsystem -c /home/gpadmin/conf/initgp_config -e=lhr -s openeuler2203gpdb6 -P 5433 -S /opt/greenplum/data/master_standby/gpseg-1 |
gpcc在欧拉系统上的问题
gpcc报错rsync: symbol lookup error: rsync: undefined symbol: ZSTD_compressStream2
在欧拉系统上安装gpcc时，会报错“rsync: symbol lookup error: rsync: undefined symbol: ZSTD_compressStream2”
解决：将以下代码加入所有节点的/home/gpadmin/.bashrc中，包括mdw和smdw和所有数据节点
1 2 | echo "export LD_LIBRARY_PATH=''" >> /home/gpadmin/.bashrc export LD_LIBRARY_PATH='' |
本人提供Oracle(OCP、OCM)、MySQL(OCP)、PostgreSQL(PGCA、PGCE、PGCM)等数据库的培训和考证业务，私聊QQ646634621或微信dbaup66，谢谢！
