原 MPP架构之Greenplum的安装配置初级版
发布日期 · 已更新  
Tags: 原创GreenPlum安装配置分布式数据库MPP数据仓库
简介
Greenplum是一个面向数据仓库应用的关系型数据库,因为有良好的体系结构,所以在数据存储、高并发、高可用、线性扩展、反应速度、易用性和性价比等方面有非常明显的优势。Greenplum是一种基于PostgreSQL的分布式数据库,其采用sharednothing架构,主机、操作系统、内存、存储都是自我控制的,不存在共享。
本质上讲Greenplum是一个关系型数据库集群,它实际上是由数个独立的数据库服务组合成的逻辑数据库。与RAC不同,这种数据库集群采取的是MPP(Massively Parallel Processing)架构。跟MySQL、Oracle 等关系型数据不同,Greenplum可以理解为分布式关系型数据库。
关于Greenplum的更多信息请访问https://greenplum.org/
下载
可以从 Greenplum 的 GitHub 页面(https://github.com/greenplum-db/gpdb/releases)下载RPM 包,或注册并登录到 Pivotal 公司官网(https://network.pivotal.io/products/vmware-tanzu-greenplum)进行下载。
安装包大约65MB,如下:
1 | wget https://github.com/greenplum-db/gpdb/releases/download/6.19.3/open-source-greenplum-db-6.19.3-rhel7-x86_64.rpm |
环境
本文以1个master,2个segment的集群示例,OS均为CentOS 7.6:
172.72.6.40 master lhrpg40
172.72.6.41 segment1 lhrpg41
172.72.6.42 segment2 lhrpg42
只为两个segment节点配置高可用的mirror节点,master的高可用可以使用流复制实现。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 | -- 网卡 docker network create --subnet=172.72.0.0/16 lhrnw docker rm -f lhrgp40 docker run -d --name lhrgp40 -h lhrgp40 \ --net=lhrnw --ip 172.72.6.40 \ -p 64340:5432 \ -v /sys/fs/cgroup:/sys/fs/cgroup \ --privileged=true lhrbest/lhrcentos76:9.0 \ /usr/sbin/init docker rm -f lhrgp41 docker run -d --name lhrgp41 -h lhrgp41 \ --net=lhrnw --ip 172.72.6.41 \ -p 64341:5432 \ -v /sys/fs/cgroup:/sys/fs/cgroup \ --privileged=true lhrbest/lhrcentos76:9.0 \ /usr/sbin/init docker rm -f lhrgp42 docker run -d --name lhrgp42 -h lhrgp42 \ --net=lhrnw --ip 172.72.6.42 \ -p 64342:5432 \ -v /sys/fs/cgroup:/sys/fs/cgroup \ --privileged=true lhrbest/lhrcentos76:9.0 \ /usr/sbin/init docker cp /soft/open-source-greenplum-db-6.19.3-rhel7-x86_64.rpm lhrgp40:/soft/ docker cp /soft/open-source-greenplum-db-6.19.3-rhel7-x86_64.rpm lhrgp41:/soft/ docker cp /soft/open-source-greenplum-db-6.19.3-rhel7-x86_64.rpm lhrgp42:/soft/ |
修改/etc/hosts文件
在Greenplum中,习惯将Master机器叫做mdw,将Segment机器叫做sdw。dw的含义为Data Warehouse。
1 2 3 | 172.72.6.40 lhrpg40 mdw 172.72.6.41 lhrpg41 sdw1 172.72.6.42 lhrpg42 sdw2 |
mdw和sdw只是一个主机的别名,不影响程序去查找IP。
创建用户和集群配置文件
为所有节点创建gpadmin用户:
1 2 3 4 | groupadd -g 530 gpadmin useradd -g 530 -u 530 -m -d /home/gpadmin -s /bin/bash gpadmin chown -R gpadmin:gpadmin /home/gpadmin echo "gpadmin:lhr" | chpasswd |
为所有的节点创建一个all_hosts文件,包含所有节点主机名:
1 2 3 4 5 6 7 8 9 10 | su - gpadmin mkdir -p /home/gpadmin/conf/ cat > /home/gpadmin/conf/all_hosts <<"EOF" lhrgp40 lhrgp41 lhrgp42 EOF |
为所有的节点创建一个 seg_hosts文件 ,包含所有的Segment Host的主机名:
1 2 3 4 5 6 | cat > /home/gpadmin/conf/seg_hosts <<"EOF" lhrgp41 lhrgp42 EOF |
配置互信
只在master节点操作:
1 2 | ./sshUserSetup.sh -user gpadmin -hosts "lhrgp40 lhrgp41 lhrgp42" -advanced exverify –confirm chmod 600 /home/gpadmin/.ssh/config |
安装GP
在所有节点操作:
1 2 3 4 5 6 | wget https://github.com/greenplum-db/gpdb/releases/download/6.19.3/open-source-greenplum-db-6.19.3-rhel7-x86_64.rpm yum install -y apr apr-util bash bzip2 curl krb5 libcurl libevent libxml2 libyaml zlib openldap openssh openssl openssl-libs perl readline rsync R sed tar zip krb5-devel rpm -ivh open-source-greenplum-db-6.19.3-rhel7-x86_64.rpm |
默认的安装路径是/usr/local,修改该路径gpadmin操作权限:
1 | chown -R gpadmin:gpadmin /usr/local/greenplum-db |
创建目录,用作集群数据的存储目录:
1 2 | mkdir -p /opt/greenplum/data/ chown -R gpadmin:gpadmin /opt/greenplum |
配置环境变量
1 2 3 4 5 6 | -- 所有节点 echo ". /usr/local/greenplum-db/greenplum_path.sh" >> /home/gpadmin/.bashrc -- master配置 echo "export MASTER_DATA_DIRECTORY=/opt/greenplum/data/master/gpseg-1" >> /home/gpadmin/.bashrc |
数据库初始化
Greenplum 配置文件模板都在/usr/local/greenplum-db/docs/cli_help/gpconfigs
目录下,其中gpinitsystem_config是初始化 Greenplum 的模板。
在master节点操作:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | [gpadmin@lhrgp40 ~]$ cd /usr/local/greenplum-db/docs/cli_help/gpconfigs [gpadmin@lhrgp40 gpconfigs]$ ll total 52 -rw-r--r-- 1 root root 2422 Feb 26 03:09 gpinitsystem_config -rw-r--r-- 1 root root 4511 Feb 26 03:09 gpinitsystem_singlenode -rw-r--r-- 1 root root 2321 Feb 26 03:09 gpinitsystem_test -rw-r--r-- 1 root root 359 Feb 26 03:09 hostfile_exkeys -rw-r--r-- 1 root root 119 Feb 26 03:09 hostfile_gpchecknet_ic1 -rw-r--r-- 1 root root 119 Feb 26 03:09 hostfile_gpchecknet_ic2 -rw-r--r-- 1 root root 87 Feb 26 03:09 hostfile_gpcheckperf -rw-r--r-- 1 root root 255 Feb 26 03:09 hostfile_gpexpand -rw-r--r-- 1 root root 237 Feb 26 03:09 hostfile_gpinitsystem -rw-r--r-- 1 root root 96 Feb 26 03:09 hostfile_gpssh_allhosts -rw-r--r-- 1 root root 87 Feb 26 03:09 hostfile_gpssh_segonly -rw-r--r-- 1 root root 44 Feb 26 03:09 hostlist_singlenode |
在master节点操作:创建一个初始化副本 initgp_config,根据前面的配置,修改参数
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | -- 在所有节点操作(在主节点创建master目录,在从节点分布创建primary目录和mirror目录) 或 3个目录创建都可以 su - gpadmin mkdir -p /opt/greenplum/data/master mkdir -p /opt/greenplum/data/primary mkdir -p /opt/greenplum/data/mirror -- master节点 cat > /home/gpadmin/conf/initgp_config <<"EOF" declare -a DATA_DIRECTORY=(/opt/greenplum/data/primary) declare -a MIRROR_DATA_DIRECTORY=(/opt/greenplum/data/mirror) ARRAY_NAME="lhrgp" SEG_PREFIX=gpseg PORT_BASE=6000 MASTER_PORT=5432 MASTER_HOSTNAME=lhrgp40 MASTER_DIRECTORY=/opt/greenplum/data/master DATABASE_NAME=lhrgpdb MACHINE_LIST_FILE=/home/gpadmin/conf/seg_hosts EOF |
在master节点操作:执行初始化命令:
1 2 | su - gpadmin gpinitsystem -c /home/gpadmin/conf/initgp_config -h /home/gpadmin/conf/seg_hosts |
初始化数据库时根据脚本的提示操作即可,如果配置有问题,gpinitsystem 命令就不能运行成功,错误日志存储在
/home/gpadmin/gpAdminLogs
中,需要认真查看日志报错信息,
修改正确后再重新安装。
若初始化失败,需要删除/opt/greenplum/data
下的数据资源目录重新初始化。
初始化过程:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 | [gpadmin@lhrgp40 ~]$ gpinitsystem -c /home/gpadmin/conf/initgp_config -h /home/gpadmin/conf/seg_hosts 20220308:10:55:22:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Checking configuration parameters, please wait... 20220308:10:55:22:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Reading Greenplum configuration file /home/gpadmin/conf/initgp_config 20220308:10:55:22:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Locale has not been set in /home/gpadmin/conf/initgp_config, will set to default value 20220308:10:55:22:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Locale set to en_US.utf8 20220308:10:55:23:005944 gpinitsystem:lhrgp40:gpadmin-[WARN]:-CHECK_POINT_SEGMENTS variable not set, will set to default value 20220308:10:55:23:005944 gpinitsystem:lhrgp40:gpadmin-[WARN]:-ENCODING variable not set, will set to default UTF-8 20220308:10:55:23:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-MASTER_MAX_CONNECT not set, will set to default value 250 20220308:10:55:23:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Checking configuration parameters, Completed 20220308:10:55:23:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Commencing multi-home checks, please wait... .. 20220308:10:55:23:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Configuring build for standard array 20220308:10:55:23:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Commencing multi-home checks, Completed 20220308:10:55:23:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Building primary segment instance array, please wait... .. 20220308:10:55:24:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Checking Master host 20220308:10:55:25:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Checking new segment hosts, please wait... .. 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Checking new segment hosts, Completed 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Greenplum Database Creation Parameters 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:--------------------------------------- 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Master Configuration 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:--------------------------------------- 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Master instance name = lhrgp 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Master hostname = lhrgp40 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Master port = 5432 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Master instance dir = /opt/greenplum/data/master/gpseg-1 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Master LOCALE = en_US.utf8 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Greenplum segment prefix = gpseg 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Master Database = lhrgpdb 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Master connections = 250 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Master buffers = 128000kB 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Segment connections = 750 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Segment buffers = 128000kB 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Checkpoint segments = 8 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Encoding = UTF-8 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Postgres param file = Off 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Initdb to be used = /usr/local/greenplum-db-6.19.3/bin/initdb 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-GP_LIBRARY_PATH is = /usr/local/greenplum-db-6.19.3/lib 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-HEAP_CHECKSUM is = on 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-HBA_HOSTNAMES is = 0 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Ulimit check = Passed 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Array host connect type = Single hostname per node 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Master IP address [1] = 172.72.6.40 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Standby Master = Not Configured 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Number of primary segments = 1 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Total Database segments = 2 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Trusted shell = /bin/ssh 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Number segment hosts = 2 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Mirroring config = OFF 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:---------------------------------------- 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Greenplum Primary Segment Configuration 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:---------------------------------------- 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-lhrgp41 6000 lhrgp41 /opt/greenplum/data/primary/gpseg0 2 20220308:10:55:29:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-lhrgp42 6000 lhrgp42 /opt/greenplum/data/primary/gpseg1 3 Continue with Greenplum creation Yy|Nn (default=N): > y 20220308:10:56:32:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Building the Master instance database, please wait... 20220308:10:56:41:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Starting the Master in admin mode 20220308:10:56:42:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Commencing parallel build of primary segment instances 20220308:10:56:42:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Spawning parallel processes batch [1], please wait... .. 20220308:10:56:42:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Waiting for parallel processes batch [1], please wait... ................ 20220308:10:56:58:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:------------------------------------------------ 20220308:10:56:58:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Parallel process exit status 20220308:10:56:58:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:------------------------------------------------ 20220308:10:56:58:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Total processes marked as completed = 2 20220308:10:56:58:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Total processes marked as killed = 0 20220308:10:56:58:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Total processes marked as failed = 0 20220308:10:56:58:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:------------------------------------------------ 20220308:10:56:58:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Removing back out file 20220308:10:56:58:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-No errors generated from parallel processes 20220308:10:56:58:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Restarting the Greenplum instance in production mode 20220308:10:56:58:008288 gpstop:lhrgp40:gpadmin-[INFO]:-Starting gpstop with args: -a -l /home/gpadmin/gpAdminLogs -m -d /opt/greenplum/data/master/gpseg-1 20220308:10:56:58:008288 gpstop:lhrgp40:gpadmin-[INFO]:-Gathering information and validating the environment... 20220308:10:56:58:008288 gpstop:lhrgp40:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information 20220308:10:56:58:008288 gpstop:lhrgp40:gpadmin-[INFO]:-Obtaining Segment details from master... 20220308:10:56:58:008288 gpstop:lhrgp40:gpadmin-[INFO]:-Greenplum Version: 'postgres (Greenplum Database) 6.19.3 build commit:33e10eef63ae25aeb33afbefdda46b6367353800 Open Source' 20220308:10:56:58:008288 gpstop:lhrgp40:gpadmin-[INFO]:-Commencing Master instance shutdown with mode='smart' 20220308:10:56:58:008288 gpstop:lhrgp40:gpadmin-[INFO]:-Master segment instance directory=/opt/greenplum/data/master/gpseg-1 20220308:10:56:58:008288 gpstop:lhrgp40:gpadmin-[INFO]:-Stopping master segment and waiting for user connections to finish ... server shutting down 20220308:10:56:59:008288 gpstop:lhrgp40:gpadmin-[INFO]:-Attempting forceful termination of any leftover master process 20220308:10:56:59:008288 gpstop:lhrgp40:gpadmin-[INFO]:-Terminating processes for segment /opt/greenplum/data/master/gpseg-1 20220308:10:57:00:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Starting gpstart with args: -a -l /home/gpadmin/gpAdminLogs -d /opt/greenplum/data/master/gpseg-1 20220308:10:57:00:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Gathering information and validating the environment... 20220308:10:57:00:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Greenplum Binary Version: 'postgres (Greenplum Database) 6.19.3 build commit:33e10eef63ae25aeb33afbefdda46b6367353800 Open Source' 20220308:10:57:00:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Greenplum Catalog Version: '301908232' 20220308:10:57:00:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Starting Master instance in admin mode 20220308:10:57:00:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information 20220308:10:57:00:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Obtaining Segment details from master... 20220308:10:57:00:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Setting new master era 20220308:10:57:00:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Master Started... 20220308:10:57:01:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Shutting down master 20220308:10:57:01:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Commencing parallel segment instance startup, please wait... . 20220308:10:57:02:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Process results... 20220308:10:57:02:008313 gpstart:lhrgp40:gpadmin-[INFO]:----------------------------------------------------- 20220308:10:57:02:008313 gpstart:lhrgp40:gpadmin-[INFO]:- Successful segment starts = 2 20220308:10:57:02:008313 gpstart:lhrgp40:gpadmin-[INFO]:- Failed segment starts = 0 20220308:10:57:02:008313 gpstart:lhrgp40:gpadmin-[INFO]:- Skipped segment starts (segments are marked down in configuration) = 0 20220308:10:57:02:008313 gpstart:lhrgp40:gpadmin-[INFO]:----------------------------------------------------- 20220308:10:57:02:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Successfully started 2 of 2 segment instances 20220308:10:57:02:008313 gpstart:lhrgp40:gpadmin-[INFO]:----------------------------------------------------- 20220308:10:57:02:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Starting Master instance lhrgp40 directory /opt/greenplum/data/master/gpseg-1 20220308:10:57:03:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Command pg_ctl reports Master lhrgp40 instance active 20220308:10:57:03:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Connecting to dbname='template1' connect_timeout=15 20220308:10:57:03:008313 gpstart:lhrgp40:gpadmin-[INFO]:-No standby master configured. skipping... 20220308:10:57:03:008313 gpstart:lhrgp40:gpadmin-[INFO]:-Database successfully started 20220308:10:57:03:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Completed restart of Greenplum instance in production mode 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Scanning utility log file for any warning messages 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[WARN]:-******************************************************* 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[WARN]:-Scan of log file indicates that some warnings or errors 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[WARN]:-were generated during the array creation 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Please review contents of log file 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-/home/gpadmin/gpAdminLogs/gpinitsystem_20220308.log 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-To determine level of criticality 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-These messages could be from a previous run of the utility 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-that was called today! 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[WARN]:-******************************************************* 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Greenplum Database instance successfully created 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:------------------------------------------------------- 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-To complete the environment configuration, please 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-update gpadmin .bashrc file with the following 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-1. Ensure that the greenplum_path.sh file is sourced 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-2. Add "export MASTER_DATA_DIRECTORY=/opt/greenplum/data/master/gpseg-1" 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:- to access the Greenplum scripts for this instance: 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:- or, use -d /opt/greenplum/data/master/gpseg-1 option for the Greenplum scripts 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:- Example gpstate -d /opt/greenplum/data/master/gpseg-1 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Script log file = /home/gpadmin/gpAdminLogs/gpinitsystem_20220308.log 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-To remove instance, run gpdeletesystem utility 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-To initialize a Standby Master Segment for this Greenplum instance 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Review options for gpinitstandby 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:------------------------------------------------------- 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-The Master /opt/greenplum/data/master/gpseg-1/pg_hba.conf post gpinitsystem 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-has been configured to allow all hosts within this new 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-array to intercommunicate. Any hosts external to this 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-new array must be explicitly added to this file 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-Refer to the Greenplum Admin support guide which is 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:-located in the /usr/local/greenplum-db-6.19.3/docs directory 20220308:10:57:05:005944 gpinitsystem:lhrgp40:gpadmin-[INFO]:------------------------------------------------------- |
GP验证
若初始化成功,则GP自动启动,可以看到master节点上的5432已经在listen了,psql进入数据库,开始greenplum之旅。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 | [gpadmin@lhrgp40 ~]$ netstat -tulnp | grep 5432 tcp 0 0 0.0.0.0:5432 0.0.0.0:* LISTEN 8362/postgres tcp6 0 0 :::5432 :::* LISTEN 8362/postgres [gpadmin@lhrgp40 ~]$ psql -d lhrgpdb psql (9.4.26) Type "help" for help. lhrgpdb=# \l List of databases Name | Owner | Encoding | Collate | Ctype | Access privileges -----------+---------+----------+------------+------------+--------------------- lhrgpdb | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 | postgres | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 | template0 | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 | =c/gpadmin + | | | | | gpadmin=CTc/gpadmin template1 | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 | =c/gpadmin + | | | | | gpadmin=CTc/gpadmin (4 rows) lhrgpdb=# show port; port ------ 5432 (1 row) lhrgpdb=# show listen_addresses; listen_addresses ------------------ * (1 row) lhrgpdb=# select * from gp_segment_configuration order by 1; dbid | content | role | preferred_role | mode | status | port | hostname | address | datadir ------+---------+------+----------------+------+--------+------+----------+---------+------------------------------------ 1 | -1 | p | p | n | u | 5432 | lhrgp40 | lhrgp40 | /opt/greenplum/data/master/gpseg-1 2 | 0 | p | p | n | u | 6000 | lhrgp41 | lhrgp41 | /opt/greenplum/data/primary/gpseg0 3 | 1 | p | p | n | u | 6000 | lhrgp42 | lhrgp42 | /opt/greenplum/data/primary/gpseg1 (3 rows) [gpadmin@lhrgp40 ~]$ gpstate 20220308:11:25:54:010475 gpstate:lhrgp40:gpadmin-[INFO]:-Starting gpstate with args: 20220308:11:25:54:010475 gpstate:lhrgp40:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.19.3 build commit:33e10eef63ae25aeb33afbefdda46b6367353800 Open Source' 20220308:11:25:54:010475 gpstate:lhrgp40:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.26 (Greenplum Database 6.19.3 build commit:33e10eef63ae25aeb33afbefdda46b6367353800 Open Source) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Feb 24 2022 23:24:35' 20220308:11:25:54:010475 gpstate:lhrgp40:gpadmin-[INFO]:-Obtaining Segment details from master... 20220308:11:25:54:010475 gpstate:lhrgp40:gpadmin-[INFO]:-Gathering data from segments... 20220308:11:25:54:010475 gpstate:lhrgp40:gpadmin-[INFO]:-Greenplum instance status summary 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:----------------------------------------------------- 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Master instance = Active 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Master standby = No master standby configured 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Total segment instance count from metadata = 2 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:----------------------------------------------------- 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Primary Segment Status 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:----------------------------------------------------- 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Total primary segments = 2 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Total primary segment valid (at master) = 2 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Total primary segment failures (at master) = 0 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Total number of postmaster.pid files missing = 0 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Total number of postmaster.pid files found = 2 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Total number of postmaster.pid PIDs missing = 0 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Total number of postmaster.pid PIDs found = 2 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Total number of /tmp lock files missing = 0 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Total number of /tmp lock files found = 2 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Total number postmaster processes missing = 0 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Total number postmaster processes found = 2 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:----------------------------------------------------- 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Mirror Segment Status 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:----------------------------------------------------- 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:- Mirrors not configured on this array 20220308:11:25:55:010475 gpstate:lhrgp40:gpadmin-[INFO]:----------------------------------------------------- |
启动和关闭
使用gpstart -a 启动greenplum;使用gpstop -a关闭greenplum:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 | [gpadmin@lhrgp40 ~]$ gpstop -a 20220308:11:06:34:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Starting gpstop with args: -a 20220308:11:06:34:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Gathering information and validating the environment... 20220308:11:06:34:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information 20220308:11:06:34:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Obtaining Segment details from master... 20220308:11:06:34:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Greenplum Version: 'postgres (Greenplum Database) 6.19.3 build commit:33e10eef63ae25aeb33afbefdda46b6367353800 Open Source' 20220308:11:06:34:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Commencing Master instance shutdown with mode='smart' 20220308:11:06:34:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Master segment instance directory=/opt/greenplum/data/master/gpseg-1 20220308:11:06:34:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Stopping master segment and waiting for user connections to finish ... server shutting down 20220308:11:06:35:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Attempting forceful termination of any leftover master process 20220308:11:06:35:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Terminating processes for segment /opt/greenplum/data/master/gpseg-1 20220308:11:06:35:009117 gpstop:lhrgp40:gpadmin-[INFO]:-No standby master host configured 20220308:11:06:35:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Targeting dbid [2, 3] for shutdown 20220308:11:06:35:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Commencing parallel segment instance shutdown, please wait... 20220308:11:06:35:009117 gpstop:lhrgp40:gpadmin-[INFO]:-0.00% of jobs completed 20220308:11:06:36:009117 gpstop:lhrgp40:gpadmin-[INFO]:-100.00% of jobs completed 20220308:11:06:36:009117 gpstop:lhrgp40:gpadmin-[INFO]:----------------------------------------------------- 20220308:11:06:36:009117 gpstop:lhrgp40:gpadmin-[INFO]:- Segments stopped successfully = 2 20220308:11:06:36:009117 gpstop:lhrgp40:gpadmin-[INFO]:- Segments with errors during stop = 0 20220308:11:06:36:009117 gpstop:lhrgp40:gpadmin-[INFO]:----------------------------------------------------- 20220308:11:06:36:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Successfully shutdown 2 of 2 segment instances 20220308:11:06:36:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Database successfully shutdown with no errors reported 20220308:11:06:36:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Cleaning up leftover gpmmon process 20220308:11:06:36:009117 gpstop:lhrgp40:gpadmin-[INFO]:-No leftover gpmmon process found 20220308:11:06:36:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Cleaning up leftover gpsmon processes 20220308:11:06:36:009117 gpstop:lhrgp40:gpadmin-[INFO]:-No leftover gpsmon processes on some hosts. not attempting forceful termination on these hosts 20220308:11:06:36:009117 gpstop:lhrgp40:gpadmin-[INFO]:-Cleaning up leftover shared memory [gpadmin@lhrgp40 ~]$ gpstart -a 20220308:11:06:41:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Starting gpstart with args: -a 20220308:11:06:41:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Gathering information and validating the environment... 20220308:11:06:41:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Greenplum Binary Version: 'postgres (Greenplum Database) 6.19.3 build commit:33e10eef63ae25aeb33afbefdda46b6367353800 Open Source' 20220308:11:06:41:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Greenplum Catalog Version: '301908232' 20220308:11:06:41:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Starting Master instance in admin mode 20220308:11:06:41:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information 20220308:11:06:41:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Obtaining Segment details from master... 20220308:11:06:41:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Setting new master era 20220308:11:06:41:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Master Started... 20220308:11:06:42:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Shutting down master 20220308:11:06:42:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Commencing parallel segment instance startup, please wait... . 20220308:11:06:44:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Process results... 20220308:11:06:44:009285 gpstart:lhrgp40:gpadmin-[INFO]:----------------------------------------------------- 20220308:11:06:44:009285 gpstart:lhrgp40:gpadmin-[INFO]:- Successful segment starts = 2 20220308:11:06:44:009285 gpstart:lhrgp40:gpadmin-[INFO]:- Failed segment starts = 0 20220308:11:06:44:009285 gpstart:lhrgp40:gpadmin-[INFO]:- Skipped segment starts (segments are marked down in configuration) = 0 20220308:11:06:44:009285 gpstart:lhrgp40:gpadmin-[INFO]:----------------------------------------------------- 20220308:11:06:44:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Successfully started 2 of 2 segment instances 20220308:11:06:44:009285 gpstart:lhrgp40:gpadmin-[INFO]:----------------------------------------------------- 20220308:11:06:44:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Starting Master instance lhrgp40 directory /opt/greenplum/data/master/gpseg-1 20220308:11:06:44:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Command pg_ctl reports Master lhrgp40 instance active 20220308:11:06:44:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Connecting to dbname='template1' connect_timeout=15 20220308:11:06:44:009285 gpstart:lhrgp40:gpadmin-[INFO]:-No standby master configured. skipping... 20220308:11:06:44:009285 gpstart:lhrgp40:gpadmin-[INFO]:-Database successfully started |
配置远程登录
1、本地登陆修改密码
1 2 3 4 5 6 7 8 | [gpadmin@lhrgp40 gpseg-1]$ psql -d lhrgpdb psql (9.4.26) Type "help" for help. lhrgpdb=# \password gpadmin Enter new password: Enter it again: lhrgpdb=# |
2、修改pg_hba.conf
1 2 3 | echo "host all all all md5" >> /opt/greenplum/data/master/gpseg-1/pg_hba.conf -- 使改动生效切不中断服务gpstop -u |
3、远程登录
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | C:\Users\lhrxxt>psql -U gpadmin -h192.168.8.8 -p 64340 -d lhrgpdb Password for user gpadmin: psql (14.0, server 9.4.26) Type "help" for help. lhrgpdb=# \l List of databases Name | Owner | Encoding | Collate | Ctype | Access privileges -----------+---------+----------+------------+------------+--------------------- lhrgpdb | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 | postgres | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 | template0 | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 | =c/gpadmin + | | | | | gpadmin=CTc/gpadmin template1 | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 | =c/gpadmin + | | | | | gpadmin=CTc/gpadmin (4 rows) lhrgpdb=# select * from pg_user; usename | usesysid | usecreatedb | usesuper | usecatupd | userepl | passwd | valuntil | useconfig ---------+----------+-------------+----------+-----------+---------+----------+----------+----------- gpadmin | 10 | t | t | t | t | ******** | | (1 row) |
基本运维操作
启动数据库服务
gpstart,系统会自检,并提示是否启动服务,选择y,启动服务
gpstart -a 则系统无任何提示,进行启动
gpstart -q 如果不希望屏幕输出
gpstart -h 具体的选项帮助说明
ps -ef|grep postgre 查看相关的服务进程如何关闭数据库服务
gpstop
gpstop -M fast 想强行关闭服务
gpstop -u 重启系统
该工具提供了-t选项,增加允许的超时设置。这对系统关闭时存在大量回滚数据的情况非常有用(过去的默认超时是60秒)
gpstop -h 获取选项帮助gpstate :显示Greenplum数据库运行状态,详细配置等信息 常用可选参数:
-c:primary instance 和 mirror instance 的对应关系
-m:只列出mirror 实例的状态和配置信息
-f:显示standby master 的详细信息
-s:查看详细状态,如在同步,可显示数据同步完成百分比
--version,查看数据库version
(也可使用pg_controldata查看数据库版本和postgresql版本) 该命令默认列出数据库运行状态汇总信息,常用于日常巡检。
我们可以通过任何装有psql客户端的机器连接GP数据库,比如
1 | psql -d template1 -U gpadmin -p 5432 -h mdw |
其中-d指定了连接数据库的名称,-U指定了连接数据库的用户名,也称为角色,-p指定了连接使用的端口,默认值是5432,-h指定了master对外服务的主机名。
修改用户密码:alter role xxx with password 'xxx'
巡检
参考
https://docs.greenplum.org/6-16/install_guide/install_gpdb.html
https://blog.csdn.net/DongGeGe214/article/details/80923257
http://docs-cn.greenplum.org/v5/ref_guide/system_catalogs/catalog_ref-tables.html#topic1
相关文章
- MPP架构之Greenplum的安装配置高级版(企业配置)
- Greenplum 6和Greenplum 7基于华为欧拉openEuler22.03的编译安装和二进制安装及相关错误处理
- GreenPlum中的is null可以走索引吗?is null和''空字符串一样吗?
- 将GreenPlum数据库从6升级到7版本(大版本升级)
- GreenPlum通过gpbackup和gprestore在MinIO的S3存储中的异地备份和还原
- GreenPlum中查询主机历史CPU、内存、磁盘、网络等情况(gpmetrics.gpcc_system_history)
- GreenPlum查询表有数据,但是drop表报错does not exist,vacuumdb也报错
- GreenPlum中的union可能比union all要快很多
- gpload加载数据报错 ProgrammingError: ERROR: column "relstorage" does not exist(GP7中的pg_class没有relstorage字段)
- gpload加载数据报错 UnboundLocalError: local variable 'has_seq_bool' referenced before assignment
- GreenPlum启动报错ModuleNotFoundError: No module named 'gppylib'
- GreenPlum查询参数报错schema gp_toolkit does not exist
- GP7.2在欧拉22.03上安装的一些报错处理
- GreenPlum相关软件在博通网站的下载方式
- GreenPlum 7.2.0新特性介绍
- 在GreenPlum中配置PgBouncer连接池并使用Prometheus+Grafana对pgbouncer监控
- GreenPlum启动报错could not load library /usr/local/greenplum-db-6.27.0/lib/postgresql/pg_cron.so的故障处理
- GreenPlum 6.27.0版本新特性及pg_cron模块配置定时任务说明
- 在GreenPlum中,如果知道某个数据节点sdw1上的进程号,其CPU使用率特别高,那么如何查询master上对应的进程号呢?
- GreenPlum循环删除不需要的外部表