一、前言
openGauss 6.0.0 LTS 版本于2024年9月30日正式上线发布,openGauss 6.0.0 LTS 是社区最新发布的长期支持版本,版本生命周期为3年。本次发布包含2个数据库服务端安装版本:企业版、轻量版。另外,openGauss 7.0.0-RC1 版本将在2025年3月30日进行发布。
芬达大佬的dbops自动化部署工具1.6版本现在已经支持opengauss6.0的自动化部署了。今天分享一下在centos7.9虚拟机里面使用dbops工具部署opengauss6.0企业版(一主两从)的整个过程。
二、dbops工具介绍
dbops 是一套高效的 ansible playbook 集合,目前是一个可以自动化安装和部署生产级别的 MySQL 及周边生态的工具。dbops 正在持续迭代开发中,未来将支持更多功能。你可以查看 roadmap (待补充)以了解 dbops 已经支持的和未来将要支持的功能。开发历史请查看 releasehistory。目前,支持了 MySQL5.7、8.0、8.4,openGauss5.0、6.0,磐维数据库V2.0-S2.0.3,磐维数据库V2.0-S3.0.0。
dbops主要解决的问题:
各类生产环境的规范化建设(如开源数据库部署、高可用架构部署、监控工具部署、备份脚本部署)
实例生命周期中的备份、监控、事态感知与修复方案下发 roadmap (待开发)
生命周期结束时的清理与资源回收
使用前注意:
dbops 是个人开发的,因此并未经过大量的测试。如果你打算在生产系统中使用它,请详细阅读源代码,通过公司的安全扫描,并确保在测试环境中进行充分的测试,以避免高危操作。由于 dbops 目前只专注于部署,因此在建设期直接使用基本上不会有问题。
dbops 服务器端(也就是安装 ansible 的位置)和被控端(即实际要部署软件/架构的节点)只支持以下操作系统。
操作系统 | 说明 |
---|---|
RHEL7 系列 | 理论上支持 RHEL7 及其兼容系统,包括:RedHat7、CentOS7 等等 |
RHEL8 系列 | 理论上支持 RHEL8 及其兼容系统,包括:RedHat8、CentOS8、AnolisOS8 等等 |
RHEL9 系列 | 理论上支持 RHEL9 及其兼容系统,包括:RockyLinux9 等等 |
Euler 系列 | 理论上支持 Euler 及其兼容系统,包括:openEuler 20.03 LTS、openEuler 22.03 LTS 、openEuler 24.03 LTS等等 |
支持的系统和数据库架构
ansible
mysql-ansible
opengauss-ansible(目前版主还没更新opengauss支持的最新版本文档,其实6.0企业版已支持)
三、dbops工具下载&opengauss软件包下载
dbops工具下载地址:
https://gitee.com/fanderchan/dbops
opengauss软件包下载地址:
https://opengauss.org/zh/download/
四、使用dbops工具部署opengauss数据库
环境说明
dbops版本:1.6
数据库环境如下:
主机名 | ip地址 | OS版本 | 内存、CPU | 节点角色 | 数据库端口 | cmserver端口 |
---|---|---|---|---|---|---|
node1 | 192.*.*.60 | Centos7.9 | 4G 、 1个双核 | 主节点 | 15400 | 15000 |
node2 | 192.*.*.62 | Centos7.9 | 4G 、 1个双核 | 从节点 | 15400 | 15000 |
node3 | 192.*.*.64 | Centos7.9 | 4G 、 1个双核 | 从节点 | 15400 | 15000 |
1. 第一台机器上下载dbops工具
[root@node1 ~]# wget https://gitee.com/fanderchan/dbops/releases/download/dbops.1.6.20241029/dbops.1.6.20241029-Linux-x86_64.tar.gz
[root@node1 ~]# tar -zxvf dbops.1.6.20241029-Linux-x86_64.tar.gz -C /usr/local/
2. 安装dbops自带的绿色版ansible
[root@node1 ~]# cd /usr/local/dbops
[root@node1 dbops]# cd portable-ansible-v0.5.0-py3
[root@node1 portable-ansible-v0.5.0-py3]# sh setup_portable_ansible.sh
[root@node1 portable-ansible-v0.5.0-py3]# source ~/.bashrc
检查ansible是否安装成功
[root@node1 portable-ansible-v0.5.0-py3]# ansible --version
ansible 2.10.5
config file = None
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/dbops/portable-ansible-v0.5.0-py3/ansible/ansible
executable location = /usr/local/dbops/portable-ansible-v0.5.0-py3/ansible
python version = 3.6.8 (default, Nov 14 2023, 16:29:52) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
[root@node1 portable-ansible-v0.5.0-py3]#
3. 上传opengauss安装包到downloads目录
# 操作系统是centos7.9,所以下载centos平台的opengauss安装包,文件名如下:
openGauss-All-6.0.0-CentOS7-x86_64.tar.gz
cd /usr/local/dbops/opengauss_ansible/downloads
mv /opt/soft/openGauss-All-6.0.0-CentOS7-x86_64.tar.gz ./
4. 修改ansible的hosts.ini
cd /usr/local/dbops/opengauss_ansible/inventory/
vi hosts.ini
[dbops_opengauss]
192.*.*.60 ansible_user=root ansible_ssh_pass="'密码'" #改为你的ip和密码,只支持root不能改root
192.*.*.62 ansible_user=root ansible_ssh_pass="'密码'" #改为你的ip和密码,只支持root不能改root
192.*.*.64 ansible_user=root ansible_ssh_pass="'密码'" #改为你的ip和密码,只支持root不能改root
[all:vars]
#ansible_python_interpreter=/usr/bin/python3
5. 修改playbook参数和执行
如果有东西需要改,那么改common_config.yml
和vars/var_enterprise_deployment.yml
(/usr/local/dbops/opengauss_ansible/playbooks目录下)即可。
修改common_config.yml:
cd /usr/local/dbops/opengauss_ansible/playbooks
vi common_config.yml
#截止至 20241029,dbops支持的opengauss最高版本是6.0.0,最小需要修改以下参数:
opengauss_version: "6.0.0"
#本次部署共修改了3个参数:
opengauss_version: "6.0.0"
opengauss_port: 15400
gausshome: /data/{{ db_type }}
common_config.yml中有1处需特别注意:
如果数据库安装目录不是/database,需要修改gausshome参数
修改后的common_config.yml文件内容如下:
[root@node1 playbooks]# cat common_config.yml
## In most cases, just adjust these two parameters.
opengauss_version: "6.0.0"
opengauss_port: 15400
opengauss_user: omm
db_type: opengauss # opengauss, panweidb
panweidb_dbcompatibility: B # A, B, C, and PG. They are compatible with Oracle, MySQL, Teradata, and PostgreSQL, respectively. Only applicable to the PanWeiDB.
#Directory of openGauss installation package
opengauss_packages_dir: ../downloads/
## linux opengauss run user group and password
opengauss_group: dbgroup
opengauss_user_password: Dbops@9999
## opengauss install directory
gausshome: /data/{{ db_type }}
## The password needs to meet the password complexity requirements
## Complexity requirements: at least 8 digits, including numbers, upper and lower case letters, and special symbols.
opengauss_admin_user: "{{ opengauss_user }}"
opengauss_admin_password: Dbops@8888
## opengauss postgresql.cnf
## Feature Control Switch
fcs_skip_db_mount_verification: 1
fcs_skip_check_ntpd_or_chrony_running: 1
fcs_auto_download_opengauss: 1
fcs_create_opengauss_fast_login: 1
fcs_opengauss_use_jemalloc: 0
fcs_disable_command_history: 0
## Feature Control Switch vars
fcs_create_opengauss_fast_login_name: "db{{ opengauss_port }}"
[root@node1 playbooks]#
修改vars/var_enterprise_deployment.yml:
vi var_enterprise_deployment.yml
# 本次部署修改了1个参数
cm_server_port: 15000
修改后的vars/var_enterprise_deployment.yml文件内容如下:
[root@node1 playbooks]# cat vars/var_enterprise_deployment.yml
# vars loading order: common_config -> this file
cluster_name: "opengauss"
gaussdb_app_path: "{{ gausshome }}/app"
gaussdb_log_path: "{{ gausshome }}/log"
tmp_mppdb_path: "{{ gausshome }}/tmp"
gaussdb_tool_path: "{{ gausshome }}/tool"
core_path: "{{ gausshome }}/corefile"
gaussdb_data_path: "{{ gausshome }}/data"
cm_dir: "{{ gausshome }}/cm"
cm_server_port: 15000
tmp_tar_packages_path: "{{ gausshome }}/soft"
audit_directory: "{{ gausshome }}/pg_audit"
archive_dest: "/archive"
[root@node1 playbooks]#
执行playbook:
cd /usr/local/dbops/opengauss_ansible/playbooks
ansible-playbook enterprise_deployment.yml
# 会先pre_tasks检查你的配置是否正确,然后你输入comfirm确认后,就一直等待直到部署完成即可。
confirm
安装过程部分截图如下:
...
ASK [../roles/enterprise_deployment : Transfer openGauss install package to remote host] ***************************************************************************************************************************************************
changed: [192.168.100.62]
changed: [192.168.100.64]
changed: [192.168.100.60]
TASK [../roles/enterprise_deployment : Unarchive openGauss install package to /data/opengauss/soft] *****************************************************************************************************************************************
changed: [192.168.100.62]
changed: [192.168.100.64]
changed: [192.168.100.60]
TASK [../roles/enterprise_deployment : Touch unarchive_opengauss_package_finished file] *****************************************************************************************************************************************************
changed: [192.168.100.62]
changed: [192.168.100.64]
changed: [192.168.100.60]
TASK [../roles/enterprise_deployment : Unarchive openGauss openGauss-OM-6.0.0-CentOS7-x86_64.tar.gz to /data/opengauss/soft] ****************************************************************************************************************
changed: [192.168.100.62]
changed: [192.168.100.64]
changed: [192.168.100.60]
TASK [../roles/enterprise_deployment : Touch unarchive_opengauss_bz2_finished file] *********************************************************************************************************************************************************
changed: [192.168.100.62]
changed: [192.168.100.64]
changed: [192.168.100.60]
TASK [../roles/enterprise_deployment : Config cluster_config_template.xml and transfer to the first_host /data/opengauss/soft] **************************************************************************************************************
changed: [192.168.100.64]
changed: [192.168.100.62]
changed: [192.168.100.60]
TASK [../roles/enterprise_deployment : Execute gs_preinstall script on every node] **********************************************************************************************************************************************************
changed: [192.168.100.64]
changed: [192.168.100.62]
changed: [192.168.100.60]
TASK [../roles/enterprise_deployment : Clear content from the marker to the end of file] ****************************************************************************************************************************************************
changed: [192.168.100.62]
changed: [192.168.100.64]
changed: [192.168.100.60]
TASK [../roles/enterprise_deployment : Add Gauss OM IP Hosts Mapping to /etc/hosts if not present] ******************************************************************************************************************************************
changed: [192.168.100.64]
changed: [192.168.100.62]
changed: [192.168.100.60]
TASK [../roles/enterprise_deployment : Install opengauss to use OM] *************************************************************************************************************************************************************************
skipping: [192.168.100.62]
skipping: [192.168.100.64]
included: /usr/local/dbops/opengauss_ansible/roles/enterprise_deployment/tasks/install_opengauss.yml for 192.168.100.60
TASK [../roles/enterprise_deployment : Transfer expect script to remote host] ***************************************************************************************************************************************************************
changed: [192.168.100.60]
TASK [../roles/enterprise_deployment : Execute expect script to run gs_install] *************************************************************************************************************************************************************
ok: [192.168.100.60]
TASK [../roles/enterprise_deployment : Remove expect script from remote host] ***************************************************************************************************************************************************************
changed: [192.168.100.60]
TASK [../roles/enterprise_deployment : Copy gs_guc.sh.j2 to /tmp] ***************************************************************************************************************************************************************************
skipping: [192.168.100.62]
skipping: [192.168.100.64]
changed: [192.168.100.60]
TASK [../roles/enterprise_deployment : Execute gs_guc.sh as opengauss_user] *****************************************************************************************************************************************************************
skipping: [192.168.100.62]
skipping: [192.168.100.64]
changed: [192.168.100.60]
TASK [../roles/enterprise_deployment : Restart service with gs_om] **************************************************************************************************************************************************************************
skipping: [192.168.100.62]
skipping: [192.168.100.64]
changed: [192.168.100.60]
TASK [../roles/enterprise_deployment : OpenGauss status] ************************************************************************************************************************************************************************************
ok: [192.168.100.60]
ok: [192.168.100.64]
ok: [192.168.100.62]
TASK [../roles/enterprise_deployment : Set up alias in .bashrc named —— db15400] ************************************************************************************************************************************************************
[WARNING]: Module remote_tmp /home/omm/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually
changed: [192.168.100.62]
changed: [192.168.100.64]
changed: [192.168.100.60]
TASK [../roles/enterprise_deployment : Touch enterprise_deployment_finish.flag] *************************************************************************************************************************************************************
changed: [192.168.100.62]
changed: [192.168.100.64]
changed: [192.168.100.60]
PLAY RECAP **********************************************************************************************************************************************************************************************************************************
192.168.100.60 : ok=108 changed=45 unreachable=0 failed=0 skipped=32 rescued=0 ignored=0
192.168.100.62 : ok=89 changed=35 unreachable=0 failed=0 skipped=35 rescued=0 ignored=0
192.168.100.64 : ok=89 changed=35 unreachable=0 failed=0 skipped=35 rescued=0 ignored=0
Playbook run took 0 days, 0 hours, 11 minutes, 57 seconds
[root@node1 playbooks]#
整个安装过程会完成检查并自动安装缺少的软件依赖包、设置内核参数、关闭透明大页、关闭防火墙、关闭RemoveIPC等准备工作,进行预安装,直到最后安装成功。安装成功后会自动配置数据库基线。在自己本地虚拟机配置比较低的情况下整个安装过程只花费了11分钟57秒,缩短了传统人工部署时间,大大提高了工作效率。
五、检查数据库集群状态
[omm@node1 ~]$ gs_om -t status --detail
[ CMServer State ]
node node_ip instance state
------------------------------------------------------------------
1 node1 192.*.*.60 1 /data/opengauss/cm/cm_server Primary
2 node2 192.*.*.62 2 /data/opengauss/cm/cm_server Standby
3 node3 192.*.*.64 3 /data/opengauss/cm/cm_server Standby
[ Cluster State ]
cluster_state : Normal
redistributing : No
balanced : Yes
current_az : AZ_ALL
[ Datanode State ]
node node_ip instance state
---------------------------------------------------------------------------
1 node1 192.*.*.60 6001 15400 /data/opengauss/data P Primary Normal
2 node2 192.*.*.62 6002 15400 /data/opengauss/data S Standby Normal
3 node3 192.*.*.64 6003 15400 /data/opengauss/data S Standby Normal
数据库集群运行正常。
六、通过gsql客户端访问opengauss数据库
[omm@node1 ~]$ gsql -d postgres -p15400 -r
gsql ((openGauss 6.0.0 build aee4abd5) compiled at 2024-09-29 18:38:08 commit 0 last mr )
Non-SSL connection (SSL connection is recommended when requiring high-security)
Type "help" for help.
#检查数据库版本
openGauss=# select version();
version
-------------------------------------------------------------------------------------------------------------------------------------------------------
(openGauss 6.0.0 build aee4abd5) compiled at 2024-09-29 18:38:08 commit 0 last mr on x86_64-unknown-linux-gnu, compiled by g++ (GCC) 10.3.0, 64-bit
(1 row)
#查看数据库列表
openGauss=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+-------+----------+---------+-------+-------------------
postgres | omm | UTF8 | C | C |
template0 | omm | UTF8 | C | C | =c/omm +
| | | | | omm=CTc/omm
template1 | omm | UTF8 | C | C | =c/omm +
| | | | | omm=CTc/omm
(3 rows)
#创建数据库
openGauss=# create database testdb;
CREATE DATABASE
#切换到创建的数据库testdb
openGauss=# \c testdb
Non-SSL connection (SSL connection is recommended when requiring high-security)
You are now connected to database "testdb" as user "omm".
#创建用户
testdb=# create user testusr with password '******';
NOTICE: The encrypted password contains MD5 ciphertext, which is not secure.
CREATE ROLE
#切换到创建的用户testusr
testdb=# \c - testusr
Password for user testusr:
Non-SSL connection (SSL connection is recommended when requiring high-security)
You are now connected to database "testdb" as user "testusr".
#查看relation
testdb=> \d
No relations found.
#创建表、插入数据
testdb=> create table testtab(id int,name varchar(20));
CREATE TABLE
testdb=> insert into testtab values(1,'a');
INSERT 0 1
#查看用户列表
testdb=> \du
List of roles
Role name | Attributes | Member of
-----------+------------+-----------
testusr | | {}
#查看schema列表
testdb=> \dn
List of schemas
Name | Owner
-----------------+---------
blockchain |
coverage |
cstore |
db4ai |
dbe_perf |
dbe_pldebugger |
dbe_pldeveloper |
dbe_sql_util |
pkg_service |
public |
snapshot |
sqladvisor |
testusr | testusr
(13 rows)
#查看当前schema下的表
testdb=> \dt
List of relations
Schema | Name | Type | Owner | Storage
---------+---------+-------+---------+----------------------------------
testusr | testtab | table | testusr | {orientation=row,compression=no}
(1 row)
七、通过dbeaver客户端访问opengauss数据库
7.1 下载jdbc驱动
解压jdbc驱动包
[root@node1 ~]# tar -zxf openGauss-JDBC-6.0.0.tar.gz -C /opt/soft/
[root@node1 ~]# cd /opt/soft/
[root@node1 soft]# ls
opengauss-jdbc-6.0.0.jar postgresql.jar README_cn.md README_en.md
[root@node1 soft]#
7.2 dbeaver设置
菜单栏选择数据库->选择驱动管理器->选择新建->选择库->添加文件->点击找到类
菜单栏选择数据库->选择新建数据库连接->选择刚才添加的opengauss驱动
填写JDBC URL、用户名、密码
jdbc url格式:jdbc:opengauss://ip地址:port/数据库名
7.2 dbeaver测试连接
连接失败解决办法:
# 用下面命令设置客户端ip的白名单
[omm@node1 ~]$ gs_guc reload -I all -N all -h "host all all 192.168.0.0/16 sha256"
The gs_guc run with the following arguments: [gs_guc -I all -N all -h host all all 192.168.0.0/16 sha256 reload ].
Begin to perform the total nodes: 3.
Popen count is 3, Popen success count is 3, Popen failure count is 0.
Begin to perform gs_guc for datanodes.
Command count is 3, Command success count is 3, Command failure count is 0.
Total instances: 3. Failed instances: 0.
ALL: Success to perform gs_guc!
再次连接测试:
八、虚拟机里碰到的问题
1、Network card names are different
原因:三台虚拟机上都有两块网卡且网卡名不一致
解决办法:
1、先把三台主机的网卡修改一致
2、删除所有节点的网卡比对文件
rm -rf net_aliases.txt
3、重新部署
ansible-playbook enterprise_deployment.yml
九、总结
整个安装过程非常丝滑,dbops自动化部署工具大大提高了运维人员的工作效率,减少了手动部署过程中容易出现人为错误,如配置错误、遗漏步骤等错误的发生,提高了部署的准确性和可靠性,让运维人员把精力花费在更有价值的工作上。
点击阅读原文跳转作者文章