首页
常用命令
About Me
推荐
weibo
github
Search
1
linuxea:gitlab-ci之docker镜像质量品质报告
48,996 阅读
2
linuxea:如何复现查看docker run参数命令
20,462 阅读
3
Graylog收集文件日志实例
18,023 阅读
4
git+jenkins发布和回滚示例
17,602 阅读
5
linuxea:jenkins+pipeline+gitlab+ansible快速安装配置(1)
17,574 阅读
ops
Openvpn
Sys Basics
rsync
Mail
NFS
Other
Network
HeartBeat
server 08
Code
Awk
Shell
Python
Golang
virtualization
KVM
Docker
openstack
Xen
kubernetes
kubernetes-cni
Service Mesh
Data
Mariadb
PostgreSQL
MongoDB
Redis
MQ
Ceph
TimescaleDB
kafka
surveillance system
zabbix
ELK Stack
Open-Falcon
Prometheus
Web
apache
Tomcat
Nginx
自动化
Puppet
Ansible
saltstack
Proxy
HAproxy
Lvs
varnish
更多
音乐
影视
music
Internet Consulting
最后的净土
软件交付
持续集成
gitops
devops
登录
Search
标签搜索
kubernetes
docker
zabbix
Golang
mariadb
持续集成工具
白话容器
linux基础
nginx
elk
dockerfile
Gitlab-ci/cd
最后的净土
基础命令
jenkins
docker-compose
gitops
haproxy
saltstack
Istio
marksugar
累计撰写
676
篇文章
累计收到
140
条评论
首页
栏目
ops
Openvpn
Sys Basics
rsync
Mail
NFS
Other
Network
HeartBeat
server 08
Code
Awk
Shell
Python
Golang
virtualization
KVM
Docker
openstack
Xen
kubernetes
kubernetes-cni
Service Mesh
Data
Mariadb
PostgreSQL
MongoDB
Redis
MQ
Ceph
TimescaleDB
kafka
surveillance system
zabbix
ELK Stack
Open-Falcon
Prometheus
Web
apache
Tomcat
Nginx
自动化
Puppet
Ansible
saltstack
Proxy
HAproxy
Lvs
varnish
更多
音乐
影视
music
Internet Consulting
最后的净土
软件交付
持续集成
gitops
devops
页面
常用命令
About Me
推荐
weibo
github
搜索到
7
篇与
MongoDB
的结果
2022-03-19
linuxea:mongodb 4.4.6副本集配置笔记
version : 4.4.6先决条件1,防火墙互相放行270202, 节点配置应该一致副本集解决不了写入瓶颈,如果副本集规模越大,写入性能越会下降,副本集可以应对读多的场景序号ip配置1172.16.100.108*16 / hdd(推荐ssd)2172.16.100.118*16 / hdd(推荐ssd)3172.16.100.128*16 / hdd(推荐ssd)echo never > /sys/kernel/mm/transparent_hugepage/enabledecho never > /sys/kernel/mm/transparent_hugepage/defrag3, deploy.sh#!/bin/bash \# auther: mark \# descriptions: the shell scripts install mongodb 4.4.6 version and start script and firewalld if ! grep ntp.aliyun.com /var/spool/cron/root >/dev/null 2>&1;then (crontab -l; echo -e "10 * * * * ntpdate ntp.aliyun.com") | crontab -;fi timedatectl set-timezone Asia/Shanghai hostnamectl set-hostname mongodb1 tar xf mongodb-linux-x86_64-rhel70-4.4.6.tgz -C /usr/local cd /usr/local/ ln -s mongodb-linux-x86_64-rhel70-4.4.6 mongodb mkdir /data/mongodb/{data,logs,pid,conf} -p groupadd mongodb useradd -g mongodb mongodb chown -R mongodb.mongodb /data/mongodb ln -s /usr/local/mongodb/bin/mongo /usr/local/bin/ cp /etc/firewalld/zones/public.xml /etc/firewalld/zones/public.xml.oldone cat > /etc/firewalld/zones/public.xml << EOF <?xml version="1.0" encoding="utf-8"?> <zone> <short>Public</short> <service name="ssh"/> <service name="dhcpv6-client"/> <port protocol="tcp" port="27020"/> </zone> EOF systemctl restart firewalld cat > /etc/systemd/system/mongodb.service << EPF [Unit] Description=mongodb Service After=network.target syslog.target [Service] Environment=ZOO_LOG_DIR=/data/mongodb/logs SyslogIdentifier=mongodb User=mongodb Group=mongodb Type=forking LimitFSIZE=infinity LimitCPU=infinity LimitAS=infinity LimitMEMLOCK=infinity LimitNOFILE=64000 LimitNPROC=64000 PIDfile=/mydata/data/mongodb/pid/mongod.pid ExecReload=/bin/kill -s HUP $MAINPID ExecStart=/usr/local/mongodb/bin/mongod -f /data/mongodb/conf/mongodb.conf ExecStop=/usr/local/mongodb/bin/mongod --shutdown -f /data/mongodb/conf/mongodb.conf Restart=on-failure PrivateTmp=true [Install] WantedBy=default.target EPF systemctl daemon-reload chmod +x /etc/systemd/system/mongodb.service systemctl enable mongodb示例: 172.16.100.10/11/12echo "mongodb soft nofile 64000" >> /etc/security/limits.conf echo "mongodb hard nofile 64000" >> /etc/security/limits.conf echo "mongodb soft nproc 32000" >> /etc/security/limits.conf echo "mongodb hard nproc 32000" >> /etc/security/limits.conf echo "never" > /sys/kernel/mm/transparent_hugepage/enabled echo "never" > /sys/kernel/mm/transparent_hugepage/defragLimitFSIZE=infinity LimitCPU=infinity LimitAS=infinity LimitMEMLOCK=infinity LimitNOFILE=64000 LimitNPROC=64000配置文件10配置systemLog: destination: file logAppend: true path: /data/mongodb/logs/mongod.log storage: dbPath: /data/mongodb/data journal: enabled: true directoryPerDB: true wiredTiger: engineConfig: cacheSizeGB: 8 directoryForIndexes: true processManagement: fork: true pidFilePath: /data/mongodb/pid/mongod.pid net: port: 27020 bindIp: 0.0.0.0 #bindIp: 0.0.0.0,mongodb1,localhost # ip and hostname maxIncomingConnections: 5000 #security: # keyFile: /data/mongodb/conf/keyfile # authorization: enabled replication: # oplogSizeMB: 1024 replSetName: rs0先注释security字段,没有密码登陆。三台节点防火墙开启27020互通初始化集群直接使用mongo连入admin/usr/local/mongodb/bin/mongo 172.16.100.10:27020 --authenticationDatabase admin配置仲裁priority。如果节点配置不一样,根据配置大小调整权重比例config = { _id:"rs0", members:[ {_id:0,host:"172.16.100.10:27020",priority:90}, {_id:1,host:"172.16.100.11:27020",priority:90}, {_id:2,host:"172.16.100.12:27020",arbiterOnly:true} ] };或者不配置。节点少,不配置仲裁config = { _id:"rs0", members:[ {_id:0,host:"172.16.100.10:27020",priority:90}, {_id:1,host:"172.16.100.11:27020",priority:90}, {_id:2,host:"172.16.100.12:27020",priority:90} ] };配置集群要进入admin> use admin switched to db admin > config = { _id:"rs0", ... members:[ ... {_id:0,host:"172.16.100.10:27020",priority:90}, ... {_id:1,host:"172.16.100.11:27020",priority:90}, ... {_id:2,host:"172.16.100.12:27020",priority:90} ... ] ... } { "_id" : "rs0", "members" : [ { "_id" : 0, "host" : "172.16.100.10:27020", "priority" : 90 }, { "_id" : 1, "host" : "172.16.100.11:27020", "priority" : 90 }, { "_id" : 2, "host" : "172.16.100.12:27020", "priority" : true } ] }使用rs.initiate(config);进行初始化> rs.initiate(config); { "ok" : 1 }使用rs.status()查看状态rs0:SECONDARY> rs.status() { "set" : "rs0", "date" : ISODate("2021-05-18T02:37:24.202Z"), "myState" : 1, "term" : NumberLong(1), "syncSourceHost" : "", "syncSourceId" : -1, "heartbeatIntervalMillis" : NumberLong(2000), "majorityVoteCount" : 2, "writeMajorityCount" : 2, "votingMembersCount" : 3, "writableVotingMembersCount" : 2, "optimes" : { "lastCommittedOpTime" : { "ts" : Timestamp(1621305433, 1), "t" : NumberLong(1) }, "lastCommittedWallTime" : ISODate("2021-05-18T02:37:13.348Z"), "readConcernMajorityOpTime" : { "ts" : Timestamp(1621305433, 1), "t" : NumberLong(1) }, "readConcernMajorityWallTime" : ISODate("2021-05-18T02:37:13.348Z"), "appliedOpTime" : { "ts" : Timestamp(1621305433, 1), "t" : NumberLong(1) }, "durableOpTime" : { "ts" : Timestamp(1621305433, 1), "t" : NumberLong(1) }, "lastAppliedWallTime" : ISODate("2021-05-18T02:37:13.348Z"), "lastDurableWallTime" : ISODate("2021-05-18T02:37:13.348Z") }, "lastStableRecoveryTimestamp" : Timestamp(1621305431, 3), "electionCandidateMetrics" : { "lastElectionReason" : "electionTimeout", "lastElectionDate" : ISODate("2021-05-18T02:37:11.853Z"), "electionTerm" : NumberLong(1), "lastCommittedOpTimeAtElection" : { "ts" : Timestamp(0, 0), "t" : NumberLong(-1) }, "lastSeenOpTimeAtElection" : { "ts" : Timestamp(1621305421, 1), "t" : NumberLong(-1) }, "numVotesNeeded" : 2, "priorityAtElection" : 90, "electionTimeoutMillis" : NumberLong(10000), "numCatchUpOps" : NumberLong(0), "newTermStartDate" : ISODate("2021-05-18T02:37:11.879Z"), "wMajorityWriteAvailabilityDate" : ISODate("2021-05-18T02:37:13.328Z") }, "members" : [ { "_id" : 0, "name" : "172.16.100.10:27020", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 733, "optime" : { "ts" : Timestamp(1621305433, 1), "t" : NumberLong(1) }, "optimeDate" : ISODate("2021-05-18T02:37:13Z"), "syncSourceHost" : "", "syncSourceId" : -1, "infoMessage" : "", "electionTime" : Timestamp(1621305431, 1), "electionDate" : ISODate("2021-05-18T02:37:11Z"), "configVersion" : 1, "configTerm" : 1, "self" : true, "lastHeartbeatMessage" : "" }, { "_id" : 1, "name" : "172.16.100.11:27020", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 22, "optime" : { "ts" : Timestamp(1621305433, 1), "t" : NumberLong(1) }, "optimeDurable" : { "ts" : Timestamp(1621305433, 1), "t" : NumberLong(1) }, "optimeDate" : ISODate("2021-05-18T02:37:13Z"), "optimeDurableDate" : ISODate("2021-05-18T02:37:13Z"), "lastHeartbeat" : ISODate("2021-05-18T02:37:23.871Z"), "lastHeartbeatRecv" : ISODate("2021-05-18T02:37:23.378Z"), "pingMs" : NumberLong(0), "lastHeartbeatMessage" : "", "syncSourceHost" : "172.16.100.10:27020", "syncSourceId" : 0, "infoMessage" : "", "configVersion" : 1, "configTerm" : 1 }, { "_id" : 2, "name" : "172.16.100.12:27020", "health" : 1, "state" : 7, "stateStr" : "SECONDARY", "uptime" : 22, "lastHeartbeat" : ISODate("2021-05-18T02:37:23.872Z"), "lastHeartbeatRecv" : ISODate("2021-05-18T02:37:23.876Z"), "pingMs" : NumberLong(0), "lastHeartbeatMessage" : "", "syncSourceHost" : "", "syncSourceId" : -1, "infoMessage" : "", "configVersion" : 1, "configTerm" : 1 } ], "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1621305433, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1621305433, 1) }授权root用户创建super man用户db.createUser({user: "root", pwd: "linuxea.com", roles: [{role: "root", db: "admin"}]}); db.createUser({user: "root", pwd: "linuxea.com", roles: [{role: "dbAdminAnyDatabase", db: "admin"}]}); db.createUser({user: "marksugar", pwd: "TdmMzIyNjRmMjViOTc1MGIwZGU", roles: [{role: "dbAdminAnyDatabase", db: "admin"}]});创建普通用户https://www.jianshu.com/p/0a7452d8843ddb.createUser( { user: "marksugar2",pwd: "TdmMzIyNjRmMjViOTc1MGIwZGU",roles: [ { role:"readWrite", db: "marksugar" }] }) db.createUser( { user: "marksugar",pwd: "TdmMzIyNjRmMjViOTc1MGIwZGU",roles: [ { role:"dbAdmin", db: "marksugar" }] }) db.createUser({ user: "marksugar",pwd: "TdmMzIyNjRmMjViOTc1MGIwZGU",roles: [ { role: "dbAdmin", db: "marksugar" }]}) db.createUser({ user: "marksugar",pwd: "TdmMzIyNjRmMjViOTc1MGIwZGU",roles: [ { role: "dbOwner", db: "marksugar" }]})如下:修改密码 db.createUser({user: "root", pwd: "linuxea.com", roles: [{role: "root", db: "admin"}]}) 执行结果 Successfully added user: { "user" : "root", "roles" : [ { "role" : "root", "db" : "admin" } ] }创建完成可以使用如下命令验证授权mongo -u root -p "linuxea.com" 17.168.0.175:27020/admin rs0:PRIMARY> db.get admin.get mongo -u marksugar -p "TdmMzIyNjRmMjViOTc1MGIwZGU" 17.168.0.175:27020/marksugar rs0:PRIMARY> db.get marksugar.get验证集群登入:mongo 172.16.100.10:27020/admin[root@localhost ~]# mongo 172.16.100.10:27020/admin MongoDB shell version v4.4.6 connecting to: mongodb://172.16.100.10:27020/admin?compressors=disabled&gssapiServiceName=mongodb Implicit session: session { "id" : UUID("e73613cb-6004-46b0-b229-8356c6ed8cf5") } MongoDB server version: 4.4.6 --- The server generated these startup warnings when booting: 2021-05-18T10:25:11.991+08:00: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem 2021-05-18T10:25:13.535+08:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted 2021-05-18T10:25:13.535+08:00: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. We suggest setting it to 'never' 2021-05-18T10:25:13.535+08:00: /sys/kernel/mm/transparent_hugepage/defrag is 'always'. We suggest setting it to 'never' 2021-05-18T10:25:13.535+08:00: Soft rlimits too low 2021-05-18T10:25:13.535+08:00: currentValue: 1024 2021-05-18T10:25:13.535+08:00: recommendedMinimum: 64000 --- --- Enable MongoDB's free cloud-based monitoring service, which will then receive and display metrics about your deployment (disk utilization, CPU, operation statistics, etc). The monitoring data will be available on a MongoDB website with a unique URL accessible to you and anyone you share the URL with. MongoDB may use this information to make product improvements and to suggest MongoDB products and deployment options to you. To enable free monitoring, run the following command: db.enableFreeMonitoring() To permanently disable this reminder, run the following command: db.disableFreeMonitoring() ---查看库show dbsrs0:PRIMARY> show dbs admin 0.000GB config 0.000GB local 0.000GB创建库use yxtops-testrs0:PRIMARY> use yxtops-test switched to db yxtops-test查看db.getName()rs0:PRIMARY> db.getName() yxtops-test插入:db.student.insert([{'name':'mark'},{}])rs0:PRIMARY> db.student.insert([{'name':'mark'},{}]) BulkWriteResult({ "writeErrors" : [ ], "writeConcernErrors" : [ ], "nInserted" : 2, "nUpserted" : 0, "nMatched" : 0, "nModified" : 0, "nRemoved" : 0, "upserted" : [ ] })查询rs0:PRIMARY> show tables; student rs0:PRIMARY> db.student.find() { "_id" : ObjectId("60a329d15d32fd9c982ccde1"), "name" : "mark" } { "_id" : ObjectId("60a329d15d32fd9c982ccde2") }验证登陆其他节点: mongo 172.16.100.11:27020/admin/usr/local/mongodb/bin/mongo -u root -p 'linuxea.com' 172.16.100.11:27020 --authenticationDatabase admin打开读rs0:SECONDARY> rs.slaveOk()进入其他库验证rs0:SECONDARY> use yxtops-test switched to db yxtops-test rs0:SECONDARY> show tables; student rs0:SECONDARY> db.student.find() { "_id" : ObjectId("60a329d15d32fd9c982ccde1"), "name" : "mark" } { "_id" : ObjectId("60a329d15d32fd9c982ccde2") }用户认证集群与集群之间要使用Keyfile。首先创建keyfile。而后复制到其他节点[root@mongodb1 ~]# cd /data/mongodb/conf/ [root@mongodb1 conf]# openssl rand -base64 756 > keyfile [root@mongodb1 conf]# chmod 400 keyfile [root@mongodb1 conf]# pwd /mydata/data/mongodb/conf chown mongodb.mongodb keyfile将内容复制到其他两个节点:cd /mydata/data/mongodb/conf for i in 11 12;do scp keyfile 172.16.100.$i:/mydata/data/mongodb/conf/;done for i in 11 12;do ssh 172.16.100.$i chmod 400 /mydata/data/mongodb/conf/keyfile;done for i in 11 12;do ssh 172.16.100.$i chown mongodb.mongodb /mydata/data/mongodb/conf/keyfile;done或者直接把keyfil文件内容粘贴出来复制到其他两个节点cat > keyfile << EOF kSb+lVdnA9K9/fJnldG65Vvem3r0dKHX/4ag0qcAIZ1Bl87VxmqVho5vKtDMZLN1 uQE9aZ4A3gZ0LFgdPpIXkAHazP9ZqNqBfIJO6JRpBDClFuN8KU1FIJI1ghyVTTAh sjPgdGr8NB7bnloPfm92LwT5kArNMuAd6aqa5EggmpnUZmxkpc81msx39S2mLsxC YEerP0Rj0j8ZA36hsgOsHy+PGvADhjcscqIFAbcqKSpP0CJGZC2WQLUo6EfL0/u9 ChLcUJp0FNgW2vguVXyOjwIhv+x9Rn7fHGokidjnVl54sylPz4AYBtgdmLXsmxi8 5JFXl56kT5W/SD5tvG1IJtteGJfyrWmVqOjbAqhQd6JSQ5zOvRVCUqpndxVy/IRP fw1thjQdBkpcLzf92rUqTUSyANsPV+M1/3L6VEU0KmarkL5yxQvKHlYR1/MtSr7+ bI1UvPsXAMdXWoyZUuZDAeh+J1mMlebeNZ7lYJagskd4SVk9sQoQfmAEnjTuJvSz xpvW4cbJIjzGx58rrfZ/DX+JowhDcsom3Icvl/PlV2JK3CEs4Rna0C70z4KZ0W7B uEY34aTi3IKS3dXQ06qNu3Zja3Zeff9CK2tW0CUs+MiO7Jqp1pVVZRHT6mrG3HWF VbcmvOiHBoEt0/YoMSU6fQsJMgk16Zv53GF0UvY3ZF3x1w6OKT5u7kp9OzHWnPr0 oK6fogfEgi94VBpdSHOXKUyeEI+0LgX8xSb4rmTn4Ud3tBCxXbnqlgVzEuBG+r55 chO4Xmk53FDW5UGtBC9QMWCtGT2I3vymEmKDlnIpzqQXJoKNOQi8UcIw/fluZeaG a1tcaaNn7Vrt9eNfgrBy1w7jL/6FGWF+lhbRmlTAMOMU6k4nGWinaZZA/MAmQmr8 55Y99wT18feHlt+FDH0eSewk6HnTW5QxZYgudP+V89/2vMQHH5vXEBD0JeDHSrpY rkTWULfr9s8nXxoDr5tbLxP1oc+CULt1pZOwXH+Exx5fQuT8 EOF chown mongodb.mongodb keyfile chmod 400 keyfile在配置文件中取消注释security: keyFile: /mydata/data/mongodb/conf/keyfile authorization: enabled而后关闭三台mongodb节点,关闭完成后再依次启动删除用户rs0:PRIMARY> db.dropUser("marksugar") true登陆开启认证后登陆:创建操作等,只能在PRIMARY节点操作才可以。通过rs.status()查看/usr/local/mongodb/bin/mongo -u root -p 'linuxea.com' 172.16.100.10:27020 --authenticationDatabase admin创建其他库和授权库用户use marksugar db.createUser({user: "marksugar",pwd: "TdmMzIyNjRmMjViOTc1MGIwZGU",roles: [ { role: "dbAdmin", db: "marksugar" }]}); db.createUser({user: "marksugar", pwd: "TdmMzIyNjRmMjViOTc1MGIwZGU", roles: [{role: "dbOwner", db: "marksugar"}]}); db.createUser({ user: "marksugar",pwd: "TdmMzIyNjRmMjViOTc1MGIwZGU",roles: [ { role: "dbAdmin", db: "marksugar" }] }) db.createUser({ user: "marksugar",pwd: "TdmMzIyNjRmMjViOTc1MGIwZGU",roles: [ { role: "dbAdmin", db: "marksugar" }]}) db.createUser({user: "root", pwd: "linuxea.com", roles: [{role: "root", db: "admin"}]}); db.createUser({user: "root", pwd: "linuxea.com", roles: [{role: "dbAdminAnyDatabase", db: "admin"}]});插入rs0:PRIMARY> use marksugar switched to db marksugar rs0:PRIMARY> db marksugar rs0:PRIMARY> db.marksugar.insert({"name":"mark"}) WriteResult({ "nInserted" : 1 }) rs0:PRIMARY> show dbs admin 0.000GB config 0.000GB marksugar 0.000GB local 0.001GB rs0:PRIMARY>登陆marksugar/usr/local/mongodb/bin/mongo -u marksugar -p "TdmMzIyNjRmMjViOTc1MGIwZGU" 172.16.100.10:27020/marksugar监控授权db.grantRolesToUser("root", [{role:"__system", db:"admin"}]) db.grantRolesToUser("root", [{role:"dbAdminAnyDatabase", db:"admin"}]);如下:mongo -u root -p "linuxea.com" 172.16.100.10:27020/admin rs0:PRIMARY> use admin switched to db admin rs0:PRIMARY> db.grantRolesToUser("root", [{role:"__system", db:"admin"}]) rs0:PRIMARY> db.grantRolesToUser("root", [{role:"dbAdminAnyDatabase", db:"admin"}]);副本读rs.slaveOk()或者rs.secondaryOk()日志清理logpath日志中存储了日志过程。将会保留7天#!/bin/bash IPADDRES=172.16.100.10:27020 DBNAME=admin USERNAME="USER_NAME" PASSWORDS="PASSWORD" LOGPATHS=/var/log/mongodb LOGDYA=7 # mongodb logpath logfile roll mongo ${IPADDRES}/${DBNAME} --authenticationDatabase ${DBNAME} -u ${USERNAME} -p "${PASSWORDS}" --eval "db.runCommand({logRotate:1});" sleep 3s # logfile delete last 7 days find ${LOGPATHS}/mongod.log.20* -type f -mtime +${LOGDYA} -delete # 1 12 * * * /bin/bash /data/script/mongomore.sh其他参考mongodb4.4.8复制(副本)集简单配置mongodb
2022年03月19日
1,252 阅读
0 评论
0 点赞
linuxea:mongodb4.4.8分片集群搭建
引用geekdemo课程4核8G的分片集群配置域名解析和分片目录创建分片复复制集并初始化创建config复制集并初始化初始化分片集群加入第一个分片创建分片表加入第二个分片架构如下分布情况安装mkdir /data/db && cd /data/ wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel70-4.4.8.tgz tar xf mongodb-linux-x86_64-rhel70-4.4.8.tgz ln -sn mongodb-linux-x86_64-rhel70-4.4.8 mongodb export PATH=$PATH:/data/mongodb/bin echo "export PATH=$PATH:/data/mongodb/bin" > /etc/profile.d/mongodb.sh source /etc/profile.d/mongodb.sh[root@Node-172_16_100_91 /data]# mongod -version db version v4.4.8 Build Info: { "version": "4.4.8", "gitVersion": "83b8bb8b6b325d8d8d3dfd2ad9f744bdad7d6ca0", "openSSLVersion": "OpenSSL 1.0.1e-fips 11 Feb 2013", "modules": [], "allocator": "tcmalloc", "environment": { "distmod": "rhel70", "distarch": "x86_64", "target_arch": "x86_64" } }配置域名解析配置hosts模拟域名记录172.16.100.91 instance1 mongo1.linuxea.com mongo2.linuxea.com 172.16.100.92 instance2 mongo3.linuxea.com mongo4.linuxea.com 172.16.100.93 instance3 mongo5.linuxea.com mongo6.linuxea.com添加之前,我们在每个节点修改hostname,分布是mogodb_instance1,mogodb_instance2,mogodb_instance3for i in static pretty transient; do hostnamectl set-hostname mogodb_instance1 --$i; done写到/etc/profile中而后添加hostsecho "172.16.100.91 mogodb_instance1 mongo1.linuxea.com mongo2.linuxea.com" >> /etc/hosts echo "172.16.100.92 mogodb_instance2 mongo3.linuxea.com mongo4.linuxea.com" >> /etc/hosts echo "172.16.100.93 mogodb_instance3 mongo5.linuxea.com mongo6.linuxea.com" >> /etc/hosts创建目录1,3,5执行mkdir -p /data/shard1/ /data/config/2,4,5执行mkdir -p /data/shard2/ /data/mongos/开始搭建在三台节点都创建用户并修改文件属性groupadd -r -g 490 mongodb useradd -u 490 -s /sbin/nologin -c 'mongodb server' -g mongodb mongodb -M初始化参数echo "never" > /sys/kernel/mm/transparent_hugepage/defrag echo "never" > /sys/kernel/mm/transparent_hugepage/enabled chown -R mongodb.mongodb /data/shard*复制集在member1/member3/member5上执行以下命令。注意以下参数:shardsvr: 表示这不是一个普通的复制集,而是分片集的一部分;wiredTigerCacheSizeGB: 该参数表示MongoDB能够使用的缓存大小。默认值为(RAM - 1GB) / 2不建议配置超过默认值,有OOM的风险;因为我们当前测试会在一台服务器上运行多个实例,因此配置了较小的值;bind_ip: 生产环境中强烈建议不要绑定外网IP,此处为了方便演示绑定了所有IP地址。类似的道理,生产环境中应开启认证--auth,此处为演示方便并未使用;在三台节点都运行如下命令sudo -u mongodb /data/mongodb/bin/mongod --bind_ip 0.0.0.0 --replSet shard1 --dbpath /data/shard1 --logpath /data/shard1/mongod.log --port 27010 --fork --shardsvr --wiredTigerCacheSizeGB 2防火墙放行firewall-cmd --permanent --zone=public --add-rich-rule='rule family="ipv4" source address="172.16.100.0/24" port port="27010" protocol="tcp" accept' firewall-cmd --reload firewall-cmd --list-all运行完成后,就有三台节点已经启动,而后组合这三台节点为复制集用这三个实例搭建shard1复制集:任意连接到一个实例,例如我们连接到member1.example.com: mongo --host mongo1.linuxea.com:27010初始化shard1复制集。我们使用如下配置初始化复制集: rs.initiate({ _id: "shard1", "members" : [ { "_id": 0, "host" : "mongo1.linuxea.com:27010" }, { "_id": 1, "host" : "mongo3.linuxea.com:27010" }, { "_id": 2, "host" : "mongo5.linuxea.com:27010" } ] });执行> rs.initiate({ ... _id: "shard1", ... "members" : [ ... { ... "_id": 0, ... "host" : "mongo1.linuxea.com:27010" ... }, ... { ... "_id": 1, ... "host" : "mongo3.linuxea.com:27010" ... }, ... { ... "_id": 2, ... "host" : "mongo5.linuxea.com:27010" ... } ... ] ... }); { "ok" : 1 } shard1:SECONDARY> 只需要稍等,shard1:SECONDARY> 状态将会切换为shard1:SECONDARY> 使用 rs.status()查看状态shard1:PRIMARY> rs.status() { "set" : "shard1", "date" : ISODate("2021-08-07T14:54:37.331Z"), "myState" : 1, "term" : NumberLong(1), "syncSourceHost" : "", "syncSourceId" : -1, "heartbeatIntervalMillis" : NumberLong(2000), "majorityVoteCount" : 2, "writeMajorityCount" : 2, "votingMembersCount" : 3, "writableVotingMembersCount" : 3, "optimes" : { "lastCommittedOpTime" : { "ts" : Timestamp(1628348069, 1), "t" : NumberLong(1) }, "lastCommittedWallTime" : ISODate("2021-08-07T14:54:29.354Z"), "readConcernMajorityOpTime" : { "ts" : Timestamp(1628348069, 1), "t" : NumberLong(1) }, "readConcernMajorityWallTime" : ISODate("2021-08-07T14:54:29.354Z"), "appliedOpTime" : { "ts" : Timestamp(1628348069, 1), "t" : NumberLong(1) }, "durableOpTime" : { "ts" : Timestamp(1628348069, 1), "t" : NumberLong(1) }, "lastAppliedWallTime" : ISODate("2021-08-07T14:54:29.354Z"), "lastDurableWallTime" : ISODate("2021-08-07T14:54:29.354Z") }, "lastStableRecoveryTimestamp" : Timestamp(1628348049, 2), "electionCandidateMetrics" : { "lastElectionReason" : "electionTimeout", "lastElectionDate" : ISODate("2021-08-07T14:54:08.916Z"), "electionTerm" : NumberLong(1), "lastCommittedOpTimeAtElection" : { "ts" : Timestamp(0, 0), "t" : NumberLong(-1) }, "lastSeenOpTimeAtElection" : { "ts" : Timestamp(1628348037, 1), "t" : NumberLong(-1) }, "numVotesNeeded" : 2, "priorityAtElection" : 1, "electionTimeoutMillis" : NumberLong(10000), "numCatchUpOps" : NumberLong(0), "newTermStartDate" : ISODate("2021-08-07T14:54:09.234Z"), "wMajorityWriteAvailabilityDate" : ISODate("2021-08-07T14:54:10.227Z") }, "members" : [ { "_id" : 0, "name" : "mongo1.linuxea.com:27010", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 204, "optime" : { "ts" : Timestamp(1628348069, 1), "t" : NumberLong(1) }, "optimeDate" : ISODate("2021-08-07T14:54:29Z"), "syncSourceHost" : "", "syncSourceId" : -1, "infoMessage" : "", "electionTime" : Timestamp(1628348048, 1), "electionDate" : ISODate("2021-08-07T14:54:08Z"), "configVersion" : 1, "configTerm" : -1, "self" : true, "lastHeartbeatMessage" : "" }, { "_id" : 1, "name" : "mongo3.linuxea.com:27010", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 39, "optime" : { "ts" : Timestamp(1628348069, 1), "t" : NumberLong(1) }, "optimeDurable" : { "ts" : Timestamp(1628348069, 1), "t" : NumberLong(1) }, "optimeDate" : ISODate("2021-08-07T14:54:29Z"), "optimeDurableDate" : ISODate("2021-08-07T14:54:29Z"), "lastHeartbeat" : ISODate("2021-08-07T14:54:36.949Z"), "lastHeartbeatRecv" : ISODate("2021-08-07T14:54:36.137Z"), "pingMs" : NumberLong(0), "lastHeartbeatMessage" : "", "syncSourceHost" : "mongo1.linuxea.com:27010", "syncSourceId" : 0, "infoMessage" : "", "configVersion" : 1, "configTerm" : -1 }, { "_id" : 2, "name" : "mongo5.linuxea.com:27010", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 39, "optime" : { "ts" : Timestamp(1628348069, 1), "t" : NumberLong(1) }, "optimeDurable" : { "ts" : Timestamp(1628348069, 1), "t" : NumberLong(1) }, "optimeDate" : ISODate("2021-08-07T14:54:29Z"), "optimeDurableDate" : ISODate("2021-08-07T14:54:29Z"), "lastHeartbeat" : ISODate("2021-08-07T14:54:36.949Z"), "lastHeartbeatRecv" : ISODate("2021-08-07T14:54:36.048Z"), "pingMs" : NumberLong(0), "lastHeartbeatMessage" : "", "syncSourceHost" : "mongo1.linuxea.com:27010", "syncSourceId" : 0, "infoMessage" : "", "configVersion" : 1, "configTerm" : -1 } ], "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1628348069, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1628348069, 1) } shard1:PRIMARY> config 复制集configsvr是标注这是config服务器的与shard1类似的方式,我们可以搭建config服务器。在member1/member3/member5上执行以下命令:运行config实例:授权目录chown -R mongodb.mongodb /data/config/启动sudo -u mongodb /data/mongodb/bin/mongod --bind_ip 0.0.0.0 --replSet config --dbpath /data/config --logpath /data/config/mongod.log --port 27019 --fork --configsvr --wiredTigerCacheSizeGB 1防火墙放行firewall-cmd --permanent --zone=public --add-rich-rule='rule family="ipv4" source address="172.16.100.0/24" port port="27019" protocol="tcp" accept' firewall-cmd --reload firewall-cmd --list-all连接到member1:mongo --host mongo1.linuxea.com:27019初始化config复制集:rs.initiate({ _id: "config", "members" : [ { "_id": 0, "host" : "mongo1.linuxea.com:27019" }, { "_id": 1, "host" : "mongo3.linuxea.com:27019" }, { "_id": 2, "host" : "mongo5.linuxea.com:27019" } ] });执行> rs.initiate({ ... _id: "config", ... "members" : [ ... { ... "_id": 0, ... "host" : "mongo1.linuxea.com:27019" ... }, ... { ... "_id": 1, ... "host" : "mongo3.linuxea.com:27019" ... }, ... { ... "_id": 2, ... "host" : "mongo5.linuxea.com:27019" ... } ... ] ... }); { "ok" : 1, "$gleStats" : { "lastOpTime" : Timestamp(1628348518, 1), "electionId" : ObjectId("000000000000000000000000") }, "lastCommittedOpTime" : Timestamp(0, 0) }稍等等状态从config:SECONDARY> 变为config:PRIMARY> config:SECONDARY> config:SECONDARY> config:SECONDARY> config:PRIMARY> config:PRIMARY> config:PRIMARY> mongos配置mongos的搭建比较简单,我们在member2/member4/member6上搭建3个mongos。注意以下参数:configdb: 表示config使用的集群地址;开始搭建:目录授权chown -R mongodb.mongodb /data/mongos/运行mongos进程:sudo -u mongodb /data/mongodb/bin/mongos --bind_ip 0.0.0.0 --logpath /data/mongos/mongos.log --port 27017 --configdb config/mongo1.linuxea.com:27019,mongo3.linuxea.com:27019,mongo5.linuxea.com:27019 --fork放行防火墙firewall-cmd --permanent --zone=public --add-rich-rule='rule family="ipv4" source address="172.16.100.0/24" port port="27017" protocol="tcp" accept' firewall-cmd --reload firewall-cmd --list-all连接到任意一个mongos,此处我们使用member1:mongo --host mongo1.linuxea.com:27017将shard1加入到集群中:sh.addShard("shard1/mongo1.linuxea.com:27010,mongo3.linuxea.com:27010,mongo5.linuxea.com:27010");执行mongos> sh.addShard("shard1/mongo1.linuxea.com:27010,mongo3.linuxea.com:27010,mongo5.linuxea.com:27010"); { "shardAdded" : "shard1", "ok" : 1, "operationTime" : Timestamp(1628349200, 3), "$clusterTime" : { "clusterTime" : Timestamp(1628349200, 3), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } } } mongos> 查看mongos> sh.status() --- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("610ea073af74e0f6aa13bd64") } shards: { "_id" : "shard1", "host" : "shard1/mongo1.linuxea.com:27010,mongo3.linuxea.com:27010,mongo5.linuxea.com:27010", "state" : 1 } active mongoses: "4.4.8" : 3 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: No recent migrations databases: { "_id" : "config", "primary" : "config", "partitioned" : true } mongos> 创建分片表上述示例中我们搭建了一个只有1个分片的分片集。在继续之前我们先来测试一下这个分片集。连接到分片集:mongo --host mongo1.linuxea.com:27017sh.status();mongos> sh.status() --- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("610ea073af74e0f6aa13bd64") } shards: { "_id" : "shard1", "host" : "shard1/mongo1.linuxea.com:27010,mongo3.linuxea.com:27010,mongo5.linuxea.com:27010", "state" : 1 } active mongoses: "4.4.8" : 3 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: No recent migrations databases: { "_id" : "config", "primary" : "config", "partitioned" : true } config.system.sessions shard key: { "_id" : 1 } unique: false balancing: true chunks: shard1 1024 too many chunks to print, use verbose if you want to force print创建一个分片表:foo为库名sh.enableSharding("foo"); // 指定库 sh.shardCollection("foo.bar", {_id: 'hashed'}); // 指定foo.bar表,分片键id作为hashed sh.status();执行mongos> sh.enableSharding("foo"); { "ok" : 1, "operationTime" : Timestamp(1628349504, 4), "$clusterTime" : { "clusterTime" : Timestamp(1628349504, 4), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } } }1mongos> sh.shardCollection("foo.bar", {_id: 'hashed'}); { "collectionsharded" : "foo.bar", "collectionUUID" : UUID("0bec2da5-e75b-4de4-82be-292584633af9"), "ok" : 1, "operationTime" : Timestamp(1628349791, 2), "$clusterTime" : { "clusterTime" : Timestamp(1628349791, 2), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } } } mongos> sh.status()查看状态mongos> sh.status() --- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("610ea073af74e0f6aa13bd64") } shards: { "_id" : "shard1", "host" : "shard1/mongo1.linuxea.com:27010,mongo3.linuxea.com:27010,mongo5.linuxea.com:27010", "state" : 1 } active mongoses: "4.4.8" : 3 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: No recent migrations databases: { "_id" : "config", "primary" : "config", "partitioned" : true } config.system.sessions shard key: { "_id" : 1 } unique: false balancing: true chunks: shard1 1024 too many chunks to print, use verbose if you want to force print { "_id" : "foo", "primary" : "shard1", "partitioned" : true, "version" : { "uuid" : UUID("eb727625-7da5-4da6-a162-25ce2ee2973d"), "lastMod" : 1 } } foo.bar shard key: { "_id" : "hashed" } unique: false balancing: true chunks: shard1 2 { "_id" : { "$minKey" : 1 } } -->> { "_id" : NumberLong(0) } on : shard1 Timestamp(1, 0) { "_id" : NumberLong(0) } -->> { "_id" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 1) 任意写入若干数据:use foo for (var i = 0; i < 10000; i++) { db.bar.insert({i: i}); }执行插入10000条数据mongos> use foo switched to db foo mongos> for (var i = 0; i < 10000; i++) { ... db.bar.insert({i: i}); ... } WriteResult({ "nInserted" : 1 })此时在查看databases字段多了一个foo"_id" : "foo", "primary": 表示每个分片集群里面都有一个主shard,有些操作会在primary上进行partitioned: true: 表示这张表已经分片了,并且会在下面列出"_id" : "foo", "primary" : "shard1", "partitioned" : true, mongos> sh.status() --- Sharding Status --- .............. databases: { "_id" : "config", "primary" : "config", "partitioned" : true } config.system.sessions shard key: { "_id" : 1 } unique: false balancing: true chunks: shard1 1024 too many chunks to print, use verbose if you want to force print { "_id" : "foo", "primary" : "shard1", "partitioned" : true, "version" : { "uuid" : UUID("eb727625-7da5-4da6-a162-25ce2ee2973d"), "lastMod" : 1 } } foo.bar shard key: { "_id" : "hashed" } unique: false balancing: true chunks: shard1 2 { "_id" : { "$minKey" : 1 } } -->> { "_id" : NumberLong(0) } on : shard1 Timestamp(1, 0) { "_id" : NumberLong(0) } -->> { "_id" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 1) 向分片集群加入新的分片创建复制集下面我们搭建shard2并将其加入分片集中,观察发生的效果。使用类似shard1的方式搭建shard2。在member2/member4/member6上执行以下命令:授权chown -R mongodb.mongodb /data/shard2/启动sudo -u mongodb /data/mongodb/bin/mongod --bind_ip 0.0.0.0 --replSet shard2 --dbpath /data/shard2 --logpath /data/shard2/mongod.log --port 27011 --fork --shardsvr --wiredTigerCacheSizeGB 1firewall-cmd --permanent --zone=public --add-rich-rule='rule family="ipv4" source address="172.16.100.0/24" port port="27011" protocol="tcp" accept' firewall-cmd --reload firewall-cmd --list-all用这三个实例搭建shard2复制集:任意连接到一个实例,例如我们连接到member2.example.com:mongo --host mongo2.linuxea.com:27011初始化shard2复制集。我们使用如下配置初始化复制集:rs.initiate({ _id: "shard2", "members" : [ { "_id": 0, "host" : "mongo2.linuxea.com:27011" }, { "_id": 1, "host" : "mongo4.linuxea.com:27011" }, { "_id": 2, "host" : "mongo6.linuxea.com:27011" } ] });执行> rs.initiate({ ... _id: "shard2", ... "members" : [ ... { ... "_id": 0, ... "host" : "mongo2.linuxea.com:27011" ... }, ... { ... "_id": 1, ... "host" : "mongo4.linuxea.com:27011" ... }, ... { ... "_id": 2, ... "host" : "mongo6.linuxea.com:27011" ... } ... ] ... }); { "ok" : 1 }而后稍作等待shard2:SECONDARY> 切换为shard2:PRIMARY> shard2:SECONDARY> shard2:SECONDARY> shard2:SECONDARY> shard2:PRIMARY> shard2:PRIMARY> 如果没有看到也要在rs.status()中找到primaryshard2:PRIMARY> rs.status() { .................. "members" : [ { "_id" : 0, "name" : "mongo2.linuxea.com:27011", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 110, "optime" : { "ts" : Timestamp(1628350780, 1), "t" : NumberLong(1) ................ }, { "_id" : 1, "name" : "mongo4.linuxea.com:27011", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 66, "optime" : { ............. }, { "_id" : 2, "name" : "mongo6.linuxea.com:27011", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 66, "optime" : { "ts" : Timestamp(1628350780, 1), "t" : NumberLong(1) .......... }加入第二个分片连接到任意一个mongos。此处使用mongo1:mongo --host mongo1.linuxea.com:27017将shard2加入到集群中:sh.addShard("shard2/mongo2.linuxea.com:27011,mongo4.linuxea.com:27011,mongo6.linuxea.com:27011");mongos> sh.addShard("shard2/mongo2.linuxea.com:27011,mongo4.linuxea.com:27011,mongo6.linuxea.com:27011"); { "shardAdded" : "shard2", "ok" : 1, "operationTime" : Timestamp(1628351344, 3), "$clusterTime" : { "clusterTime" : Timestamp(1628351344, 4), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } } }观察sh.status():sh.status();mongos> sh.status() --- Sharding Status --- ....... shards: { "_id" : "shard1", "host" : "shard1/mongo1.linuxea.com:27010,mongo3.linuxea.com:27010,mongo5.linuxea.com:27010", "state" : 1 } { "_id" : "shard2", "host" : "shard2/mongo2.linuxea.com:27011,mongo4.linuxea.com:27011,mongo6.linuxea.com:27011", "state" : 1 } active mongoses: ........ unique: false balancing: true chunks: shard1 1 shard2 1 { "_id" : { "$minKey" : 1 } } -->> { "_id" : NumberLong(0) } on : shard2 Timestamp(2, 0) { "_id" : NumberLong(0) } -->> { "_id" : { "$maxKey" : 1 } } on : shard1 Timestamp(2, 1) mongos> 可以发现原本shard1上的两个chunk被均衡到了shard2上,这就是MongoDB的自动均衡机制。现在就可以通过mongos进行操作了集群重启恢复先启动configsudo -u mongodb /data/mongodb/bin/mongod --bind_ip 0.0.0.0 --replSet config --dbpath /data/config --logpath /data/config/mongod.log --port 27019 --fork --configsvr --wiredTigerCacheSizeGB 1在启动复制集1和复制集2,有几个启动几个sudo -u mongodb /data/mongodb/bin/mongod --bind_ip 0.0.0.0 --replSet shard1 --dbpath /data/shard1 --logpath /data/shard1/mongod.log --port 27010 --fork --shardsvr --wiredTigerCacheSizeGB 2sudo -u mongodb /data/mongodb/bin/mongod --bind_ip 0.0.0.0 --replSet shard2 --dbpath /data/shard2 --logpath /data/shard2/mongod.log --port 27011 --fork --shardsvr --wiredTigerCacheSizeGB 1最后启动mongossudo -u mongodb /data/mongodb/bin/mongos --bind_ip 0.0.0.0 --logpath /data/mongos/mongos.log --port 27017 --configdb config/mongo1.linuxea.com:27019,mongo3.linuxea.com:27019,mongo5.linuxea.com:27019 --fork
2021年10月02日
1,493 阅读
0 评论
0 点赞
1
2
...
4