首页
mongodb
Linux
云原生
常用bat文件
Maven教程
设计模式
Oracle常用知识梳理
更多……
申请加入课程
MongDB简介
MongDB及其附属工具介绍和安装
MongoDB 备份(mongodump)与恢复(mongorestore)
MongoDB基础DML和DDL操作
MongoDB 可视化工具
MongoDB 索引
MongoDB 聚合
MongDB 复制与分片集群
MongoDB 监控
MongoDB 查询分析
MongoDB 概念解析
MongoDB GridFS
MongoDB Java
Redis、Memcache和MongoDB的区别
MongoDB的其它关键知识点
MongoDB k8s单节点部署和备份
MongoDB 的配置mongodb.conf
MongoDB k8s下部署 Replica Set
MongoDB 状态查询
MongoDB 数据库的日志解析
MongoDb 常见错误处理方法收集
MongoDB 一次性能测试和问题解决
MongoDB k8s下部署 Replica Set
星辰
2020-11-26
0
0
2267
人
0
人评论
0
人举报
[TOC] # MongoDB k8s下部署 Replica Set 副本集相关知识的介绍请查看专题下的文章《[MongDB 复制与分片集群](https://www.itnoteshare.com/course/11/175/previewcourse.htm "MongDB 复制与分片集群")》 **说明:** 如果k8s集群用的存储卷是分布式存储卷的话,用mongo-sidecar的方式去搭建更为简单,本次版主使用部署3个`StatefulSet`原因是版主的k8s集群采用的是`nfs`存储,并不能算是真正的分布式存储,为了确保通过副本集实现有效的数据备份,需要开启多个`StatefulSet`然后分存储卷需要分别挂载到不同的nfs服务器。 ## K8s 下搭建 MongoDB过程记录 **环境介绍:** 此次搭建环境是k8s,实际操作是在kubesphere上操作的。 **mongo版本:** `mongo:4.4` **说明** * 搭建的集群没有设置密码,设置密码参数配置请自行添加,在命令里面开启密码验证,并在环境变量中加入密码即可。 * 本次搭建没有建存储卷,这块需要自行建立 **节点数:** 设置了3个副本 ### 创建3个mongo服务 **第一步:** 创建3个`StatefulSet`,脚本如下,创建其他几个时请自行修改其中的标识`mongo-rep-work`为`mongo-rep-work2`和`mongo-rep-work3`,这里只贴一份yaml文件,其他内容是一样的 StatefulSet.yaml ``` kind: StatefulSet apiVersion: apps/v1 metadata: name: mongo-rep-work2 namespace: jiangyin labels: app: mongo-rep-work2 annotations: kubesphere.io/alias-name: mongo-rep-work2 kubesphere.io/description: |- mongo-rep-work2 mongo:4.4 spec: replicas: 1 selector: matchLabels: app: mongo-rep-work2 template: metadata: creationTimestamp: null labels: app: mongo-rep-work2 annotations: kubesphere.io/containerSecrets: '' kubesphere.io/restartedAt: '2020-11-26T11:33:28.075Z' logging.kubesphere.io/logsidecar-config: '{}' spec: containers: - name: container-zp3y70 image: 'mongo:4.4' command: - mongod - '--replSet' - rs0 - '--bind_ip' - 0.0.0.0 ports: - name: tcp-27017 containerPort: 27017 protocol: TCP resources: limits: cpu: '4' memory: 4000Mi requests: cpu: 200m memory: 200Mi terminationMessagePath: /dev/termination-log terminationMessagePolicy: File imagePullPolicy: IfNotPresent restartPolicy: Always terminationGracePeriodSeconds: 30 dnsPolicy: ClusterFirst serviceAccountName: default serviceAccount: default securityContext: {} affinity: {} schedulerName: default-scheduler serviceName: mongo-rep-work2-9ha52x podManagementPolicy: OrderedReady updateStrategy: type: RollingUpdate rollingUpdate: partition: 0 revisionHistoryLimit: 10 ``` **第二步:** 创建对应的`Service`,下面仅贴一个的yaml文件其他几个自行修改值`mongo-rs-svc`为`mongo-rs2-svc`和`mongo-rs3-svc` Service.yaml ``` kind: Service apiVersion: v1 metadata: name: mongo-rs-svc namespace: jiangyin labels: app: mongo-rs-svc annotations: kubesphere.io/alias-name: mongo-rs-svc kubesphere.io/description: mongo-rs-svc spec: ports: - name: http-27017 protocol: TCP port: 27017 targetPort: 27017 nodePort: 30384 selector: app: mongo-rep-work clusterIP: 10.233.22.140 type: NodePort sessionAffinity: ClientIP externalTrafficPolicy: Cluster sessionAffinityConfig: clientIP: timeoutSeconds: 10800 ``` ### 集群初始化 进入上面任何一个服务的容器,比如`mongo-rep-work-0`执行初始化脚本,建立副本集之间的关系 ``` # 连接mongo,由于没创建密码所以不需要密码 mongo --port 27017 # 初始化集群关系 https://docs.mongodb.com/manual/reference/method/rs.initiate/ rs.initiate( { _id: "rs0", version: 1, members: [ { _id: 0, host : "mongo-rs-svc:27017" }, { _id: 1, host : "mongo-rs2-svc:27017" }, { _id: 2, host : "mongo-rs3-svc:27017" } ] } ) # 以上脚本由于采用的是服务名创建关联,所以只能部署在集群中的应用只能通过服务名连接,外部连接不到,如果需要外部能连接,可以把服务名和端口换成外部ip和端口,如下 rs.initiate( { _id: "rs0", version: 1, members: [ { _id: 0, host : "192.168.3.161:30384" }, { _id: 1, host : "192.168.3.161:31963" }, { _id: 2, host : "192.168.3.161:31313" } ] } ) # 执行完后验证关系 rs.status() ``` ### 验证数据是否有同步 ``` # 进入主节点,这个需要连接上去看脚本提示是不是primary,默认只有主节点可以写入 mongo --port 27017 # 创建数据库 use noteshare_mongo # 插入测试数据 db.noteshare_mongo.insert({"name":"noteshare"}) show dbs show tables db.noteshare_mongo.find() # 主节点能查到数据 # 进入从节点查询下上面写入的数据是否可以查到,如果可以查到说明已经成功复制了数据 ``` ### 处理问题记录 * MongoDb的“not master and slaveok=false”错误及解决方法 此问题问子节点无法查询数据,通过以下命令开启即可查询 `rs.slaveOk();` ### mongodb副本集修改配置问题 如果已经初始化了集群,该如何修改配置呢,请查看一下相关指令说明 ``` # 查看配置rs.config();需要找到primary主机,在该主节点服务器上才有权限修改配置 rs.config() # 移除原配置文件中的已经变更地址的主机 rs.remove("ip:port") # 添加新的地址主机 rs.add("ip:port") # 设置priority优先级和修改配置 >var config = rs.config() >config.members[2].priority=2 # 重新更新配置 >rs.reconfig(config) # 添加仲裁节点 rs.addArb("192.168.0.3:27019"); # 删除仲裁节点 rs.remove("192.168.0.3:27019"); ## 初始化时添加仲裁节点 rs.initiate( { _id: "rs0", version: 1, members: [ { _id: 0, host : "192.168.3.161:30384" }, { _id: 1, host : "192.168.3.161:31963" }, { _id: 2, host : "192.168.3.161:31313",arbiterOnly:true} ] } ) ``` ### 开启mongo认证 ``` mongo 副本集的认证需要使用--keyfile的形式来做 生成keyFile字符串 随便找台服务器生成 openssl rand -base64 756 将字符串配置为k8s的秘钥文件 # 挂载秘钥 挂载到临时目录,读写 /mongoconf/keyFile 启动命令: /bin/sh,-c 参数 cp /mongoconf/keyFile /home/keyFile;chmod 600 /home/keyFile;mongod --replSet=rs0 --bind_ip=0.0.0.0 --auth --keyFile=/home/keyFile # 启动3个mongo # 建立集群连接 用rs.initiate # 登录主节点 use admin # 创建账号密码 db.createUser({user:'admin',pwd:'NoteShare2020',roles:['userAdminAnyDatabase']}) db.auth("admin","NoteShare2020") # 更改为最高权限,之前创建的权限不够,无法恢复数据库 db.updateUser('admin',{pwd:'NoteShare2020',roles:[{role:'root',db:'admin'}]}) # 注意用navicat连接时,如果之前连接过未输入密码的,需要删除重新建立连接,感觉有缓存啥的存在误导 # 恢复数据到集群 在primary上进行,在没有赋予root权限时会报错,加上权限就好了。 mongorestore -h 192.168.3.161:30384 -uadmin -pNoteShare2020 --authenticationDatabase admin -d qyzc_jy --dir /home/backup/2020_11_18/qyzc_jy # 对恢复的库设置账号密码和附加权限,在主节点上进行 rs0:PRIMARY> use admin switched to db admin rs0:PRIMARY> db.auth("admin","NoteShare2020") 1 rs0:PRIMARY> use qyzc_jy switched to db qyzc_jy rs0:PRIMARY> db.createUser({user:'noteshare',pwd:'NoteShare2020',roles:['readWrite']}) Successfully added user: { "user" : "noteshare", "roles" : [ "readWrite" ] } ``` ### 单节点数据迁移到集群 ``` # 注意不能把之前的单节点直接加入集群,那样会把单节点的数据同步成集群先用的数据,会造成数据丢失 # 暂停服务器防止数据的继续写入 ===版主这里的应用暂时没有要求那么高,允许短时不提供服务,这里应该探索不断服务的方式来迁移,后续再研究下。 # 把单节点的数据备份下来 # 在主节点上通过恢复命令把数据恢复的集群中 mongorestore -d qyzc_jy --dir /home/backup/2020_11_18/qyzc_jy # 应用更新连接到集群服务检查数据情况 mongo.properties # mongo settings dataBaseName=qyzc_jy #hostName=192.168.3.161 #port=31013 #userName=test #password=test2020 maxWaitTime=300000 socketTimeout=100000 maxConnectionLifeTime=86400000 connectTimeout=43200000 ##副本集模式下 ##是否副本集 默认 false isReplica=true ##主机地址之间以英文逗号分隔 hostName=192.168.3.161,192.168.3.161,192.168.3.161,192.168.3.161 ##主机端口之间以英文逗号分隔,个数与主机地址匹配,不够的默认为最后一个 port=30384,31963,31313,32702 #链接池数量 #connectionsPerHost=100 #连接超时时间默认10,000 #connectTimeout=10000 #最大等待时间默认120,000 #maxWaitTime=120000 #I/O socket读写时间,默认0代表不限制 #socketTimeout=0 ``` ### 通过挂载mongo.conf配置文件部署 发现挂载没起作用,可能挂载方式有问题还需要了解下原理。不知道是不是挂载是在启动mongo之后的动作,导致挂载进去的文件没起作用。 另外可以研究下`mikefarah/yq:2.4.1`容器的处理过程,看到较多是用这个来处理挂载配置文件的。 ### 操作打印留痕 ``` # mongo --port 27017 MongoDB shell version v4.4.2 connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb Implicit session: session { "id" : UUID("063f2976-cad8-477e-ba16-91c1738d9668") } MongoDB server version: 4.4.2 Welcome to the MongoDB shell. For interactive help, type "help". For more comprehensive documentation, see https://docs.mongodb.com/ Questions? Try the MongoDB Developer Community Forums https://community.mongodb.com --- The server generated these startup warnings when booting: 2020-11-26T11:24:09.169+00:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted 2020-11-26T11:24:09.169+00:00: You are running this process as the root user, which is not recommended 2020-11-26T11:24:09.170+00:00: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. We suggest setting it to 'never' 2020-11-26T11:24:09.170+00:00: /sys/kernel/mm/transparent_hugepage/defrag is 'always'. We suggest setting it to 'never' --- --- Enable MongoDB's free cloud-based monitoring service, which will then receive and display metrics about your deployment (disk utilization, CPU, operation statistics, etc). The monitoring data will be available on a MongoDB website with a unique URL accessible to you and anyone you share the URL with. MongoDB may use this information to make product improvements and to suggest MongoDB products and deployment options to you. To enable free monitoring, run the following command: db.enableFreeMonitoring() To permanently disable this reminder, run the following command: db.disableFreeMonitoring() --- > rs.initiate( ... { ... _id: "rs0", ... version: 1, ... members: [ ... { _id: 0, host : "mongo-rs-svc:27017" }, ... { _id: 1, host : "mongo-rs2-svc:27017" }, ... { _id: 2, host : "mongo-rs3-svc:27017" } ... ] ... } ... ) { "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1606389935, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1606389935, 1) } rs0:SECONDARY> rs0:SECONDARY> rs.status() { "set" : "rs0", "date" : ISODate("2020-11-26T11:26:19.508Z"), "myState" : 1, "term" : NumberLong(1), "syncSourceHost" : "", "syncSourceId" : -1, "heartbeatIntervalMillis" : NumberLong(2000), "majorityVoteCount" : 2, "writeMajorityCount" : 2, "votingMembersCount" : 3, "writableVotingMembersCount" : 3, "optimes" : { "lastCommittedOpTime" : { "ts" : Timestamp(1606389976, 1), "t" : NumberLong(1) }, "lastCommittedWallTime" : ISODate("2020-11-26T11:26:16.269Z"), "readConcernMajorityOpTime" : { "ts" : Timestamp(1606389976, 1), "t" : NumberLong(1) }, "readConcernMajorityWallTime" : ISODate("2020-11-26T11:26:16.269Z"), "appliedOpTime" : { "ts" : Timestamp(1606389976, 1), "t" : NumberLong(1) }, "durableOpTime" : { "ts" : Timestamp(1606389976, 1), "t" : NumberLong(1) }, "lastAppliedWallTime" : ISODate("2020-11-26T11:26:16.269Z"), "lastDurableWallTime" : ISODate("2020-11-26T11:26:16.269Z") }, "lastStableRecoveryTimestamp" : Timestamp(1606389946, 3), "electionCandidateMetrics" : { "lastElectionReason" : "electionTimeout", "lastElectionDate" : ISODate("2020-11-26T11:25:46.253Z"), "electionTerm" : NumberLong(1), "lastCommittedOpTimeAtElection" : { "ts" : Timestamp(0, 0), "t" : NumberLong(-1) }, "lastSeenOpTimeAtElection" : { "ts" : Timestamp(1606389935, 1), "t" : NumberLong(-1) }, "numVotesNeeded" : 2, "priorityAtElection" : 1, "electionTimeoutMillis" : NumberLong(10000), "numCatchUpOps" : NumberLong(0), "newTermStartDate" : ISODate("2020-11-26T11:25:46.263Z"), "wMajorityWriteAvailabilityDate" : ISODate("2020-11-26T11:25:47.020Z") }, "members" : [ { "_id" : 0, "name" : "mongo-rs-svc:27017", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 131, "optime" : { "ts" : Timestamp(1606389976, 1), "t" : NumberLong(1) }, "optimeDate" : ISODate("2020-11-26T11:26:16Z"), "syncSourceHost" : "", "syncSourceId" : -1, "infoMessage" : "", "electionTime" : Timestamp(1606389946, 1), "electionDate" : ISODate("2020-11-26T11:25:46Z"), "configVersion" : 1, "configTerm" : 1, "self" : true, "lastHeartbeatMessage" : "" }, { "_id" : 1, "name" : "mongo-rs2-svc:27017", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 44, "optime" : { "ts" : Timestamp(1606389976, 1), "t" : NumberLong(1) }, "optimeDurable" : { "ts" : Timestamp(1606389976, 1), "t" : NumberLong(1) }, "optimeDate" : ISODate("2020-11-26T11:26:16Z"), "optimeDurableDate" : ISODate("2020-11-26T11:26:16Z"), "lastHeartbeat" : ISODate("2020-11-26T11:26:18.259Z"), "lastHeartbeatRecv" : ISODate("2020-11-26T11:26:19.266Z"), "pingMs" : NumberLong(0), "lastHeartbeatMessage" : "", "syncSourceHost" : "mongo-rs-svc:27017", "syncSourceId" : 0, "infoMessage" : "", "configVersion" : 1, "configTerm" : 1 }, { "_id" : 2, "name" : "mongo-rs3-svc:27017", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 44, "optime" : { "ts" : Timestamp(1606389976, 1), "t" : NumberLong(1) }, "optimeDurable" : { "ts" : Timestamp(1606389976, 1), "t" : NumberLong(1) }, "optimeDate" : ISODate("2020-11-26T11:26:16Z"), "optimeDurableDate" : ISODate("2020-11-26T11:26:16Z"), "lastHeartbeat" : ISODate("2020-11-26T11:26:18.258Z"), "lastHeartbeatRecv" : ISODate("2020-11-26T11:26:19.266Z"), "pingMs" : NumberLong(0), "lastHeartbeatMessage" : "", "syncSourceHost" : "mongo-rs-svc:27017", "syncSourceId" : 0, "infoMessage" : "", "configVersion" : 1, "configTerm" : 1 } ], "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1606389976, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1606389976, 1) } rs0:PRIMARY> ```
所有评论列表
点我发表评论