MongoDB 配置Sharding Cluster 基于Ubuntu

本教程基于Ubuntu 14.04.2 LTS \n \l和mongoDB 3.0, 配置了3个 Config Servers(在产品环境中被要求),2个Query Router查询分发(最少1个必要),和3个shard Server(最少2个)。

注意确保在所有上述Server上已经安装了MongoDB 3.0以上版本,否则首先安装MongoDB,要在Ubuntu上安装MongoDB可以按照这个教程,《升级MongoDB到3.0.2并启用wiredTiger存储引擎 基于ubuntu 14.04》,https://www.askmac.cn/archives/upgrade-to-mongodb-302-wiredtiger.html

how-to-do-mongodb-sharding

 

以下是HOSTNAME 和 IP 列表,注意本文的网络拓扑按照如下配置,如果与之不同则需要相应修改

 10.132.24.62    config0.dbdao.com
 10.132.24.60    config1.dbdao.com
 10.132.24.61    config2.dbdao.com
 10.132.24.59    query0.dbdao.com 
 10.132.24.63    query1.dbdao.com 
 10.132.24.64    shard0.dbdao.com 
 10.132.24.65    shard1.dbdao.com 
 10.132.24.67    shard2.dbdao.com 

第一步配置config server

 

需要在配置query router和shard server前配置3个 mongoDB config server:
以下需要在3个 mongoDB config server上均需以root用户执行

以下在三个config server上配置目录


 mkdir -p /m01/mongo-metadata
 chown mongodb:mongodb /m01/mongo-metadata


在三个config server上分别执行,注意 IP要一一对应

 config0: mongod --configsvr --dbpath /m01/mongo-metadata --bind_ip 10.132.24.62 --port 35001
 config1: mongod --configsvr --dbpath /m01/mongo-metadata --bind_ip 10.132.24.60 --port 35001
 config2: mongod --configsvr --dbpath /m01/mongo-metadata --bind_ip 10.132.24.61 --port 35001

 

 

第二步配置Query Router Instances

分别在2个Query Router服务器上执行下列的命令

 

确保2个Query Router服务器上的mongd服务已经关闭

su - root 
 service mongod stop
 启动mongos


在2个Query Router服务器上执行下列命令
su - root 

 mongos --configdb config0.dbdao.com:35001,config1.dbdao.com:35001,config2.dbdao.com:35001

 第三步 增加shard到集群

 

注意这里没有配置Replication Set RS ,如果需要配置replica-set可以关注maclean今后的文章。

以root用户在三个sharding server上执行下面的命令:

 

 

 service mongod stop
 mkdir -p /m01/mongo-data
 chown mongodb:mongodb /m01/mongo-data
 
以下是针对不同server分别执行


shard0:  mongod --dbpath /m01/mongo-data --bind_ip 10.132.24.64 --port 35001 --storageEngine  wiredTiger
shard1:  mongod --dbpath /m01/mongo-data --bind_ip 10.132.24.65 --port 35001 --storageEngine  wiredTiger
shard2:  mongod --dbpath /m01/mongo-data --bind_ip 10.132.24.67 --port 35001 --storageEngine  wiredTiger


之后登陆query router来增加shard

sh.addShard( "shard0.dbdao.com:35001" )
sh.addShard( "shard1.dbdao.com:35001" )
sh.addShard( "shard2.dbdao.com:35001" )

 

之后可以在DB级别启用sharding

mongo query0.dbdao.com:27017

 use test_db
db
show dbs
sh.enableSharding("test_db")

mongos> db.databases.find()
{ "_id" : "admin", "partitioned" : false, "primary" : "config" }
{ "_id" : "test_db", "partitioned" : true, "primary" : "shard0000" }


在collection级别启用sharding

mongo query0.dbdao.com:27017
use test_db


db.test_collection.ensureIndex( { _id : "hashed" } )


mongos> db.test_collection.ensureIndex( { _id : "hashed" } )
{
        "raw" : {
                "shard0.dbdao.com:35001" : {
                        "createdCollectionAutomatically" : true,
                        "numIndexesBefore" : 1,
                        "numIndexesAfter" : 2,
                        "ok" : 1
                }
        },
        "ok" : 1
}
mongos> sh.shardCollection("test_db.test_collection", { "_id": "hashed" } )

{ "collectionsharded" : "test_db.test_collection", "ok" : 1 }
mongos> 
mongos> use test_db
switched to db test_db
mongos> for (var i = 1; i <= 500; i++) db.test_collection.insert( { x : i } ) WriteResult({ "nInserted" : 1 }) mongos> db.test_collection.find();
{ "_id" : ObjectId("554b296c160953211da4b523"), "x" : 2 }
{ "_id" : ObjectId("554b296c160953211da4b522"), "x" : 1 }
{ "_id" : ObjectId("554b296c160953211da4b524"), "x" : 3 }
{ "_id" : ObjectId("554b296c160953211da4b526"), "x" : 5 }
{ "_id" : ObjectId("554b296c160953211da4b529"), "x" : 8 }
{ "_id" : ObjectId("554b296c160953211da4b525"), "x" : 4 }
{ "_id" : ObjectId("554b296c160953211da4b52c"), "x" : 11 }
{ "_id" : ObjectId("554b296c160953211da4b52b"), "x" : 10 }
{ "_id" : ObjectId("554b296c160953211da4b527"), "x" : 6 }
{ "_id" : ObjectId("554b296c160953211da4b52d"), "x" : 12 }
{ "_id" : ObjectId("554b296c160953211da4b52f"), "x" : 14 }
{ "_id" : ObjectId("554b296c160953211da4b528"), "x" : 7 }
{ "_id" : ObjectId("554b296c160953211da4b52e"), "x" : 13 }
{ "_id" : ObjectId("554b296c160953211da4b530"), "x" : 15 }
{ "_id" : ObjectId("554b296c160953211da4b52a"), "x" : 9 }
{ "_id" : ObjectId("554b296c160953211da4b531"), "x" : 16 }
{ "_id" : ObjectId("554b296c160953211da4b532"), "x" : 17 }
{ "_id" : ObjectId("554b296c160953211da4b533"), "x" : 18 }
{ "_id" : ObjectId("554b296c160953211da4b53b"), "x" : 26 }
{ "_id" : ObjectId("554b296c160953211da4b534"), "x" : 19 }
Type "it" for more
mongos> db.test_collection.count();
500



mongos> sh.status();
--- Sharding Status --- 
  sharding version: {
        "_id" : 1,
        "minCompatibleVersion" : 5,
        "currentVersion" : 6,
        "clusterId" : ObjectId("554b241f4df23a46a60f6a9c")
}
  shards:
        {  "_id" : "shard0000",  "host" : "shard0.dbdao.com:35001" }
        {  "_id" : "shard0001",  "host" : "shard1.dbdao.com:35001" }
        {  "_id" : "shard0002",  "host" : "shard2.dbdao.com:35001" }
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours: 
                2 : Success
                1 : Failed with error 'could not acquire collection lock for test_db.test_collection to migrate chunk [{ : MinKey },{ : MaxKey }) :: caused by :: Lock for migrating chunk [{ : MinKey }, { : MaxKey }) in test_db.test_collection is taken.', from shard0000 to shard0002
  databases:
        {  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }
        {  "_id" : "test_db",  "partitioned" : true,  "primary" : "shard0000" }
                test_db.test_collection
                        shard key: { "_id" : "hashed" }
                        chunks:
                                shard0000       2
                                shard0001       2
                                shard0002       2
                        { "_id" : { "$minKey" : 1 } } -->> { "_id" : NumberLong("-6148914691236517204") } on : shard0000 Timestamp(3, 2) 
                        { "_id" : NumberLong("-6148914691236517204") } -->> { "_id" : NumberLong("-3074457345618258602") } on : shard0000 Timestamp(3, 3) 
                        { "_id" : NumberLong("-3074457345618258602") } -->> { "_id" : NumberLong(0) } on : shard0001 Timestamp(3, 4) 
                        { "_id" : NumberLong(0) } -->> { "_id" : NumberLong("3074457345618258602") } on : shard0001 Timestamp(3, 5) 
                        { "_id" : NumberLong("3074457345618258602") } -->> { "_id" : NumberLong("6148914691236517204") } on : shard0002 Timestamp(3, 6) 
                        { "_id" : NumberLong("6148914691236517204") } -->> { "_id" : { "$maxKey" : 1 } } on : shard0002 Timestamp(3, 7) 

> use test_db;
switched to db test_db						
> db.serverStatus();
{
        "host" : "shard1.dbdao.com:35001",
> db.test_collection.count();
171


> db.serverStatus();
{
        "host" : "shard0.dbdao.com:35001",


> use test_db;
switched to db test_db
> 
> db.test_collection.count();
169

>  db.serverStatus();
{
        "host" : "shard2.dbdao.com:35001",


> use test_db;
switched to db test_db
> db.test_collection.count();
160



以上三个sharding server的数据分别为 171、169、160 , 正好为总数500.

Comments

  1. vi /etc/hosts3 Config Servers (Required in production environments)2 Query Routers (Minimum of 1 necessary)4 Shard Servers (Minimum of 2 necessary) 10.132.24.62 config0.dbdao.com 10.132.24.60 config1.dbdao.com 10.132.24.61 config2.dbdao.com 10.132.24.59 query0.dbdao.com 10.132.24.63 query1.dbdao.com 10.132.24.64 shard0.dbdao.com 10.132.24.65 shard1.dbdao.com 10.132.24.67 shard2.dbdao.com 第一步配置config server,他们需要可用在query router和shards可配置前。 使用root 登陆,先创建目录 mkdir -p /m01/mongo-metadata chown mongodb:mongodb /m01/mongo-metadata mongod –configsvr –dbpath /m01/mongo-metadata –bind_ip 10.132.24.62 –port 35001 The server will start outputting information and will begin listening for connections from other components.Repeat this process exactly on the other two configuration servers. The port number should be the same across all three servers. config0: mongod –configsvr –dbpath /m01/mongo-metadata –bind_ip 10.132.24.62 –port 35001 config1: mongod –configsvr –dbpath /m01/mongo-metadata –bind_ip 10.132.24.60 –port 35001 config2: mongod –configsvr –dbpath /m01/mongo-metadata –bind_ip 10.132.24.61 –port 35001 升级MongoDB到3.0.2并启用wiredTiger存储引擎 基于ubuntu 14.04 第二步配置Query Router Instances service mongod stop 启动mongos mongos –configdb config0.dbdao.com:35001,config1.dbdao.com:35001,config2.dbdao.com:35001 保持上述语句在多个query上一样执行 第三步 增加shard到集群 这里没有用到configure replica sets service mongod stop mkdir -p /m01/mongo-data chown mongodb:mongodb /m01/mongo-data shard0: mongod –dbpath /m01/mongo-data –bind_ip 10.132.24.64 –port 35001 –storageEngine wiredTigershard1: mongod –dbpath /m01/mongo-data –bind_ip 10.132.24.65 –port 35001 –storageEngine wiredTigershard2: mongod –dbpath /m01/mongo-data –bind_ip 10.132.24.67 –port 35001 –storageEngine wiredTigermongo query0.dbdao.com:27017sh.addShard( “shard0.dbdao.com:35001” )sh.addShard( “shard1.dbdao.com:35001” )sh.addShard( “shard2.dbdao.com:35001” )为Collection启用Sharding mongo config0.dbdao.com:35001connecting to: query0.dbdao.com:27017/testServer has startup warnings: 2015-05-07T04:37:31.488-0400 I CONTROL ** WARNING: You are running this process as the root user, which is not recommended.2015-05-07T04:37:31.488-0400 I CONTROL mongos> mongos> use test_dbswitched to db test_dbmongos> dbtest_dbmongos> show dbsadmin (empty)config 0.016GBmongos> sh.enableSharding(“test_db”){ “ok” : 1 }mongos> use configswitched to db configmongos> db.databases.find(){ “_id” : “admin”, “partitioned” : false, “primary” : “config” }{ “_id” : “test_db”, “partitioned” : true, “primary” : “shard0000” }mongos> mongos> mongos> mongos> use test_dbswitched to db test_dbmongos> mongos> use test_dbswitched to db test_dbmongos> db.test_collection.ensureIndex( { _id : “hashed” } ){ “raw” : { “shard0.dbdao.com:35001” : { “createdCollectionAutomatically” : true, “numIndexesBefore” : 1, “numIndexesAfter” : 2, “ok” : 1 } }, “ok” : 1}mongos> sh.shardCollection(“test_db.test_collection”, { “_id”: “hashed” } ){ “collectionsharded” : “test_db.test_collection”, “ok” : 1 }mongos> mongos> use test_dbswitched to db test_dbmongos> for (var i = 1; i <= 500; i++) db.test_collection.insert( { x : i } )WriteResult({ “nInserted” : 1 })mongos> db.test_collection.find();{ “_id” : ObjectId(“554b296c160953211da4b523”), “x” : 2 }{ “_id” : ObjectId(“554b296c160953211da4b522”), “x” : 1 }{ “_id” : ObjectId(“554b296c160953211da4b524”), “x” : 3 }{ “_id” : ObjectId(“554b296c160953211da4b526”), “x” : 5 }{ “_id” : ObjectId(“554b296c160953211da4b529”), “x” : 8 }{ “_id” : ObjectId(“554b296c160953211da4b525”), “x” : 4 }{ “_id” : ObjectId(“554b296c160953211da4b52c”), “x” : 11 }{ “_id” : ObjectId(“554b296c160953211da4b52b”), “x” : 10 }{ “_id” : ObjectId(“554b296c160953211da4b527”), “x” : 6 }{ “_id” : ObjectId(“554b296c160953211da4b52d”), “x” : 12 }{ “_id” : ObjectId(“554b296c160953211da4b52f”), “x” : 14 }{ “_id” : ObjectId(“554b296c160953211da4b528”), “x” : 7 }{ “_id” : ObjectId(“554b296c160953211da4b52e”), “x” : 13 }{ “_id” : ObjectId(“554b296c160953211da4b530”), “x” : 15 }{ “_id” : ObjectId(“554b296c160953211da4b52a”), “x” : 9 }{ “_id” : ObjectId(“554b296c160953211da4b531”), “x” : 16 }{ “_id” : ObjectId(“554b296c160953211da4b532”), “x” : 17 }{ “_id” : ObjectId(“554b296c160953211da4b533”), “x” : 18 }{ “_id” : ObjectId(“554b296c160953211da4b53b”), “x” : 26 }{ “_id” : ObjectId(“554b296c160953211da4b534”), “x” : 19 }Type “it” for moremongos> db.test_collection.count();500mongos> db.test_collection.count();500mongos> db.test_collection.count();500mongos> db.test_collection.count();500mongos> sh.status();— Sharding Status — sharding version: { “_id” : 1, “minCompatibleVersion” : 5, “currentVersion” : 6, “clusterId” : ObjectId(“554b241f4df23a46a60f6a9c”)} shards: { “_id” : “shard0000”, “host” : “shard0.dbdao.com:35001” } { “_id” : “shard0001”, “host” : “shard1.dbdao.com:35001” } { “_id” : “shard0002”, “host” : “shard2.dbdao.com:35001” } balancer: Currently enabled: yes Currently running: no Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: 2 : Success 1 : Failed with error ‘could not acquire collection lock for test_db.test_collection to migrate chunk [{ : MinKey },{ : MaxKey }) :: caused by :: Lock for migrating chunk [{ : MinKey }, { : MaxKey }) in test_db.test_collection is taken.’, from shard0000 to shard0002 databases: { “_id” : “admin”, “partitioned” : false, “primary” : “config” } { “_id” : “test_db”, “partitioned” : true, “primary” : “shard0000” } test_db.test_collection shard key: { “_id” : “hashed” } chunks: shard0000 2 shard0001 2 shard0002 2 { “_id” : { “$minKey” : 1 } } –>> { “_id” : NumberLong(“-6148914691236517204”) } on : shard0000 Timestamp(3, 2) { “_id” : NumberLong(“-6148914691236517204”) } –>> { “_id” : NumberLong(“-3074457345618258602”) } on : shard0000 Timestamp(3, 3) { “_id” : NumberLong(“-3074457345618258602”) } –>> { “_id” : NumberLong(0) } on : shard0001 Timestamp(3, 4) { “_id” : NumberLong(0) } –>> { “_id” : NumberLong(“3074457345618258602”) } on : shard0001 Timestamp(3, 5) { “_id” : NumberLong(“3074457345618258602”) } –>> { “_id” : NumberLong(“6148914691236517204”) } on : shard0002 Timestamp(3, 6) { “_id” : NumberLong(“6148914691236517204”) } –>> { “_id” : { “$maxKey” : 1 } } on : shard0002 Timestamp(3, 7) use test_db; db.test_collection.count();

Comment

*

沪ICP备14014813号-2

沪公网安备 31010802001379号