某个MongoDB数据库出现如下错误”assertion 10334 BSONObj size: 1684627202 (0x64695F02) is invalid. Size must be between 0 and 16793600(16MB) First element: EOO ns:local.oplog.$main query:{ query: {}, orderby: { $natural: -1 } }”一例：
2015-05-12T16:12:43.872+0800 I - ***** SERVER RESTARTED *****
2015-05-12T16:12:43.883+0800 I - [initandlisten] MongoDB starting : pid=1697 port=27019 dbpath=arb/mongodb master=1 64-bit host=ubuntu
2015-05-12T16:12:43.883+0800 I - [initandlisten]
2015-05-12T16:12:43.884+0800 I - [initandlisten] ** NOTE: This is a development version (2.7.8) of MongoDB.
2015-05-12T16:12:43.884+0800 I - [initandlisten] ** Not recommended for production.
2015-05-12T16:12:43.884+0800 I - [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2015-05-12T16:12:43.884+0800 I - [initandlisten]
2015-05-12T16:12:43.884+0800 I - [initandlisten] db version v2.7.8
2015-05-12T16:12:43.884+0800 I - [initandlisten] git version: 0e9cb3b20498b9f167afaff7a5c4a4d1da7e06a2
2015-05-12T16:12:43.884+0800 I - [initandlisten] build info: Linux build4.nj1.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49
2015-05-12T16:12:43.884+0800 I - [initandlisten] allocator: tcmalloc
2015-05-12T16:12:43.884+0800 I - [initandlisten] options: { config: "/etc/mongodb.conf", net: { bindIp: "0.0.0.0" }, processManagement: { fork: true, pidFilePath: "arb/mongodb/mongodb.pid" }, sharding: { clusterRole: "configsvr" }, storage: { dbPath: "arb/mongodb", preallocDataFiles: false }, systemLog: { destination: "file", logAppend: true, path: "ar/log/mongodb.log" } }
2015-05-12T16:12:43.887+0800 I JOURNAL [initandlisten] journal dir=arb/mongodb/journal
2015-05-12T16:12:43.888+0800 I JOURNAL [initandlisten] recover : no journal files present, no recovery needed
2015-05-12T16:12:43.914+0800 I - [initandlisten] Assertion: 10334:BSONObj size: 1684627202 (0x64695F02) is invalid. Size must be between 0 and 16793600(16MB) First element: EOO
mongod(main+0x134) [0x7bbf34]
libc.so.6(__libc_start_main+0xF5) [0x7f13b661aec5]
mongod(+0x3EEC01) [0x7eec01]
----- END BACKTRACE -----
2015-05-12T15:55:47.951+0800 I COMMANDS [initandlisten] assertion 10334 BSONObj size: 1684627202 (0x64695F02) is invalid. Size must be between 0 and 16793600(16MB) First element: EOO ns:local.oplog.$main query:{ query: {}, orderby: { $natural: -1 } }
2015-05-12T15:55:47.951+0800 I COMMANDS [initandlisten] ntoskip:0 ntoreturn:1
2015-05-12T15:55:47.952+0800 I STORAGE [initandlisten] exception in initAndListen: 13106 nextSafe(): { $err: "BSONObj size: 1684627202 (0x64695F02) is invalid. Size must be between 0 and 16793600(16MB) First element: EOO", code: 10334 }, terminating
2015-05-12T15:55:47.952+0800 I COMMANDS [initandlisten] now exiting
2015-05-12T15:55:47.952+0800 I NETWORK [initandlisten] shutdown: going to close listening sockets...
2015-05-12T15:55:47.952+0800 I NETWORK [initandlisten] removing socket file: /tmp/mongodb-27019.sock
2015-05-12T15:55:47.952+0800 I NETWORK [initandlisten] shutdown: going to flush diaglog...
2015-05-12T15:55:47.952+0800 I NETWORK [initandlisten] shutdown: going to close sockets...
2015-05-12T15:55:47.952+0800 I STORAGE [initandlisten] shutdown: waiting for fs preallocator...
2015-05-12T15:55:47.952+0800 I STORAGE [initandlisten] shutdown: final commit...
2015-05-12T15:55:47.953+0800 I STORAGE [initandlisten] shutdown: closing all files...
2015-05-12T15:55:47.954+0800 I STORAGE [initandlisten] closeAllFiles() finished
2015-05-12T15:55:47.954+0800 I JOURNAL [initandlisten] journalCleanup...
2015-05-12T15:55:47.954+0800 I JOURNAL [initandlisten] removeJournalFiles
2015-05-12T15:55:47.958+0800 I STORAGE [initandlisten] shutdown: removing fs lock...
2015-05-12T15:55:47.958+0800 I COMMANDS [initandlisten] dbexit: rc: 100
.
针对该 BSONObj size invalid，主要是mongodb读取到数据文件后发现存在BSON对象的大小超过16MB的非法对象所以报错，由于其认为数据库可能存在损坏，所以会强制关闭mongd实例
针对此类案例一般考虑：
1、repair database ,修复数据库
2、如果确实无法修复数据库，则考虑从备份中恢复数据库
一、repair database的例子：
mkdir /m01/mongo-data02
mongodb数据文件位于 /m01/mongo-data01 ，使用repairpath指定修复后的文件的位置。
注意/m01/mongo-data01 下需要有journal
root@shard0:~#
root@shard0:~# mongod -dbpath /m01/mongo-data01 --repair --repairpath /m01/mongo-data02 --storageEngine wiredTiger|less
2015-05-12T04:39:41.234-0400 I CONTROL [initandlisten] MongoDB starting : pid=19575 port=27017 dbpath=/m01/mongo-data01 64-bit host=shard0.dbdao.com
2015-05-12T04:39:41.234-0400 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2015-05-12T04:39:41.234-0400 I CONTROL [initandlisten]
2015-05-12T04:39:41.234-0400 I CONTROL [initandlisten]
2015-05-12T04:39:41.234-0400 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2015-05-12T04:39:41.234-0400 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2015-05-12T04:39:41.234-0400 I CONTROL [initandlisten]
2015-05-12T04:39:41.235-0400 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2015-05-12T04:39:41.235-0400 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2015-05-12T04:39:41.235-0400 I CONTROL [initandlisten]
2015-05-12T04:39:41.235-0400 I CONTROL [initandlisten] db version v3.0.0
2015-05-12T04:39:41.235-0400 I CONTROL [initandlisten] git version: a841fd6394365954886924a35076691b4d149168
2015-05-12T04:39:41.235-0400 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1f 6 Jan 2014
2015-05-12T04:39:41.235-0400 I CONTROL [initandlisten] build info: Linux ip-10-179-177-12 3.13.0-24-generic #46-Ubuntu SMP Thu Apr 10 19:11:08 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49
2015-05-12T04:39:41.235-0400 I CONTROL [initandlisten] allocator: tcmalloc
2015-05-12T04:39:41.235-0400 I CONTROL [initandlisten] options: { repair: true, storage: { dbPath: "/m01/mongo-data01", engine: "wiredTiger", repairPath: "/m01/mongo-data02" } }
2015-05-12T04:39:41.262-0400 W - [initandlisten] Detected unclean shutdown - /m01/mongo-data01/mongod.lock is not empty.
2015-05-12T04:39:41.263-0400 W STORAGE [initandlisten] Recovering data from the last clean checkpoint.
2015-05-12T04:39:41.263-0400 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),statistics=(fast),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2015-05-12T04:39:41.317-0400 I STORAGE [initandlisten] Repairing size cache
2015-05-12T04:39:41.320-0400 I STORAGE [initandlisten] WiredTiger progress session.verify 2
2015-05-12T04:39:41.320-0400 I STORAGE [initandlisten] Verify succeeded on uri table:sizeStorer. Not salvaging.
2015-05-12T04:39:41.321-0400 I STORAGE [initandlisten] Repairing catalog metadata
2015-05-12T04:39:41.323-0400 I STORAGE [initandlisten] WiredTiger progress session.verify 2
2015-05-12T04:39:41.323-0400 I STORAGE [initandlisten] Verify succeeded on uri table:_mdb_catalog. Not salvaging.
2015-05-12T04:39:41.329-0400 I STORAGE [initandlisten] repairDatabase local
2015-05-12T04:39:41.329-0400 I STORAGE [initandlisten] Repairing collection local.startup_log
2015-05-12T04:39:41.332-0400 I STORAGE [initandlisten] WiredTiger progress session.verify 2
2015-05-12T04:39:41.332-0400 I STORAGE [initandlisten] Verify succeeded on uri table:collection-0--8766758909797526255. Not salvaging.
2015-05-12T04:39:41.338-0400 I INDEX [initandlisten] build index on: local.startup_log properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "local.startup_log" }
2015-05-12T04:39:41.338-0400 I INDEX [initandlisten] building index using bulk method
2015-05-12T04:39:41.343-0400 I STORAGE [initandlisten] repairDatabase test
2015-05-12T04:39:41.343-0400 I STORAGE [initandlisten] Repairing collection test. otherStuff
2015-05-12T04:39:41.349-0400 I STORAGE [initandlisten] WiredTiger progress session.verify 2
2015-05-12T04:39:41.350-0400 I STORAGE [initandlisten] Verify succeeded on uri table:collection-4--6404712422294369090. Not salvaging.
2015-05-12T04:39:41.355-0400 I INDEX [initandlisten] build index on: test. otherStuff properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "test. otherStuff" }
2015-05-12T04:39:41.355-0400 I INDEX [initandlisten] building index using bulk method
2015-05-12T04:39:41.360-0400 I STORAGE [initandlisten] Repairing collection test.alpha
关于repair 更多信息可以参考 http://docs.mongodb.org/manual/reference/command/repairDatabase/
二、通过原有备份的mongodump来恢复数据
Comment