[SERVER-1891] Audit "DDL" operations Created: 05/Oct/10  Updated: 16/Nov/21  Resolved: 14/Nov/13

Status: Closed
Project: Core Server
Component/s: Logging, Security
Affects Version/s: 1.7.0
Fix Version/s: 2.5.4

Type: New Feature Priority: Major - P3
Reporter: Alvin Richards (Inactive) Assignee: Eric Milkie
Resolution: Done Votes: 13
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PDF File U_INS_generic_v8r1.6_Checklist_20100827.pdf    
Issue Links:
Depends
is depended on by DOCS-1834 Document Auditing "DDL" operations Closed
is depended on by SERVER-11028 shutdown on Audit failure Closed
is depended on by SERVER-4321 MongoDB Logging Related Issue Closed
Duplicate
is duplicated by SERVER-8359 Database changes are written to an au... Closed
is duplicated by SERVER-8876 DDL and Authentication Audit Logs Closed
Related
related to SERVER-11192 Audit system cannot ascribe DDL opera... Closed
related to SERVER-11594 JSON format audit records Closed
is related to SERVER-7091 Include authenticated user in log mes... Closed
Participants:

 Description   

Problem:
A frequent request is to log any "DDL" operation that occurs
– drop collection
– ensureIndex
– dropIndex
– etc.

This would allow an organization to audit these changes, info the could be recorded would be
– operation
– date / time
– client connection (hostname, ip)
– etc.

Solution:
There are a couple of possible way this could work

  • Log these operations into the existing log file
  • Log these operations into a Capped Collection (but will impact DB throughput)
  • Log these operations into a new audit log file


 Comments   
Comment by Githook User [ 12/Dec/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-1891 added comment explaining that the shardCollection actiontype is only for the auditlog
Branch: master
https://github.com/mongodb/mongo/commit/c47b59b9461760398ca5027310704b6d8f6a2376

Comment by Githook User [ 12/Dec/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-1891 added shardCollection actiontype for the auditlog
Branch: master
https://github.com/mongodb/mongo/commit/8d3220958d1ffd5f9bb89f1e574ef8235075bef5

Comment by Githook User [ 12/Dec/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-1891 change maxsize to maxSize for addShard auditlogging
Branch: master
https://github.com/mongodb/mongo/commit/6923d227113a44bdec68e450a27a191284d465a1

Comment by Githook User [ 12/Dec/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-1891 correct actiontypes for auditlogging of sharding events
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/c3f0a4fb9562d8d0d6420fafd172c9169d2a1046

Comment by Githook User [ 12/Dec/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-1891 change maxsize to maxSize for addShard auditlogging
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/effd595128394f62a9dee182017c3d04058e5ce6

Comment by auto [ 05/Nov/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-1891 Hookup audit logging for sharding events.
Branch: master
https://github.com/mongodb/mongo/commit/3d97d00c3a94283241210c0abc27eb0039093ae0

Comment by auto [ 05/Nov/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-1891 Add audit logging functions for sharding events
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/303e45ef0e7412c3df660e731d0e174663538111

Comment by auto [ 11/Oct/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-1891 fix auditlog parameter parsing to work properly with no args
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/fbce1e8e6cd834b341f47601db2e44afa3523719

Comment by auto [ 11/Oct/13 ]

Author:

{u'username': u'erh', u'name': u'Eliot Horowitz', u'email': u'eliot@10gen.com'}

Message: SERVER-1891: connect auditing hooks for databases/collections/index creation/drop
Branch: master
https://github.com/mongodb/mongo/commit/843df1203958c4fd55c2078ff11098697084bf98

Comment by auto [ 11/Oct/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-1891 Hookup audit logging for user management and DDL events.
Branch: master
https://github.com/mongodb/mongo/commit/cbd990534291bd55d5e7ac6508ae0ae499c2a719

Comment by auto [ 11/Oct/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-1891 Add audit logging hooks for user management and DDL events.
Branch: master
https://github.com/mongodb/mongo/commit/3293653f6d6f8d3bcd9e1ae11107923dd056d35e

Comment by auto [ 11/Oct/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-1891 Add audit logging functions for user management and DDL events
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/184ba5018fc3a7742b6ad85611a97f7b80af3a0b

Comment by auto [ 11/Oct/13 ]

Author:

{u'username': u'milkie', u'name': u'Eric Milkie', u'email': u'milkie@10gen.com'}

Message: SERVER-1891 added syslog support and fixed up command line options as per spec
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/9f96f52dba8a5a0db14318be34289d722c8ce656

Comment by auto [ 10/Oct/13 ]

Author:

{u'username': u'milkie', u'name': u'Eric Milkie', u'email': u'milkie@10gen.com'}

Message: SERVER-1891 support BSON output
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/1cc32c4451f3d4fceed9cce5e7eaf443be209db5

Comment by auto [ 10/Oct/13 ]

Author:

{u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}

Message: Revert "SERVER-1891 Hookup audit logging for user management and DDL events."

This reverts commit 91480e08d9a6dd41ad6118ee0bf5461a99cbdbc3.
Branch: master
https://github.com/mongodb/mongo/commit/9630c5a8595ee00fa1902db0b181a8a4beb4dc8a

Comment by auto [ 09/Oct/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-1891 Hookup audit logging for user management and DDL events.
Branch: master
https://github.com/mongodb/mongo/commit/91480e08d9a6dd41ad6118ee0bf5461a99cbdbc3

Comment by auto [ 09/Oct/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-1891 do not auditlog authorization successes
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/480b0c5a2f78e30c5f2aba114b5d9f46cf498a2b

Comment by auto [ 09/Oct/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-1891 Add audit logging hook for renameCollection
Branch: master
https://github.com/mongodb/mongo/commit/d29d8008c20143e52ed11890d8728ed19e4f56eb

Comment by auto [ 09/Oct/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-1891 Add audit logging hooks for user management and DDL events.
Branch: master
https://github.com/mongodb/mongo/commit/42b1dfce39fc77156a6bbad531893d6deee6e049

Comment by auto [ 07/Oct/13 ]

Author:

{u'username': u'dannenberg', u'name': u'matt dannenberg', u'email': u'matt.dannenberg@10gen.com'}

Message: SERVER-1891 add new ActionTypes
Branch: master
https://github.com/mongodb/mongo/commit/fb65792d774388ce9a4fdf7c4b2cc45d08dc0ee8

Comment by auto [ 26/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Add hooks to audit authentications using SASL mechanisms.
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/937ac601e3b2327d05d49a77389f895497fcb69b

Comment by auto [ 26/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Implementation of audit::logAuthentication() for subscription build.
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/a01212d753c455f4778a75eb8c21359a7faae1fe

Comment by auto [ 26/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Add hooks to audit authentications using MONGODB-CR and MONGODB-X509.
Branch: master
https://github.com/mongodb/mongo/commit/78b54e5608d1a49da4228ee2b45489a9d0cc9182

Comment by auto [ 26/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Move some useful utility functions to audit_private.

{h,cpp}

.
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/ca50288f82cc86006dccd1aedc9f023268e00dc3

Comment by auto [ 26/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Report dates in audit log using same format as diagnostic log.
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/0718ce40fa24005a4bb2d7a39da37752697a63ab

Comment by auto [ 23/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Audit authorization checks.

Logs success and failure of all authorization checks to the audit log, when auditing
is enabled.
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/cb6634e54413b8af1b4d86b175e9168484f892fa

Comment by auto [ 22/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Add redactForLogging() to Command.

Command::redactForLogging(mutablebson::Document* cmdObj) transforms "cmdObj"
into a form suitable for writing to logs. This patch provides a sample
implementation for the user management commands that censors password fields,
and updates the audit hook for commands, appropriately.
Branch: master
https://github.com/mongodb/mongo/commit/aa44eb2ce901ab2d82e8faa0799550ca15ccaf7e

Comment by auto [ 19/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Add audit logging hooks for authorization checks in mongos.
Branch: master
https://github.com/mongodb/mongo/commit/ede1257b6fc1169beedcf59b241fc07b921ddcb9

Comment by auto [ 19/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Fix SSL builds.
Branch: master
https://github.com/mongodb/mongo/commit/f7dfc4db117e0111aabf34c422a03b446c56bffa

Comment by auto [ 19/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Make it possible to get the local and remote SockAddr from an AbstractMessagingPort.
Branch: master
https://github.com/mongodb/mongo/commit/129d2ddff0d3d0b8cb041473525af41c63cd1470

Comment by auto [ 19/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Consolidate auth failure/success behavior in authenticate command, for easier logging.
Branch: master
https://github.com/mongodb/mongo/commit/f29364047e9328b689d52dc45222452bbee4b84e

Comment by auto [ 17/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Fix test broken by commit 0eb227c15841da86dbf9d21e7e593c7659040963.
Branch: master
https://github.com/mongodb/mongo/commit/beb5a898a6646b12a48977829c39fe084911ee34

Comment by auto [ 17/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Add audit logging hooks for authorization checks in mongod.
Branch: master
https://github.com/mongodb/mongo/commit/0eb227c15841da86dbf9d21e7e593c7659040963

Comment by auto [ 17/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Clean up includes some in sock.h and listen.h.
Branch: master
https://github.com/mongodb/mongo/commit/fec9c2a9fa948f9a3e3e1f3ef0ddd753532b3b24

Comment by auto [ 17/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Add hook for auditing access control checks for commands.
Branch: master
https://github.com/mongodb/mongo/commit/457e50ba8c19cc6df8d865a870268a5a5462de8b

Comment by auto [ 17/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Consolidate command authorization checking logic.

This reverts commit 2e2a6fdffdba369a0594962267e5bc7bb47a3f3a and also fixes the build break that the
original "Consolidate command authorization checking logic" patch introduced.
Branch: master
https://github.com/mongodb/mongo/commit/f62d600cb0a2680b72a35023e812140da50056ca

Comment by auto [ 10/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: Revert "SERVER-1891 Consolidate command authorization checking logic."

This reverts commit d430713c403fa6b065337cf7e480ed70940631ab.
Branch: master
https://github.com/mongodb/mongo/commit/2e2a6fdffdba369a0594962267e5bc7bb47a3f3a

Comment by auto [ 10/Jul/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-1891 Consolidate command authorization checking logic.
Branch: master
https://github.com/mongodb/mongo/commit/d430713c403fa6b065337cf7e480ed70940631ab

Comment by Matt Veitas [ 27/Jun/13 ]

I will second the comments made by Jason as this is the only thing holding us back from using Mongo in our environment that falls under HIPAA security regulations.

Comment by Jason Denizac [ 11/May/13 ]

Was referred to this ticket by Alvin at MongoSF today. Audit logging, especially valid and invalid authentication attempts, is very important for healthcare industry (HIPAA) security regulations. Other aspects of compliance can be dealt with elsewhere in the system, eg on disk encryption, application-level content access logging, etc. But it's important also to log the connection between the application and the db.

Comment by David McLennan [ 05/Sep/12 ]

Financial environments have similar auditing requirements - after reviewing the attached DoD document, sections DG0141, DG0142 & DG0145 are close to what we require. Essentially any auditing solution needs to:

i) Audit all authentication actions with metadata (username, Source IP, DB, Success/Fail etc.)

ii) Audit all security related configuration changes (new users, changed passwords, changed permissions)

iii) The audit records need to be immutable (i.e. they cannot be modified or deleted by normal commands. Typically done by logging to a filesystem external to the instance or to the windows event log / Unix syslogd)

iv) Audit all commands by "highly privileged" users. For current releases of MongoDB, I would expect this would boil down to logging all queries by any admin user.

Comment by Will LaForest [ 04/Jan/12 ]

DISA STIG check list for DoD accreditation.

Comment by Will LaForest [ 04/Jan/12 ]

Auditing will be required in order to get widespread of use in DoD. I have attached the DISA DBMS STIG Document which talks about the checks used to determine if a DBMS is in compliance. If you search for audit you can find the pertinent checks.

Comment by Alvin Richards (Inactive) [ 06/May/11 ]

1) Need to log the user (through user authentication) who was logged in who performed the operation
2) Limited the auditing by DB or Collection

Generated at Thu Feb 08 02:58:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.