By Juraj Holub
Last Validated on May 28 2021 · Originally Published on May 28, 2021 · Viewed 2.9k times

Introduction

MongoDB is a multi-platform document database. It is one of the most popular NoSQL databases, which means that it is not based on the relational database schema. Instead of that, it is based on documents similar to the JSON format. In general, the database is the basis of almost every backend, and administrators want to log this service. MongoDB offers various configurable options for a custom set-up of the database logging.

In this tutorial, you will do following actions:

  • You will install the latest version of the MongoDB server.
  • You will learn about the most important configuration options related to database logging, and you will configure your own configuration setup.
  • You will understand the structure of the MongoDB log records and view them.

Prerequisites

You will need:

  • Ubuntu 20.04 distribution including the non-root user with sudo access.
  • You should understand the basics of syslog, and what is the JSON.

Step 1 — Installing Latest Version of MongoDB Server

This tutorial uses MongoDB 4.4 Community Edition. To install this MongoDB version, you can use the official mongodb-org package for the apt. Ubuntu provides also the package mongodb but this package is not maintained by the official MongoDB support. The package mongodb uses different configuration file format incompatible with this tutorial. In case that you have already installed the mongodb package on your Ubuntu system, you must first uninstall it before proceeding with these instructions.

Ubuntu requires the public GPG Key used by the package management system. You can download it with wget and include it to the Ubuntu packages with the apt-key utility (sudo required):

$ wget -qO - <https://www.mongodb.org/static/pgp/server-4.4.asc> | sudo apt-key add -

If the command succeeds then the apt shows the OK message. In case that you receive an error message, you must install package gnupg with the utility apt (sudo required):

$ sudo apt install gnupg

After successful installation retry importing the public key:

$ wget -qO - <https://www.mongodb.org/static/pgp/server-4.4.asc> | sudo apt-key add -

Now, let's create the file with the list of the MongoDB resources. You must create the file mongodb-org-4.4.list in the system directory /etc/apt/sources.list.d/mongodb-org-4.4.list (sudo required):

$ sudo nano /etc/apt/sources.list.d/mongodb-org-4.4.list

Fill, the new file with following configuration:

deb [ arch=amd64,arm64 ] <https://repo.mongodb.org/apt/ubuntu> focal/mongodb-org/4.4 multiverse

The configuration defines the source of the MongoDB repository for the apt. For further information, see the Ubuntu official documentation about source lists. Now, you can save and close the new file.

Your Ubuntu distribution now includes the required key and repository for the mongodb-org package. Let's update apt repositories and install the mongodb-org with the apt (sudo required):

$ sudo apt update
$ sudo apt install mongodb-org

At this point, your Ubuntu distribution includes the latest version of the official MongoDB server.


Step 2 — Configuring the MongoDB

MongoDB configuration is stored in the file /etc/mongod.conf. The configuration file uses YAML formatting. This file stores all configuration options for the database. There are also all supported settings for the configuration of database logging. The logging configuration is covered with the option systemLog. The following sample of YAML configuration shows the most important options included in the systemLog configuration:

systemLog:
   destination: <string>
   path: <string>
   logAppend: <boolean>
   verbosity: <int>
   logRotate: <string>
   timeStampFormat: <string>
   quiet: <boolean>

Note, that the sample is just the template without real values (type of value is indicated in the angle brackets, for example <int>).

The YAML syntax depends on the text alignment. Use spaces instead of tab, because the tab is not supported character for alignment.


Destination

The destination option specifies where to send all log generated by MongoDB. There are two possible values:

  • file: The database stores log into the custom file. If you set this value you must specify the destination file with the option path.
  • syslog: The logs are stored and maintained with the Syslog daemon. If you don't know the Syslog daemon, you can read our linux logging tutorial, where we cover the basics.
  • Not specifying either file or syslog sends all logging output to stdout.

Thy Syslog advantage is the centralised approach. You can easily find relevant logs with journalctl (How to Control Journald with Journalctl). On the other side, the Syslog timestamps are inaccurate because the daemon generates a timestamp when the message is recorded, not when the database generates the message.


Path

The path option specifies the absolute path to the custom log file. In general, the Linux default directory for logs is /var/log. However, you can use any directory you want.

This option is relevant only if the destination is set to file.


Log Append

The logAppend option is a boolean flag. If false, MongoDB backup the actual log and creates the new one after each service restart. Otherwise, the database appends records after service restart to the existing log. If not defined, the default value is set to false.


Verbosity

The verbosity option is an integer in the range from 0 to 5 that determines the minimal level of log message severity to be recorded:

  • 0 - MongoDB includes only informational messages.
  • 1 to 5 - The database includes informational and debugs messages with severity equal to verbosity level.

We will learn more about severity levels in the next step.


Log Rotate

The logRotate option set how the MongoDB rotate logs. If you don't know what the log rotation is, you can read the tutorial How to Manage Logs with Logrotate on Ubuntu 20.04. The option can hold two possible values:

  • rename: The MongoDB server renames active log by appending a timestamp to the filename, opens, and create a new active log file.
  • reopen: The server closes the log file and reopens the log with the same name, expecting that the logrotate daemon already rotates the closed file. In this case, the logAppend option must be set to true.

Timestamp Format

The timeStampFormat option set the format of timestamp in ISO-8601 format:

  • iso8601-utc: Displays timestamps in Coordinated Universal Time (UTC).
  • iso8601-local: Displays timestamps in local time in the ISO-8601 format.

For example, the date 2021-05-28 will be formatted as:

  • 2021-05-28T16:00:00,000000000+00:00: UTC.
  • 2021-05-28T18:00:00,000000000+02:00: Local time for America, Los Angeles.

Quiet

The quiet option is the boolean flag. If true, then the server runs in a quiet mode that attempts to limit the amount of output. By the official documentation, it is not recommended to use quiet in production systems as it may make tracking problems during particular connections much more difficult.


Editing Configuration

At last, let's edit the configuration file and fill the log configuration option with its own values. Keep in mind, that the values used in this example are not mandatory and you should use values that match your use case.

You can open the file /etc/mongod.conf (sudo required for file editing):

$ sudo nano /etc/mongod.conf

The file contains following lines that holds log configuration:

systemLog:
   destination: file
   path: "/var/log/mongodb/mongod.log"
   logAppend: true

As you can see, there are not used all options that we already know. Now, let's replace the entire systemLog YAML structure with our template and fill it with some setup:

systemLog:
   destination: file
   path: "/var/log/mongodb/mongod.log"
   logAppend: true
   verbosity: 0
   logRotate: "rename"
   timeStampFormat: "iso8601-local"
   quiet: false

When you set up your own values, you can save and close the configuration file. At last, you must restart the MongoDB service to apply a new configuration with systemctl (sudo required):

$ sudo systemctl restart mongod.service

Now, the service restarts and the new configuration should be loaded. However, let's check that you don't include any error into the configuration file by viewing the service status with systemctl:

$ systemctl status mongod.service

You’ll see the program’s output appear on the screen:

Output
● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; disabled; vendor preset: enabled)
     Active: active (running) since Tue 2021-05-25 12:02:51 CEST; 5 sec
       Docs: <https://docs.mongodb.org/manual>
   Main PID: 79931 (mongod)
     Memory: 277.5M
     CGroup: /system.slice/mongod.service
             └─79931 /usr/bin/mongod --config /etc/mongod.conf

May 25 12:02:51 alice systemd[1]: Started MongoDB Database Server.

The output shows that the service is actively running for a 5 seconds. So, the service restart was successful.


Step 3 — Viewing the Log Message Structure

Each log message of MongoDB service is in the JSON format. The message contains following fields:

  • Timestamp - Timestamp of the log message, in ISO-8601 format.
  • Severity - Severity code that indicates message priority.
  • Component - The field determines the category of the message.
  • Context - Name of the thread generating the log statement.
  • Id - Unique identifier of the log record.
  • Message - String with raw message generated by database.
  • Attributes - List of attributes that identify the client connected to the MongoDB server.

Here is an example of the log record:

{
  "t": {
    "$date": "2020-05-01T15:16:17.180+00:00" // Timestamp
  },
  "s": "I", // Severity
  "c": "NETWORK", // Component
  "ctx": "listener", // Context
  "id": 12345, // Id
  "msg": "Listening on", // Message
  "attr": {
    "address": "127.0.0.1" // Atributes
  }
}

The fields "c" (component) and "s" (severity) are enumeration types. Let's view possible values.


Severity

The severity level begins from debug information to a fatal error. You can set a threshold for minimal severity level to be logged with verbosity option in the configuration file /etc/mongod.conf. There are three severity levels that will be always logged:

  • F: Fatal error.
  • E: Error.
  • W: Warning.

Then there are 6 severity levels that will be logged only if they satisfy the set verbosity minimum:

  • I: Informational, for verbosity level 0.
  • D1  -  D5: Debug, for verbosity level > 0. Since the MongoDB version 4.2, the debug severity uses 5 levels that refer to verbosity value (for example, if the verbosity=3 then the D1, D2, D3 debug level are logged).

Components

Let's list the most common components:

  • NETWORK: Messages related to network activities, such as accepting connections.
  • QUERY: Messages related to queries, including query planner activities.
  • ROLLBACK: Messages related to rollback operations.
  • WRITE: Messages related to write operations, such as update commands.
  • RECOVERY: Messages related to storage recovery activities.
  • JOURNAL: Messages related specifically to storage journaling activities.
  • COMMAND: Messages related to database commands, such as count.

You can view full list of the components in the official MongoDB documentation.


Viewing the Log

Now, let's put it all together and view the log file. You can view the last 10 records of the log file /var/log/mongodb/mongod.log with the utility tail (sudo required):

$ sudo tail /var/log/mongodb/mongod.log

You’ll see the program’s output appear on the screen:

Output
{"t":{"$date":"2021-05-25T11:34:57.739+02:00"},"s":"F",  "c":"-",        "id":23091,   "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":28595,"file":"src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp","line":948}}
{"t":{"$date":"2021-05-25T11:34:57.739+02:00"},"s":"F",  "c":"-",        "id":23092,   "ctx":"initandlisten","msg":"\\n\\n***aborting after fassert() failure\\n\\n"}

{"t":{"$date":"2021-05-25T11:35:32.947+02:00"},"s":"I",  "c":"CONTROL",  "id":20698,   "ctx":"main","msg":"***** SERVER RESTARTED *****"}
{"t":{"$date":"2021-05-25T11:35:32.947+02:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2021-05-25T11:35:32.964+02:00"},"s":"W",  "c":"ASIO",     "id":22601,   "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"}
{"t":{"$date":"2021-05-25T11:35:32.964+02:00"},"s":"I",  "c":"NETWORK",  "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2021-05-25T11:35:32.965+02:00"},"s":"I",  "c":"STORAGE",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":76188,"port":27017,"dbPath":"/var/lib/mongodb","architecture":"64-bit","host":"moncuska"}}
{"t":{"$date":"2021-05-25T11:35:32.965+02:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"4.4.6","gitVersion":"72e66213c2c3eab37d9358d5e78ad7f5c1d0d0d7","openSSLVersion":"OpenSSL 1.1.1f  31 Mar 2020","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2004","distarch":"x86_64","target_arch":"x86_64"}}}}

The output shows a few records in JSON format we already discussed in the previous step. You can read the "msg" fields and find out that the log records reports about server restart.

Alternatively, if you set it up to store logs into Syslog you can use utility journalctl for viewing the log records.


Conclusion

In this tutorial, you installed the latest version of the MongoDB server. You learnt about all the most important configuration options related to database logging, and you configured your own configuration setup. At last, you viewed the MongoDB custom log.