Mayukh's Blog

Sessions and Transactions in MongoDB

Mayukh Bhowmick's photo
Mayukh Bhowmick

Table of contents

Pre-requisites:
Must know how to use mongoose to create models, run queries, writes, updates and creating a connection.

ACID

ACID transactions in MongoDB ensure data integrity and consistency for operations involving multiple documents. Here's a brief explanation:

ACID stands for:

  • Atomicity: All operations in a transaction either succeed together or fail together.

  • Consistency: The database remains in a consistent state before and after the transaction.

  • Isolation: Concurrent transactions don't interfere with each other.

  • Durability: Once a transaction is committed, it remains permanent even in case of system failures.

What does sessions and transactions do?

Sessions are a way of maintaing a consistent view of the database.

And transactions let you execute multiple operations and potentially undo all the operations if one of them fails.

Sounds too technical?

Before coming to the dictionary definition, allow me to explain you

  • why we need sessions and transactions?

  • do we need both or either of them?

  • What scenarios are best for sessions and transactions?

  • What to avoid when using transactions?

I believe the best way to understand something is through a real-case scenario:

Scenario 1: Session but no transactions

Imagine this...

Alex and James, two users visiting a smartphone on an e-commerce website.

Only one phone is left in stock.

Now the checkout process for both the users looks something like this:

  • Verify item availability

  • Calculate total price

  • Apply discounts

  • Process payment

  • Update inventory

Let’s assume the product details of phone looks something like this on mongodb

// product details of a phone
{
  "_id": ObjectId("651234567890abcdef"),
  "name": "Galaxy S24 Ultra",
  "brand": "Samsung",
  "model": "S24 Ultra",
  "stock": 1 // only one stock left for purchase
  // ... other information
}

Now, what is the work of session here...

Throughout the checkout process, Alex enters a session, and James also enters into a different session.

Now what session does is, it takes a snapshot of the current database, meaning, even if there are updates on database, outside of Alex's session, for that particular data ( say old data is old_data, after update, the data is new_data),
Alex will still read old_data.

Now let's say when Alex is between "Apply discounts" and "Process payment" steps of the checkout process and If only one phone is left, and James makes the first request to purchase it, the stock will get reduced from 1 -> 0.

But Alex will still see that 1 phone is available.

And after James' purchase, Alex can also purchase it, as it shows that one phone is still left, because of sessions, even though the stock is zero, leading to overselling of a product.

Scenario 2: Session and Transactions

This is where transaction comes in to play,

if we use transactions, then the optimistic concurrency control feature of mongodb will check if the data has been changed since Alex's session started...

Since the following document has been changed since the session has started for Alex, transaction will fail, transaction will fail, and the entire process will roll back.

If no, then transaction will be successfull and commited.

In Alex's case, transaction will faill and rollback, because data has been already changed and updated due to James' purchase, and the entire process of purchase for Alex will fail.

Note:

  • You can use sessions without transactions for operations where you want a consistent view of the database but don't need the all-or-nothing guarantee of transactions.

  • Every transaction uses a session, whether you create it explicitly or MongoDB creates it implicitly.

Where to use the session object?

Following are the methods where we should associate the session.

  1. Query Methods:

    • .find()

    • .findOne()

    • .findOneAndUpdate()

    • .findOneAndDelete()

    • .findOneAndReplace()

    • .findById()

    • .findByIdAndUpdate()

    • .findByIdAndDelete()

    • .countDocuments()

    • .distinct()

Example:

    await Model.find({}).session(session);

  1. Update Methods:

    • .updateOne()

    • .updateMany()

Example:

    await Model.updateOne({}, {}, { session });

  1. Delete Methods:

    • .deleteOne()

    • .deleteMany()

Example:

    await Model.deleteOne({}, { session });

  1. Aggregate:

    • .aggregate()
    await Model.aggregate([]).session(session);

  1. Create Methods:

    • .create()

    • .insertMany()

    await Model.create([{ name: 'John' }], { session });

  1. Document Operations:

    • .save()

    • .remove()

    const doc = new Model({ name: 'John' });
    await doc.save({ session });

  1. Populate: When using .populate(), the session is automatically passed to the populated query if the original query used a session.

  2. Subdocuments: Operations on subdocuments will automatically use the session of the parent document.

How to use sessions and transactons using mongoose?

For any function that executes DB operations, here's how to use sessions and transactions

const mongoose = require('mongoose');

async function transfer(fromId, toId, amount) {
  const session = await mongoose.startSession();
  session.startTransaction();

  try {
    const fromAccount = await Account.findById(fromId).session(session);
    if (fromAccount.balance < amount) throw new Error('Insufficient funds');

    // Simulate delay
    await new Promise(resolve => setTimeout(resolve, 1000));

    await Account.findByIdAndUpdate(fromId, { $inc: { balance: -amount } }).session(session);
    await Account.findByIdAndUpdate(toId, { $inc: { balance: amount } }).session(session);

    await session.commitTransaction();
    console.log('Transfer complete');
  } catch (error) {
    await session.abortTransaction();
    console.error('Transfer failed:', error);
  } finally {
    session.endSession();
  }
}

  1. mongoose.startSession():

    • Creates a new session and initializes a consistent snapshot of the database.

        const session = await mongoose.startSession();

    • Note: A session by itself doesn't provide atomicity; it's used to maintain a consistent view of the database across multiple operations.

  2. session.startTransaction():

    • Starts a new transaction within a session and sets the beginning of a series of operations that should be executed as a single, atomic unit.

        session.startTransaction();

    • Note: This must be called on a session object. It prepares the session to group subsequent operations.

  3. session.commitTransaction():

    • Commits (finalizes) a transaction, meaning it applies all changes made within the transaction to the database.

        await session.commitTransaction();

    • Note: If this succeeds, all operations in the transaction are permanently applied. If it fails (due to conflicts), none of the operations are applied.

  4. session.abortTransaction():

    • Aborts (cancels) a transaction, meaning it discards all changes made within the transaction.

        await session.abortTransaction();

    • Note: This is typically used in a catch block to handle errors and revert changes if something goes wrong during the transaction.

  5. session.endSession():

    • Ends a session and closes the session and releases any server resources associated with it.

        session.endSession();

    • Note: This should be called whether the transaction was committed, aborted, or even if no transaction was started.

A much clearer view of transaction and session in use:

const session = await mongoose.startSession();
try {
  session.startTransaction();

  // Perform database operations here
  // All operations should use the session

  await session.commitTransaction();
  console.log('Transaction successfully committed.');
} catch (error) {
  await session.abortTransaction();
  console.log('Transaction aborted:', error);
} finally {
  session.endSession();
}

Summary:

  • startSession() creates the session.

  • startTransaction() begins the transaction.

  • If all operations succeed, commitTransaction() is called to apply the changes.

  • If an error occurs, abortTransaction() is called to discard the changes.

  • endSession() is always called at the end to clean up, regardless of success or failure.

This ensures that a series of operations either all succeed together or all fail together, maintaining database consistency. It's particularly useful for operations that involve multiple documents or collections that need to be updated atomically.

When multiple operations are tyring to modify the same document simultaneously you get errors like MongoError: Write Conflict. Sessions and transactions prevent that from happening.

What to avoid when using Transactions

The use of transactions should be done carefully.

  • Avoid nested transactions: Nested transactions are not supported by MongoDB. Look at the below code carefully.

      function someWork(){
      const session = await mongoose.startSession();
      await session.startTransaction(); // first transaction

      // some code

      // not supported, will throw error
      await session.startTransaction(); // second transaction

      // some code
  }

    We have two transactions here…

    This is a nested transaction and mongodb doesn’t support it because inside a session, only one transaction can be active at a time.

    It will throw an error that says something like transaction is already in progress.

    And if you want to use multiple transactions within the same session, then you need to commit the first transaction and then start another transaction.

      function someWork(){
      const session = await mongoose.startSession();
      await session.startTransaction(); // first transaction

      // some code

      await session.commitTransaction();

      await session.startTransaction(); // second transaction

      // some code

      await session.commitTransaction();

      await session.endSession();
  }

  • Dont use too many transaction: Try to keep the total no. of transactions as minimum as possible, because sometimes managing too many transactions becomes expensive and can be a load on performance and can also lead to database inconsistency since every transaction is isolated.

  • Transaction timeout: The default time limit of a transaction is 60s, which means any no. of operations inside a transaction should be completed within 60s.

    If any function takes more than 60s, try to optimize your operation.

    Some suggestions on optimizations

    • Use promise.all() for consecutive operations independent of each other.

    • Use lean() for mongoose find methods.

    • If you have many write operations inside a for loop, consider using insertMany.

    • User bulkWrite method of any mongoose model for executing multiple operations at the same time like update, delete, insert.

Why do we need to associate sessions with DB operations?

Associating a session with database operations is crucial for maintaining consistency and is especially important when using transactions.

  1. Why associate sessions with operations:

    a) Consistency: All operations within a session see a consistent snapshot of the database at the point the session started. Sessions ensure that operations are causally consistent, meaning they're executed in the order you specify.

    b) Transaction support: For transactions to work, all operations must be part of the same session.

    c) How to associate a session:

    You associate a session with an operation by either:

    • Chaining .session(session) to the query:

        await Model.insertMany(data).session(session);

    • Passing it in the options object:

        await Model.updateOne( { session } );

  2. What happens if you don't associate the session:

    a) Outside of transactions:

    • Operations will execute against the current state of the database, not the consistent snapshot of your session.

    • You lose the consistency guarantees that sessions provide.

b) Within transactions:

  • The operations won't be part of the transaction.

  • They will execute immediately against the database, outside the transaction's control.

  • These operations won't be rolled back if the transaction is aborted.

const session = await mongoose.startSession();
session.startTransaction();

try {
  // This is part of the transaction
  const user = await User.findOne({ _id: userId }).session(session);

  // This is NOT part of the transaction
  const post = await Post.findOne({ _id: postId });

  user.postCount++;
  post.likes++;

  // This update is part of the transaction
  await user.save({ session });

  // This update is NOT part of the transaction
  await post.save();

  await session.commitTransaction();
} catch (error) {
  await session.abortTransaction();
} finally {
  session.endSession();
}

In this example:

  • The user operations are part of the transaction and will be rolled back if the transaction is aborted.

  • The post operations are not part of the transaction. They will persist even if the transaction is aborted, potentially leading to inconsistent data.

To fix this, we would need to associate the session with all operations:

const post = await Post.findOne({ _id: postId }).session(session);
// ...
await post.save({ session });

Associating sessions with your operations is crucial for maintaining consistency, especially within transactions. Failing to do so can lead to operations being executed outside the intended transactional context, potentially resulting in data inconsistencies.

Conclusion

Use of sessions and transactions helps you run operations successfully, and prevent data inconsistency at all costs. Have look through the documentation here.

Also go ahead and play with transactions to get a better understanding.

Did you find this article valuable?

Support Mayukh's Blog by becoming a sponsor. Any amount is appreciated!

MongoDBDatabasesbackend