Chapter 3.5☕ 20 min read

Aggregation Pipeline — Advanced Queries

The aggregation pipeline transforms and analyzes data through a series of stages. Like a Swiggy factory assembly line for your data.

01What is Aggregation Pipeline? — Data Analysis Ka Power Tool

The aggregation pipeline is MongoDB's most powerful feature for data analysis. Think of it like a factory assembly line for your data:

Raw data goes in one end → passes through a series of stages (each transforms the data) → comes out the other end as analyzed results.

Each stage in the pipeline:

Takes the output from the previous stage
Performs a specific operation (filter, group, sort, etc.)
Passes the result to the next stage

Swiggy Example — Understanding the Pipeline Concept:

Imagine Swiggy wants to know: "Which biryani category earned the most money this month?"

The pipeline would be:

$match — Only orders from this month (filter out irrelevant data)
$group — Group by biryani category (Chicken, Mutton, Veg) and sum the totals
$sort — Sort by revenue descending (highest earner first)
$limit — Show top 3 categories only
$project — Format the output nicely

In Mongoose, you use Model.aggregate([stage1, stage2, stage3, ...]) to run the pipeline.

// Aggregation pipeline syntax
const result = await Order.aggregate([
  { $match: { status: 'delivered' } },       // Stage 1: Filter
  { $group: { _id: '$category', total: { $sum: '$total' } } },  // Stage 2: Group
  { $sort: { total: -1 } },                    // Stage 3: Sort
  { $limit: 5 }                                // Stage 4: Limit
]);

Warning! Aggregation pipeline results are PLAIN JavaScript objects — not Mongoose documents. You can't use methods like .save() on them. They're read-only analysis results.

02$match — Pehla Stage, Documents Filter Karna

$match is the filter stage. It's like the security guard at Swiggy's warehouse who only lets relevant packages through. Always put $match FIRST in your pipeline to reduce the data flowing through subsequent stages.

// $match — Filter documents (always first for performance!)
const result = await Order.aggregate([
  {
    $match: {
      status: 'delivered',           // Only delivered orders
      createdAt: {
        $gte: new Date('2025-01-01'), // From January 2025
        $lte: new Date('2025-01-31')  // To January 2025
      },
      total: { $gte: 100 }            // Only orders above Rs.100
    }
  }
]);

// Equivalent Mongoose query:
// await Order.find({ status: 'delivered', total: { $gte: 100 } })

$match uses standard MongoDB query operators:

Operator	Meaning	Example
`$eq`	Equals	{ status: { $eq: 'delivered' } }
`$ne`	Not equal	{ status: { $ne: 'cancelled' } }
`$gt`	Greater than	{ total: { $gt: 500 } }
`$gte`	Greater than or equal	{ total: { $gte: 100 } }
`$lt`	Less than	{ total: { $lt: 200 } }
`$in`	In array	{ status: { $in: ['delivered', 'confirmed'] } }
`$regex`	Pattern match	{ name: { $regex: 'chicken', $options: 'i' } }

Performance tip: Put $match as the FIRST stage whenever possible. MongoDB can use indexes to speed up $match, but only if it's first. Filtering early means later stages process less data.

03$group — Data Ko Groups Mein Organize Karna

$group is the most powerful aggregation stage. It groups documents by a specified field and performs calculations on each group. Think of it like grouping Swiggy orders by restaurant, then calculating each restaurant's total sales.

// $group — Group documents and calculate aggregates
const result = await Order.aggregate([
  { $match: { status: 'delivered' } },
  {
    $group: {
      _id: '$category',           // Group by category field
      totalOrders: { $sum: 1 },    // Count documents in each group
      totalRevenue: { $sum: '$total' },  // Sum of total field
      averageOrderValue: { $avg: '$total' },   // Average
      minOrder: { $min: '$total' },          // Minimum
      maxOrder: { $max: '$total' },          // Maximum
      orderIds: { $push: '$_id' }           // Collect IDs into an array
    }
  }
]);

// Result:
// [
//   { _id: 'Chicken', totalOrders: 150, totalRevenue: 37500, ... },
//   { _id: 'Mutton',  totalOrders: 80,  totalRevenue: 28000, ... },
//   { _id: 'Veg',     totalOrders: 45,  totalRevenue: 9000,  ... }
// ]

The _id in $group specifies what field to group by. Use $fieldName to reference a field from the current document.

Common Accumulators in $group:

Accumulator	Description	Example
`$sum`	Total of field values	{ totalRevenue: { $sum: '$total' } }
`$avg`	Average of field values	{ avgPrice: { $avg: '$price' } }
`$min`	Minimum value	{ cheapest: { $min: '$price' } }
`$max`	Maximum value	{ costliest: { $max: '$price' } }
`$count`	Count documents (newer syntax)	{ $count: 'total' }
`$push`	Collect values into array	{ names: { $push: '$name' } }
`$first`	First value in group	{ firstOrder: { $first: '$_id' } }

04$sort, $project aur $limit — Data Ko Perfect Shape Dena

After $match and $group, you'll often need to sort, shape, and limit your results. Here are the key stages for finishing your pipeline.

$sort — Order the results

// $sort — 1 for ascending, -1 for descending
const result = await Order.aggregate([
  { $match: { status: 'delivered' } },
  { $group: { _id: '$restaurant', revenue: { $sum: '$total' } } },
  { $sort: { revenue: -1 } }  // Highest revenue first
]);

$limit — Take only top N results

// $limit — Always use AFTER $sort to get meaningful top results
const result = await Order.aggregate([
  { $match: { status: 'delivered' } },
  { $group: { _id: '$restaurant', revenue: { $sum: '$total' } } },
  { $sort: { revenue: -1 } },
  { $limit: 5 }  // Top 5 restaurants only
]);

$project — Shape the output (rename, include, exclude fields)

// $project — Like SELECT in SQL, shapes the final output
const result = await Order.aggregate([
  { $match: { status: 'delivered' } },
  { $group: {
    _id: '$restaurant',
    revenue: { $sum: '$total' },
    orders: { $sum: 1 },
    avgOrderValue: { $avg: '$total' }
  }},
  { $sort: { revenue: -1 } },
  {
    $project: {
      _id: 0,                    // Hide _id
      restaurant: '$_id',        // Rename _id to restaurant
      revenue: 1,                // Include revenue (1 = include)
      orders: 1,                 // Include orders
      avgOrderValue: {           // Round to 2 decimal places
        $round: ['$avgOrderValue', 2]
      },
      revenueCategory: {         // Add computed field
        $cond: {
          if: { $gte: ['$revenue', 10000] },
          then: 'High',
          else: 'Low'
        }
      }
    }
  }
]);

Pipeline order matters! $sort then $limit gives you the TOP results. $limit then $sort gives you the wrong results. Always: $match → $group → $sort → $limit → $project.

05Complete Pipeline — Swiggy Sales Report Example

Let's build a complete real-world example: Swiggy's Monthly Sales Report. This pipeline combines everything we've learned.

// controllers/report.controller.js — Swiggy Sales Report
import Order from '../models/Order.js';

// GET /api/reports/monthly-sales?year=2025&month=1
export const getMonthlySalesReport = async (req, res) => {
  try {
    const year = parseInt(req.query.year) || new Date().getFullYear();
    const month = parseInt(req.query.month) || (new Date().getMonth() + 1);

    const startDate = new Date(year, month - 1, 1);
    const endDate = new Date(year, month, 0, 23, 59, 59);

    const report = await Order.aggregate([
      // Stage 1: Filter to this month's delivered orders
      {
        $match: {
          status: 'delivered',
          createdAt: { $gte: startDate, $lte: endDate }
        }
      },

      // Stage 2: Unwind items array (each item becomes its own document)
      { $unwind: '$items' },

      // Stage 3: Lookup biryani details
      {
        $lookup: {
          from: 'biryanis',
          localField: 'items.biryani',
          foreignField: '_id',
          as: 'biryaniData'
        }
      },
      { $unwind: '$biryaniData' },

      // Stage 4: Group by category
      {
        $group: {
          _id: '$biryaniData.category',
          totalSold: { $sum: '$items.qty' },
          revenue: { $sum: { $multiply: ['$biryaniData.price', '$items.qty'] } },
          ordersCount: { $sum: 1 },
          avgPrice: { $avg: '$biryaniData.price' }
        }
      },

      // Stage 5: Sort by revenue descending
      { $sort: { revenue: -1 } },

      // Stage 6: Shape the output
      {
        $project: {
          _id: 0,
          category: '$_id',
          totalSold: 1,
          revenue: { $round: ['$revenue', 2] },
          ordersCount: 1,
          avgPrice: { $round: ['$avgPrice', 2] },
          percentageOfTotal: {
            $round: [{
              $multiply: [
                { $divide: ['$revenue', { $sum: '$revenue' }] },
                100
              ]
            }, 1]
          }
        }
      }
    ]);

    // Get overall summary
    const summary = await Order.aggregate([
      { $match: { status: 'delivered', createdAt: { $gte: startDate, $lte: endDate } } },
      {
        $group: {
          _id: null,
          totalRevenue: { $sum: '$total' },
          totalOrders: { $sum: 1 },
          avgOrderValue: { $avg: '$total' }
        }
      },
      {
        $project: {
          _id: 0,
          totalRevenue: { $round: ['$totalRevenue', 2] },
          totalOrders: 1,
          avgOrderValue: { $round: ['$avgOrderValue', 2] }
        }
      }
    ]);

    res.json({
      success: true,
      data: {
        period: { year, month },
        summary: summary[0] || { totalRevenue: 0, totalOrders: 0, avgOrderValue: 0 },
        categoryBreakdown: report,
        generatedAt: new Date().toISOString()
      }
    });
  } catch (error) {
    res.status(500).json({ success: false, message: error.message });
  }
};

What this pipeline does:

$match — Filters to only delivered orders in the specified month
$unwind — Splits each order's items array (one doc per item)
$lookup — Joins with the biryanis collection to get category and price
$group — Groups by category, calculates total sold, revenue, avg price
$sort — Orders by revenue descending (best performers first)
$project — Cleans up the output, adds computed fields (percentage of total)

This single pipeline replaces HUNDREDS of lines of manual JavaScript code that would loop through arrays, make separate queries, and calculate totals manually. That's the power of the aggregation pipeline!

Key Takeaways

✅ Aggregation pipeline = assembly line for data analysis — stages transform data step by step
✅ $match — Filter documents (ALWAYS put first for performance)
✅ $group — Group by field and calculate aggregates ($sum, $avg, $min, $max, $push)
✅ $sort — Order results (1 = ascending, -1 = descending)
✅ $limit — Take only top N results (use AFTER $sort)
✅ $project — Shape output: rename fields, compute new fields, include/exclude
✅ Correct stage order: $match → $group → $sort → $limit → $project
✅ Pipeline results are plain JS objects (not Mongoose docs) — no .save() on them
✅ The aggregation pipeline replaces hundreds of lines of manual JavaScript data processing

Previous3.4 — Database Relationships — ref aur populate()📋 All Chapters Next Chapter4.1 — Authentication — JWT Setup →