How I Built a Serverless Expense Tracker Using AWS and Generative AI

Introduction

I've always struggled with tracking my expenses. This year, I resolved to improve by reviewing my bank statements each month. However, this quickly became overwhelming for two main reasons. First, like most people, I have multiple bank accounts. Second, some of my banks require me to manually request statements, which is not only inconvenient but also easy to forget.

Additionally, my transaction emails are scattered across various inboxes, and when statements finally arrive, they often come in different formats. Before I could start analyzing my spending, I was already spending too much time on data collection.

What I really wanted was a single automated system that could:

Collect all transaction alerts across my banks in one place.
Extract the actual transaction details (amount, description, date).
Store them in a structured format.
Generate a monthly spending report without requiring any manual work from me.

After researching and failing to find an existing (and free) solution that met all my requirements, I decided to build one myself. The idea I came up with was to create a pipeline that automatically forwards my transaction emails to a central location. From there, a serverless function is triggered to use Generative AI to extract transaction details, store them in a database, and then, every month, another serverless function generates a spending report. I built the system using AWS SES, Amazon S3, Gemini AI, and DynamoDB.

Architectural Overview

Step-by-Step Implementation

1. Setting Up a Domain Identity

The first step was to direct all transactional emails to a central inbox so that my Lambda function could process them efficiently. Amazon SES (Simple Email Service) was perfect for this task because it can receive emails and apply rules to determine how each message should be handled. However, before SES can send or receive emails on your behalf, it must verify your domain identity. Domain verification involves adding specific DNS records that prove ownership of your domain, while enabling DomainKeys Identified Mail (DKIM) adds an extra layer of authentication to ensure that outgoing messages are trusted and not tampered with.

Since I already had a registered domain, I decided to verify it directly. From the AWS console, I navigated to SES → Verified Identities and created a new identity using the domain option. SES then generated a set of CNAME records that needed to be added to my domain’s DNS configuration. I logged into my domain provider and published the records exactly as provided. After a short wait for DNS propagation, SES marked the domain as verified. This verification allowed SES to handle email authentication via DKIM, enabling me to send and receive messages securely from my own domain.

The next step was to set up an MX record so Amazon SES could begin receiving incoming emails. The MX record essentially tells other mail servers that AWS is authorized to accept mail for your domain. In my case, within Namecheap’s DNS settings, I created a new MX record with the host set to ses and the value pointing to inbound-smtp.<aws-region>.amazonaws.com. Once saved, any email sent to an address under the subdomain ses.mydomain.com would be routed directly to SES, ready to be processed and passed along to my Lambda function later on.

2. Receipt Rules

With SES now able to receive emails for my domain, the next step was to configure receipt rules. A receipt rule defines how incoming mail is handled based on specific recipient conditions that you set.

I started by creating a rule set, which is simply a collection of individual receipt rules. Within that set, I added a new rule and specified a recipient condition — this could be a domain, subdomain, or specific email address. If no recipient condition is defined, SES will process any email sent to the subdomain (e.g., anything@ses.mydomain.com), which works but isn’t ideal for this use case. To narrow it down, I configured the rule to only trigger for emails sent to a specific subdomain, e.g transactions@ses.mydomain.com.

Once the condition was set, I added two actions to the rule:

Store the incoming email in an S3 bucket.
Invoke a Lambda function — the Lambda picks up the email from the bucket and processes it to extract transaction details like amount, merchant, and description.

3. Email Forwarding

With the receiving pipeline in place, I needed a way to automatically forward transaction alert emails from my bank to Amazon SES. Fortunately, Gmail makes this easy with its built-in filters and forwarding rules.

For each bank I receive alerts from, I created a Gmail filter that detects transaction emails based on keywords in the subject or sender address. I then set the filter to forward those emails to the same address defined in my SES receipt rule.

Now, every time a new transaction alert arrives in my inbox, Gmail automatically forwards it to SES, which stores it in S3 and triggers my Lambda function for processing.

4. Processing Transactions

Once Amazon SES delivers an incoming email to S3, the Lambda function is automatically triggered with an event that contains metadata about the message. The first step inside the function is to fetch the actual email file from S3 so it can be processed. Each record in the event includes a messageId, which corresponds to the S3 object key where SES stored the raw email:

for record in event.get("Records", []):
    ses_message = record.get("ses", {}).get("mail", {})
    message_id = ses_message.get("messageId")

    raw_email = s3_client.get_object(Bucket=TXN_EMAILS_BUCKET_NAME, Key=message_id)["Body"].read()

This snippet retrieves the raw email file (stored in MIME format) as a byte stream. The MIME structure contains everything — headers, sender, subject, and multiple body parts (plain text, HTML, or even attachments).

To make sense of it, the Lambda uses Python’s built-in email library, which can parse these complex message structures into something readable.

msg = BytesParser(policy=policy.SMTP).parsebytes(raw_email)
body = msg.get_body(preferencelist=("plain", "html"))

This ensures that no matter how the bank formats the email, the function picks the most human-readable part, either plain text or HTML. The extracted text is then stored in msg_body and becomes the foundation for the next step: AI-powered parsing.

To interpret and structure the transaction data, the function relies on Gemini 1.5 Flash, Google’s generative AI model. Gemini takes the raw message text and converts it into a clean JSON object based on a custom prompt that defines the schema and extraction rules.

That prompt contains a detailed instruction block like this:

Extract transaction details from the following message: {msg}

Schema Definition:
{
    "amount": "number - The monetary value of the transaction (required)",
    "transactionType": "string - Either 'credit' or 'debit' (required)",
    "paymentMethod": "string - Method of payment (e.g., 'cash', 'card', 'bank transfer', 'check') (required)",
    "date": "string - Transaction date in ISO 8601 format (YYYY-MM-DD) (required)",
    "description": "string - Brief description of the transaction (required)",
    "category": "string - Transaction category (e.g., 'food', 'transport', 'utilities') (optional)",
    "merchant": "string - Name of the merchant/recipient (optional)"
}

Instructions:
1. Extract all required fields from the message
2. Convert amounts to numerical values (e.g., "₦20.50" → 20.50)
3. Standardize dates to ISO 8601 format
4. If a required field cannot be determined from the message, use null
5. Use the most appropriate category based on the description
6. Clean and standardize merchant names when available

Examples:
Input: "Spent 25.99 at Walmart on groceries yesterday using my debit card"
Output: {
    "amount": 25.99,
    "transactionType": "debit",
    "paymentMethod": "card",
    "date": "2024-03-19",  // Assuming today is 2024-03-20
    "description": "Grocery purchase at Walmart",
    "category": "food",
    "merchant": "Walmart"
}

Input: "Received $500 bank transfer from John for rent"
Output: {
    "amount": 500.00,
    "transactionType": "credit",
    "paymentMethod": "bank transfer",
    "date": "2024-03-20",  // Assuming today's date
    "description": "Rent payment from John",
    "category": "housing",
    "merchant": "John"
}

Error Handling:
- If the message is unclear or ambiguous, provide best-effort extraction
- For missing required fields, use null instead of empty strings
- For amounts in foreign currency, convert to default currency if possible
- For dates without a year, assume the current or most recent year

Return the extracted details in valid JSON format.

When the Lambda runs, it loads this prompt file, replaces {msg} with the actual email content, and sends the combined text to Gemini for processing:

prompt = prompt.replace("{msg}", str(msg_body))
response = model.generate_content(prompt)

Gemini responds with a JSON block that represents the extracted transaction. To safely isolate that data from the rest of the response text, the function looks for JSON wrapped in triple backticks:

json_match = re.search(r"```json\s*([\s\S]+?)\s*```", response.text, re.MULTILINE)

Once a match is found, the JSON is cleaned, parsed, and converted into a Python dictionary.

At the end of this step, the pipeline transforms each raw email message into clean, structured, and queryable data. For example, a transaction alert is parsed into a standardized format like:

{
  "amount": 12500.00,
  "transactionType": "debit",
  "paymentMethod": "card",
  "date": "2025-10-05",
  "description": "Purchase at Ebeano Supermarket",
  "category": "groceries",
  "merchant": "Ebeano Supermarket"
}

The final step is to persist this structured data to DynamoDB. After parsing, the function connects to DynamoDB via the boto3 interface:

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(TXN_TABLE_NAME)
table.put_item(Item={"message_id": message_id, **data})

The message_id from SES serves as a unique identifier to prevent duplicate inserts if SES ever retries delivery. This ensures idempotency in the pipeline.

5. Generating Monthly Reports

Now that the pipeline is complete, I created another Lambda function that automatically generates a monthly spending report in PDF format. This report summarizes the previous month's activity, including total income, total expenses, net income, and top spending categories and merchants. It also visualizes spending patterns with pie and bar charts, then uploads the finished PDF to an S3 bucket for long-term storage.

5.1 Overview

This Lambda function is automatically triggered by an Amazon EventBridge Scheduler (formerly known as CloudWatch Events). The EventBridge rule uses a cron expression to call the function at the start of each month. For example:

cron(0 6 1 * ? *)

This means:

0 6 → run at 06:00 UTC,
1 → on the 1st day of every month
* ? * → regardless of day of week or year.

5.2 Fetching and Filtering Transactions

The function starts by connecting to DynamoDB and retrieving all stored transactions:

dynamodb = boto3.resource("dynamodb", region_name=REGION)
table = dynamodb.Table(TXN_TABLE_NAME)
response = table.scan()
items = response["Items"]

If DynamoDB pagination is enabled, the function continues scanning until all items are collected.

Next, it determines the previous month based on the current date:

current_date = datetime.now()
if current_date.month == 1:
    previous_month = f"{current_date.year - 1}-12"
else:
    previous_month = f"{current_date.year}-{current_date.month - 1:02d}"

5.3 Aggregating Data

Once the transactions are filtered, the Lambda aggregates key financial metrics and spending breakdowns:

total_income = 0.0
total_expenses = 0.0
categories = defaultdict(float)
merchants = defaultdict(float)

for txn in previous_month_transactions:
    amount = parse_amount(txn["amount"])
    if txn["transactionType"].lower() == "credit":
        total_income += amount
    else:
        total_expenses += abs(amount)
    categories[txn["category"]] += abs(amount)
    merchants[txn["merchant"]] += abs(amount)

It then builds a summary dictionary that captures both numerical and categorical insights:

report = {
    "month": previous_month,
    "summary": {
        "total_transactions": len(previous_month_transactions),
        "total_income": total_income,
        "total_expenses": total_expenses,
        "net_income": total_income - total_expenses,
    },
    "top_categories": sorted(categories.items(), key=lambda x: x[1], reverse=True)[:10],
    "top_merchants": sorted(merchants.items(), key=lambda x: x[1], reverse=True)[:10],
    "transactions": previous_month_transactions,
}

5.4 Creating Charts and the PDF Report

The Lambda function uses the ReportLab library to create a well-formatted PDF that includes both data tables and visual charts. It generates two charts: a pie chart that shows spending by category and a bar chart that highlights the top merchants.

categories_dict = dict(report["top_categories"][:8])
pie_chart = create_pie_chart(categories_dict, "Spending by Category")

merchants_dict = dict(report["top_merchants"][:8])
bar_chart = create_bar_chart(merchants_dict, "Spending by Merchant")

Finally, the PDF is generated and written to the Lambda’s temporary directory:

output_path = Path("/tmp/reports")
pdf_file = output_path / f"transaction_report_{previous_month}.pdf"
generate_monthly_pdf_report(previous_month, report, report["top_categories"], report["top_merchants"], pdf_file)

5.5 Uploading the Report to S3

After generating the report, it’s uploaded to an S3 bucket for long-term storage:

s3_client.upload_file(
    str(pdf_file),
    s3_bucket,
    f"monthly_reports/{previous_month}/transaction_report_{previous_month}.pdf"
)

5.6 Lambda Entry Point

The function entry point ties everything together:

def lambda_handler(event, context):
    dynamodb = boto3.resource("dynamodb", region_name=REGION)
    table = dynamodb.Table(TXN_TABLE_NAME)
    reports_s3_bucket = os.environ.get("REPORTS_S3_BUCKET")
    output_dir = "/tmp/reports"

    generate_monthly_report(
        table,
        output_dir,
        s3_bucket=reports_s3_bucket,
        s3_prefix=""
    )

    return {"statusCode": 200, "body": "Monthly report generated successfully"}

At the end of this process, the application produces a clean, structured PDF summary of transactions, categorized and formatted for easy review. Below is an example of the generated PDF report (not real data):

Conclusion

Building this project was a fun and fulfilling experience. What began as a simple idea to automate how I track my expenses gradually evolved into a complete, serverless ecosystem powered by AWS and Generative AI. I’ve been using the system for a few months now, and while it’s occasionally unsettling to face the raw numbers, it has also made me more intentional and responsible with my finances.

There's still room for improvement, especially in refining how transactions are categorized. For example, POS transactions often lack detailed descriptions, making them harder to classify accurately. One idea I'm considering is adding a Telegram bot that asks me to provide short descriptions right after these transactions happen. This small human-in-the-loop step would enrich the data and make the insights even more reliable.

If you’d like to explore the code or try building something similar yourself, the lambda implementation is available here,

Command Palette

Comments