The Story Behind FinMind: From Crumpled Receipts to an AI Financial Assistant

How a frustrating spreadsheet became my most ambitious AI-assisted software project.

Imagine we are sitting in a quiet, cozy corner of a warm coffee shop. Outside, rain is tapping gently against the windowpane, and the aroma of freshly ground espresso fills the air. You look across the table, take a slow sip of your cappuccino, and ask me:

"So... tell me about this project. What was it actually like to build it?"

I would probably take a slow breath, lean back, laugh a little, and tell you about the evening I sat at my desk surrounded by a mountain of crumpled grocery receipts, a half-empty cup of cold coffee, and an Excel spreadsheet that was actively mocking my spending habits.

That was the night FinMind was born. Not out of a desire to build a flashy portfolio piece, but out of absolute, unadulterated frustration.

---

The Spark

As I entered the final year of my engineering degree, one thing became clear—I had learned how to build software, but I still hadn't learned how to manage my own money. I wasn't studying finance; I was studying engineering, which meant I could design systems on paper but had no idea where my money disappeared to every month. Whenever my parents transferred the semester's allowance, I'd feel briefly rich. Two weeks later, my account was near zero and I genuinely couldn't explain why.

I tried every personal finance tool I could find. Most were either too tedious—requiring me to manually type in every expense—or too noisy, pulling in a wall of bank transactions and categorizing my local diner breakfast as "Utilities." None of them made me feel connected to my money. They made me feel managed by it.

So I went back to basics: a spreadsheet. Tabs for income, expenses, savings. Conditional formatting. It worked for a few weeks—until one evening, I accidentally pasted a value into the wrong column, shifting my entire spending ledger by one row. The formulas didn't break. They just calculated the wrong numbers. I spent three weeks believing I had saved ₹20,000 more than I actually had.

By the time I caught the error, my budget was ruined.

I closed the laptop and sat for a while. We have high-resolution cameras and machine learning in our pockets. Why was I still manually typing numbers into cells like a 1980s accountant?

I wasn't looking for a better database. I was looking for a frictionless bridge between physical receipts and a digital budget. More than that—I was trying to answer a question that kept bothering me: could software make me more mindful about money instead of just recording it?

So I decided to build it. I named it FinMind.

Building Something That Wasn't Enough

The First Version

I sketched the architecture on a Saturday morning. At first, I considered splitting the frontend and backend into separate projects—then realized that would mean constantly switching between two codebases for every small change. I chose a modern full-stack framework so the UI and APIs could evolve together, a document database because receipts fit naturally into nested structures, and an authentication service so I didn't spend three weeks writing login code.

A week later, I had my first working dashboard. Dark mode, glassmorphism cards, a modal that slid up when you clicked "Add Transaction." I showed it to my roommate that evening. He clicked around, nodded, said "Looks clean."

V1 Dashboard Prototype – the first working interface.

I used it for three days. On the fourth day, I stopped.

When the Mirror Showed the Truth

It was simply too much work. Buying lunch meant six clicks and a dozen keystrokes. I had built a beautiful interface around the exact problem I was trying to eliminate—tedious data entry. The design was polished. The product was broken.

I stared at the screen for a while, then opened a new file and started planning the scanner.

Sometimes the most important feedback you get is from yourself, after you actually try to use the thing you built.

The Breakthrough

The Receipt That Refused to Cooperate

I still remember taking a photo of a wrinkled supermarket receipt and watching the application fail spectacularly. Half the items disappeared. The total became nonsense. I laughed—then realized OCR was going to be much harder than I expected.

Real-world receipts are printed on cheap, glossy thermal paper. Folded in your pocket, smudged, sometimes faded. The open-source OCR tool I started with read characters like a drunk person. A bottle of Coca-Cola became C0c4-C014. A price of ₹450.00 became ~45O.o0. My code to parse this quickly spiraled into a 500-line monster of nested rules that collapsed the moment I tried a receipt from a different store.

Early OCR output – barely legible.

That was the first time I realized building AI products isn't about making the model smarter—it's about asking it better questions.

A Weekend of Self-Doubt

A friend who sat next to me in systems class finished his entire CRUD project over one weekend while I was on my third week trying to understand why a receipt scanner couldn't read a coffee receipt. I remember thinking, "Maybe this project is too ambitious. Maybe I should just build a to-do app like everyone else."

One evening I nearly deleted the whole repository. Then I thought: what if I try one more approach first?

I kept going. Not because I was confident—because I was curious.

The First Time AI Surprised Me

Instead of writing rules to parse raw text, I tried sending a crumpled photo of a diner receipt to a vision language model—an AI that can actually see an image and reason about its contents.

I clicked "Run" expecting another failure. Five seconds later, a perfectly structured response appeared in my terminal. I stared at the screen for a few seconds before realizing I hadn't written a single parsing rule. It had mapped the pancakes to 'Restaurants', the orange juice to 'Beverages', and normalized the transaction date—completely ignoring the massive crease down the middle of the paper.

I must have uploaded the same receipt ten times that evening, not because I needed to test it anymore, but because I couldn't stop smiling every time it worked.

I texted a friend at midnight: "Hey—the receipt scanner actually works." He replied: "Go to sleep." I didn't.

That evening I wasn't thinking about OCR anymore. I was thinking about possibility. For the first time, FinMind felt like it had an identity.

The OCR demo that finally worked.

That was the moment the project stopped feeling like an experiment and started feeling like a product.

When Success Became the Next Problem

Serverless platforms place strict time limits on how long a backend function can run. A raw phone photo was large enough that uploading it plus calling the vision API together would push past those limits—and the request would die with a timeout error mid-process.

After two evenings of profiling, I realized the problem wasn't my server. It was the image itself. I had been sending six megabytes of information when the AI only needed to read a few hundred characters. I built a small compression utility that ran entirely in the browser, shrinking the image before it ever touched the network. The upload suddenly felt instant.

I spent two evenings blaming my server. Turns out, the server had done nothing wrong.

The Bug Hidden in Windows

Then came the strangest bug of the entire project.

My database connection kept dying with no explanation. Everything looked correct. After hours of digging through Node.js release notes and network forums, I discovered it was a networking default specific to my Windows development environment—the kind of issue that doesn't appear in any tutorial because it sits three layers below the application. The fix was a handful of lines at the very bottom of my configuration file.

The connection succeeded instantly.

Some of the hardest bugs aren't in your code. They're hiding underneath it.

The Teammate That Never Slept

A Search Box That Finally Understood Me

With receipts flowing in, I built a standard search box. You typed "coffee," it found "coffee." But I wanted to ask something like "Did I buy anything sweet when I was traveling last month?"—and get an actual answer.

A keyword search had no concept of meaning, only of matching. I spent an evening reading about vector embeddings—essentially a way of representing language mathematically so that similar ideas land near each other, even when the words are different. Once I built this, the system retrieved a Starbucks receipt listing a "Blueberry Muffin" when I asked about something sweet—despite the word "sweet" never appearing anywhere in the original receipt.

Vector embeddings – mapping meaning mathematically.

That was the moment search stopped feeling like filtering data and started feeling like talking to someone who actually remembered my life.

The First Conversation That Felt Real

Once I could find relevant receipts by meaning, I fed those directly into a chat interface. The AI would answer questions using only my actual financial data—my own transactions as its knowledge base.

The first version was embarrassingly verbose. It wrote paragraphs of polite introduction when all I wanted was a blunt answer. I spent hours tuning it—strict rules, word limits, personality guidelines—until I had something that fired back instantly:

"🚨 Yes! You've spent ₹1,240.00 on coffee this month, which is 34% higher than your historical average. You might want to skip the next Starbucks run to stay within your ₹1,000.00 budget limit."

The chat interface – a financial mirror.

That was the assistant I'd wanted from the beginning—not a chatbot, but a financial mirror.

Trying to See Around Corners

I wanted the app to warn me before I overspent, not after. My first forecasting attempt completely failed—one rent payment convinced the algorithm I'd be bankrupt by day thirty. I experimented with several approaches before finding one that blended my current month's spending trajectory with my historical average, smoothing out one-off spikes.

The day I finished it, I ran it on my own data. The prediction was surprisingly close to my actual spending at the end of the month—close enough that I stopped thinking of it as a demo and started trusting it.

That was a strange feeling: watching software predict my own behavior better than I could.

The Architecture I Had to Compromise

My lowest point came trying to run authentication inside a part of the system that simply didn't support what I needed. My build compiler crashed every time I tried, and four consecutive nights of workarounds led only to infinite redirects and security loopholes. I felt completely out of my depth.

Ultimately, I moved authentication out of that layer and into a standard route. The perfect architecture gave way to one that actually shipped.

Engineering isn't about writing pristine, theoretical code. It's about making pragmatic tradeoffs under real constraints.

The Quiet Moment After Deployment

When I finally deployed the first stable version, I didn't immediately open the website.

I just sat there for a minute staring at the deployment success message. After weeks of chasing bugs—DNS timeouts, rendering errors, middleware crashes, OCR failures—it felt strangely quiet.

My phone showed 2:47 AM.

For the first time in weeks, the terminal was silent. That silence felt like success.

Looking Back

The Bug That Made Me Laugh

You can't build a project like this without making some hilarious mistakes.

During testing, I accidentally left a recurring test script running on a 5-minute cron job. My alerts were hooked to my email. I woke up to 240 emails warning me that my coffee habit at 'The Grind' would cost ₹1,440,000.00 a month if I continued buying coffee every five minutes.

I sat in my bed laughing hysterically at my own inbox. The math wasn't wrong.

The code did exactly what I told it to do. That's usually the problem.

What AI Actually Changed

Semantic search turned out to be a form of financial therapy. Asking "Why did I spend so much last Wednesday?" would yield: "You had a long workday, bought energy drinks, ordered pizza, and took a taxi to manage fatigue." That wasn't just a search result—it was context. It connected my spending to my life, showed me that my expenses weren't random subtractions; they were reflections of my energy, my stress, and my lifestyle.

The value of AI in personal finance isn't just automation; it's reflection.

The Tools I Used Along the Way

Like most developers today, I used AI-assisted tools at various points during the build—for generating boilerplate, exploring API syntax, and quickly scaffolding component layouts. That's not meaningfully different from searching Stack Overflow or reading documentation; it's just faster. The tools were useful, and I used them where they made sense.

But there's something important about where AI helps and where it doesn't. It could generate a Mongoose schema in seconds, but it couldn't debug the Windows DNS issue. It could scaffold a React component, but it hallucinated outdated MongoDB Vector Search syntax. When I asked it to solve the latency problem, it suggested buying faster servers. I had to figure out that the real answer was compressing the image on the client side before it ever left the browser.

Every meaningful decision in this project—the architecture, the debugging, the tradeoffs—was mine to make. The tools made me faster. They didn't make the decisions.

Knowing when to use a tool, and when to put it down and think, might be the most underrated skill in software engineering.

What Being a Good Developer Actually Means

Before FinMind, I thought being a good developer meant knowing more—more algorithms, more frameworks, more syntax. After FinMind, I think it means something different. It means understanding how systems connect, making decisions under uncertainty, and being honest about what you don't know yet. I'm still learning that lesson every time I start a new project.

If I opened this repository today, I'd probably rewrite half of it. And that's actually a good thing. It means I've grown. The code I wrote a few months ago reflects the engineer I was then. I'm hoping the code I write after graduation reflects someone even better.

Why This Project Will Always Matter

The moment I remember most clearly isn't a line of code. It's a Tuesday afternoon when a classmate visited my room and I showed him a demo. He took a photo of a crumpled receipt from his wallet—one that had been folded in half and was barely legible—and held it up to his laptop camera. Five seconds later, every item appeared on screen, categorized and timestamped. He looked at me and said, "Wait. The computer actually read that?"

I laughed. Because yes. That was the whole point. And somehow, after months of failures, it finally worked.

Later, someone close to me tried it for the first time. They photographed a grocery receipt they'd dug out of their bag, half-crumpled, and watched every item appear on screen, categorized and clean. They looked up and said, "This is the first time your coding has actually meant something to me." I didn't have a reply for that.

As I prepare to graduate, I already know this project has changed the way I'll approach every piece of software I build next. Not because of the technologies—but because of what building it taught me about failure, patience, and what software can do when it's built around a real human problem.

We don't build software to impress other machines. We build it to make our human lives just a little bit simpler, one receipt at a time.

And for me, that's what FinMind will always represent—not the most complex project I've built, but the project that taught me how I want to build software.

The desk, the receipt, the coffee, and the code – a moment of clarity.