Tax filing is genuinely unpleasant when you have an NRI (Non-Resident Indian) status and income across multiple countries. You are not just filing a single return. You are reconciling employment income, foreign investment income, figuring out what tax treaties exempt and what they do not, making sure each income source landed in the right section of the right form. The foreign tax credit calculation alone takes more thought than most of the rest of the return. It takes a few hours every year, and most of that time is not thinking. It is hunting for documents, cross-referencing slips against entries, and second-guessing whether a number belongs in the right field.
A few friends face the same situation. Some of them were paying CPAs (professional tax accountants) to do it, which felt like an expensive solution to a problem that was fundamentally about document collection and field mapping. I figured I would build something for myself first, then share it if it worked. So I started building.
The Build
The idea was straightforward. The pipeline had five stages: collect all the relevant documents, extract the field values from each one, map them to the correct sections of the tax form, verify the entries against the source documents, and file. Each stage felt tractable on its own.
Fig. 1 — The intended pipeline. Each stage felt independently solvable.
So I ran experiments. I took an employment slip as a PDF and asked Claude to extract the fields. It worked. I tried browser automation to navigate the tax filing platform and fill in a form field. It worked. I wrote a prompt to map an extracted value to the correct section of the tax form. It worked. Every single experiment I ran came back with a result I could use.
This is where things got interesting. Not in a good way.
The Problem With Experiments That Always Work
Here is what I did not notice at the time: none of those experiments cost me very much. Claude was writing the extraction logic. Claude was debugging the browser automation. Claude was reasoning about which field the number belonged in. I was reviewing outputs, adjusting prompts, moving to the next piece. The work felt real. The progress felt undeniable.
But I was measuring the wrong thing. I was measuring whether each step was possible, not whether the destination was reachable.
This is where AI-assisted building gets interesting, and not entirely in a bad way. When you build without AI, the effort of each step is yours. A PDF extractor that takes three days and keeps breaking tells you something: maybe this is harder than you thought, maybe the problem is not worth solving this way. The friction is data. With AI absorbing the implementation cost, that signal goes quiet. Experiments succeed cheaply. That speed is genuinely useful — you cover more ground, test more ideas, learn faster. The low effort is a feature, not a flaw.
The problem shows up later, at integration. When you try to connect the pieces into a pipeline, the failures are qualitatively different. They are harder to debug, more time-consuming, and more demoralizing — precisely because the individual steps felt so easy. The contrast is what gets you. Each experiment worked. Why is putting them together so hard?
I kept going because each experiment worked. That is the honest answer. There was no natural pause where the build felt hard enough to stop and ask whether I was heading somewhere worth going. The pause came eventually, but it came from a week of debugging integration failures, not from the build feeling hard in the way builds used to feel hard.
Fig. 2 — AI removes the friction signal at each step; the signal reappears at integration, harder and later
Where It Actually Broke
So what does integration failure actually look like? In this project, it came in three forms.
Browser automation is brittle. The tax filing platform is a web application that changes. Selectors break. Login flows change. A script that worked last week fails this week because a button moved. This is not an AI problem. It is a fundamental property of browser automation that has been true for a decade. AI does not fix it; it just makes it easier to write the brittle script in the first place.
PDF extraction has the same property. A clean, machine-generated employment slip extracts cleanly. A scanned document, a slightly different layout from a different employer, an investment slip in a different format: each one is a new failure mode. The experiment works on the example I tested it with. The real world has a different distribution. This is one of the deeper problems in data work generally: at any meaningful scale, inputs are never as standardised as your test set assumed. Tax documents are a particularly bad example of this: every employer, every financial institution, every country produces a slightly different format for what is nominally the same information.
Then there is authentication. To automate this properly, the system needs access to my tax authority account, my payroll platform, my brokerage. Each of those is behind a login, with MFA, with session management, with terms of service that do not contemplate an AI agent acting on your behalf. There is no clean interface for this today. The authenticated data access infrastructure that would make this tractable (where a system could securely act on your behalf across services that require login) does not exist yet. This is, I think, one of the real unsolved gaps in the AI tooling space right now.
Each of these failures is real. But they only surfaced when I tried to connect the experiments into a pipeline. Individually, every piece worked.
The Accounting I Should Have Done Earlier
After a week where every fix surfaced a new failure, I stopped and did the math I should have done at the start.
How long does it actually take me to file my taxes manually? A few hours. Document collection, cross-referencing, entry, verification. It is not pleasant, but it is finite and it happens once a year.
How long had I spent building the automation? Weeks, spread across evenings and weekends. And the automation was not finished. And even if it were finished, it would require maintenance every year as the tax platform changes, as tax rules update, as my document types shift.
The ratio was always wrong. I just could not feel it because the AI was absorbing the implementation cost, not me. The hours of work were being counted in tokens, not in my time. Tokens do not show up in your gut the way a long debugging session does.
What I Actually Used in the End
I still used Claude to file my taxes. Just not the way I planned.
The pipeline did not run end-to-end. But several pieces held up on their own.
Document collection turned out to be independently valuable from the start. Tracking every incoming slip across the year (employment slips, investment statements, foreign income documents), knowing what was expected and what had not arrived yet, was a problem worth solving on its own. It did not need the full automation to work.
Using Claude as a judge for tax calculations was the next piece that held up. Running the numbers, checking the logic against the applicable rules, catching inputs that looked off: that worked reliably even when the surrounding automation did not.
The most useful piece was the browser and human loop. Browser automation with a stored login could extract the current field values from the tax filing platform. Brittle, but workable with me in the loop catching the breaks. That gave Claude something concrete to reason against. The method that worked best was asking Claude to go gap by gap: one discrepancy at a time, reason through it, validate the logic, and flag where it needed my input before moving on. That kind of structured debate — Claude working through each potential deduction or mismatch, me pushing back with my actual situation, Claude recalculating — was genuinely useful in a way a solo manual review would not have been.
The honest picture is: the integrated pipeline failed. The individual experiments, applied with a human in the loop, produced real value. That is a different outcome than pure failure, and it took stopping the build to see it clearly.
Fig. 3 — The integrated pipeline failed; the individual pieces, used with a human in the loop, held up
The Check I Should Have Run
I published an article last week called The Validation Loop, about using a set of three questions to pressure-test any output before acting on it. The idea was that my instinct as a critical reviewer, the habit of asking "does this actually hold up?", is the thing AI amplifies most usefully. Not the generation, but the interrogation of what was generated.
The tax assistant is what happens when I forgot to apply it to the build itself.
When experiments keep succeeding, it is psychologically hard to stop and ask whether the project is working. Success feels like validation. Each working experiment feels like a confirmation that the destination is reachable. You keep going because why would you stop? The Validation Loop is designed for outcomes. But the same logic applies to the build: at each iteration, is this moving me closer to the actual outcome, or am I just generating activity?
The check that would have caught it earlier is simpler than the full loop. Before you build: what does the manual version actually cost? Not in frustration. In hours, per year, across the realistic number of people who will use this. If the answer is small, the automation needs to be very cheap to build and very reliable to maintain. If it is neither, the math will not work, no matter how good the individual experiments are.
I am still building. The next project has that question at the top of the brief.
I filed my taxes manually this year, with Claude checking the sections I was uncertain about. My friends still paid the CPA.
Since writing this: Perplexity recently launched a tax assistant with the review workflow described above as a named feature. They accept document collection as a manual step, which is the part I tried to automate.