Building an assistant that carries work across conversations

Five jobs in one conversation

The assistant does five kinds of work: tasks, news briefings, meeting summaries, email drafting and document search. The backend is Python with FastAPI, the orchestration is a LangGraph graph, durable records live in PostgreSQL, and Teams and Planner are reached through Microsoft Graph. It was a work project and I contributed to it, so there's no public repo to point at. I'll talk about decisions rather than code.

No single capability was the hard part. The hard part was that one conversation hops between them. Someone creates a task, moves the deadline, asks for a briefing, then comes back to the task. That raised two problems for me. The system has to remember work across turns without treating a chat transcript as a record. And some of what it produces leaves the system, as a message in Teams or a task in Planner, and you can't fix that by editing a chat bubble.

A router and a handful of specialists

I started from the workflow, not the message. A news agent has to produce a briefing in a requested format. A task agent has to keep one specific record and its current state. If both just return free-form text, the interface looks simple and the backend becomes impossible to trust, so each specialist got its own tools and its own idea of a correct result.

In front of them sits a router with a deliberately small job: pick the specialist and pass along the shared context. The payoff is that adding a capability means adding a specialist and a routing option, not touching the ones that already work.

The cost shows up when a request spans two specialists, like drafting an email about a task. That only works if the shared context is accurate, which is why the conversation state ended up mattering as much as the tools. I never measured how often the router picks the right specialist.

The checkpoint is not the task

A LangGraph checkpoint records where a run is so the conversation can pick up later. That's useful, but it describes the run, not the business object. If someone asks about a task from another screen or another conversation, the answer has to come from the task record in PostgreSQL, not from whatever the model said earlier.

This is the part I spent most of my time on: the lifetime of the PostgreSQL checkpoint connection, and asynchronous task persistence. Those are the seams where a good demo turns flaky. Connections need a defined lifetime, and what the user sees afterwards should be what was actually saved, not what the model said it would save.

Keeping the two layers apart also made bugs easier to place. A stuck conversation and a wrong task status are different problems with different places to look.

Tasks are records, not paragraphs

The task tools list, create, update and complete work, and status, assignee, priority and deadline are real fields. The list operation takes a status filter, and update logic parses dates before they reach the service layer.

So "push it to next Friday" becomes a record operation instead of another paragraph in the chat. It needs the task being referred to, the field being changed and a date that parses. If any of those is missing, the assistant returns a validation response instead of guessing. A guessed deadline looks exactly like a correct one in the chat, and that's what worried me.

Two smaller rules came out of that: an absent field is not the same as a field the user wants to change, and the task is reloaded from the database before it's shown, so the reply can't be stale.

Filter the news before polishing it

For briefings, the news agent has a model score each candidate article for relevance and drops the low scorers before the report is assembled. A beautifully summarized article that nobody needed is still noise, and filtering first means the later steps work on less.

I also worked on pulling report structure out of sample PDF and DOCX files. The sample supplies headings, ordering and presentation, so a briefing follows whatever format was asked for instead of one layout hard-coded in the generator. Export then writes the result as PDF or DOCX so it can be used outside the chat.

Asking before touching Teams or Planner

Teams and Planner put the assistant close to real work, so I built human-in-the-loop confirmation for those integrations. The assistant shows the proposed task or message, waits for a decision, and goes ahead with only what was approved.

Two failures pushed me there. One is acting on a misunderstanding, like creating a task with the wrong assignee; confirmation catches that before the side effect. The other is repeating an action after an interrupted request, like sending the same message twice. Persisting the proposal and the decision gives the system something to check before it acts again.

I haven't run any of this against a production tenant, so permissions, retries and what Microsoft Graph does on partial failures are still open for me.

What I'd do next

Adding another specialist turned out to be the easy part. Making the boundary between a conversation and an external side effect dependable is harder, and that's where I'd spend the next round: task completion on real requests, deadline parsing failures, whether readers find the briefings relevant, and how often an interrupted request produces a duplicate action.

Takeaways

• Design the workflow first; specialists get their own tools and their own definition of a correct result.
• A graph checkpoint, a task table and a model reply are different kinds of state, and none can stand in for another.
• Structured task fields and date parsing turn a chat request into a record operation you can inspect.
• Confirming before external actions limits what gets sent to what the user approved.