AI-Powered Email Quote Bot

The Feature

I shipped an AI Bot that consumed PDF forms over email and replied with quotes. I configured an LLM backend to parse the documents, automatically fill an insurance submission, fill gaps with assumptions, and reply with quotes.

Time-to-Value: From Days to Minutes

By allowing brokers to simply forward their incoming requests to our Quote Bot instead of having to log in, we effectively created a faster way to triage and focus on profitable deals.

Customer Research

Customer insights discovered in the past were amplified by recent interviews: Brokers felt overwhelmed by the proliferation of provider portals, which pulled them out of their inboxes. Brokers live in their inbox day to day. This caused our team to rethink the boundaries of our product and of UI design.

“I hate logging into portals, because my email is my to-do list. The information you need is in documents in my inbox. I wish there was a way to avoid re-entering it.” – User

Project Context

The application experience, specifically the long and technical questionnaire is a larger problem that I worked on across multiple projects over a long time, both iterating and inventing new approaches. Bypassing that workflow initially via Email Quote Bot was the last in a chain of projects:

The Solution

I had previously concepted related ideas, like designing Outlook plugins, but there were practical obstacles e.g., customers’ restrictive IT policies, and it would be hard to prototype. These constraints didn’t apply to AI-over-email. We decided to allow brokers to get rough quotes right from their inbox by submitting existing documents:

My Teams and Responsibilities

I ran the project end-to-end:

Planning for the Pilot

I identified key traits of our ideal user:

• Brokers that value speed/volume over accuracy (ok with rough number they can revise later)
• Brokers comfortable making assumptions, whose whose clients rarely revise application to bind (e.g., Apogee)
• Brokers that receive filled applications via email (e.g., last year’s app for other carriers, broker’s form)

For one of the selected customers, I negotiated our desired workflow and target metrics:

Conflict Resolution Story – Busy Customer

I cultivated long-term relationships with customers. For this pilot, I got buy-in from 6 ideal users based on my deep knowledge of their workflow. One of them I previously visited at their office. As a result, securing their participation in a 2-week pilot was relatively easy. I facilitated a time saving estimation exercise with each party.

Another company was in a busy season. When I reached out with a follow-up request to one of the end users, the manager pushed back over email, saying, “We can’t spend time doing that. Perhaps this isn’t going to work for us.”

I knew we had a standing call the next day, so I simply said “No worries, we can touch on this tomorrow”. I spent the evening preparing a 3-slide proposal. The following day, I walked the customer through it. In Slide 1, I reiterated their goals and the projected time saving, which meant ability to handle more business. Then I explained how we needed some sample documents to calibrate our solution. The customer gladly agreed. The account manager and PM, who were on the call listening afterward said I handled it perfectly.

“Wizard of Oz” Prototype

The best way to get real-life feedback is to put something real in the hands of customers. Wizard of Oz is a prototyping technique where you tell customers they are using a real solution, but you fake it behind the scenes.

For two weeks, I monitored my inbox and impersonated an AI chatbot (assembling submissions and attaching quotes manually).

My research objectives included questions like:

• What’s maximum time we can take to respond so the solution is acceptable?
• Would users be comfortable sending follow-up questions to a chatbot?
• Would users even remember to use the new workflow? (old habits die hard in this industry)
• How would users react to assumptions? What information would users need to verify?
• And most importantly: would the document extraction tech be good enough to produce quotes most of the time?
• It was a chance to collect more real documents to refine our prompts for data extraction

Useful Technique: Intentionally Fail

I sometimes intentionally gave failed responses, from complete failure to parse the document to “I was able to extract this partial info here, but it’s not enough for a quote. Please take additional steps…” Knowing how customers would react to fail states is a key part of prototyping. Sometimes fail states elicit behaviors that you wouldn’t catch otherwise e.g., would users try to push back on the response, would they try to resubmit, or modify their submission?

Learnings From The Prototype

User Testing the Bot Response

I needed users to verify the details being used to get quotes, because it could come from a variety of sources: extracted from docs, online search or repair by AI, or static defaults.

I tested subtle differences in how information in the Bot response was organized to speed up scanning: layout for lowest mental load, desired granularity of response, top factors to highlight. For example, Version A draws attention to missing data filled from defaults, while B draws attention to what was strictly extracted from documents vs. all other sources:

Click here to expand more details…

I surveyed the pilot participants over email. One user replied: “B is better” while another said “Example A is more practical.” So I dug in further with the latter: “What makes version A more practical?”

The user clarified: “Version A is more practical because it captures more info and leaves us just 4 additional bullets to double check… Version B captures less info and requires more time to validate those 8 bullets filled out using guesses and defaults.” This was not true as both versions show identical facts and only differ in how they’re organized. The feedback gave insights about perceived complexity. After talking to the customer more, we decided Version B would be better.

Later I tried Version H (order of priority and more granular distinctions), but someone found it too complex. I then created Version I (static order in tabular layout) thinking it’s easier, but I was wrong. Despite the predictable order, the mental load was actually higher, because a person perceived columns as multiple lists instead of one. I also saw that a granular a breakdown like in Version H was too much, so I went with a simpler original version:

Negotiating With LLM Vendor

As design leader, I regularly got involved in negotiation with various stakeholders, vendors, lawyers, underwriters, and others on broader issues impacting UX.

For this project, we were using an AI provider for extracting data from PDFs. I suggested the vendor and my team set up a shared Slack channel, where I ended up giving a lot of feedback and making feature requests. I articulated the benefits well enough that the vendor actually shipped new features based on my feedback, in time to be useful to us.

Insisting on Test Strategy for Prompts

I felt strongly that we needed a bulk testing approach to be sure our submission-to-quote ratio was 80%+. The top UX consideration was: The product does what it promises. Consistently.

To ensure a consistent success rate converting submissions into quotes, I had to get creative with designing and testing prompts. The AI vendor did not have a mass-test feature, but I wanted a systematic approach. I built a JavaScript browser automation to screen-scrape our available tools and enable bulk prompt testing. I also established a procedure for engineering to regression test prompt changes and log the results. Here is an example of my log, showing pass/fail with various prompt variants across dozens of document samples:

Product Risks I Considered

I lead team discussions on the following topics. These are things we needed to be ok with, actively mitigate, or learn about from our pilot.

Risk	Description	Mitigation
Accuracy	AI is faster but will it hallucinate or extract data accurately?	Restrict to high priority data like name, state, revenue
Choice of Providers	Brokers won’t use AI, because it doesn’t support their key carriers	Reinforce message that it’s zero effort to try; hinges on our carriers’ premiums’ being competitive over time; also on the number of carriers they get (3+ good quotes are hard to pass up even if 1 is missing)
Usability	Broker is annoyed if data is extracted incorrectly and fixing it takes even longer	Overall testing showed good results but could impact specific carriers (need to monitor long-term)
Assumptions	Broker can make better assumptions than AI	Research showed brokers are ok with a rough number and reasonable assumptions e.g., 3 employees should be enough and this can be configured for each broker

Go-to-Market Planning

As the feature was nearing completion, I kept the release on track:

• Pitched the latest customer feedback to engineering
• Updated customers on final release date and scope
• Wrote product brief for sales, support, marketing
• Ran Q&A session with sales/support
• Discussed upcoming release with marketing

…

Mental Shift: When Lo-Tech Wins

I spent years designing and refining a user-friendly, streamlined application flow, viewing email and spreadsheets as tools to replace. Over time, however, I realized there are cases where PDFs, spreadsheets, and email are the best solutions.

I started to consider that some user personas, like “producers” (salespeople), rarely log into software—they’re busy on calls and traveling, often delegating tasks. Instead of ignoring them or forcing unwanted solutions, I focused on simplicity. For example, while they wouldn’t log in, they might click a link in a spreadsheet.

This project was a step toward meeting customers where they are and thinking more broadly about where our product begins and ends.

User Testing a New Proposal Type

On a call with a customer team, I noticed one participant having a hard time mapping the new workflow to how they are used to working. We were using the same words but meant different things (more on the research here). The detailed proposal we generated for all deals was reinforcing the wrong mental model.

My solution was a “1 Pager Proposal” that was optimal for deals quoted through Quote Bot:

Takeaway

One thing I learned through years of product work is that every decision we make that solves a problem opens a new challenge, which constantly requires either making successive trade-offs or opens up a new opportunity. I call it a chain of consequences:

More on this here: Emergent Chains of Consequences (LinkedIn)

Thank you for visiting.