Category: case study

The Feature

To apply for cyber insurance, a broker has to answer many technical questions. I helped unify and streamline questionnaire from different providers and designed an AI assistant to help users answer tough questions:

Core Problem

Brokers want quotes from multiple providers. More providers means more questions upfront. More questions means it’s harder to persuade seekers to consider insurance. And it’s wasted effort for brokers if clients never purchase. Making this worse is lack of unified approach among the different providers, leading to repetitive and inconsistent questions:

Problem Timeline

I chipped away at this problem across multiple projects over a long time, both iterating and inventing new approaches:

Where Questionnaire Fits Into the Workflow

There are multiple features tackling this challenge and several alternative workflows aimed at different kinds of brokers. I’ve highlighted the core workflow and where Filling the Underwriting Questionnaire sits:

Customer Research

In my continuous discovery with customers, Sales, and Support, I saw how less sophisticated brokers struggled with the technical questions asked by insurance providers.

First Task: Unify and Streamline Questionnaires

There were many repetitive and inconsistent questions among providers. Driving a unified approach, I made myself an expert on the questions. Here is a sample question about backups:

Notice how terminology, specificity, and structure can vary between providers.

Finally, I negotiated and secured alignment from all our provider teams and internally, based on a rock-solid argument:

1. Lack of consistency in questionnaire is hurting UX (I shared broker feedback)
2. Provider will lose business to peers
3. Changes I propose are safe and responsible
4. Changes are consistent with peers

I collaborated with folks across the organization as well as our customers and vendors:

But It’s Still A Lot of Tough Questions!

I learned a lot about cyber risk in my research with brokers. Many teams, however, didn’t have the luxury of a cyber expert they could reach out to. They were still intimidated by cyber questions.

If we could educate brokers seamlessly with inline assistance, they could:

• be more confident in selling
• make better assumptions
• streamline information gathering from clients

UX and Strategic Considerations

I worked with the Head of Product to decide how we can quickly ship some kind of AI assistant that would be useful. Besides chipping away at a known problem, this project was an opportunity:

⚡ Key Risk: Mitigating legal risk (e.g., AI giving bad advice) was key, which is why I decided on a fixed set of prompts to address top broker questions.

Design Choices

Measuring Success

We instrumented analytics to see how often the feature was triggered and on which questions. We needed more time (given our long sales and adoption cycles) to establish a direct impact on the bottom line. There were larger factors in play e.g., strategically we soon shifted focus to bypassing questions instead (which meant fewer “doer” users would need to consult the AI Assistant, at least initially). So there were rapid iterations on other fronts chipping away at similar problems.

We collected qualitative feedback from the Support team doing onboarding, which suggested that having an AI assistant could reduce adoption anxiety (“What if I offer cyber and a client asks me something I don’t know?”). Also, the marketing efforts around AI generated some sales leads, which is an important thing for an early-stage startup. And the AI development experience within the team yielded more opportunities later.

The Feature

I shipped an AI Bot that consumed PDF forms over email and replied with quotes. I configured an LLM backend to parse the documents, automatically fill an insurance submission, fill gaps with assumptions, and reply with quotes.

Time-to-Value: From Days to Minutes

By allowing brokers to simply forward their incoming requests to our Quote Bot instead of having to log in, we effectively created a faster way to triage and focus on profitable deals.

Customer Research

Customer insights discovered in the past were amplified by recent interviews: Brokers felt overwhelmed by the proliferation of provider portals, which pulled them out of their inboxes. Brokers live in their inbox day to day. This caused our team to rethink the boundaries of our product and of UI design.

“I hate logging into portals, because my email is my to-do list. The information you need is in documents in my inbox. I wish there was a way to avoid re-entering it.” – User

Project Context

The application experience, specifically the long and technical questionnaire is a larger problem that I worked on across multiple projects over a long time, both iterating and inventing new approaches. Bypassing that workflow initially via Email Quote Bot was the last in a chain of projects:

The Solution

I had previously concepted related ideas, like designing Outlook plugins, but there were practical obstacles e.g., customers’ restrictive IT policies, and it would be hard to prototype. These constraints didn’t apply to AI-over-email. We decided to allow brokers to get rough quotes right from their inbox by submitting existing documents:

My Teams and Responsibilities

I ran the project end-to-end:

Planning for the Pilot

I identified key traits of our ideal user:

• Brokers that value speed/volume over accuracy (ok with rough number they can revise later)
• Brokers comfortable making assumptions, whose whose clients rarely revise application to bind (e.g., Apogee)
• Brokers that receive filled applications via email (e.g., last year’s app for other carriers, broker’s form)

For one of the selected customers, I negotiated our desired workflow and target metrics:

Conflict Resolution Story – Busy Customer

I cultivated long-term relationships with customers. For this pilot, I got buy-in from 6 ideal users based on my deep knowledge of their workflow. One of them I previously visited at their office. As a result, securing their participation in a 2-week pilot was relatively easy. I facilitated a time saving estimation exercise with each party.

Another company was in a busy season. When I reached out with a follow-up request to one of the end users, the manager pushed back over email, saying, “We can’t spend time doing that. Perhaps this isn’t going to work for us.”

I knew we had a standing call the next day, so I simply said “No worries, we can touch on this tomorrow”. I spent the evening preparing a 3-slide proposal. The following day, I walked the customer through it. In Slide 1, I reiterated their goals and the projected time saving, which meant ability to handle more business. Then I explained how we needed some sample documents to calibrate our solution. The customer gladly agreed. The account manager and PM, who were on the call listening afterward said I handled it perfectly.

“Wizard of Oz” Prototype

The best way to get real-life feedback is to put something real in the hands of customers. Wizard of Oz is a prototyping technique where you tell customers they are using a real solution, but you fake it behind the scenes.

For two weeks, I monitored my inbox and impersonated an AI chatbot (assembling submissions and attaching quotes manually).

My research objectives included questions like:

• What’s maximum time we can take to respond so the solution is acceptable?
• Would users be comfortable sending follow-up questions to a chatbot?
• Would users even remember to use the new workflow? (old habits die hard in this industry)
• How would users react to assumptions? What information would users need to verify?
• And most importantly: would the document extraction tech be good enough to produce quotes most of the time?
• It was a chance to collect more real documents to refine our prompts for data extraction

Useful Technique: Intentionally Fail

I sometimes intentionally gave failed responses, from complete failure to parse the document to “I was able to extract this partial info here, but it’s not enough for a quote. Please take additional steps…” Knowing how customers would react to fail states is a key part of prototyping. Sometimes fail states elicit behaviors that you wouldn’t catch otherwise e.g., would users try to push back on the response, would they try to resubmit, or modify their submission?

Learnings From The Prototype

User Testing the Bot Response

I needed users to verify the details being used to get quotes, because it could come from a variety of sources: extracted from docs, online search or repair by AI, or static defaults.

I tested subtle differences in how information in the Bot response was organized to speed up scanning: layout for lowest mental load, desired granularity of response, top factors to highlight. For example, Version A draws attention to missing data filled from defaults, while B draws attention to what was strictly extracted from documents vs. all other sources:

Click here to expand more details…

I surveyed the pilot participants over email. One user replied: “B is better” while another said “Example A is more practical.” So I dug in further with the latter: “What makes version A more practical?”

The user clarified: “Version A is more practical because it captures more info and leaves us just 4 additional bullets to double check… Version B captures less info and requires more time to validate those 8 bullets filled out using guesses and defaults.” This was not true as both versions show identical facts and only differ in how they’re organized. The feedback gave insights about perceived complexity. After talking to the customer more, we decided Version B would be better.

Later I tried Version H (order of priority and more granular distinctions), but someone found it too complex. I then created Version I (static order in tabular layout) thinking it’s easier, but I was wrong. Despite the predictable order, the mental load was actually higher, because a person perceived columns as multiple lists instead of one. I also saw that a granular a breakdown like in Version H was too much, so I went with a simpler original version:

Negotiating With LLM Vendor

As design leader, I regularly got involved in negotiation with various stakeholders, vendors, lawyers, underwriters, and others on broader issues impacting UX.

For this project, we were using an AI provider for extracting data from PDFs. I suggested the vendor and my team set up a shared Slack channel, where I ended up giving a lot of feedback and making feature requests. I articulated the benefits well enough that the vendor actually shipped new features based on my feedback, in time to be useful to us.

Insisting on Test Strategy for Prompts

I felt strongly that we needed a bulk testing approach to be sure our submission-to-quote ratio was 80%+. The top UX consideration was: The product does what it promises. Consistently.

To ensure a consistent success rate converting submissions into quotes, I had to get creative with designing and testing prompts. The AI vendor did not have a mass-test feature, but I wanted a systematic approach. I built a JavaScript browser automation to screen-scrape our available tools and enable bulk prompt testing. I also established a procedure for engineering to regression test prompt changes and log the results. Here is an example of my log, showing pass/fail with various prompt variants across dozens of document samples:

Product Risks I Considered

I lead team discussions on the following topics. These are things we needed to be ok with, actively mitigate, or learn about from our pilot.

Risk	Description	Mitigation
Accuracy	AI is faster but will it hallucinate or extract data accurately?	Restrict to high priority data like name, state, revenue
Choice of Providers	Brokers won’t use AI, because it doesn’t support their key carriers	Reinforce message that it’s zero effort to try; hinges on our carriers’ premiums’ being competitive over time; also on the number of carriers they get (3+ good quotes are hard to pass up even if 1 is missing)
Usability	Broker is annoyed if data is extracted incorrectly and fixing it takes even longer	Overall testing showed good results but could impact specific carriers (need to monitor long-term)
Assumptions	Broker can make better assumptions than AI	Research showed brokers are ok with a rough number and reasonable assumptions e.g., 3 employees should be enough and this can be configured for each broker

Go-to-Market Planning

As the feature was nearing completion, I kept the release on track:

• Pitched the latest customer feedback to engineering
• Updated customers on final release date and scope
• Wrote product brief for sales, support, marketing
• Ran Q&A session with sales/support
• Discussed upcoming release with marketing

…

Mental Shift: When Lo-Tech Wins

I spent years designing and refining a user-friendly, streamlined application flow, viewing email and spreadsheets as tools to replace. Over time, however, I realized there are cases where PDFs, spreadsheets, and email are the best solutions.

I started to consider that some user personas, like “producers” (salespeople), rarely log into software—they’re busy on calls and traveling, often delegating tasks. Instead of ignoring them or forcing unwanted solutions, I focused on simplicity. For example, while they wouldn’t log in, they might click a link in a spreadsheet.

This project was a step toward meeting customers where they are and thinking more broadly about where our product begins and ends.

User Testing a New Proposal Type

On a call with a customer team, I noticed one participant having a hard time mapping the new workflow to how they are used to working. We were using the same words but meant different things (more on the research here). The detailed proposal we generated for all deals was reinforcing the wrong mental model.

My solution was a “1 Pager Proposal” that was optimal for deals quoted through Quote Bot:

Takeaway

One thing I learned through years of product work is that every decision we make that solves a problem opens a new challenge, which constantly requires either making successive trade-offs or opens up a new opportunity. I call it a chain of consequences:

More on this here: Emergent Chains of Consequences (LinkedIn)

Thank you for visiting.

When natural disasters, power outages, and other major events cause excessive losses, insurers can’t pay out all their customers. Reinsurance is when insurance companies resell a part of their risk to other insurance companies.

An insurance underwriter goes to a reinsurance broker who would go into Relay to find other insurance companies to help. I built an application-quote-compare workflow. And the Risk Tower shows exactly how insurers will share the risk.

In the example above, I wrote a $50M policy, but I can only cover some of it (beige portions). So, I put out a call for quotes on the 3 dark layers. I’ve found a taker for 50% of the topmost layer (blue).

Product Overview

This case study focuses on the Quote Comparison screen, specifically the Risk Tower visualization. However, this 1-minute overview of the whole product I shipped may help to convey the bigger picture.

Where Risk Tower Fits Into Workflow

As I researched examples and interviewed experts, I saw that sophisticated brokers manually create visual aids for 3 reasons:

1. Summarize request for providers
2. Track progress/gaps
3. Present to clients

Process for the Visual Design

What Was Most Difficult

Domain Expertise

No one on the product team knew about reinsurance, but I quickly became a domain expert through research and customer conversations. Customers often shared contradictory information, used inconsistent language, and described complex, varying workflows. To address this, we hired former brokers, and I brought in external SMEs to prepare before engaging with customers.

Ambiguity and Scope

I was inundated with information, making it hard to see which risks are worth taking, which problems are worth solving, and which customer stories represent wider market opportunities rather than rabbit holes. To get unstuck, I used techniques like visual mapping, breaking down issues, making assumptions, and seeking input from others.

Big Picture Challenges

Brokers spent significant time assembling complex risk towers. But what qualifies as complex? Is complex really our target segment? Will what I’m designing scale?

Multiple Participants, Multiple Views

Designing multi-participant experiences meant always considering how a change on one side affected other parties. For example:

• Broker submitting an application users tower to request and view quotes
• Insurance Provider viewing the application uses tower to see where their quotes lie with respect to what is requested

Here is what it looked like for a broker to enter a risk tower and view quotes visually:

On the insurance provider side, things were similar but with important differences. For example, we had to be careful not to reveal information the broker needed to keep confidential for any given provider.

There were also multiple internal personas with different interests. For example, managers needed to a way to restrict how big of a tower you can put together:

Sample UI Challenge: Overlapping Layers

Quotes could overlap, hiding other quotes underneath. Over time, I introduced several features to help the users disambiguate layers and make it easier to assemble the Tower.

Version 1 simply ignored this potential problem while I continue with research. Later I added a disclaimer:

I later added an intro animation for visual interest and to highlight how layers stack:

Later, what I decided was to add interactions to tackle overlapping more directly:

• ability to hide/pin quotes to the graph
• on-hover dotted-line preview of where the quote is next to other quotes
• mini-preview on each quote card, showing that quote in isolation

Sorry for the quality but this screengrab from a public demo:

I carried this “mini-preview” component into this dashboard header for example:

Here is an exploratory concept for overlapping layers and making the tower more interactive. This one came from a conversation with a customer to resolve “oversubscribed” horizontal layers. Its advantage is that is shows the number of participants (which overlapping hides). It might have worked if most carriers quoted within the requested layer but it didn’t work more generally:

In a later iteration of this concept I explored the ability to show number of layers and drill into and sign down the oversubscribed layers:

Aiding Analysis

When speaking to one customer, I saw that they were using Excel to keep track of which quotes had conditions attached. One provider might take a layer as-is, while another might take it with strings attached. To help customers make an apples to apples comparison, I designed an interaction that highlighted the presence of conditions:

As a result, the user could prevent situations where a claim happens, and it’s not covered, because an exclusion was overlooked. That could be worth millions.

Growth Tactics: Interactive Widget

To give organic leads a taste of how our app visually structures risk, I designed a simplified widget for our home page. The visitor could add some layers and then was guided to sign up and continue their submission:

Other growth tactics included an onboarding flow with tips, such as this one here explaining how to Accept, Counter, or Decline an offer (screengrab from a demo video):

I also designed a role play tutorial which welcomed new users:

Measuring Success

We knew our feature saved time for brokers who put these together by hand, but this effort varied widely among brokers. We also knew from customer feedback that the Tower added a valuable “delight” factor.

To evaluate the overall submission workflow including the Tower visualization, I continuously interviewed customers and helped them quantify their time saving from getting, comparing, and choosing quotes:

In this 1 minute testimonial, a customer says they could process submissions 50% faster:

Future Iterations

I wanted to move layer labels into the tower itself. So I made the tower larger to fit them. I also explored the ability to add free annotations to any part of the tower:

I also spent time concepting replacing quote cards with a table comparison for more complex deals:

I also concepted ways to assign layers-to-providers if it should come to that (many-to-many):

Closing

Reinsurance was a complex enterprise product I worked on for 2-3 years. I played a key role is shaping vague and conflicting information into actionable concepts we could test and ship.