An inbox nobody owned and no support team coming. Left behind a system that answered each customer for their own situation.
- Walked into
- An unstaffed support inbox, and nothing flowing from customer problems back to the product.
- Built
- An AI support system that answered each person for their employer, region, and programs, with a human on anything involving money.
- Kept running
- The system, and the process that turned support conversations into product and engineering decisions.
I joined an early-stage startup as its first business hire after the raise. The company sold to employers and served their employees, so every customer relationship involved two layers of people with different questions. The brief was broad: bring structure to how the company ran.
The inbox nobody owned
Support was the most urgent part. Nobody was staffing the inbox. Customers couldn’t get issues resolved, and nothing they ran into made it back to the people building the product. This early, the answer wasn’t going to be a support team. It had to be a system that one person could build and a small company could run.
Doing it by hand first
The first thing I did was answer the tickets myself. Automating before you understand the questions just means answering them wrong faster.
What I learned was that most questions didn’t have a single right answer. The correct response depended on who was asking: which employer they worked for, which programs that employer offered, what rules it had set, and where the person lived. Two people asking the same question could need different answers. That was the real challenge, and it’s the part a generic support tool would have gotten wrong.
I used the ticket data to build an FAQ and knowledge base around the questions people actually asked. But a knowledge base on its own would give everyone the same answer.
Teaching it who’s asking
So the next layer was data plumbing. I fed the AI the attributes of the person it was talking to: their employer, their region, the programs they had access to, and their company’s rules. On top of that, I built rules telling it how to apply the knowledge base for each group of customers.
It first clicked in testing. We asked questions as specific users, such as someone in a particular region with a particular set of programs, and the answers were specific to that person and correct. It clicked again in production, when an employer admin had a long, multi-question conversation with it and every answer was right.
Spending the human where it counts
From day one, the system could have shown users their financial information. We held back. Money is where a wrong answer does the most damage, and we hadn’t yet proven the responses were accurate enough to trust. Anything involving financial data kept a human in the loop, and we let the system handle more on its own only as it proved accurate.
The other place human attention mattered was listening. When a person reads every ticket, the company hears what customers are struggling with. Automate carelessly and that goes quiet. So we built a process to keep reviewing what came through and turn it into input for product and engineering decisions.
The system and that process were running before I left, and they kept running after.
What travels
A startup can’t hire its way out of a support problem, and it shouldn’t try to automate its way out blindly. The approach that works under tight resources is to do the work by hand until you understand the details, build the automation around those details, and spend your scarce human time on the two things a system can’t be trusted with yet: the expensive mistakes, and listening to what customers are telling you.
Then widen what the system handles as it earns trust. Building in stages is what makes a small team’s effort count.