Finley started as an internal experiment — “what if an AI assistant could manage prospecting, outreach, and follow-up on its own?” After several iterations, some surprises, and a few battle scars, here are the most important lessons we’ve learned from building and scaling AI tools in the real world.
What’s Harder Than It Looks
Prompt Engineering Is Not Plug-and-Play
Getting GPT to behave the way you want is an art. Tiny changes in how you phrase a prompt can shift the tone, break the format, or change how the model interprets the task. After much trial and error, we settled on highly structured prompts, JSON-format outputs, and eventually OpenAI function calling to ensure predictable results.
Lesson: You can’t just ask the model nicely — you have to guide it like a junior teammate with high IQ and zero memory.
Structuring Context Is More Work Than You Think
Every email Finley writes uses live context: lead info, company metadata, sender identity, and thread history. The challenge wasn’t fetching the data — it was how to package all of it into compact, consistent, and token-efficient blocks. We had to continuously tweak how we prioritize fields, collapse text, and structure inputs for reliability across use cases.
Lesson: The hard part isn’t generating content — it’s structuring input.
What’s Overrated
“Just Use AI” Isn’t a Strategy
There’s a common myth that AI will “just figure it out.” In practice, AI needs scaffolding — prompts, fallbacks, logging, validation layers, retries, and system-level reasoning. Finley doesn’t just “call GPT.” It builds multi-step context, parses structured output, validates fields, enforces rules, and logs every action in Salesforce.
Lesson: AI is the engine — but you still need to build the car.
More Models ≠ More Value
With so many options out there, it’s tempting to try every new model or API — but we’ve found that consistency trumps novelty. We now use just two OpenAI models:
gpt-4o-minifor summarization, email generation, and thread assistancegpt-4ofor reasoning-heavy tasks like lead scoring
This lets us optimize performance, cost, and reliability without juggling too many dependencies.
Lesson: Two well-integrated models beat five half-integrated ones.
Advice to Future Builders
Here are five things we’d recommend to anyone building similar AI systems:
- Log everything — even when it “works,” it might be doing something weird.
- Use queues to decouple AI steps — retries become trivial, and debugging becomes easier.
- Structure your context early — sloppy context becomes your biggest blocker later.
- Pick a single source of truth — mixing state across systems is fragile and hard to debug.
- Invest in monitoring and fallback logic — especially when your AI talks to customers.
Conclusion
Finley’s not perfect, but it runs on its own. That’s been the real achievement — not just building an AI tool, but building a system where the AI is part of the team. And the lessons we’ve learned along the way? They’re as much about product design and engineering as they are about machine learning.
If you’d like to learn more about applying these lessons in your own projects, contact us — we’d love to share insights and explore possibilities together.