Barceloneta Premium: we gave the model tools and it did not use them

We built it the way almost everything sold as an agent gets built today. A model with a set of tools at its disposal, each one able to look something up or record something, plus the instruction to use them when needed.

And what can go wrong there? That a lot of the time it did not use them. The tool was not failing, the connection was not failing and there was no error to look at. The model simply decided it could answer without calling the tool.

That came out in two ways and neither is good. Either it made the answer up, because a model missing a fact fills the hole with something that sounds reasonable. Or it got stuck, telling the person it could not go any further when it could.

What stands out about this case is that on paper everything was right. The tools existed, the instruction to use them was written down and it worked in testing. An instruction to a model is a request, not a guarantee. Here the request was nothing less than remembering to look before speaking.

What we did was take away from the model the decision about where the conversation stands. That decision moved to a separate mechanism, written in code, that keeps track of where each chat has got to.

It runs on two things. Which pieces of information have already been collected, because knowing the budget and not the paperwork is not the same as the other way round. And which stage of questions the conversation is in, because there is an order and the questions do not all get asked at once.

With those two, the mechanism always knows the exact situation. And that is where the model comes back in, doing what it is good at: reading what the person just wrote and deciding which tool fits that particular state.

The split is the usual one and that is why it works. The code keeps count and the model interprets. Neither one does the other’s job, so there is no longer any point where remembering something depends on a model’s goodwill.

The agent does not fill in a form and send it. It asks the way somebody from the team would ask, gathering over the course of the conversation what the agency needs to know: solvency, whether there are pets, whether anyone smokes in the property and a few other things the agency defines.

With that, the request gets classified as suitable or not. The criteria are set in advance, so the classification is always the same for the same answers and does not depend on the words somebody happened to use.

And where do those criteria live? The answer is the part that usually surprises people. In a spreadsheet table, in the agency’s own Drive, which they open and edit whenever they like. If they change their mind about a requirement tomorrow, they change it there.

Nobody has to tell us, nobody has to wait for a release on our side and there is no place in the system where that criterion is written down twice. Whoever sets the rules is whoever answers for them. That is why they live where that person can reach them.

The agent closes nothing. Once it has gathered the information, it sends the team a summary with the classification and a paragraph explaining why. Somebody at the agency reads it and decides whether to book the viewing.

That separation is not decoration. An automatic classification about people is a recommendation. The decision about who gets to rent a home belongs to whoever answers for it.

And there is something the system does not do, on purpose. If the interested person stops replying, the conversation stays exactly where it was. It does not push, it does not remind them on the third day and it does not write again a week later.

That was decided because chasing somebody who has gone cold does not pay off for the agency. It is one less feature to build, one less to maintain and one less to explain when somebody asks why a robot is writing to them.

The summary arrives by email, which at first glance looks like the lazy option when the agency has a CRM. The reason is duller and more common than it sounds.

The agency’s CRM lets you read its data from outside, but it does not let you write to it. It has read endpoints rather than a full interface, so no external system can leave anything inside it.

You can fight that or you can accept it. We build for what is there, so the verdict goes out through the channel that does work and reaches the person who has to decide just as fast.

It is the kind of detail that shows up in no demo and decides the whole design. Before promising anyone that the result lands in their system, it is worth checking whether their system lets anything land.

The agency puts what it gets back at more than three hours a day, in handling incoming requests alone. The team stopped doing triage and moved to booking viewings, which is the part of the job that brings in revenue.

The person asking, on top of that, gets an answer straight away and at any hour, including the peaks where they used to be left waiting.

The signal that matters most to us came later. The agency is extending the agent to home sales and to internal processes, on the same foundation that already filters rentals.