IBM is working with researchers at the University of Twente in the Netherlands to develop a system that listens in on calls received at a call center, uses speech recognition to identify topics that a caller is asking about, looks up related information, and displays it to a call center agent. Sort of like the annoying kid who sat next to you in class and was always trying to impress you by pointing out how much he knew.

The system actually does sounds interesting, but I’m pretty skeptical. First, the speech recognition system has to recognize what a caller is asking about. The prototype currently uses keyword recognition. That can work well, assuming you choose your grammar very wisely and you actually do quite a bit better than simple keyword recognition. Without putting the keywords into some context, the application is likely to behave like a loyal, but very stupid dog, running off and fetching lots of information that is tangential to what the caller is really interested in. You then run the risk of the system distracting the call center agent and leading him or her to pay more attention to the info the system fetched than to what the customer is saying.

The problem is that the system must not only understand the caller’s utterances and dig up useful, related info, but it must also present that info to the agent in a way that the agent can then effectively present back to the caller. You now have a lot of steps at which misinterpretation can occur, sort of like the old telephone game.

Another advertised feature of the system is that it can warn the agent if it does not hear a warning that the agent is supposed to give the caller, such as about penalties for not making mortgage payments.

If the system does not “hear” the keywords of that warning the operator will receive a sharp on-screen reminder before the call ends.

And what kind of a “sharp” reminder would this be, a swift poke in the ribs, a high voltage shock on the nose? Ah, I forget this is an English magazine and my use of certain adjectives doesn’t all jive with the Queen’s use.