Feedback on paper of Reed, Norman and Jennings KIND OF CONTRIBUTION The paper identifies a problem of finding a common communication language that will allow communication between agents that have not been designed to interoperate. This is not an entirely new problem (as the related work discussion makes clear) but it is an important but neglected one, and their approach is novel. They propose a new voting algorithm for this task, called SVM, and illustrate how it can be applied via a worked example. SVM requires the existence of a mutually agreed and understood 'semantic space' to define the voting choices. An example of such a space is outlined. Some example translations are given of common communicative primitives into a shared semantic space. FLAWS * SVM and semantic spaces are only illustrated on a single example and do not appear to have been implemented, despite their simplicity. * There is no thorough-going evaluation of SVM or semantic spaces, only a single illustrative example. * It is not made explicit what hypothesis any evaluation would evaluate. There are lots of potential candidates, but none is highlighted. * The related work outlines a lot of other work, but does not attempt much comparison with the authors' proposal. * The organisation of the paper is poor. Although the individual sentences are well written, overall it rambles and has no clear focus. * The difficult aspects of this task for an agent are: selecting which actions to suggest; ensuring that the final set of primitive actions is sufficient to construct all the plans that an agent requires; deciding that another agent's suggestion is good enough for you to support; when an agent has the casting vote, deciding which actions to vote for, given that its own agent may not be involved in the tie; etc. None of this was discussed, but solutions to these problems are essential for the process to work. HYPOTHESES AND THEIR EVALUATION There were no unambiguously flagged and explicitly evaluated hypotheses in the paper. There were numerous remarks that could be interpreted as hypotheses. Of these, I have selected below those that seemed to me to best reflect what they seemed to trying to achieve or that had some suggestion of an evaluation. They are also a fairly representative sample, as many of the claims were similar. * The motivating hypothesis behind this work is that there are no definitive (absolute) definitions of agent communication primitives that are universally applicable." (p230) This use of 'hypothesis' is misleading. This is not really a claim about the work done, but more a background assumption motivating the project. * "The key argument of this article is that the mapping of semantic space provides a means to systematically analyze inter-agent communication" (p234) This mapping is done for a few primitive communication actions, but this is not enough to show that such mapping can always be done. In particular, it looks like these exemplar actions were also used as development examples. There is always a danger that a novel action will embody dimensions or positions in the dimensions not present in the current semantic space, and so will resist mapping. * "It can be shown that SVM will terminate under reasonable conditions." (p244) What follows is an argument that the space of possible primitive actions is finite (although potentially quite large). It is left implicit that one action is removed from this space on each round and that, therefore, the process will terminate. * "The current approach, in contrast, offers a potential means for determining a language with good fit for purpose quickly and efficiently." (p259) No argument was offered that SVM produced languages that were a "good fit for purpose" or that SVM was either quick or efficient. Indeed, it was observed that a dogged agent would succeed in getting all its preferred primitive actions into the language. So, potentially, the language could consist of all possible actions. Also, from the the termination argument, one can conclude that, in the worst case, SVM was exponential in n^d, where the semantic space has d dimensions each with n positions. * "It enables agents that have not been designed to interoperate to come to a mutual and explicit understanding about the semantics of the communication primitives." (p249) The agents /did/ have to be designed to share the semantic space used to generate the shared action language, so some "mutual and explicit understanding" is pre-provided. They could then use this semantic space to understand the primitive actions being proposed by the other agents. * "Specifically, we argued that a simple voting mechanism is the most appropriate method for organizing this semantic fixing activity." The argument for SVM was crude. It was argued that SVM was at the mid-point of a range of possible processes and that both extreme ends of this range were undesirable in different ways. YOUR REVIEWS Generally, good reviews. Here are some of my most common criticisms. * A wide variety of potential hypotheses were identified. Many of them were essentially equivalent to the ones listed above. Some were merely part of the description of the techniques used or were background assumptions motivating the project. Some were too vague to be realistically evaluated. * You were uncritical of the related work, some even claiming that it did include an adequate comparison with the proposals in this paper. I thought the related work mostly described the rival work and had very little comparing or contrasting. * Many of you saw semantic spaces and SVM as two separate techniques. But SVM needed a space of actions to vote for, so I saw them as one integrated technique. Would you see representation with logical formulae and deduction on these formulae as two techniques or one? I guess both perspectives are valid. * Few people commented on the presentation, which I thought was overly long and rambling. For a long way into the paper, for instance, it was not clear where it was going. * No one commented on the evident lack of implementation. Given that that was a key criticism of the 3rd review, I thought you'd be sensitised to that issue.