Feedback on Clark et al Paper

Hypotheses: 

	Clark et al do not advance an explicit
hypothesis. Indeed, this is one of the main criticisms of the
paper. There is an implicit one that:

	Multi-agent system technology can be successfully used to
	solve the problem of distributed information retrieval. 

However, they do not contrast this success, either positively or
negatively, with rival approaches, e.g. those that use more
conventional information retrieval technology. 

Evidence:

	The authors have (presumably) implemented a multi-agent
system and applied it to the task of distributed information
retrieval. However, there is no account of any
evaluation. Indeed, the absence of an evaluation and some
vagueness in the description, do make one wonder if a
fully-functioning implementation ever existed. An example of
vagueness is the reference at the end of section 1 to "an
external thesaurus (like WordNet)". Was WordNet actually used, or
some other thesaurus, in which case, which one?

Possible Criticisms: Since 1997, when this paper was published,
there has been a great deal of work on distributed information
retrieval, e.g. the rise of Google and work on named entity
recognition for extracting meaning from free text. We must not
judge this paper by 2006 standards. Nevertheless, there is a lot
wrong with it in purely methodological terms.

* The absence of an explicit hypothesis. This makes it difficult
to know what any evaluation would actually be evaluating. 

* The absence of an evaluation. Without an evaluation, how can we
assess whether the system was a success or how the approach
compares with rival ones? Information retrieval has standard
measures of precision and recall, but these terms are not even
mentioned let alone measured. 

* Related work is thin. The work is only compared with other
multi-agent approaches, and even this comparison is
superficial. We really want to know how multi-agent approaches
compare with rival ones, but these are not mentioned. The
comparison with other multi-agent approaches only says that they
tackle only part of the problem, but we also want to know how
they compare at the technological level and in terms of success.