Feedback on Reviews of Nourani Paper.


I feel I ought to apologise for asking you to read such a terrible paper,
especially as the first review, but I had a good reason for asking you to read
it, and for asking you to read it first; I wanted to spur you to have the
courage of your convictions. It would be easy for you to be intimidated by the
deluge of jargon and authoritative sounding assertions, and to give the author
the benefit of the doubt. But you need to learn from the outset that it is the
author's responsibility to make it clear what claim they are making and to
convince you of this claim with clear evidence. If they fail to do this then at
the least they have made a presentational failure. You may also suspect that
they have made a scientific failure.

I start with an advantage, because this man has form. He seems to rely on the
shotgun principle of firing off lots of papers in the hope that some will find
their target. Surprisingly many seem to do so. This is clear evidence that the
refereeing and editing processes in our field are failing. Your generation has
the responsibility to put that right.

FLAWS: Turning to the paper itself, it mostly consists of a wide-ranging and
shallow survey of many areas of AI and Philosophy, consisting of jargon-ridden
assertions and poorly supported conclusions, with no examples to illustrate the
work he is describing. None of this is self-contained, with many references to
both his own previous papers and those of others. It is thus very hard to verify
any of his claims, since you would have to read or already be familiar with so
many other areas, including a large body of the author's previous work. None of
this is helped by the poor presentation, which is riddled with spelling errors,
ungrammatical and convoluted sentences, sloppy citation and poor
translations. The title does not reflect the content. In particular, haptic
computing (touch sensing) is hardly discussed. Neither is it clear what the
problem of "visual ontology" is or what solution is proposed to it. Finally,
"beliefs, desires and intentions" are universally called BDI; I've no idea where
BID came from.

HYPOTHESES: Interestingly, collectively you identified many different possible
hypotheses. Almost any of his various assertions could be interpreted as a claim
or hypothesis. My favourite would be his three theorems and one proposition,
since this is how theoretical computer science expresses its hypotheses, so
there are unambiguous claims here, two of which appear not just to be
repetitions of claims from previous papers.

EVIDENCE: The various assertions are apparently justified by the preceding
expositions, but are usually non-sequiturs or just hard to make sense of. Two of
the theorems are unaccompanied by proofs. One has a proof overview. However,
since no mathematical definitions are given of the relevant concepts: Morph
Gentzen Logic, virtual tree model, etc, it is impossible to understand exactly
what claim is being made, let alone to evaluate the one proof outline. There
appears to be no implementation and, thus, no experimental evidence. 

COMMON MISTAKES: Here are some particular points that people got wrong:

1. Identifying "exploratory investigation to suggest a hypothesis" as a kind of
contribution. Normally, some experimental work would throw up some
patterns. There were no such experiments reported. Neither was a hypothesis
found and announced. Many people justified this classification by pointing to
what was essentially a literature survey. 

2. Considering how damning a criticism most of you made of the paper, I found
some of your final numerical assessments to be generous. Don't be afraid to use
the full scale, e.g. 1.

3. Some of you claim to have found proofs of the various theorem statements.
There *is* a discussion following one theorem and a paragraph labelled as "proof
overview" following another. However, as an ex-mathematician myself, there was
nothing here I could recognise as even a proof overview.  Without formal
definitions of the concepts involved, I don't see how a proof is even possible.

4. Some people thought Nourani was proposing a model of a natural system. This
would be Cognitive Science, e.g. trying to build a computational model of some
aspect of the mind. It would require experimental comparisons, e.g. with human
cognition. There is nothing of this kind in the paper.

5. Some people thought Nourani had built a system, e.g. the DESIRE framework.
Although I admit it is anything but clear, my reading is that this is just a
theoretical framework and that there is no implementation. Certainly, no
experimentation with such a system is reported.

6. Some of you made heroic attempts to precis the paper in a coherent way, and
came up with very plausible accounts. It is natural to try to make sense of the
incomprehensible, but I suspect you were reading more into the paper than was
actually present. Of course, you must try hard to find the meaning in a badly
presented paper -- but not too hard, to the extent that you invent meaning. 

7. If you choose any of the "other" options, then specify the criticism,
e.g. "the paper is deficient in some other way (please specify)".