Rogue Thoughts

I found these old reviews from when I submitted the article to Religion. The peer reviews were generally very positive, but the editors remarks were not that positive. See for your self:

Reviewers’ comments:




Reviewer #1:   

Against anthropological orthodoxy, which locates the responsibility for the credibility of divination practices in characteristics of diviners, this paper advances what might be called the “cognitive repair” hypothesis.  That hypothesis holds that it is the mind’s automatic attempts to resolve the puzzle that the ritual actions present that is the source of divination practices’ credibility with clients.  Rituals displace the intentional component of action, provoking a search for an underlying agent.  This leads to inferences about unobvious agents, who are, therefore, attributed counter-intuitive properties.  Those properties include access to knowledge that these agents reveal via their influence on the patterns diviners interpret.  Thus, the author concludes, divination practices gain their credibility (primarily?) from clients’ confidence in the influence of these unusual agents on the divinations’ outcomes.  The paper is mostly about two empirical
studies.  The first looks at subjects’ perceptions of various actions by a diviner and finds, in a design that disguises the study’s goal, that subjects are significantly more likely to find the resulting advice credible in ritual actions than in alternative actions in which the diviner intentionally established the pattern. Although agreeing with Boyer that ritual credibility turns on the displacement of intention, the author maintains that it turns, specifically, on the status of the posited agent who is behind the whole transaction.  The second study tests this subsidiary hypothesis. 
        This paper is clear (for the most part), well-organized, and direct.  It demonstrates that hypotheses in this area have testable consequences and offers a plausible program of research for testing them, in fact.  It also exhibits a level of sophistication with psychological experimentation, experimental design, and statistical analysis that is refreshing in the literature on divination, ritual, religion, etc.  Attention to the following items (which are in no particular order) would improve the paper.

1. In the paragraph at the top of p. 4 is insufficiently restrictive to characterize divination (other than by using the language of divination throughout).  Why, for example, would science and scientists not also satisfy the conditions the author provides?
2.  The introductory account (near the top of p. leading to the discussion of the first experiment is inaccurate.  The dependent measure is not “the likelihood that the main character acted on the information” but rather subjects’ ratings of that likelihood.
3.  To avoid putting off some readers for the wrong reasons, alternatives to the language of “primitive” and “modern” people on p. 9 might be wise.  (The point, after all, is not about the people but about their cultural settings.)

4.  Having two independent raters and doing the Cohen Kappa test for their agreement would be more convincing than checking the agreement of one rater with the experimenter. 

5.  In the middle of p. 10, the author introduces the second measure about subjects’ ratings of the likelihood that the client found the advice worthwhile, however, the author does not discuss (here) what this measure safeguards against.  It is not obvious.

6.  Expand a bit on the relevant material at the very top of p. 10, so that in the section on “Design” that follows it will be more clear what “(Sets)” is.

7.  The large paragraph on p. 11 is unclear on multiple fronts.  Say explicitly what ACC and AD are.  Say which rating is higher in the Kurabi story.  In what sense is that rating “in this set” (as opposed to “with this set”)?  Clarify what is referred to by “Action type” in the final sentence.  (There is ambiguity here in the use of the term “Action,” given all that the paragraph discusses.)  Is the preposition in the final sentence of this paragraph correct?  Should it be “of” Action type, instead of “on” Action type? 
8. The opening sentence of the second paragraph on p. 14 is oddly framed.  It is “possible” to attribute the effect to universal processes, but that they are “not culturally variable” is not something that the experimental evidence provided here supports.  At a minimum, that would require carrying out the study cross-culturally.   

9.  The first rationale on p. 15 for experiment two does not seem to the point.  Experiment 2 tests whether or not the status of a putative associated agent influences subjects’ judgments about others’ assessments of the credibility of a divination, per the account of the experiment provided in the next paragraph. 

10.  The author should state clearly that the results of experiment two fail to distinguish between two possible measures of a god’s prestige, viz., the frequency with which sacrifices are made to the god as opposed to the value of the sacrifices made to the god.
11.  The first two sentences of the middle paragraph on p. 18 are puzzling.  How does the comment about non-believers’ probable preferences follow?  Indeed, it is seems inconsistent with the comment about the preference of “people in general” in the first sentence.

12.  The discussion in the penultimate paragraph on p. 19 would be enhanced by including an example from the ethnographic literature.




Reviewer #2:

This is a fine contribution to the cognitive science of religion, interesting, original, and well written. Some typos still remain, though (e.g. “Litteratur;” Boyer & Ramble is listed twice). Methodology is quite alright. However, is the author(s) wish to make the contribution still stronger, the concepts of “agency” and “counterintuitiveness” could be explained in more detail. Barrett’s (2000, 2004) idea of HADD thus might be included. You do not make it clear enough what you actually mean by counter-intuitive agents. Some references with this in mind:

Boyer, Pascal, & Harold Clark Barrett. (2005). Domain-specificity and intuitive ontology. In David Buss (Ed.), The Handbook of evolutionary psychology, 96-118. Hoboken, NJ: John Wiley. [about agency detection]

Blakemore, S.-J., P. Boyer, M. Pachot-Clouard, Andrew Meltzoff, C. Segerbarth, & J. Decety. (2003). The detection of contingency and animacy from simple animations in the human brain. Cerebral Cortex 13, 837-44. [about agency detection]

Guthrie, Stewart (E.). (1993). Faces in the clouds. New York: Oxford University Press. [A classic!]

Pyysiäinen, Ilkka, Marjaana Lindeman, & Timo Honkela. (2003). Counterintuitiveness as the hallmark of religiosity. Religion 33(4), 341-55.

Saler, Benson, & Charles A. Ziegler. (2006). Atheism and the apotheosis of agency. Temenos 42(2), 7-41.

Wegner, Daniel M. (2002). The illusion of conscious will. Cambridge, MA: MIT Press. [Includes discussion of Ouija]







Reviewer #3:

1.the article is well written and well researched, although covering only a small area oof research the limitations of which the author, it seems, is not aware of.
2. the research findings represent what Danish people who do not practise divination, think about divination. One suspects that practitioners of divination as, for instance, in Africa, act on different principles; and one expects that the author’s hypotheses will be falsified when exposed to research based on serious fieldwork outside European culture.
3. the article couls be improved considerably by reflecting on the methodological problem indicated above (of generalizing the results from the löimited research area) and by taking into consideration research reports on the practice of divination as, for instance, in Africa (saee, for instance, U.H.Danfulani’s dissertation “Pebbles and Deities” (1995)or his contribution to the volume edited by Gerrie ter Haar: “Imagining Evil”, 2007).  

2.

Editors comments

I quite liked your paper’s extended and nuanced engagement with ritual theory in light of a fascinating set of empirical material. However, some theoretical and methodological questions still trouble me. In the end, these concerns are sufficient to lead me to decline to publish this paper in RELIGION. My co-editor has read the paper, the reviews and my comments below, and he concurs. I will comment on your paper in order both to do you the justice of pinpointing my difficulties and to offer you some thoughts regarding alternative approaches that might assist you in revising what is, my critique notwithstanding, an interesting and potentially quite valuable paper.
I go through it page-wise 
We begin with what might appear as a detail: P. 3: “Consequently the question arises: why does the Ouija board seem to take on a personality when obviously there is none?” Actually, in the anecdote you quote on p. 2 the board does not take on a personality at all, but “the Ouijas written words seemed to take on a personality”. It is not the board, but the words! This is a significant difference, one that has important implications for your analysis.
Fundamentally, I am not persuaded by the displacement of intention theory of ritual. Nor do I find the ritual mode of action to be counter-intuitive: it can only be perceived as such on the basis of a very narrow definition of ordinary action (whatever that may be) like ‘Peter eating an apple.’ My general scepticism with this theory aside, it fails to convince me as you present it. You say “that the immediate goal cannot be referred to the beliefs and desires of the agent.” This reflects the observer’s position (as indicated by “is readily inferable”). So, the reference is actually an inference from the observer’s point of view, it seems: an observer is able to infer that Peter’s “goal is to eat because he is hungry (desire), and that he thinks that eating the apple will relieve the hunger (belief).” Now, Peter crosses himself and Paul will not be able to infer his goal, desire, and belief. However, this distinction between transparent and opaque inference does not correspond to that between ordinary and ritual action.  There are many types of ordinary actions where Paul will not be able to draw the same sort of inferences that are possible in the case of Peter’s eating an apple: Say Peter reads a book: Paul will not know his goal (Entertainment? Education? Planning a trip? Reading a game manual? Sexual arousal?), nor his desires, nor his beliefs. If this does not work, then repairing the “deficiency” does not apply either.
I would think that what we have is a case of deference or deferral, rather than the speculative mechanism that you reconstruct. Your experiment actually appears to measure the former. Let’s take the Kurabi story. To begin with, even where the diviner takes the pebbles in his right hand, beats his hand etc., this is not unintentional as you take it to be. It strikes me as very much an intentional act (the diviner certainly acts intentionally!). However, the intentionally executed action is archetypical as H&L would have it (or not encoded by himself as Rappaport would have it), and the resulting effect of is beyond his control—hence it is deferred, and it seems to me that it is the intentional deference in a deferred mode (= not encoded by oneself) plus the expectation of an intentional result that requires the assumption of an agent. So, my theoretical disagreement here has a methodological counterpart, since I don’t believe that your experiments measure quite what you take them to. Or at least, if I have understood you correctly.
(A note on the dendrologist example: I would have expected Eva to be female, but you write “he”.)
Let’s take a look at experiment 2. Here also I have a couple of problems. To begin with, I think your prestige-strategy is problematic, because, as far as I know, in divination it is not necessarily the most prestigious agents who are invoked, but often less-prestigious ones.
As to the special result of your experiment, I think the fact that Para is given “dried bread” is a deformation compared to the other two conditions where the content of the respective meals is not specified. That may, to some extent, account for the greater difference between the Low and Medium prestige conditions than that between the Medium and High.
I am a bit confused about your criticism of Boyer. Discussing your finding you write: “If ascription of divine agency is secondary to divination techniques as Boyer thinks, we would expect no difference in preference of diviner. If on the other hand, the representation of a counter-intuitive agent is central to divination, we should expect the prestige-bias to create a preference for the diviners communicating with gods of high prestige.” (18-19) From this summary and from the way you initially frame your criticism of Boyer on p. 16, it seems that your theory focuses on counter-intuitive agency whereas Boyer focuses on divination techniques (or ritual form). However, this is not at all what you are testing in your experiment, for the experiment tests the impact of “prestige”, i.e. a specific representation of counterintuitive agency. In order to more clearly investigate techniques, the experiment would need to be devised in such a manner that technique would be held to work even without agency. As it stands now, if may well refine a point in your own theory (but see above); however, it does not refute Boyer.
Finally, your example from Cicero puzzles me. I am wondering whether your interpretation is supported by this source, or whether it is made to fit into your scheme. As I read it, admittedly naively, the anecdote shows that the people were dissatisfied because they felt fooled, and because the rules of the game were fooled around with; there was no deferral, as I would have it; the technique did not allow the situation to speak for itself, as Boyer would have it. I see no evidence for “failed displacement of intention” here.
In sum, if I were not the editor of the journal but were rather invited to publish a comment on your paper, I would do so along the lines sketched above. Being the editor, however, I feel that your paper is sufficiently problematic that I cannot publish it. However, if I have misunderstood you completely, or should you feel that you could reframe your argument so as to make a more compelling case (or if I convinced you to modify your theory), please feel free to submit a new version. Your overall approach to the empirical engagement with ritual theory is work of great value, and work that RELIGION is definitely interested in publishing. My concern is with the details, primarily that the fit between experimental design and hypothesis is not tight enough to ground your argument effectively.
I understand that this is a disappointing decision. I hope that this process will have contributed to your work, helping you to refine your theory. In all sincerity, I thank you for you generosity and patience in allowing RELIGION to consider publishing this work.

I could post my replies if anyone found them interesting, but the article in it’s present form reflects these replies.

It is always interesting to get peer review on your stuff. I have received very different quality though. Every now and then a reviewer just wants to read something different, and does not really care to read the article at hand all too attentively. That is of course my subjective opinion.
But who peer reviews the peer reviewers?
I thought why not post the reviews on the internet so it’s open to anyone what the peer reviewers review. Here is a review I got from an undisclosed journal recently on this article

I read the article you sent earlier in the meantime. All in all, I am a bit surprised about the number of people who already reviewed it given its present quality.
 
Leaving the intentional structure of a human action (priest) within a ritual largely open leaves room for supernatural intentionality to come in. Not only more so than in human intentional action but also more so than in human coincidental (ritual performed before client posed question: same physical event but at different time) or in human accidental action (same physical event but ritual carried out unintentionally). The latter two comparisons are my wild assumptions, since unfortunately the results for the last two conditions were not reported in the present paper. Since all 4 conditions seem relevant and interesting to me, I wonder why they were not reported.
 
 

Whereas I think the theoretical background of the research is interesting and relevant, I am less convinced by the empirical work.
Here are some reasons why I think the paper needs some more work:
1.   The abstract is unfinished.
2.   A native speaker needs to read through the whole text.
3.   Exp. 1: Parts of the methodology could be clearer. I found the details of the randomization in experiment 1 confusing and still am not sure about how it was done.
4.   Exp. 1: Variables that were formerly described as dependent variables (action, advice, p.11) show up as independent variables in an ANOVA without further notice in the text. Only the table informs the reader that a new dependent variable was created that treats responses to the ‘advice’ and ‘action’ question equally and that is labelled ‘credibility’. This might be a good choice for ‘advice’ but I am less sure in the case of ‘action’ – also because it seems that action is rated significantly higher than advice – more people predict someone will act on the advice of the ritual master, than they see the advice as valuable (=credible).
5.   I guess, I expected an ANOVA including action type (2: intentional/ritual) x question type (2: action/advice) and story (3: kurabi, dendrologist, banban) with results for main effects and interaction effects (does not look there are any). Instead, the way results are presented here is somewhat fragmented.
6.   About the ‘advice; and ‘action’ question. I am wondering if the probably significant main effect for question type (‘advice’ vs. ‘action’) is sufficiently discussed at the end of experiment 1. One idea brought forward in the introduction is that the Danish participants might differentiate in their predictions (difference b/w advice and action) more for the indigenous vignettes (not very credible but person will act anyway) than for the western one. They don’t which is interesting.  Secondly and more generally, I am wondering if the advice isn’t what counts more for the analysis of the hypothesis (amount of human vs. supernatural intentionality in ritual action and its influence on credibility of the outcome of a ritual): So I wonder why the ‘action’ question was asked in the first place? I am not sure if this was explained somewhere. There was an explanation why the ‘advice’ question was added but not why the ‘action’ question was chosen in the first place. This is a rather minor point but since both questions are there, I was wondering about the interpretation of probably diverging results.
 
Further remarks:
If Cohen’s d is calculated even better but why make it sound like it is some ‘extra’ analysis: why not simply add it in an extra column next to the p values.
At times, things that belong in one section show up in another section. For example in experiment 2, the idea Boyer supposedly entertains about divination techniques (no preference for diviner) should not be discussed in the results section. It belongs either in the discussion or even better in both introduction and discussion. Further on, in the results section, the measures should be taken at face value: That participants chose the gods who get more (and) or bigger offerings is the result of the 2nd experiment, NOT that they see these gods as more privileged – the latter is an interpretation and belongs into the discussion section. About this point I would like to make a comment about alternative interpretations that are not discussed: First of all, the participants could base their judgements on no other information than the differing regularity/amount of offerings (all other being equal). Secondly, other than attributing different levels of prestige to the different gods, it is to me as likely that participants consider the offerings as rewards or some kind of reinforcement schedule for the Gods. I give more therefore I should get more. Interestingly my own notion in picking one of the gods was to choose the -in my view- most human one because I considered him most trustworthy. As participant I would have gone for the ‘meals every day god’ rather than for the ‘banquet’ god, since the first seems to need food regularly, whereas the other one seems to be a spoiled (and therefore possibly capricious) diva . Anyway, the author sees a shortcoming in the methods of experiment 2 in that the two dimensions regularity and amount of offerings are not clearly varied in the different options for the offerings given. I am sorry to say this but all in all, the second experiment does not convince me.
In conclusion and from my point of view this paper needs some revision before it can be published.
 

Maybe I will publish my responses later. Actually I got the idea to collect a lot of other peer reviews to do a bit more research on peer review.

I wrote this review of Roger Becks book: The Religion of the Mithras Cult in the Roman Empire – Mysteries of the Unconquered Sun for the Journal of Roman Studies (2009). the book is interesting because it uses cognitive science to throw light on history. This was also the approach I took in my PhD dissertation “The Dissemination of Divination in the Roman Republic”
I really like the eclectic approach of Beck, but eventually you get caught up in too many views on the subject. Some views are not consistent with each other, and it is difficult to find out what is what. That is the problem with eclecticism. Ok. Here it is:

Roger Beck: The Religion of the Mithras Cult in the Roman Empire – Mysteries of the Unconquered Sun, Oxford University Press 2006

By Anders Lisdorf

Roger Beck should be no stranger to anyone who have even the slightest knowledge on Roman Mithraism. Indeed one could legitimately claim that he is one of a handful of “grand old men” who have been vital in shaping contemporary research on Mithraism and he has dedicated several decades to the study of Mithraism. This is why it is all the more impressive to see him provide a new look at this subject without stubbornly clinging to old views, but with an open explorative mind set worthy of a true scholar. The Religion of the Mithras Cult in the Roman Empire is a synthesis of Beck’s earlier work with new theoretical advances particularly in the field of the cognitive science of religion. This makes for an interesting eclectic journey through one of the most mysterious cults in the Roman Empire.
Beck starts by showing that contemporary scholarship, with a few exceptions, has reached a dead end in interpreting the meaning of Mithraism mainly because of the paucity of textual evidence. This has resulted in interpretation being replaced with a narrow positivism focusing purely on the material evidence which exists in abundance compared to the textual. This impasse in traditional approaches is attributable to a number of problems: they undervalue the literary testimony in favour of the monumental; they undervalue the significance of the mithraeum; since there is no doctrine, it has been implicitly assumed that the mysteries did not entail any sophisticated cognitive enterprise; and there has been a total disregard of semantics and semiotics. The resolution is to be found in new methods in the cognitive science of religion. Instead of the old hermeneutics (of Franz Cumont) or the new narrow positivism (of Manfred Clauss), the mysteries should be seen ‘(..) as a system of symbols both complex and orderly, apprehended by the initiates in cult life and especially in ritual’ (4).
One of the questions that have been given much attention in the history of research is the search or the original Mithraic doctrine. Beck is very clear that this search is futile. Rather than on a doctrine, Mithraism was based on a limited number of fundamental principles that Beck calls axioms. They are: 1) Deus sol invictus Mithras and 2) Harmony of tension in opposition. These principles may at first glance appear equally mysterious to the modern scholar. The first axiom aims at clarifying that Mithras is the central deity of the religion and he is the unconquered sun. The second axiom makes clear the central significance of opposites such as night and day in Mithraism. These axioms find expression in a number of motifs such as the sacred story of Mithras, the cosmos, the sublunary world and the destiny of human souls. The symbol system, the object proper of the study, conveys these axioms and motifs and is manifested in three distinct structures: the central scene of the tauroctony, the mithraeum and the organizational structure of the 7 grades. These structures were apprehended by the initiates primarily though ritual action, but also through the iconography, exchange of words (to avoid the word teaching) and ethical behaviour consonant with the mysteries.
Throughout the book this interpretative scheme is filled out with impressively meticulous analysis of the textual evidence, the symbolic structure of the mithraeum and the tauroctony. These analyses illustrate that the mithraic symbol system was communicated in the idiom of, what Beck terms “star talk”, where the central focus is the mystery of the descent and ascent of the human soul. There is no doubt that the Beck’s knowledge of all extant evidence is comprehensive and the interpretation he provides is among the most coherent and all encompassing.
There are however also a number of critical points. That Beck’s approach is eclectic is perhaps an understatement. He picks and chooses whatever part of any theory as long as it throws some light on the Mithras cult. The upside of this is a very inspiring and creative analysis. The downside is occasional confusion and theoretical inconsistency. An example of this is his explicit appropriation of the ideas of the cognitive anthropologist Dan Sperber. Later Beck endorses the idea of thick description of Clifford Geertz. Probably no two theories could be more inconsistent with each other. According to Sperber symbols don’t really exist, whereas symbols constitute the texture of culture according to Geertz.
Beck also uses modern ethnographic evidence. More precisely he uses the the practices of the Chamulas, a people of Southern Mexico, who also have a sun cult, as an example. It is difficult to see the theoretical justification of a one to one comparison, since there are also great differences between the Chamulas and mithraists. It seems that Beck thinks sun cults are predetermined to use a specific symbolism (82). This seems to deserve some further explication and justification: could we just just pick and choose from the ethnographic record the religion that comes closest to our personal interpretation of the Mithras cult or any other cult?
That said, the overall impression is that this book should be regarded as a quarry of information and good ideas on the Mithras cult. Along with the work of Richard Gordon and Luther Martin it is the best bid contemporary scholarship has on what sort of religion the Mithras cult was.