7
$\begingroup$

This question seems to have found an error in a computational geometry paper on CoRR, and David Eppstein in an answer confirmed the error. I haven't checked it myself, but I wanted to ask if we should set up some policy to contact an author with a pointer to questions of this sort. This seems to have a lot of potential as an additional service to the community this site could provide (part of "Peer Review 2.0").

A concrete suggestion: if a Q&A locates/seems to locate an error in a paper, a moderator should email the corresponding author of the paper with a pointer to the question, and comment in the question that this happened.

$\endgroup$
4
  • 2
    $\begingroup$ there seems to be some debate on the appropriateness of such question (in the absence of prior contact with the authors) meta.cstheory.stackexchange.com/questions/204/… $\endgroup$
    – Suresh Venkat
    Commented Sep 2, 2010 at 2:56
  • 1
    $\begingroup$ Thanks. I had seen that question here on meta, but before David Eppstein commented. It seems to me, though, if someone asks and someone else answers, it's a fait accompli, unless we can erase questions, which seems like a power to use extremely rarely, if at all. So maybe put in the FAQ to suggest contacting the authors first, but if the situation arises, why not give an author a chance to respond (or fix a problem)? $\endgroup$
    – Aaron Sterling
    Commented Sep 2, 2010 at 10:12
  • $\begingroup$ Is it possible for moderators to delete entire threads of questions/answers? That would make things easier. $\endgroup$
    – Robin Kothari
    Commented Sep 3, 2010 at 3:32
  • $\begingroup$ no idea. hopefully we will find out. I think it might be possible. $\endgroup$
    – Suresh Venkat
    Commented Sep 3, 2010 at 5:06

3 Answers 3

Reset to default
4
$\begingroup$

A general observation:

This point about notifying an author before making a public display of an error goes to the larger point about the community norms. We might include some text specifically to the point about notifying authors in the FAQ, but also the larger point that people should be respectful on the site (in questions, answers, and comments). [Comments tend to be the wild west on these sites, but it would be good to enforce standards (by example) there too...]

So long as all questions are asked in good faith and in the spirit of collaboration, trying to cover every possible subclass of problem should be less of an issue.

Does anyone know of a good, short essay on the collaborative scientific process (peer review, etc.)? I think that it will go a long way to characterize the site Zeitgeist.

$\endgroup$
1
  • 1
    $\begingroup$ A short google search turned this up michaelnielsen.org/blog/…, but I think that we can do better. Surely Popper wrote about this? $\endgroup$
    – Shane
    Commented Sep 2, 2010 at 15:36
4
$\begingroup$

I think there are two separate issues here.

If you think you found an error in a paper, you should never discuss it on a public forum before giving the author a chance to think about and correct it first, in my opinion. Of course, it is sometimes difficult to tell whether a point which you have realized is an error in a paper or not, but if you think that there is a good chance that it is an error, it means that there is a good chance that your post on a public forum might embarrass the author of the paper, which is probably a situation you want to avoid. Although this is not specific to this particular website, a sentence or two on this point in FAQ on cstheory.stackexchange.com may be useful.

(Therefore, as a tangential topic, I have some uneasy feeling about the discussions on the recent paper by Vinay Deolalikar. I have never followed the discussions closely enough to draw any reliable conclusion, though.)

However, a question or a discussion may reveal an error of a paper in an unexpected way. In that case, it would be better that someone contact the author to let him/her know about the finding. I do not think that moderators are the most suitable for this job, but if someone not in the discussion notices a case like that and it does not seem that anyone has contacted the author, it is probably good to draw the attention of the moderators (by flagging the post) and ask the moderators to handle the issue than to contact the author directly. To make this work, if someone (in the discussion) contacts the author, it is a good idea for him/her to drop a line in the forum so that other people do not have to worry about the possibility that the author might not be notified of the discussion.

$\endgroup$
6
  • 1
    $\begingroup$ I think this all seems reasonable. Maybe you could amend the FAQ accordingly ? meta.cstheory.stackexchange.com/questions/225/… $\endgroup$
    – Suresh Venkat Mod
    Commented Sep 4, 2010 at 20:40
  • $\begingroup$ @Suresh: I added an entry to the FAQ. I think that it is the best to merge it with the existing entry, but currently I do not have a good idea how to merge. $\endgroup$
    – Tsuyoshi Ito
    Commented Oct 3, 2010 at 17:46
  • 1
    $\begingroup$ @Tsuyoshi: Do you think your opinion (which is that one should not discuss errors in a published paper in public without informing the authors first) is a widely believed in the CS community? I'm asking just because I find it hard to believe this. I would have expected that public discussions on public papers is completely fine (and public discussions on privately-circulated papers is not fine). But the community just had an entirely public discussion on a private paper of Deolalikar, so I think the community does not mind public discussions on public papers at least. $\endgroup$
    – Robin Kothari
    Commented Oct 3, 2010 at 18:43
  • 1
    $\begingroup$ @Robin: Well, honestly, it depends. The most important rule is, as Shane stated, be respectful. The seemingly same actions become either acceptable or not depending on whether the actions are done out of respect. But I find it very dangerous to discuss other people’s error in public, and I hope that people share this view. $\endgroup$
    – Tsuyoshi Ito
    Commented Oct 3, 2010 at 18:56
  • $\begingroup$ @Tsuyoshi, I think it's actually ok to have separate answers, because one is about scope (CAN I ask questions about theorems in papers) and one is about best practices (what should I do if I think I've found a bug). So maybe you should merely cross link the first to the second. $\endgroup$
    – Suresh Venkat Mod
    Commented Oct 3, 2010 at 19:44
  • 1
    $\begingroup$ @Robin, The Deolalikar discussion only happened because someone leaked the paper, and then Lipton asked Deolalikar if he would consent at that point to a public discussion. In general, I think being conservative on this front is better: the worst-case scenario is that the bug takes longer to be made public. But if you go public too soon, the worst-case scenario is more unpleasant (authors get wrongfully slammed with the 'bug' label, which even if refuted later is hard to shake). . BTW there's potentially something brewing here: cstheory.stackexchange.com/questions/1876 $\endgroup$
    – Suresh Venkat Mod
    Commented Oct 3, 2010 at 19:48
2
$\begingroup$

This is relevant to the other thread too, but I thought I'll just post my comment here. It is a combined answer to the questions asked in both threads.

The other question, which reads "Are specific questions about Theorem X in paper Y fine?" actually covers two types of questions from posters.

Type 1: Questions like "How does one reach eq 3.14 from eq 3.15 in paper X, page Y? I tried the following ... but it didn't work."

I think most people would agree that these questions are appropriate for this site. The other type of question is the following:

Type 2: "How does the proof of Theorem 3 in [citation] handle the case of disconnected graphs? I don't see how it works for that case."

This question might be pointing out a genuine error in a paper. But then again it might just be something the poster did not understand. What I want to say is that it is often hard to tell whether there is a mistake/confusion/non-understanding on your part, or an error in the paper. These correspond to type 1 and 2 questions respectively. If the poster is not able to distinguish between the two types of questions, and if we make it a policy to only allow Type 1 questions and not Type 2, this will be hard for posters.

Would a solution like the following work?

We allow both type of questions. Questions of type 1 are handled as they usually are by the community. If we realize that it is a question of type 2 (i.e., there are answers which indicate that there is a genuine mistake in the paper), then a moderator emails the author of the paper and informs him of the thread, and offers to have the thread deleted.

Now the author may choose to have the thread deleted, or reply on the thread explaining how the poster has misunderstood and it is not an error, or reply thanking the poster for finding the error, or do nothing, etc. (I am assuming that moderators can delete threads.)

$\endgroup$
2
  • 4
    $\begingroup$ I'm not getting why the thread should get deleted if there is in fact a bug ! isn't that a good reason to keep the thread ? $\endgroup$
    – Suresh Venkat
    Commented Sep 3, 2010 at 5:06
  • 2
    $\begingroup$ From David Eppstein's comment in the other thread, I understood that if there is a bug we are subjecting "the authors to public embarrassment." I thought perhaps deleting the thread (if the author doesn't want the thread to be public) will solve that problem. $\endgroup$
    – Robin Kothari
    Commented Sep 3, 2010 at 5:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .