On modifying our scope: A proposal

Question

There's been a lot of chatter of late about our scope, the declining number of questions being asked per day, and whether we're being excessively strict about question scope.

This question is intended to revisit the debate over scope. I'll outline some reasons why I think it's a good idea to revisit the question, and propose a new scope definition that we can debate.

Why revisit our scope ?

Our original intent in defining our scope was to shelter the site from a potential deluge of basic algorithms questions, and to model our community on the very successful MathOverflow model. I think two things have changed since that time:

1. We now have a natural forum (cs.stackexchange.com) for the more "basic" algorithms questions, and the cstheory "brand" is well established as a 'research-only' venue. So we have an easy process to shunt questions that are obviously undergrad level.

2. As has been observed, our overall participation numbers have slowed, as well as the pace of new upvoted questions. Given how many people I personally know who aren't active on the site, I see no reason why our numbers should be slowing.

My view of the problem

We are not the same as the mathoverflow community. TCS is much smaller to start with, and there's a much smaller gap between the "starting grad student question" and "advanced research question" in TCS than in math. By limiting ourselves to the research end of the spectrum, we are

• excluding questions from people who might be 'research-level' in a sub area but aren't experts in some other domain. This is often mitigated by reputation (I might be able to get away with asking a basic complexity theory question), but it shouldn't be !
• creating the impression of an unfriendly site (which is a bigger problem in terms of maintaining the audience)
• spending far too much time policing the boundaries of what's acceptable, instead of being willing to accept some small fraction of lower-level questions in order to also bring in more interesting ones.

A proposal

We should allow questions that might be asked by a grad student in a grad theory class, even if they are not 'research-level', as long as they are looking for a deeper understanding. For example, a question "What is Sauer's lemma' could be dismissed because it's on Wikipedia, but "How should I go about proving an upper bound on VC dimension" would be ok.

This is not perfect as examples go, and I'm happy to hear modifications and suggestions.

But please keep the discussion constructive and focused. Just because your questions are closed doesn't mean that there's a problem. Conversely, just because you like the site just as it is doesn't mean you can't try to imagine something even more useful to the community.

Answer 1

I'm in favor of rejecting undergraduate (and most graduate) homework problems, admitting anything that seems to come from a practitioner with a real theory question (however easy) that arises in practice, and using judgment for the other cases. There is an overlap in level between the first two, but I think you can often tell from the question which is which, and if you can't, we can query the OP to decide.

My reasoning is: if you encourage questions from people with real problems, you will occasionally get very interesting theory problems that somebody actually cares about, and I'd like to see more of these. If you encourage questions from people with homework problems, all you'll ever get is a deluge of even more homework problems.

I don't think we should close questions from practitioners as being at too low a level, as this sends out a message "don't ask us questions; you're too stupid". If we get a lot of questions whose answer is, say, the assignment problem, we could close them as duplicates of each other, even if the questions aren't exactly the same.

Answer 2

We should allow questions that might be asked by a grad student in a grad theory class, even if they are not 'research-level', as long as they are looking for a deeper understanding. For example, a question "What is Sauer's lemma' could be dismissed because it's on Wikipedia, but "How should I go about proving an upper bound on VC dimension" would be ok.

This seems like a good start to me. Especially as someone from a university with a small theory group, having a reference on certain proof techniques would be great.

I think that we could agree that the following examples are too broad/ill-formed:

• How do I prove a lower bound?
• How do I use the Lovasz Local Lemma?

But perhaps the following is a bit more focused:

• Is a cell probe lower bound appropriate for X application?

I'm interested to hear thoughts on this aspect.

Answer 3

Though the scope of this site is mostly defined in terms of topics and level, clientele is another important factor. For question that are in the overlap between here and 'general audience' sites, the difference is mostly in the type of answer that such questions get. In such cases, users may prefer one site over another depending on the type of answer they want.

For example, I am a mathematical logician, so my relation with TCS is mostly tangential. As a result, my knowledge of the field is very uneven. It's fairly common for me to encounter something in TCS that I don't know but is not necessarily that high-level, something that a young graduate student in TCS may ask. However, the student and I need very different types of answer to this question. On the one hand, a carefully crafted and detailed answer makes a lot of sense for the student since they are seeking to learn more and hone their skill set. On the other hand, a brief explanation skipping over routine steps is perfect for me. I really just need to plug a gap in my knowledge base, my general skills are already well developed so I can easily fill in the missing steps. Even if the question is the same, it's better for me to ask it here and it's better for the student to ask elsewhere.

I think the scope of this site should allow this kind of flexibility. It wouldn't hurt to be explicit about this in the statement of purpose. This is also a good place to remind users to add motivation and background for their question. Otherwise, it's very difficult for the community to assess whether this is the right place for a borderline question. This is difficult, but it's not a bad idea to try to cultivate a certain level of tolerance for questions that are not clearly out of scope. Users who do not find the type of answers they want will eventually find their way elsewhere.

Answer 4

I think there are several issues here that are being confused. I'll number the issues and give examples to illustrate the point. I'm not really advocating anything in this post, I'm just trying to clear the confusion. It seems to me that people are talking past each other just because they're arguing about different issues

---

Issue 1: "Deep understanding" questions.

I think everyone would be fine with the OP demonstrated understanding of the standard answer and asked for a deep understanding.

Non-example: "What's the definition of AC0?". This question can be answered by reading any standard text on circuit complexity, and thus should be closed.

Example: "Why does AC0 allow unbounded fanin AND gates? NC0 has a gate-independent definition (allow all 2 input gates), why does AC0 focus on AND/OR gates? Changing the gates changes the class, so what's the motivation?" This is an interesting question, and someone on this site might have an excellent answer. It should remain open.

Conclusion: I think Issue 1 is a non-issue. I don't think we are divided about this issue. If the OP demonstrates understanding of the standard answer and asks a "motivation" or "understanding" type question, I think people wouldn't vote to close. (Please leave a comment if you think this conclusion is incorrect.)

---

Issue 2: Is the function IsInScope() solely a function of the question or does it also depend on who asked it?

This is the issue raised by Peter. While we don't want to entertain undergrad homework questions, often questions from people outside the field are at the level of undergrad homework questions.

Example: "I'm stuck on this problem [followed by a description of the problem] while designing a maze-navigating robot. Any suggestions on how to solve it?" Answer: "This is NP-complete using the following simple reduction from longest path. However, you could try the following heuristics."

Any problem that is a simple reduction from longest path might be an assignment problem in an undergrad course. So if this were asked by an undergrad, we should close it. But Peter suggests that it should be kept open if asked by a practitioner outside the field.

Conclusion: This issue is not resolved. I don't think people are in agreement about what to do about this or how to distinguish whether the OP is an undergrad or a practitioner in another field.

---

Issue 3: Grad level questions: Should we lower the bar from research-level to grad-level?

These are questions that would be considered basic grad level in field X by persons who study field X. However, as in Issue 2, these might come from people outside the field, for whom the question is not easy at all. But let's assume that questions come from grad students in the field so as to not confuse this issue with Issue 2.

These questions might be homework questions for grad courses (which means they can be difficult questions) or questions asked by grad students to the instructor during/after class.

Maybe this issue separates into two issues. The first is about grad-level homework questions and the second is about questions asked by a grad student, which are not research-level, but the answer can be found in a grad-level text book.

Conclusion: Currently I don't think there is consensus on this. Grad-level homework questions are sometimes closed. Questions whose answers are in grad-level text-books are often closed. It's not clear if the proposal is to change this situation or not.

Answer 5

We should be specific not only about what is on-topic but also about what is off-topic.

I think typical questions at the level of undergraduate homework/assignment/textbook exercise should remain off-topic, i.e. a question is off-topic if

• it is an exercise from a standard undergraduate textbook, or
• it is a typical assignment/homework problem in a typical undergraduate course on the topic.

I think the idea of this lower-bound on cstheory's scope boils down to the following:

A question is off-topic if a typical (or maybe even somewhat bright) undergraduate in CS who has taken a typical undergraduate course (or has read a typical undergraduate-level textbook) on the topic is expected to know the answer.

An exception can made for questions where the OP shows that they know the standard answer to the question and explicitly ask for a deeper understanding or insight (which a typical undergraduate is unlikely to be able to provide).

I think this is a lower-bound if we want to keep the attention of the target community of the site (TCS researchers in academia/industry and graduate students, i.e. people whose job is research in TCS). If we go below that level I think we will loose users from that community.

Answer 6

The question about scope is an important one. The answers and comments make one thing clear: we have not only to decide what we want, but also why we want it. Because in the end, whatever we do, we have to do it for the right reasons.

For example, why change the scope at all? So far, we have vehemently defended it against numerous incurions, shooting down arguments like low stats, ivory tower and lack of non-experts to ask questions with the argument that the community works and likes what it has. What changed? Some numbers and a couple of disgruntled people outside of our target group can hardly be the reason.

Another example, what are the real reasons for not allowing undergrad questions? Given that the same question has been proclaimed OK if it is asked by somebody who is not an undergrad student (be it practitioner or researcher of another field), the ban has the taste of hypocrisy; is the only reason something like "I don't want to be plagued by noisy students in my free time"? If so, that would be a weak reason¹.

That said, if the scope is to be changed, I see two possibilites:

1. Extend vertically, that is non-research level TCS questions.
2. Extend horizontally, that is to research-level questions in other CS subdisciplines.

Variant 1 is problematic in the light of the above. Either you create a very weird and not effectively enforcable set of rules, or you allow all TCS questions (which does not seem to be the dominant wish). The notion of "level" has always been squishy, but at least "research" is relatively clearly defined. "Grad-level" is already incredibly ambiguous².

Variant 2 is less problematic in principle but all but impossible practically: there are no experts of non-theoretic CS here (presumably), the site has its brand as "theoretical". I fail to see how you could get experts from other fields engaged here. Essentially, they would not want to for the same reasons the founders of cstheory did not want to be part of a all-encompassing CS community: low signal-to-noise ratio (because of the dominance of TCS).

In either case, we have to find an answer (with good reasons!) to the question: why not go all the way and join forces with cs.SE?³

---

1. I strongly feel that the reason for rejection has to be found in the question. In particular, it has to be independent of the person asking. Otherwise, you get weird judgements and (perceived and actual) discrimination.
2. We have grad students (in AI) that come to us with next to no TCS knowledge. Literally. They are taught things like finite automata from the basics up. So, would a typical undergrad question posed by an actual grad student (who is an undergrad in TCS) be ok?
3. That would be a horrible idea right now, but it will hopefully become a viable option in the future.

Answer 7

Disclaimer: I am a moderator on Computer Science Stack Exchange. While I try to judge this proposal on its own merit, it is likely that the time and energy I have invested in CS.SE influences me.

I'm afraid that broadening the scope in this way would destroy the existing community. A Stack Exchange site is defined by two things: the kind of questions it accepts, and the audience it attracts. Even though each question is in principle judged on its intrinsic merits, the long-term effect is to create a community.

On this site, the community is centered around researchers in theoretical computer science. Participants tend to be researchers in TCS, researchers in other fields, or graduate students aspiring to become researchers. The participants who are welcomed are generally those who came with the expectation of participating in a community dominated by TCS researchers.

I do not see a problem with allowing more applied questions. As long as academics and people who want to fit in with academics dominate the site, that isn't a problem. It may be difficult to bootstrap an applied CS community (it would have been a lot easier to do when the site was young), but if it works, it won't change the tone of the site, and if it fails, the site won't have changed.

On the other hand, opening the door to more basic questions means opening the door to a broader audience. This should be a carefully controlled process. In particular, I advise you not to try to draw on Stack Overflow as an audience base. A majority of Stack Overflow does not know or care what it means to be a scientist. The kind of people you would attract are those who now post on Programmers. If you attract those people, be prepared to moderate their content. And to moderate them.

Reacting to specific points in your proposal:

We now have a natural forum (cs.stackexchange.com) for the more "basic" algorithms questions, and the cstheory "brand" is well established as a 'research-only' venue. So we have an easy process to shunt questions that are obviously undergrad level.

Keep in mind that CS.SE is not exclusively for basic questions. It is for questions at all level, ranging all the way from CS 101 homework to applied research (and even research in theory, but we suggest CSTheory.SE to the rare asker who comes with a theory question that seems to go unanswered because it is too hard).

excluding questions from people who might be 'research-level' in a sub area but aren't experts in some other domain

From what I see (I admit to not following CSTheory.SE very closely), this already happens: as long as there's a feeling that peers are talking with peers, questions tend to be appreciated, left open and answered.

creating the impression of an unfriendly site (which is a bigger problem in terms of maintaining the audience)

I think the community is doing an adequate job of retaining the audience it likes in and keeping the undesirables in.

All in all, while the current guidelines may be stricter than this, I think an intelligent application of the existing policies already gives you what you want.

Answer 8

1st, the good news. the site has many experts subscribing & signed up. thats a huge accomplishment and not easy to pull off. it has a high quality of content. it has indeed attracted many experts.

next, the bad news: the experts are not really engaged a lot on the site (a subjective observation after 0.5 yr, admittedly short, although not compared to the overall lifetime of the site), also indicated by decreasing questions even with increasing membership. there are a few high-engagement experts but there are many low-engagement experts.

by "experts", think of ppl that are highly qualified in TCS & which we would like to participate & hear their contributions but dont nec have high site rep. this could be quantified further with good statistics & analysis, but unfortunately it takes awhile to find statistics like this from current data.

so, the big question is, why is that?

imho one of the highest priorities of the site is to get high participation from experts but not merely in answering questions. at least imho it is a degeneration of the site purpose if the experts are active but only mainly to "shoot down" questions via negative votes, negative comments, constant reference to the FAQ/charter etc.-- that seems like "pushing the river"...

what do experts in the field like to talk about? what are their own questions?

at times seemingly theres an unstated/unspoken/mistaken belief/groupthink/folklore around here that poor questions scare away the experts from participating. however, there are many experts on here, and there are many that are not really participating that much, and personally, I doubt its because of poor question quality.

would like to see a charter that encourages higher participation esp by experts, decreases some strictness on questions, aims toward a high/increasing level of question volume. at least, if site participation is going down even as membership is going up, something is wrong with the current approach.

---

so what is on charter? any topics that the experts like to discuss, but possibly not expert level-questions on those topics. but yes, emphasis on expert level answers. all of which are discovered not a priori but in fact a posteriori through the voting system.

moreover to a large degree, and perhaps contrary to conventional wisdom, often the quality of a question cannot be judged immediately on seeing it. the quality of a question is discovered over time. especially with research & mathematics, questions initially that sound innocuous turn out to be deep, or questions that seem deep turn out to be innocuous.

unclosed questions, on significant TCS topics, lying around with no answers are harmless on this site, and serve as "search engine bait" for experts to find and answer/revisit later. note that votes can be changed over time on the same Q/A (ie one can change ones vote from up to down or down to up after time or revisions--key functionality there).

apparently mainly a large, healthy diversity of content on expert topics is what attracts experts & encourages participation. a dribble of new questions daily is not sufficient. its not interesting.

I say, lets figure out creative/colorful ways to grow question volume, which is the main driver of span/breadth/diversity of content, (to reverse the current dramatic decline even with increasing members!) without so much focus on supposed question quality (which is not as objectively measurable or perceivable as many here would imply, and maybe even an unattainable/illusory goal) and instead let the software sort it out via voting and tagging mechanisms.

why are many experts not posting much questions or answers on this site? there is also a pattern/tendency, imho, of high rep users mainly answering questions, but not asking many, and then perhaps even turning around and saying that question quality (by everyone else) is poor quality.

recall, low voted questions disappear from the main page quickly. its as if they dont even exist. so this stackexchange software is inherently designed to deal with high question volume & participation & may break down somewhat in its design with low question volume.

from some statistics the question volume is down to 50% of the initial beta period and the close ratio has doubled within a month even on the lower volume. so yeah, lets revisit the charter & figure why the overall formula doesnt seem to be working any more given new conditions (other CS beta site etc)