Response to comments by Kiran on IR

Replying to Formats of Rounds - II : Infinite Rebounds, Kiran said:

the reason why infinite rebounds is not that relied upon is pretty simple, its in the name itself. only if the number of questions are infinite or to be practical significantly large, can you apply this format. if there are not many questions (atleast 20 X no. of teams), some teams might feel high and dry in terms of number of questions put to them. that element of fairness is ensure in D&P. moreover, if one of your biggest arguments against D&P is the fact that one team cld end up answeing a direct imm. after a pass, thats not a strong enuff point. having been a veteran in quizzes, i wld still subscribe to D&P inspite of some amount of unfair situations.

Niranjan adds:

a simulation of infinite rebounds vs D&P (latest code - none of you have it yet..;-)) indicates that differences in team abilities result in a more pronounced variation in points obtained using IR as compared to D&P. this considers varying levels of question difficulty, team streaks and question-to-question streaks across a number of quizzes. though it does take a number of quizzes to establish this (as kiran says), an average of limited number of quizzes (say, 10?) is enough to substantiate this.

i think, the most important aspect in every quiz is the uniformity in difficulty level of the questions. but perhaps this is the most difficult thing to accomplish. more often than not more than one person are involved in making a quiz and they have to 'settle; for a varying level of difficulty. the question of fairness arises mainly due to this aspect and therefore needs addressing.

another sub-problem is to arrange teams such that the order does not give an unfair advantage to certain teams. am working on the same and will soon come up with something worth stating (so i hope).

Here's my response to these points (thanks for the different point of view, Kiran - really helped me examine my arguments, of course I still don't agree with you :-) ):

• Only if the number of questions are infinite or to be practical significantly large, can you apply this format.if there are not many questions (atleast 20 X no. of teams), some teams might feel high and dry in terms of number of questions put to them. that element of fairness is ensure in D&P.

  I personally don't think that given a quiz with a decent number of questions (>30), this is a valid point. I don't think the number need to be very large for IR to be fairer than D&P. The reason for this view is my measure of fairness is not the number of questions "directly" posed to a question (that is the number of first attempts for a team), but rather it is the number of questions a team gets to attempt, be it direct or passed. I don't see any difference in them, especially if the distribution of questions and consistency of level is assured. I feel that ideally speaking, one must be able to answer any question that is posed (of course this doesn't happen practically), and the framing & quality of the questions must not be a hindrance to getting the answer if I know it. In this regard, I don't see why we must distinguish between a direct attempt and a passed attempt. What fundamental difference lies between them? I am willing to concede that there is a psychological difference only, that we feel we have a better chance with Directs than with Passed question. I feel that's only a mental block. Getting a question direct does not greatly improve my chances of getting a question suited to me, nor does a passed question inhibit my chances in any way especially, if I'm the only person among all who can get it. Consequently, I feel that the number of attempts is a better indicator of fairness between formats. If I can say that all teams have received an equal number of attempts (of course, not at the same questions, as we know the ideal case that all teams attempt all questions and the winner is one that gets most right is not a practical one in our stage finals conduct), then that format is fine.

• moreover, if one of your biggest arguments against D&P is the fact that one team cld end up answeing a direct imm. after a pass, thats not a strong enuff point.

  I don't consider that to be the biggest argument. There is no single argument against D&P, it is the combination of the "different values for different attempts", "fixed starting points of questions", "teams getting direct + pass right" etc. Teams may answer more questions than anyone else and still lose. Also the number of attempts at questions can be hampered by the fact that the origin of the next question is always fixed and potentially against the teams later in the order. A theoretical example:

  Teams A to F. Teams E & F are the best of the six and can usually answer any question put to them. Teams A to D will never correctly answer a question posed to them. In D&P, in a round, Team F will answer 5 questions, 4 passed & 1 direct. Team F (presumably furious by now :-)) get their direct right. Number of attempts per team is heavily tilted in favour of E.
  In IR, both E & F will end up getting equal number of attempts and points - a fair reflection of the strengths.

  Of course, a counter-argument to the above (thanks Samrat ) is that if rounds are reversed after half the quiz, F gets the same advantage and things balance out. But the fact remains that in a round, there is a skew. Now assume that (again thanks to Samrat ;-) ) each round was a theme based round and that F was better at Sports than E (in such a round, E would get lesser questions right than F given the same number of attempts). If that round was from A to F, then F would lose out. (Hope this makes sense :-) ).

  The point is that number of attempts should be similar for all teams. Clustering of attempts around a team is to be avoided. I think D&P will result in different numbers of attempts, perhaps not everytime, but atleast sometimes. I also suspect that IR may do better in this respect. The format should not inherently breakdown in situations where one team gets significantly more chances than others. I agree with Niranjan that distribution and consistency is the key aspect, but I feel we can concentrate on it once we have assured ourselves that the underlying framework will not distort that distribution, or cause its own problems.

  During the last few quizzes, I have tried to collect some numbers on the questions in the finals. What I did was to note for question the team it was directed to and the team that got it right. Using this data, I calculated the number of attempts per team. The quizzes were my Pulse quiz, the VIT quiz and the Brand Equity Quiz. I couldn't manage this at B.C Joshi and I left this task incomplete at the Chakravyuuh quiz for there was so much else going on to make it a valid example to use :-). I'll be putting these stats in a separate posts. I think it is too early to point out any inferences, but there are some interesting trends in them.

• I thought of a new format that could possibly marry the ideas of D&P and IR - a separate post for that (Hanc marginis exiguitas non caperet! ;-)).