Showing posts with label theory. Show all posts
Showing posts with label theory. Show all posts

Sunday, May 16, 2010

Private rumblings from the Pune Brand Equity 2010 quiz

Niranjan and I teamed up for the Pune round of Brand Equity 2010 quiz where we finished 3rd. Neither of us is any good at biz-quizzing, and so admittedly, a mercenary attitude fueled the decision to travel the entire diameter of Pune. Despite the middle-level placing (not bad, given the presence and three-blade-close-shave elimination of some very good teams), we can't help a couple of rants.

On the difference between data mining and data compression

Or, to compress the question, the various differences between data compression. I hate having to answering such school-type questions1, but I guess we're always ready to pawn our soul for 10 points :-). So we defined it as being techniques "for reducing the size of data for transmission/storage etc" (paraphrasing), but were ruled wrong. I'm not sure what the correct answer was, but I think it was to the effect of "extracting only that info from data that is needed" (memory fails me, as I was still reeling from the surprise). Without going deeply into data compression, there are different kinds: lossy (like in MP3s where you say that some data/info is unimportant enough to be thrown away) or lossless (such as in file compression, where you have a different, smaller form of your data to cheaply store or transmit). I'm not sure what was needed here.

The quizmaster did assert that he knew what he was looking for, he knew it would be one of this tricky ones, and he had sources who helped him "nail this one". He also did apologise a couple of times later, saying that he knew it was one of those borderline ones (i.e. for the inherent ambiguity, not that he was wrong about it), which was kind of him, I suppose. However:

  • It's a terrible kind of question to ask - school homework-ish at best. Neither was it entertaining.
  • If a quiz-setter knows a question/answer is inherently vague or prone to misinterpretation, then he'd be better off not asking it
  • If it's not a domain you are familiar with, then you are dependent on hearing keywords in such answers - not a good idea
  • Don't ask for 'layman' interpretations of technical questions - my layman lives in Shivajinagar, yours may live in Salt Lake.
On yet another case study of the demerits of Direct-and-Pass

I don't have much of a problem with big quizzes being designed to appeal to non-quizzing audiences, but why must they mess with the fairness of the rounds? The use of D&P over Infinite Rebounds (Infinite Bounce to non-Pune quizzers) is a menace that everyone in the game knows about. Most (and all good) quizzes use the IR system for their regular rounds because it guarantees one thing : between two shots at questions for any team, all other teams will have had a chance to attempt a question. IR does not smooth the fairness of the content or the fortune of receiving an easy question. But unlike D&P, you can be assured that your team will not twiddle its thumbs while the rest get to answer. Think of it as all teams receiving (almost) the same number of deliveries to bat to, at any given interval in time.

Yet another illustration of this from the other day: after 2 rounds of passing (one forward, one reverse), the number of attempts (out of 12 questions): two: A, B, F; three: E; five: C, D. Five vs. Two - that kind of imbalance would never happen in IR.

(This is no reflection on the qualities of the other teams, esp. the winners - just a commentary on why a quiz format shouldn't be so uneven. Let luck of the draw play its role in the content and team placement, but not the number of attempts.)

So why do big quizzes still employ an outdated and inferior system? Makes no sense at all.


1. read up for the future: the difference between ink pens and fountain pens; between MP3s and MP4s; between first slip and third slip (about a feet?)

Sunday, December 24, 2006

The Mod Mod Infinite Rebounds

Ages ago, I'd written a post on Infinite Rebounds as part of a series discussing scoring formats in quizzes. That post has a couple of errors which came to notice in time, and is slightly obsolete now. This current post aims to correct those errors and record the current set of conventions that are followed in IR (at least in the BC circles). This post is also partly aimed at perhaps reaching a consensus on how we should run this method here - it is after all the most popular form of conducting a quiz in these parts - so as to avoid the usual raised eyebrows just before the finals of a quiz begin when the quizmaster announces his nth variation.

Infinite Rebounds a.k.a. Infinite Bounds (IR) is one of the algorithms to answer the timeless quizzing question of "who gets the next question?". One answer is in these commonly agreed conventions is IR:

(Some terms and abbreviations:
* Q{n-1} : the previous question which has been completed
* Q{n} : the next question which is to be asked
* S: the team that successfully answered Q{n-1}
* P: the team that started Q{n-1}
* Normal passing direction: The usual passing order that a quiz begins with i.e. A->B->...->F->A...
* Reverse: F->E->...->A->F...
* S-1(P-1): the team seated "before" S(P) in the Normal passing direction
* S+1(P+1): the team seated "after" S(P) in the Normal passing direction
)


Rule 1. Q{n} after Q{n-1} was correctly answered: The choice of the team to receive Q{n} depends on how Q{n-1} was answered. If Q{n-1} was answered correctly by a team S, Q{n} goes to S+1.


Analysis:
* IR attempts to keep things "fair" for all teams by keeping the number of attempts on questions as equal as possible. "Equality" not only over the entire length of a quiz, but also over short windows of time. In IR, this means that at any point, no team will have an unfairly greater number of attempts at questions than any other team.
* We know that the fairest possible system is for all teams to simultaneously attempt all questions. But this is not possible for all questions in a stage-final-format, and so every algorithm needs to address where a question starts and how it passes. Here 'fairness' means for teams to at least attempt almost, if not exactly, the same number of questions. For this, one needs to also convince oneself that theoretically there is no difference between attempting a question on the direct or on a pass.
* In earlier systems like Direct-and-Pass, some teams could enjoy several more attempts than others in a single round. Perhaps over the length of the quiz, the teams may have similar number of attempts, but in a local window, this will not be true.
* A Team is not penalised for sitting next to a "good" team that gobbles up all questions before they can get to it.
* IR always increments the next question marker, ensuring that no team can get an attempt again unless all teams after it have had one attempt since.
* We can state the fundamental principle of IR as: "keep the number of attempts equal". Earlier, we thought it would make sense to re-start next to the team that last got points. But on reflection, this is not quite correct.

Rule 2. Q{n} after Q{n-1} was not correctly answered:
(Before modification) If Q{n-1} went through unanswered, Q{n} will go to P+1 i.e. the team next to the one that started the previous question.
(Modification 1) If Q{n-1} went unanswered, Q{n} goes back to P i.e. the same team with which the previous question started.


Analysis:
* Earlier, it was felt that no team should start a question twice in a row. However, if we want to keep the number of attempts equal, then here the old rule violated the principle. This is because the team that started the last question is now pushed to the end of the new cycle. Unless the question is unanswered by the rest, this team loses an attempt while the others gain one.
* Applying the modification ensures that at any given point in the quiz, no team can have more than 1 additional attempts over any other team in the final. (Having "1 more" means that the other teams will attempt questions to square the count before this team can increment its count.) In the previous system, this could not be ensured. In fact, data shows that a difference of 2 or more is often seen with the older system.
* This modification is as simple to run, and more importantly, it does not penalise a team just because the quiz-setter asked a tough question that none of the teams could answer.
Rule 3. Next question after partial points:
(Before modification) Sometimes a question is made up of multiple sub-answers. Suppose such a question Q{n-1} is attempted by all teams, some of which who give partial correct answers. At the end, the quiz-master awards partial points to those teams with correct contributions. In this case, who gets Q{n}? Previously, the question would be awarded to the team next in passing order to the last team to get points.
(Modification 2) Here, since all teams have attempted the previous question, the question heads back again to P.
(There is no consensus on this point)


Analysis:
* Again, the previous system believed in re-starting next to the team that last got points. Instead, if we appeal to the spirit of the principle of equal attempts, Q{n} should go back to P, as everyone's had an attempt on the previous question.
Rule 4. Next question after round reversal: A question is asked and completed. The order of passing now reverses. Who gets the next question? If modification #1 is implemented and if the last question was unanswered, then Q{n} goes back to P. This is a trivial case. In all the other cases, there have been several variations here and no consensus has really emerged. The variations:

* Q{n} is given to S-1 (or P+1 in case no modification is applied) and the passing is in Reverse
* Q{n} is given to S+1 and the passing then Reverses
* Breaking with IR here, Q{n} starts at the other extreme i.e. Team F (assuming the previous half started with Team A)


Analysis:
* It seems to me that each of the above rules have their pros and cons, and hence their respective advocates. None of these seem to be able to give any firm guarantees, so I think this much dissimilarity in number of attempts can be tolerated. Perhaps the best way is to toss a coin and pick the starting team in each segment randomly. Random is better than Bad :-)

The rest of the 'philosophical' aspects can be read in the previous posts. I hope to bring out the emphasis on the "equality in number of attempts" to help guide all minor quibbles. Of course, there is no intention of imposing some kind of standard, but perhaps leading to a convention that say most open quizzes in Pune and even Bombay can adopt. My personal basket of conventions would be to include both the modifications on points 2 and 3. Hopefully, quizmasters will think clearly about the options available to them instead of making an arbitrary and dogmatic choice. Of course, there are those who say IR is dogma :-), but it's the best we've got at the moment, is it not?

Kindly comment on issues.

P.S.: I remarked that "theoretically, there is no difference between attempting a question on the direct or on a pass". This is not quite true. In fact, practically, there is a difference between all teams when they attempt. The earlier you attempt a question, you have less to work with from answers previously given, but you also benefit from lack of "Forward Bias" (example embedded in this post). But overall, does it matter? I think not.

P.P.S: Apologies for the use of short-forms etc. etc. - all in the name of brevity (esp. if you consider the size of this post!).

P.^3.S: Special mention of Arnold D'Souza (as promised) for independently re-discovering modification #1 in Pune circles :-)

Sunday, February 15, 2004

Formats of Rounds - II

Update (25 Dec 2006): See this post for an update on this subject. Some obsolete/incorrect portions have been struck off. In continuation to Formats of Rounds - I

Infinite Rebounds

Since Direct and Passing doesn't work in its usual form for reasons noted before, it has quickly been replaced by the Infinite Rebounds system as the de facto quizzing format in all the good quizzes. A caveat here is that of the qualifier "good quizzes", as a great many "popular" quizzes aren't aware of this system, or if they are, choose to shy away from employing it - an exploration later.

The commonly agreed conventions of Infinite Rebounds:

  1. The choice of the team to receive the next question depends on how the previous question was answered. If the previous question was answered by a team, the next one is put to the team next in order from it. Alternately, if it went unanswered, the new question is simply advanced to the team next to the one that first received the last question.

    As an example, consider six teams A to F. Say a question starts with Team B. Turns are taken in order to attempt answers and Team E eventually answers correctly. In this case Team F would receive the next question. In case the question was left unanswered, Team C would receive first crack at the next question.

  2. No question may be posed first to the same team twice in a row. (More elaboration on this later in this post.)

  3. The beginning of a new round does not require resetting of the asking order i.e. unlike in the D&P method, each round does not start with the same team or does not have a predecided beginning point. An element of randomness appears in this system as it is not evident who would have answered the last question of the preceding round. This prevents a great deal of rigging (nothing can prevent quizmasters "choosing" their questions, of course, but it hampers rigging when the sequence of questions is already embedded in a visual presentation as is usually the case these days).

  4. Each question has the same value for the correct answer.

  5. No question in the rounds with using the Infinite Rebounds system has penalties i.e. no negative scoring for wrong answers.

According to me, the whole philosophy of Infinite Rebounds can be summed up with the sentence: Be as fair as possible to the participants. That is really the crux of the structure. If another system which is even better at being fair is possible, I don't have any doubts that it will replace Infinite Rebounds as we know it today.

I'd like to spend some more time further examining the implications of the above conventions. The point about the equal value and penalties aren't really a fundamental part of the structure of Infinite Rebounds, which is after all a set of rules for passing. However, they do conform to the spirit behind encouraging the use of this collection of rules - so they've been bundled up under the same banner. If someone so desired, a system of different values for direct and for passed questions may be employed, or points subtracted for guessing wildly thus inhibiting the choice to dare an answer, but that would bring what I like to think of as Instant Recoil (it can be seen on the faces of experienced quizzers on stage and looks as if they've swallowed a raccoon, and by the immediate broadcast of this feeling to fellow quizzers around).

The first rule presents the main algorithm of the format. It gets rid of one of the fundamental problems of the D&P method where a team could get a passed question correct and then a direct one for itself, boosting its tally without other teams having had a say in proceedings. Infinite Rebounds puts the team with the last correct answer at the end of the queue, forcing the movement of the "first attempt marker" to constantly circulate (the "first attempt marker" indicates how teams got first crack at a question). Additionally, this implicitly provides an element of randomness that is very useful in preventing many of the known methods of "black magic" that happens at quizzes :-). Teams also need not be very glum about having drawn seats next to so-called "studs" that in the D&P method would've eaten up much of the food supply of questions. Instead, such a team may be rewarded for its position, as it could potentially get many more first shots at questions. This one fact is enough to serve as an endorsement of this format over the conventional system. The clinching factor is that on most occasions, the team that has given the most correct answers wins. Of course, the number of correct answers is a factor of the number of attempts a team receives. There are cases where among two equally balanced teams slug it out and one team sneaks through because it got more questions to answer than its principal rival, but both teams will agree that it wasn't because the system did them in - the winners still had to provide more correct answers than the runners-up. Contrast this with the D&P team where quite often in a tight contest, the winning team could've had lesser attempts, lesser number of correct answers and still win.

The second point in the list, the one about not giving first attempt of a question to the same team twice in a row, requires some elucidation. Such a case may not be readily apparent. This can happen when (taking an example) say, Team B gets the next question in a round. They can't answer it and it passes normally till Team A (the last team in line for that question) answers it correctly. Now, by the normal passing rule which simply states that the team next in order to the one that got the last answer gets the new one, Team B would again gets a chance at the question. Since much of Infinite Rebounds is in protest of the injustices of the D&P method where this occurence is common and hated, it makes sense to prevent this as a special case and advance the first attempt marker to Team C. This would ensure that things aren't stuck at Team B's end.

Even if some do not agree with this point, it does make sense if one considers the spirit of the Infinite Rebounds philosophy. I speak from personal experience. In 2002, Swapnil did a quiz (called "NQuirer"?) at the BC. We got into this exact situation, and my team was in the position of Team B and would've benefited as we were not doing very well and needed every question we could get. Since this was quite a new situation (for some reason, no one at the BC had ever made a special note of this scenario), it wasn't quite clear who would get the next question, us or the team next to us. Many of the "learned figures", Harish and Niranjan quickly spotted the solution (the obvious one) i.e. we shouldn't get the chance. We (or rather me, since I was the senior guy in my team, so musn't pull Anup Mankar or Amrish into this), in a fit of silly gamesmanship, insisted that there was no rule or precedence governing this, and we should get it. I don't recall what was decided (most likely, it went against us by consensus, as I usually don't get my way at the BC ;-) ), but it did throw up a case for which we had no convention to govern. I will be the first to admit I was wrong that day and to avoid this situation in future, I like to make a special mention of it. That's why all these words are spent on it, so as to avoid acrimony in the use of such a splendid system as this.

For some incomprehensible reason, Infinite Rebounds isn't yet the universal standard for the conduct of general rounds of quizzes ("general" as in normal passing rounds that don't have any special funk such as Rapid Fire or Single Specialities etc.). It is inexplicable that even veteran quizhosts hang on to the D&P method for dear life. Either they are ignorant about it, or do not have the intellect to comprehend it (which must make it a pea-sized intellect), or they must be resisting its use. I don't claim to say that I can conclusively prove it is a fairer system than any other, but experience, personal and among the fraternity is overwhelmingly positive about it, or atleast in rejecting the conventional system. I also suspect some of the ancient quizhosts perceive a sense of having to give up their dignity and adopt a newer system; perhaps it would undermine all their previous quizzes. What else would explain one quiz organizer (a veteran quizzer himself) completely rejecting Infinite Rebounds and describing it as (paraphrased) "nonsense"? Unless he knew something we didn't. Would you hang on to a fast-eroding prestige by being stubborn or be fair to the participants? I think a great deal of quizzing inconsistencies are because these reputed quizmasters don't spend enough time thinking about their conduct of their quizzes and prefer keeping the show audience-centric rather than participant-centric. I don't buy the argument that Infinite Rebounds is hard to conduct; in fact it makes life easier. The usual points of confusion for someone conducting a quiz are how much does the team that just got the correct answer get in terms of points and who gets the next question. Both these are easy to answer in this format: everyone gets the same number of points each time and one just has to remember the last correct answer and/or the last starting point. As in many other walks of life, simple is elegant.

In an effort to try comparing and evaluating the efficacy of the two popular methods, Niranjan wrote a small simulation whose source code (in C) he posted on Inquizitive - apparently he's improved it since. It would make even more sense to run it with actual data from live quizzes rather than just test data, so some of us are trying to record actual sequences - I managed to do so from the last quiz I conducted, an internal office event called Pulse. I still haven't run the data against the sim. If anyone's interested in the code, just leave a comment behind here.

To summarise, I think since Infinite Rebounds is both widely accepted and perceived to be fair, its use must be encouraged all around. If somehow it can be introduced into the school quizzing circuit, it would expose kids to that at an earlier stage. Otherwise, it remains something of an elitist choice, but one that definitely exposes a badly organized quiz from the others. Of course, the most ideally fair way to hold a quiz is to allow everyone to attempt the same questions simultaneously and then the one who gets the most correct answers wins. This is obviously not possible in a "final-on-stage" situation. We can only get algorithms that approach that optimal solution. Infinite Rebounds is definitely here to stay as one such solution that is an excellent substitute for the practical conduct of a quiz.

Comments/Criticisms/Commendations/Contributions welcome as usual