tag:blogger.com,1999:blog-9538596226415593632024-03-08T01:48:23.949-08:00A Numbers GameOpen source debate statisticsA Numbers Gamehttp://www.blogger.com/profile/18412505663480533746noreply@blogger.comBlogger10125tag:blogger.com,1999:blog-953859622641559363.post-49941391027358866262009-11-08T13:48:00.000-08:002009-11-08T13:52:13.218-08:00RKS scale not dramatically increasing predictive accuracy<p>
If moving to the Ross K. Smith 100-point scale increases the accuracy of speaker points, and if speaker points accurately reflect a team's abilities, we might expect that the scale change would help speaker points predict future winners. To my surprise, there is not a dramatic increase in the predictive accuracy of speaker points in the three large tournaments that moved to the new scale while maintaining the same number of teams and preliminary rounds.
<p>
I used the following model: Predictions are made for rounds five and later, using at least the first four preliminary rounds. The model makes no prediction if the two teams have different win-loss records. If they have the same win-loss record and the same total points, the model also does not make a prediction. Otherwise, it predicts that whichever team had the higher points for the predicting rounds will win the debate.
<p>
If two teams have the same record for rounds one through four, but they don't meet until round eight, the model still predicts the winner based on their points in rounds one through four, since the teams might well have debated in round five. It also makes predictions based on their points in rounds one through five, one through six, and one through seven, assuming their win-loss records are the same after rounds five, six, and seven. All predictions are based on rounds one through four, five, six, or seven; no predictions ever discard any early prelims.
<h4>Wake Forest, 8 prelims</h4>
<table border="1">
<tr><th>Year <th>Teams <th>Tied points, no prediction <th>Correct prediction <th>Incorrect prediction <th>Accuracy
<tr><td>2006-2007 <td>138 <td>11 <td>89 <td>50 <td>63.0%
<tr><td>2007-2008 <td>134 <td>1 <td>101 <td>79 <td>56.1%
</table>
<h4>UNLV open division, 7 prelims</h4>
<table border="1">
<tr><th>Year <th>Teams <th>Tied points, no prediction <th>Correct prediction <th>Incorrect prediction <th>Accuracy
<tr><td>2008-2009 <td>56 <td>0 <td>32 <td>23 <td>58.2%
<tr><td>2009-2010 <td>54 <td>0 <td>20 <td>11 <td>64.5%
</table>
<h4>Harvard, 8 prelims</h4>
<table border="1">
<tr><th>Year <th>Teams <th>Tied points, no prediction <th>Correct prediction <th>Incorrect prediction <th>Accuracy
<tr><td>2008-2009 <td>80 <td>9 <td>57 <td>27 <td>66.1%
<tr><td>2009-2010 <td>87 <td>4 <td>68 <td>33 <td>66.7%
</table>A Numbers Gamehttp://www.blogger.com/profile/18412505663480533746noreply@blogger.com3tag:blogger.com,1999:blog-953859622641559363.post-50262507334106723652009-11-04T14:08:00.000-08:002009-11-05T13:18:34.149-08:00RKS translation at Harvard<p>
The Harvard tournament this year had a <a href="http://cedadebate.org/forum/harvard/how-the-tournament-will-be-run-paired-etc/">very explicit and very simple translation</a> from the 30-point scale to the 100-point (or Ross K. Smith) scale. Using the <a href="http://www.planetdebate.com/files/view/3087">results packet</a>, we can see how the judging pool interpreted the RKS scale:
<p>
<img src="http://chart.apis.google.com/chart?cht=lxy&chs=430x500&chtt=Harvard+cumulative+point+distributions&chf=c,s,000000&chg=10,12.5,1,0&chxt=x,y,r&chxr=1,60,100,5|2,26,30,.5&chxl=0:|0%25|10%25|20%25|30%25|40%25|50%25|60%25|70%25|80%25|90%25|100%25&chdl=2003|2004|2005|2006|2007|2008|2009&chco=ff0000,ff8000,ffff00,00ff00,00ffff,0000ff,ff00ff&chds=0,100,60,100,0,100,60,100,0,100,60,100,0,100,60,100,0,100,60,100,0,100,60,100,0,100,60,100&chd=t:0.9,4.5,7.7,24.8,49,79.7,96,99.6,100|57.5,62.5,67.5,72.5,77.5,82.5,87.5,92.5,97.5|0.8,2.6,15.2,42.9,75.9,94.6,99,99.9,100|62.5,67.5,72.5,77.5,82.5,87.5,92.5,97.5,100|1,3.3,5.7,17.8,39.4,77.5,95,99.4,100|57.5,62.5,67.5,72.5,77.5,82.5,87.5,92.5,97.5|0.1,1.2,2.3,11.4,34.3,73,94.7,99.6,100|52.5,62.5,67.5,72.5,77.5,82.5,87.5,92.5,97.5|0.4,0.9,2.1,14.9,40.1,75.5,94.6,99.5,99.9,100|57.5,62.5,67.5,72.5,77.5,82.5,87.5,92.5,97.5,100|1,3,14.1,37.4,69.8,94.4,99.8,100|62.5,67.5,72.5,77.5,82.5,87.5,92.5,97.5
|0,0,0,0,0,0,0.4,0.4,0.7,0.7,0.9,2.8,3.7,5.4,6.6,8.7,14.3,16.2,20.6,24.9,29.9,40.5,45.2,50.8,56.9,63.6,72.2,77.7,82.7,87.9,90.8,94.5,96,96.9,97.8,98.4,99.6,99.8,100,100,100,100
|59.5,60.5,61.5,62.5,63.5,64.5,65.5,66.5,67.5,68.5,69.5,70.5,71.5,72.5,73.5,74.5,75.5,76.5,77.5,78.5,79.5,80.5,81.5,82.5,83.5,84.5,85.5,86.5,87.5,88.5,89.5,90.5,91.5,92.5,93.5,94.5,95.5,96.5,97.5,98.5,99.5,100">
<p>
If we apply the Harvard translation to historical tournaments, the switch to RKS this year was accompanied by more point inflation than usual. The older data (drawn from <a href="http://debateresults.com/">debateresults.com</a>) shows that a median speaker used to earn between 27.8 and 28.0 points (78 to 80 points, under RKS). This year, he or she earned 28.25 points (82.5, under RKS). A 28.0 (80, under RKS) used to mean 55th to 65th percentile. This year, it meant about 35th percentile.
<p>
It will be interesting to see how many judges <a href="http://cedadebate.org/forum/harvard/more-on-the-100-(99)-point-scale/">circled the indicator</a> on the front of their ballots indicating that they were conforming to the suggested scale translation.
<p>
It should also be noted that Harvard's points this year were substantially lower than <a href="http://blog.anumbersgame.net/2009/10/ross-k-smith-scale-and-standards.html">Kentucky's points this year</a>:
<p>
<img src="http://chart.apis.google.com/chart?cht=lxy&chs=430x400&chd=t:0,0,0.1,0.4,0.9,1.2,1.5,2.4,3.7,4.7,6.3,8.1,11,14,16.4,20.5,25.6,33.3,42.3,51,60.8,70,78.7,85.1,89.2,93.2,96,97.7,98.6,99.3,99.8,99.9|64,65,69,70,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99|0,0,0,0,0,0,0.4,0.4,0.7,0.7,0.9,2.8,3.7,5.4,6.6,8.7,14.3,16.2,20.6,24.9,29.9,40.5,45.2,50.8,56.9,63.6,72.2,77.7,82.7,87.9,90.8,94.5,96,96.9,97.8,98.4,99.6,99.8,100,100,100,100|59.5,60.5,61.5,62.5,63.5,64.5,65.5,66.5,67.5,68.5,69.5,70.5,71.5,72.5,73.5,74.5,75.5,76.5,77.5,78.5,79.5,80.5,81.5,82.5,83.5,84.5,85.5,86.5,87.5,88.5,89.5,90.5,91.5,92.5,93.5,94.5,95.5,96.5,97.5,98.5,99.5,100&chds=0,100,60,100,0,100,60,100&chxt=y,x&chxr=0,60,100,5&chxl=1:|0%25|10%25|20%25|30%25|40%25|50%25|60%25|70%25|80%25|90%25|100%25&chco=0000ff,ff0000&chg=10,12.5,4,0,0,0&chtt=Cumulative+point+distributions,+2009-2010&chdl=Kentucky|Harvard&chf=c,s,000000">A Numbers Gamehttp://www.blogger.com/profile/18412505663480533746noreply@blogger.com0tag:blogger.com,1999:blog-953859622641559363.post-38808080680744641012009-10-11T20:44:00.000-07:002009-10-13T16:41:04.077-07:00The Ross K. Smith scale and standards(<a href="http://code.google.com/p/anumbersgame/wiki/SpeakerPointScale">Click here for more on speaker point scale changes</a>)
<h2>Judge scale variation</h2>
<p>
In a <a href="http://www.ndtceda.com/pipermail/edebate/2009-October/079837.html">post on edebate</a> (<a href="http://www.mail-archive.com/edebate@www.ndtceda.com/msg09994.html">mirror</a>, <a href="http://article.gmane.org/gmane.education.region.usa.edebate/10353">mirror</a>), Brian DeLong suggests that tournaments adopting the 100-point (RKS) scale provide an interpretation of the scale to judges to make sure points are allocated fairly. His motivation is based partially on a discrepancy he saw between judges in the <a href="http://article.gmane.org/gmane.education.region.usa.edebate/10334">Kentucky results from this year</a> (<a href="http://www.ndtceda.com/pipermail/edebate/2009-October/079818.html">mirror</a>): "Some people are on this 87=average boat, while others place average at around 78-80ish". Here is a chart of the average points given out by judges at Kentucky this year:
<p align="center">
<img src="
http://chart.apis.google.com/chart
?cht=lc
&chs=430x300
&chd=t:1.65,2.48,2.48,4.13,2.48,2.48,8.26,10.74,11.57,14.88,15.7,13.22,4.96,1.65,1.65,1.65,0,0
&chds=0,20
&chxt=x,y
&chxr=0,78,95,1
&chxl=1:|0%|5%|10%|15%|20%
&chm=B,ff7777,0,0,0
&chco=ff0000
&chg=29.41,25,4,3,11.76,0
&chtt=Average+points+by+judges|Kentucky+'09-'10
"/>
<p>
Some of these discrepancies could be between judges who simply saw teams of different abilities. One way correct for this is to compare the points given by each judge to the points given to the same competitors by other judges. From this, we can see how much a judge skews high or low. Applying this skew to the actual average points (86.37), we get an estimate of each judge's perception of the average:
<p align="center">
<img src="
http://chart.apis.google.com/chart
?cht=lc
&chs=430x300
&chd=t:1.65,0,3.31,3.31,2.48,2.48,4.96,7.44,8.26,13.22,17.36,10.74,7.44,7.44,7.44,0.83,1.65,0
|1.65,2.48,2.48,4.13,2.48,2.48,8.26,10.74,11.57,14.88,15.7,13.22,4.96,1.65,1.65,1.65,0,0
&chds=0,20
&chxt=x,y
&chxr=0,78,95,1
&chxl=1:|0%|5%|10%|15%|20%
&chm=B,7777ff,0,0,0
&chco=0000ff,ff0000
&chg=29.41,25,4,3,11.76,0
&chtt=Estimated+average+points+by+judges|Kentucky+'09-'10
"/>
<p>
As in the first chart, the red line show the actual average point distribution; the blue line shows the distribution of estimates of judge perceptions of the average. To get a feel for how the estimate works, here are two examples:
<ul>
<li>Alex Lamballe gave an average of 90.83 points, but other judges judging the same debaters gave them an average of 83.00 points. His skew is 7.83 points, so we estimate that he perceives the average points to be 7.83 points higher than the true average of 86.37. His estimated perceived average is 94.20.
<li>Justin Kirk gave an average of 79.50 points, but other judges judging the same debaters gave them an average of 90.00 points. His skew is -10.50 points, so we estimate that he perceives the average points to be 10.50 points lower than the true average of 86.37. His estimated perceived average is 75.87.
</ul>
<p>
From these two extreme examples, it should be clear that this method of estimating judge-perceived averages is quite inexact. I think it is mostly useful as a check to ensure that the point skew in the first graph is not solely due to the differing quality of teams seen by different judges. Clearly, there is some variation in what judges think "average" is.
<p>
But how can we check if Kentucky showed more variation in judge point scale than other tournaments? One way is to measure judge scale variation with a test that has a scale-invariant expected distribution. The Mann-Whitney U test is approximately normal for reasonably-sized samples, so we can use that to find the Z-score for each judge at a tournament. The larger the variance of judge Z-scores, the more variation there is in judge point scale. (As above, the two samples are the points given by a certain judge and the points given to the same debaters by the rest of the pool.)
<p align="center">
<img src="
http://chart.apis.google.com/chart
?cht=bvs
&chbh=a
&chs=430x300
&chd=t:1.54,1.82,1.67,1.77,1.78,1.54,0|0,0,0,0,0,0,2.30
&chds=1,3
&chxt=x,y
&chxr=1,1,3,0.5
&chxl=0:|2003|2004|2005|2006|2007|2008|2009
&chg=0,12.5,4,3,0,0
&chco=ff8000,000090
&chtt=Judge+variance+at+Kentucky
&chdl=30+points|100+points"/>
<p>
The 2009 tournament showed more judge scale variance than any of the other Kentucky tournaments in the debateresults.com database. If we compare the distribution of judge scale Z-scores from 2009 to the combined distribution of judge scale Z-scores from Kentucky in 30-point-scale years:
<p align="center">
<img src="
http://chart.apis.google.com/chart
?cht=lc
&chs=430x300
&chd=t:0,0.2,0.3,2.1,4.6,16,23.6,23.4,14.6,9.3,4.3,1.1,0.3,0.2
|0,2.5,1.6,5.7,5.7,13.1,16.4,17.2,13.9,12.3,5.7,4.1,1.6,0
&chds=0,25
&chxt=x,y
&chxr=0,-6.5,6.5
&chxl=1:|0%|5%|10%|15%|20%|25%
&chco=0000ff,ff0000
&chg=15.38,0,4,3,3.85,0
&chtt=Judge+variance+at+Kentucky
&chdl=30+points|100+points
"/>
<p>
There is clearly more judge scale variation under the 100-point scale.
<p>
The Wake Forest tournament was the first tournament to change to the 100 point scale, in 2007. There was no corresponding jump in judge scale variance at Wake that year, but Wake <a href="http://groups.wfu.edu/debate/DixieClassic/Shirley%20Information%202007.pdf">provided a reference scale with extensive documentation</a>.
<p align="center">
<img src="
http://chart.apis.google.com/chart
?cht=bvs
&chbh=a
&chs=430x300
&chd=t:1.82,1.52,1.77,1.59,0,0|0,0,0,0,1.61,1.72
&chds=1,3
&chxt=x,y
&chxr=1,1,3,0.5
&chxl=0:|2003|2004|2005|2006|2007|2008
&chg=0,12.5,4,3,0,0
&chco=ff8000,000090
&chtt=Judge+variance+at+Wake
&chdl=30+points|100+points"/>
<h2>Reference point scales</h2>
<p>
DeLong also suggested a translation of the 30-point scale to the 100-point scale. The Kentucky judges from 2008 and 2009 also implicitly suggest a translation between the scales by the distribution of their votes. For instance, a 27 was the at 10th percentile of points given in 2008; this value is closest to an 80 in 2009, since 80 was at the 11th percentile in 2009.
<p>
The chart below compares DeLong's translation with the implicit translation by the Kentucky and Wake judges. It also charts the translations <a href="http://www.ndtceda.com/pipermail/edebate/2009-September/079662.html">proposed by Michael Hester</a> (<a href="http://article.gmane.org/gmane.education.region.usa.edebate/10178">mirror</a>, <a href="http://www.mail-archive.com/edebate@www.ndtceda.com/msg09829.html">mirror</a>) and <a href="http://www.the3nr.com/2009/10/12/the-100-speaker-point-system/trackback/">Roy Levkovitz</a> as well as a very simple "last digit" translation, calculated subtracting 20 from a score under the 30-point system and then multiplying by 10: 27.5 becomes 75, 29.0 becomes 90, and so on.
<!--
<table border="1">
<tr><th>old<th>DeLong<th>Kentucky 2008 & 2009 judges<th>last digits<th>Hester et al.<th>30pt percentile
<tr><td>30 <td>100<td> <td>100<td>100 <td>
<tr><td>29.6<td> 96<td> <td> 96<td> <td>
<tr><td>29.5<td> 95<td>98 <td> 95<td> <td>99.8
<tr><td>29.0<td> 90<td>94 <td> 90<td> <td>96.6
<tr><td>28.9<td> 89<td> <td> 89<td> <td>
<tr><td>28.5<td> 85<td>91 <td> 85<td>89.5<td>84.5
<tr><td>28.4<td> 84<td> <td> 84<td> <td>
<tr><td>28 <td> 79<td>88 <td> 80<td>82.5<td>58.7
<tr><td>27.9<td> 78<td> <td> 79<td> <td>
<tr><td>27.5<td> <td>84.5<td> 75<td>74.5<td>29.5
<tr><td>27 <td> 72<td>80 <td> 70<td>70 <td>10.1
<tr><td>26.9<td> 71<td> <td> 69<td> <td>
<tr><td>26.5<td> <td>75 <td> 65<td> <td>2.3
<tr><td>26.0<td> 60<td>72 <td> 60<td> <td>0.6
</table>
-->
<p align="center">
<img src="
http://chart.apis.google.com/chart
?cht=lxy
&chs=430x370
&chd=t:26,26.5,27,27.5,28,28.5,29,29.5,30
|60,65,70,75,80,85,90,95,100
|26,26.5,27,27.5,28,28.5,29,29.5
|72,75,80,84.5,88,91,94,98
|26,26.5,27,27.5,28,28.5,29,29.5,30
|70,72.5,77,83,88,92,95,96,98
|26,26.9,27,27.9,28,28.4,28.5,28.9,29,29.5,29.6,30
|60,71,72,78,79,84,85,89,90,95,96,100
|27,27.5,28,28.5,30
|70,74.5,82.5,89.5,100
|27.25,27.75,28.35,28.75,29.25,30
|69.5,79.5,87.5,93.5,97.5,100
&chds=26,30,60,100,26,30,60,100,26,30,60,100,26,30,60,100,26,30,60,100,26,30,60,100
&chxt=x,y
&chxr=0,26,30,0.5|1,60,100,5
&chco=dddddd,ff0000,6060ff,00ee00,ff00ff,ff8000
&chm=o,ffffff,0,-1,6,1
&chg=12.5,12.5,4,0,0,0
&chtt=Scale+conversion
&chdl=last+digit|KY+'08+and+'09|Wake+'06+and+'07|DeLong|Hester|Levkovitz
&chf=c,s,000000
"/>
<p>
The implicit translation induced by the Kentucky judging from 2008 is:
<table align="center" border="1">
<tr><th>30pt scale<th>100pt scale
<tr><td>29.5<td>98
<tr><td>29.0<td>94
<tr><td>28.5<td>91
<tr><td>28 <td>88
<tr><td>27.5<td>84.5
<tr><td>27 <td>80
<tr><td>26.5<td>75
<tr><td>26.0<td>72
</table>
<p>
The other translations (except Wake) are lower for most of the scale.
<p>
Another way to compare translations is to see how they would affect the cumulative point distributions. For example, at Kentucky in 2009, a 30th percentile performance would earn an 85. At Wake in 2007, a 70th percentile performance would earn a 90.
<p align="center">
<img src="
http://chart.apis.google.com/chart
?cht=lxy
&chs=430x400
&chd=t:1.2,3.3,16.9,42,75.4,93.5,99.6
|62.5,67.5,72.5,77.5,82.5,87.5,92.5,97.5
|0,0,0.1,0.4,0.9,1.2,1.5,2.4,3.7,4.7,6.3,8.1,11,14,16.4,20.5,25.6,33.3,42.3,51,60.8,70,78.7,85.1,89.2,93.2,96,97.7,98.6,99.3,99.8,99.9
|64,65,69,70,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99
|0,0.6,1.2,1.8,2.5,3.2,5.6,8.2,9.9,11.9,14.1,17.7,21.5,25.1,30.2,35.8,42.7,48.8,53.6,58.8,63.9,71.6,79.2,84.5,89.8,93.9,96.9,98.8,99.5,99.8
|69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98
|0.6,10.1,58.7,84.5,96.6,99.8
|60,72,79,85,90,95
|10.1,29.5,58.7,84.5,100
|70,74.5,82.5,89.5,100
|10.1,42,75.4,93.5,99.6,100
|69.5,79.5,87.5,93.5,97.5,100
|0.25,9.75,30.25,54.25,72.25,79.75,89.75,94.75,99.75
|70,74.5,79.5,83.5,87.5,89.5,92.5,94.5,98.5
&chds=0,100,60,100,0,100,60,100,0,100,60,100,0,100,60,100,0,100,60,100,0,100,60,100,0,100,60,100
&chxt=y,x
&chxr=0,60,100,5
&chxl=1:|0%|10%|20%|30%|40%|50%|60%|70%|80%|90%|100%
&chco=dddddd,ff0000,6060ff,00ee00,ff00ff,ff8000,ffff00
&chm=o,ffffff,0,-1,6,1
&chg=10,12.5,4,0,0,0
&chtt=Scale+cumulative+distributions
&chdl=last+digit|KY+2009|Wake+2007|DeLong|Hester|Levkovitz|Rubaie
&chf=c,s,000000
"/>
<p>
(Included in this chart is the <a href="http://judgephilosophies.wikispaces.com/page/diff/Rubaie%2C+Brian/94976536">point scale of Brian Rubaie</a>, which he gives in terms of position, rather than a translation from the 30-point scale.)
<p>
In the cumulative distribution chart, as in the translation chart, the "last digit" translation is close to the proposal of DeLong. The comparatively larger point range Hester gives to the 40%-85% area is alluded to in his proposal: "these two have large ranges b/c they are the areas i want to distinguish the most". Levkovitz has a similar reasoning:
<blockquote>
The difference to me between a 29.5 and 30 is quite negligible; both displayed either perfection or near perfection so their range should be smaller. But it is within what we call now 28.5, 28, and 27.5s that I want to have a larger range of points to choose from. The reasoning for this is that I want to be able to differentiate more between a solid 28 and a low 28, or a strong 28.5 and low 28.
</blockquote>
<!-- LocalWords: edebate DeLong Lamballe RKS
-->A Numbers Gamehttp://www.blogger.com/profile/18412505663480533746noreply@blogger.com2tag:blogger.com,1999:blog-953859622641559363.post-70458940108073988352009-09-11T19:06:00.000-07:002009-09-11T22:30:47.993-07:00Topic side biasEvery summer there is discussion about the side bias of prospective controversy areas and resolutions. Using the data from <a href="http://debateresults.com/">DebateResults.com</a>, we can see <a href="http://code.google.com/p/anumbersgame/wiki/TopicSideBias">what side bias past resolutions have exhibited</a>.
<p>
<a href="http://code.google.com/p/anumbersgame/wiki/TopicSideBias"><img src="http://chart.apis.google.com/chart?cht=lc&chs=400x300&chm=b,0000ffaa,0,1,1%7Cr,ff0000,0,0.665,0.669%7Co,000000,2,-1,5,1&chd=t:-4.3,-16.8,-12.2,-2.1,-14.5,-7.3%7C4.3,-8.2,-3.8,6.0,-6.2,0.4%7C-0.0,-12.4,-8.0,1.9,-10.3,-3.4&chds=-20,10,-20,10,-20,10&chxt=x,x,y,y,x&chxl=2:%7C%7C%7C10%25%20neg%7C%7Ceven%7C%7C10%25%20aff%7C0:%7C03-04%7C04-05%7C05-06%7C06-07%7C07-08%7C08-09%7C1:%7CEurope%7Cenergy%7CChina%7Ccourts%7Csecurity%7Cagriculture%7C4:%7C%7C%7C%7C%7Cguarantee%7C%7C3:%7C%7Cbias%7C&chco=000000,000000,ffff00&chtt=Topic+side+bias%7C90%25+confidence+interval&chg=0,33.33,4,3,0,0&chdlp=t&dummy=img.png" /></a>
<p>
Under the Bradley-Terry model, the energy, China, and security guarantee topics had small, yet highly statistically significant (<tt>p < .001</tt>), negative bias. The other three topics show no statistically significant bias either way.
<p>
The BT model suggests that the largest topic side bias in the past six years was 12.4%, on the energy topic. With a 12.4% neg bias, in an otherwise evenly matched round, the neg has a 52.9% chance of winning. For more details, see <a href="http://code.google.com/p/anumbersgame/wiki/TopicSideBias#Bradley-Terry">the description of how the bias is measured</a>.A Numbers Gamehttp://www.blogger.com/profile/18412505663480533746noreply@blogger.com1tag:blogger.com,1999:blog-953859622641559363.post-79132074895420148462009-07-28T16:17:00.000-07:002009-07-28T16:23:51.593-07:00Update on speaker point scalesIn response to <a href="http://www.cross-x.com/vb/showthread.php?p=1740435#post1740435">a question from cross-x.com</a>, I checked to see if the point distributions are any smoother for teams that might clear. <a href="http://code.google.com/p/anumbersgame/wiki/SpeakerPointScale#Point_Clustering_For_Winning_Teams">It turns out that they aren't</a>, except to the extent that the distributions have a narrower range:
<p>
<a href="http://code.google.com/p/anumbersgame/wiki/SpeakerPointScale#Point_Clustering_For_Winning_Teams"><p align="center">
<img src="http://chart.apis.google.com/chart?cht=lxy&chs=400x300&chd=t:68.000,70.000,72.000,73.000,74.000,75.000,76.000,77.000,78.000,79.000,80.000,81.000,82.000,83.000,84.000,85.000,86.000,87.000,88.000,89.000,90.000,91.000,92.000,93.000,94.000,95.000,96.000,97.000,98.000,99.000|2,10,1,4,8,32,16,19,27,18,70,25,45,78,66,112,68,81,93,78,82,43,23,12,2,5,4,1,2,5|75.000,76.000,77.000,79.000,80.000,81.000,82.000,83.000,84.000,85.000,86.000,87.000,88.000,89.000,90.000,91.000,92.000,93.000,94.000,95.000,96.000,97.000,98.000|2,1,1,1,2,3,4,10,8,28,18,24,51,40,97,53,66,64,23,26,10,5,3&chds=65,100,0,206.4,65,100,0,108.0&chco=ff0000,00cc00,aa44ff,ff7700&chtt=Wake%20%2708-%2709|speaker%20point%20distribution%20by%20percentage&chdl=losing%20record|winning%20record&chxt=x,y&chxr=0,65,100,5|1,0,20&chg=14.29,0,3,2&dum.png">
</p></a>A Numbers Gamehttp://www.blogger.com/profile/18412505663480533746noreply@blogger.com0tag:blogger.com,1999:blog-953859622641559363.post-69882112678658715722009-07-23T13:40:00.000-07:002009-07-23T13:53:20.573-07:00Changing the speaker point scale<a href="http://www.gsudebate.org/events_at_gsu/gsu_national_college_debate.html">The Georgia State tournament is moving to a 100-point speaker point scale next year</a>. This will make them the third large [<a href="#small-tournament-scale-change">N.B.</a>] tournament to use a non-standard scale. <a href="http://code.google.com/p/anumbersgame/wiki/SpeakerPointScale">I have put up some charts of how changing the scale affected point distributions at Wake and USC</a>.
<p>
<a href="http://code.google.com/p/anumbersgame/wiki/SpeakerPointScale#Point_Distribution"><p align="center"><img src="http://chart.apis.google.com/chart?cht=lxy&chs=400x350&chd=t:26.000,26.500,27.000,27.500,28.000,28.500,29.000,29.500|0.0070,0.0170,0.1450,0.2490,0.3460,0.1720,0.0550,0.0090|24.500,25.000,25.500,25.750,26.000,26.250,26.500,26.750,27.000,27.250,27.500,27.750,28.000,28.250,28.500,28.750,29.000,29.250,29.500|0.0009,0.0009,0.0018,0.0009,0.0054,0.0018,0.0161,0.0107,0.0688,0.0518,0.1518,0.0964,0.2089,0.1241,0.1366,0.0527,0.0482,0.0161,0.0063|26.000,26.300,26.500,26.600,26.700,26.800,26.900,27.000,27.100,27.200,27.300,27.400,27.500,27.600,27.700,27.800,27.900,28.000,28.100,28.200,28.300,28.400,28.500,28.600,28.700,28.800,28.900,29.000,29.100,29.200,29.300,29.400,29.500,29.800|0.0054,0.0023,0.0070,0.0008,0.0016,0.0062,0.0031,0.0559,0.0132,0.0264,0.0349,0.0179,0.0994,0.0179,0.0287,0.0621,0.0303,0.1545,0.0272,0.0396,0.0745,0.0287,0.0978,0.0256,0.0272,0.0264,0.0093,0.0411,0.0054,0.0062,0.0140,0.0023,0.0062,0.0008&chds=24.5,30,0,0.35,24.5,30,0,0.35,24.5,30,0,0.35&chco=ff0000,00cc00,aa44ff&chtt=USC%20%27Alan%20Nichols%27%20Debate%20Tournament|speaker%20point%20distribution%20by%20percentage&chdl=%2706-%2707|%2707-%2708|%2708-%2709&chxt=x,y&chxr=0,24.5,30|1,0,35&chg=9.09,0,3,2,9.09&chm=o,000000,0,-1,5&dum.png"></p></a>
<p>
As you might expect, the number of distinct speaker points per round and per speaker both increased. On the other hand, judges didn't use the whole range of points. At Wake (100-point scale), points clustered around multiples of 5. At USC (30-point scale, finer granularity), points clustered around the old half-point scale.
<p>
Are any other tournament directors planning on switching point scales? If you considered doing so, but decided not to, what stopped you? If you are switching, why did you choose the scale you chose?
<p>
<a name="small-tournament-scale-change">N.B.</a> Two other (smaller) tournaments changed speaker point granularity since '03-'04: the Weber State round robin used a granularity of 0.1 at their '08-'09 tournament, and the Northern Illinois tournament seems to have changed from the usual half-point granularity to a full-point granularity for their '05-'06 tournament.A Numbers Gamehttp://www.blogger.com/profile/18412505663480533746noreply@blogger.com2tag:blogger.com,1999:blog-953859622641559363.post-48882430303085537612009-05-13T13:27:00.000-07:002009-05-13T15:35:03.993-07:00CEDA Nationals vs NDT attendance<p>
<a href="http://repository.anumbersgame.net/wiki/Attendance">I have put up some charts about CEDA Nats vs. NDT attendance</a>.
<p>
The summary:
<ul>
<li>There was a sharp drop off in CEDA Nats attendance in 2009. Some coaches have suggested this was due to its placement in Pocatello, Idaho.
<li>From 2004-2008, 57% of CEDA Nats competitors debated almost exclusively in the open division at other tournaments the year in question. CEDA Nats 2009 had a novice breakout, but we won't be able to see if or how attendance changed until Bruschke's 2008-2009 database dump comes out. Even then, any analysis will have to contend with the decreased total attendance.
<li>Fewer than half of NDT teams skip CEDA Nats. There is very little correlation between NDT success and skipping CEDA.
</ul>
<p align="center">
<a href="http://repository.anumbersgame.net/wiki/Attendance#NDT_preference">
<img src="http://chart.apis.google.com/chart?cht=bvg&chbh=a&chm=D,,1,0,1&chbh=a&chs=400x300&chd=t1:2,3,1,1,1,1,1,1,2,3,2,1,1,2,3,2,3,3,2,2,0,1,1,0,2,1,3,2,2,3,2,1,3,2,2,3,1,2,0,2,0,1,3,2,1,0,2,2,3,1,2,2,1,2,1,3,2,1,1,0,2,0,1,2,3,2,2,1,1,2,1,2,2,2,0,2,1,2|-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.85,1.75,1.65,1.65,1.6,1.65,1.65,1.75,1.8,1.8,1.8,1.8,1.8,1.9,1.9,1.85,1.9,1.8,1.75,1.65,1.65,1.65,1.65,1.75,1.85,1.8,1.75,1.7,1.7,1.75,1.65,1.65,1.7,1.6,1.6,1.55,1.55,1.6,1.55,1.6,1.5,1.6,1.55,1.45,1.45,1.55,1.65,1.65,1.6,1.5,1.55,1.5,1.5,1.55,1.55,1.5,1.45,1.4,1.45,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0,-1.0&chds=0,3,0,3&chxt=x,y&chxl=1:||25%|50%|75%|100%|0:|worst|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||best|&chco=FFFFAA&chma=0,30,0,0&chg=200,50.1,1,0&dummy=img.png"/>
</a>
<p>
<a href="http://repository.anumbersgame.net/source/browse/trunk/tests/attendance/">The raw numbers and the code used to generate them are available.</a>A Numbers Gamehttp://www.blogger.com/profile/18412505663480533746noreply@blogger.com0tag:blogger.com,1999:blog-953859622641559363.post-34228787720982360242009-05-07T13:00:00.000-07:002009-05-07T13:12:32.907-07:00Hybrid teams<p>
<a href="http://www.ndtceda.com/pipermail/edebate/2009-May/078643.html">Liz Kim suggested that hybrid teams be allowed to debate at the NDT.</a>
<p>
<a href="http://repository.anumbersgame.net/wiki/HybridTeams">I was curious about the number and nature of hybrid teams.</a>
<h2>Sample:</h2>
<p align="center">
All teams:
<p align="center">
<img align="center" src="http://chart.apis.google.com/chart?cht=p&chd=t:2.70,64.65&chs=300x200&chl=Hybrid%7C&chco=FF0000,0000FFA0&dummy=img.png">
<p align="center">
Hybrid teams:
<p align="center">
<img align="center" src="http://chart.apis.google.com/chart?cht=p&chd=t:2.0,9.3,15.7&chs=250x100&chl=All+ringer%7CMixed%7CAll+drone&chco=FF0000A0&dummy=img.png">
<h2>Summary:</h2>
<p>
In the average year, about 6% of debaters participate in a hybrid
team. 23% of these only debate on hybrid teams.
<p>
50% of tournaments have some hybrid entries, but those tournaments
generate 58% of the ballots.
<p>
In addition to <a href="http://repository.anumbersgame.net/wiki/HybridTeams">the charts</a>, the <a href="http://repository.anumbersgame.net/source/browse/trunk/tests/hybrids/">raw data</a> is also available.A Numbers Gamehttp://www.blogger.com/profile/18412505663480533746noreply@blogger.com1tag:blogger.com,1999:blog-953859622641559363.post-55779260554711110002009-05-04T00:03:00.000-07:002009-05-04T08:27:39.412-07:00Charts of tournament win-loss networks<p><a href="http://repository.anumbersgame.net/wiki/TournamentCharts">I have posted charts showing win-loss networks of college tournaments from 2003 through 2008.</a>
<table border="0" align="center">
<tr align="center">
<td align="center">
<iframe src="http://www.gmodules.com/gadgets/ifr?url=http%3A%2F%2Fanumbersgame.googlecode.com%2Fsvn%2Ftrunk%2Ftests%2Fteam-rating%2Fscc-visualize%2Fshow0708.xml&parent=http%3A%2F%2Fcode.google.com%2Fhosting&synd=code" id="gadget0" name="gadget0" frameborder="0" width="400" height="300"></iframe>
</td>
</tr>
<tr>
<td align="center">Click on any image to view a full-sized version.
</td>
</tr>
</table>
<p>You can <a href="http://code.google.com/p/anumbersgame/wiki/TournamentWinLossNetworkCharts">browse through</a> <a href="http://anumbersgame.googlecode.com/svn/trunk/tests/team-rating/scc-visualize/DebateResults0304/">or</a> <a href="http://anumbersgame.googlecode.com/svn/trunk/tests/team-rating/scc-visualize/DebateResults0405/">down</a><a href="http://anumbersgame.googlecode.com/svn/trunk/tests/team-rating/scc-visualize/DebateResults0506/">load</a> <a href="http://anumbersgame.googlecode.com/svn/trunk/tests/team-rating/scc-visualize/DebateResults0607">the</a> <a href="http://anumbersgame.googlecode.com/svn/trunk/tests/team-rating/scc-visualize/DebateResults0708/">charts</a> and <a href="http://code.google.com/p/anumbersgame/source/browse/trunk/tests/team-rating/scc-visualize/">the source code used to create them</a>.A Numbers Gamehttp://www.blogger.com/profile/18412505663480533746noreply@blogger.com0tag:blogger.com,1999:blog-953859622641559363.post-45715033096465778722009-04-29T12:25:00.000-07:002009-04-29T12:30:00.226-07:00Judge Side Bias<p>
Using Jon Bruschke's databases on <a href="http://debateresults.com/">debateresults.com</a>, I have calculated some historical judge side bias.
<p>
For instance, for the 2006-2007 season (the courts topic), the aff won 50.10% of all ballots. The judges below are those most likely to differ from the general judge population that year:
<p>
Aff bias:
<ol>
<li>Joe Koehle, Aff:56, Neg:30</li>
<li>Kirk Evans, Aff:26, Neg:10</li>
<li>Austin Carson, Aff:10, Neg:1</li>
</ol>
<p>
Neg bias:
<ol>
<li>Joe Patrice, Aff:22, Neg:45</li>
<li>Greta Stahl, Aff:26, Neg:50</li>
<li>Erik Holland, Aff:1, Neg:10</li>
<li>Brian DeLong, Aff:23, Neg:44</li>
<li>John Nagy, Aff:15, Neg:32</li>
</ol>
<p>
<a href="http://code.google.com/p/anumbersgame/wiki/JudgeSideBias">Other years and more statistical details (as well as the code used to calculate them) are available.</a>A Numbers Gamehttp://www.blogger.com/profile/18412505663480533746noreply@blogger.com7