Announcement

Collapse
No announcement yet.

Case Stats

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    I doubt they will acknowledge it's this bad. Last year the case balance wasn't great either after regionals (even with all the noise caused by weaker teams it was running around 55% P in R3). They gave the overall balance which wasn't unreasonable (again because of all the noise) in the memo, but they mysteriously left out the R3 balance for the first time in years and called it "a fairly balanced case." If they did that for 55%, I don't know what they will do for this much imbalance.

    Comment


    • #17
      Stats from Week 1 of Regionals:

      Richmond:
      Total P wins: 34.5 (43%)
      Total D wins: 45.5 (57%)
      R3 P wins: 10 (50%)
      R3 D wins: 10 (50%)

      State College:
      Total P wins: 51.5 (49%)
      Total D wins: 52.5 (51%)
      R3 P wins: 15 (58%)
      R3 D wins: 11 (42%)

      Washington D.C.:
      Total P wins: 50.5 (53%)
      Total D wins: 45.5 (47%)
      R3 P wins: 11 (46%)
      R3 D wins: 13 (54%)

      Overall Stats:
      Total P wins: 136.5 (49%)
      Total D wins: 143.5 (51%)
      R3 P wins: 36 (51%)
      R3 D wins: 34 (49%)

      Frankly, I was somewhat surprised by these. The case is looking a lot more balanced in the wider pool of regionals than it was in the top invites. As some people pointed out earlier, I suspect a lot of this is noise. I also suspect that the D bias is going to be less pronounced with weaker teams (who will have less creative D theories).

      I wanted to see, though, what happens when we look only at the top teams at each tournament. This tells us what the case balance would look like if no changes were made before ORCS. So I looked at what happened if you restricted the stats to rounds where both teams ended up winning 5 or more ballots (i.e. the ones who will be going to ORCS either through direct or open bid).

      Turns out there were 18 rounds between top teams.
      P wins: 11 (32%)
      D wins: 23 (68%)

      This matches what we were seeing with those top invites. Hopefully, if this pattern continues, AMTA will correct the imbalance before ORCS.

      Comment


      • #18
        Stats from Week 2 of Regionals:

        Louisville:
        Total P wins: 37 (46.3%)
        Total D wins: 43 (53.8%)
        R3 P wins: 8 (40%)
        R3 D wins: 12 (60%)

        St. Paul:
        Total P wins: 52 (50%)
        Total D wins: 52 (50%)
        R3 P wins: 16 (62%)
        R3 D wins: 10 (38%)

        Columbia:
        Total P wins: 37 (38.5%)
        Total D wins: 59 (61.5%)
        R3 P wins: 5.5 (23%)
        R3 D wins: 18.5 (77%)

        Columbus:
        Total P wins: 50 (52.1%)
        Total D wins: 46 (47.9%)
        R3 P wins: 12 (50%)
        R3 D wins: 12 (50%)

        Fresno:
        Total P wins: 39.5 (41.1%)
        Total D wins: 55.5 (58.9%)
        R3 P wins: 14 (58%)
        R3 D wins: 10 (42%)

        Seattle:
        Total P wins: 47 (49%)
        Total D wins: 49 (51%)
        R3 P wins: 14.5 (60%)
        R3 D wins: 9.5 (40%)

        Overall Stats (Running):
        Total P wins: 399 (47%)
        Total D wins: 449 (53%)
        R3 P wins: 106 (50%)
        R3 D wins: 106 (50%)



        As usual, I also ran stats on the top teams (the teams with 5 or more ballots), and its looking a bit better this week, but still not great.
        P wins: 50.5 (43%)
        D wins: 67.5 (57%)
        Last edited by J.D. Lorean; February 11th, 2018, 11:04 PM.

        Comment


        • #19
          Stats from Week 3 of Regionals:

          Dallas:
          Total P wins: 48.5 (43.3%)
          Total D wins: 43 (56.7%)
          R3 P wins: 14 (50%)
          R3 D wins: 14 (50%)

          Jackson:
          Total P wins: 42.5 (44.3%)
          Total D wins: 53.5 (55.7%)
          R3 P wins: 11 (45.8%)
          R3 D wins: 13 (54.2%)

          Cedar Rapids:
          Total P wins: 58 (52.3%)
          Total D wins: 53 (47.7%)
          R3 P wins: 12.5 (44.6%)
          R3 D wins: 15.5 (55.4%)

          Cincinnati:
          Total P wins: 42.5 (40.9%)
          Total D wins: 61.5 (59.1%)
          R3 P wins: 9 (34.6%)
          R3 D wins: 17 (65.4%)

          Joliet:
          Total P wins: 47 (45.2%)
          Total D wins: 57 (54.8%)
          R3 P wins: 17 (65.4%)
          R3 D wins: 9 (34.6%)

          Kansas City:
          Total P wins: 41 (39.4%)
          Total D wins: 63 (60.6%)
          R3 P wins: 11 (42.3%)
          R3 D wins: 15 (57.7%)

          New Haven:
          Total P wins: 52.5 (46.1%)
          Total D wins: 61.5 (53.9%)
          R3 P wins: 16 (57.1%)
          R3 D wins: 12 (42.8%)
          Tempe:
          Total P wins: 50 (48.1%)
          Total D wins: 54 (51.9%)
          R3 P wins: 16.5 (63.5%)
          R3 D wins: 9.5 (36.5%)

          Toeka:
          Total P wins: 49.5 (51.6%)
          Total D wins: 46.5 (48.4%)
          R3 P wins: 12.5 (52.1%)
          R3 D wins: 11.5 (47.9%)

          Overall Stats (Running):
          Total P wins: 830.5 (46.3%)
          Total D wins: 962.5 (53.7%)
          R3 P wins: 225.5 (50.3%)
          R3 D wins: 222.5 (49.7%)

          As usual, I also ran stats on the top teams (the teams with 5 or more ballots). The data looks pretty much the same as last week.
          P wins: 99 (42.7%)
          D wins: 133 (57.3%)

          Comment


          • #20
            With the new case changes, I've started up the stats again. Only two ORCS this weekend so the sample sizes are going to be pretty small, but for what its worth here they are:

            Wilmington:
            Total P wins: 51 (53.1%)
            Total D wins: 45 (46.9%)
            R3 P wins: 9.5 (40%)
            R3 D wins: 14.5 (60%)

            Santa Monica:
            Total P wins: 45 (46.9%)
            Total D wins: 51 (53.1%)
            R3 P wins: 8 (33.3%)
            R3 D wins: 16 (66.7%)

            Overall
            Total P wins: 96 (50%)
            Total D wins: 96 (50%)
            R3 P wins: 17.5 (36.5%)
            R3 D wins: 30.5 (63.5%)

            The 50-50 split overall is really nice for the case committee if they can keep it up. The round three numbers are a bit more concerning, as they seem to be showing the old D bias back in force. That said, if you look at top teams vs top teams (I'm still measuring by 5 ballots or more since that was the lowest ballot record of a qualifying team), we get the opposite results for bias with 9 P wins (64.3%) to 5 D wins (35.7%). So, as mentioned above, it may just be that the sample sizes for both computations are too small to have meaningful results yet, and we will need to wait until next weekend to get a clear picture.

            Comment


            • #21
              Originally posted by J.D. Lorean View Post
              With the new case changes, I've started up the stats again. Only two ORCS this weekend so the sample sizes are going to be pretty small, but for what its worth here they are:

              Wilmington:
              Total P wins: 51 (53.1%)
              Total D wins: 45 (46.9%)
              R3 P wins: 9.5 (40%)
              R3 D wins: 14.5 (60%)

              Santa Monica:
              Total P wins: 45 (46.9%)
              Total D wins: 51 (53.1%)
              R3 P wins: 8 (33.3%)
              R3 D wins: 16 (66.7%)

              Overall
              Total P wins: 96 (50%)
              Total D wins: 96 (50%)
              R3 P wins: 17.5 (36.5%)
              R3 D wins: 30.5 (63.5%)

              The 50-50 split overall is really nice for the case committee if they can keep it up. The round three numbers are a bit more concerning, as they seem to be showing the old D bias back in force. That said, if you look at top teams vs top teams (I'm still measuring by 5 ballots or more since that was the lowest ballot record of a qualifying team), we get the opposite results for bias with 9 P wins (64.3%) to 5 D wins (35.7%). So, as mentioned above, it may just be that the sample sizes for both computations are too small to have meaningful results yet, and we will need to wait until next weekend to get a clear picture.
              Any thoughts on the root of the immense deflation in point differentials? Certainly ORCS contains more evenly matched teams, but there are so few double digit victories, even among high-low pairings (see UCLA A, a traditionally dominant team, won no ballot by larger than 8, and even that was accompanied by a 9 point loss from the other judge). The highest point differential of any team was a +28 by UCLA B against Oregon State, followed by a three way tie with +21 between Irvine A against Riverside, Howard A against Rutgers A (given that both qualified for NCT -- wow), and UVA A against both Elon and SMU.

              Comment


              • #22
                For a team to make ORCS, they have to be at least marginally competent - most insane regional PDs come from extreme mismatches in competition.

                Comment


                • #23
                  Point differential is also extremely based on the judge. Some judges grade everything comparatively, others take off a point every time some uses wrong pronouns, says umm, or looks at them funny. I think using point differential to determine case balance doesn't make a lot of sense. I also think we should be cautious about just using R3 as a metric, often times even if a team had a decent record through Day 1, they may have an over-inflated score at the time. See P1 v. D1 at wilmington, Patrick Henry A vs. SMU... I wouldn't really say that those are equal top teams, but just because PH swept doesn't make me think the case is defense sided. I am really impressed that the overall trend is 50/50 though. It feels prosecution sided, but maybe people have already come up with really creative theories.

                  Comment


                  • #24
                    Originally posted by DefenseMid View Post
                    For a team to make ORCS, they have to be at least marginally competent - most insane regional PDs come from extreme mismatches in competition.
                    Would disagree on the point differential part. In my day I was part of a number of dumpster fire regionals teams that qualified against pretty talented but young C and D teams. My senior year felt like a 3-5 performance given how competent the teams we hit were (I belonged to a program that doesn't check tab at the student level until the end). We went 7-1.

                    Something I've noticed this year is the insane number of rounds with splits of one ballot wildly in favor of the defense and a narrow one in favor of the prosecution. I actually judged a round of two top-10 team at an invitational this year where I had a prosecution team narrowly lose my ballot, not because the defense theory was impressive (in fact, it yielded five soft impeachments during the defendant's cross examination alone), but because the prosecution's framing and response to the claims of the defense weren't as effective as they could've been (given the available evidence). In fact, the defense theory was kind of ridiculous if you know how mock trial works. My colleague, who hadn't ever seen the case, gave the ballot to the defense by double digits.

                    Certainly there are disagreements in opinions about how trials go, but I'd be interested in seeing PD (especially in rounds with wild splits) used as a contextual metric of case evenness. Yale's weighted partial balloting system comes to mind, but I certainly think the high level math involved in it makes it hard to use. Looking at a performance like UVA's at Downtown (where they won 9.5/10 defense ballots, 5 by 10 or more) but the 4 prosecution ballots they didn't lose were by one or a tie. Certainly PD unto itself is probably a bad metric, but when coupled with opposing sides' overall CS (and a CS broken down by prosecution and defense), we can yield more useful data than whether there was a win or a loss. I certainly think there's more a problem if the average defense is winning 55% of trials by 10 or greater rather than prosecutions winning 60% of all trials with an average PD of +3.

                    Comment


                    • #25
                      Originally posted by Adevans View Post
                      Point differential is also extremely based on the judge. Some judges grade everything comparatively, others take off a point every time some uses wrong pronouns, says umm, or looks at them funny. I think using point differential to determine case balance doesn't make a lot of sense. I also think we should be cautious about just using R3 as a metric, often times even if a team had a decent record through Day 1, they may have an over-inflated score at the time. See P1 v. D1 at wilmington, Patrick Henry A vs. SMU... I wouldn't really say that those are equal top teams, but just because PH swept doesn't make me think the case is defense sided. I am really impressed that the overall trend is 50/50 though. It feels prosecution sided, but maybe people have already come up with really creative theories.
                      This year's case is actually really interesting because of how much it is shaped by personal opinion. I worked with a team this season and my words of advice were to execute perfectly on everything up to the second prosecution witness because looking at ballots (and scorer's comments) -- it appears that people make up their minds and score accordingly on everything after that point. I'd be interested in whether AMTA has ways to assess scorer bias that aren't in the memos, because the comments (and the insane splits) make me think so much of the scores in this case are determined by scorer opinions about the fact pattern rather than the arguments themselves.

                      And on the prosecution heavy bit, I've actually always thought that from an evidentiary and argument perspective, this is a case that favors the prosecution overwhelmingly. Judges who are unfamiliar with the fact pattern (and the fact that other, uncalled witnesses often time answer the questions they have) seem to break for the defense. Scores from Santa Monica definitely seem to reflect a return to case balance.

                      Comment


                      • #26
                        This week things settled into a more even pattern. The D bias is back (at least in the overall stats). It's not nearly as strong as it was at regionals, though, so that's an improvement:

                        Geneva:
                        Total P wins: 40 (41.7%)
                        Total D wins: 56 (58.3%)
                        R3 P wins: 16 (66%)
                        R3 D wins: 8 (33%)

                        Greenville:
                        Total P wins: 48 (50%)
                        Total D wins: 48 (40%)
                        R3 P wins: 14 (58%)
                        R3 D wins: 10 (42%)

                        Central Islip:
                        Total P wins: 40.5 (42.2%)
                        Total D wins: 55.5 (57.8%)
                        R3 P wins: 10 (42%)
                        R3 D wins: 14 (58%)

                        Overall:
                        Total P wins: 224.5 (46.8%)
                        Total D wins: 255.5 (55.2%)
                        R3 P wins: 57.5 (47.9%)
                        R3 D wins: 62.5 (52.1%)

                        Looking at the top v top rounds, there were 28 P wins and 28 D wins, so thats a perfect 50-50 split so far.

                        Comment


                        • #27
                          For anyone interested, here are the stats on the NCT Case:

                          Total P Wins: 113 (46%)
                          Total D Wins: 156 (54%)
                          R3 P Wins: 27.5 (37.7%)
                          R3 D Wins: 45.5 (62.3%)

                          Comment


                          • #28
                            Defense Bias seems to be quite real for the NCT case.

                            Since R4 is not power protected, do you have stats for that as well?
                            Last edited by goldfish687; April 25th, 2018, 12:45 PM. Reason: spelling

                            Comment


                            • #29
                              Originally posted by goldfish687 View Post
                              Defense Bias seems to be quite real for the NCT case.

                              Since R4 is not power protected, do you have stats for that as well?
                              Sorry, I was running on autopilot after regionals/ORCS. R4 went 51.4% D. So the total R3/R4 percentage was 56.9% D

                              Comment

                              Working...
                              X