Announcement

Collapse
No announcement yet.

Electronic Tab Room for Invitationals.

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Electronic Tab Room for Invitationals.

    Our program previously attended a tournament with an electronic tab room where the tab summary was shared online with competitors minutes after the round was over. We found this system to be very efficient from a competitor's perspective and are looking to adopt the same system for our invitational tournament.

    Do you believe an electronic tab room is overall better than a traditional one? Does anyone use it at their tournament? Just looking for some guidance from more experienced tournament hosts/tab directors.

  • #2
    For the two years I used balanced weighted partial ballots (link) to run my invitationals, I emailed an electronic tab summary out after each round simply because the math was so complicated that just viewing the tab cards wasn't particularly helpful. I stopped using BWPB and emailing out the tab summary in part because it was an additional extra step that added more time to the tabulation process. I think if you have enough people that one person can be dedicated to copying ballots onto the computer as they come in it would be worthwhile, but I struggled to get enough team members that weren't competing to help with other duties in tab.

    In short, I see no down side, but would take steps to make sure it doesn't slow down the process of getting round pairings done.
    Ray Barr (the Amazing)
    "Bow ties are cool."

    Comment


    • #3
      To be completely honest, I dislike electronic tab rooms. We've only been to a few tournaments that use them, but the value of using the standard tab cards is it allows everyone to check your work. When you have 5+ sets of eyes besides the reps/tab directors watching to make sure everything is correct, you're less likely to have mistakes. In a perfect world, if someone designed a tab room software that was error-proof, I'd have no problem with it being used. I'm not aware of such a thing existing, and until it does, tournaments should just learn the tab card system and use it. It doesn't prevent you from entering results into a spreadsheet and sharing it (i.e. say a running Google Sheet that competitors can view, which is an intriguing idea) but it gives coaches/captains the chance to follow along and be sure things are done right.

      Comment


      • #4
        In my experience, there are three tasks in Tab that are still done by hand:
        1. Tabulating ballots
        2. Pairing rounds
        3. Individual awards
        When I was still competing for Yale, one of my teammates (Annie) who ran Tab for our Invitational built an electronic tab system using a combination of Google Sheets and some programming language I can't recall, and this is how it dealt with these three items (as far as I can remember):
        1. Annie made a Google Form where Tab volunteers would input each individual score, along with the two teams competing in the round (which would be populated from a pairings list). The computer would then calculate the winner of each ballot, along with the PD.
        2. Annie wrote a program that would automatically pair each round given Wins, CS, PD, impermissibles, etc.
        3. Annie used a Google spreadsheet that took as inputs each round pairing, as well as rosters submitted by each team. Tab volunteers would then enter the side of each ranked individual (P/D), which would then populate that team's roster, then the Tab volunteer would click the appropriate name. So this spreadsheet had one row for each round-trial-role-rank combination. A separate sheet within the same file kept a running total of top ranks, which made it easy at the end of the tournament to identify award winners.
        I really liked what Annie built for the individual awards (and I hope to recreate it and bring it back for the Yale Invitational next year). Even if it didn't save much time, it was a big QOL improvement, and having the computer identify the top ranking witnesses/attorneys at the end was a big plus.

        That said, I think individual awards are separate from tabulating and pairing, and this is where Ben's concerns about accuracy come into play for me: I think an electronic tab system needs to shave off substantial time in order to justify the black boxiness of it all, and I don't think such a system can exist if we still use paper ballots. My view is that the tradeoff favors electronic tab iff the system can fully automate both tabulation and pairing. And while we're still using paper ballots with judges with messy handwriting who like to write a lot more than 28 scores, I don't think it's possible to implement any sort of scanner that feeds the scores into the system. If the scores still have to be entered by hand, I don't see much time being saved versus standard tabulating by hand.

        Also, a program that ostensibly pairs rounds has its own issues. First, there's the programming challenge of handling edge cases (or even just difficult cases where, like, all but two rounds are impermissible). Second, it's not immediately obvious to me how you could integrate both tabulating and pairing into the same program. If you can't do that, and you have to manually enter Wins, CS, PD, etc. after each round for each team in order to run the pairings, you lose out on a lot (if not all) of the time savings.

        At the end of the day, I don't see an electronic tab system being better or faster than tab by hand, as long as your tab room is sufficiently staffed. The past three Yale Invitationals, we've had ~50 teams across two divisions, and thanks to a veritable army of volunteers from other programs, our Tab room ran quickly and smoothly by hand.

        TL;DR: Give every judge an iPad with an e-ballot.
        Last edited by lookatmeguo; July 11th, 2019, 11:19 AM.

        Comment


        • #5
          Totally agree with lookatmeguo. Certain tasks (individual awards especially) can and should be done by computer if you develop an effective system. I also really like the idea of a running results sheet that competitors and coaches can use in real time to refer to results. But as long as we're using paper ballots, tournaments should use tabulation cards for transparency and error avoidance. At the very least, if you're using an electronic tab system, you need to be prepared to explain it to me and let me see the underlying data so I can verify it was entered correctly.

          Comment


          • #6
            I hope that the programmers and tech savvy among us keep trying to crack this nut, because automation of the tab process would make tournaments more efficient and also make tournament hosting more accessible. I recall at least 2 attempts by people several years ago to write programs that auto-paired rounds. If you scroll back far enough in Perjuries, you could probably find links to them. I don't remember seeing one that had all the kinks worked out. Writing a script for initial pairings is easy enough, but scripting all of the permutations for resolving impermissibles proves to be a challenging quest. If Yale's pairings program managed to do that error-free, including all complex impermissibles with multiple switches, then I really hope they dig that up and make it available for peer review. If nothing else, having a computer program run the pairings would be a great way to double check your work if you do it manually. I would pay to use that program.

            I agree that tablets with e-ballots is the theoretically best solution, since it eliminates manual entry in the tab room, which is the biggest time consumer. Of course you'll have unsavvy judges who don't know how to use it, plus possible connectivity problems (imagine the screen resets and you lose all your scores). Plus the cost of 20+ tablets is unrealistic, unless you get a grant or something. Easily $1,000+ total even if you get the cheapest ones.

            But I would totally support a tournament that beta-tests e-Ballots for certain savvy judges. If you emailed me a Google Sheets or Google forms link to insert my blues scores, I could just do that from my phone, no problem, and the tab room could be seeing my scores in real time. I could just indicate once I'm "done", and then my ballot would be tabulated even before the sheets get walked to the tab room. If anybody ends up doing that kind of beta test please share.


            Comment


            • #7
              Empire has used different flavors of electronic tabbing the past few years. It largely operates off of Google sheets with a tournament volunteer in each courtroom or each pair of courtrooms entering the data off of the completed scoresheets. I've not been the person in charge of it, but the best description I can give is that the spreadsheet gets laggy and slow as the tournament progresses because of the extent of the data it's carrying and the nature of the calculations it's performing. (That's more an issue with Google sheets itself than with the concept, which would probably work much faster as a native program.) The tab room still checks the paper ballot against the spreadsheet data, so there isn't necessarily a huge time savings because of having to still basically manually check the e-data against the paper data for entry errors. It's not necessarily a *bad* system but I don't think it has enough significant benefit over the manual system to justify the resultant lack of transparency.

              Now, if someone wants to make an app for judge assignments...take my money...
              I post in my personal capacity, not on behalf of AMTA.

              Comment


              • #8
                Originally posted by geneva View Post
                If Yale's pairings program managed to do that error-free, including all complex impermissibles with multiple switches, then I really hope they dig that up and make it available for peer review.
                If I recall correctly, Annie was also using Weighted Partial Ballots for that tournament, which would have rendered unnecessary the Wins/CS/PD/coinflip hierarchy of tiebreaks, though impermissibles still could've been an issue.

                Originally posted by geneva View Post
                If nothing else, having a computer program run the pairings would be a great way to double check your work if you do it manually.
                Elizabeth Bays, Michael Byrnes, and I have talked on and off the past few years about writing a pairings program, but life's gotten in the way. Maybe this year.

                Comment


                • #9
                  Originally posted by MizzouMock View Post
                  Now, if someone wants to make an app for judge assignments...take my money...
                  I've never agreed more with anything on this website.

                  Comment


                  • #10
                    I've been working on electronic tabulation (+ pairings + judge/room assignment + summary and score analysis) as a downtime project without being aware of existing solutions, so I'm super-curious as to what's made prior implementations (with paper ballots) error-prone or unreliable. Not just in terms of user experience/trust but in terms of (if people can share those) the implementation hurdles.

                    MizzouMock What makes Empire's Sheets solution laggy, for example? I've (sadly enough) worked on both very performant and very laggy Sheets workbooks in the past, and ime the biggest driver of lag has been calculations with a lot of dependencies. Most Sheets calculations consume barely any resources other than human-noticeable time, so I feel like if you parallelize/design well enough you shouldn't face any serious lag. How many computations even happen anyway? If you're just doing straight-forward tab, is there some large dependency chain that emerges? I can't think of (m)any calculations that have deep dependencies like that.

                    I guess the bar for lag is a lot lower when you have to enter new results while calculations are occurring. Was Empire's lag creating that type of issue- delaying human input while calculations were happening? Or was it even more serious than that?

                    Re:validation- I thought the validation system (minimum of 3 pairs of eyes/ballot) in standard tabulation was fairly reliable when you force people to add up ballots in different ways. It also seemed fairly easy to translate- you have satellite tab add up ballots by hand (as the first validator already does), main tab input tab data score-by-score (so 30 scoring entries per ballot, taking 15-30s/ballot with a row-by-row Tab...Return entry pattern, which- when not parallelized- is still <1h/round for a 60-team 3-judge tournament), and have built-in validation to detect anything that doesn't add up before acting as the 3rd validator and computing the margin.

                    Another consideration, though- if we managed to not only automate but standardize the process (and make it open to all teams, thereby achieving the accessibility goal), then we could:

                    a) generate process improvements like what lookatmeguo mentioned, with more dependable support for edge-cases + trustworthy clever solutions
                    b) get rid of the ugly inconsistency/tendency of tab summaries to be shitty, unpublished, and/or error-prone
                    c) get more ambitious over time instead of having a bunch of programs essentially working on the same problem, hitting the same hurdles, and failing to gain trust/traction
                    d) adopt OCR if we feel like it; bad handwriting is often still legible by computer (at least when it comes to numbers) + very likely to trigger error-detection if you misread a digit, especially if you have multiple OCR approaches working concurrently and validating one another

                    As a side-note, if there's an ongoing effort to tackle this, I'd be down to just merge my work into it. I've done everything so far in Google Sheets (+ am planning to throw in some Apps Script/JavaScript-with-lower-standards for some of the fancier aspects) but if a proper solution to this requires actual webapp development, it shouldn't be too hard to translate what I have so far.

                    If people wanna get ambitious about the scope of finally bringing mock trial into the 1990s, the domain mocktri.al is still available and would be kind of neat to host it all on.

                    I just feel like calculating wins/CS/OCS/PD/the likelihood that someone will post about the round afterward on Confessions, and pairing based on all that- it can't be that hard a problem. Call me arrogant, but I think that the blocker here isn't the problem space; it's the kind of solution you get without good design + engineering effort + heaps of validation. It's the same reason United.com is trash while Google Flights is often not trash- the problem space alone doesn't much constrain the quality of a good solution.
                    Last edited by Zephaniah; July 12th, 2019, 01:18 AM.
                    things are really heating up in the lawyer impersonation fandom

                    Comment


                    • #11
                      Would somebody be willing to share a link to an example of an electronic tab?

                      Comment


                      • #12
                        Originally posted by Zephaniah View Post
                        As a side-note, if there's an ongoing effort to tackle this, I'd be down to just merge my work into it. I've done everything so far in Google Sheets (+ am planning to throw in some Apps Script/JavaScript-with-lower-standards for some of the fancier aspects) but if a proper solution to this requires actual webapp development, it shouldn't be too hard to translate what I have so far.

                        If people wanna get ambitious about the scope of finally bringing mock trial into the 1990s, the domain mocktri.al is still available and would be kind of neat to host it all on.

                        I just feel like calculating wins/CS/OCS/PD/the likelihood that someone will post about the round afterward on Confessions, and pairing based on all that- it can't be that hard a problem. Call me arrogant, but I think that the blocker here isn't the problem space; it's the kind of solution you get without good design + engineering effort + heaps of validation. It's the same reason United.com is trash while Google Flights is often not trash- the problem space alone doesn't much constrain the quality of a good solution.
                        +1 agree.

                        The bigger question is if someone built an electronic tab, would anyone use it?

                        Comment


                        • #13
                          ^ Wrong way to frame the question. Better to ask what features/characteristics/etc. would make an electronic tab highly likely to gain adoption.

                          I suspect the reduced pain when it comes to generating tab summaries is already an easy way to appeal to tab directors at less established tournaments. Moreover, a reduction in labor cost per round (for inexperienced + coachless tournaments, tab is a huge labor suck) is pretty appealing.

                          An inefficiently-run tournament just sucks up hours of commitment that your team would almost certainly prefer to have spent on actual practice.
                          things are really heating up in the lawyer impersonation fandom

                          Comment

                          Working...
                          X