Colorado chose Free & Fair to build a risk-limiting audit (RLA) system to be used statewide beginning with the November 2017 general election. First developed in 2008, RLAs promote evidence-based confidence in election outcomes by comparing a random sampling of paper ballots to their corresponding Cast Vote Records (CVRs), which are the electronic records indicating how the marks on a ballot were interpreted as votes by the tabulation computers. This will be the first time anywhere in the United States that risk-limiting audits are conducted on a regular, statewide basis. You can follow along with the development process on the project GitHub repository.
Free & Fair has already prototyped an open source risk-limiting audit tool called OpenRLA, for RLAs of election contests in single jurisdictions. The production RLA system being developed for Colorado will facilitate statewide, multi-county and individual county audits. Like OpenRLA, the RLA system developed for Colorado will be released under an open source license (GPL Version 3).
About RLAs
Risk-limiting audits provide strong statistical evidence that a jurisdiction’s voting system accurately interpreted and tabulated voter markings on paper ballots, with relatively little hand counting. The “risk limit” is the largest chance that an outcome-changing error in the initial tabulation will not be discovered and corrected in the audit. If the risk limit is 5% and the outcome wouldn’t match the result of a full, accurate count of the paper ballots, there is at least a 95% chance that the audit will correct the outcome. The risk limit calculation is based on worst-case assumptions, so the actual chance of correcting a wrong outcome may be much larger than 95%, depending on how and why the outcome is wrong. Mathematically speaking, with a 5% risk limit, a risk-limiting audit reduces any uncertainty about the correctness of the tabulation outcome by a factor of 20. The risk limit is one of two factors which determine how many ballots must be checked. The lower the risk limit, the more ballots that must be checked. The other factor is the contest’s margin of victory.
To conduct a risk-limiting audit, paper ballots from a race to be audited are randomly selected and compared to their corresponding digital representations. If the margin of victory is large, the number of ballots that must be checked can be as low as dozens. But, if the margin of victory is small, a larger number of random ballots will have to be checked to provide sufficient statistical evidence that the digital count is sufficiently accurate. RLAs can sometimes lead to a complete hand count, but usually only if the margin is razor thin, or if the tabulation system is not functioning properly. RLAs are an efficient, transparent way to determine whether the computers have correctly counted the votes.
About traditional post-election audits
Traditional post-election audits typically entail randomly selecting a few precincts or voting machines, and checking the associated results with a hand count of the paper ballots. While these audits do provide some evidence that the machines have correctly interpreted voters’ intent, they audit only a few samples, often using an artificially created subset of results, rather than the actual election results. Therefore, the level of confidence yielded by a traditional post-election audit provides far less evidence at higher cost. Colorado, as one of the first states to have the detailed reporting needed for ballot-level RLAs, is pioneering these more efficient RLAs.
In some places, traditional post-election audits don’t even check the accuracy of the interpretation of voter intent. Where there are no paper ballots, there is no reliable record of voter intent. In these jurisdictions, all that can be audited is the ability of the computer to add correctly.
Details about Colorado’s new RLA system
The Secretary of State will establish the risk limit and select the contests based on closeness of margins, number of eligible electors, workload on counties, and other factors. Counties will then use the RLA system to upload ballot manifests, cast vote records (CVRs), and summary results to a central server. The RLA system will use this data to determine the initial set of ballots to be checked for each contest based on the risk limit and the margin of victory. The RLA system will report this information to each county, where the Audit Board, consisting of representatives of different political parties, will retrieve the appropriate ballots and record their interpretations of voter intent in the RLA system. If there are too many mismatches between the Audit Board’s interpretations and the choices reported by the vote-tallying system, additional ballots may need to be randomly selected until either the risk limit is satisfied or a complete hand count is conducted.
Links to info about RLAs: