EVT
Election Verification Toolkit
Guide to understanding our testing
Office of Cook County Clerk David Orr
Cook County, Illinois - Summer, 2013
PreLAT:
Pre-election Logic & Accuracy Testing
a. Ensure that machines are functioning
b. Programming is correct
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Two Independent Proofs
a. Logic = programming
b. Accuracy = mechanics of machines & ballots
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Logic (programming)
* some potential errors
a. That challenged candidate – did he really get taken off in
the final database?
b. Ballot Style found in new precinct – was it added correctly
c. Late changes to core databases (street data, voter files,
candidate filing) that can get out of synch with the
database that has created your ballots and programmed
your machines.
d. Random issues introduced by programmers
e. Hacks
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Accuracy
* some potential errors
a. Misplacement of the “Crease Gap”
- (the space set aside for the fold line on prefolded
mail ballots)
b. Printing company error
c. Miscalibrated touchscreens
d. Bad read heads or poor ‘darkness calibration’ of paper
ballot scanners
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Patterned results
a. Gore
1
b. Bush
2
c. Mickey Mouse
3
d. Nader
1
e. Tancredo
2
f. Donald Duck
9
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Method of Analysis
Visual Scan
a. Special election with two candidates in a county of 40
precincts
- Check 80 lines for a 1, 2 pattern
b. Cook County Federal/Gubernatorial General
- 1673 precincts
- 70 judicial retention contests (yes/no)
- 140 lines per precinct
- Avg. of 30 regular contests (Federal, State, local &
judicial) x 2-3 candidates
- 60 lines per precinct
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Method of Analysis
Visual Scan
(continued)
334,600 lines (200 / precinct x 1673 pcts.
a. Would your staff notice an error? Would you yourself?
b. You might catch 1, 2, 3, 1, 3, 3 …
c. But would you catch a contest that was present in a
precinct where it didn’t belong?
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Method of Analysis
Electronic Verification
a. Expected results
- Build a database
- Electronically compare results
- Look at merely 100 or 200 lines with discrepancies
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Database Build - linkages to build the
“Expected Results” table
a. Precinct to ballot style
b. Ballot style to district
c. District to contest
- Village of Markham, with Mayor’s contest, but also
Clerk and Treasurer contests
d. Contest to candidate
e. Candidate to votes
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Database Build
comparing Prelat
Results to Expected Results
a. Import prelat results
b. Where prelat results are accurate, the Expected Result
minus the Prelat Result = 0
Candidate
Prelat
Expected
Discrepancy
Gore
1
1
0
Bush
2
2
0
Mickey Mouse
2
3
-1
Nader
1
1
0
Tancredo
2
2
0
Donald Duck
9
3
-6
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Why did Donald get 6 extra votes?
a.“he’s just popular”
b.Staff accidentally included a bunch of extra ballots in the
test deck
c.There were 3 other candidates, who should have received
1, 2 and 3 votes, but the ballot counting machine added
those 6 votes to Donald’s.
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Discrepancy Analysis
Why did Mickey get one vote too few?
a. The paper ballot was marked too lightly
b. The read head on the ballot counter is wrongly
calibrated, and it missed a mark that we believe it should
have read.
c. The printer left two other candidates on the ballot
above Mickey’s name who have been removed; Mickey
shows up as candidate number 5, and staff naturally
gave him 2 votes instead of 3.
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Discrepancy Analysis
Prove it
a. It’s not enough to assume you know why.
- Find the ballot that’s mismarked.
- Is the ‘ballots counted’ total off? That could
prove a ballot was accidentally run twice
b. Check Touchscreen Paper Trails
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Break from the Pattern Intentionally
a. If anyone has hacked you, a 1-2-3 prelat pattern isn’t
hard to fake
b. Introduce a handful of extra ballots or mismarked
ballots, misvoted touchscreens.
c. Verifying that some discrepancies reflect your
intentional breaks from pattern gives you greater
confidence that the machines are counting correctly
d. Intentional errors can be easier to verify on
touchscreens than inadvertent ones – you can pull the
touchscreen immediately and check the results tape.
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Building the Prediction – check marks by words at
lower left indicate tables that have loaded
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Adding / Defining Tests (for touchscreen data,
scanner data, etc.)
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Our test pattern for touchscreens includes a basic
1-2-3 repeating pattern, and a ‘special pattern’ of
1 additional vote for each of the first two candidates
on the ballot (the manual- and card-activated votes)
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Filtering a discrepancy report to show the summary
for one precinct with two touchscreens and no errors.
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
The discrepancy report filtered to show one precinct
with an error.
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
A precinct with some problems:
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
A screen showing the touchscreen test for Barrington precincts. Each
line reflects a single touchscreen. Most of them have no
discrepancies. Look at precinct 2, where there is a discrepancy of 2
on one of the touchscreens. We’ll go to see where the discrepancy
was in a moment.
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Here we’ve zeroed in on touchscreen 4632 in Barrington Precinct 1.
We see that there every candidate got the predicted number of votes.
Notice the first two lines – Karen Darch and the write-in line for
Barrington Village President – instead of 1 / 2, the pattern is 2 / 3.
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
The reason for the different pattern is that we run a vote simulation
which creates a 1-2-3 pattern in every contest, and then we add a
manual-activated vote to the first candidate on the ballot and a card-
activated vote to the second. Darch and the write-in both received
one extra vote. Here is where we let the program know about the
‘special’ portion of our expected vote pattern (the contest involved is
contest 11 – notice way over to the right of the contest drop-down
you’ll see that we’ve clicked to indicate contest 11.)
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Here is the test for the touchscreen with the discrepancy in Precinct
2. You can see that Pete Douglas, a trustee candidate, received 1 extra
vote. We were able to go to that touchscreen and determine that a
staffer had misinterpreted the manual activation instructions and
added an unexpected vote for Douglas on this touchscreen – it
showed up in the paper trail.
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Post-Election Audit – verifying that
votes are counted accurately
a. Vote Canvass ensures accurate reporting of what machines
counted
b. Ballot Canvass ensures ballot count matches voter count,
or discrepancies are explained
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Why Audit the Election
a. Judges make errors
b. Judge might feel tempted to cheat.
c. Contacting judges to learn more about even innocent
discrepancies shows you’re watching, providing a powerful
deterrent against cheating
d. Let’s you engage in “micro-training” by identifying and
training on places where specific judges miss specific
things
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Vote Canvass
a. Reading published results against machine tapes
b. Was a wrong cartridge version entered into results?
c. Was anything garbled through machine or human error
d. In our experience, errors rarely if never found
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Ballot to Voter Canvass
a. Comparing votes counted to voters who signed in
b. Tracking discrepancies
- We find small discrepancies occasionally
- Most are obviously innocent
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Completely benign errors
a. Misnumbered applications
b. Math errors
- In a year with a 2
nd
card for our judicial ballot, our
ballot count may consist of
Touchscreen A + Touchscreen B + Touchscreen C + (Scanner Count) / 2
- a lot of judges stumble on that “divided by 2”
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Real Problems
a. “Hidden Touchscreen Falloff ”
b. Scanner Breakdown Mistakes
c. Wrong ballot box
d. Judges cheating
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Hidden Touchscreen Falloff =
Voters leaving a touchscreen without
remembering to cast a ballot
a. If you’ve ever left your ATM card in a machine, you
should sympathize
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Scanner Breakdown Mistakes
a. Case A – The scanner is repaired or begins working again,
but judges don’t notice that when it comes back on, the
previously counted votes are still there, so they re-run
these ballots.
b. Case B – The scanner breaks down and judges set aside
ballots until it’s repaired. After repair, they forget to rerun
the set aside ballots, though they run other ballots
c. Case C – The scanner breaks down and is never fixed;
judges fail to notify us and simply assume we’ll count the
rest of the paper ballots turned in.
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Wrong Precinct’s Ballot Box
Nuff said.
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Judges stuffing the ballot box
a. Would you catch a judge who voted extra ballots?
b. If not for our Ballot to Voter Canvass, we fear we might
not.
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
A method of judge assessment
a. Did the judge pool in precinct 12 fail to balance their votes
and voters properly more than once?
b. Do they understand the record keeping requirements?
c. Do they know we are watching.
d. It’s time for a new judge or two, or closer oversight from
office staff.
Office of Cook County Clerk David Orr
Cook County, Illinois Summer, 2013
Office of Cook County
Clerk David Orr
Document Outline - Election Verification Toolkit
- PreLAT: Pre-election Logic & Accuracy Testing
- Two Independent Proofs
- Logic (programming) * some potential errors
- Accuracy * some potential errors
- Patterned results
- Method of Analysis Visual Scan
- 334,600 lines (200 / precinct x 1673 pcts.
- Method of Analysis Electronic Verification
- Database Build - linkages to build the “Expected Results” table
- Database Build comparing Prelat Results to Expected Results
- Discrepancy Analysis
- Why did Donald get 6 extra votes?
- Why did Mickey get one vote too few?
- Prove it
- Break from the Pattern Intentionally
- Post-Election Audit – verifying that votes are counted accurately
- Why Audit the Election
- Vote Canvass
- Ballot to Voter Canvass
- Completely benign errors
- Real Problems
- Hidden Touchscreen Falloff = Voters leaving a touchscreen without remembering to cast a ballot
- Scanner Breakdown Mistakes
- Wrong Precinct’s Ballot Box
- Judges stuffing the ballot box
- A method of judge assessment
Dostları ilə paylaş: |