Overview
This project is a relatively simple demonstration of mapping technology available to R users, using Oregon election precincts and results from the 2020 general election.
Originally, I compiled Portland, Oregon metro district precinct data for the 2020 presidential election. Using mapping tools and R database management features, I generated colored maps that show precinct level electoral results.
In some ways, the result evokes the New York Times’s extremely detailed map of the 2020 election which first appeared following the election of 2016. The Times’s tool uses the Mapbox platform and the Open Street Maps layer.
Like my project, the Times’s tool adapts precinct data. At the precinct level, compiling this data is labor intensive, and precinct results are usually not aggregated by states along with the results reported by counties. Rather, they are reported by individual counties, and typically not provided as exportable datasets, but as PDF’s with inconsistent formatting. They noted, “scraped and standardized precinct-level election results from around the country, and joined this tabular data to precinct GIS data to create a nationwide election map.” 1.
PDF scraping software such as Tabula can help the process but some kind of manual transcription or editing is necessary. As the Times’s tool shows, the compilation is incomplete at this writing (2/10/2021). 2 The data used by the Times are available in a GEOjson download which I posted on the Github site.
My project uses shapefiles from various agencies around Oregon and extracts downloaded from OpenElections and some of the counties.
The map uses tmap’s Leaflet implementation and let’s users drill into a display that shows address-level two party election result with color coding and vote detail. This example displays percentages of the two party presidential election results.
This post includes the maps and the code that generated them. The supporting data tables and shapefiles can be found at https://github.com/ianrmcdonald/oregon_precincts_2020
Load Libraries
This list and sequence should be cleaned up. I’m using the easypackages function libraries.
Generate a centering address
This chunk can produce an address marker using the geocode_OSM function from tmap. An example is shown here. In the next version this will use a text input for Shiny.
address_input <- c("Portland State University")
getAddress <- function(address_text) {
address_raw <- geocode_OSM(address_text)
address <- tibble(place = c(address_text), longitude = address_raw$coords["x"],
latitude = address_raw$coords["y"])
address_output <- st_as_sf(address, coords = c("longitude",
"latitude"), crs = "NAD83", agr = "constant")
return(address_output)
}
address_sf <- getAddress(address_input)
Load and process shapefiles and sf objects
County and state websites generally use shapefiles, but some counties have switched to GEOJSON. I read the rawfiles as sf objects using st_read function. In this case, I combine an Oregon shapefile for counties with Clark County in Washington, included because Clark County is part of the Portland Metro area.
At some point, I need to find a way to convert map objects used in ggplot into sf objects.Somehow this will involve the st_as_sf function but I haven’t figured it out yet. The usmaps package is very handy for quick map generation, and it uses Alaska and Hawaii properly, but my intent is to use tmap whenever possible.
I used the Metro website to find precinct shapefiles for Multnomah, Clackamas, and Washington counties. Portland State University posted 2018 shapefiles for the remainder of the state, and they appear to work fine for the 2020 election. Clark County WA came from the county website. The New York Times extract appears to have reliable precinct shapefiles for Oregon as well, but I obtained these shapefiles before the Times tool was published.
I added labels from the USMAP fips lookup. Is there a better lookup table provided by the Census bureau from tidycensus?
Notice that I transform the sf objects into a coordinate reference system of NAD83, although they appear to have been read with that CRS. The bind_rows function doesn’t appear to work otherwise. I"m probably assigning the CRS reduantly but my early attempts would often fail because of CRS inconsistency.
I added a state boundary layer because I have included Clark County data from Washington state. Oregon precinct boundaries came from https://opendata.imspdx.org/dataset/november-2018-election-oregon-results-by-precinct. This link hasn’t worked for some time.
Load County Shapefiles
oregon_counties_sf <- st_read("shapefiles/oregon_counties/counties.shp",
quiet = TRUE) %>% st_transform(or_counties, crs = "NAD83") %>%
select(!COUNTY)
washington_state_counties_sf <- st_read("shapefiles/washington_counties/WA_County_Boundaries.shp",
quiet = TRUE) %>% st_transform(crs = "NAD83") %>% mutate(STFID = as.character(JURISDIC_5))
washington_state_sf <- st_union(washington_state_counties_sf)
clark_county_sf <- washington_state_counties_sf %>% filter(JURISDIC_2 ==
"Clark")
Often, I use the dplyr function bind_rows to consolidate sf objects. It generally works fine, even when there are slight differences in field names that need to be reconciled.
oregon_counties_and_clark_sf <- bind_rows(oregon_counties_sf,
clark_county_sf) %>% st_transform(crs = "NAD83")
The usmap package provides a fips_info function that generates county names in a fips table from a single column of FIPS input. This chunk attaches names to the county shapefile. The Tidycensus package can provide the same content.
This chunk generates a county level map with the state bounary layer. This step applies the st_union function to generate the state boundary.
oregon_state_sf <- st_union(oregon_counties_sf)
oregon_bb <- st_bbox(oregon_counties_and_clark_sf)
This chunk creates a simple map of counties without precincts.
tmap_mode("plot")
oregon_county_tm <- tm_shape(oregon_counties_and_clark_sf, bbox = oregon_bb) +
tm_text("county", size = 0.5) + tm_polygons(alpha = 0, id = "county") +
tm_shape(oregon_state_sf) + tm_borders(col = "orange", lwd = 2.5)
oregon_county_tm
Load Precinct Shapefiles
As noted above, Metro Portland the rest of the state are processed separately.
metro_portland_precinct_sf <- st_read("shapefiles/metro_portland_precinct/precinct.shp",
quiet = TRUE) %>% select(precinct = PRECINCTID, county = COUNTY,
geometry) %>% mutate(county = case_when(county == "W" ~ "Washington",
county == "M" ~ "Multnomah", county == "C" ~ "Clackamas",
TRUE ~ "Other")) %>%
st_transform(crs = "NAD83")
clark_wa_precincts_sf <- st_read("shapefiles/clark_precinct_shapefiles/Precinct.shp",
quiet = TRUE) %>% select(precinct = PRECINCT, geometry) %>%
mutate(precinct = str_c("K", as.character(precinct)), county = "Clark") %>%
st_transform(crs = "NAD83")
Create a county level sf object that includes only the four counties of Metro Portland.
Merge Clark County WA into the metro Portland precinct file.
metro_portland_precinct_sf <- bind_rows(metro_portland_precinct_sf,
clark_wa_precincts_sf)
We generate a map that shows precinct detail within counties. The Oregon map excludes the Portland metro area counties.
tmap_mode("plot")
tmap_options(max.categories = 50)
metro_portland_tm <- tm_shape(metro_portland_precinct_sf) + tm_polygons(col = "county",
legend.show = FALSE, id = "county") + tm_shape(metro_portland_counties_sf) +
tm_text("county", fontface = "bold")
metro_portland_tm
We generate an sf object with precincts outside the Portland Metro area
or_precinct_sf <- st_read("shapefiles/oregon_precincts/OregonPrecinctsNov2018.shp",
quiet = TRUE) %>% filter(!County %in% c("Multnomah", "Washington",
"Clackamas")) %>% st_transform("NAD83") %>% rename(county = County,
precinct = Precinct)
oregon_counties_wo_PDX_sf <- oregon_counties_and_clark_sf %>%
filter(!county %in% c(c("Multnomah", "Washington", "Clackamas",
"Clark")))
or_precinct_tm <- tm_shape(or_precinct_sf) + tm_polygons(col = "county",
legend.show = FALSE, id = "county") + tm_shape(oregon_counties_wo_PDX_sf) +
tm_text("county", size = 0.75, fontface = "bold") + tm_shape(oregon_state_sf) +
tm_borders(col = "orange", lwd = 2.5)
or_precinct_tm
Load vote data for 2020 presidential election
This section processes the various files with precinct level election results. A lot of inconsistencies about candidate and party labeling are handled in these routines.
Build an data format function based on thee Open Elections data format
This function applies the Open Election data format, excluding some counties that are processed separately. In some cases, the Open Election data doesn’t identify the correct precincts, while Metro Portland data was obtained before Open Elections processed it.
Process votes for most Oregon counties
Process every county except the list of pre-processed counties below. The open data version of files for some counties outside of Portland were incomplete, so I found other sources for them (primarily the county websites).
Process Lane County vote totals, hand tabulated from county pdf.
lane_votes <- read_csv("data/votes/lane.csv", col_types = cols(.default = "d",
precinct = "c")) %>% mutate(county = "Lane") %>% rename(DEM = Biden,
REP = Trump) %>% mutate(OTH = Total - DEM - REP) %>% select(county,
precinct, DEM, REP, OTH)
Marion, Polk, and Yamhill County voter data. Note these use the Open Elections standard format. Portland Metro counties generated further down.
This chunk was necessary because the consolidated state file for Open Elections had issues with these three counties. In theory, this function could be applied to all individual county files.
open_election_col_types = cols(.default = "c", votes = "d")
counties <- c("marion", "polk", "yamhill")
counties_csv_names <- str_c("data/votes/20201103__or__general__",
counties, "__precinct.csv")
marion_polk_yamhill_votes <- map_df(counties_csv_names, open_election_format_votes)
Tillamook County, hand tabulated from the published county pdf
tillamook_votes <- read_csv("data/votes/tillamook_votes.csv",
col_types = cols(.default = "c", DEM = "d", REP = "d", OTH = "d")) %>%
rename(precinct = precinct_votefile) %>% select(county, precinct,
DEM, REP, OTH)
Consolidate the five additional counties with or_precincts_votes, then print a sample of ten records from the consolidated table.
or_precincts_votes <- bind_rows(or_precincts_votes, tillamook_votes,
lane_votes, marion_polk_yamhill_votes)
or_precincts_votes %>% sample_n(10)
# A tibble: 10 x 5
county precinct REP DEM OTH
<chr> <chr> <dbl> <dbl> <dbl>
1 Marion 403 1701 1512 103
2 Marion 352 193 214 11
3 Josephine Precinct 18 1002 418 49
4 Tillamook Foley 204 247 15
5 Marion 342 4 2 0
6 Clatsop 23 - South Clatsop 597 576 29
7 Jackson Precinct 009 486 248 26
8 Deschutes Precinct 30 906 1176 53
9 Lincoln 02 Alsea 309 297 10
10 Columbia 44 Rural Vernonia 575 245 22Read in a lookup table that matches vote file precinct names and shape file precinct names
lookup <- read_csv("data/votes_shapefile_lookup.csv", col_types = cols(.default = "c")) %>%
mutate(county = str_to_title(tolower(county)))
lookup %>% sample_n(10)
# A tibble: 10 x 3
county precinct_votefile precinct
<chr> <chr> <chr>
1 Crook Precinct 16 OR_CROOK_16
2 Josephine Precinct 21 OR_JOSEPHINE_21
3 Washington Precinct 396 OR_WASHINGTON_396
4 Multnomah 4706 OR_MULTNOMAH_4706
5 Multnomah 3102 OR_MULTNOMAH_3102
6 Lane 762 OR_LANE_0762
7 Multnomah 4503 OR_MULTNOMAH_4503
8 Washington Precinct 431 OR_WASHINGTON_431
9 Linn Precinct 095 OR_LINN_095
10 Washington Precinct 308 OR_WASHINGTON_308Combine lookup table and shapefile.
Two columns have white space in the text and needs to be trimmed. Could replace regular expressions with simpler tidy version; here I use a regular expression.
The final version of the or_precinct_sf object appears here. The df version of the sf object is handy for viewing and validation. Print a sample of ten reecords.
or_precinct_sf <- inner_join(or_precinct_sf, or_precincts_votes,
by = c(county = "county", precinct_votefile = "precinct")) %>%
select(county, precinct = precinct_votefile, REP, DEM, OTH,
geometry) %>% st_transform(crs = "NAD83")
or_precinct_df <- st_drop_geometry(or_precinct_sf)
or_precinct_df %>% sample_n(10)
county precinct REP DEM OTH
1 Marion 578 25 22 2
2 Baker BakerCountry 627 133 15
3 Lane 1123 170 1747 104
4 Columbia 10Milton 492 405 26
5 Linn Precinct018 1070 1147 116
6 Klamath Precinct37 387 451 37
7 Marion 336 3 5 0
8 Jackson Precinct015 1471 1976 104
9 Umatilla 109R-ADAMS,OUTSIDECITY 48 10 0
10 Jefferson No.16CampSherman 83 156 4Process the Metro Portland Vote tables
This section process vote tables for Multnomah, Clackamas, and Washington, plus Clark County WA
Clackamas County is somplicated because their reports combine several precincts. This chunk consolidates the affected precincts.
clackamas_precincts_combined <- tribble(~p1, ~p2, "007", "010",
"070", "071", "099", "100", "103", "104", "251", "252", "361",
"362", "417", "418") %>% mutate(PRECINCTID = str_c("C", p1,
"_", p2))
combine_clackamas_precincts <- function(df, insert_df) {
insert_df <- insert_df %>% mutate(p1 = str_c("C", p1), p2 = str_c("C",
p2))
df <- df %>% rename(PRECINCTID = precinct)
extract <- df %>% filter(PRECINCTID %in% insert_df$p1 | PRECINCTID %in%
insert_df$p2)
remaining <- df %>% rows_delete(extract, by = "PRECINCTID")
insert_df <- insert_df %>% pivot_longer(cols = c(p1, p2),
names_to = "p", values_to = "OLD_PRECINCTID")
insert_df <- insert_df %>% rename(NEWPRECINCTID = PRECINCTID,
PRECINCTID = OLD_PRECINCTID) %>% select(NEWPRECINCTID,
PRECINCTID)
extract <- inner_join(extract, insert_df, by = "PRECINCTID") %>%
select(-PRECINCTID) %>% rename(PRECINCTID = NEWPRECINCTID)
df <- remaining %>% rows_insert(extract, by = "PRECINCTID") %>%
rename(precinct = PRECINCTID)
return(df)
}
The function consolidates Clackamas precincts and reinserts them into metro_portland_precinct_sf
metro_portland_precinct_sf <- combine_clackamas_precincts(metro_portland_precinct_sf,
clackamas_precincts_combined)
metro_portland_precinct_df <- st_drop_geometry(metro_portland_precinct_sf)
As noted earlier, the Metro Portland vote data were generated by hand before it became available in the Open Elections extract.
multnomah_votes_df <- read_csv("data/votes/multnomah_votes.csv",
col_types = cols(.default = "d", precinct = "c")) %>% mutate(county = "Multnomah",
PRECINCTID = str_c("M", precinct)) %>% select(county, precinct = PRECINCTID,
DEM, REP, OTH)
clackamas_votes_df <- read_csv("data/votes/clackamas_votes.csv",
col_types = cols(.default = "d", precinct = "c")) %>% mutate(county = "Clackamas",
precinct = str_c("C", precinct))
washington_votes_df <- read_csv("data/votes/washington_county_votes.csv",
col_types = cols(.default = "d", precinct = "c")) %>% mutate(county = "Washington",
precinct = str_c("W", precinct), OTH = TOT - DEM - REP) %>%
select(county, precinct, DEM, REP, OTH)
clark_votes_df <- read_csv("data/votes/clark_votes.csv", col_types = cols(.default = "d",
precinct = "c")) %>% mutate(county = "Clark", precinct = str_c("K",
precinct), OTH = TOT - DEM - REP) %>% select(county, precinct,
DEM, REP, OTH)
metro_portland_votes_df <- bind_rows(multnomah_votes_df, clark_votes_df,
washington_votes_df, clackamas_votes_df)
metro_portland_votes_df %>% sample_n(10)
# A tibble: 10 x 5
county precinct DEM REP OTH
<chr> <chr> <dbl> <dbl> <dbl>
1 Clark K561 238 268 22
2 Clackamas C415 50 76 2
3 Clackamas C005 1286 1209 100
4 Clark K108 903 257 58
5 Clark K296 497 271 28
6 Washington W410 2064 1036 112
7 Multnomah M3611 4694 640 155
8 Washington W379 5092 1623 235
9 Multnomah M5004 3672 2567 214
10 Clackamas C031 1458 1047 96Join metro Portland vote tables and sf objects.
Here we check for precincts in each table that don’t match. In this case, a non-populated precinct in Clackamas County is the only unmatched precinct.
check <- anti_join(metro_portland_precinct_df, metro_portland_votes_df,
by = c("county", "precinct"))
check
precinct county
1 C000 ClackamasCreate the final consolidated sf object for Metro Portland. Report a sample of ten records.
metro_portland_precinct_sf <- inner_join(metro_portland_precinct_sf,
metro_portland_votes_df, by = c("county", "precinct")) %>%
select(county, precinct, DEM, REP, OTH) %>% mutate(precinct = str_sub(precinct,
2))
or_precinct_sf <- bind_rows(or_precinct_sf, metro_portland_precinct_sf)
or_precinct_df <- st_drop_geometry(or_precinct_sf)
or_precinct_df %>% sample_n(10)
county precinct REP DEM OTH
1 Washington 325 0 3 0
2 Clark 450 497 749 45
3 Clark 575 300 165 15
4 Clark 940 709 463 41
5 Umatilla 126-HERMISTON,CITYOF 687 564 43
6 Washington 435 1608 2335 157
7 Washington 390 687 1534 42
8 Clackamas 415 76 50 2
9 Washington 324 280 219 20
10 Deschutes Precinct27 374 1830 59Build the base Leaflet map object
In this case, the leaflet map object is generated by tmap’s viewer mode. Some additional formatting variables added in this chunk.
or_precinct_sf <- or_precinct_sf %>% mutate(Votes = DEM + REP +
OTH) %>% mutate(DemMOV = round((DEM/Votes - REP/Votes), 3) *
100) %>% mutate(vpct = abs(round((DEM/Votes - REP/Votes),
3) * 100)) %>%
mutate(pwinner = ifelse(DEM > REP, "DEM", "REP")) %>%
mutate(pct_lbl = str_c(pwinner, " ", vpct)) %>%
mutate(hover = str_c(county, " ", precinct, ": Victory Margin = ",
pct_lbl, " Total Votes = ", Votes)) %>%
select(county, precinct, DemMOV, vpct, DEM, REP, OTH, hover)
The final or_precinct_votes table has this structure:
or_precinct_df <- st_drop_geometry(or_precinct_sf)
or_precinct_df %>% sample_n(10)
county precinct DemMOV vpct DEM REP OTH
1 Deschutes Precinct42 -7.9 7.9 1525 1795 112
2 Washington 341 -10.3 10.3 151 188 20
3 Clark 961 18.5 18.5 286 194 18
4 Umatilla 116-ATHENA,OUTSIDECITY -19.9 19.9 56 85 5
5 Malheur 14-NorthRuralVale -67.7 67.7 117 643 17
6 Washington 438 -21.1 21.1 109 169 7
7 Baker Baker#4 -53.1 53.1 275 937 34
8 Clackamas 284 40.3 40.3 204 83 13
9 Multnomah 4510 41.7 41.7 1647 662 51
10 Clark 637 26.1 26.1 438 252 24
hover
1 Deschutes Precinct42: Victory Margin = REP 7.9 Total Votes = 3432
2 Washington 341: Victory Margin = REP 10.3 Total Votes = 359
3 Clark 961: Victory Margin = DEM 18.5 Total Votes = 498
4 Umatilla 116-ATHENA,OUTSIDECITY: Victory Margin = REP 19.9 Total Votes = 146
5 Malheur 14-NorthRuralVale: Victory Margin = REP 67.7 Total Votes = 777
6 Washington 438: Victory Margin = REP 21.1 Total Votes = 285
7 Baker Baker#4: Victory Margin = REP 53.1 Total Votes = 1246
8 Clackamas 284: Victory Margin = DEM 40.3 Total Votes = 300
9 Multnomah 4510: Victory Margin = DEM 41.7 Total Votes = 2360
10 Clark 637: Victory Margin = DEM 26.1 Total Votes = 714# find some table functionality besides kbl that works
Draw the map
Note that we created code that can control the midpoint between the colored Democratic and Republican precincts on the map (which defaults to 0.) Here, we have set the midpoint to +16, the Democrats’ margin of victory in Oregon. This means the colors relative significance but sometimes show Democratic precincts as red.
# Eliminate NaN's from the table listed here. If a precinct
# has zero votes it winds up with this result
tm_min = -100
tm_max = 100
min_max_bound <- function(x, min = tm_min, max = tm_max) {
min(max(x, min), max)
}
rwb <- colorRampPalette(c("#ff0000", "white", "#0000fa"))(256)
midpoint <- 16
pctiles <- midpoint + c(-80, -70, -60, -50, -40, -20, -5, 0,
5, 20, 40, 50, 60, 70, 80)
pctiles <- c(tm_min, map_dbl(pctiles, min_max_bound), tm_max)
bb_small <- bb(address_input, ext = 10)
tmap_mode("view")
tmap_options(basemaps = c("OpenStreetMap"))
or_precincts_tm <- tm_shape(or_precinct_sf, bbox = bb_small) +
tm_polygons(col = "DemMOV", n = length(10), id = "hover",
style = "fixed", breaks = pctiles, alpha = 0.6, palette = rwb,
midpoint = NA, legend.show = TRUE) + tm_view(set.zoom.limits = c(6,
15)) +
tm_shape(oregon_counties_and_clark_sf, bbox = bb_small) + tm_borders(lwd = 1,
col = "blue")
or_precincts_tm