I spent $400 on UpWork for a developer to build my firm a custom web scraper for Nevada political donations, and I'm sharing it with you for free.
As a campaign professional who has raised money in five different states, I'm deeply familiar with the different arcane setups that state campaign finance reporting websites have.
If you are lucky, you will be operating in a state like Washington that has a more modern campaign finance database with easily downloadable CSV files.
https://www.pdc.wa.gov/browse/campaign-explorer/candidate?filer_id=INSLJ%20%20110&election_year=2020
Here in Nevada, we are blessed with a campaign finance database that looks like it comes out of the early 2000s.
Even worse, there is no easy way to directly download individual candidate reports for analysis. These reports are designed to be easily printable and not easily downloadable.
Here's a snapshot of what a Nevada report typically looks like:
(The screenshot below is from Gov. Sisolak's 2021 CE filing)
Each new row of boxes contains all donations in a given reporting period for a given donor.
This is great if you want to see all of the donations that a specific entity made during a reporting period, but incredibly counterproductive if you'd like to download this data as a CSV file instead.
If you try to copy and paste each page onto excel, here is what you get:
The result is a pile of excel rows that are incredibly difficult to organize and sort.
Previously, I had a bunch of Interns working on sorting this data by hand. I've even spent hours on weekends copying and pasting donation records onto a google sheet to onboard a new client.
I'm ashamed to admit it, but I've wasted a ton of my own time and the time of others by having them do a repetitive task that is far more suitable for robots than humans.
Here's where the web scraper comes in. By using python to sort and comb data, you can instantly create digestible CSV files of campaign finance reports that are easy to sort and organize.
You don't even need to know how to code or put together a python script. Upwork gives you (or your campaign) access to hundreds of talented developers at prices that are likely cheaper than you'd think.
Here's a snapshot of the finished product:
(The data is derived from Commissioner Michael Naft's finance reports)
For a copy of the python script. Please send an email to Kalani.
Here's how to successfully use this script:
Use a Windows OS (Sorry, Mac users. If anyone knows how to adapt python scripts for Mac, let me know, and I can update this article.)
Download the latest version of Python
Create a new folder and place this script file in it
Download individual campaign finance reports from the NV SOS website as "HTML" files and place them in this folder. You do this by going to each individual report in your browser, right-clicking each webpage, and then clicking on "Save As...".
Run the script, and it should work! A new CSV file called "output" should appear with neatly combed data in it.
You can save as many individual reports as HTML files as you like, and this script will sort them all into one single output file.
I strongly believe that as fundraisers, our job should be focused as much as possible on making meaningful solicitations to the right people with the right medium and message. Spending hours on manual data entry or administrative work that could be automated only hampers our ability to raise more money and foster new relationships.
Have any questions? Send me a note at Kalani@tissotsolutions.com.