Today I was faced with a daunting task. As previously mentioned, I am the developer behind Govistics, a government spending statistic tool. We are in the process of updating some data including the list of governments in the database. I won’t bore you with all the details but I decided to try out my Python skills.
The task was to update some names of the 94,000 records. It read in a CSV, loaded it into a pandas data frame, selected a subset based on a criteria, steps through the subset making request for JSON encoded data, then output the results into another CSV.
This was a great first real world exercise! Honestly I muddled through it and followed example code found in the documentation for the most part.
But it is nice to see evidence of progress being made. A few of weeks ago if you were to tell me I would be writing Python scripts to help my wrangle some data I would have laughed. But now it has become a reality.
I am also embolden to create some API wrappers for some open data sources, now that I have a working example of how to make and process a JSON request.