Status

My First Real-World Data Wrangling with Python

Today I was faced with a daunting task.  As previously mentioned, I am the developer behind Govistics, a government spending statistic tool.  We are in the process of updating some data including the list of governments in the database.  I won’t bore you with all the details but I decided to try out my Python skills.

The task was to update some names of the 94,000 records.  It read in a CSV, loaded it into a pandas data frame, selected a subset based on a criteria, steps through the subset making request for JSON encoded data, then output the results into another CSV.

This was a great first real world exercise!  Honestly I muddled through it and followed example code found in the documentation for the most part.

But it is nice to see evidence of progress being made.  A few of weeks ago if you were to tell me I would be writing Python scripts to help my wrangle some data I would have laughed.  But now it has become a reality.

I am also embolden to create some API wrappers for some open data sources, now that I have a working example of how to make and process a JSON request.

Advertisements