The developers of RStudio have created this wonderful package called shiny. It allows you to create little apps using R. You can deploy your app on a server or used their cloud based service. I played around with it and have created my first app and have chosen to use the cloud based solution. My first app is the BEA Data Explorer.
Since shiny allows for visualization of the data I think it is a great tool when you are exploring a data set for the first time. It helps you to get a feel for the data. This app allows me to look at the BEA data for metros. I can easily filter the data, view the relationships between two variables, and even cluster them using k-means. I would love to hear what you think of my app!
After a short delay in grading I received notification that I have completed the Getting and Cleaning Data Course.
This class explained how to pull in data from a variety of formats (i.e. excel, XML, JSON, MySQL, etc.). It introduced me to the data.table package in R which is awesome. It was not always taught in a very clear way.
I was taking Core Concepts in Data Analysis at the same time which was very challenging. That is why my blog has fell silent for the last little bit (because I was too busy to write). I am waiting to see how that class turns out.
I have been using the skills gained in this class to write scripts to pull data from the BEA’s API using R. Hopefully I will have more to share on this in a later post.
I have begun writing a script that I would like to pass on. It pulls in GDP by MSA data using the BEA’s API into R. This is the beginning of an analysis I am hoping to preform. Hopefully others find this helpful.
# Load the needed libraries
# Set my BEA API key
bea.api.key <- "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
bea.api.url <- "http://www.bea.gov/api/data"
# Read in the FIPS codes for the MSA's
msa = read.csv("bea_msa_fips.csv",head=TRUE,sep=",")
fips = msa$FIPS
# Create data frame to hold the API results
df <- data.frame()
# Loop through the MSA's to request the MSA GDP
for (i in 1:length(fips))
# Make the request to the API
req <- paste("http://www.bea.gov/api/data/?&UserID=", bea.api.key, "&method=GetData&datasetname=RegionalData&KeyCode=GDP_MP&GeoFIPS=", fips[i], "&ResultFormat=json", sep="")
# Parse the JSON response
res <- fromJSON(getURL(req))
# Since there are multiple years in the response we need to loop through them
for (j in 1:length(res$BEAAPI$Results$Data))
# Append the API result to the data frame
r <- as.data.frame(res$BEAAPI$Results$Data[j])
df <- rbind(df,r)
msa.gdp <- dcast(df, GeoName + GeoFips ~ TimePeriod, value.var="DataValue")
If you are interested in using this you will need this csv.
I have been busy working on the BEA API Python wrapper when I came across a great blog post – R vs Python Round 3. This blog post introduced me to Quandl, which may deliver all the economic data I could ever want. I haven’t had the time to really dig in. We’ll see if it is needed.
I have also been toying with R. I do this seasonal adjustment of employment data every month at work. I would love to move the work out of SPSS and into R. Especially since R can integrate with our MySQL data warehouse.
I have been busy putting my budding Python skills to work. I am making a wrapper for the BEA API. I have some developed some rudimentary features. An API request is loaded into a dictionary. You call the get_time_series() method and it will return a dictionary with values of the time series data.
I am programming it for Python 3, since this is probably the direction the language is going. Once I add in some more functions and clean up the code a bit I’ll be happy to share it with all. I will then move on to a BLS and a FRED wrapper.
The Bureau of Economic Analysis (or BEA) has an API. They are the people who come up with the GDP estimates (among other things). You can register for an API key, and read the documentation (their user guide is quite good). The URI is http://www.bea.gov/api/data. You can get data in JSON and XML formats.
Once I become more proficient with Python, I plan on programming a wrapper for this API. I will announce when that project is complete here.