Using requests Module to Search FRED Metadata

notr

FRED data with requests¶

The FRED API is so easy to use that finding what series to read is the hard part. There are hundreds of thousands of series, too many even to list in a simple html table. API calls through the requests module can help.

For pulling individual series, we do not need an API key, so it's simple for anyone to copy a notebook and follow along. For pulling metadata, a key is required. There is a key in the notebook.

So many FRED sequences have non-farm descriptions that it made me wonder what series they have for farms, so we take a look at that.

Imports¶

In [31]:

# imports
import pandas as pd
import requests 
import json 

# CONST
fredKey = "see notebook" # api key for notebook

requests.get¶

The requests.get method takes a string argument. The string has the following components connected with anpersands

• base tags search url and tag to look for
• api key
• file_type=json

The decode method for the content attribute from that output is a json string we'll interrogate.

In [18]:

reqS = "&".join(("""https://api.stlouisfed.org/fred/tags/series?tag_names=%s
api_key=%s
file_type=json""" %("farm", fredKey)).split("\n"))
respS = requests.get(reqS).content.decode()
len(respS)

Out[18]:

573397

The string from the request can be sent to json.loads to obtain a dictionary.

In [32]:

tD = json.loads(respS)
tD.keys()

Out[32]:

dict_keys(['realtime_start', 'realtime_end', 'order_by', 'sort_order', 'count', 'offset', 'limit', 'seriess'])

It's the "seriess" entry we are interested in, and that is a list of dictionaires.

In [33]:

seriesL = tD["seriess"]
len(seriesL)

Out[33]:

838

So there are over 800 series with "farm" in the tags. Just browsing one shows a dictionary.

In [34]:

seriesL[0]

Out[34]:

{'frequency': 'Annual',
 'frequency_short': 'A',
 'group_popularity': 1,
 'id': 'A0282AUSA398NNBR',
 'last_updated': '2012-08-16 11:38:36-05',
 'notes': 'Series Is Presented Here As 3 Variables: (1) Original Data, 1915-46; (2) Original Data 1946-63; (3) Seasonally Adjusted Data, 1939-46. Included In This Series Are: (1) New Housing Units And Additions And Alterations To Existing Units On Places Classified As Farms, According To The 1960 Census Definition; And (2) Other Buildings And Structures Used In Farm Production, Such As Barns, Storage Houses, Smoke Houses, Fences, Wells, Etc. Excluded Are Operations Which Are An Integral Part Of Farming, Such As Plowing, Terracing, And Digging Of Drainage Ditches. For Further Information On Sources, Collection Procedures, And Coverage, See Introductory Notes On Bls-Dept. Of Commerce Series, And Lipsey & Preston, Pp. 263-264. Source: Business & Defense Services Administration And Bureau Of Labor Statistics Publications (See Note).\n\nThis NBER data series a02182a appears on the NBER website in Chapter 2 at http://www.nber.org/databases/macrohistory/contents/chapter02.html.\n\nNBER Indicator: a02182a',
 'observation_end': '1946-01-01',
 'observation_start': '1915-01-01',
 'popularity': 1,
 'realtime_end': '2018-05-28',
 'realtime_start': '2018-05-28',
 'seasonal_adjustment': 'Not Seasonally Adjusted',
 'seasonal_adjustment_short': 'NSA',
 'title': 'Private Farm Construction for United States',
 'units': 'Millions of Current Dollars',
 'units_short': 'Mil. Of Current $'}