MacroEcon-WrapPy

Economics Wrapper for data across multiple domains and sources (FRED, BEA, BLS, IPUMS, etc.). The structures allow for smooth workflows for data ingest, transformations, graphing, and modeling.

This module is used independently of any particular API-wrapper. However, some wrappers are suggested and used in the examples. An API-wrapper Adapter interface is available for improved integration with this module's primary classes. Where no API-wrapper functionality exists, such as directly downloading files or site scraping, this module makes direct connections to a site by implementing the Extractor interface.

Install and Configure

Create a file (SECRETS.yaml) with all appropriate keys and values.

pip install macroecon_wrappy

Usage

Select a source and associated wrapper. For this example, we will use the fredapi wrapper with the FRED data source. The Adapter is provided for this wrapper, but you can create your own if you use a different API-wrapper.

Set the authentication key for the wrapper.

Add external libraries using the --dev argument, such as:

uv add --dev waybackpack

<TODO: add from notebooks/test.ipynb>

Using the FredApi Adapter enables seemless integration with the rest of the module, in particular, incorporating data series into Measures.

<TODO: add from notebooks/test.ipynb>

Work directly with the Measure for various tasks and transformations.

Test

uv run pytest --collect-only

Develop

Its often best to work with the notebook while quickly developing.

uv pip install -e .

Then simply import into the notebook

import macroecon_wrappy as me

But ensure to remove it when done:

uv pip uninstall macroecon_wrappy

Provisioned API-Wrappers

These API-wrappers are supported by this module with Adapters:

• fredapi
• treasury_fiscaldata: us-federal-treasury-python-api
  • requires install with pip install ./test_modules/us-federal-treasury-python-api/.
• internet_archive
• yfinance

API Structures

• Relation
  • multiple Measures, Epochs, Events related using a mathematical expression
• Measure line on a graph
  • (ie consumer inflation) contains multiple Metrics (cpi, core cpi, pce, bpp, etc.)
  • list of Metrics
  • Metric - continuous timeseries data
    • pd.Series with metadata
• Epoch polygon on a graph
  • (ie recessions) contain multiple Epochs (expansion, contraction, etc.)
  • pd.DataFrame of Epochs with metadata
  • Span - window of time
    • single row of pd.DataFrame with metadata
• Event dot on a graph
  • (ie fall of enron) contains multiple Points (news articles, blog posts, anecdotes, etc.)
  • pd.DataFrame of Points with metadata
  • Point in time (ie news article)
    • single row of pd.DataFrame with metadata
• Geom abstract structure with metadata

Adapter - interacts with api wrapper to get data Extractor - interacts with site to get files and extract data

Roadmap

• Span of duration_days == 0 is Event
• Adapters and Extractors should enforce output of
  • ?list of dicts,
  • with opinionated methods for how to use with data structures (Span, Metric, etc.)
  • return raw structure (dict, pd.DataFrame)
  • return macroecon structure (Measure, Epoch, Event, Geom)
  • enable caching of data using module, top-level config, and work with api-wrappers' own caching
• what is load.py doing? dependencies: RM_sources
• awesome quant

Data sources: Adapter, Extractor

• various APIs
  • useful sources: Energy Information Administration (EIA), treasury
  • additional: IMF, census, UN Comm(odity)Trade,
• datareader
  • list of sources
  • free api: enigma, econdb
  • useful sources: tsp, worldbank, oecd, eurostat, fama/french, ...
• internet archive (Adapter)
  • need better text integration with plots
• hindenburg
  • headline: https://hindenburgresearch.com/
• treasury, additional frb sites
  • example: https://github.com/bdecon/econ_data/blob/master/APIs/Treasury.ipynb
  • treasury home: https://home.treasury.gov/policy-issues/financing-the-government/interest-rate-statistics
• macromicro / trading economics
  • jpm global pmi: https://en.macromicro.me/series/20151/jp-morgan-global-manufacturing-pmi
  • jpm global pmi: https://tradingeconomics.com/world/manufacturing-pmi
• quandl / nasdaq data link
  • some is free
  • api wrapper: https://github.com/Nasdaq/data-link-python
  • data: https://data.nasdaq.com/search?filters=%5B%22Free%22%5D

Functionality: graphs, digitizers, transformations, cache / storage, etc.

• finplot
• fixed income
  • rates
  • financepy
  • book code
• equities
  • pandas-ta
    • solve issue
  • ta-lib
• economics
  • econ
  • online econ
• altair graphs
  • repo: https://github.com/vega/altair
  • candlestick
  • deep understanding of candlestick
  • legend select
  • side charts
  • update after creation
  • ~~shaded regions and recession bars: ~~
    • vertical
    • horizonal
    • bars
    • discussion
• ggplot, plotnine
  • multiple y-axes
  • ref
  • ref
• metric file storage, PyStore
  • update with pyarrow
  • original: https://aroussi.com/post/fast-datastore-for-pandas-time-series-data
• Wrapper adapters and source extractors
• plot digitizers
  • https://github.com/echemdb/svgdigitizer
  • https://github.com/peterstangl/svg2data
  • https://github.com/dilawar/PlotDigitizer