v0.3.20210220

General/my.core changes:

• a330567 deprecate my.cfg, instead my.config can (and should be) used directly

• 0534c5c new cli feature: module management

  cli: add hpi module install and hpi module requires

  relevant: #12, #79

• 97650ad add discovery_pure module to get modules and their dependencies via ast module

• f90599d make module discovery rely on =ast= module
  Hopefully it will make it more robust & much faster.

• 07f901e helpers for automatic dataframes from sequences of NamedTuple/dataclass

• 4012f9b more generic functions to jsonify data

• 746c3da core.pandas: allow specifying schema; add tests

• 5313984 add tmp_config context manager for test & adhoc patching

• df9a7f7 core.pandas: add check for 'error' column + add empty one by default

• e81dddd proprely resolve class properties in make_config + add test

Modules:

• some innitial work on filling InfluxDB with HPI data

• pinboard

  • 42399f6: breaking backwards compability, use pinbexport module directy

    Use 'hpi module install my.pinboard' to install it

    relevant: #79

• stackexchange

  • 63c825a use GDPR data for votes
  • ddea816 use proper pip package, add stat

• bluemaestro

  • 6d9bc29 populate grafana
  • 1899b00 investigation of data quality + more sanity checks
  • d77ab92 get rid of unnecessary file, move to top level

• runnerup

  • 6b45133 Initial parser for RunnerUp data which I'm now using instead of Endomondo

Misc:

• f102101 core/windows: fix get_files and its tests

• 56d5587 CI: clean up tox config a bit, get rid of custom lint script

• d562f00

  tests: run all tests, but exclude tests specific to my computer from CI
  controllable via HPI_TESTS_KARLICOSS=true

• improved mypy coverage

General:

• documentation improvements, my.config in the package now serves both as documentation & tests helper
• my.kython: deprecated, moved to my.core
• my.core.error: error handling enhancements
• stat now supports pandas.DataFrame
• minor enhancements to hpi doctor

Module changes:

• new: my.time.tz: provides localize function to attach a timezone to an arbitrary date #96
• my.location: exposes a combined log of user's location, with fallback to 'default' home locations #95 #99
• my.fbmessenger: switch to use a proper PIP package
• new: my.taplog data provider

Misc: more tests & mypy running on CI

See commit changelog for more details.

Special thanks to @seanbreckenridge for discussions!

core:

• core config section, allow enabling/disabling modules
• better module detection
• experimental helpers to supply errors with datetime information
• experimental helpers to make it easier to override config, e.g. see https://github.com/karlicoss/dashboard/blob/ff15295d8118494f986ed13757e022ae2aa18000/src/dashboard/data.py#L99-L117 for example usage
• nicer warnings
• initial core.pandas helpers

cli: hpi doctor enhancements

modules:

• big update for instapaper, reddit, ghexport, hypothesis, rescuetime, pocket.
  DEPRECATED: the old way of specifying dependecies either as

  • config parameters (e.g. hypexport = "/path/to/hypexport_on_disk")
  • symlinks in the config (e.g. my/config/repos/hypexport)

  These dependencies are proper PIP packages now, see #79 for the discussion about the change
  This is backwards compatible, but please reinstall them properly as soon as you can (see the updated instructions in the corresponding repos, e.g. https://github.com/karlicoss/instapexport#setting-up)

• provide fake data providers for some modules

• better mypy coverage for some modules

• for other minor updates, see the commit messages

• core helper to open databases as read-only/immutable (useful for Docker mounts)
• some cachew experiments, including cachew base directory (still work in progress)
• more stat functions and improvements for stats
• initial work on using proper virtualenv for DAL helpers (using emfit to start with)

modules affected:

• reddit
• pdfs
• bluemaestro
• emfit
• twint
• rescuetime

core:

• support '' for "empty paths"
• helper for computing stats, add to more modules

github:

• deprecated my.coding.github, see ca39187
• support data arbitration from GDPR and ghexport, see d7aff1b

kobo: see #62

• breaking: kobuddy should be properly installed (instead of old my.config.repos.kobuddy)
• breaking: export_dir renamed to export_path for consistency,

pocket:

• now uses pockexport for the data, see f175acc