wiretap | ˈwʌɪətap |
[noun]
A concealed device connected to a telephone or other communications system that allows a third party to listen or record conversations.
[verb]
To install or to use such a device.
This library provides a small set of tools that help you to observe the execution of functions and multimethods. It is designed to be used in (dev) environments where you want to gain insights without having to modify/annotate code.
As a git dep:
io.github.beoliver/wiretap {:git/sha "12a3640"}As a Maven dep:
io.github.beoliver/wiretap {:mvn/version "0.0.20"}The simplest way to use wiretap is with the record namespace:
(require '[wiretap.record :as rec])
;; Start recording all functions in a namespace
(def recorder (rec/start! {:globs ["my-app.core"]}))
;; Run your code
(my-app.core/some-function 42)
;; Play back the formatted trace
(rec/playback recorder)
;; Access raw events for analysis
(rec/events recorder)
;; Clean up when done
(rec/stop! recorder)Glob patterns use * and ** wildcards to match namespaces:
;; * matches a single namespace segment
(rec/start! {:globs ["my-app.*.utils"]})
;; Matches: my-app.core.utils, my-app.web.utils
;; Does NOT match: my-app.utils, my-app.core.web.utils
;; ** matches zero or more segments
(rec/start! {:globs ["my-app.**"]})
;; Matches: my-app.core, my-app.core.utils, my-app.web.handlers.auth
;; Record multiple patterns
(rec/start! {:globs ["my-app.core" "my-app.util.*"]})
;; Record specific functions
(rec/start! {:vars [#'my-app.core/foo #'other.ns/bar]})
;; Combine both approaches
(rec/start! {:globs ["my-app.**"]
:vars [#'other.ns/helper]})(def recorder (rec/start! {:globs ["my-app.**"]}))
;; Run some code to generate events
(my-app.core/process-data {:user-id 123})
;; Display formatted trace output
(rec/playback recorder)
;; TRACE t123: (my-app.core/process-data {:user-id 123})
;; TRACE t124: | (my-app.db/fetch-user 123)
;; TRACE t124: | => {:id 123 :name "Alice"}
;; TRACE t123: => {:processed true}
;; Access events directly for custom analysis
(def events (rec/events recorder))
;; Filter for specific functions
(->> (rec/events recorder)
(into []
(comp (filter :post?)
(filter #(= 'fetch-user (:name %)))
(map :result))))
;; Clear recorded events (keeps recording)
(rec/wipe! recorder)
;; Stop recording and restore original functions
(rec/stop! recorder)Wiretap works by wrapping function calls. Any var whose value is an instance of Fn or MultiFn (created via fn or defmulti) can be wiretapped.
As a user of the library, you provide a function that will be called before and/or after any wiretapped function or method is invoked. This function can be used to perform any side effecting operations - for example swapping values in an atom, or simply calling println. This custom function is passed a context map that contains information about the invocation of the var. For more information see wiretap context.
(ns user)
(defn foo [x] (inc x))
(defn bar [x] (foo (dec x)))user=> (require '[wiretap.wiretap :as wiretap])
nil
user=> (wiretap/install! #(println (if (:pre? %) ">" "<") (:name %)) [#'foo #'bar])
(#'user/foo #'user/bar)
user=> (+ 10 (bar 1))
> bar
> foo
< foo
< bar
11Depending on the type of the var value, different strategies are used to wrap the invocation of the var. The following table shows the supported types and the corresponding wrapping strategy.
| Instance | wiretap |
|---|---|
Fn |
Wraps invocation of the function |
MultiFn |
Wraps invocation of the selected method determined by the dispatch-value |
Depending on whether the user provided function is called before or after the invocation of the wiretapped var, the context map will contain different information. The following table shows the keys that will be present in the context map and when they will be present.
| Key | When | Value |
|---|---|---|
:id |
pre/post | Uniquely identifies the call. Same value for pre and post calls. |
:name |
pre/post | A symbol. Taken from the meta of the var. |
:ns |
pre/post | A namespace. Taken from the meta of the var. |
:function |
pre/post | The value that will be applied to the value of :args. |
:thread |
pre/post | The name of the thread. |
:stack |
pre/post | The current stacktrace. |
:depth |
pre/post | Number of wiretapped function calls on the stack. |
:args |
pre/post | The seq of args that value of :function will be applied to. |
:start |
pre/post | Nanoseconds since some fixed but arbitrary origin time. |
:parent |
pre/post | The context of the previous wiretapped function on the stack. |
:pre? |
pre | true |
:post? |
post | true |
:stop |
post | Nanoseconds since some fixed but arbitrary origin time. |
:result |
post | The result of applying the value of :function to :args. |
:error |
post | Any exception caught during computation of the result. |
If the wiretapped var is a multimethod then the following information will also be present.
| Key | When | Value |
|---|---|---|
:multimethod? |
pre/post | true |
:dispatch-val |
pre/post | The dispatch value used to select the method. |
For most use cases - simple recording and playback:
(start! {:globs [...] :vars [...]})- Start recording matching functions(events recorder)- Get vector of recorded events(playback recorder)- Display formatted trace to console(wipe! recorder)- Clear recorded events (keeps recording)(stop! recorder)- Stop recording and restore original functions
Returns a "recorder" map with :vars and :events keys.
For custom tracers and advanced use cases:
(install! tracer-fn vars)- Install custom tracer on vars(install-pre! tracer-fn vars)- Tracer called only before invocation(install-post! tracer-fn vars)- Tracer called only after invocation(uninstall! vars)- Remove wiretap from vars(uninstall!)- Remove all wiretaps
Your tracer function receives the context map on each call.
Helper functions for working with vars and traces:
(glob-regex pattern)- Convert glob string to regex Pattern(ns-matches regex)- Find namespace symbols matching regex(ns-matches-vars regex)- Find vars in matching namespaces(globs-vars glob-patterns)- Find vars matching glob patterns(ns-vars & namespaces)- Get all vars from namespaces(wiretapped? var)- Check if var is currently wiretapped(display-trace events)- Format and print event sequence
Documentation hosted here
Assume that we have a multimethod
(defmulti m1 :name)
(defmethod m1 :foo [x] {:the-foo x})
(defmethod m1 :bar [x] {:the-bar x})We can wiretap this as follows
user=> (wiretap/install!
#(when (:pre? %)
(println (select-keys % [:name :dispatch-val :args]))) [#'m1])
(#'user/m1)
user=> (m1 {:name :foo})
{:name m1, :dispatch-val :foo, :args ({:name :foo})} ;; printed line
{:the-foo {:name :foo}}
user=> (m1 {:name :bar})
{:name m1, :dispatch-val :bar, :args ({:name :bar})} ;; printed line
{:the-bar {:name :bar}}However, if we add a new method then it will not be wiretapped.
user=> (defmethod m1 :baz [x] {:the-baz x})
#multifn[m1 0x2f3911be]
user=> (m1 {:name :baz})
{:the-baz {:name :baz}}The methods that were wiretapped, remain wiretapped.
user=> (m1 {:name :foo})
{:name m1, :dispatch-val :foo, :args ({:name :foo})} ;; printed line
{:the-foo {:name :foo}}When uninstalling, the wiretapped methods are replaced with the original ones.
user=> (wiretap/uninstall! [#'m1])
(#'user/m1)
user=> (m1 {:name :foo})
{:the-foo {:name :foo}}
user=> (m1 {:name :bar})
{:the-bar {:name :bar}}
user=> (m1 {:name :baz})
{:the-baz {:name :baz}}Assume that we have the following namespace definitions...
(ns user)
(defn simple [x] (inc x))
(defn call-f [f x] (f x))
(defn pass-simple [x] (call-f simple x))To show how wiretap events can be used - we will generate traces similar to those of the clojure/tools.trace library. All we need to do is write a function that can take a wiretap context map and perform some io (call to println).
(defn ^:wiretap.wiretap/exclude my-trace
[trace-id-atom {:keys [id pre? depth name ns args result] :as ctx}]
(let [trace-id (if pre? (gensym "t") (get @trace-id-atom id))
trace-indent (apply str (take depth (repeat "| ")))
trace-value (if pre?
(str trace-indent (pr-str (cons (symbol (ns-resolve ns name)) args)))
(str trace-indent "=> " (pr-str result)))]
(if pre?
(swap! trace-id-atom assoc id trace-id)
(swap! trace-id-atom dissoc id))
(println (str "TRACE" (str " " trace-id) ": " trace-value))))To make things interesting - we will persist all of the contexts and then run our trace function on the data. Repeatable traces!
user=> (require '[wiretap.tools :as tools])
nil
user=> (def history (atom []))
#'user/history
user=> (wiretap/install! #(swap! history conj %) (tools/ns-vars *ns*))
(#'user/pass-simple #'user/simple #'user/call-f)
user=> (pass-simple 1)
2
user=> (count @history)
6
user=> (run! (partial my-trace (atom {})) @history)
TRACE t8018: (user/pass-simple 1)
TRACE t8019: | (user/call-f #function[clojure.lang.AFunction/1] 1)
TRACE t8020: | | (user/simple 1)
TRACE t8020: | | => 2
TRACE t8019: | => 2
TRACE t8018: => 2
nilOr using the record namespace:
user=> (require '[wiretap.record :as rec])
nil
user=> (def recorder (rec/start! {:globs ["user"]}))
#'user/recorder
user=> (pass-simple 1)
2
user=> (rec/playback recorder)
TRACE t8021: (user/pass-simple 1)
TRACE t8022: | (user/call-f #function[clojure.lang.AFunction/1] 1)
TRACE t8023: | | (user/simple 1)
TRACE t8023: | | => 2
TRACE t8022: | => 2
TRACE t8021: => 2
nilNow that we have a history of events, we can perform other operations on them! In the previous example we called pass-simple passing the value 1. Let's use the spec-provider library to infer some specs from the trace.
(require '[spec-provider.provider :as sp])
(defn result-spec [history var-obj]
(let [var-ns (:ns (meta var-obj))
var-name (:name (meta var-obj))
examples (->> history
(filter (fn [{:keys [post? error ns name]}]
(and post?
(nil? error) ;; ignore results if error thrown
(= ns var-ns)
(= name var-name))))
(map :result))]
(sp/pprint-specs
(sp/infer-specs (set examples) (keyword (name (ns-name var-ns))
(name var-name)))
var-ns 'spec)))We can now use the function to infer the spec of the return value for a function - even if we never called it directly.
=> (result-spec @history #'simple)
(spec/def ::simple integer?)
=> (call-f simple 2.0)
3.0
=> (result-spec @history #'simple)
(spec/def ::simple (spec/or :double double? :integer integer?))Or using the record namespace:
=> (result-spec (rec/events recorder) #'simple)
(spec/def ::simple integer?)- https://github.com/clojure/tools.trace
- https://github.com/circleci/bond
- https://github.com/alexanderjamesking/spy
- https://github.com/technomancy/robert-hooke
clj -X:test
clj -Sdeps '{:deps {wiretap/wiretap {:git/url "https://github.com/beoliver/wiretap/" :git/sha "12a3640"}}}' -e "(require '[wiretap.wiretap :as wiretap] '[wiretap.tools :as wiretap-tools])" -r
Checking out: https://github.com/beoliver/wiretap/ at 12a3640994ff8241cdd38995d16c481bcf143c53
WARNING: Implicit use of clojure.main with options is deprecated, use -M
user=> (def foo (fn [x] (+ x x)))
#'user/foo
user=> (wiretap/install! #(when (:post? %) (clojure.pprint/pprint %)) [#'foo])
(#'user/foo)
user=> (foo 10)
{:args (10),
:parent nil,
:ns #object[clojure.lang.Namespace 0x309028af "user"],
:name foo,
:start 298028669941875,
:function #object[user$foo 0x44841b43 "user$foo@44841b43"],
:stop 298028670197791,
:result 20,
:thread "main",
:post? true,
:id "7bd32775-d675-48ab-8d42-c56924ed7ee3",
:stack
[[java.lang.Thread getStackTrace "Thread.java" 1602],
[wiretap.wiretap$wiretap_var_BANG_$wiretapped__149 doInvoke "wiretap.clj" 17],
[clojure.lang.RestFn applyTo "RestFn.java" 137],
[clojure.lang.AFunction$1 doInvoke "AFunction.java" 31],
[clojure.lang.RestFn invoke "RestFn.java" 408],
[user$eval223 invokeStatic "NO_SOURCE_FILE" 1],
[user$eval223 invoke "NO_SOURCE_FILE" 1],
[clojure.lang.Compiler eval "Compiler.java" 7194],
[clojure.lang.Compiler eval "Compiler.java" 7149],
[clojure.core$eval invokeStatic "core.clj" 3215],
[clojure.core$eval invoke "core.clj" 3211],
[clojure.main$repl$read_eval_print__9206$fn__9209 invoke "main.clj" 437],
[clojure.main$repl$read_eval_print__9206 invoke "main.clj" 437],
[clojure.main$repl$fn__9215 invoke "main.clj" 458],
[clojure.main$repl invokeStatic "main.clj" 458],
[clojure.main$repl_opt invokeStatic "main.clj" 522],
[clojure.main$repl_opt invoke "main.clj" 518],
[clojure.main$main invokeStatic "main.clj" 664],
[clojure.main$main doInvoke "main.clj" 616],
[clojure.lang.RestFn applyTo "RestFn.java" 137],
[clojure.lang.Var applyTo "Var.java" 705],
[clojure.main main "main.java" 40]],
:depth 0}
20
user=> (wiretap/uninstall!)
(#'user/foo)
user=> (foo 20)
40