This repository contains a language toolchain that processes a simplified Lisp language and outputs FFmpeg’s filtergaphs.
From a notoriously complex and error-prone string-based syntax, the filtergraph is reified into a first-class, composable unit. This in turn enables (dramatic) improvements in creative freedom, programmability, maintainability and reliability.
Note: This is not a wrapper for FFmpeg (or bindings). This is a language solution for a language problem, ie. the excessive information density of FFmpeg’s syntax.
Problem: FFmpeg filtergraphs are fragile string concatenations where a single misplaced comma or bracket can cause cryptic failures.
Solution: Bioscoop represents filtergraphs as immutable data structures:
Instead of "scale=1920:1080,overlay", users write:
(chain (scale {:width 1920 :height 1080})
(overlay))Benefits:
- Type safety: Parameters are validated before execution
- Immutability: Filtergraphs can be safely composed and transformed
- Explicitness: Parameter names are spelled out (compare and contrast with FFmpeg’s positional arguments).
Problem: FFmpeg lacks native composition mechanisms, forcing developers to manually manage complex filter concatenation.
Solution: Built-in composition protocol:
;; Seamless composition
(compose scale-graph overlay-graph crop-graph)Benefits:
- Modular design: Complex pipelines built from simple components
- Reusability: Filtergraphs become first-class composable units
- Testability: Individual components can be tested in isolation
Problem: FFmpeg’s label management is tightly coupled with filtergraphs. Users have to manually label filtergraphs, it is error-prone and difficult to debug.
Solution: Decoupling of labels and filtergraphs.
This is achieved internally by implementing labels as metadata on filters:
(-> (make-filter 'scale {:width 1920 :height 1080})
(with-labels ["in"] ["out"]))User-facing syntax stays close the FFmpeg convention: input labels at the left of a filterchain, output labels at the right.
;; This is equivalent: "[in]scale=1920:1080[scaled]"
[["in"] (scale {:width 1920 :height 1080}) ["out"]]However, because filtergraphs are first-class, we can now write:
(defgraph scaled (scale {:with 1920 :height 1080})) ;; first-class filtergraph, independent of labels
(bioscoop [["in"] scaled ["out"]]) ;; attach labels to filtergraphBenefits:
- Decoupling: Labels and filtergraphs can be handled separately
- Automatic label generation: No more manual label tracking
- Label validation: Prevents duplicate or missing labels
- Metadata preservation: Labels persist through transformations
Problem: FFmpeg filtergraphs are one-way - once created as strings, they can’t be easily analyzed or modified.
Solution: Round-trip transformation capabilities:
;; Parse FFmpeg string to data structure
(def parsed (ffmpeg/parse "scale=1920:1080,overlay"))
;; Modify the data structure
(def modified (update-in parsed [:chains 0 :filters] conj (crop {:width 800})))
;; Render back to FFmpeg string
(to-ffmpeg modified) ; => "scale=1920:1080,overlay,crop=width=800"Benefits:
- Analysis: Programmatically inspect and analyze existing filtergraphs
- Transformation: Modify filtergraphs without string manipulation
- Migration: Update old filtergraph syntax to new patterns
In Ffmpeg, filters take parameters. This is what makes them flexible, expressive and powerful. However, those parameters need to be hard-coded in the filtergraph expression. Not so with bioscoop.
(defgraph transition (xfade {:transition "fade" :duration 1 :offset 9}))
(defn n-transition [n offset] (for [i (range n)]
(-> transition
(update-in [:chains 0 :filters 0 :args] assoc :bioscoop.domain.specs.effects/offset (+ i offset (* i offset)))
(update-in [:chains 0 :filters 0] with-labels [(if (zero? i) (str "out" i) (str "t" i)) (str "out" (inc i))] [(str "t" (inc i))]))))Problem: FFmpeg parameters are validated at runtime, often with unclear error messages.
Solution: Values passed to the filters are validated through specs.
(s/def ::width (s/and int? pos?))
(s/def ::height (s/and int? pos?))
(s/def ::scale (s/keys :req-un [::width ::height]))
;; Validation happens before FFmpeg executionBenefits:
- Early error detection: Catch invalid parameters before FFmpeg runs
- Clear error messages: Know exactly which parameter failed validation
- Documentation: Specs serve as living documentation for filter parameters (type help and the name of the filter to see the spec).
An IEEE Conference paper is available that expounds the concept. In a nutshell, AST convergence is a technique to enable a single transformation on the AST while processing multiple input modalities. This is how Bioscoop manages to be an external DSL and an internal one at the same time. It offers standalone compilation and macro expansion without code duplication. This is achieved by having the macro emitting the same parse tree than the parser.
- External DSL Path: Text → Instaparse Parser → AST →
transform-ast - Internal DSL Path: Clojure Forms → Macro → AST →
transform-ast
In classic Lisp systems, external DSLs would typically use a separate parser (like a PEG parser) while internal DSLs use macros that directly generate target code. The key here is that both paths converge on the same AST structure before the transformation phase.
The Association of Moving Image Archivists (AMIA) provides Open Source resources that support their mission. The following examples were largely inspired by the FFmpeg artschool.
FFmpeg syntax:
"cellauto=rule=110:start_full=false:stitch=true:size=1024x1024[cell];[0:v]format=pix_fmts=yuva420p[img];[cell][img]overlay"Bioscoop program:
(require '[bioscoop.macro :refer [bioscoop defgraph]]
'[bioscoop.built-in])
(defgraph cellular (cellauto {:rule 110 :start_full false :stitch true :size "1024x1024"}))
(defgraph presentation (compose [cellular ["cell"]]
[["0:v" ] (format {:pix_fmts "yuva420p"}) ["img"]]
[["cell"] ["img"] (overlay)]))
(def filtergraph #(to-ffmpeg presentation))FFmpeg syntax:
"[1:v]format=gbrp10le[v1];[0:v]format=gbrp10le[v0];[v1][v0]scale2ref[v1][v0];[v0][v1]blend=all_mode=pinlight,format=yuv422p10le[v]"Bioscoop program:
(require '[bioscoop.macro :refer [bioscoop defgraph]]
'[bioscoop.built-in])
(defgraph formatting (format {:pix_fmts "gbrp10le"}))
(defgraph blending (chain (blend {:all_mode "pinlight"})
(format {:pix_fmts "yuv422p10le"})))
(def filtergraph #(to-ffmpeg (bioscoop (compose [["0:v"] formatting ["v0"]]
[["1:v"] formatting ["v1"]]
[["v1"] ["v0"] (scale2ref) ["s1"] ["s0"]]
[["s0"] ["s1"] blending]))))
Ffmpeg syntax:
"format=yuv420p10le|yuv422p10le|yuv444p10le|yuv440p10le,split=10[b0][b1][b2][b3][b4][b5][b6][b7][b8][b9];[b0]crop=iw/10:ih:(iw/10)*0:0,lutyuv=y=512:u=512:v=512:y=bitand(val\,pow(2\,10-1))*pow(2\,1)[b0c];[b1]crop=iw/10:ih:(iw/10)*1:0,lutyuv=y=512:u=512:v=512:y=bitand(val\,pow(2\,10-2))*pow(2\,2)[b1c];[b2]crop=iw/10:ih:(iw/10)*2:0,lutyuv=y=512:u=512:v=512:y=bitand(val\,pow(2\,10-3))*pow(2\,3)[b2c];[b3]crop=iw/10:ih:(iw/10)*3:0,lutyuv=y=512:u=512:v=512:y=bitand(val\,pow(2\,10-4))*pow(2\,4)[b3c];[b4]crop=iw/10:ih:(iw/10)*4:0,lutyuv=y=512:u=512:v=512:y=bitand(val\,pow(2\,10-5))*pow(2\,5)[b4c];[b5]crop=iw/10:ih:(iw/10)*5:0,lutyuv=y=512:u=512:v=512:y=bitand(val\,pow(2\,10-6))*pow(2\,6)[b5c];[b6]crop=iw/10:ih:(iw/10)*6:0,lutyuv=y=512:u=512:v=512:y=bitand(val\,pow(2\,10-7))*pow(2\,7)[b6c];[b7]crop=iw/10:ih:(iw/10)*7:0,lutyuv=y=512:u=512:v=512:y=bitand(val\,pow(2\,10-8))*pow(2\,8)[b7c]; [b8]crop=iw/10:ih:(iw/10)*8:0,lutyuv=y=512:u=512:v=512:y=bitand(val\,pow(2\,10-9))*pow(2\,9)[b8c];[b9]crop=iw/10:ih:(iw/10)*9:0,lutyuv=y=512:u=512:v=512:y=bitand(val\,pow(2\,10-10))*pow(2\,10)[b9c]; [b0c][b1c][b2c][b3c][b4c][b5c][b6c][b7c][b8c][b9c]hstack=10,format=yuv422p10le,drawgrid=w=iw/10:h=ih:t=2:c=cyan@1"Bioscoop program:
(require '[bioscoop.macro :refer [bioscoop defgraph]]
'[bioscoop.built-in])
(defgraph formatting (chain (format {:pix_fmts "yuv420p10le|yuv422p10le|yuv444p10le|yuv440p10le"})
(split {:outputs 10})))
(defgraph bitplane (chain (crop {:out_w "iw/10" :out_h "ih" :x "(iw/10)*0" :y "0"})
(lutyuv {:y "'bitand(val,pow(2,10-1))*pow(2,1)'" :u "512" :v "512"})))
(defgraph stacking (chain (hstack {:inputs 10})
(format {:pix_fmts "yuv422p10le"})
(drawgrid {:width "iw/10" :height "ih" :thickness "2" :color "cyan@1"})))
(defn n-formatting [n]
(list (-> formatting
(update-in [:chains 0 :filters 1] with-output-labels (into [] (for [i (range n)] (str "b" i)))))))
(defn n-stack [n]
(list (-> stacking
(update-in [:chains 0 :filters 0] with-input-labels (into [] (for [i (range n)] (str "b" i "c")))))))
(defn n-bitplane [n]
(for [i (range n)]
(-> bitplane
(update-in [:chains 0 :filters 0 :args] assoc :bioscoop.domain.specs.crop/x (str "(iw/10)*" i))
(update-in [:chains 0 :filters 1 :args] assoc :bioscoop.domain.specs.lut/y (str "'bitand(val,pow(2,10-" (inc i) "))*pow(2," (inc i) ")'"))
(update-in [:chains 0 :filters 0] with-input-labels [(str "b" i)])
(update-in [:chains 0 :filters 1] with-output-labels [(str "b" i "c")]))))
(def filtergraph
#(to-ffmpeg (bioscoop (let [n 10]
(compose (n-formatting n) (n-bitplane n) (n-stack n))))))Ffmpeg syntax:
"format=gbrp10[formatted];[formatted]split[a][b];[a]lagfun=decay=.99:planes=1[a];[b]lagfun=decay=.98:planes=2[b];[a][b]blend=all_mode=screen:c0_opacity=.5:c1_opacity=.5,format=yuv422p10le[out]"Bioscoop program:
(require '[bioscoop.macro :refer [bioscoop defgraph]]
'[bioscoop.built-in])
(defgraph formatting (chain (format {:pix_fmts "gbrp10"})
(split {:outputs 2})) )
(defgraph fun (lagfun {:decay 0.99 :planes 1}))
(defn n-fun [n]
(for [i (range n)]
(-> fun
(update-in [:chains 0 :filters 0 :args] assoc
:bioscoop.domain.specs.lagfun/decay (/ (- 99 i) 100)
:bioscoop.domain.specs.lagfun/planes (inc i))
(update-in [:chains 0 :filters 0] with-labels [(str "i" i)] [(str "o" i )]))))
(defgraph blending (chain (blend {:all_mode "screen" :c0_opacity 0.5 :c1_opacity 0.6})
(format {:pix_fmts "yuv422p10le"})))
(def filtergraph #(to-ffmpeg (bioscoop (compose [formatting ["i0"] ["i1"]]
(n-fun 2)
[["o0"] ["o1"] blending]))))Note: Instead of the top-level defgraph, Bioscoop also allows for local bindings with a let.
(def filtergraph #(to-ffmpeg
(bioscoop
(let [formatting (chain (format {:pix_fmts "gbrp10"})
(split {:outputs 2}))
blending (chain (blend {:all_mode "screen" :c0_opacity 0.5 :c1_opacity 0.6})
(format {:pix_fmts "yuv422p10le"}))]
(compose [formatting ["i0"] ["i1"]]
(n-fun 2)
[["o0"] ["o1"] blending])))))If you have created something with Bioscoop, please send me a link to your project for inclusion below.
Presentation of photography work with the Ken Burns effect. Click on the image below to play a Youtube video.
While the language proper is feature-complete, no binary is shipping
yet (you can build it yourself). The bioscoop macro wraps the compiler
in a Clojure environment, and exposes the functionality in a REPL. If
you are familiar with Clojure, then your needs are met. If you are not
and wished you could work with the standalone compiler, please let me
know that you are interested. Please consider becoming a sponsor to
voice that interest. Thank you!
- ☑ Bioscoop language toolchain
- ☐ More filters
- ☐ CLI for the standalone compiler (GraalVM binary)