This release adds a number of performance upgrades, and completes the port of all code in the mechanics package of the original scmutils library.

This will also be the final release before extracting most of the code in the library out and giving it a new identity and home as the "Emmy Computer Algebra System", housed at https://github.com/mentat-collective/emmy.

New Code / Upgrades

• #531:

  • Adds proper self-require forms to all cljc namespaces with macros; this removes the need for :include-macros true or :refer-macros and makes the life of consumers simpler.

  • Upgrades sci, test.check, core.match, timbre, clojurescript, shadow-cljs dependencies to remove various warnings.

  • Adds sicmutils.structure/symbol-set

  • Adds IHash implementations for js/BigInt, goog.math.Long, goog.map.Integer and Ratio types

• #514:

  • Modifies sicmutils.calculus.derivative/taylor-series to return a proper PowerSeries instance, which the user can call with some dx to get back the old behavior.

    The new version can take any number of arguments in addition to f.
    Supplying no arguments returns the expansion at 0; if you supply many
    arguments (totally fine!), you'll need to wrap your dx components in a
    vector before supplying them to the returned PowerSeries.

  • sicmutils.series/function-> works the same way now, and functions identically, but with a different implementation. (previously it took a single expansion point under a keyword argument :x0.)

  • The new sicmutils.calculus.derivative/symbolic-taylor-series is a port of Taylor-series-coefficients from scmutils. It has the same contract as taylor-series, except that the full expansion is performed symbolically, and the original arguments are substituted in after expansion and simplification.

  • Other changes:

    • Installs 1 as the one-like and identity-like return values for structures and vectors. A true identity element would be an identity element compatible with all entries of the structure; but as defined now, 1 is a fine choice and matches the scmutils implementation.

    • new sicmutils.differential/map-coefficients, which makes simplify slightly more efficient by filtering terms .

    • more efficient sicmutils.expression/variables-in, maybe 30% faster for big expressions; this makes a difference in the simplifier!

    • sicmutils.expression/substitute now works for proper Literal instances. Before it only worked for unwrapped literals.

    • matrix walks made slightly faster by caching a row or column before traversal

• #512:

  • adds sicmutils.mechanics.routhian, with implementations of Lagrangian->Routhian, Routh-equations, Routhian->acceleration, Routhian->state-derivative, Lagrangian-state->Routhian-state and Routhian-state->Lagrangian-state.

  • adds missing sicmutils.mechanics.{routhian,time-evolution,noether} to sicmutils.env.sci

• #509:

  • Fixes a bug with down*Matrix multiplication, and adds tests for correctness.

  • Adds sicmutils.matrix.{symmetric?,antisymmetric?} predicates

  • The mechanics port continues with sicmutils.mechanics.rigid:

    • T-rigid-body moves to T-body-Euler with an alias back to its original name. Same situation for Euler-state->L-body => L-body-Euler and Euler-state->L-space => L-space-Euler.

    • New functions: three-vector-components->antisymmetric, T-body, L-body, L-space, Euler->omega, Euler->omega-body, quaternion-state->omega-body, quaternion-state->omega-space, qw-state->L-body, qw-state->L-space, T-quaternion-state

• #511 focuses on adding more rotations and efficiency to sicmutils.quaternion. Specifically:

  • magnitude-sq and magnitude are now more efficient.

  • New functions to get to and from quaternions and various matrix representations: from-rotation-matrix, ->rotation-matrix, from-complex-matrix, ->complex-matrix, from-4x4-matrix, ->4x4-matrix

  • New instances ONE-matrix, I-matrix, J-matrix, K-matrix, matrix representations of the corresponding quaternion elements.

  • Similar to the matrix elements, we also now have ONE-tensor, I-tensor, J-tensor, K-tensor.

• #503:

  • adds sicmutils.mechanics.lagrange/Lagrangian for building function signatures of Lagrangians.

  • adds the sicmutils.mechanics.time-evolution namespace

  • adds sicmutils.mechanics.lagrange/L-axisymmetric-top, more efficient than the version in sicmutils.examples.top

  • Fleshes out sicmutils.mechanics.hamilton:

    • New functions: H-state?, compatible-H-state?, state->p, momenta, P, literal-Hamiltonian-state, L-state->H-state, H-state->L-state, H-state->matrix, matrix->H-state, make-Hamiltonian, D-phase-space, Hamiltonian->Lagrangian-procedure, Hamiltonian->Lagrangian, flow-derivative, flow-transform, standard-map-inverse, F->K, J-func, T-func, canonical-H?, canonical-K?, linear-function->multiplier, Phi, Phi*, qp-canonical?, polar-canonical-inverse, two-particle-center-of-mass , two-particle-center-of-mass-canonical, transpose-function, multiplicative-transpose, symplectic-two-form, canonical-transform?, J-matrix, symplectic?

    • F->CH moves to F->CT (F->CT is now an alias)

    • Legendre-transform-fn becomes Legendre-transform-procedure and gains more correctness tests, toggled on and off by the *validate-Legendre-transform?* dynamic variable.

• #508 adds sicmutils.mechanics.noether namespace, with Noether-integral.

• #506 tidies up the build by removing unneeded reader conditionals and replacing renames like core-= with a proper require of clojure.core.

• #502 begins the port of the remaining items in the scmutils mechanics package over the Clojure. This PR focuses on sicmutils.mechanics.lagrange, which contains functions from many files in the original mechanics folder.

  • momentum-tuple moves here from sicmutils.mechanics.hamilton

  • New functions ->L-state, ->local, ->state,state->n-dof, time, state->{q,qdot,qddot}, coordinates, velocities, accelerations, Q, Qdot, Qdotdot, literal-Lagrangian-state path->state-path (alias for the existing Gamma), Rayleigh-dissipation, qv->local-path, Lagrange-equations-first-order, (with Lagrange-equations-1 alias), Lagrangian->power-loss, T3-spherical, L3-central, Dt-procedure and the wrapping operator Dt, Euler-lagrange-operator (with Lagrange-equations-operator and LE aliases), generalized-LE.

  • Many of these are aliased into sicmutils.env. Ask if you think more should be there!

  • many new built-in Lagrangians: L-Kepler-polar, L-coupled-harmonic, L-sliding-pend, L-pendulum, L-two-particle

  • Lagrange-equations, Lagrangian->acceleration, Lagrangian->state-derivative now take a dissipation function

  • local-state-derivative aliases the 1-arity version of Lagrangian->state-derivative

  • New rectangular->polar, polar->rectangular, spherical->rectangular and rectangular->spherical that operate on coordinates, with associated r->p (new), p->r, s->r and r->s (new).

• #501 moves elliptic-integrals from sicmutils.special.elliptical-test sicmutils.special.elliptical, as it's needed by the upcoming sicmutils.mechanics.pendulum namespace.

Bug Fixes

• #531:

  • Fixes a typo in one of the rules in sicmutils.simplify.rules/expand-multiangle, closing #530

  • Forces the subexpression walk in pattern.rule/try-subexpressions with doall

  • Adds type hints to all remaining Complex calls in js, to fix issues with advanced compilation (thanks to @mhuebert for finding this)

  • #515:

• tidies up the square and cube speedups thanks to a tip from GJS

• Converts more (mul x x) to square in the derivatives of the sicmutils.generic namespace.

• fixes a bug in the numeric-zero? check of exponent's derivative.

• #503:

  • Changes the default implementation of square and cube for differentials to use (expt <> 2) etc instead of (* <> <>).

    This is a big deal! For certain expressions there's a huge blowup when you square a big symbolic term, and taking the derivative of it TWICE is very messy.

    With this change, differentials use the chain rule to calculuate the derivative of $x^2$ as $2x*x'$, instead of using the product rule and achieving a SECOND differentiatation of the same form and another multiplication: $xx' + x'x$.

    Before a judicious simplify call I added, this change dropped the runtime of the sicmutils.sicm.ch3-test suite down by 6x. After the simplify change in sicmutils.examples.top the tests were still 40% faster in that namespace.

  • Fixes a bug where the RationalFunction cube implementation actually called square.

Misc

• #532:

  • Removes the potemkin dependency by importing only what we need directly into sicmutils.util.def. This makes sense since our versions add a fork call so that they work for ClojureScript as well.

  • Moves all examples into the tests so that we don't ship them with the library. These will eventually be converted to Clerk notebooks.

  • Removes the hiccup dependency.

  • Upgrades test.chuck and removes all :include-macros true calls for that library. Only same/ish requires them now!

  • Capitalizes the "Script" in ClojureScript everywhere it appears.

• #531:

  • Drops cljsjs dependencies in favor of deps.cljs entries

The big highlights for this release are:

• Consistently (much) faster compiled function performance (9x faster for 50 seconds on double pendulum simulation)
• The beginnings of the conversion of SICMUtils into a proper Literate Library thanks to a dev dependency on Clerk.

Thanks for @borkdude for PRs and help talking through clj-kondo, SCI etc.

Detailed release notes:

Compiled Function Performance

• #496:

  • replaces the function values in sicmutils.expression.compile with symbols; I hadn't realized before that substituting in symbolic Math/sqrt, for example, was possible, vs a #(Math/sqrt %) function value. Compiled functions are now faster!

    A simulation run of the double pendulum example in the clerk-demo repository now runs in 350ms vs the former 2.2 seconds, a major win.

  • Function compilation now pre-simplifies numerical forms encountered inside a function, like (/ 1 2), instead of letting them be evaluated on every fn call.

  • All numerical forms encountered in function compilation are now converted to either double on the JVM or js/Number in javascript; this way no BigInt values etc are left around.

Clerk

#485 adds a development dependency on Nextjournal's Clerk library, and begins the process of massaging various namespaces into proper literate essays display-able with Clerk. To run these, start a REPL and follow the instructions in dev/user.clj.

- `sicmutils.calculus.derivative` and `sicmutils.differential` now render as proper literate essays, with TeX bugs fixed.

New

• #497: sicmutils.expression.compile/compile-state-fn and its non-memoized version can now take an explicit :mode argument; this will override the dynamically bound *mode*. Invalid modes supplied via :mode will cause compile-state-fn to throw an exception.

• #496, sicmutils.expression.compile:

  • gains a new, validating compiler-mode function for fetching the compiler function.

  • set-compiler-mode! now actually works. It never did!

  • New :source compile mode that returns a source code form. You can either call eval on this or call sci-eval to get an SCI-evaluated function with all proper bindings in place.

  • compile-state-fn now takes an optional options map, with support for :flatten? and :generic-params? keywords. These can be used to tune the shape of the function returned by compile-state-fn.

• #485:

  • Bumps the shadow-cljs dependency to version 2.17.4, and the included cljs version to 1.11.4. sicmutils.collection properly handles the new cljs IntegerRange class.

• sicmutils.polynomial.factor now memoizes poly->factored-expression by default. If polynomial GCD fails inside that function the computation now proceeds with a warning instead of failing.

• #492 updates the clj-kondo linters to emit custom warnings with all metadata from the original token, not just :row and :col. This fixes the ability to override or ignore individual warnings.

• #490 adds sicmutils.numerical.roots.bisect with implementations of bisection search, secant search and a mixed method found in scmutils. These all live under a bisect function.

  The data structure returned is similar to the minimization functions in the sicmutils.numeric.{unimin, multimin} namespaces. As more root-finding methods come online this should all standardize nicely.

• #491 adds sicmutils.mechanics.rotation/M->Euler, for converting from a rotation matrix to a triple of Euler angles. Now we can successfully round trip.

• #489:

  • Installs g/log and g/exp for js/BigInt instances, enabling g/log2 and g/log10 in the mix.

  • Removes most js* calls using coercive-= and js-mod. This form is internal and should be avoided.

• #484 adds sicmutils.polynomial/from-power-series, for generating a polynomial instance from some prefix of a (univariate) power series.

Bug Fixes

• #497 fixes a bug where non-numeric operations up and down were applied at compile time, throwing an error.

Misc

• #498 replaces all long-form GPL headers with "SPDX-License-Identifier: GPL-3.0".

• #485 Bumps the JDK version for Github Actions to 17 from 8.

The big feature of this release is a custom clj-kondo config for sicmutils. All macros in the library (let-coordinates, with-coordinate-functions, all of the pattern matching macros and more) now show proper linter warnings; the pattern matching macros in particular will now show helpful warnings on cases where you've made an easy-to-correct mistake in your pattern syntax.

For example, the following form:

(require '[sicmutils.rule :as r])

(r/rule (+ (? x) (? y)) => (+ (? y odd?) (? x)))

Will provide this inline linter warning, appearing as you type:

Restrictions are not allowed in consequence bindings: odd?

See here for the full list of macros handled by the config.

• The exported clj-kondo config lives here
• To install the config into your project, see the Linter instructions here

Linting

• #477 adds tight integration with the clj-kondo linter via an exported clj-kondo configuration in the resources directory. All macros in the library now offer pleasant linting to users. This is especially helpful for the macros in pattern.rule, which now can offer live feedback to pattern-matching authors.

  See doc/linting.md for details on various warnings reported, and installation instructions for the clj-kondo config.

  Thanks to @borkdude for all of his help getting this working, and making this amazing project!

  • All linter errors and warnings are now addressed, fixed and silenced for the entire codebase, both test and src directories.

  • A new Github Action will run the linter for every PR and push to master, and annotate PRs with linter warnings and errors.

Additions

• #481 adds a new x-degree argument to sicmutils.polynomial/univariate->dense, for padding the result with zeros in the case that you want to guarantee a certain dense degree in the result.

• #477:

  • com.gfredericks/test.chuck dev dependency upgraded to 0.2.13 to grab its clj-kondo exported config.

  • pattern.rule patterns can now handle spliced and unquote-spliced inputs in their symbol position.

Bug Fixes

• #481 fixes a long-standing (test-only) bug in sicmutils.polynomial-test around palindromic polynomials

• #480: sicmutils.numerical.quadrature/definite-integral now coerces the result of non-compiled integrands in cljs to double. This prevents the @kloimhardt bug where certain paths would produce BigInt instances and fail in quadrature calls.

• in #477, I found the following bugs with the help of the linter:

  • Deleted the unused sicmutils.differential/d:apply.

  • Fixed a bug with sicmutils.expression.render/->JavaScript not using the second argument to remainder.

  • deleted sicmutils.numerical.quadrature.common in favor of sicmutils.generic/infinite?

  • Fixed a broken integrator in sicmutils.numerical.quadrature.simpson38, and fixed the tests to actually stress this code.

  • Fixed a bug where sicmutils.numerical.quadrature.substitute/exponential-upper was not actually using its input function!

  • unused simplify argument removed from sicmutils.simplify.rules/non-negative-factors! and all uses.

  • Bug fix in sicmutils.special.elliptic/jacobi-elliptic-functions; deep in the gnarly fn, one of the branches returned nil instead of its required values. Thank you, linter!

  • sicmutils.pattern/template will no longer error in the 1-arity case when some form contains a binding entry like (? (fn [m] ...)). Instead, the function will be passed an empty map.

This release takes us a good way toward completing the port of the kernel, the beating heart, of the original scmutils Scheme library. First, the highlights, and then the thematically organized CHANGELOG entries captured in this release.

• We now have a full Quaternion implementation! The implementation is built out of the best stuff I could find in every Quaternion library out there, well documented and fully generic. In the future we should be able to specialize various methods to native numerics, making this type very fast.

• Lots of new generics, including all remaining trigonometric functions and their inverses. Thanks to John D Cook's 'Bootstrapping a minimal math library' for his inspiration on the defaults and implementation order of these new functions.

• The Matrix, Structure and Tensor APIs are now fully ported. This led to some big efficiency boosts, and the ability to solve systems of linear equations with various combinations of matrices, up and down structures and row and column matrices.

• Functional folds! A big rewrite of the floating-point summation routine code led to some (I think) original discoveries about how to write polynomial interpolation, rational function interpolation and Richardson extrapolation as functional folds.

• Lots of performance improvements! One of the early examples from SICM involves finding a path that "minimizes action" given some Lagrangian. This example is down from many seconds to 100ms on my machine.

Read on for detailed release notes.

Quaternions

• #461 adds sicmutils.quaternion, with a full arithmetic implementation and the beginnings of a rotation API. Quaternions are implemented like vectors of length 4, and implement all appropriate Clojure protocols. All arithmetic is compatible with all scalars and complex numbers.

  • Accessors: get-r, real-part, get-i, get-j, get-k, complex-1, complex-2, ->complex-pair, ->vector, three-vector

  • Predicates: real?, zero?, one?, pure?, unit?

  • Constants: ZERO, ONE, I, J, K

  • Reader literal #sicm/quaternion takes a 4-vector or a single entry.

  • Constructors: make, from-complex, spherical, semipolar, multipolar, cylindrospherical, cylindrical

  • More: arity, evaluate, partial-derivative, magnitude-sq, normalize, commutator

  • Transcendental functions: exp log, cos, sin, tan sinh, cosh, tanh. Many more transcendentals will work, thanks to their default implementations.

  • Arithmetic and generics: +, -, *, /, dot-product, cross-product, conjugate, magnitude, expt, sqrt, simplify, infinite?, solve-linear-right, solve-linear

  • Rotation-related: from-angle-normal-axis, from-angle-axis, pitch, roll, yaw, ->angle-axis

New Generics

• #458:

  • Default implementation of g/negative? returning false for literal numbers and symbols. This was required to get g/abs working for polynomials and rational functions with symbolic coefficients.

• #448:

  • new g/infinite? generic with implementations for all numeric types, complex numbers, differential instances. Defaults to false for all other types. (Also aliased into sicmutils.env/infinite?).

• #449:

  • All missing trigonometric functions have been filled in sicmutils.generic and aliased in sicmutils.env:

    • Inverse cotangent: acot
    • inverse secant: asec
    • inverse cosecant: acsc
    • hyperbolic (inverse hyperbolic) cotangent: coth and acoth
    • hyperbolic (and inverse hyperbolic) secant: sech and asech
    • hyperbolic (and inverse hyperbolic) cosecant: csch and acsch

    All of these have default implementations and derivatives defined. They'll work out of the box for all types with atan defined (and potentially exp, sqrt and log.)

    Thanks to John D Cook's 'Bootstrapping a minimal math library' for his inspiration on the defaults and implementation order of these new functions.

  • expt gains a new default implementation for non-native-integral powers, making expt work for any type with exp, log and mul defined.

  • sqrt gains a default implementation for all types implementing exp, mul and log.

  • All trig functions now have derivatives and docstrings.

  • New sinc, tanc, sinhc, tanhc functions live in sicmutils.generic and are aliased into sicmutils.env. These are generically defined as (/ (sin x) x), (/ (tan x) x) (and similar with sinh and tanh), with correct definitions for 0 and infinite-valued inputs.

    These functions all support derivatives as well.

  • New default acot implementation in sicmutils.series.

Matrix / Structure / Tensors

• #469:

  • sicmutils.matrix gains:

    • literal-column-matrix, literal-row-matrix for generating slightly tidier matrices of literal entries. (See literal-matrix for the prior option.)

    • structure->matrix converts 2 tensors into explicit matrices.

    • s:solve-linear-left, s:solve-linear-right, s:divide-by-structure act on 2 tensors. These live in the matrix namespace since they depend on conversions to and from tensors and matrices.

    • make-diagonal for generating diagonal matrices with a constant element along the diagonal.

    • s->m, s:transpose and s:inverse all gain new 2-arities that provides a sane default for ls.

    • More efficient matrix invert and determinant routines, plus functions to generate type specific custom matrix inversion and determinant routines via classical-adjoint-formula, general-determinant.

    • Linear equation solving via solve, rsolve and cramers-rule.

  • sicmutils.structure gains down-of-ups?, up-of-downs?, two-up?, two-down?, two-tensor? and two-tensor-info for working with "2 tensors", ie, structures that contain structural entries of matching orientation and size.

  • Implements new generics for matrices and structures:

    • diagonal matrices respond true to v/= with a scalar if all entries along the diagonal are equal to that scalar.

    • square matrices can now g/+ and g/- with scalars; the scalar c is converted (* c I), where I is an identity matrix of the same dimension as the square matrix.

    • (g/acot M) now expands the matrix M into a nice power series, more efficient than the previous default.

    • Thanks to solve and cramers-rule, the following g/div combinations now work: matrix/scalar, scalar/square-matrix, column-matrix/square-matrix, row-matrix/square-matrix, up/square-matrix, down/square-matrix, matrix/square-matrix.

    • new solve-linear implementations between square matrices and up down, row and column matrices, and between structures and scalars.

    • new solve-linear-right between row-matrix+square-matrix, down+square-matrix and scalar+structure.

Folds

• #456:

  • Richardson extrapolation is now implemented as a functional fold. The exposition in sicmutils.polynomial.richardson discusses this; the namespaces gains richardson-fold, richardson-sum and richardson-scan.

• #451:

  • new sicmutils.algebra.fold namespace:

    • New folds: kahan-babushka-neumaier (aliased as kbn), kahan-babushka-klein and and kbk-n macro for generating higher-order kahan-babushka-klein variants. generic-sum-fold folds using sicmutils.generic/+.

    • sicmutils.util.aggregate/kahan-fold now lives here, named kahan.

    • fold->sum-fn and fold->scan-fn generate functions like sicmutils.util.aggregate.{sum,scan} specialized to the supplied fold. See the docstrings for the multiple arities supported

    • fold primitives: count, constant, min, max.

    • fold combinator join allows compound folds to be built out of primitive folds.

  • Upgrades to sicmutils.util.aggregate:

    • scanning-sum renamed to scan

    • halt-at deleted in favor of the built-in halt-when that I didn't know about!

    • scan and sum now both use a dynamic binding, *fold*, to set the fold they use for aggregation. By default, this is set to the new kahan-babushka-neumaier-fold.

    • The three-arity version of sum now uses transducers, saving a pass over the input range.

    • pairwise-sum implements pairwise summation, an error-limiting technique for summing vectors. Use the dynamic binding *cutoff* to set where pairwise-sum bails out to normal summation.

  • Upgrades to sicmutils.rational-function.polynomial:

    • The folds in this namespace now follow the fold contract laid out in sicmutils.algebra.fold, implementing all three arities correctly.

    • I realized that the fold implementation here should /not/ return a full row every time it processes a previous row; a far better present implementation would return the best estimate so far. Then you could build a scan from that fold to see the estimates evolve lazily as new points are added. This has better performance, it turns out, than the original method!

    • added a bunch to the exposition to make the advantages clear.

  • Upgrades to sicmutils.rational-function.interpolate:

    • fold interface upgraded, similar to the polynomial interpolation notes.

    • New bulirsch-stoer-fold, bulirsch-stoer-sum and bulirsch-stoer-scan functions. These are similar to the modified-** versions but use the bulirsch-stoer algorithm, instead of modified-bulirsch-stoer.

    • modified-bulirsch-stoer-fold-fn renamed to modified-bulirsch-stoer-fold, to match the naming scheme of other "folds" in the library.

    • modified-bulirsch-stoer-fold renamed to `modi...

Small bugfix release to push out #396 .

• #396:

  • fixes a bug in the SCI version of define-coordinates which didn't allow
    any rebinding of manifolds.

  • Removes the bindings key from sicmutils.env.sci/context-opts.
    babashka/sci#637 is a bug with variable rebinding
    that occurs when :bindings is in play. Instead of relying on this key,
    evaluate (require '[sicmutils.env :refer :all]) against your SCI
    environment to get all bindings.

  • bumps the default version of SCI to 0.2.7.

This release cleans up the differential geometry utilities in SICMUtils and fixes a couple of bugs that prevented the Einstein Field Equations from running. These now work well, and very fast!

Highlights

• A new define-coordinates macro
• Einstein's field equations now run as tests: https://github.com/sicmutils/sicmutils/blob/main/test/sicmutils/fdg/einstein_test.cljc#L41
• All code listings from Functional Differential Geometry now work and are implemented as tests in the FDG test directory.
• You can now define complex numbers using the #sicm/complex reader literal:

#sicm/complex [1.2 3.6]    ;; 1.2+3.6i
#sicm/complex [1.2]        ;; 1.2
#sicm/complex 1.4          ;; 1.4
#sicm/complex "1.2 + 3.6i" ;; 1.2+3.6i

A big thanks to @phasetr for pushing on this in this discussion: #380

New Features

• #348:

  • Adds a new single arity version of
    sicmutils.util.permute/permutation-parity, which returns the parity of a
    permutation relative to its sorted version.

  • sicmutils.complex/complex can now take a single string argument in both
    Clojure and Clojurescript.

  • Expands the complex number literal parser to take these forms, in addition
    to the previously-supported string argument:

#sicm/complex [1.2 3.6]    ;; 1.2+3.6i
#sicm/complex [1.2]        ;; 1.2
#sicm/complex 1.4          ;; 1.4
#sicm/complex "1.2 + 3.6i" ;; 1.2+3.6i

• #393:

  • Forms like (let-coordinates [(up x y) R2-rect] ...) will now work even if up is not present in the environment. Previously this syntax was valid, but only if up had been imported.

  • Adds the sicmutils.calculus.coordinate/define-coordinates macro, also aliased into sicmutils.env. This macro allows you to write forms like

(define-coordinates (up t x y z) spacetime-rect)
(define-coordinates [r theta] R2-polar)

and install a set of bindings for a manifold's coordinate functions, basis vector fields and basis form fields into a namespace. This is used liberally in Functional Differential Geometry. (You might still prefer let-coordinates for temporary binding installation.)

• Converts many of the sicmutils.fdg test namespaces to use the new define-coordinates macro, making for a presentation closer to the book's.

• Fixes a Clojurescript warning in sicmutils.util warning due to redefinition of clojure.core/uuid

• #381:same.ish/Approximate implemented for sicmutils.structure/Structure, allowing ish? comparison of up and down structures with approximate entries. Require sicmutils.generator for this feature. (NOTE: because protocols are implemented for the LEFT argument, (ish? <vector> (down ...)) will still return true if the values are approximately equal, even though a <vector> is technically an up and should NOT equal a down. Do an explicit conversion to up using sicmutils.structure/vector->up if this distinction is important.)

• #381:

  • Section 7.3 of FDG implemented as tests in sicmutils.fdg.ch7-test.

  • #382 adds tests for all code forms in Chapter 8 of FDG.

  • Many new tests and explorations ported over from covariant-derivative.scm. These live in sicmutils.calculus.covariant-test.

• #384:

  • Adds sicmutils.fdg.ch9-test, with tests for all forms from FDG's 9th chapter.

  • Tests from sicmutils.fdg.einstein-test now all work, and quite fast. The functions in this namespace comprise some of the exercises from FDG chapter
    9. (Einstein's Field Equations hung until this PR... getting these working
    is a huge achievement for me, and, in some sense, the final milestone of the
    Big Port from scmutils.)

  • Adds sicmutils.function/memoize, a metadata-and-function-arity preserving version of clojure.core/memoize.

  • Adds new manifold? and manifold-family? functions in sicmutils.env and sicmutils.calculus.manifold. These are enabled by new :type :sicmutils.calculus.manifold/{manifold,manifold-family} keys in the appropriate structures in the manifold namespace. Manifolds and manifold families will now respond with these keywords to sicmutils.value/kind.

• #386:

  • Aliases sicmutils.mechanics.hamilton/phase-space-derivative into sicmutils.env, and adds sicmutils.sr.frames/base-frame-maker. The latter function makes it easier to write reference frames like the-ether, as with the home variable in chapter 11 of FDG.

  • Adds all code listings from chapters 10 and 11 of FDG as sicmutils.fdg.{ch9,ch10}-test.

Functions moves, API changes

• #381: sicmutils.calculus.coordinate/generate moves to sicmutils.calculus.manifold/c:generate; this supports a bugfix where 1-dimensional manifolds like R1-rect, aka the-real-line, return a coordinate prototype of a single element like t instead of a structure with a single entry, like (up t). Thanks to @phasetr for the bug report that led to this fix, and @gjs for finding and fixing the bug.

Bug Fixes

• #394 fixes a bug with derivatives of functions that returned a map... but where the map was actually meant to represent some other type, by holding a :type key. We do this for manifold families and manifold points, as two examples. Now, instead of recursing into the values, the system will correctly throw an error. (You can fix this by using a defrecord instead of a map and implementing sicmutils.differential/IPerturbed.)

#381:

• same.ish/Approximate now defers to sicmutils.value/= for equality between Symbol and other types. This lets ish? handle equality between symbols like 'x and literal expressions that happen to wrap a single symbol.

• Cartan->Cartan-over-map now does NOT compose (differential map) with its internal Cartan forms. This fixed a bug in a code listing in section 7.3 of FDG.

• timeout exceptions resulting from full GCD are now caught in tests using sicmutils.simplify/hermetic-simplify-fixture. Previously, setting a low timeout where simplification failed would catch and move on in normal work, but fail in tests where fixtures were applied

• #382:

  • Makes the name argument to sicmutils.operator/make-operator optional. name now defaults to '???.

• #384:

  • in sicmutils.calculus.indexed, with-argument-types and with-index-types now both correctly set the arity of the returned function, in addition to the argument types or indices. sicmutils.function/arity will now work correctly with indexed or typed functions.

  • The sicmutils.calculus.manifold/ICoordinateSystem now has a uuid function, for internal comparison of coordinate systems. This is here so that points can cache coordinate system representations by UUID. Before this change, changing the coordinate prototype, or attaching metadata to a coordinate system would break its cache entry in manifold points. (This was the killer for the Einstein Field Equations!)

  • sicmutils.calculus.manifold/{coordinate-prototype,with-coordinate-prototype} now store and retrieve the coordinate prototype from metadata. This plus the previous change allows manifold points to correctly cache their coordinate representations.

  • sicmutils.calculus.manifold/manifold acts as identity on manifolds now. Previously it only worked on coordinate systems.

• #376 adds more type hints to the ratio.cljc namespace. This fully solves the advanced compilation issues we were seeing.

• #374: Demos, thanks to @sigmaxipi!

• #379 fixes typos in a couple of the equations in richardson.cljc, closing #377. Thanks to @leifp for the report.

Incremental release that bumps the Fraction.js dependency to 4.1.1. This includes cljsjs/packages#2190, which makes bigfraction.js compatible with advanced compilation.

• #372 bumps the Fraction.js dependency to 4.1.1.

This is an incremental bugfix release to get Clojurescript advanced compilation into shape.

• #371:

  • fixes a subtle bug with extern inference on fraction.js/bigfraction.js. Thanks to @sigmaxipi for this report!

  • removes overridden factory constructors like ->Polynomial. I had originally done this for functions that held a metadata field, so that the user could leave it out and have it default to nil... but advanced Closure compilation can't understand the ns-unmap call, so it has to go.

  • Many unary functions on Operator, Structure, Series, PowerSeries, Polynomial and RationalFunction now preserve metadata. Binary functions between two instances of any of these still return a new object with metadata == nil.

(If you have any questions about how to use any of the following, please ask us at our Github Discussions page!)

This release focused on improving the expressiveness and performance of the three simplification engines in SICMUtils:

• sicmutils.polynomial and sicmutils.rational-function are now quite well fleshed out, with full polynomial and rational function APIs and many generics.

• The polynomial and rational function simplifiers work by round-tripping expressions through these types, depending on each namespace to emit symbolic expressions in "canonical form". This process is now much faster! On one important Bianchi Identity benchmark in sicmutils.fdg.bianchi-test, one test that formerly took close to 30 minutes now runs in 30 seconds, and all see a 60-fold improvement.

• By default, these simplifiers emit expressions with all terms multiplied out; the new factor function in sicmutils.env lets you factor expressions, overriding this default.

• The rule-based simplifier is now based on a powerful pattern matching engine, implemented in pattern.match and pattern.rule. sicmutils.simplify.rules now contains every rule and possible customization from the original scmutils codebase.

There is a lot in this release, all motivated by performance. Please read on for the detailed notes, and enjoy version 0.19.0!

Rule-Based Simplifier Overhaul

• #353 introduces a powerful new simplifier, ported from the new-simplify procedure in simplify/rules.scm of the scmutils library. There are now a BUNCH of new rulesets and rule simplifiers in sicmutils.simplify.rules!

  The next step with these is to massage them into separate bundles of rules that users can mix and match into custom simplifiers for objects like abstract matrices, abstract bra and ket structures, up and down, booleans (for representing equations and inequalities) and so on.

• #349 introduces a new pattern matching system, built out of matcher combinators. All of the rules in sicmutils.simplify.rules now use the new syntax offered by the library. Some notes:

  • pattern.match defines a number of "matcher combinators"; these are functions that take a map of bindings, a data input and a success continuation and either succeed by calling their continuation, or fail. Out of the box, the library provides fail, pass, with-frame, update-frame, predicate, frame-predicate, eq, bind, match-when, match-if, or, and, not, segment and sequence.

  • Additionally, any combinator that takes another combinator can ALSO take a pattern form like '?x. See pattern.syntax for the full, rich range of syntax allowed. These are all functions, so you'll have to quote your symbols at this stage.

  • Passing a matcher combinator to pattern.match/matcher to generate a matcher object. This is a function from some data input to a map of bindings on success, or an explicit pattern.match/failure object on failure. Test for failure with pattern.match/failed?.

  • A combination of a matcher and a "consequence function" is called a "rule". A consequence is a function that takes a binding map and either returns a new result or fails by returning nil or false. (Don't worry, you can succeed with these values too by wrapping them in sicmutils.rule/succeed.)

    Rules are the heart of the whole simplification mechanism in sicmutils! To learn about how to build these, see the documentation for pattern*, pattern, consequence, template, rule*and rule.

  • pattern.rule gives you some starter rules, and many combinators you can use to build more and more powerful and complex sets of rules. These are pass, fail, predicate, return, branch, choice*, choice, pipe*, pipe, n-times, attempt, guard, iterated, while, until, fixed-point and trace.

  • Rules are nice for rewriting entire expressions recursively, from the bottom up or top down. This is called "term rewriting". A big motivation for this rewrite was to make it easy to build custom term rewriters for types like abstract matrices or abstract up and down structures. You can use your rules to rewrite structures recursively with bottom-up, top-down, iterated-bottom-up and iterated-top-down. ruleset*, ruleset, rule-simplifier and term-rewriting capture some common patterns the library uses to go from rules => term rewriters.

  • If you want ideas about how to use the pattern matching library to rewrite expressions, see sicmutils.simplify.rules for many examples.

• #354 adds SCI support for all macros and functions in the new pattern matching namespaces, and adds these to the namespaces exposed via sicmutils.env.sci.

Rational Function, Polynomial Simplifiers

• #341 takes on a large rewrite of the rational function and polynomial simplfiers. One goal of this project was to improve the performance of the Bianchi Identities in sicmutils.fdg.bianchi-test, and I'm happy to say that they are now a good bit faster than the original scmutils implementation.

  sicmutils.polynomial and sicmutils.rational-function are now solid data structures of their own, with many operations installed into the generic system. These are now valuable and useful outside of their role in the simplifier.

  This was a large project, and many small improvements and bugfixes snuck in. Here is the full list:

  • v/kind now works for sorted-map instances.

  • GCD in Clojurescript is now fast and efficient between all combinations of js/BigInt and js/Number, and in Clojure between all combinations of clojure.lang.BigInt, BigInteger, Long and Integer.

  • on the JVM, GCD now works properly with rational numbers. Previously anything non-integral would return 1; now (gcd 1/2 1/3) properly returns 1/6.

  • g/exact-divide now succeeds for all non-exact ::v/scalar types (symbols, floats, etc) either if the denominator is zero, or if the two arguments are equal. Else, it throws, just like before.

  • A multi-arity call to sicmutils.generic/* now stops if it encounters a 0, rather than attempting to multiply all remaining items by 0.

  • The default function for sicmutils.generic/lcm protects against overflow by dividing only a single one of its arguments a and b by (gcd a b).

  • (g/lcm 0 0) now properly returns 0.

  • New sicmutils.util.aggregate/{monoid,group} functions let you build multi-arity aggregations out of binary combination functions, with an option to bail early at "annihilator" values, like 0 for multiplication.

  • New multi-arity lcm and gcd implementations for symbolic expressions appropriately handle 0 and 1 on either side, as well as the case where both arguments are equal.

  • In the sicmutils.numsymb namespace, thanks to monoid and group, the '*, '/, '-, '+, 'or, 'and, 'gcd, 'lcm and '= operations now have efficient multi-arity implementations that stop computing when they receive an annihilator, like 0 for multiplication or true for or. Access these via (sicmutils.numsymb/symbolic-operator <symbol>).

  • sicmutils.series/PowerSeries gains arg-scale and arg-shift functions; these are identical to sicmutils.function/arg-{scale,shift}, but preserve the PowerSeries type. (#367 proposes making these functions generic.)

  • New sicmutils.ratio/IRational protocol, with numerator and denominator functions implemented for ratios and for the RationalFunction data type. These two are now exposed in sicmutils.env.

  • sicmutils.simplify.rules/*divide-numbers-through-simplify?* is now true by default; numbers in the denominator will now automatically pull up into the numerator. All tests now reflect this setting.

  • Any analyzer generated from sicmutils.expression.analyze can now act on both bare, unwrapped expressions (raw lists etc) and on sicmutils.expression.Literal instances. This means that you can now call sicmutils.simplify/{*rf-simplify*,*poly-simplify*} as functions and canonicalize some form with either simplifier without triggering a full simplification. A small win, but ice.

  • sicmutils.polynomial.factor got a major rewrite, and now exposes a few functions like poly->factored-expression, factor-expression and factor.

    • factor is tremendously useful! Call factor (it's aliased into sicmutils.env) on any expression to factor out all possible terms. This makes it much easier to see where there is some cancellation lurking, in, say, some expression you know should equal zero (a residual).

  • bugfix: sicmutils.expression.Literal instances now compare their contained expression via sicmutils.value/=.

  • sicmutils.rules/constant-elimination can now eliminate constants from expressions with any arity, not just binary forms.

  Now, the three big namespaces... sicmutils.polynomial, sicmutils.rational-function and sicmutils.polynomial.gcd all got a big overhaul.

  • sicmutils.polynomial notes:

    • Polynomial uses a new sparse representation for its "power product" term; this, plus an arithmetic rewrite, makes the whole system much faster for larger numbers of variables (for all #s, really).

    • Polynomial instances implement many more Clojure(script) protocols. They can hold metadata; they can be evaluated as functions of their indeterminates, and seq now returns a sequence of terms.

    • Polynomial extends sicmutils.function/IArity and differential/IPerturbed, so you can use sicmutils.function/arity, and take derivatives of functions that return polynomials.

    • In their arithmetic, Polynomial instances will drop down to bare coefficients whenever some multiplication or addition removes all indeterminates. All binary arithmetic exposed in the namespace can han...

This release focused on porting over all of the material required to run every piece of code from Sussman and Wisdom's "Functional Differential Geometry". The namespaces are lightly documented; the situation is better than the original library, but will only get better as I work through the material and add commentary.

There is a huge amount of functionality and material here! We can run many examples from general and special relativity, and the tests are full of exercises from the classic "Gravitation" book by Misner, Thorne and Wheeler (MTW).

Notable changes from the rest of the library:

• Operator instances are slightly more efficient with their addition and multiplication, handling zero? and one? cases appropriately

• Structures can now hold metadata

• We've extended the SICMUtils generics to Clojure's Map and Set data structures. These can now combine with +. Maps are treated as sparse infinite-dimensional vector spaces, and can multiply with symbolic or numeric scalars.

• ModInt instances are now correctly equal to numbers (when those numbers mod down to the ModInt instance's residue).

What's next?

The next major change will be an overhaul of the simplifier to make it work fast enough to solve Einstein's field equations in a reasonable amount of time, maybe even in the browser. Polynomial GCD is slow, but #341 will make it fast.

On to the detailed notes!

Functional Differential Geometry

• From #339:

  • The new sicmutils.calculus.covariant/Lie-D can compute the Lie derivative for coordinates.

  • sicmutils.calculus.frame lets us create relativistic reference frames for investigating special relativity problems. This namespace aliases the following functions into sicmutils.env: 'frame?', make-event, event?, claim, coords->event, event->coords, ancestor-frame, frame-name, frame-owner and frame-maker.

  • sicmutils.calculus.hodge-star implements the Hodge star operator from chapter 10 of Functional Differential Geometry, plus Gram Schmidt orthonormalization. This namespace aliases the following functions into sicmutils.env: Gram-Schmidt, orthonormalize and Hodge-star.

  • sicmutils.calculus.indexed ports over the scmutils work on indexed objects and typed functions. This namespace aliases the following functions into sicmutils.env: argument-types, with-argument-types, index-types, with-index-types, typed->indexed, indexed->typed, typed->structure, structure->typed, i:outer-product and i:contract.

  • sicmutils.calculus.manifold gains coordinate-system?, which (predictably) returns true if its argument is a coordinate system, false otherwise. chart and point also take relativistic reference frames in addition to coordinate systems; the returned function converts to and from coordinates and events, rather than coordinates and manifold points.

  • Div, Grad, Curl and Lap move from sicmutils.calculus.derivative to sicmutils.calculus.vector-calculus. This namespace also contains versions of these operators from Functional Differential Geometry. This namespace aliases the following functions into sicmutils.env: divergence, curl, gradient and Laplacian (along with the others mentioned).

  • lots of new namespaces available in sicmutils.env.sci, soon to be deployed to Nextjournal: sicmutils.calculus.{hodge-star, indexed, vector-calculus}, and sicmutils.sr.{boost,frames}.

  • sicmutils.sr.boost describes boosts from special relativity, covered in chapter 11 of Functional Differential Geometry. This namespace aliases the following functions into sicmutils.env: make-four-tuple, four-tuple->ct, four-tuple->space, proper-time-interval, proper-space-interval, general-boost, general-boost2 and extended-rotation.

  • sicmutils.sr.frames implements relativistic reference frames from special relativity, covered in chapter 11 of Functional Differential Geometry. This namespace aliases the following functions into sicmutils.env: make-SR-coordinates, SR-coordinates?, SR-name, make-SR-frame, the-ether, boost-direction, v:c, coordinate-origin, add-v:cs and add-velocities.

• From #338:

  • sicmutils.fdg.bianchi-test verifies the Bianchi identities; this was a challenge posed by GJS, and getting it working exposed a few bugs and triggered the rest of the work in this PR. Thank you, GJS!

  • covariant-derivative now properly handles the case of functions with argument types attached.

  • added covariant-differential to sicmutils.calculus.covariant.

  • aliased all functions from various namespaces in sicmutils.calculus into sicmutils.env.

  • adds sicmutils.calculus.metric, with the following functions exposed in sicmutils.env:

    • coordinate-system->metric-components, coordinate-system->metric, coordinate-system->inverse-metric, literal-metric, components->metric, metric->components, metric->inverse-components, metric-over-map, lower, vector-field->oneform-field, drop1, raise, oneform-field->vector-field, raise1, drop2, raise2, trace2down, trace2up, sharpen, S2-metric

    • sicmutils.calculus.metric/invert is exposed as metric:invert to match the scmutils naming scheme.

  • adds sicmutils.calculus.connection, with the following functions exposed in sicmutils.env:

    • make-Christoffel-1, metric->Christoffel-1, metric->Christoffel-2, literal-Christoffel-1, literal-Christoffel-2, metric->connection-1, metric->connection-2, literal-Cartan, structure-constant

• #337:

  • adds sicmutils.calculus.curvature, with these new functions and many tests from the classic "Gravitation" book: Riemann-curvature, Riemann, Ricci, torsion-vector, torsion and curvature-components

  • form fields now have NO identity operator, since they multiply by wedge, not
    composition.

• #328 adds many utilities for "Functional Differential Geometry".

  • vector fields, in sicmutils.calculus.vector-field:

    • new functions: basis-components->vector-field,
      vector-field->basis-components

    • vector fields now implement v/zero? and v/zero-like by returning
      proper vector fields.

  • form fields, in sicmutils.calculus.vector-field:

    • new functions: nform-field?, basis-components->oneform-field, oneform-field->basis-components and function->oneform-field (aliased as differential-of-function)

    • Alt, alt-wedge provide alternate wedge product definitions

    • form fields now implement v/zero? and v/zero-like by returning proper form fields that retain their rank.

    • form fields now correctly multiply via * by using sicmutils.calculus.form-field/wedge, instead of composition.

  • maps between manifolds, in sicmutils.calculus.map:

    • new function: pushforward-function

    • differential becomes differential-of-map, aliased back as differential

  • sicmutils.calculus.covariant gains new functions: Cartan?, Christoffel?, Cartan->Christoffel, symmetrize-Christoffel, symmetrize-Cartan, Cartan->Cartan-over-map, geodesic-equation, parallel-transport-equation.

  • sicmutils.calculus.covariant/vector-field-Lie-derivative can now handle structural inputs.

New Functions, Functionality

• From #342:

  • Added sicmutils.calculus.derivative/D-as-matrix and sicmutils.matrix/as-matrix, ported from scmutils.

  • converted sicmutils.modint.ModInt to a deftype; this allows ModInt instances to be = to non-ModInt numbers on the right, if the right side is equal to the residue plus any integer multiple of the modulus. v/= gives us this behavior with numbers on the LEFT too, and ModInt on the right.

    • This change means that :i and :m won't return the residue and modulus anymore. sicmutils.modint gains new residue and modulus functions to access these attributes.

  • The JVM version of sicmutils gains more efficient gcd implementations for Integer and Long (in addition to the existing native BigInteger gcd), thanks to our existing Apache Commons-Math dependency.

  • sicmutils.structure/dual-zero aliases compatible-zero to match the scmutils interface. Both are now aliased into sicmutils.env.

  • Structure instances can now hold metadata (#339).

• From #339:

  • In sicmutils.mechanics.rotation:

    • gains aliases for R{xyz} in rotate-x, rotate-y and rotate-z.

    • R{x,y,z}-matrix now alias rotate-{x,y,z}-matrix.

    • Added new functions angle-axis->rotation-matrix and the mysterious,
      undocumented wcross->w from scmutils

    • rotate-{x,y,z}-tuple are now aliased into sicmutils.env.

  • Operator instances now ignore the right operator in operator-operator addition if the left operator passes a v/zero? test. Contexts are still appropriately merged.

  • in sicmutils.simplify.rules, the sqrt-contract ruleset now takes a simplifier argument and attempts to use it to simplify expressions internal to a square root. As an example, if two square roots in a product simplify to the same expression, we can drop the wrapping square root; otherwise multiplication is pushed under the root as before.

    • Added a missing rule in simplify-square-roots that handles roots of exponents with odd powers.

  • sicmutils.matrix changes:

    • generate has a new 2-arity version; if you supply a single dimension the returned matrix is square.

    • diagonal? returns true if its argument is a diagonal matrix, false
      otherwise.

  • A new namespace, sicmutils.util.permute...