Bignums as a separate opam package #498

letouzey · 2017-03-22T12:44:29Z

Not fully ready yet (but no real showstopper either), this PR is to mark my intention to migrate Bignums (Int31, BigN, BigZ, BigQ) out of Coq repository, and place them instead in a dedicated opam package, ideally as soon as Coq 8.7.

Why ?

Relatively small audience for this library (hardcore computationists ;-)
Important compilation overhead on the stdlib
First step toward a more component-based and customizable standard library

Already done

Regrouped all relevant files to a same location (currently plugins/bignums)
Created a dedicated Makefile (based on coq-makefile)

Still to be done

Create and populate a separate git repository with the content of plugins/bignums
Corresponding opam package
Identify and adapt third-party developments that use bignums
Could maybe the "retroknowledge" code about Int31 come someday as a plugin ?

ejgallego · 2017-03-22T12:49:34Z

Once you have the separate repository maybe you would like to update the .travis.yml file too.

letouzey · 2017-03-22T12:55:02Z

Indeed

maximedenes · 2017-03-22T13:01:05Z

Could maybe the "retroknowledge" code about Int31 come someday as a plugin ?

As I mentioned two working groups ago, I'm working on rebasing primitive integers on trunk. I'm afraid this work goes a bit against making the retroknowledge modular, since it adds an Int constructor to terms. It also switches from 31 to 63 bit integers.

However, it is still doable to ship separately the libraries built on top of it. The treatment of 63-bit integers themselves is fairly lightweight.

silene · 2017-03-22T13:01:14Z

Could maybe the "retroknowledge" code about Int31 come someday as a plugin?

Precisely because there is no way in the foreseeable future to move the retroknowledge code out of Coq, I would suggest not to remove Int31.v from Coq either. All the other files are fine though.

letouzey · 2017-03-23T10:11:52Z

Ok for leaving in the stdlib the stuff about Int31 (and hence the definition of cyclic types).

maximedenes · 2017-03-29T21:09:15Z

What is the transition strategy for user developments?

maximedenes · 2017-05-24T23:37:13Z

During the WG discussion, we agreed to merge this. We were worried about the backward compatibility of developments importing the OPAM package and even suggested to publish an empty package for 8.6. But @letouzey made the remark offline that if the package compiles with both 8.6 and 8.7, no action is required to ensure backward compatibility, apart from setting the right dependency constraints in the OPAM package.

letouzey · 2017-05-31T14:50:58Z

An external repository for bignums code is ready here https://github.com/coq/bignums .
I'll see now how to provide an opam package out of this.
The master branch of this repository compiles with coq trunk, and there is also a v8.6 branch.
The only difference between the two is CErrors.user_err_loc vs. CErrors.user_err in the syntax plugin. My dear compatibility zealots @maximedenes @ejgallego, how are we expected to provide
code that works both with 8.6 and 8.7 if there isn't any compat layer or deprecation delay for changes such as these CErrors functions ?

ejgallego · 2017-05-31T14:53:50Z

@letouzey you are right, we should add the removed function back, in a deprecated way, sorry for that.

This change was done when we had not settled on Ocaml 4.02.3 so deprecation was not available.

Anyways, for most plugins, the only solution is indeed to have their own compat layer or to ship two different files. However note that given the amount of changes in 8.7

code that works both with 8.6 and 8.7

is really a futile goal.

maximedenes · 2017-05-31T14:55:29Z

My dear compatibility zealots @maximedenes @ejgallego

:)

We mean this for scripts. Given the current shape of the plugin API, it would be counter productive to guarantee compatibility. On this particular example, I'm a bit surprised, though, I thought @ejgallego had made sure there was a way to maintain compatibility, following a comment by Guillaume.

ejgallego · 2017-05-31T14:56:52Z

I re-added some missing functions, but not this one in particular.

ejgallego · 2017-05-31T15:00:35Z

Indeed as @maximedenes points out, given the current OCaml API model and the things discussed for the next Coq versions, ML-level compatibility will be far from stellar I'm afraid.

Adding your plugin to Travis can help alleviate some of the pain. However IMO given what the release schedule is going to be we should really recommend a synchronized release model to plugin authors. That is to say: if you are a plugin author, when Coq 2019.04 comes out, you release the 2019.04 version of your plugin, then you move development to Coq 2019.10.

IMHO this is feasible now as a 6 months rate should be OK for Coq. [Note that other projects such as GCC or Linux deem 6 months too long and they release way often].

ejgallego · 2017-05-31T16:43:56Z

Sorry @letouzey , user_err_loc is there in trunk, so I am not sure I understand what the problem is?

letouzey · 2017-05-31T17:02:38Z

@ejgallego, indeed user_err_loc is there today, but the trunk repo I was using at that moment was a few days old. Anyway, since these recent changes also brought your CAst stuff, all hope of number_syntax.ml compatible to both 8.6 and 8.7 is now lost. Well, so be it...

letouzey · 2017-05-31T17:04:30Z

Rebased, and directly propose the removal of BigN, BigZ, BigQ now that we have an separate repo for them ( https://github.com/coq/bignums ).

maximedenes · 2017-06-02T21:55:19Z

There multiple failures. Some of them are expected (CoLoR relying on BigN), some of them easy to fix (doc), some unexpected (CompCert).

maximedenes · 2017-06-06T09:01:24Z

@letouzey Are you working on a fix, or should we postpone this to 8.8?

letouzey · 2017-06-06T14:12:29Z

@maximedenes : Working on it today...

letouzey · 2017-06-06T15:22:02Z

Rebased + fixed make doc

letouzey · 2017-06-06T15:33:00Z

The CompCert failure is a really minor consequence of the removal of BigNumPrelude, which was (ab)using Hints a lot. Since this failure is unique and easy to fix in CompCert, I'll propose a patch for it (and an overlay here in the meantime).

For devs really relying on bignums (Color, MathClasses and its dependency Formal_topology), I'll provide overlays declaring my new repository as dependency for these devs

letouzey · 2017-06-06T19:31:24Z

@ejgallego : does the new version of travis stuff suits you ?

ejgallego · 2017-06-06T19:43:30Z

dev/ci/ci-basic-overlay.sh

+# Bignums
+########################################################################
+: ${Bignums_CI_BRANCH:=master}
+: ${Bignums_CI_GITURL:=https://github.com/coq/bignums.git}


minor nit, it should be bignums_CI_BRANCH etc... as I hope soon much of this boilerplate is handled automagically, but the name should correspond to the entry in ci-bignums

maximedenes · 2017-06-09T11:57:33Z

@ejgallego I believe @letouzey is waiting for your feedback on this one.

ejgallego · 2017-06-09T13:33:44Z

Oh sorry I didn't see it, sure it looks almost fine, they were really nits. However I would like to see the install stuff corrected, but it is not a blocker for a merger as we can take care of it later.

ejgallego · 2017-06-09T13:39:30Z

dev/ci/ci-color.sh


+# Setup Bignums
+
+source ${ci_dir}/ci-bignums.sh


I see your point here but I don't think we want to use source to install packages, please, define an install function similarly to what we do for math-comp.

I don't think we want to use source to install packages

Why not ? I personally find it cleaner this way, with stuff about bignums located in ci-bignums.sh rather that who knows elsewhere. But I won't fight over this. I'll implement what you ask if I've 5min today, otherwise feel free to do what you what if that gets merged.

ejgallego · 2017-06-09T13:40:03Z

dev/ci/ci-bignums.sh

+then
+    source ${ci_dir}/ci-common.sh
+fi
+


See my comment below.

ejgallego · 2017-06-09T13:40:45Z

dev/ci/ci-color.sh

+sed -i -e "s/From Coq Require Export BigN/From Bignums Require Export BigN/" ${Color_CI_DIR}/Util/*/*.v
+sed -i -e "s/From Coq Require Import BigZ/From Bignums Require Import BigZ/" ${Color_CI_DIR}/Util/*/*.v
+sed -i -e "s/From Coq Require Export BigZ/From Bignums Require Export BigZ/" ${Color_CI_DIR}/Util/*/*.v
+


This should be eventually integrated in CoLoR, I suggest we ask Frédéric to do an 8.7 branch.

Didn't we say at the WG that this would even be a backward compatible fix?

Please be precise when you speak of compatibility, since it might mean many many things. Yes the above change is "backward compatible" in the way I've been asked to ensure : after applying this change, CoLoR may still compiles with Coq 8.6 ... at the condition that the new bignums package (available both for 8.6 and 8.7) is correctly installed first.

@ejgallego : agreed concerning the need of a 8.7 branch of CoLoR, but the sed stuff should do meanwhile (no nice overlay tricks for svn, right ?). And I don't think it's worth bothering Frédéric before this PR gets merged.

Sure as I said below I am just adding notes so we don't forget, none of my remarks is blocking.

ejgallego · 2017-06-09T13:41:28Z

dev/ci/ci-formal-topology.sh

+# Setup Bignums
+
+source ${ci_dir}/ci-bignums.sh
+


replace by install_bignum

ejgallego · 2017-06-09T13:43:10Z

dev/ci/ci-math-classes.sh

+
+source ${ci_dir}/ci-bignums.sh
+
 # Setup Math-Classes


letouzey · 2017-06-09T14:59:58Z

Rebased + removed the CompCert overlay (already merged :-).

letouzey · 2017-06-11T18:08:43Z

@maximedenes : all green here, at last :-)

See now https://github.com/coq/bignums Int31 is still in the stdlib. Some proofs there has be adapted to avoid the need for BigNumPrelude.

See PR#498 rocq-prover/rocq#498 In addition to this commit, you should also have fetched and compiled the new bignums packages before compiling math-classes : git clone https://github.com/coq/bignums.git && cd bignums && make && make install

letouzey force-pushed the outsource-bignums branch from 2970c3d to d785952 Compare March 23, 2017 18:18

silene mentioned this pull request Mar 29, 2017

A versioned, independent, development model for the standard library rocq-prover/rfcs#18

Closed

maximedenes added the needs: discussion Further discussion is needed. label Apr 10, 2017

maximedenes added needs: fixing The proposed code change is broken. and removed needs: discussion Further discussion is needed. labels May 25, 2017

ejgallego added this to the 8.7 milestone May 30, 2017

letouzey force-pushed the outsource-bignums branch from d785952 to cdc4e0f Compare May 31, 2017 17:03

maximedenes removed the needs: fixing The proposed code change is broken. label Jun 1, 2017

maximedenes added the needs: fixing The proposed code change is broken. label Jun 2, 2017

letouzey force-pushed the outsource-bignums branch from cdc4e0f to 264aca6 Compare June 6, 2017 15:20

ejgallego reviewed Jun 6, 2017

View reviewed changes

letouzey force-pushed the outsource-bignums branch from 36598d1 to ded6399 Compare June 6, 2017 20:56

ejgallego approved these changes Jun 9, 2017

View reviewed changes

ejgallego reviewed Jun 9, 2017

View reviewed changes

dev/ci/ci-bignums.sh

then

source ${ci_dir}/ci-common.sh

fi

Copy link

Member

ejgallego Jun 9, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment below.

ejgallego reviewed Jun 9, 2017

View reviewed changes

dev/ci/ci-formal-topology.sh

# Setup Bignums

source ${ci_dir}/ci-bignums.sh

Copy link

Member

ejgallego Jun 9, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace by install_bignum

ejgallego reviewed Jun 9, 2017

View reviewed changes

dev/ci/ci-math-classes.sh

source ${ci_dir}/ci-bignums.sh

# Setup Math-Classes

Copy link

Member

ejgallego Jun 9, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

letouzey force-pushed the outsource-bignums branch from ded6399 to d0c5f46 Compare June 9, 2017 14:59

This was referenced Jun 12, 2017

math-classes : switch to the external Bignums library rocq-community/math-classes#41

Merged

corn : switch to the external Bignums library rocq-community/corn#42

Merged

letouzey added 5 commits June 13, 2017 10:30

BigNums: remove files about BigN,BigZ,BigQ (now in an separate git repo)

2951071

See now https://github.com/coq/bignums Int31 is still in the stdlib. Some proofs there has be adapted to avoid the need for BigNumPrelude.

[travis] adapt CoLoR compilation to depend on the bignum package

fe972a3

[travis] overlay + extra deps for math-classes (and formal-topology)

08e86c0

[travis] extra test ci-bignums (+factorize other scripts)

0fd563c

[travis] overlay for corn

268ccbb

letouzey force-pushed the outsource-bignums branch from 22b1555 to 268ccbb Compare June 13, 2017 08:33

coqbot merged commit 268ccbb into rocq-prover:trunk Jun 13, 2017

letouzey deleted the outsource-bignums branch June 14, 2017 13:03

Zimmi48 added kind: cleanup Code removal, deprecation, refactorings, etc. and removed needs: fixing The proposed code change is broken. labels Jul 20, 2017

Zimmi48 added the zARCHIVED: standard library Previously standard library (do not use anymore, now its own repo). label Oct 2, 2017

coqbot mentioned this pull request Oct 18, 2017

The standard library should not define inconsistent notations #5026

Closed

Uh oh!

Bignums as a separate opam package #498

Bignums as a separate opam package #498

Uh oh!

Conversation

letouzey commented Mar 22, 2017

Why ?

Already done

Still to be done

Uh oh!

ejgallego commented Mar 22, 2017

Uh oh!

letouzey commented Mar 22, 2017

Uh oh!

maximedenes commented Mar 22, 2017

Uh oh!

silene commented Mar 22, 2017

Uh oh!

letouzey commented Mar 23, 2017

Uh oh!

maximedenes commented Mar 29, 2017

Uh oh!

maximedenes commented May 24, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

letouzey commented May 31, 2017

Uh oh!

ejgallego commented May 31, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maximedenes commented May 31, 2017

Uh oh!

ejgallego commented May 31, 2017

Uh oh!

ejgallego commented May 31, 2017

Uh oh!

ejgallego commented May 31, 2017

Uh oh!

letouzey commented May 31, 2017

Uh oh!

letouzey commented May 31, 2017

Uh oh!

maximedenes commented Jun 2, 2017

Uh oh!

maximedenes commented Jun 6, 2017

Uh oh!

letouzey commented Jun 6, 2017

Uh oh!

letouzey commented Jun 6, 2017

Uh oh!

letouzey commented Jun 6, 2017

Uh oh!

letouzey commented Jun 6, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maximedenes commented Jun 9, 2017

Uh oh!

ejgallego commented Jun 9, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

maximedenes commented May 24, 2017 •

edited

Loading

ejgallego commented May 31, 2017 •

edited

Loading

ejgallego commented Jun 9, 2017 •

edited

Loading