Thanks to visit codestin.com
Credit goes to github.com

Skip to content

CLI (redux) #6133

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Feb 26, 2022
Merged

CLI (redux) #6133

merged 20 commits into from
Feb 26, 2022

Conversation

ethomson
Copy link
Member

@ethomson ethomson commented Dec 6, 2021

A command-line interface for libgit2. The goal is for it to be git-compatible.

  1. The libgit2 developers can more easily dogfood libgit2 to find bugs and performance issues.

  2. There is growing usage of libgit2's examples as a client; libgit2's examples should be exactly that - simple code samples that illustrate libgit2's usage. This satisfies that desire for a client more directly.

  3. By producing a client ourselves, we can better understand the needs of client creators, possibly producing a shared "middleware" for commonly-used pieces of client functionality like interacting with external tools.

  4. Since git is the reference implementation, we may be able to benefit from git's unit tests, running their test suite against our CLI to ensure correct behavior.

We've done a lot of the preliminary refactoring work already, so this is a relatively small set of changes (at least compared to #5507).

  1. The source tree is rearranged: the "pure" utility code is moved into the util folder and the remainder of the libgit2 code is moved into the libgit2 folder. This makes it easier to use the utility code from other tools (like a CLI).

  2. A new executable has been added. At present, all the commands are linked into a single executable, which statically links libgit2, but the design is intended to be flexible enough to produce a binary-per-command or to dynamically link libgit2 if this proves useful.

  3. Two command-line tools were introduced, cat-file and hash-object. I chose these since they're relatively simple - one of the few examples of git commands that more or less do a single thing - and having the ability to inspect the ODB is going to be extremely useful as we start looking at SHA256.

@twaritwaikar
Copy link

I am not a libgit2 developer but I use libgit2 as a tool in both my hobby and work, so just wanted to get some understanding of this change.

If I understand correctly, this is going to be a CLI which allows interfacing to libgit2 functions directly? So is it like a replacement for git, but for power users? (Power user meaning users that would use it like Git but with more explicit plumbing commands)

Or is it a completely CI helper sort of thing which makes CI and testing easier?

And is there a possibility that the CLI ends up being a faster, more advanced replacement for Git?

Thanks! And thank you for maintaining this library!

@NattyNarwhal
Copy link
Contributor

I get the impression it's basically turning the examples (which are kinda simple reimplementations of Git commands or similar) into a more fully featured porcelain of its own. That'd be useful for CI, making sure git has independent reimplementations, etc.

I think this is a good idea for other libgit2 users; without the samples, I'd have been lost in using some functionality. Some stuff is also hairy enough (i.e. rebases) I'd like to see a sample to make sure I feel confident in my approach, so having a realistic implementation could be great there.

@ethomson ethomson force-pushed the ethomson/cli_redux branch 4 times, most recently from 2d96ebc to 2b140a3 Compare December 10, 2021 13:44
@boretrk
Copy link
Contributor

boretrk commented Dec 11, 2021

Overall I think this is a great idea. Being able to use some of gits unit tests would hopefully avoid a lot of issues people encounter.

I agree with @pks-t in #5507 that the util library seems a bit out of place, but maybe I misunderstand the purpose of it.
Is the idea that util mainly is a collection of functions that is shared between the cli and libgit2 or is the intention that all users should use functions in util too?

If it is the former it sort of removes the dogfooding aspect of this change.
IMO the ability to use gits unit tests still justifies the change.
https://github.com/libgit2/libgit2/blob/main/docs/projects.md indicates that an earlier idea was to go this way with the example code, but that leads to poor examples since most code will be unrelated to the library.
Having a separate cli and making the example code less functional and more explanatory would be better I think.

If it is the latter case, why is it better to have it in a separate area rather than with the rest of the library?
More specifically, how would a developer know if a function belongs in one or the other?

@boretrk
Copy link
Contributor

boretrk commented Dec 11, 2021

An alternative would be to keep the cli in a separate project. If it is "just another library user" then the cli and the adapted git tests can be kept separately.
The downside to that approach is that changes that breaks the cli tests won't be caught until they find their way into the cli project.

@ethomson
Copy link
Member Author

ethomson commented Dec 11, 2021

Is the idea that util mainly is a collection of functions that is shared between the cli and libgit2

That's right. Users should not be using the things in util, and we won't export them as public API. It's things like our string manipulation functions. But it's not shared state or only intended to be used by just the cli and libgit2.

You can imagine that our CLI - really any C application - needs some utility library to do things like at a level above the standard C library, and give us cross-platform mechanisms for doing so. Working with strings, reading files, slightly more complex data structures like stacks and such. We could do this with something like glib. And, in fact, if we were to reboot libgit2 then using glib would probably be smart. But I don't really want to add a new dependency for the CLI when we've already got our own utility functions to do these things, and in a way that the developers are comfortable with.

In fact, I think about the util folder in very much the same way I think about glib. It's grown out of libgit2 to be its own standalone utility library (just like glib grew out of gimp). And so it should be able to be its own standalone library that anybody could use. I intend to pull a copy of that folder out, call it gu (for "git utilities") and s/git_/gu_/g and make it available as a standalone project. I want this utility library in a different project that I'm working on

So I'm looking at util as a standalone dependency that both libgit2 and the CLI depend on. We could go "all in" on this plan and move util into deps, and give it its own name and unique function signatures (eg gu_str instead of git_str). But ... I don't know, it feels a little ... heavy handed? Presumptuous?

My goal was to take an incremental approach towards getting there. But maybe what we have here is too noncommittal and vague.

If it is the former it sort of removes the dogfooding aspect of this change.

I hope not! The CLI is intended to only ever interact with the public functions of libgit2. The CLI very intentionally only includes include/git2/*.h so that it only talks to public API.

None of the utility functions are actually necessary to interact with libgit2, and we shouldn't ever share any data between the cli and libgit2 through the utility library.

(This is actually more aspirational than fact today. The error handling code does share state because I haven't unravelled that piece yet. I'm a bit torn on how to handle it - should the utility code to have its own separate error handling, or should it have none and users bring their own, or some optional pluggable model... I just haven't found a model that I love yet, but we certainly could make a quick and dirty change to make it happen.)

Does this clarify the goals here or are you still concerned?

@boretrk
Copy link
Contributor

boretrk commented Dec 11, 2021

No, it clarifies the idea behind it pretty well.
I with that goal I don't really think it is a problem to move the functionality to deps at all but I also don't see it as something that needs to be hurried.

I'm not exactly sure how to phrase my concern regarding the dogfooding part.
I guess I think that it could be easy to be content with a library interface that is only easy to use as long as you have the util library, but as long as the library is available externally and/or one keeps that in mind then that might not be a problem.

@ethomson
Copy link
Member Author

guess I think that it could be easy to be content with a library interface that is only easy to use as long as you have the util library, but as long as the library is available externally and/or one keeps that in mind then that might not be a problem.

Thanks - I want to be mindful of this as that's definitely very much not the goal. Certainly I will keep this in mind but I will think about some ways that we might be able to mitigate this. 🤔

@ethomson ethomson force-pushed the ethomson/cli_redux branch 2 times, most recently from b319dfe to 22dac6f Compare January 28, 2022 15:37
@ethomson ethomson force-pushed the ethomson/cli_redux branch 4 times, most recently from d6d27af to 3f88889 Compare February 16, 2022 04:10
Newer gcc is complaining about `object` being potentially not
initialized; initialize it.
The `git_error_set` function is useful for callers who implement
backends and advanced callbacks.  Expose it.
Instead of simply including the utility files directly, make them a
cmake object library for easy reusability between other projects within
libgit2.

Now the top-level `src` is responsible for platform selection, while the
next-level `libgit2` and `util` configurations are responsible for
identifying what objects they include.
The `git2internal` target is actually the git library; call it such so
that IDE users have visibility into it.
Like we want to separate libgit2 and utility source code, we want to
separate libgit2 and utility tests.  Start by moving all the tests into
libgit2.
@ethomson ethomson force-pushed the ethomson/cli_redux branch 3 times, most recently from 80c87a8 to 53c6f65 Compare February 26, 2022 18:22
Introduce a command-line interface for libgit2.  The goal is for it to
be git-compatible.

1. The libgit2 developers can more easily dogfood libgit2 to find bugs,
   and performance issues.

2. There is growing usage of libgit2's examples as a client; libgit2's
   examples should be exactly that - simple code samples that illustrate
   libgit2's usage.  This satisfies that need directly.

3. By producing a client ourselves, we can better understand the needs
   of client creators, possibly producing a shared "middleware" for
   commonly-used pieces of client functionality like interacting with
   external tools.

4. Since git is the reference implementation, we may be able to benefit
   from git's unit tests, running their test suite against our CLI to
   ensure correct behavior.

This commit introduces a simple infrastructure for the CLI.

The CLI is currently links libgit2 statically; this is because the
utility layer is required for libgit2 _but_ shares the error state
handling with libgit2 itself.  There's no obviously good solution
here without introducing annoying indirection or more complexity.
Until we can untangle that dependency, this is a good step forward.

In the meantime, we link the libgit2 object files, but we do not include
the (private) libgit2 headers.  This constrains the CLI to the public
libgit2 interfaces.
Our argument parser (https://github.com/ethomson/adopt) includes a
function to print a usage message based on the allowed options.  Omit
this and use a cutom function that understands that we have subcommands
("checkout", "revert", etc) that each have their own options.
Add a framework for commands to be defined, and add our first one,
"help".  When `git2_cli help` is run, the `cmd_help` function will be
invoked with the remaining command line arguments.  This allows users to
invoke `git2_cli help foo` to get information about the `foo` subcommand.
Support `help <command>` by re-invoking the command itself with the
`--help` argument.  This allows us to keep the help logic with the
commands itself.
Introduce a simple command that emulates `git cat-file`.
Introduce a simple command that emulates `git hash-object`.
@ethomson
Copy link
Member Author

OK, I feel like this has been waiting long enough. Now that v1.4 is out the door... 🚢

@ethomson ethomson merged commit 1327dbc into main Feb 26, 2022
@ethomson ethomson deleted the ethomson/cli_redux branch February 26, 2022 20:10
@ethomson ethomson mentioned this pull request Feb 26, 2022
DavidKorczynski added a commit to DavidKorczynski/oss-fuzz that referenced this pull request Mar 16, 2022
AdamKorcz pushed a commit to google/oss-fuzz that referenced this pull request Mar 16, 2022
@carlosmn
Copy link
Member

Sorry it took me so long to go through this, I had mapped out a couple of thoughts in my head and then I guess never typed it out, oops.

So overall, yeah it makes sense to have a cli even if it's just to show some pattern and/or make sure the API makes some sense. My only comment there is that, if the main focus of this repository is the library, I would have expected src/ to directly contain the library code and the cli to be elsewhere (a subdir, or another dir in the top-level) but that's rather minor.

I would also be cautious with making the util library too general, lest you end up also trying to maintain another glib.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants