Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@nuko8
Copy link

@nuko8 nuko8 commented Oct 1, 2017

Currently, in Vim, there are two different classes of configuration for macOS.
One is with --enable-darwin (default) and the other with --disable-darwin.

The former takes advantage of the API specific to macOS, thereby enabling system clipboard integration and character set conversion of which encoding is specific to that OS. The latter is identical with what is commonly used for other UNIX-like platforms.

When "the darwin feature of Vim" (h: mac-darwin-feature) was merged into our code base, it appears that some adjustments were done incompletely for some reason.

As a result, for non-darwin Vim, has('mac') returns 0, and :version doesn't displays anything which is compared to "MacOS X (unix) version" for darwin Vim.

That makes it hard to write Vim scripts for non-darwin Vim.

That's not the only issue. Even for the darwin configuration/build, the configure script refers to the feature as MACOS, the compiler command-line calls it MACOS_X_UNIX, a header file gives it two different nicknames, MACOS_X and MACOS, and they are used in several *.c files in an obscure way. That's anything but manageable.

Also, despite the fact that the support for Mac OS Classic has been discontinued since Vim 7, there are quite a few remnants of it in our code base. Obviously, that gives unnecessary overhead to code reading and maintenance.

One could look away the last two issues and write a quite simple patch to fix the primary issue. Actually, I did that. But that had made me uneasy because I knew such a way of solution would only get the situation worse and get back at me someday, even if it could be helpful for the moment.

My new solution is to examine every use of the MACOS.* macros, determine what each is meant for, and get the build system neat and tidy on the basis of that research. This is not an attempt to change the build system in favor of someone's view or preferences, but an attempt to give it a better shape so that it will clarify itself, what each part of it does and how each relates to other. I'd like to call it a refactoring, even if it happens to include substantial changes such as bug fixes as "side effects" of it.

To ease the review, the whole patch is logically divided into several commits (ie., it's not a chronological accumulation), and each commit addresses its own issue only. Honestly, I'm not sure how much it could help people review the patch, but both the whole of it and each of the commits constituting it are easily available to them on GitHub (AFAIK), I try using it.

The patch consists of about 20 commits. The first three commits will clarify the goal. The three commits following them fix some breakage in the build system resulted from them. Then many minor adjustments follow. The last two commits clean up what I think is the root cause of the headache. Please refer to the commit messages for details.

Sorry for the wall of text and thank you for taking your time. I welcome any comment on the proposed patch and open to any proposal.

@codecov-io
Copy link

codecov-io commented Oct 1, 2017

Codecov Report

Merging #2178 into master will decrease coverage by <.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2178      +/-   ##
==========================================
- Coverage   74.21%    74.2%   -0.01%     
==========================================
  Files          90       90              
  Lines      131978   131984       +6     
  Branches    30862    28988    -1874     
==========================================
  Hits        97942    97942              
- Misses      34011    34017       +6     
  Partials       25       25
Impacted Files Coverage Δ
src/main.c 55.12% <ø> (ø) ⬆️
src/if_ruby.c 56.46% <ø> (ø) ⬆️
src/mbyte.c 59.83% <ø> (ø) ⬆️
src/option.c 84.47% <ø> (ø) ⬆️
src/evalfunc.c 83.34% <ø> (ø) ⬆️
src/undo.c 81.76% <ø> (ø) ⬆️
src/window.c 81.09% <ø> (+0.06%) ⬆️
src/message.c 68.26% <ø> (ø) ⬆️
src/version.c 80.3% <ø> (ø) ⬆️
src/pty.c 55.55% <ø> (ø) ⬆️
... and 20 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 19a3d68...1768ab0. Read the comment docs.

src/vim.h Outdated
* MACOS_X compiling for MacOS X
* MACOS_X_DARWIN compiling with the darwin feature for MacOS X
* MACOS compiling for either one
* MACOS compiling for any of Macintosh operating systems
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder what this is for, now that Classic has been ripped out. There are now no Macintosh operating systems supported by Vim other than "MACOS_X".

@nuko8
Copy link
Author

nuko8 commented Oct 1, 2017 via email

@nuko8
Copy link
Author

nuko8 commented Oct 2, 2017

Having replied to @chdiza , just to be sure for myself, I looked over all the places where MACOSs are used this morning, and found that some of them still looked fishy.

For example, we have two MACOSs in misc2.c:newenv(). But if we combine that fact with the declaration of environ in misc2.c, taking it into account that Mac OS X has the system global environ(7) (Not sure about its early versions, though), we see that those two MACOSs are a wrong substitute for MACOS_CLASSIC.

Probably, they have been sitting there since MACOS_CLASSIC was introduced, and overlooked because they are obviously harmless for Vim to work properly. But..., yeah, I'll look into this new issue and update the PR if necessary. Maybe, after making sure that such a misuse is completely expelled from the code base, I'll replace MACOS with MACOS_X, because the former can be interpreted as either "Mac OS" or "macOS", or both. Eliminating this sort of ambiguity is within the scope of the PR, anyway.

nuko8 added 26 commits October 3, 2017 09:02
Currently, MACOSX is defined only when the darwin feature is enabled.

The variable has been used to specify platform-specific build parameters
needed for the if_* modules and the Carbon GUI to compile and link.

But this is strange since those modules are inherently irrelevant to the
feature.

Also, there's no good reason to exclude any of the modules from
non-darwin featured builds.

To correct those unreasonable things,

1.  Abandon MACOSX.
2.  Define the new variable MACOS_X for all macOS build, and have it
    take over the role of MACOSX.
3.  There has been a place where MACOSX is truly meant for the darwin
    feature, that is, the block where Cocoa is conditionally added to
    $LIBS.  To this particular usage, it isn't reasonable to replace
    MACOSX with MACOS_X.  To address this, introduce a new variable
    called MACOS_X_UNIX dedicated to the block.

Be careful not to confuse this MACOS_X shell variable with the C macro
of the same name currently defined in vim.h and used throughout the
source files.  That possible confusion will be resolved with some of
patches of this patch set afterwards by changing the current usage of
the C macro meticulously so that its semantics will be consistent with
that of the shell variable.

For macOS builds, making the names of the shell variables and those of
the corresponding C macros consistent is one of the goals of the patch
set.  Note that, with this patch, that goal is partially accomplished
for MACOS_X_UNIX.
Currently, when the darwin feature is enabled, Vim is unconditionally
linked against Cocoa, but that's too much for most of the cases.

For Tiny, os_macosx.m is excluded from the build to avoid link errors.
Then, if +multi_byte is not required for the build, we could exclude
os_mac_conv.c, too.  In other words, we don't need the empty
os_mac_conv.o to build an -multi_byte Vim.

As to Small, it's possible for Vim to have +clipboard with +multi_byte.
But that obviously compromises the interoperability of the clipboard.

This patch corrects those unreasonable things.

The following table summarizes the proposed changes:

                          (*1)
Feature             Clipboard   Extra Files             Framework
====================================================================
      -multi_byte   disabled    None                    None
tiny  --------------------------------------------------------------
      +multi_byte   disabled    os_mac_conv.c           CoreServices
--------------------------------------------------------------------
      -multi_byte   enabled     os_macosx.m             AppKit (*2)
                                os_mac_conv.c (vim#3)
small --------------------------------------------------------------
      +multi_byte   enabled     os_macosx.m             AppKit
                                os_mac_conv.c
--------------------------------------------------------------------

*1: For macOS, FEAT_CLIPBOARD is defined in vim.h for FEAT_SMALL.
*2: The linker automatically adds CoreServices in favor of AppKit.
*3: As mentioned above, +clipboard prefers +multi_byte.
Make the Mac-related feature list items easier to use by overhauling
macOS-relevant part of the build system.

The result can be skimmed easily only by reading the changes of a few
lines in runtime/doc/eval.txt.

With the changes, non-darwin Vim on macOS returns 1 for has('mac')
instead of 0.

To accomplish that, we could introduce a new compile flag to indicate
that the host system is macOS.  This is the easiest way to address the
issue, but there's a thing to note.

Why has non-darwin Vim returned 0 for has('mac')?  Why has that been
overlooked or remainted unfixed?

Personally, I think one of the reasons is that the transposition of
MACOS_X and MACOS_X_UNIX; the former is defined only if the latter is
defined.  While MACOS_X looks like covering all builds, it actually
doesn't.  It's a shorthand of MACOS_X_UNIX.  Very counter-intuitive.
MACOS_X should represent all the builds, and MACOS_X_UNIX should
represent the subset of MACOSX_X covering the darwin feature; by that,
the subset of non-darwin builds are represented even better.

Also, in src/configure.ac, what is referred to as "darwin" in our
documents is represented by the shell variable MACOSX, while in the
source code, the same notion is represented by MACOS_X_UNIX or MACOS_X
as its shorthand.  Context-dependent various symbols for the same single
notion.  Even if one can handle them with care, they are definitely very
confusing.  Preferably, they should be renamed so that people can work
with them comfortably.

Hence, it is true that adding a new flag solves the current problem
quite easily, but the whole situation will get more convoluted.  We need
an overhaul to get it back to be managable.

To accomplish that, this patch changes the semantics of the C macro
MACOS_X so that it will represent all the builds on macOS.

In theory, for darwin builds, the change shouldn't affect them except
for Tiny without multi_byte since it's identical with that for
non-darwin.

For non-darwin builds, we have to examine each occurance of MACOS_X,
MACOS_X_UNIX and MACOS in the source code one by one to determine what
exactly it has been meant for, and correct it without breaking darwin
builds if necesary.

Basically, patches following this commit are for that laborious task;
they will fix breakages or side effects resulting from the proposed
changes with this commit.

To begin with, they will fix breakages in the build system so that we
will run 'make' and 'make test' successfully for every possible build on
macOS.  After them, patches for fixing breakages or side effects which
are hard to catch will follow.
This is the only case where a darwin build falls back to that for
non-darwin (*1).

The cause of the breakages is that os_mac.h is now included even for
non-darwin builds, which results in some conflict with os_unix.h.

This issue could be solved quite easily by not including the header file
against non-darwin builds.

However, since non-darwin builds are those on macOS, people hardly see
why os_mac.h is excluded for them.  Rather, we should modify os_mac.h so
that it can coexist with os_unix.h in favor of the MACOS_X of the new
semantics.

To be more specific,

1.  Change around USE_CR: Without this, 'make test' would abort at a
    very early phase.
2.  Change around MAXPATHL: Without this, MAXPATHL and BASENAMELEN would
    conflict with those coming from os_unix.h.
3.  Change around SIGPROTOARG: Without this, HAVE_AVAIL_MEM would remain
    defined and cause a link error.

With this commit, 'make' and 'make test' should work for all dawrin
builds.

*1: The darwin feature was made on top of the previous build system
    where non-darwin was default.  Hence, although the current default
    builds are darwin, this sort of fallback, that restoring to
    non-default value, happens.
If MACOS_CONVERT is set, some of the object files are compiled to have
references to the functions defined in os_mac_conv.c which implement
part of the darwin feature.

Therefore, the macro shouldn't be defined for non-darwin builds.

With this commit, 'make' and 'make test' should work for all non-darwin
builds except 'make test' for the GUIs on XQuartz.
Since MACOS_X is now defined for non-darwin builds, MACOS is defined for
them, too.  This makes Test_set_guioptions() fail against the GUIs on
XQuartz.

To fix that, MACOSX is replaced with FEAT_GUI_MAC.  Obviously,
FEAT_GUI_MAC would be still more appropriate than MACOS here even if
the semantics of MACOS_X wasn't changed.  It is worth doing that for
it's own sake.

With this commit, 'make test' should work for all non-darwin builds, in
particular, the GUIs on XQuartz.
For non-darwin, MACOS_X is defined while MACOS_CONVERT is not.  Let
non-darwin rely only on libiconv.
Currently, the GUIs on XQuartz, which are all non-darwin, don't work
with the -f command-line option.  This commit fixes it.
Although if_python.c, if_python3.c and if_ruby.c refer to MACOS_X_UNIX,
at several places, none of them uses the functionality provided by
os_mac_conv.c or os_macosx.m, i.e., the darwin feature.  Rather, those
modules should be independent each other.

We should replace MACOS_X_UNIX with MACOS_X there.
All the occurrence of MACOS_X_UNIX should be replaced with MACOS_X since
they aren't used to specify the darwin feature.
Since this option is only for the Mac GUI, it's of no use for the GUIs
on XQuartz. Let E519 work for them, instead.
Conversion of LF into CR-LF is not specific to darwin alone.
Since MACOS_X_UNIXs here are irrelevant to the darwin feature, they
should have been MACOS_X.
The condition defined(MACOS) already appears to define FEAT_HUGE.
sys/errno.h defines EILSEQ if __DARWIN_C_LEVEL >= __DARWIN_C_FULL, and
the condition holds true for newer Mac OS X.
After all, MACOS_X_UNIX is meant for taking advantage of macOS-specific
API rather than for making use of Unix-proper API.

Probably, this is the last piece to prevent the lasting misunderstanding
and confusion.

Still, the name remains confusing because Darwin is the name of the
kernal of macOS.  Hopefully, someday someone will give a better name to
what we call "the darwin feature of Vim" today.
Although the support for the platform in question has been discontinued
since Vim 7, there're still remnants of it.  Remove them.

N.B. At some places, MAC_OS_CLASSIC has been used.  This may be a typo
or a direct reference to a macro defined in a system header file.
Either way, it is supposed to share the same fate with MACOS_CLASSIC.
Now that MACOS_CLASSIC is completely left out, the condition
"defined(MACOS) && !defined(MACOS_X)" is identically false.  Remove the
code blocks subject to the condition.

Also, if_ruby.c has had the macro 'macintosh' which is considered a
synonym for MAC_OS_CLASSIC.  Naturally, that will be removed, too.

Finally, macOS has <sys/ioctl.h>, and the third parameter of open(2) is
varargs.  Adjust pty.c accordingly.
It is misleading that MACOS sounds like it still covers MAC OS Classic.
Those two MACOSs are a wrong substitute for MACOS_CLASSIC, since Mac OS
X has the global environ(7) variable.
These are the pre-defined macros of compilers for Mac OS Classic.
That #undef is unnecessary for the recent versions.

The last change to the block is dated 2008-08-09, and the latest Mac OS
X those days was 10.5.  So...
Seems like the macro MACOS here is actually meant for MACOS_CLASSIC.
No reason to exclude the block for Mac OS X.
- Now that MACOS_X and MACOS_X_DARWIN have been made distinguishable,
  and MACOS_CONVERT is part of the latter, use MACOS_CONVERT for the
  hunk of lines 516--523 since the MACOS there is meant for
  MACOS_X_DARWIN.
- Mac OS X has mblen(3).
@nuko8 nuko8 force-pushed the mac-build-overhaul branch from fdab4b0 to 5cb2687 Compare October 3, 2017 13:35
@nuko8
Copy link
Author

nuko8 commented Oct 3, 2017

I'll look into this new issue and update the PR if necessary.

Done with the last six commits.

I'll replace MACOS with MACOS_X, ...

Will be done tomorrow.

nuko8 added 7 commits October 4, 2017 19:52
Now that os_mac.h is included via vim.h for all builds for Mac OS X, the
declaration is no longer needed.
Now that none of MACOSs that are meant for MACOS_CLASSIC remains,
we can safely replace MACOS with MACOS_X.
For non-darwin Tiny and Small builds for Mac OS X having XQuartz
installed, the resulting executable is linked against X11 libraries
since mblen() is inadvertently replaced with _Xmblen().

To fix that, add a check for mblen() to configure.ac, add a check for
HAVE_MBLEN to after the checks for __amigaos4__ and __ANDROID__, move
the preprocessor block for replacing mblen() with _Xmblen() to the place
where Xlocale.h is included in order to make the code cleaner, and
correct the indentation level of a #if block to reflect the nesting
depth correctly.

This commit should also fixes unexpected X11 dependency seen in some
multibyte-enabled builds on other platforms.
Now that MACOS_X is alwayed defined for any macOS builds, some #if
statements are evaluated to be constant, and thus redundant.  Reduce
them.
@chdiza
Copy link

chdiza commented Oct 28, 2017

I am unclear as to why a build of a console-only app is now linked to the AppKit framework. The system vim that comes with macOS doesn't have this.

@nuko8
Copy link
Author

nuko8 commented Oct 29, 2017

I am unclear as to why a build of a console-only app is now linked to the AppKit framework.

Before the inclusion of the patch, that link had been done against Cocoa, and this implies that the app is also linked against AppKit; the linking is done by the linker at build time implicitly to resolve Cocoa's dependency automatically.

For Vim to support the darwin feature, in particular, system clipboard integration, it needs NSPasteboard of AppKit and hence has always been linked against it when the feature is included.

The system vim that comes with macOS doesn't have this.

This is because, since it doesn't support system clipboard integration, there's no need for AppKi or Cocoa for it.

I've never heard of any reason why Apple avoids the feature, but one thing sure to me is, if one runs Mac without GUI (i.e., console mode) and invokes a vim which is linked against any of GUI frameworks such as AppKIt or Cocoa, launchd will probably start up the windowing system and prevent him from keep working in console mode.

@chdiza
Copy link

chdiza commented Oct 29, 2017

By "console-only", I meant in-terminal-only, not the special "console" mode in which Aqua isn't even running.

But thanks for the explanation, that makes sense.

adizero pushed a commit to adizero/vim that referenced this pull request May 19, 2018
Problem:    Mac features are confusing.
Solution:   Make feature names more consistent, add "osxdarwin". Rename
            feature flags, cleanup Mac code. (Kazunobu Kuriyama, closes vim#2178)
janlazo added a commit to janlazo/neovim that referenced this pull request Aug 24, 2019
Problem:    Mac features are confusing.
Solution:   Make feature names more consistent, add "osxdarwin". Rename
            feature flags, cleanup Mac code. (Kazunobu Kuriyama, closes vim/vim#2178)
vim/vim@d057301
justinmk pushed a commit to neovim/neovim that referenced this pull request Aug 24, 2019
Problem:    Mac features are confusing.
Solution:   Make feature names more consistent, add "osxdarwin". Rename
            feature flags, cleanup Mac code. (Kazunobu Kuriyama, closes vim/vim#2178)
vim/vim@d057301
@ychin ychin mentioned this pull request Aug 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants