you dont want to use find_package inside the FindPCRE2.cmake module
instead you will want to check for the presence of the config and fall back to module mode from the top-level CMakeLists:

# try config mode
find_package(PCRE2 ... CONFIG)
if (NOT PCRE2_FOUND)
  # fallback to module mode
  find_package(PCRE2 MODULE REQUIRED)
endif ()

else the internal imported target in the FindPCRE2 module in a manner that matches the exported target is the right way to go

note that if pcre2 is compiled using autotools no cmake config file is generated

instead you will want to check for the presence of the config and fall back to module mode

Yes, that's what I'm doing by using find_package(PCRE2 CONFIG). CONFIG or NO_MODULE tells CMake to use config mode search first, and then if this succeeds. I'm not sure why this shouldn't be done from the find module as the logic is still 1. do config mode search 2. if failed, then use heuristics. The CMake-provided FindCURL uses this exact pattern.

note that if pcre2 is compiled using autotools no cmake config file is generated

Yes, that's why there's heuristic logic for the pcre2.h header, etc. Aside from the CI checks I've tested locally:

• PCRE 10.45, 10.46 built with CMake, on both WSL1 Ubuntu 22.04 and 64-bit Windows
• PCRE 10.39 that came installed with WSL1 Ubuntu 22.04 and has no CMake config support

Oops, there was one commit I forgot to push. It basically just adds a find_package_handle_standard_args call like in FindCURL if config mode search succeeds to report on the located components.

Hm, got a couple failing CI jobs. Not sure how they're related to this PR however.

One failure is with linux / r gcc (pull_request) with an R segfault with the li_boost_shared_ptr C++ test.

make[1]: *** [Makefile:44: li_boost_shared_ptr.cpptest] Error 1

I see several warnings like this for each of the Boost shared_ptr tests (manually wrapped for readability):

In file included from /usr/include/boost/smart_ptr/detail/requires_cxx11.hpp:9,
                 from /usr/include/boost/smart_ptr/shared_ptr.hpp:17,
                 from /usr/include/boost/shared_ptr.hpp:17,
                 from li_boost_shared_ptr_wrap.cpp:1523:
/usr/include/boost/smart_ptr/detail/requires_cxx11.hpp:19:1: note: ‘#pragma message: C++03 support was deprecated
in Boost.SmartPtr 1.82 and will be removed in Boost.SmartPtr 1.84. Please open an issue in
https://github.com/boostorg/smart_ptr if you want it retained.’

Totally unrelated issue with windows / python 3.13 msvc (pull_request) where linking is failing:

LINK : fatal error LNK1104: cannot open file 'python313t.lib'

Seems like this is a known issue with 3.13.4 (fix in 3.13.5 it seems).

I an novice when it comes to CMAKE.
I do have 2 general questions:

1. Can the change, make the use of CMAKE more simply?
   In .github/workflows/windows-cmake.yml
   Can we for example remove the
   -DPCRE2_INCLUDE_DIR=...
-DPCRE2_LIBRARY=...
2. Can we add a cmake test on Ubuntu.

I personally prefer to see an actual change in the testing when we make changes 😄

Hey, sorry for getting back to this late. I can respond to some of your questions:

1. Yes. My changes to FindPCRE2.cmake are to better integrate with typical CMake practices, so in this case, it means defining the PCRE2 install root with -DPCRE2_ROOT=/path/to/pcre2 in the CMake command, or if PCRE2 is installed in some system location on the CI machine, e.g. ./usr/local/, you might be able to omit these altogether.
2. Yes, I can try and do that. Definitely agree with your statement about testing.

For 2., would that mean adding something like .github/workflows/windows-cmake.yml but for Ubuntu? Like a linux-cmake.yml, with similar actions as the existing windows-cmake.yml?

Hey, sorry for getting back to this late. I can respond to some of your questions:

1. Yes. My changes to FindPCRE2.cmake are to better integrate with typical CMake practices, so in this case, it means defining the PCRE2 install root with -DPCRE2_ROOT=/path/to/pcre2 in the CMake command, or if PCRE2 is installed in some system location on the CI machine, e.g. ./usr/local/, you might be able to omit these altogether.
2. Yes, I can try and do that. Definitely agree with your statement about testing.

For 2., would that mean adding something like .github/workflows/windows-cmake.yml but for Ubuntu? Like a linux-cmake.yml, with similar actions as the existing windows-cmake.yml?

Seems logic to use linux-cmake.yml.
It is better to verify with @wsfulton.
He decide on naming 😄

A linux-cmake.yml would make sense.

Relevant docs will need to be updated in the same pull request too, yes.

Oops, totally missed the comment. But ok, this sounds good. I think the relevant doc is Doc/Manual/Windows.html?

Doc/Manua

Yes, "Building swig.exe using CMake"

Hey, I have some time and have come back to this PR.

I am tweaking windows-cmake.yml to ensure that the "new" PCRE2 detection by CMake works on CI with NuGet PCRE2. Am also working on the linux-cmake.yml while concurrently updating the SWIG HTML docs for Windows CMake.

Also unrelated to the actual work in this PR, but is there a way to get the AppVeyor jobs to run concurrently with the GitHub Actions jobs? CI job throughput would of course increase if we didn't have 100+ GitHub Actions jobs waiting for several serialized AppVeyor jobs (which takes like 20+ minutes overall) to finish.

Also unrelated to the actual work in this PR, but is there a way to get the AppVeyor jobs to run concurrently with the GitHub Actions jobs? CI job throughput would of course increase if we didn't have 100+ GitHub Actions jobs waiting for several serialized AppVeyor jobs (which takes like 20+ minutes overall) to finish.

We reduce AppVeyor jobs to 3 jobs because of the serialising issue.
This is why we add windows.yml, to reduce the jobs in AppVeyor.
We used to run AppVeyor jobs for 6 hours in average compare to 1 hour in average of today.
I did submit #2976 for testing python with MingW64 on GitHub windows runner in the hope we can reduce the jobs on AppVeyor further.
The PR is under review.

Anyhow the average time of all Linux testing running on GitHub Ubuntu is around 1 hour.
So usually the AppVeyor testing ends around 10 to 15 minutes after the Linux testing ends.
It is by far better than what we had in the past.

We reduce AppVeyor jobs to 3 jobs because of the serialising issue. This is why we add windows.yml

Hm, I see. I guess compared to before there's already a big improvement, but I'm not sure why GitHub Actions will not let the GitHub Actions jobs run concurrently with the AppVeyor job. That seems a bit strange to me (unless it is by design).

I did submit #2976 for testing python with MingW64 on GitHub windows runner

One thing I had considered doing was seeing if the CMake config could be used to replicate the current Makefile example setup. CMake scripting in my opinion is an improvement over autotools because it works natively on Windows. That is, we can build SWIG using native MSVC, without MinGW or MSYS2, and then create native DLL modules for each language. Since CMake handles both configuration + build, we can add the necessary configuration logic the way we want to.

I've worked CMake pretty hard in both my personal and work projects. My last job, I was maintaining a C++ library with some SWIG modules I wrote, and had CMake configure C#, Python, and Java SWIG wrapping tests (there were some typemaps I had written in those .i files) as part of the build, although only conditionally (e.g. skip C# if no C# compiler). However, I know that SWIG has a very large multi-language test suite, so this kind of replication is nontrivial. I may do some experimentation in a feature branch, but I wouldn't have much expectation of merging to master :)

We reduce AppVeyor jobs to 3 jobs because of the serialising issue. This is why we add windows.yml

Hm, I see. I guess compared to before there's already a big improvement, but I'm not sure why GitHub Actions will not let the GitHub Actions jobs run concurrently with the AppVeyor job. That seems a bit strange to me (unless it is by design).

AppVeyor serialize by user, so each user can run a single job.
Also the resources allocate to SWIG are limited, which means you are queue with other jobs from other users.
While GitHub Actions run in parallel per user, as long as there are GitHub runners available.
You can think like that:
GitHub provides ~10 runners.
While AppVeyor provides 1 runner.

I did submit #2976 for testing python with MingW64 on GitHub windows runner

One thing I had considered doing was seeing if the CMake config could be used to replicate the current Makefile example setup. CMake scripting in my opinion is an improvement over autotools because it works natively on Windows. That is, we can build SWIG using native MSVC, without MinGW or MSYS2, and then create native DLL modules for each language. Since CMake handles both configuration + build, we can add the necessary configuration logic the way we want to.

I've worked CMake pretty hard in both my personal and work projects. My last job, I was maintaining a C++ library with some SWIG modules I wrote, and had CMake configure C#, Python, and Java SWIG wrapping tests (there were some typemaps I had written in those .i files) as part of the build, although only conditionally (e.g. skip C# if no C# compiler). However, I know that SWIG has a very large multi-language test suite, so this kind of replication is nontrivial. I may do some experimentation in a feature branch, but I wouldn't have much expectation of merging to master :)

This is a question to @wsfulton.
I do not know if the SWIG project is willing to replace autoconf with CMake.
The current windows.yml use both MSVC and mingW64 compilers, and we need to test both.
As some users are unable or unwilling to use MSVC on Windows.
We try to support our users, not tell them what to do.
If you can support both MSVC and mingW64 compilers, then we can talk about using CMake. MSYS2 is just environment used to install the mingW64 compiler and the libraries we need to build with it.
Same as using NuGet to install libraries needed to build with MSVC.
Also, we need CMake to support Linux as well.
We use autoconf for Linux (and other POSIX/Unix like Android and Apple systems).
I do not think we want to maintain a separate build for Windows.
The CMake today build the swig binary with MSVC on windows only without any testing.

Another remark. The time today of the configure with autoconf is around ~1 m. Switch to CMake will not change the testing time, as most of the time is on instillation, building and above all the testing under Examples/test-suite/ Examples/.

You can think like that: GitHub provides ~10 runners. While AppVeyor provides 1 runner.

Yes I understand that; just find it weird that the AppVeyor run blocks all the 100+ GitHub Actions jobs. Maybe it has something to do with how AppVeyor integrates with GitHub somehow... interesting,

I do not know if the SWIG project is willing to replace autoconf with CMake.

Exactly, even if autotools is not replaced, there is still the question of maintenance. I like CMake since it means a lot of the configuration/probing and build orchestration logic is portable onto Windows vs. autotools, which is definitely a plus, but replicating the existing functionality of how the examples/test suite is configured is not trivial at all :(

The current windows.yml use both MSVC and mingW64 compilers, and we need to test both.

It seems like mingw-w64 actually recommends CMake as one build option. CMake itself also provides a way to check if compilation is being done for MinGW. Even MSYS2 seems to have some page for CMake. Honestly, this is quite surprising for me--I have not used MinGW since the original 32-bit one years ago and so have not been following the changes :)

I do not think we want to maintain a separate build for Windows.

Of course; that's exactly the problem that CMake historically helps address, as you can use CMake for building on several Unix-like systems and for Windows. This avoids needing a separate build solution just for Windows, which I agree is not ideal.

I would still say my opinion is that although I would be in favor of moving to CMake, I recognize that replicating the current Makefile test-suite configuration is not simple at all. And definitely agree that it's a big decision to think about.

Anyways, happy to wrap this PR up if the documentation + CI configuration changes are satisfactory. I will consider experimenting with CMake to build some of examples and test suite tests, but I am not interested in getting those changes merged to master anytime soon. However, I do think having a feature branch/draft PR may be interesting for monitoring progress.

You can think like that: GitHub provides ~10 runners. While AppVeyor provides 1 runner.

Yes I understand that; just find it weird that the AppVeyor run blocks all the 100+ GitHub Actions jobs. Maybe it has something to do with how AppVeyor integrates with GitHub somehow... interesting,

https://docs.github.com/en/actions/concepts/runners/github-hosted-runners
https://www.appveyor.com/docs/parallel-testing/
See "Test categories"

I do not know if the SWIG project is willing to replace autoconf with CMake.

Exactly, even if autotools is not replaced, there is still the question of maintenance. I like CMake since it means a lot of the configuration/probing and build orchestration logic is portable onto Windows vs. autotools, which is definitely a plus, but replicating the existing functionality of how the examples/test suite is configured is not trivial at all :(

The current windows.yml use both MSVC and mingW64 compilers, and we need to test both.

It seems like mingw-w64 actually recommends CMake as one build option. CMake itself also provides a way to check if compilation is being done for MinGW. Even MSYS2 seems to have some page for CMake. Honestly, this is quite surprising for me--I have not used MinGW since the original 32-bit one years ago and so have not been following the changes :)

I do not think we want to maintain a separate build for Windows.

CMake is used for a long time. It does make sense these projects support it.
As you talk on Windows and MSVC, it was important to make sure you understand the expectation from such a task.
In terms of OS, compilers and testing with multiple languages.

Of course; that's exactly the problem that CMake historically helps address, as you can use CMake for building on several Unix-like systems and for Windows. This avoids needing a separate build solution just for Windows, which I agree is not ideal.

I would still say my opinion is that although I would be in favor of moving to CMake, I recognize that replicating the current Makefile test-suite configuration is not simple at all. And definitely agree that it's a big decision to think about.

It is your call.
I would start with a new issue with this topic something like "Moving from autoconf to the CMake"
And there you can talk to @wsfulton and @ojwb about that.
If and how to do that.

See "Test categories"

The test categories are only in the context of the AppVeyor job; I'm mostly just not sure why the GitHub Actions jobs are all blocking for the AppVeyor job. Maybe it's just how GitHub/AppVeyor are integrated...

it was important to make sure you understand the expectation from such a task

Yes, totally understand. It is not a simple thing to do given all the language + compiler + platform combinations.

It is your call. I would start with a new issue with this topic something like "Moving from autoconf to the CMake"

Sounds good; maybe when I have something to show I will consider starting a new issue. For now, it's just an idea :)

See "Test categories"

The test categories are only in the context of the AppVeyor job; I'm mostly just not sure why the GitHub Actions jobs are all blocking for the AppVeyor job. Maybe it's just how GitHub/AppVeyor are integrated...

AppVeyor runner do not block GitHub actions.
The GitHub actions usually ends before AppVeyor jobs ends.
The total checks split between GitHub actions and AppVeyor jobs.
Why do you try to complicate?

it was important to make sure you understand the expectation from such a task

Yes, totally understand. It is not a simple thing to do given all the language + compiler + platform combinations.

I was refer to the size of the task ...

It is your call. I would start with a new issue with this topic something like "Moving from autoconf to the CMake"

Sounds good; maybe when I have something to show I will consider starting a new issue. For now, it's just an idea :)

Your call

One thing I had considered doing was seeing if the CMake config could be used to replicate the current Makefile example setup.

There are various pull requests which improve CMake and add testing, some of them quite advanced, but if I recall correctly none of them were ever quite finished off and had problems compared to the autotools test-suite and slow (but that might be different now?). I suggest you search for them in github and the two mailing lists. I think there was one or two that ran the test suite for one of the target languages. I suggest finding one of the tickets and following up or creating a new one and summarise all the previous attempts in it to give a status. Get copilot to do it???

One thing I had considered doing was seeing if the CMake config could be used to replicate the current Makefile example setup.

Indeed. An incremental improvement in parallel would be the only feasible approach to adding CMake support and maybe it would replace autools in the long term once proved. It would require someone to commit to doing it and maintaining it.

Regarding Appveyor, the goal is to stop using it and fully switch to Github Actions, but it's not the most important thing right now. Am concerned we'll lose the ability to test some older versions as GHA images seem to be focusing more and more on just making the very latest available.

Anyway regarding this merge request, could someone else familiar with CMake please give it an approval and I'll then look at merging it. I think that would be you @jschueller given you've mostly recently maintained the CMakeLists.txt file.

AppVeyor runner do not block GitHub actions. The GitHub actions usually ends before AppVeyor jobs ends.

At least initially, I see the AppVeyor job runs first while the GitHub Actions jobs are waiting. It's possible that after some time, while the AppVeyor job is running, that the GitHub Actions jobs start actually running, but I haven't confirmed this because I'm usually doing something else once I see the AppVeyor job going. This behavior actually seems new; I didn't notice before.

I could be wrong or it's possible GitHub Actions might have some scheduling policy changes or even just been busy.

I was refer to the size of the task ...

Yes, I know... there is a lot of cover and therefore a lot to maintain.

I suggest you search for them in github and the two mailing lists.

Yeah, I see at least #2646 which is rather ambitious and also has a lot of proposed CI changes. I'm not sure how well it covers the existing SWIG test suite logic, however. Consider this snippet from Examples/test-suite/CMakeLists.txt:

	add_test(NAME ${name}
			COMMAND $<TARGET_FILE:Swig_swig> -${ARGS_LANGUAGE} ${ARGS_EXTRA_ARGS} ${name}.i
	)

This doesn't do any of the C/C++ module compilation; it just generates the C++/target language wrappers. Definitely the goal is to at least incrementally match some of the rules in Examples/Makefile.in. Perhaps even an incremental step is simply setting up the necessary CMake configuration for just detecting required target language development headers, SDKs, etc.

Get copilot to do it???

:)

Indeed. An incremental improvement in parallel would be the only feasible approach to adding CMake support and maybe it would replace autools in the long term once proved. It would require someone to commit to doing it and maintaining it.

Agree; transitioning a build system isn't doable as a drive-by change.

Am concerned we'll lose the ability to test some older versions as GHA images seem to be focusing more and more on just making the very latest available.

Yes, I remember there was an ubuntu-18.04 runner at some point but this was decommissioned in 2022.

AppVeyor runner do not block GitHub actions. The GitHub actions usually ends before AppVeyor jobs ends.

At least initially, I see the AppVeyor job runs first while the GitHub Actions jobs are waiting. It's possible that after some time, while the AppVeyor job is running, that the GitHub Actions jobs start actually running, but I haven't confirmed this because I'm usually doing something else once I see the AppVeyor job going. This behavior actually seems new; I didn't notice before.

Not at all.
GitHub wait for available runners.
The GitHub runners are simply busy with other jobs from other PRs and other users.
There is no binding between GitHub runners and AppVeyor runner.
The only connection is that the jobs are queue once you push the commit.
And the check finish after both GitHub jobs and AppVeyor are done.
Again you try to complicate where it is not so.

I could be wrong or it's possible GitHub Actions might have some scheduling policy changes or even just been busy.

I was refer to the size of the task ...

Yes, I know... there is a lot of cover and therefore a lot to maintain.

Not at all. GitHub wait for available runners. The GitHub runners are simply busy with other jobs from other PRs and other users.

Ok, understood. That makes more sense, but their waiting makes it seem like they are waiting for the AppVeyor job (which usually starts instantly). I'm not trying to "complicate" anything; just my observation of what seemed to be happening.

Not at all. GitHub wait for available runners. The GitHub runners are simply busy with other jobs from other PRs and other users.

Ok, understood. That makes more sense, but their waiting makes it seem like they are waiting for the AppVeyor job (which usually starts instantly). I'm not trying to "complicate" anything; just my observation of what seemed to be happening.

You are probably Lucy.
You push at times when GitHub runners are busy.

I would start with a new issue with this topic something like "Moving from autoconf to the CMake"
And there you can talk to wsfulton and ojwb about that.

While autotools are not perfect and SWIG's autotools config is uglier than many, I would not be in favour of switching to cmake.

Every time I have to interact with cmake I end up cross, sad and frustrated, so I have no desire to use cmake more than I absolutely have to. I'm certainly not going to work on it in a voluntary capacity - if we did switch, I'd just stop doing any work on SWIG's build system.

I can't see myself switching focus from autotools to CMake either, it'll be too much work. We need a CMake maintainer really, but for now I need someone who is familiar with CMake to validate/check the pull request to approve it and then I'll be happy to merge it.

While autotools are not perfect and SWIG's autotools config is uglier than many, I would not be in favour of switching to cmake.

I mostly agree (even though I like/tolerate CMake). There's already been a lot of SWIG investment into autotools.

Although it's better to discuss the topic in a different thread, there's only a few benefits with CMake that come to mind:

• Better support for native MSVC compilation on Windows (most Windows use cases will involve MSVC, not MinGW)
• Higher-level tying together of the tests + examples via custom rules (I've had some success with the Python examples)
• Better optimal/minimal rebuild using SWIG's -MMD -MF <file> options via custom rules
• Can swap out the "real" build system backend (e.g. Ninja instead of Make; Ninja is a bit faster for parallel builds)

I'm not sure that these benefits outweigh the upfront conversion cost, however.

While autotools are not p

I'm sure you can plot a better cons and pros.
I'll leave it to another discuss thread.

But I would like to reply to one point.
Although we support gladly Windows and MSVC.
We do not prefer Windows or MSVC over any other OS or other compilers used by developers.
Claiming CMake is better with Windows and MSVC is by far not sufficient.
Only if CMake is better is most OSs and most compilers we will consider using it.

The main focus of the autoconf is building the testing suite with different target languages with different compilers on different OSes.

@phetdam

Let up put the number of tests we do today:
Currently we have 137 tests:

• 120 test on Linux (Ubuntu) using GCC and CLang (and some other non Microsoft compilers).
• 6 tests on Windows using MingW-W64 GCC
• 1 test on AppVeyor using Windows with Mingw GCC
• 7 tests on Windows using MSVC
• 2 tests on AppVeyor using Windows with MSVC
• 1 test with CMake using MSVC

So you propose to potentiality improve 9 tests out of 136 tests, we do today.
And you consider that a pro?
I am more worried about disrupting the other systems: Unixs, Android and Apple systems, which we do not test every PR, than improving working with MSVC.

Hope you get the proportions.

Erez

So you propose to potentiality improve 9 tests out of 136 tests

At the very least, I don't think the MSVC tests have the same test coverage. E.g. the Windows CMake jobs only build SWIG, but don't build and run the large test suite that is available on the other compiler/OS combinations.

CMake also supports the other listed systems in addition to native Windows using MSVC.

So you propose to potentiality improve 9 tests out of 136 tests

At the very least, I don't think the MSVC tests have the same test coverage. E.g. the Windows CMake jobs only build SWIG, but don't build and run the large test suite that is available on the other compiler/OS combinations.

The 9 test:

• 7 tests on Windows using MSVC
• 2 tests on AppVeyor using Windows with MSVC
  Are full coverage with autoconf and all tests running for several targets languages, like C#, Python and Java.

CMake also supports the other listed systems in addition to native Windows using MSVC.

I did not imply CMake is limit in usage.
I was referring to usage of MSVC on Windows as less important,
Most tests run on Linux using GCC.
A good list of pros and cons to switching to CMake should start there.
Does CMake is better on Linux?
Does CMake is easier to use with GCC?

It doesn't seem useful to continue to discuss cmake vs autotools here - it should already be clear we aren't going to switch the default. Let's focus on the patch here - tangential discussion only works against getting this merged.