`--jit` option to execute without creation of executable file #2564

Vipul-Cariappa · 2024-02-29T07:03:02Z

Using LLVM JIT.

Example:
With --jit flag

❯ time ./src/bin/lpython --jit ./examples/expr2.py
25
./src/bin/lpython --jit ./examples/expr2.py  0.01s user 0.00s system 97% cpu 0.013 total

Without

❯ time ./src/bin/lpython ./examples/expr2.py      
25
./src/bin/lpython ./examples/expr2.py  0.02s user 0.01s system 99% cpu 0.037 total

Vipul-Cariappa · 2024-02-29T09:56:55Z

Both of the failing workflows are WASM related. One of them says error: use of undeclared identifier 'execute_python_using_jit'. Even though it is declared and defined in the preceding lines.

Thirumalai-Shaktivel

Also, I think we should test this, try resolving my review above, later we will figure out how to test this.

Thirumalai-Shaktivel · 2024-03-07T06:01:01Z

src/bin/lpython.cpp

+    } catch (LCompilers::LCompilersException&) {}
+    auto llvm_end = std::chrono::high_resolution_clock::now();
+    times.push_back(std::make_pair("LLVM to binary", std::chrono::duration<double, std::milli>(llvm_end - llvm_start).count()));
+    print_time_report(times, time_report);


Let's do this only for LLVM backend.
This seems duplicated, try utilising or simplifying the function: compile_python_to_object_file

Thirumalai-Shaktivel · 2024-03-07T06:02:29Z

Please mark this PR ready for review once it is ready.

ubaidsk · 2024-03-07T13:17:20Z

src/bin/lpython.cpp

+        if (to_jit) {
+            compiler_options.emit_debug_info = false;
+            compiler_options.emit_debug_line_column = false;
+            compiler_options.generate_object_code = false;
+            return execute_python_using_jit(arg_file, runtime_library_dir,
+                    lpython_pass_manager, compiler_options, time_report);
+        }


One of them says error: use of undeclared identifier 'execute_python_using_jit'. Even though it is declared and defined in the preceding lines.

This should be wrapped inside #ifdef HAVE_LFORTRAN_LLVM #endif. Hopefully, it should fix the CI failure.

Vipul-Cariappa · 2024-03-09T15:12:15Z

@Thirumalai-Shaktivel, Regarding testing out this feature. I was thinking to add the following

run_test(
                f"{filename} JIT",
                "llvm",
                "lpython --jit {infile}",
                filename,
                update_reference)

after llvm test at run_tests.py.

But I get

raise FileNotFoundError(
FileNotFoundError: The reference json file 'tests/reference/llvm-expr14-f21489f.json' for expr14.py JIT does not exist

when I run it.

I do not properly understand the methodology used to test existing features. Suggestion and guidance would be helpful here.

Thirumalai-Shaktivel · 2024-03-09T15:27:32Z

I think we need to test this using integration_tests/run_tests.py. It is used for testing most of our runtime tests.
Checkout integration_tests/CMakeLists.txt. If you are blocked, do let me know.

Thirumalai-Shaktivel

The implementation looks good so far.

ubaidsk

This looks good! Thanks for this! Now, you just need to add an integration test for this. You can leverage the existing EXTRA_ARGS support. For example:

diff --git a/integration_tests/CMakeLists.txt b/integration_tests/CMakeLists.txt
index 44b703eae..1db5d10b8 100644
--- a/integration_tests/CMakeLists.txt
+++ b/integration_tests/CMakeLists.txt
@@ -313,11 +313,10 @@ endmacro(RUN_UTIL)
 macro(RUN)
     set(options FAIL NOFAST NOMOD ENABLE_CPYTHON LINK_NUMPY NO_WARNINGS)
     set(oneValueArgs NAME IMPORT_PATH COPY_TO_BIN REQ_PY_VER)
-    set(multiValueArgs LABELS EXTRAFILES)
+    set(multiValueArgs LABELS EXTRAFILES EXTRA_ARGS)
     cmake_parse_arguments(RUN "${options}" "${oneValueArgs}"
                         "${multiValueArgs}" ${ARGN} )
 
-    set(RUN_EXTRA_ARGS "")
     set(RUN_FILE_NAME ${RUN_NAME})
 
     if (RUN_LINK_NUMPY)
@@ -826,3 +825,6 @@ COMPILE(NAME import_order_01     LABELS cpython llvm c) # any
 
 # LPython emulation mode
 RUN(NAME lpython_emulation_01    LABELS cpython NOMOD)
+
+RUN(NAME jit_01 LABELS cpython llvm
+    EXTRA_ARGS --jit)

ubaidsk · 2024-03-10T11:26:19Z

We need an integration test for this. Please mark this as "Ready for review" when ready. Thanks for working on this! I appreciate it.

integration_tests/CMakeLists.txt

Vipul-Cariappa · 2024-03-10T14:06:59Z

We expect

exit_02_jit
exit_04_jit
exit_02b_jit
exit_02c_jit

expr_18_jit
if_03_jit

to fail (i.e. exit with non-zero code).

What should I do here? When the exit code is supposed to be non-zero?

const_03_jit
expr_13_jit
test_c_interop_02_jit
test_c_interop_03_jit
test_c_interop_05_jit
bindc_03_jit
bindc_05_jit
bindc_06_jit
structs_07_jit
structs_13_jit
structs_18_jit
structs_19_jit
structs_20_jit
sizeof_01_jit
enum_05_jit

fails because it has an external dependency.

I believe we should skip these tests under JIT.

test_argv_01_jit and print_list_tuple_03_jit look like proper errors. Debugging required.

EDITED:

I would also want to add that test_dict_nested1_jit fails in the GitHub Workflow and passes in my local computer. I suspect this is due to some memory-related bug. Return of element allocated on stack or use after free (this is my guess, I am not sure). print_list_tuple_03_jit may also be a bug in similar lines.

Vipul-Cariappa · 2024-03-10T14:23:00Z

integration_tests/CMakeLists.txt

+    #     unset(RUN_EXTRA_ARGS)
+    #     set(RUN_EXTRA_ARGS --jit)
+    #     set(RUN_NAME "${RUN_NAME}_JIT")
+    #     RUN_UTIL(RUN_FAIL RUN_NAME RUN_FILE_NAME RUN_LABELS RUN_EXTRAFILES RUN_NOMOD RUN_EXTRA_ARGS RUN_COPY_TO_BIN)


I didn't intend to push this. I was testing stuff out in my local computer, I should have removed this before pushing. Sorry. Will remove it in the next commit.

Vipul-Cariappa · 2024-03-10T15:07:06Z

@Thirumalai-Shaktivel, @Shaikh-Ubaid Look at #2595. If you could verify that they are definitely memory-related bugs it would be helpful.
I guess progress on this PR would be halted till we figure out and resolve #2595.!?

I did not look at the AST and ASR. I will investigate them tomorrow.

ubaidsk · 2024-03-24T13:06:37Z

Please mark it as "Ready for review" when ready.

Vipul-Cariappa · 2024-03-24T13:32:28Z

I believe this PR is ready for review/merge.

CC: @Shaikh-Ubaid

I will explain some aspects of the modifications I have made to run the tests:

Add necessary options in the cmake(-JIT) tests to skip tests that depend on external C/C++ code. (JIT compiling code with external dependency does not make much sense. We cannot load in a static library at runtime.)
Also testing integration with symengine. I am dynamically loading in symengine.so using dlopen if --enable-symengine is passed.
Similarly, I am dynamically loading in python3.so if --enable-cpython is passed.

Note: LLVM backend does not support BindPython ABI yet.

There are only 2 tests that fail. They fail due to segfault. And related to #2595. They are not caused by any new bug I have introduced. My code happens to trigger it.

I tried investigating the segfault, but have not found its cause yet. It is very difficult to debug the LLVM IR directly. Presently I am trying to do that. Also fixing them should not be part of this PR.

Also, the Windows CI is not running to completion. Please read this comment I made in the zulip chat.

ubaidsk · 2024-03-25T21:25:40Z

@Vipul-Cariappa From a high level it looks good. I will review it thoroughly tomorrow.

There are only 2 tests that fail. They fail due to segfault. And related to #2595. They are not caused by any new bug I have introduced. My code happens to trigger it.

We will need the CI to be green to merge this. So, we either need to get the two failing tests to work or we need to completely skip running these two tests with --jit(I think right now we just skip checking the return code).

certik · 2024-04-02T15:58:56Z

Thanks!

The advantages of this approach:

faster
no intermediate files
gets rid of the Clang dependency for linking

Disadvantages:

Now we need to handle the loading of shared libraries (cpython and symengine, if used), and we do not allow static libraries

The handling of shared libraries is something we will need for interactive mode as well, so I think we need this. The current implementation works for Conda, but it will fail for, say, Ubuntu, which uses a lib64 (I think) prefix. We will need to add options to select where these libraries are installed. This code should be refactored into some utility file, so that we don't have these platform dependent ifdefs in the main driver.

To finish this, let's get all tests passing, and then let's polish the implementation, so that we can merge it. Then we can improve upon it.

Vipul-Cariappa · 2024-04-03T14:27:58Z

To finish this, let's get all tests passing

we need to completely skip running these two tests with --jit(I think right now we just skip checking the return code).

I am trying to debug and fix #2595. It would be better to fix the issues and then merge this PR. Skipping the failing tests will require quite a lot of modifications to the integration_tests/CMakeLists.txt. Which should later be undone, when the issue is fixed.
Please have a look at the recent comments I have made at #2595 (Help Required).

Vipul-Cariappa · 2024-04-04T06:13:15Z

e7760b0 fixes #2595
I tested with both AddressSanitizer and Valgrind.

Vipul-Cariappa · 2024-04-04T07:51:33Z

Ready for review. @Shaikh-Ubaid @certik

ubaidsk · 2024-04-18T20:20:08Z

@Vipul-Cariappa Can you rebase the branch on top of latest main and resolve any conflicts?

remove the try catch block around the execution of global_init & global_stmts

…t` to `compile_python_using_llvm` removed duplicated code

Co-authored-by: Shaikh Ubaid <[email protected]>

…lers#2564) * Add --jit option to execute without creation of executable file * execute global_init & global_stmts after checking their existence remove the try catch block around the execution of global_init & global_stmts * CI fix & raise error for non-llvm backend with jit * combined `compile_python_to_object_file` and `execute_python_using_jit` to `compile_python_using_llvm` removed duplicated code * Add testing mechanism * Testing with cpython and symengine * skipping jit tests with external dependency * skipping testing JIT that depend on syms or cpython * support to use cpython and symengine with JIT * Trigger CI/CD * windows fix * WASM ci fix * allow few tests to fail under `--jit` * fix for lcompilers#2595 * factored out loading cpython and symengine libraries * wasm related fix * windows ci fix * windows ci fix * windows ci fix * updated for consistency Co-authored-by: Shaikh Ubaid <[email protected]> * updated for consistency * Apply suggestions from code review Co-authored-by: Shaikh Ubaid <[email protected]> --------- Co-authored-by: Shaikh Ubaid <[email protected]>

Thirumalai-Shaktivel reviewed Mar 7, 2024

View reviewed changes

Thirumalai-Shaktivel marked this pull request as draft March 7, 2024 06:02

ubaidsk reviewed Mar 7, 2024

View reviewed changes

Vipul-Cariappa marked this pull request as ready for review March 9, 2024 15:06

Vipul-Cariappa requested review from ubaidsk and Thirumalai-Shaktivel March 9, 2024 15:06

Thirumalai-Shaktivel reviewed Mar 9, 2024

View reviewed changes

ubaidsk reviewed Mar 9, 2024

View reviewed changes

ubaidsk marked this pull request as draft March 10, 2024 11:26

ubaidsk reviewed Mar 10, 2024

View reviewed changes

integration_tests/CMakeLists.txt Show resolved Hide resolved

Vipul-Cariappa commented Mar 10, 2024

View reviewed changes

Vipul-Cariappa marked this pull request as ready for review March 24, 2024 05:30

Vipul-Cariappa marked this pull request as draft March 24, 2024 06:10

Vipul-Cariappa marked this pull request as ready for review March 24, 2024 13:32

Vipul-Cariappa mentioned this pull request Mar 25, 2024

Prevent calling abort while handling the SIGABRT signal #2626

Merged

Vipul-Cariappa requested a review from ubaidsk April 4, 2024 07:51

Vipul-Cariappa and others added 22 commits April 20, 2024 11:02

Add --jit option to execute without creation of executable file

2da048b

execute global_init & global_stmts after checking their existence

5a96c18

remove the try catch block around the execution of global_init & global_stmts

CI fix & raise error for non-llvm backend with jit

8a0a847

combined compile_python_to_object_file and `execute_python_using_ji…

895ae8f

…t` to `compile_python_using_llvm` removed duplicated code

Add testing mechanism

8659948

Testing with cpython and symengine

602359a

skipping jit tests with external dependency

4ae8463

skipping testing JIT that depend on syms or cpython

c138728

support to use cpython and symengine with JIT

31b4d02

Trigger CI/CD

f1cd329

windows fix

caec554

WASM ci fix

ed6dd6e

allow few tests to fail under --jit

3be9723

fix for lcompilers#2595

ca42626

factored out loading cpython and symengine libraries

4fdfe04

wasm related fix

cbac670

windows ci fix

a35dc02

windows ci fix

f6f62b9

windows ci fix

ff5ac20

updated for consistency

ea044d5

Co-authored-by: Shaikh Ubaid <[email protected]>

updated for consistency

5ae201e

Apply suggestions from code review

6d9432b

Co-authored-by: Shaikh Ubaid <[email protected]>

Vipul-Cariappa force-pushed the jit branch from 693d8e7 to 6d9432b Compare April 20, 2024 05:36

ubaidsk approved these changes Apr 20, 2024

View reviewed changes

ubaidsk merged commit 69b826e into lcompilers:main Apr 20, 2024

Vipul-Cariappa mentioned this pull request Apr 20, 2024

AddressSanitizer Errors for test_dict_nested1.py and print_list_tuple_03.py: requested allocation size exceeds maximum supported size #2595

Closed

Vipul-Cariappa deleted the jit branch May 2, 2024 14:08

--jit option to execute without creation of executable file #2564

--jit option to execute without creation of executable file #2564

Uh oh!

Conversation

Vipul-Cariappa commented Feb 29, 2024

Uh oh!

Vipul-Cariappa commented Feb 29, 2024

Uh oh!

Thirumalai-Shaktivel left a comment

Choose a reason for hiding this comment

Uh oh!

Thirumalai-Shaktivel Mar 7, 2024

Choose a reason for hiding this comment

Uh oh!

Thirumalai-Shaktivel commented Mar 7, 2024

Uh oh!

ubaidsk Mar 7, 2024

Choose a reason for hiding this comment

Uh oh!

Vipul-Cariappa commented Mar 9, 2024

Uh oh!

Thirumalai-Shaktivel commented Mar 9, 2024

Uh oh!

Thirumalai-Shaktivel left a comment

Choose a reason for hiding this comment

Uh oh!

ubaidsk left a comment

Choose a reason for hiding this comment

Uh oh!

ubaidsk commented Mar 10, 2024

Uh oh!

Uh oh!

Vipul-Cariappa commented Mar 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Vipul-Cariappa Mar 10, 2024

Choose a reason for hiding this comment

Uh oh!

Vipul-Cariappa commented Mar 10, 2024

Uh oh!

ubaidsk commented Mar 24, 2024

Uh oh!

Vipul-Cariappa commented Mar 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ubaidsk commented Mar 25, 2024

Uh oh!

certik commented Apr 2, 2024

Uh oh!

Vipul-Cariappa commented Apr 3, 2024

Uh oh!

Vipul-Cariappa commented Apr 4, 2024

Uh oh!

Vipul-Cariappa commented Apr 4, 2024

Uh oh!

ubaidsk commented Apr 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

`--jit` option to execute without creation of executable file #2564

`--jit` option to execute without creation of executable file #2564

Vipul-Cariappa commented Mar 10, 2024 •

edited

Loading

Vipul-Cariappa commented Mar 24, 2024 •

edited

Loading

ubaidsk commented Apr 18, 2024 •

edited

Loading