-
Notifications
You must be signed in to change notification settings - Fork 171
Use optimisation flags for C compiler in lpython decorator #2201
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
9f5cde4
Use -funroll-loops -ffast-math optimisations
czgdp1807 0b54b65
Make backend compulsory and allow passing custom optimisation flags a…
czgdp1807 4da626c
Allow passing optional custom optimisation flags
czgdp1807 5cff2f6
Make backend compulsory if optimisations are provided
czgdp1807 ebcfc22
Fix doc
czgdp1807 dce735d
Run failed cases verbosely
czgdp1807 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -3,6 +3,7 @@ | |
import ctypes | ||
import platform | ||
from dataclasses import dataclass as py_dataclass, is_dataclass as py_is_dataclass | ||
import functools | ||
|
||
|
||
# TODO: this does not seem to restrict other imports | ||
|
@@ -647,37 +648,49 @@ def ccallable(f): | |
def ccallback(f): | ||
return f | ||
|
||
class lpython: | ||
""" | ||
The @lpython decorator compiles a given function using LPython. | ||
class LpythonJITCache: | ||
|
||
The decorator should be used from CPython mode, i.e., when the module is | ||
being run using CPython. When possible, it is recommended to use LPython | ||
for the main program, and use the @cpython decorator from the LPython mode | ||
to access CPython features that are not supported by LPython. | ||
""" | ||
def __init__(self): | ||
self.pyfunc2compiledfunc = {} | ||
|
||
def compile(self, function, backend, optimisation_flags): | ||
if function in self.pyfunc2compiledfunc: | ||
return self.pyfunc2compiledfunc[function] | ||
|
||
if optimisation_flags is not None and backend is None: | ||
raise ValueError("backend must be specified if backend_optimisation_flags are provided.") | ||
|
||
if backend is None: | ||
backend = "c" | ||
|
||
def __init__(self, function): | ||
def get_rtlib_dir(): | ||
current_dir = os.path.dirname(os.path.abspath(__file__)) | ||
return os.path.join(current_dir, "..") | ||
|
||
self.fn_name = function.__name__ | ||
fn_name = function.__name__ | ||
# Get the source code of the function | ||
source_code = getsource(function) | ||
source_code = source_code[source_code.find('\n'):] | ||
|
||
dir_name = "./lpython_decorator_" + self.fn_name | ||
dir_name = "./lpython_decorator_" + fn_name | ||
if not os.path.exists(dir_name): | ||
os.mkdir(dir_name) | ||
filename = dir_name + "/" + self.fn_name | ||
filename = dir_name + "/" + fn_name | ||
|
||
# Open the file for writing | ||
with open(filename + ".py", "w") as file: | ||
# Write the Python source code to the file | ||
file.write("@pythoncallable") | ||
file.write(source_code) | ||
|
||
if backend != "c": | ||
raise NotImplementedError("Backend %s is not supported with @lpython yet."%(backend)) | ||
|
||
opt_flags = " " | ||
if optimisation_flags is not None: | ||
for opt_flag in optimisation_flags: | ||
opt_flags += opt_flag + " " | ||
|
||
# ---------------------------------------------------------------------- | ||
# Generate the shared library | ||
# TODO: Use LLVM instead of C backend | ||
|
@@ -687,12 +700,14 @@ def get_rtlib_dir(): | |
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure if the --fast` option works in the C backend. If it does, then we can use it in line 684? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It doesn't I think. |
||
gcc_flags = "" | ||
if platform.system() == "Linux": | ||
gcc_flags = " -shared -fPIC " | ||
gcc_flags = " -shared -fPIC" | ||
elif platform.system() == "Darwin": | ||
gcc_flags = " -bundle -flat_namespace -undefined suppress " | ||
gcc_flags = " -bundle -flat_namespace -undefined suppress" | ||
else: | ||
raise NotImplementedError("Platform not implemented") | ||
|
||
gcc_flags += opt_flags | ||
|
||
from numpy import get_include | ||
from distutils.sysconfig import get_python_inc, get_python_lib, \ | ||
get_python_version | ||
|
@@ -706,17 +721,38 @@ def get_rtlib_dir(): | |
|
||
# ---------------------------------------------------------------------- | ||
# Compile the C file and create a shared library | ||
shared_library_name = "lpython_module_" + fn_name | ||
r = os.system("gcc -g" + gcc_flags + python_path + numpy_path + | ||
filename + ".c -o lpython_module_" + self.fn_name + ".so " + | ||
filename + ".c -o " + shared_library_name + ".so " + | ||
rt_path_01 + rt_path_02 + python_lib) | ||
assert r == 0, "Failed to create the shared library" | ||
self.pyfunc2compiledfunc[function] = (shared_library_name, fn_name) | ||
return self.pyfunc2compiledfunc[function] | ||
|
||
def __call__(self, *args, **kwargs): | ||
import sys; sys.path.append('.') | ||
# import the symbol from the shared library | ||
function = getattr(__import__("lpython_module_" + self.fn_name), | ||
self.fn_name) | ||
return function(*args, **kwargs) | ||
lpython_jit_cache = LpythonJITCache() | ||
|
||
# Taken from https://stackoverflow.com/a/24617244 | ||
def lpython(original_function=None, backend=None, backend_optimisation_flags=None): | ||
""" | ||
The @lpython decorator compiles a given function using LPython. | ||
|
||
The decorator should be used from CPython mode, i.e., when the module is | ||
being run using CPython. When possible, it is recommended to use LPython | ||
for the main program, and use the @cpython decorator from the LPython mode | ||
to access CPython features that are not supported by LPython. | ||
""" | ||
def _lpython(function): | ||
@functools.wraps(function) | ||
def __lpython(*args, **kwargs): | ||
import sys; sys.path.append('.') | ||
lib_name, fn_name = lpython_jit_cache.compile( | ||
function, backend, backend_optimisation_flags) | ||
return getattr(__import__(lib_name), fn_name)(*args, **kwargs) | ||
return __lpython | ||
|
||
if original_function: | ||
return _lpython(original_function) | ||
return _lpython | ||
|
||
def bitnot(x, bitsize): | ||
return (~x) % (2 ** bitsize) | ||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This documentation should be somewhere.