Convert back to RST, add section on runtime loading

pythonnet · filmor · Sep 26, 2022 · May 26, 2022 · Sep 19, 2022 · Sep 26, 2022
commit 13f2a9c16e0eceaf451414f0b6deff06c0733c77
diff --git a/doc/requirements.txt b/doc/requirements.txt
@@ -4,9 +4,6 @@ sphinx
 furo>=2022.9.15
 pygments>=2.7
 
-# Markdown
-myst-parser
-
 # C# via doxygen
 breathe
 git+https://github.com/rogerbarton/sphinx-csharp.git
diff --git a/doc/source/codecs.rst b/doc/source/codecs.rst
@@ -0,0 +1,69 @@
+.. _codecs:
+
+Codecs
+======
+
+Python.NET performs some conversions between .NET and Python automatically.
+For example, when Python calls this C# method:
+
+.. code:: csharp
+
+   void Foo(int bar) { ... }
+
+via ``Foo(42)``, Python value ``42`` of type ``int`` will be
+automatically converted to .NET type ``System.Int32``. Another way to
+invoke those conversions is to call ``dotNetObject.ToPython()``
+(available as an extension method) or ``pyObject.As<T>()`` to convert
+``PyObject`` to .NET.
+
+An incomplete list of Python types, that are converted between Python
+and .NET automatically: most numeric types, ``bool``, ``string``,
+``Nullable<T>`` to its ``Value`` or ``None`` and back, etc.
+
+A custom conversion (**Codec**) can be defined by implementing one of the (or
+both) interfaces:
+
+- ``Python.Runtime.IPyObjectDecoder`` to marshal Python objects to .NET
+
+.. code:: csharp
+
+   interface IPyObjectDecoder {
+     bool CanDecode(PyObject objectType, System.Type targetType);
+     bool TryDecode<T>(PyObject pyObj, out T value);
+   }
+
+-  ``Python.Runtime.IPyObjectEncoder`` to marshal .NET objects to Python
+
+.. code:: csharp
+
+   interface IPyObjectEncoder {
+     bool CanEncode(System.Type);
+     PyObject TryEncode(System.Object);
+   }
+
+Once implemented, instances have to be registered with
+``Python.Runtime.PyObjectConversions.RegisterEncoder``/``-Decoder``. One
+can override *some* of the default conversions by registering new
+codecs.
+
+Codec priorities
+~~~~~~~~~~~~~~~~
+
+When multiple codecs are registered, the runtime will first try the ones, that
+were registered earlier. If you need to have some grouping of codecs by
+priority, create and expose
+``Python.Runtime.Codecs.EncoderGroup``/``-.DecoderGroup``. For example:
+
+.. code:: csharp
+
+   public static EncoderGroup HighPriorityEncoders{ get; } = new EncoderGroup();
+
+   void Init() {
+     PyObjectConversions.RegisterEncoder(HighPriorityEncoders);
+     var lowPriorityEncoder = new SomeEncoder();
+     PyObjectConversions.RegisterEncoder(lowPriorityEncoder);
+   }
+
+   ... some time later
+
+   HighPriorityEncoders.Add(new SomeOtherEncoder());
diff --git a/doc/source/conf.py b/doc/source/conf.py
@@ -10,16 +10,15 @@
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #
-# import os
-# import sys
-# sys.path.insert(0, os.path.abspath('.'))
+import os, sys
+sys.path.insert(0, os.path.abspath('../..'))
 
 
 # -- Project information -----------------------------------------------------
 
-project = 'Python.NET'
-copyright = '2022, The Python.NET Project Contributors'
-author = 'The Python.NET Project Contributors'
+project = "Python.NET"
+copyright = "2022, The Python.NET Project Contributors"
+author = "The Python.NET Project Contributors"
 
 
 # -- General configuration ---------------------------------------------------
@@ -28,14 +27,14 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
-    'myst_parser',
-    'breathe',
-    'sphinx.ext.autodoc',
-    'sphinx_csharp',
+    "breathe",
+    "sphinx.ext.autodoc",
+    "sphinx_csharp",
+    "sphinx.ext.intersphinx",
 ]
 
 # Add any paths that contain templates here, relative to this directory.
-templates_path = ['_templates']
+templates_path = ["_templates"]
 
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
@@ -48,12 +47,14 @@
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
-html_theme = 'furo'
+html_theme = "furo"
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
-html_static_path = ['_static']
+html_static_path = ["_static"]
 
-breathe_projects = { "pythonnet": "../doxygen_xml" }
-breathe_default_project = 'pythonnet'
+breathe_projects = {"pythonnet": "../doxygen_xml"}
+breathe_default_project = "pythonnet"
+
+intersphinx_mapping = {"clr-loader": ("https://pythonnet.github.io/clr-loader/", None)}
diff --git a/doc/source/dotnet.md b/doc/source/dotnet.md
diff --git a/doc/source/dotnet.rst b/doc/source/dotnet.rst
@@ -0,0 +1,114 @@
+Embedding Python into .NET
+==========================
+
+.. warning::
+    Because Python code running under Python.NET is inherently
+    unverifiable, it runs totally under the radar of the security
+    infrastructure of the CLR so you should restrict use of the Python
+    assembly to trusted code.
+
+The Python runtime assembly defines a number of public classes that
+provide a subset of the functionality provided by the Python C-API.
+
+These classes include PyObject, PyList, PyDict, PyTuple, etc.
+
+At a very high level, to embed Python in your application one will need
+to:
+
+-  Reference ``Python.Runtime.dll`` (e.g. via a ``PackageReference``)
+-  Call ``PythonEngine.Initialize()`` to initialize Python
+-  Call ``PythonEngine.ImportModule(name)`` to import a module
+
+The module you import can either start working with your managed app
+environment at the time its imported, or you can explicitly lookup and
+call objects in a module you import.
+
+For general-purpose information on embedding Python in applications, use
+www.python.org or Google to find (C) examples. Because Python.NET is so
+closely integrated with the managed environment, one will generally be
+better off importing a module and deferring to Python code as early as
+possible rather than writing a lot of managed embedding code.
+
+.. note::
+    Python is not free-threaded and uses a
+    global interpreter lock to allow multi-threaded applications to interact
+    safely with the Python interpreter. Much more information about this is
+    available in the Python C-API documentation on the www.python.org
+    Website.
+
+When embedding Python in a managed application, one has to manage the
+GIL in just the same way you would when embedding Python in a C or C++
+application.
+
+Before interacting with any of the objects or APIs provided by the
+``Python.Runtime`` namespace, calling code must have acquired the Python
+global interpreter lock by calling the ``PythonEngine.AcquireLock``
+method. The only exception to this rule is the
+``PythonEngine.Initialize`` method, which may be called at startup
+without having acquired the GIL.
+
+When finished using Python APIs, managed code must call a corresponding
+``PythonEngine.ReleaseLock`` to release the GIL and allow other threads
+to use Python.
+
+A ``using`` statement may be used to acquire and release the GIL:
+
+.. code:: csharp
+
+   using (Py.GIL())
+   {
+       PythonEngine.Exec("doStuff()");
+   }
+
+The AcquireLock and ReleaseLock methods are thin wrappers over the
+unmanaged ``PyGILState_Ensure`` and ``PyGILState_Release`` functions
+from the Python API, and the documentation for those APIs applies to the
+managed versions.
+
+Passing C# Objects to the Python Engine
+---------------------------------------
+
+This section demonstrates how to pass a C# object to the Python runtime.
+The example uses the following ``Person`` class:
+
+.. code:: csharp
+
+   public class Person
+   {
+       public Person(string firstName, string lastName)
+       {
+           FirstName = firstName;
+           LastName = lastName;
+       }
+
+       public string FirstName { get; set; }
+       public string LastName { get; set; }
+   }
+
+In order to pass a C# object to the Python runtime, it must be converted
+to a ``PyObject``. This is done using the ``ToPython()`` extension
+method. The ``PyObject`` may then be set as a variable in a ``PyScope``.
+Code executed from the scope will have access to the variable:
+
+.. code:: csharp
+
+   // create a person object
+   Person person = new Person("John", "Smith");
+
+   // acquire the GIL before using the Python interpreter
+   using (Py.GIL())
+   {
+       // create a Python scope
+       using (PyScope scope = Py.CreateScope())
+       {
+           // convert the Person object to a PyObject
+           PyObject pyPerson = person.ToPython();
+
+           // create a Python variable "person"
+           scope.Set("person", pyPerson);
+
+           // the person object may now be used in Python
+           string code = "fullName = person.FirstName + ' ' + person.LastName";
+           scope.Exec(code);
+       }
+   }