[python/c++] Connect libtiledbsoma to tiledbsoma readers for dataframes [WIP]

[johnkerl]

Status

• Currently WIP
• SOMADataFrame and SOMAIndexedDataFrame are on this PR
• For reviewer mercy, SOMASparseNdArray and SOMADenseNdArray will be on a separate PR

PR context

This is the three in a group of three related PRs:

• [python] Conform to spec by reading as pyarrow.Table not pyarrow.RecordBatch #355: Conform to https://github.com/single-cell-data/SOMA/blob/main/abstract_specification.md by having read returnpyarrow.Table not pyarrow.RecordBatch (as in an outdated version of that spec) -- now merged
• [python] Use true ASCII attributes in dataframes #359: Foreward-port [python] Update ASCII storage for dataframes #273 from main-old which will truly have ASCII columns, obviating the need for our util_arrow.ascii_to_unicode_pyarrow_readback -- now merged
• This PR: drop in C++ acceleration-library support for read methods, which will go in cleanly now
  • The C++ code returns pyarrow.Table and with the first PR our unit tests will be ready to go
  • When the C++ code reads Unicode cells it returns them as pyarrow.LargeBinaryArray (needing decode) but when we are properly writing ASCII cells via the Python write path then the C++ code will read ASCII cells and return them as strings (no longer needing decoding)

[johnkerl]

The fail is here:

E           RuntimeError: TypeError: type of argument "node" must be a type; got ast.Compare instead

https://gist.github.com/johnkerl/f93ad47c4bfa29fbb929f4e2d3cd83f0

Now we are intentionally suppressing type-checking for apis/python/src/tiledbsoma/query_condition.py here:
https://github.com/single-cell-data/TileDB-SOMA/blob/main/apis/python/src/tiledbsoma/query_condition.py

However, typeguard seems to blithely bulldoze right past that, as evidenced by its surfacing type-check issues involving query_condition.py.

On the third hand, here
https://github.com/single-cell-data/TileDB-SOMA/blob/main/apis/python/tests/__init__.py#L4
I've tried this diff:

diff --git a/apis/python/tests/__init__.py b/apis/python/tests/__init__.py
index 5df0213..8817d3e 100644
--- a/apis/python/tests/__init__.py
+++ b/apis/python/tests/__init__.py
@@ -1,7 +1,16 @@
 import pyarrow as pa
-from typeguard.importhook import install_import_hook
+from typeguard.importhook import TypeguardFinder, install_import_hook

-install_import_hook("tiledbsoma")
+class CustomFinder(TypeguardFinder):
+    def should_instrument(self, module_name: str):
+        #print("\nSI >>", module_name, "<<\n")
+        if module_name == 'tiledbsoma.query_condition':
+            return False
+        if module_name == 'tiledb.query_condition':
+            return False
+        return True
+
+install_import_hook("tiledbsoma", cls=CustomFinder)

and verified that the query_condition modules are being ignored -- and still typeguard persists in its

E           RuntimeError: TypeError: type of argument "node" must be a type; got ast.Compare instead

😠

[johnkerl]

typeguard issue resolved on this commit:
cfb14ae

[johnkerl]

... behind that now-opened door lie more still-locked doors; upon them is written more typeguard

[johnkerl]

Closing in favor of #400