2.4.0

Full details on the release can be found here.

Additions:

• Support for q/kdb+ 4.1 documentation here added as an opt-in capability, this functionality is enabled through setting PYKX_4_1_ENABLED environment variable.

>>> import os
>>> os.environ['PYKX_4_1_ENABLED'] = 'True'
>>> import pykx as kx
>>> kx.q.z.K
pykx.FloatAtom(pykx.q('4.1'))

• Added support for Python 3.12.
  • Support for PyArrow in this python version is currently in Beta.
• Added conversion of NumPy arrays of type datetime64[s], datetime64[ms], datetime64[us] to kx.TimestampVector
• Added Table.sort_values(), Table.nsmallest() and Table.nlargest() to the Pandas like API for sorting tables.
• Table.rename() now supports non-numerical index columns and improved the quality of errors thrown.
• Added the reconnection_attempts key word argument to SyncQConnection, SecureQConnection, and AsyncQConnection IPC classes. This argument allows IPC connection to be automatically re-established when it is lost and a server has reinitialized.

>>> import pykx as kx
>>> conn = kx.SyncQConnection(port = 5050, reconnection_attempts=4)
>>> conn('1+1')    # Following this call the server on port 5050 was closed for 2 seconds
pykx.LongVector(pykx.q('2'))
>>> conn('1+2')
WARNING: Connection lost attempting to reconnect.
Failed to reconnect, trying again in 0.5 seconds.
Failed to reconnect, trying again in 1.0 seconds.
Connection successfully reestablished.
pykx.LongAtom(pykx.q('3'))

• Added --reconnection_attempts option to Jupyter %%q magic making use of the above IPC logic changes.
• Addition of environment variable/configuration value PYKX_QDEBUG which allows debugging backtrace to be displayed for all calls into q instead of requiring a user to specify debugging is enabled per-call. This additionally works for remote IPC calls and utilisation of Jupyter magic commands.

>>> import os
>>> os.environ['PYKX_QDEBUG'] = 'True'
>>> import pykx as kx
>>> kx.q('{x+1}', 'e')
backtrace:
  [2]  {x+1}
         ^
  [1]  (.Q.trp)

  [0]  {[pykxquery] .Q.trp[value; pykxquery; {if[y~();:(::)];2@"backtrace:
                    ^
",.Q.sbt y;'x}]}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/anaconda3/lib/python3.8/site-packages/pykx/embedded_q.py", line 230, in __call__
    return factory(result, False)
  File "pykx/_wrappers.pyx", line 493, in pykx._wrappers._factory
  File "pykx/_wrappers.pyx", line 486, in pykx._wrappers.factory
pykx.exceptions.QError: type

Fixes and Improvements:

• Resolved segfaults on Windows when PyKX calls Python functions under q.

>>> import pykx as kx
>>> kx.q('{[f;x] f  x}', sum, kx.q('4 4#til 16'))
pykx.LongVector(pykx.q('24 28 32 36'))

• Updated kdb Insights Core libraries to 4.0.8, see here for more information.
• Updated libq 4.0 version to 2024.03.04 for all supported OS’s.
• Fix issue where use of valid C backed q code APIs could result in segmentation faults when called.

>>> import pykx as kx
>>> isf = kx.q('.pykx.util.isf')
>>> isf
pykx.Foreign(pykx.q('code'))
>>> isf(True)
pykx.BooleanAtom(pykx.q('0b'))

• Each call to the PyKX query API interned 3 new unique symbols. This has now been removed.

Beta Features

• Addition of Compress and Encrypt classes to allow users to set global configuration and for usage within Database partition persistence.

Standalone

>>> import pykx as kx
>>> compress = kx.Compress(algo=kx.CompressionAlgorithm.gzip, level=8)
>>> kx.q.z.zd
pykx.Identity(pykx.q('::'))
>>> compress.global_init()
pykx.LongVector(pykx.q('17 2 8'))
>>> encrypt = kx.Encrypt(path='/path/to/the.key', password='PassWord')
>>> encrypt.load_key()

Database

>>> import pykx as kx
>>> compress = kx.Compress(algo=kx.CompressionAlgorithm.lz4hc, level=10)
>>> db = kx.DB(path='/tmp/db')
>>> db.create(kx.q('([]10?1f;10?1f)', 'tab', kx.q('2020.03m'), compress=compress)
>>> kx.q('-21!`:/tmp/db/2020.03/tab/x')
pykx.Dictionary(pykx.q('
compressedLength  | 140
uncompressedLength| 96
algorithm         | 4i
logicalBlockSize  | 17i
zipLevel          | 10i
'))