Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

misc design improvements #15

Merged
merged 10 commits into from
Nov 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 14 additions & 44 deletions .github/workflows/publish-to-pypi.yml
Original file line number Diff line number Diff line change
@@ -1,56 +1,26 @@
name: Publish Python 🐍 distributions 📦 to PyPI and TestPyPI
name: Publish Python 🐍 distributions 📦 to PyPI

on:
workflow_dispatch: {}
push:
branches:
- main
pull_request:
branches:
- main
types:
- closed
release:
types: [published]
workflow_dispatch:

jobs:
build-n-publish:
name: Build and publish Python 🐍 distributions 📦 to PyPI and TestPyPI
build:
name: build and upload release to PyPI
runs-on: ubuntu-latest
environment:
name: release
environment: "release"
permissions:
id-token: write # IMPORTANT: this permission is mandatory for trusted publishing

steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.9"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pytest
pip install -e .

- name: Test with pytest
run: |
pytest

- name: Install pypa/build
run: >-
pip install build --user

- name: Build a source tarball and a binary wheel
run: >-
python3 -m build --sdist --wheel --outdir dist/ .
- name: Install uv
uses: astral-sh/setup-uv@v2

- name: Publish distribution 📦 to Test PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
repository-url: https://test.pypi.org/legacy/
skip-existing: true
- name: Build Package
run: uv build

- name: Publish distribution 📦 to PyPI
if: startsWith(github.ref, 'refs/tags/v')
uses: pypa/gh-action-pypi-publish@release/v1
- name: Publish package distributions to PyPI
run: uv publish
43 changes: 43 additions & 0 deletions .github/workflows/python-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Test package

on:
push:
branches: [main]
pull_request:
branches:
- main
- dev
workflow_dispatch:

jobs:
build:
strategy:
matrix:
python-version: [3.9, "3.10", "3.11", "3.12"]
os:
- "ubuntu-latest"
- "windows-latest"
- "macos-latest"
runs-on: ${{matrix.os}}

steps:
- uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v2
with:
enable-cache: true
cache-dependency-glob: "uv.lock"

- name: Set up Python ${{ matrix.python-version }}
run: uv python install ${{ matrix.python-version }}

- name: Install the project
run: uv sync --all-extras --dev

# - name: Lint
# run:
# uv tool run ruff check --output-format=github src

- name: Run tests
run: uv run pytest tests
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -159,3 +159,5 @@ cython_debug/
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
.idea/
# "I am become God." -Chad

src/pycbsdk/__version__.py
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ config = cbsdk.get_config(nsp_obj)
print(config)
```

You may also try the provided test script with `python -m pycbsdk.examples.print_rates` or via the shortcut: `pycbsdk_print_rates`.
You may also try the provided test script with `python -m pycbsdk.examples.print_rates` or via the shortcut: `pycbsdk-rates`.

## Introduction

Expand Down
4 changes: 4 additions & 0 deletions docs/img/pycbsdk_design.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
49 changes: 32 additions & 17 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,27 +1,42 @@
[tool.poetry]
[project]
name = "pycbsdk"
version = "0.1.4"
description = "Pure Python interface to Blackrock Neurotech Cerebus devices"
authors = ["Chadwick Boulay <[email protected]>"]
authors = [
{ name = "Chadwick Boulay", email = "[email protected]" },
]
license = "MIT"
readme = "README.md"
packages = [
{ include = "pycbsdk", from = "src" }
requires-python = ">=3.9"
dynamic = ["version"]
dependencies = [
"numpy",
"aenum>=3.1.15",
"ifaddr>=0.2.0",
]

[tool.poetry.dependencies]
python = ">=3.9,<3.13"
numpy = "^1.26.4"
aenum = "^3.1.15"
ifaddr = "^0.2.0"
[project.optional-dependencies]
test = [
"pytest>=8.3.3",
]

[tool.poetry.group.dev.dependencies]
typer = "^0.9.0"
pytest = "^8.1.1"
[project.scripts]
pycbsdk-rates = "pycbsdk.examples.print_rates:main"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
requires = ["hatchling", "hatch-vcs"]
build-backend = "hatchling.build"

[tool.hatch.version]
source = "vcs"

[tool.poetry.scripts]
ezmsg-monitor = "pycbsdk.examples.print_rates:main"
[tool.hatch.build.hooks.vcs]
version-file = "src/pycbsdk/__version__.py"

[tool.hatch.build.targets.wheel]
packages = ["src/pycbsdk"]

[tool.uv]
dev-dependencies = [
"ruff>=0.6.8",
"typer>=0.12.5",
]
5 changes: 1 addition & 4 deletions src/pycbsdk/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1 @@
import importlib.metadata


__version__ = importlib.metadata.version("pycbsdk")
from .__version__ import __version__ as __version__
6 changes: 3 additions & 3 deletions src/pycbsdk/cbhw/device/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,9 @@ def __init__(self, params: Params):
_: [] for _ in CBChannelType
}
# Init config_callbacks as a defaultdict that will create an empty list on-the-fly for unseen keys.
self.config_callbacks: defaultdict[
CBPacketType, typing.List[CBPktCallBack]
] = defaultdict(lambda: [])
self.config_callbacks: defaultdict[CBPacketType, typing.List[CBPktCallBack]] = (
defaultdict(lambda: [])
)
self._params = params
self.packet_factory = CBPacketFactory(protocol=self._params.protocol)
self.pkts_received = 0
Expand Down
27 changes: 10 additions & 17 deletions src/pycbsdk/cbhw/device/nsp.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,10 @@
import copy
from ctypes import Structure, create_string_buffer
import logging
import queue
import socket
from collections.abc import Callable
from enum import IntEnum, Flag, IntFlag
from typing import Optional, Type
from enum import IntEnum, IntFlag
from typing import Optional
import struct
import threading
import time
Expand Down Expand Up @@ -244,7 +243,7 @@ class LNCRate:
def GetLNCRate(key) -> int:
try:
return LNCRate.lnc_rates[key]
except KeyError as e:
except KeyError:
print("Error with LNC rate key.")
return 0

Expand Down Expand Up @@ -286,9 +285,7 @@ def __init__(self, params: Params, **kwargs):
self._monitor_state["time"] = 1

# Placeholders for IO
self._pkt_handler_thread = None
self._sender_queue = None
self._receiver_queue = None
self._io_thread = None

self._register_basic_callbacks()
Expand Down Expand Up @@ -317,6 +314,10 @@ def __init__(self, params: Params, **kwargs):
self._params.inst_port,
)

# Start the packet handler thread. We don't expect it to receive any packets until `.connect()` is called.
self._pkt_handler_thread = PacketHandlerThread(self)
self._pkt_handler_thread.start()

@property
def device_addr(self) -> tuple[str, int]:
return self._device_addr
Expand Down Expand Up @@ -1121,7 +1122,7 @@ def set_transport(
event = self._config_events["sysrep"]
logger.debug(f"Attempting to set transport to {transport.upper()}")
if not self._send_packet(pkt, event=event, timeout=timeout):
logger.warning(f"Did not receive SYSREPTRANSPORT in expected timeout.")
logger.warning("Did not receive SYSREPTRANSPORT in expected timeout.")

def get_transport(self, force_refresh=False) -> int:
if force_refresh:
Expand All @@ -1136,18 +1137,13 @@ def reset(self) -> int:

# region IO
def connect(self, startup_sequence: bool = True) -> int:
self._receiver_queue = queue.SimpleQueue()

self._pkt_handler_thread = PacketHandlerThread(self._receiver_queue, self)
self._io_thread = CerebusDatagramThread(
self._receiver_queue,
self._pkt_handler_thread.receiver_queue,
self._local_addr,
self._device_addr,
self._params.protocol,
self._params.recv_bufsize,
)

self._pkt_handler_thread.start()
self._io_thread.start()
# _io_thread.start() returns immediately but takes a few moments until its send_q is created.
time.sleep(0.5)
Expand Down Expand Up @@ -1185,9 +1181,6 @@ def disconnect(self):
self._io_thread.join()
self._pkt_handler_thread.join()

del self._receiver_queue
self._receiver_queue = None

logger.info("Disconnected successfully.")

def _startup_sequence(self) -> CBError:
Expand Down Expand Up @@ -1254,7 +1247,7 @@ def _send_packet(
if event is not None:
res = event.wait(timeout=timeout)
if not res:
logger.debug(f"timeout expired waiting for event")
logger.debug("timeout expired waiting for event")
return False
return True

Expand Down
13 changes: 9 additions & 4 deletions src/pycbsdk/cbhw/handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,17 +24,19 @@ class PacketHandlerThread(threading.Thread):
and should use atomic operations only.
"""

def __init__(
self, receiver_queue: queue.SimpleQueue, device: DeviceInterface, **kwargs
):
def __init__(self, device: DeviceInterface, **kwargs):
super().__init__(**kwargs)
self._recv_q = receiver_queue
self._recv_q = queue.SimpleQueue()
self._device = device
self._continue = False
self._packet_factory = CBPacketFactory(protocol=device._params.protocol)
self._stop_event = threading.Event()
self.daemon = True

@property
def receiver_queue(self) -> queue.SimpleQueue:
return self._recv_q

def run(self) -> None:
last_group_time = -1
last_group_data = None
Expand Down Expand Up @@ -118,6 +120,9 @@ def run(self) -> None:
)
self.warn_unhandled(pkt)

del self._recv_q
self._recv_q = None

def stop(self):
self._stop_event.set()

Expand Down
11 changes: 9 additions & 2 deletions src/pycbsdk/cbhw/io/datagram.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ def datagram_received(self, data: bytes, addr: tuple[str, int]) -> None:
self._recv_queue.put((pkt_time, chid, pkt_type, dlen, data))

def error_received(self, exc: Exception) -> None:
logger.error("Error received: ", exc)
logger.error(f"Error received: {exc}")

def connection_lost(self, exc: Exception) -> None:
logger.debug("Data receiver connection closed.")
Expand Down Expand Up @@ -143,9 +143,16 @@ async def _receiver_coro(self):
)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_DONTROUTE, True)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, self._buff_size)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True)
# sock.settimeout(10)
# sock.setblocking(False)
sock.bind(self._recv_addr)
try:
sock.bind(self._recv_addr)
except OSError as e:
logger.error(
f"Cannot bind to {self._recv_addr}. Central may have exclusive access to the port on "
f"this machine. Error: {e}"
)

loop = asyncio.get_event_loop()
# Create a future that should only return when the UDP connection is lost
Expand Down
1 change: 0 additions & 1 deletion src/pycbsdk/cbhw/packet/abstract.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
import struct
import numpy as np
import numpy.typing
from .. import config
from .common import (
CBPacketType,
CBSpecialChan,
Expand Down
Loading