feat: IDSSE Dataset #147

manuba95 · 2024-09-05T09:13:18Z

This PR adds the initial version of the dataset accompanying the paper "An integrated dataset of synchronized spatiotemporal and event data in elite soccer" which is currently in submission at Nature Scientific Data. The dataset contains seven full matches of position and event data from the German Bundesliga first and second division.

This PR should be fully functional, however documentation, testing and cosmetics will be improved in the next commits.

draabe

hi, looks great!

just some cosmetics

floodlight/io/datasets.py

draabe · 2024-09-06T18:50:26Z

tests/test_io/test_datasets.py

+@pytest.mark.unit
+def test_dfl_get() -> None:
+
+    dataset = DFLDataset()
+    event_objects, position_objects = dataset.get()
+    assert isinstance(event_objects[0]["firstHalf"]["Home"], Events)
+    assert isinstance(event_objects[0]["secondHalf"]["Away"], Events)
+    assert isinstance(event_objects[1]["Home"], Teamsheet)
+    assert isinstance(event_objects[1]["Away"], Teamsheet)
+    assert isinstance(event_objects[2], Pitch)
+
+    assert isinstance(position_objects[0]["firstHalf"]["Home"], XY)
+    assert isinstance(position_objects[0]["secondHalf"]["Away"], XY)
+    assert isinstance(position_objects[1]["secondHalf"], Code)
+    assert isinstance(position_objects[1]["firstHalf"], Code)
+    assert isinstance(position_objects[2]["secondHalf"], Code)
+    assert isinstance(position_objects[2]["firstHalf"], Code)
+    assert isinstance(position_objects[3]["Home"], Teamsheet)
+    assert isinstance(position_objects[3]["Away"], Teamsheet)
+    assert isinstance(position_objects[4], Pitch)


appreciate the effort, but how long does this take? feels like it may blow up our unit test suite from 8 seconds capable of running offline to 1 minute requiring internet connection

I agree on the duration part. This test alone takes about 1:15 min on my machine. I will remove the test for now. I think for this dataset, a good test could be checking the respective links for valid responses as internet connection is already required e.g., for the StatsBombOpenDataset tests.
However, I expect the final link to change (probably remove the private token) once the paper is published. So this could be added in a future fix?

yes, let's do it that way! probably make sense to split up our test-suite at some point with fast unit tests and a dedicated test battery for the parsing that might take longer, so they only need to be tested when working on that module or on pull requests. feel free to open another PR or issue on that

draabe · 2024-09-07T11:13:04Z

Hi @manuba95 and @VellaRo

one more general comment: I've noticed you have been using the requests module for file downloads in the added parsers/datasets. Is there a reason for it? I know we have it as a transitive dependency, but so far we've been relying exclusively on urllib. I know requests is widely recommend and liked, but all we ever going to do http wise is send get requests, so basic functionality only. And that's always going to be faster with urllib as we don't need to load a gazillion fancy request features. (we might want to update to urllib3, though).

Reason I'm asking is because we have dedicated download functions in the io/utils.py submodule, but the new parsers are deviating from this. I'd prefer if we'd think about this and see if we can find a unified way, or at least have good reasons should we end up with multiple download strategies/packages/submodules.

Any thoughts?

manuba95 · 2024-09-07T11:57:54Z

Fully agree on that! Simply a misunderstanding from my side on what the download functions in the utils would do. Changed it in the wrapper function. @VellaRo I suggest you do the same in your respective features as this is a pretty easy fix.

codecov · 2024-09-07T12:02:24Z

Codecov Report

Attention: Patch coverage is 9.09091% with 70 lines in your changes missing coverage. Please review.

Project coverage is 85.89%. Comparing base (22cb3a2) to head (ffa8bbf).
Report is 1 commits behind head on develop.

Files with missing lines	Patch %	Lines
floodlight/io/datasets.py	9.09%	70 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (22cb3a2) and HEAD (ffa8bbf). Click for more details.

HEAD has 3 uploads less than BASE

Flag BASE (22cb3a2) HEAD (ffa8bbf)

6 3

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #147      +/-   ##
===========================================
- Coverage    95.58%   85.89%   -9.70%     
===========================================
  Files           47       48       +1     
  Lines         3194     3594     +400     
===========================================
+ Hits          3053     3087      +34     
- Misses         141      507     +366

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

draabe · 2024-09-09T18:02:29Z

floodlight/io/datasets.py

+
+    @staticmethod
+    def get_pitch() -> Pitch:
+        """Returns a Pitch object corresponding to the DFL-data."""


draabe · 2024-09-09T18:03:51Z

floodlight/io/datasets.py

+    References
+    ----------
+        .. [1] `Bassek, M., Weber, H., Rein, R., & Memmert,D. (2024). An integrated
+        dataset of synchronized spatiotemporal andevent data in elite soccer. In


Suggested change

dataset of synchronized spatiotemporal andevent data in elite soccer. In

dataset of synchronized spatiotemporal and event data in elite soccer. In

draabe · 2024-09-09T18:08:21Z

tests/test_io/test_datasets.py

+@pytest.mark.unit
+def test_dfl_get() -> None:
+
+    dataset = DFLDataset()
+    event_objects, position_objects = dataset.get()
+    assert isinstance(event_objects[0]["firstHalf"]["Home"], Events)
+    assert isinstance(event_objects[0]["secondHalf"]["Away"], Events)
+    assert isinstance(event_objects[1]["Home"], Teamsheet)
+    assert isinstance(event_objects[1]["Away"], Teamsheet)
+    assert isinstance(event_objects[2], Pitch)
+
+    assert isinstance(position_objects[0]["firstHalf"]["Home"], XY)
+    assert isinstance(position_objects[0]["secondHalf"]["Away"], XY)
+    assert isinstance(position_objects[1]["secondHalf"], Code)
+    assert isinstance(position_objects[1]["firstHalf"], Code)
+    assert isinstance(position_objects[2]["secondHalf"], Code)
+    assert isinstance(position_objects[2]["firstHalf"], Code)
+    assert isinstance(position_objects[3]["Home"], Teamsheet)
+    assert isinstance(position_objects[3]["Away"], Teamsheet)
+    assert isinstance(position_objects[4], Pitch)


yes, let's do it that way! probably make sense to split up our test-suite at some point with fast unit tests and a dedicated test battery for the parsing that might take longer, so they only need to be tested when working on that module or on pull requests. feel free to open another PR or issue on that

draabe

looks good, just two typos

manuba95 added 6 commits July 24, 2024 15:47

fix: wrong branch cleanup

0b4cc9e

fix: try updated poetry

3a0a054

tests: add first test

4b9e6e8

feat: add links to data repository

016934d

refactor: streamline docs and refactor varible names

c1dea28

docs: add IDSSEDataset to module references

c99d719

draabe requested changes Sep 6, 2024

View reviewed changes

build: get dependencies up to date

501160f

fix: remove test

20657f5

fix: use urllib for download

ffa8bbf

manuba95 added 4 commits September 7, 2024 14:19

refactor: adjust paths to fit character length

18e788b

fix: typos, typing, structure

95cfaef

fix: adjust returns of .get() method

b135a72

docs: adjust returns in example

119f0f0

draabe reviewed Sep 9, 2024

View reviewed changes

draabe mentioned this pull request Sep 9, 2024

feat: Catapult API #142

Open

fix: typos

22c5ad1

draabe approved these changes Sep 10, 2024

View reviewed changes

draabe merged commit ddb0422 into floodlight-sports:develop Sep 10, 2024
7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: IDSSE Dataset #147

feat: IDSSE Dataset #147

manuba95 commented Sep 5, 2024

draabe left a comment

draabe Sep 6, 2024

manuba95 Sep 7, 2024

draabe Sep 9, 2024

draabe commented Sep 7, 2024

manuba95 commented Sep 7, 2024

codecov bot commented Sep 7, 2024

draabe Sep 9, 2024

draabe Sep 9, 2024

draabe Sep 9, 2024

draabe left a comment

	dataset of synchronized spatiotemporal andevent data in elite soccer. In
	dataset of synchronized spatiotemporal and event data in elite soccer. In

feat: IDSSE Dataset #147

feat: IDSSE Dataset #147

Conversation

manuba95 commented Sep 5, 2024

draabe left a comment

Choose a reason for hiding this comment

draabe Sep 6, 2024

Choose a reason for hiding this comment

manuba95 Sep 7, 2024

Choose a reason for hiding this comment

draabe Sep 9, 2024

Choose a reason for hiding this comment

draabe commented Sep 7, 2024

manuba95 commented Sep 7, 2024

codecov bot commented Sep 7, 2024

Codecov Report

draabe Sep 9, 2024

Choose a reason for hiding this comment

draabe Sep 9, 2024

Choose a reason for hiding this comment

draabe Sep 9, 2024

Choose a reason for hiding this comment

draabe left a comment

Choose a reason for hiding this comment