How would you model this "strand" clustering problem? [P]

https://preview.redd.it/llqlupnwng4h1.png?width=2188&format=png&auto=webp&s=7fae5860babaffa1c8bfdcb1468b374eb38ac55d

I'm currently building a computer vision application. I've managed to successfully train a YOLO model to detect the object I'm interested in for my videos.

The image above shows some visualisations of the YOLO model outputs for some of the videos. I want to essentially cluster these strands in the image into groups based on their separation distance and return a string telling me the number of strands in each group from left to right (e.g. 1-2-3).

The target value for each column in the image (where each column corresponds to a video) is 1-2-3, 1-2-3-2-3, 1-1-2-3-3-3-3 and don't worry about the fourth column for now 😄.

The rows show the x vs t, y vs t and x vs y vs t for all the detections and the points are sized based on the detection box area.

In the fourth column I have some background object detections which I want to ignore hence why I've also visualised detection box area.

I've managed to train a XGBoost classification model that gives 70ish% accuracy however Bayes error is making me think I should be able to do much better than this.

How would you approach trying to predict these strand clusterings?

Some extra info that might help; there are at max 8 groups and each group can have only at max 3 strands.