Normally, when you drag a floating connector around a shape, it moves evenly, connecting to the closest or optimal location.

If you want to restrict connectors to just the connection points, you can use snap them to the shape’s anchor points.

Enable snap to point

1. Select either the connector or the shape.
2. In the Style tab of the format panel on the right, expand the Properties section.
3. Enable the checkbox called Snap to Point.

Alternatively, manually edit the shape style and add snapToPoint=1; to the style definition.