We consider a self-assembled hybrid system, composed of a bilayer vesicle to which a number of colloids are ad-hered. Based on known results of membrane curvature elasticity, we predict that, for sufficiently deflated prolate vesicles, the colloids can self-assemble into a ring at a finite distance away from the vesicle equator, thus breaking the up–down symmetry in the system. Because the relative variation of the position of the colloidal ring along the vesicle endows the system with an effective elasticity, periodic cycles of inflation and deflation can lead to non-reciprocal shape changes of the vesicle–colloid hybrid, allowing it to swim in a low Reynolds number environment under reciprocal actuation. We design several actuation protocols that allow control over the swimming direction.