Non-coplanar helimagnetism in the layered van-der-Waals metal DyTe3

Abstract Magnetic materials with highly anisotropic chemical bonding can be exfoliated to realize ultrathin sheets or interfaces with highly controllable optical or spintronics responses, while also promising novel cross-correlation phenomena between electric polarization and the magnetic texture. The vast majority of these van-der-Waals magnets are collinear ferro-, ferri-, or antiferromagnets, with a particular scarcity of lattice-incommensurate helimagnets of defined left- or right-handed rotation sense, or helicity. Here we use polarized neutron scattering to reveal cycloidal, or conical, magnetic structures in DyTe3, where insulating double-slabs of dysprosium square nets are separated by highly metallic tellurium layers. We identify a hierarchy of energy scales with - in order of decreasing strength - antiferromagnetic exchange interactions, magnetocrystalline anisotropy, and periodic modulations of the exchange energy. The latter are attributed to magneto-elastic coupling to the unconventional charge order in DyTe3. This easily cleavable metallic helimagnet also hosts a complex magnetic phase diagram indicative of competing interactions. Our work paves the way for twistronics research, where helimagnetic layers can be combined to form complex spin textures on-demand, using the vast family of rare earth chalcogenides and beyond.