Ergot alkaloids are mycotoxins produced by fungi of the genus Claviceps, which infect cereal crops and grasses. The uptake of ergot alkaloid contaminated cereal products can be lethal to humans and animals. For food safety assessment, analytical techniques are currently used to determine the presence of ergot alkaloids in food and feed samples. However, the number of samples which can be analyzed is limited, due to the cost of the equipment and the need for skilled personnel. In order to compensate for the lack of rapid tests for the detection of ergot alkaloids, the aim of this study was to develop a specific recognition element for ergot alkaloids, which could be further applied to produce a colorimetric reaction in the presence of these toxins. As recognition elements, single-stranded DNA ligands were selected by using an iterative selection procedure named SELEX, i.e., Systematic Evolution of Ligands by EXponential enrichment. After several selection cycles, the resulting aptamers were cloned and sequenced. A surface plasmon resonance analysis enabled determination of the dissociation constants of the complexes of aptamers and lysergamine. Dissociation constants in the nanomolar range were obtained with three selected aptamers. One of the selected aptamers, having a dissociation constant of 44 nM, was linked to gold nanoparticles and it was possible to produce a colorimetric reaction in the presence of lysergamine. This system could also be applied to small ergot alkaloids in an ergot contaminated flour sample.