The Hermitage of San Felices is situated at the top of the Bilibio cliffs that make up the Haro shells, next to the Buradón cliffs. These cliffs have been a special place of worship since ancient times, developing various religious centers in the vicinity such as the Cueva de Páceta (Álava), or the Monastery of Herrera (Herrera, La Rioja).

The fortification “Castro de Bilibio” had been inhabited since the First Iron Age (7th-5th century BC), and there is also some evidence of contact with the Romans, such as the remains of Hispanic terra sigilata (ceramics) in the remains of some fortifications, some references to “Baelibio” in the nearby hermitage of San Bartolomé (Angostina, Álava), and the existence of a Roman road in the vicinity of Bilibio that communicated the Ebro valley through La Rioja to Burgos.

According to the tradition, San Felices (6th century AD) retired to pray to the cliffs of Bilibio. During this period as an anchorite lived San Millán with him, to initiate himself as an anchorite in the year 493. Finally, San Felices died in 520 and was buried in one of the caves of the mount itself, where he rested until 1090 when his remains were transferred to the San Millán Monastery and some of his relics were preserved in the church of Santo Tomás in Haro . The first hermitage consecrated to San Millán was erected in Haro itself, but the current one was built in 1710 just where the anchorite’s first resting place was located.
References: Mayoral, Mª P. Pascual, et al. La cueva de Páceta: Castro Bilibio (La Rioja)– ¿Un oratorio rupestre? Antigüedad y cristianismo, 2006, no 23, p. 719-737. Bodegas, Pablo Díaz. En torno a la traslación de San Felices de Bilibio, patrono de Haro. Berceo, 1992, no 123, p.31-47