At the end of the Middle Ages, the Caumont family built the Chateau des Milandes. They desired a more pleasant and comfortable home in keeping with the architecture and design of the era. From then on, Castelnaud was only inhabited by a few soldiers and their captain in the new living quarters of the east and south castle wings.