The archbishop in Lund had, as one of Denmark's biggest landowners, a well functioning administrative organisation. A part of this organisation was Elleholm with the castle Sjöborg. On the low-lying Alholmen (Alder islet) a village formed. The good salmon fishing in Mörrumsån (River of Mörrum), and a wide surrounding country gave a good base for trade. In 1450, the archbishop Tuve Nelson, issued town privileges to Elleholm. Six years without having to pay taxes was promised to anyone who settled in the new town and derived his livelihood there.
There were only two rows of houses on the island built by wood or half-timbered houses. In the town there was a chapel, which in the beginning of the 18th century was replaced by the present church. Farthest to the south, parted from the island by a fosse, was the house of the bishop.
Right north on the main island is the remnants of a medieval castle, Sjöborg. The castle area consists of two grass-covered plateaus separated by another filled fosse. As one of several castles of bailiffs, Sjöborg was taken and burnt in 1436 in connection to a popular rising. During the 16th century, Blekinge repeatedly was hit by war. Elleholm was taken and looted. In the year 1600 the town privileges were removed and transferred to Sölvesborg.