Between the sea and Ryssberget (a bigger hill) Sölvesborg developed. Here passed the important coastland road, Via Regia, and here was a protected harbour in Sölveviken (bay of Sölve). By the road was also the royal castle of Sölvesborg, which was the most powerful military and administrative point in the eastern Denmark. In 1445 the Danish king renewed the town privileges. It is not known when these were first issued, but it is probable that Sölvesborg becomes a town some time around year 1400. The first settlement seems to have been placed around the church. This is built in several stages during the middle ages. The oldest part is the sanctuary, which was built as a freestanding chapel during the 14th century.
The closest surrounding country was a part called Listerlandet, but goods from the north part of the county and Värend in Småland also were shipped from the harbour in Sölvesborg. The contacts between the towns around the southern Baltic were extensive, as they were with the rest of Denmark. When the new fortress town Kristianstad got its town privileges in 1614, Sölvesborg was left behind. The right to foreign trade was lost and the role as a place where trade is done was less and less dominating.
The place by the country road and the important castle made the town over and over again suffer from looting and burning by Swedish troops. After the transition to Sweden the town burnt three times, and at the last fire in 1801 destroyed all buildings except the church. After each fire disaster the town resurrected, mostly with simple half-timbered houses with roofs made of reed.