What would be my cheapest option if I wanted to get from Xiamen to Hong Kong?

The most economical way is to take a high speed train to Shenzhen North from the north railway station. The first-class seat costs CNY 181, and the second-class seat costs 150.5. Upon arrival, take metro Longhua line to Futian Port, cross the border and you'll be in HK. There is MTR East Rail Line and buses taking you to other part of HK from the border area.