Ok, sorry, my fault.
Just checked the log in SmartThings and there was an error saying that it couldn't find a device type.
I went and double check your instructions (which I should have done first) and realized that it seems like I went too fast when configuring the smartapp and I skipped the device handler.
Just added the device handler and now things work as they should!