There are some problems when sending request with fixed hardcoded token, i would prefer to capture the header and cookies for authentication and use the same for the API request again to avoid invalid authentication issues, few days before i had a same experience where structure was totally irritating but i managed to use this trick and got it working.

Again nice article and better trick to avoid issues with complicated html structure.