I'm using Scrapy and Splash (via scrape-splash) to render a page with js and images.
I'm able to send custom headers to the request for the URL that's rendered:
url = "https://8763-48c1-bf65-b80b-9747-468f-b476-5b77-579d.ngrok.io"
yield SplashRequest(url, ..., args={'headers': {"Foo": "Bar"} })
This will correctly make a request to the server with the following headers:
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/602.1 (KHTML, like Gecko) splash Version/10.0 Safari/602.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en,*
Foo: Bar
When that page is rendered, a js file is then requested, but without the custom headers:
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/602.1 (KHTML, like Gecko) splash Version/10.0 Safari/602.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en,*
Referer: https://8763-48c1-bf65-b80b-9747-468f-b476-5b77-579d.ngrok.io/
Everything I've been able to find (splash:set_custom_headers, splash:go) all refer to headers on the original request to render, but not the additional requests made during the render.