Using Puppeteer, I'd like to load a URL in Chrome and capture the following information:
- request URL
- request headers
- request post data
- response headers text (including duplicate headers like
set-cookie
) - transferred response size (i.e. compressed size)
- full response body
Capturing the full response body is what causes the problems for me.
Things I've tried:
- Getting response content with
response.buffer
- this does not work if there are redirects at any point, since buffers are wiped on navigation - intercepting requests and using
getResponseBodyForInterception
- this means I can no longer access the encodedLength, and I also had problems getting the correct request and response headers in some cases - Using a local proxy works, but this slowed down page load times significantly (and also changed some behavior for e.g. certificate errors)
Ideally the solution should only have a minor performance impact and have no functional differences from loading a page normally. I would also like to avoid forking Chrome.