I've written a function in VBA (Excel) to process an API request. For small (<150MB or so) requests it works just fine, but for large requests (>150 MB up to 300-400MB of returned data) I get:
error: 2147024882 (8007000e) Not enough storage is available to complete this operation.
Below is the code (with non-applicable pieces removed):
Function SendHTTPRequest(sURL As String, Optional sGETorPOST As String = "POST", Optional sHeaders2DArray As Variant, Optional sGetBodyOrPostArgs As String) As msxml2.ServerXMLHTTP60
Dim aheaders() As Variant, i As Long
If sGETorPOST <> "GET" And sGETorPOST <> "POST" Then Exit Function
If IsMissing(sHeaders2DArray) Then
ReDim aheaders(0 To 0, 0 To 1)
aheaders(0, 0) = "User-Agent"
aheaders(0, 1) = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
Else
ReDim aheaders(LBound(sHeaders2DArray) To UBound(sHeaders2DArray), LBound(sHeaders2DArray, 2) To UBound(sHeaders2DArray, 2))
aheaders = sHeaders2DArray
End If
Set SendHTTPRequest = New msxml2.ServerXMLHTTP60
SendHTTPRequest.Open sGETorPOST, sURL, False
For i = LBound(aheaders) To UBound(aheaders)
SendHTTPRequest.setRequestHeader aheaders(i, 0), aheaders(i, 1)
Next
If sGETorPOST = "POST" Then SendHTTPRequest.setRequestHeader "Content-type", "application/x-www-form-urlencoded"
SendHTTPRequest.send (sGetBodyOrPostArgs) '<-- Error occurs here
End Function
I've been tackling this problem for two days. I was originally doing the POST request wrong and fixed it, and I changed from XMLHTTP to ServerXMLHTTP.
Here's the applicable code calling the function:
aheaders(0, 0) = "User-Agent"
aheaders(0, 1) = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
aheaders(1, 0) = "X-Requested-With"
aheaders(1, 1) = "[redacted]"
aheaders(2, 0) = "Authorization"
aheaders(2, 1) = "Basic " & EncodeBase64(USER & ":" & PASS)
sURL = URLAPI & "/api/2.0/fo/scan/"
sArgs = "action=fetch&scan_ref=" & sRef & "&mode=extended&output_format=json"
Set httpRequest = SendHTTPRequest(sURL, "POST", aheaders, sArgs)
I've been testing in Excel 2010, 2013, and 2016. It's definitely not a resource issue. I'm running a Xeon W3550 with 12GB ram and a 500GB SSD that's less than half full.
Is there something I can do to fix this code, or is there an alternate way to make the API request?
SendHTTPRequest.Open sGETorPOST, sURL, True
- and repeatedly reading from the responseStream property of the request. Not sure if this will help though. Details of how to work with the ADODB.Stream object are here – barrowc Oct 1 '15 at 4:39