1

I'm using python to retrieve data from Successfactor API . There is a entity so heavy that the default 1000 records return per request always causes server to timeout.

I want to reduce the number of records return per request to avoid the timeout issue from server.

I have tried $top = 100 ... but can't move to the next 100 because there is no __next link in the returned result nor $offset option.

I have tried to limit the number of colums return to nonNavigationProperties. still no help.

Have anyone known how to resolve it?

below is an example:

https://xxxx.successfactors.com/odata/v2/JobRequisition?$format=JSON&&$filter=lastModifiedDateTime ge datetime'2015-08-27T00:00:00' and lastModifiedDateTime le datetime'2015-08-27T23:59:59'&$select=age,appTemplateId,assessRatingScaleName,cReq1UpApprover,cReqAddSourcing,cReqAdvertCost,cReqAdvertDetails,cReqAdvertDraftCopy,cReqAlternateLocation,cReqAppID,cReqAssignmentValue,cReqAward,cReqCancelCost,cReqCancelReason,cReqCancelbyELT,cReqChangetoterms,cReqChargeRate,cReqCompany,cReqContractDuration,cReqCoreSourcing,cReqCurrentEndDate,cReqDateShortlist,cReqDescServices,cReqEstAgencyCost,cReqEstCostOfHire,cReqExtensionEndDate,cReqExtensionStartDate,cReqFirstName,cReqFlexiWork,cReqHMPositionTitle,cReqHSE,cReqHiringManagerComments,cReqHoursPerWeek,cReqInt1Req,cReqJBComments,cReqLastName,cReqLimitedTermDuration,cReqLinkedInProject,cReqMarketMapInfo,cReqMaxRate,cReqNewDate,cReqOfferTarget,cReqOriginalREQID,cReqOriginalStartDate,cReqPOApprover1,cReqPOApprover2,cReqPOApprover3,cReqPreIdenCand,cReqPreferredName,cReqPropAccumPOSpend,cReqRateCardCode,cReqRateEndRange,cReqRateStartRange,cReqRateWSuper,cReqRecSearchFirm,cReqRemEndRange,cReqRemStartRange,cReqReplacementFor,cReqRequirements,cReqResTeamComments,cReqRetainer1,cReqRetainer2,cReqRetainer3,cReqRoleInfo,cReqRosterPatDetails,cReqSecondSourcing,cReqSrcPlan,cReqStartDate,cReqStartofRoster,cReqTesting,cReqTotalPOValuetodate,cReqTravel,cReqUserID,candidateProgress,city,closedDateTime,corporatePosting,costCenterId,country,createdDateTime,currency,defaultLanguage,deleted,department,division,erpAmount,extPostEndDate,extPostStartDate,formDataId,formDueDate,instCongofStartReq,instrConfirmStarted,instrContractExtnReq,instrContractHistory,instrContractReq,instrEmpReq,instrExecAssignmentInfo,instrFurtherInstructions,instrInterview1,instrJobBrief,instrNewHireName,instrPosDetails,instrPostingDetails,instrProposedContractCosts,instrProposedExtn,instrRateCardRate,instrReqCost,instrReqDetails,instrResTimeframe,instrRoleInfo,instrSrcPlan,instrToManager,internalStatus,intranetPosting,isDraft,jobCode,jobGrade,jobGradeCode,jobReqGUId,jobReqId,jobRole,jobStartDate,lastModifiedBy,lastModifiedDateTime,lastModifiedProxyUserId,location,numberOpenings,openingsFilled,overallScaleName,positionNumber,postalcode,ratedApplicantCount,relocationCost,reverseScale,stateProvince,statusSetId,templateId,templateType,timeToFill

1
  • would be glad the get an response to my answer, at least if it helped or not.
    – dotchuZ
    Sep 13, 2018 at 13:30

1 Answer 1

2

Did you try using $skip?

https://xxxx.successfactors.com/odata/v2/JobRequisition?
$format=JSON&$filter=lastModifiedDateTime ge datetime'2015-08-27T00:00:00' 
and lastModifiedDateTime le datetime'2015-08-27T23:59:59'
&$top=100&$skip=100

This should result in sets 100-200, instead of 0-100...

Check this: https://blogs.sap.com/2013/03/20/using-odatas-top-skip-and-count/

I think there is no "automatic" possibility to achieve this with a next-link or something else, this does not exist. I would recommend using $count first to get the total sum. Then retrieve your page size with top and skip commands.

1
  • by the way, using SAP Cloud Integration allows you to configure the page size dynamically and splits all requests into batches... which you would manually build in python.
    – dotchuZ
    Sep 12, 2018 at 9:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.