23

what are the reasons for when using cURL form linux server to access the site return an error. but opening this site in chrome succeeds.

5
  • 1
    There are a few reasons, but off the top of my head, I'd suggest that it may be because your cURL call doesn't include headers (or cookies) that the page is expecting to have in the request. If you open the url in a new private window in your browser, what happens?
    – daf
    Commented Jul 2, 2016 at 13:01
  • 1
    Had same problem. That was because of user agent and compression gzip are not in headers.
    – user6522773
    Commented Jul 2, 2016 at 13:01
  • can you provide curl request you made with error produced by it.
    – Devavrata
    Commented Jul 2, 2016 at 13:02
  • I think the problem is with SSL error. Can anyone tell me how to fix it on client side and also server side
    – john
    Commented Jul 2, 2016 at 14:00
  • @daf works with private window in the browser
    – john
    Commented Jul 2, 2016 at 15:17

4 Answers 4

12

Look browsers headers while sending your request, and add same headers to cURL request. Some servers needs some headers that browsers sends default but not cURL.

3
  • 1
    this solved my issue. server wanted a user-agent header. Commented Oct 20, 2017 at 19:54
  • 2
    Good advice. In my case it was missing --digest and -c cookies.txt
    – Alex R
    Commented Oct 27, 2018 at 18:07
  • 1
    I had to include user-agent and accept. This actually fixed an issue I had with Apache HttpClient. The exceptions weren't particularly useful, so I was scratching my head over this for days. Thanks.
    – J. Lin
    Commented Apr 18, 2020 at 15:35
2

I had 301 Moved Permanently error on curl but was working on browser.

But when I looked deeper there's Location to moved to location, which would be working. This coulde be when the server is using TLS.

$ curl -v http://www.shaharma.com/location/v1/US/zipcode/98104
*   Trying 172.111.99.100...
* TCP_NODELAY set
* Connected to www.shaharma.com (172.111.99.100) port 80 (#0)
> GET /location/v1/US/zipcode/98104 HTTP/1.1
> Host: www.shaharma.com
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< Content-Length: 0
< Location: https://www.shaharma.com/location/v1/US/zipcode/98104
< Cache-Control: max-age=0
< Expires: Fri, 01 Dec 2017 19:40:14 GMT
< Date: Fri, 01 Dec 2017 19:40:14 GMT
< Connection: Keep-Alive
< Set-Cookie: UID=52679eee-c06c-49fc-893a-69fd7e46bad0; expires=Fri, 25-Feb-2028 19:40:14 GMT; path=/; domain=.shaharma.com
< Set-Cookie: SID=1ada5fa9-0ace-4f4e-b75b-7a756b8da934; path=/; domain=.shaharma.com
< Set-Cookie: shaharma_loc_lb=p-loc-w; expires=Fri, 01-Dec-2017 19:50:14 GMT; path=/; domain=.shaharma.com
< Set-Cookie: bby_rdp=l; expires=Sat, 02-Dec-2017 19:40:14 GMT; path=/; domain=.shaharma.com
< 
* Connection #0 to host www.shaharma.com left intact

Also, you can goto brower and see the network which will have headers information.

headers

0
1

To avoid/suppress SSL error, use -k flag, ie

curl -k ...
0

For me it was because I configured the certificate generated with acme.sh in nginx like this:

ssl_certificate /root/.acme.sh/my.domain.com_ecc/my.domain.com.cer;
ssl_certificate_key /root/.acme.sh/my.domain.com_ecc/my.domain.com.key;

But I needed to configure it using the fullchain certificate:

ssl_certificate /root/.acme.sh/my.domain.com_ecc/fullchain.cer;
ssl_certificate_key /root/.acme.sh/my.domain.com_ecc/my.domain.com.key;

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.