13

I would like to download n-th image that google gives me with command line i.e. like with command wget

To search image of [something] I just go to page https://www.google.cz/search?q=[something]&tbm=isch but how do I get url of n-th search result so I can use wget?

1
  • Why did this question get closed? I do not understand the explanation I got: "We don’t allow questions about general computing hardware and software on Stack Overflow" How does that apply, especially when this question happily lived here for 6 years.
    – tom
    Oct 24, 2021 at 17:33

5 Answers 5

24

First attempt

First you need to set the user agent so google will authorize output from searches. Then we can look for images and select the desired one. To accomplish that we insert missing newlines, wget will return google searches on one single line, and filter the link. The index of the file is stored in the variable count.

$ count=10
$ imagelink=$(wget --user-agent 'Mozilla/5.0' -qO - "www.google.be/search?q=something\&tbm=isch" | sed 's/</\n</g' | grep '<img' | head -n"$count" | tail -n1 | sed 's/.*src="\([^"]*\)".*/\1/')
$ wget $imagelink 

The image will now be in your working directory, you can tweak the last command and specify a desired output file name.

You can summarize it in a shell script:

#! /bin/bash
count=${1}
shift
query="$@"
[ -z $query ] && exit 1  # insufficient arguments
imagelink=$(wget --user-agent 'Mozilla/5.0' -qO - | "www.google.be/search?q=${query}\&tbm=isch" | sed 's/</\n</g' | grep '<img' | head -n"$count" | tail -n1 | sed 's/.*src="\([^"]*\)".*/\1/')
wget -qO google_image $imagelink

Example usage:

$ ls
Documents
Downloads
Music
script.sh
$ chmod +x script.sh
$ bash script.sh 5 awesome
$ ls
Documents
Downloads
google_image
Music
script.sh

Now the google_image should contain the fifth google image when looking for 'awesome'. If you experience any bugs, let me know, I'll take care of them.

Better code

The problem with this code is that it returns pictures in low resolution. A better solution is as follows:

#! /bin/bash

# function to create all dirs til file can be made
function mkdirs {
    file="$1"
    dir="/"

    # convert to full path
    if [ "${file##/*}" ]; then
        file="${PWD}/${file}"
    fi

    # dir name of following dir
    next="${file#/}"

    # while not filename
    while [ "${next//[^\/]/}" ]; do
        # create dir if doesn't exist
        [ -d "${dir}" ] || mkdir "${dir}"
        dir="${dir}/${next%%/*}"
        next="${next#*/}"
    done

    # last directory to make
    [ -d "${dir}" ] || mkdir "${dir}"
}

# get optional 'o' flag, this will open the image after download
getopts 'o' option
[[ $option = 'o' ]] && shift

# parse arguments
count=${1}
shift
query="$@"
[ -z "$query" ] && exit 1  # insufficient arguments

# set user agent, customize this by visiting http://whatsmyuseragent.com/
useragent='Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:31.0) Gecko/20100101 Firefox/31.0'

# construct google link
link="www.google.cz/search?q=${query}\&tbm=isch"

# fetch link for download
imagelink=$(wget -e robots=off --user-agent "$useragent" -qO - "$link" | sed 's/</\n</g' | grep '<a href.*\(png\|jpg\|jpeg\)' | sed 's/.*imgurl=\([^&]*\)\&.*/\1/' | head -n $count | tail -n1)
imagelink="${imagelink%\%*}"

# get file extention (.png, .jpg, .jpeg)
ext=$(echo $imagelink | sed "s/.*\(\.[^\.]*\)$/\1/")

# set default save location and file name change this!!
dir="$PWD"
file="google image"

# get optional second argument, which defines the file name or dir
if [[ $# -eq 2 ]]; then
    if [ -d "$2" ]; then
        dir="$2"
    else
        file="${2}"
        mkdirs "${dir}"
        dir=""
    fi
fi   

# construct image link: add 'echo "${google_image}"'
# after this line for debug output
google_image="${dir}/${file}"

# construct name, append number if file exists
if [[ -e "${google_image}${ext}" ]] ; then
    i=0
    while [[ -e "${google_image}(${i})${ext}" ]] ; do
        ((i++))
    done
    google_image="${google_image}(${i})${ext}"
else
    google_image="${google_image}${ext}"
fi

# get actual picture and store in google_image.$ext
wget --max-redirect 0 -qO "${google_image}" "${imagelink}"

# if 'o' flag supplied: open image
[[ $option = "o" ]] && gnome-open "${google_image}"

# successful execution, exit code 0
exit 0

The comments should be self explanatory, if you have any questions about the code (such as the long pipeline) I'll be happy to clarify the mechanics. Note that I had to set a more detailed user agent on the wget, it may happen that you need to set a different user agent but I don't think it'll be a problem. If you do have a problem, visit http://whatsmyuseragent.com/ and supply the output in the useragent variable.

When you wish to open the image instead of only downloading, use the -o flag, example below. If you wish to extend the script and also include a custom output file name, just let me know and I'll add it for you.

Example usage:

$ chmod +x getimg.sh
$ ./getimg.sh 1 dog
$ gnome-open google_image.jpg
$ ./getimg.sh -o 10 donkey
24
  • 2
    @tom My pleasure, check out the script too!
    – ShellFish
    Jan 12, 2015 at 20:29
  • 2
    I'm afraid google doesn't supply a link to the original image in their website code. It must be javascript that handle's linking to the original image. If you can find it in google's code (using a browser's "inspect element") I'll be happy to take care of it.
    – ShellFish
    Jan 12, 2015 at 20:50
  • 2
    I was mistaken, I will look into it.
    – ShellFish
    Jan 12, 2015 at 21:14
  • 2
    The script should now take a second argument, which is either a target dir (where it will save the image as "google image.ext" or "google image(1).ext" etc if the file already exists. You can also supply a file name as second argument such as /home/user/Documents/images/image. The extension will be added automatically. -- Let me kniw if there are bugs.
    – ShellFish
    May 28, 2015 at 23:28
  • 2
    @User8547 Should be fixed now, report further bugs please.
    – ShellFish
    May 29, 2015 at 13:52
9

This is an addition to the answer provided by ShellFish. Much respect to them for working this out. :)

Google have recently changed their web-code for the image results page which has, unfortunately, broken Shellfish's code. I was using it every night in a cron job up until about 4 days ago when it stopped receiving search-results. While investigating this, I found that Google have removed elements like imgurl and have shifted a lot more into javascript.

My solution is an expansion of Shellfish's great code but has modifications to handle these Google changes and includes some 'enhancements' of my own.

It performs a single Google search, saves the results, bulk-downloads a specified number of images, then builds these into a single gallery-image using ImageMagick. Up to 1,000 images can be requested.

This bash script is available at https://git.io/googliser

Thank you.

5
  • Thanks for updating this, I will merge your answer into mine later if that's okay with you. Of course you'll get the credit you deserve!
    – ShellFish
    Apr 18, 2016 at 11:40
  • 2
    I just checked the script, you it's really nice. You should consider moving it to a public github repo though. People can then fork it and perform pull requests for possible extensions. Maybe it can even end up on public repos such as Arch's AUR! Thanks for the credit btw!
    – ShellFish
    Apr 18, 2016 at 11:44
  • 1
    Thanks ShellFish. :) Good call on GitHub. I created an account yesterday and have now uploaded the script to a public repo. I've updated the link to point to GitHub. Now - just need to learn how to use it! lol...
    – teracow
    Apr 20, 2016 at 20:36
  • 1
    I have adapted your code to work on Mac OSX: it mostly requires installing some GNU utilities and swapping them for the native ones in the googliser. You can find the script here.
    – dardo82
    Feb 13, 2017 at 21:36
  • Thanks! It lets me find portraits using OpenCV and NodeJS in the script here.
    – dardo82
    Feb 17, 2017 at 19:33
1

Python code to download high resolution images from Google. i had poste the original answer here Python - Download Images from google Image search?

Currently downloads 100 original images given a search query

Code

from bs4 import BeautifulSoup
import requests
import re
import urllib2
import os
import cookielib
import json

def get_soup(url,header):
    return BeautifulSoup(urllib2.urlopen(urllib2.Request(url,headers=header)))


query = raw_input("query image")# you can change the query for the image  here
image_type="ActiOn"
query= query.split()
query='+'.join(query)
url="https://www.google.co.in/search?q="+query+"&source=lnms&tbm=isch"
print url
#add the directory for your image here
DIR="C:\\Users\\Rishabh\\Pictures\\"+query.split('+')[0]+"\\"
header={'User-Agent':"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.134 Safari/537.36"
}
soup = get_soup(url,header)


ActualImages=[]# contains the link for Large original images, type of  image
for a in soup.find_all("div",{"class":"rg_meta"}):
    link , Type =json.loads(a.text)["ou"]  ,json.loads(a.text)["ity"]
    ActualImages.append((link,Type))

print  "there are total" , len(ActualImages),"images"


###print images
for i , (img , Type) in enumerate( ActualImages):
    try:
        req = urllib2.Request(img, headers={'User-Agent' : header})
        raw_img = urllib2.urlopen(req).read()
        if not os.path.exists(DIR):
            os.mkdir(DIR)
        cntr = len([i for i in os.listdir(DIR) if image_type in i]) + 1
        print cntr
        if len(Type)==0:
            f = open(DIR + image_type + "_"+ str(cntr)+".jpg", 'wb')
        else :
            f = open(DIR + image_type + "_"+ str(cntr)+"."+Type, 'wb')


        f.write(raw_img)
        f.close()
    except Exception as e:
        print "could not load : "+img
        print e
0

as for shelfish's answer

imagelink=$(wget -e robots=off --user-agent "$useragent" -qO - "$link" | sed 's/\"ou\"/\n\"ou\"/g' | grep '\"ou\"\:\".*\(png\|jpg\|jpeg\).*ow\"' | awk -F'"' '{print $4}' | head -n $count|tail -n1)

will work with the current google image search june 2016

0

Simple solution, for files < 4 MB only (otherwise you get a TLS error):

wget --user-agent "Mozilla/5.0" -qO - "$@" |grep video.googleusercontent.com|cut -d'"' -f2|wget --content-disposition -c -i -

Not the answer you're looking for? Browse other questions tagged or ask your own question.