8

I'm coding a scraper like the Facebook one to get info from a given url. Im just one step to finish the basic functionality. The problem so far is to remove useless images. For example, when i run some random url, i get all this images:

Scraper Object
(
    [url] => http://buzz.money.cnn.com/2012/07/23/spain-italy-short-selling/?iid=HP_Highlight
    [title] => Spain and Italy ban short selling  - The Buzz  - Investment and Stock Market News
    [description] => The Euronext 100 stock index falls sharply on renewed concerns about Spain.    Securities regulators in Spain and Italy both instituted short-selling bans Monday as financial markets tumbled.    The move is designed to limit the downward pressure
    [imageUrls] => Array
        (
            [0] => http://cnnmoneybuzzblog.files.wordpress.com/2012/07/chart_ws_index_euronext100_201272310932-09.png
            [1] => http://i.cdn.turner.com/money/.e1m/img/5.0/data/feargreed/scale.316x95.png
            [2] => http://i2.cdn.turner.com/money/.element/img/5.0/sections/contributors/ben_rooney_130.jpg
            [3] => http://i2.cdn.turner.com/money/.element/img/5.0/sections/contributors/catherine_tymkiw.02.jpg
            [4] => http://i2.cdn.turner.com/money/.element/img/5.0/sections/contributors/paul_lamonica.02.jpg
            [5] => http://i2.cdn.turner.com/money/.element/img/5.0/sections/contributors/hibah_yousuf.02.jpg
            [6] => http://i2.cdn.turner.com/money/.element/img/5.0/sections/contributors/maureen_farrell.02.jpg
            [7] => http://i2.cdn.turner.com/money/.element/img/5.0/sections/contributors/ben_rooney.02.jpg
            [8] => http://i.cdn.turner.com/money/.element/img/4.0/services/button_login.gif
            [9] => http://www.bizographics.com/collect/?fmt=gif&pid=311
            [10] => http://pixel.quantserve.com/pixel/p-5dyPa639IrgIw.gif
            [11] => http://i.cdn.turner.com/money/.element/img/1.0/misc/1.gif
            [12] => http://buzz.money.cnn.com/2012/07/23/spain-italy-short-selling/?iid=HP_Highlight//pixel.quantserve.com/pixel/p-18-mFEk4J448M.gif?labels=%2Clanguage.en%2Ctype.wpcom%2Cposttag.bonds%2Cposttag.dow%2Cposttag.ibex%2Cposttag.italy%2Cposttag.lehman%2Cposttag.milan%2Cposttag.nasdaq%2Cposttag.sp-500%2Cposttag.short-selling%2Cposttag.spain%2Cposttag.stock%2Cposttag.yields%2Cvip.cnnmoneybuzzblog
            [13] => http://stats.wordpress.com/b.gif?v=noscript
        )

)

I just need to find a way to remove all those images ending in .gif or .png and just let the .jpg inside the array, so the user can take a look and select proper one for the article.

I've tried some array functions but i think this need some regex magic for it to work in almost any url given.

P.S. I can access all array data using $info->url, $info->description and so. Just need to filter that array and it will be ready.

2
  • Off topic: CNN Money Buzz Blog shows up in a lot of the spam I get... Jul 24, 2012 at 1:25
  • I had a customer who loves linking to Money CNN. I just made my part coding the scraper :P
    – Richard-MX
    Jul 24, 2012 at 3:59

3 Answers 3

5

Without using regex:

foreach ($objects_array as $obj){
  foreach($obj->imageUrls as $key => $img){
    if(substr($img, -4) === '.gif' || substr($img, -4) === '.png'){
      unset($key);
    }
  }
}

and using regex:

$pt = '/gif$|png$/';
foreach ($objects_array as $obj){
  foreach($obj->imageUrls as $key => $img){
    $res = preg_match($pt, $img);
    if($res){
      unset($key);
    }
  }
}
3
  • You going to hate me my friend, but, it doesn't show any errors but it still shows all the image files including gifs and pngs. Actually, i removed the first foreach because there is only one object. $info only gathers info from one url. I have $info->url, $info->title, $info->description and the array $info->imageUrls. What could be wrong?
    – Richard-MX
    Jul 24, 2012 at 3:22
  • @Richard-MX of course I'm not going to hate you ;) a quick guess (since I don't have you code) try to replace: unset($key); with: unset($info->imageUrls[$key]);
    – Nir Alfasi
    Jul 24, 2012 at 3:37
  • It's working! Thanks amigo :), if you came to Cancun one day (yes, there are programmers here too :P) drinks are on me!
    – Richard-MX
    Jul 24, 2012 at 3:57
3

Try array_filter:

http://php.net/manual/en/function.array-filter.php

The callback function should be a very basic function that returns false if your regex matches .png or .gif.

0
0

You could filter after the array is built:

function only_jpgs($url){
    return substr($url, -4) == '.jpg' ? $url : false;
}

$info->imageUrls = array_filter($info->imageUrls, "only_jpgs");

But you probably should filter out the images before you build the array.

2
  • I can't get this one to work my friend. When you call "only_jpgs" from the array_filter, how does the function now what's the $url? Im getting this error array_filter() expects parameter 2 to be a valid callback, function 'only_jpgs' not found or invalid function name using it like that. Thanks.
    – Richard-MX
    Jul 24, 2012 at 3:35
  • 1
    @Richard-MX If you can't cut and paste I can't help you. Jul 24, 2012 at 4:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.