1
http://amz.com/New-Balance-WT910-Trail-Running/dp/B0098FOFCW/ref=zg_bsms_shoes_2

I don't need the last /ref=zg_bsms_shoes_2

I have the values in the urls=[]

for productlink in products:
    self.urls.append(productlink)

def save(self):
    self.br.quit()
    f=open(self.product_file,"w")
    for url in self.urls:
        f.write(url+"\n")
        f.flush()

How to strip it? Also with a fail proof if I don't have /ref=?

2
  • You may use regular expression to strip it.
    – Zhou Tai
    Jan 4, 2014 at 2:11
  • @IdleMind I think url.replace is better choice less resources :) but still I'm not sure how to do it correctly I'm new to python
    – Timy
    Jan 4, 2014 at 2:12

3 Answers 3

2

I'd strongly encourage you to start with urlparse:

In python3:

>>> import os
>>> from urllib.parse import urlparse
>>> os.path.split(urlparse(url).path)[0]
'/New-Balance-WT910-Trail-Running/dp/B0098FOFCW'

urlparse will turn the URL into all its component pieces and then you can work with the path in any number of ways, simple string splitting, os.path.split, regex, whatever you like.

In Python2 just use from urlparse import urlparse

1
>>> x = 'http://amz.com/New-Balance-WT910-Trail-Running/dp/B0098FOFCW/ref=zg_bsms_shoes_2'
>>> '/'.join(x.split('/')[:6])
'http://amz.com/New-Balance-WT910-Trail-Running/dp/B0098FOFCW'
>>> y = 'http://amz.com/New-Balance-WT910-Trail-Running/dp/B0098FOFCW'
>>> '/'.join(y.split('/')[:6])
'http://amz.com/New-Balance-WT910-Trail-Running/dp/B0098FOFCW'
5
  • 1
    '/'.join(x.split('/')[:-1]) might be preferable Jan 4, 2014 at 2:13
  • @Guy can you explain [:6]? what about the scenario of: 'http://amz.com/New-Balance-WT910-Trail-Running/dp/B0098FOFCW/' "/" at the end
    – Timy
    Jan 4, 2014 at 2:42
  • [:6] will simply take first 6 elements from the list, will always return without last / do you ask how to add it? Jan 4, 2014 at 2:47
  • @Guy weird but trying it for productlink in products: '/'.join(productlink.split('/')[:6]) self.urls.append(productlink) didnt seem to work
    – Timy
    Jan 4, 2014 at 2:51
  • for productlink in products: x = '/'.join(productlink.split('/')[:6]) self.urls.append(x) Jan 4, 2014 at 2:53
1
if 'ref' in url.split('/')[-1]: #Failsafe
    url = '/'.join(url.split('/')[:-1]
3
  • a[-1] # last item in the array a[:-1] # everything except the last item
    – alcio
    Jan 4, 2014 at 2:21
  • for url in self.urls: if 'ref' in url.split('/')[-1]: #Failsafe url = '/'.join(url.split('/')[:-1] f.write(url+"\n") f.flush() f.write(url+"\n") ^ SyntaxError: invalid syntax any idea why?
    – Timy
    Jan 4, 2014 at 2:33
  • Under which line and character is the ^ exactly ?
    – alcio
    Jan 4, 2014 at 2:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.