-3

I have the following line that I scrape directly off an html page. I want to process it to extract information from it:

var quoteDataObj = [{"symbol":".DJIA","symbolType":"symbol","code":0,"name":"Dow Jones Industrial Average","shortName":"DJIA","last":"15794.08","exchange":"Dow Jones Global Indexes","source":"Exchange","open":"15630.64","high":"15798.51","low":"15625.53","change":"165.55","currencyCode":"USD","timeZone":"EST","provider":"CNBC Quote Cache","altSymbol":".DJIA","curmktstatus":"REG_MKT","realTime":"true","assetType":"INDEX","noStreaming":"true","encodedSymbol":".DJIA"}]

I am using python to process the string above which I've saved into the variable "line". . I'm trying to write a regular expression that gets

"low":"15625.53"

However, I don't know what the number will be so I can't just search that. I've tried the following but with no luck:

last = re.search(r".*last\":.*\,", line)

Thanks!

3
  • What is line? What's this var keyword supposed to do? Are you sure you're using Python?
    – Wooble
    Feb 9, 2014 at 4:22
  • 2
    strip off that javascript, throw the rest in json.loads.
    – roippi
    Feb 9, 2014 at 4:25
  • Please see my revised edits. Feb 9, 2014 at 4:26

3 Answers 3

2

r'"low":".*?"' should work for you.

>>> re.search(r'"low":".*?"', text).group()
'"low":"15625.53"'
2
cruft,sep,payload = s.partition(' = ')

from ast import literal_eval # or 100% equivalent for this purpose, json.loads

d_in_list = literal_eval(payload)

Then you just have a regular dict in a list.

d_in_list[0]
Out[15]: 
{'altSymbol': '.DJIA',
 'assetType': 'INDEX',
 'change': '165.55',
 'code': 0,
 'curmktstatus': 'REG_MKT',
 'currencyCode': 'USD',
 'encodedSymbol': '.DJIA',
 'exchange': 'Dow Jones Global Indexes',
 'high': '15798.51',
 'last': '15794.08',
 'low': '15625.53',
 'name': 'Dow Jones Industrial Average',
 'noStreaming': 'true',
 'open': '15630.64',
 'provider': 'CNBC Quote Cache',
 'realTime': 'true',
 'shortName': 'DJIA',
 'source': 'Exchange',
 'symbol': '.DJIA',
 'symbolType': 'symbol',
 'timeZone': 'EST'}

d_in_list[0]['low']
Out[16]: '15625.53'

Though really there is a 99% chance that there exists an actual API where you can submit a query and get the above json response back without having to scrape a webpage and do janky parsing.

1
  • This works great as well, unfortunately, i couldn't find an API but for something this small, it works either way. Thanks for pointing me in this direction too! Feb 9, 2014 at 4:44
0

Another way is to strip the var ... prefix and deal with the string as a JSON object:

>>> import json
>>> data = 'var quoteDataObj = [{"symbol":".DJIA","symbolType":"symbol","code":0,"name":"Dow Jones Industrial Average","shortName":"DJIA","last":"15794.08","exchange":"Dow Jones Global Indexes","source":"Exchange","open":"15630.64","high":"15798.51","low":"15625.53","change":"165.55","currencyCode":"USD","timeZone":"EST","provider":"CNBC Quote Cache","altSymbol":".DJIA","curmktstatus":"REG_MKT","realTime":"true","assetType":"INDEX","noStreaming":"true","encodedSymbol":".DJIA"}]'
>>> json.loads(data[data.find('['):])
[{u'altSymbol': u'.DJIA', u'code': 0, u'last': u'15794.08', u'name': u'Dow Jones Industrial Average', u'noStreaming': u'true', u'exchange': u'Dow Jones Global Indexes', u'assetType': u'INDEX', u'symbol': u'.DJIA', u'realTime': u'true', u'symbolType': u'symbol', u'high': u'15798.51', u'source': u'Exchange', u'encodedSymbol': u'.DJIA', u'low': u'15625.53', u'provider': u'CNBC Quote Cache', u'curmktstatus': u'REG_MKT', u'timeZone': u'EST', u'shortName': u'DJIA', u'open': u'15630.64', u'currencyCode': u'USD', u'change': u'165.55'}]
>>> json_str = data[data.find('['):]  # Take everything from the first [ in the string
>>> json.loads(json_str)[0]['low']
u'15625.53'

You can retrieve every attributes from your array then.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.