78

I am having the following problem of mapping documents within a YAML file to a dict and properly mapping them.

I have the following YAML file, which represents a server (db.yml):

instanceId: i-aaaaaaaa
     environment:us-east
     serverId:someServer
     awsHostname:ip-someip
     serverName:somewebsite.com
     ipAddr:192.168.0.1
     roles:[webserver,php]

I load this YAML file, which I can do without any problems, I think I understand that.

instanceId = getInstanceId()
stream = file('db.yml', 'r')
dict = yaml.load_all(stream)

for key in dict:
    if key in dict == "instanceId":
        print key, dict[key]

I'd like the logic to work like the following:

  • load yaml, map to dict
  • look in every dict in the document, if the instanceId matches that which was set by getInstanceId(), then print out all of the keys and values for that document.

If I look at the map data structure from the command line, I get:

{'instanceId': 'i-aaaaaaaa environment:us-east serverId:someServer awsHostname:ip-someip serverName:someserver ipAddr:192.168.0.1 roles:[webserver,php]'}

I think I might be creating the data structure for the YAML file improperly, and on matching the contents on the dict, I am a bit lost.

Side note: I cannot load all of the documents in this file using yaml.load(), I tried yaml.load_all(), which seems to work but my main issue still exists.

3
  • 9
    Please rename dict to something else. That is bad bad. Also, can you print the value of yaml.load_all(stream) so we can see what you actually have? Lastly, it doesn't seem like you have included a good YAML sample -- where are multiple servers defined?
    – gahooa
    Oct 22, 2012 at 20:54
  • I will rename it, thank you for the best practice. yaml_load_all(stream) shows: <generator object load_all at 0x10b2648c0>. I removed the second document for troubleshooting. Oct 22, 2012 at 21:15
  • 5
    What library / module is used in the code? I think it would improve this question to add the import statement, so that people who find this question don't have to search for it and potentially find multiple possible ones. Mar 29, 2017 at 11:23

6 Answers 6

69

I think your yaml file should look like (or at least something like, so it's structured correctly anyway):

instance:
     Id: i-aaaaaaaa
     environment: us-east
     serverId: someServer
     awsHostname: ip-someip
     serverName: somewebsite.com
     ipAddr: 192.168.0.1
     roles: [webserver,php]

Then, yaml.load(...) returns:

{'instance': {'environment': 'us-east', 'roles': ['webserver', 'php'], 'awsHostname': 'ip-someip', 'serverName': 'somewebsite.com', 'ipAddr': '192.168.0.1', 'serverId': 'someServer', 'Id': 'i-aaaaaaaa'}}

And you can go from there...


So used like:

>>> for key, value in yaml.load(open('test.txt'))['instance'].iteritems():
    print key, value


environment us-east
roles ['webserver', 'php']
awsHostname ip-someip
serverName somewebsite.com
ipAddr 192.168.0.1
serverId someServer
Id i-aaaaaaaa
9
  • 1
    Hmm, I tried this but I don't get the same result using the yaml you used above: gist.github.com/3934903 Oct 22, 2012 at 22:05
  • 1
    @zippy Where is this yaml file coming from? Oct 22, 2012 at 22:10
  • 2
    Since there's no built-in module named yaml, and more than one third-party library with that name, the problem may be that you're using a different library than the one from the answer. I verified that PyYAML 3.10, both pure-Python with libyaml 0.1.4, gives the answer @JonClements shows, and so does simpleyaml 3.10. Maybe you have a broken library (which might also explain why you're trying to parse broken yaml in the first place?).
    – abarnert
    Oct 22, 2012 at 22:21
  • 1
    @zippy we're using the same version then... But colon and space is important in the format - that's worth noting Oct 22, 2012 at 22:46
  • 1
    @JonClements Ah, your last comment hit the mark on the nose. Or the nose on the mark, or whatever. Adding a space to the yaml file fixed it. key: instance value: {'environment': 'X', 'saltId': 'APPSRV-X', 'roles': ['webserver', 'php'], 'awsHostname': 'ip-someip', 'serverName': 'PHPSRV-X.example.com', 'ipAddr': '10.1.1.10', 'Id': 'i-aaaaaaaa'} Oct 22, 2012 at 22:50
24

I love to use Path, makes a beautiful one-liner

import yaml
from pathlib import Path
conf = yaml.safe_load(Path('data.yml').read_text())
8

An additional bug in your code, that doesn't have to do with YAML:

for key in dict:
    if key in dict == "instanceId": # This doesn't do what you want
        print key, dict[key]

in is an operator that works on sequence types, and also on maps. This is why this isn't a syntax error... but it doesn't do what you want.

key in dict will always evaluate to True, because all the keys you're iterating through are in the dict. So your code boils down to True == "instanceId", which will always evaluate to False, because the boolean value True is never equal to that string.

You might have noticed that the print statement doesn't produce any output; this is because it never gets called.

6

Just use python-benedict, it's a dict subclass that provides I/O support for most common formats, including yaml.

from benedict import benedict

# path can be a yaml string, a filepath or a remote url
path = 'path/to/data.yml'

d = benedict.from_yaml(path)

# do stuff with your dict
# ...

# write it back to disk
d.to_yaml(filepath=path)

It's well tested and documented, check the README to see all the features: https://github.com/fabiocaccamo/python-benedict

Install using pip: pip install python-benedict

Note: I am the author of this project

6

If you have reached this question trying to figure out how to get a python dictionary from a yaml file using the pyyaml library try the safe_load option as shown below.

import yaml
from pathlib import Path

yaml_dict = yaml.safe_load(Path("data.yml").read_text())
2
  • This just returns the string "data.yml", you need to supply an actual stream or yaml string
    – Mandera
    Aug 25, 2022 at 10:32
  • You're right. Fixed it now.
    – najeem
    Aug 25, 2022 at 12:08
4

You can use the bios package for Python3 like below:

import bios

my_dict = bios.read('data.yml')

bios is reading the raw data from a file and converting a python dict object. According to the extension of the file, it can figure out the file type.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.