95

When I run rake db:seed in my Rails app, I'm getting this error:

invalid byte sequence in US-ASCII (Argument Error)

I just added science_majors and down to my seed file, and now when I run rake db:seed it gives me this error:

invalid byte sequence error

Why is this, and how can I fix it?

part of seeds.rb

@college = College.find_or_create_by_name!('University of Pittsburgh')
if @college.update_attributes(
  url: 'university-of-pittsburgh',
  public: 'Public',
  years: '4-year',
  category: 'National University',
  calendar: 'Semester',
  location: 'Pittsburgh, PA',
  setting: 'Large City (250-500k)',
  retention: '90',
  majors: 'business/marketing: 15%|social sciences: 14%|health professions: 11%|english: 10%|engineering: 9%|psychology: 8%|biology: 7%|history: 5%',
  scholarships_link: 'http://www.oafa.pitt.edu/universityschlrs.aspx',
  map: '<iframe width="425" height="350" frameborder="0" scrolling="no" marginheight="0" marginwidth="0" src="https://maps.google.com/maps?f=q&amp;source=s_q&amp;hl=en&amp;geocode=&amp;q=university+of+pittsburgh&amp;aq=&amp;sll=40.444261,-79.970448&amp;sspn=0.142165,0.251656&amp;ie=UTF8&amp;hq=university+of+pittsburgh&amp;t=m&amp;ll=40.443486,-79.958245&amp;spn=0.010798,0.021038&amp;output=embed"></iframe><br /><small><a href="https://maps.google.com/maps?f=q&amp;source=embed&amp;hl=en&amp;geocode=&amp;q=university+of+pittsburgh&amp;aq=&amp;sll=40.431368,-79.9805&amp;sspn=0.142165,0.251656&amp;ie=UTF8&amp;hq=university+of+pittsburgh&amp;t=m&amp;ll=40.443486,-79.958245&amp;spn=0.010798,0.021038" style="color:#0000FF;text-align:left">View Larger Map</a></small>',
  street_view: '<iframe width="425" height="350" frameborder="0" scrolling="no" marginheight="0" marginwidth="0" src="https://maps.google.com/maps?f=q&amp;source=embed&amp;hl=en&amp;geocode=&amp;q=University+of+Pittsburgh,+4200+Fifth+Ave,+Pittsburgh,+PA&amp;aq=0&amp;oq=uni&amp;sll=40.444261,-79.970448&amp;sspn=0.142165,0.251656&amp;ie=UTF8&amp;hq=University+of+Pittsburgh,+4200+Fifth+Ave,+Pittsburgh,+PA&amp;ll=40.443486,-79.958245&amp;spn=0.010798,0.021038&amp;t=m&amp;layer=c&amp;cbll=40.441775,-79.956671&amp;panoid=BgWQKMiSI1fpwP7JUF16Vg&amp;cbp=12,45.25,,0,-3.34&amp;output=svembed"></iframe><br /><small><a href="https://maps.google.com/maps?f=q&amp;source=embed&amp;hl=en&amp;geocode=&amp;q=University+of+Pittsburgh,+4200+Fifth+Ave,+Pittsburgh,+PA&amp;aq=0&amp;oq=uni&amp;sll=40.431368,-79.9805&amp;sspn=0.142165,0.251656&amp;ie=UTF8&amp;hq=University+of+Pittsburgh,+4200+Fifth+Ave,+Pittsburgh,+PA&amp;ll=40.443486,-79.958245&amp;spn=0.010798,0.021038&amp;t=m&amp;layer=c&amp;cbll=40.441775,-79.956671&amp;panoid=BgWQKMiSI1fpwP7JUF16Vg&amp;cbp=12,45.25,,0,-3.34" style="color:#0000FF;text-align:left">View Larger Map</a></small>',
  housing: '<iframe src="http://www.tour.pitt.edu/tour-categories/residence-halls" scrolling="no" height="820" width="900"></iframe>',
  food: '<div style="overflow: hidden; max-width: 1000px;"><iframe scrolling="no" src="http://www.pc.pitt.edu/dining/menus.php" style="margin-left: 0; height: 2000px; margin-top: -370px; margin-bottom: -375px; width: 750px;"></iframe></div>',
  weather: '<div style="overflow: hidden; max-width: 500px;"><iframe scrolling="no" src="http://www.weather.com/weather/wxclimatology/monthly/graph/USPA1290" style="margin-left: -40px; height: 1570px; margin-top: -490px; width: 500px;"></iframe></div>',
  dorms: 'Towers (fr.)|Sutherland (fr.)|Lothrop (fr.)|Nordenberg (fr.)|Holland (fr.)|Forbes (fr.; honors)|McCormick|Brackenridge|Amos|Bruce|Panther|Pennsylvania|Ruskin|Bouquet Gardens|Oakwood Apartments|Centre-Plaza Apartments|Forbes-Craig Apartments (honors)',
  science_majors: 'Actuarial Mathematics|Applied Mathematics|Bioinformatics|Biological Sciences|Chemistry*|Computer Science|Ecology and Evolution|Environmental Geology|Environmental Studies|Geology|History and Philosophy of Science|Linguistics*|Mathematics*|Mathematics-Economics|Mathematics-Philosophy|Microbiology|Molecular Biology|Natural Sciences Area|Neuroscience*|Physics*|Physics and Astronomy|Scientific Computing|Statistics',
  social_science_majors: 'Economics*|Economics–Statistics|History*|Political Science*|Politics-Philosophy|Psychology|Sociology*|Urban Studies',
  humanities_majors: 'Africana Studies*|Africana Studies–English|Anthropology|Chinese*|Classics*|English Literature*|English Writing|Film Studies*|French*|German|History of Art & Architecture|Humanities Area|Italian*|Italian Studies|Japanese*|Liberal Studies|Music*|Philosophy*|Religious Studies*|Russian|Slavic Studies|Spanish|Studio Arts*|Theatre Arts*',
  business_majors: 'Accounting|Finance|General Management|Global Management|Marketing',
  engineering_majors: 'Bioengineering*|Chemical Engineering*|Civil Engineering*|Computer Engineering|Electrical Engineering*|Engineering Physics|Industrial Engineering*|Materials Science and Engineering*|Mechanical Engineering*',
  professional_majors: 'Administration of Justice*|Applied Developmental Psychology|Architectural Studies|Athletic Training|Clinical Dietetics and Nutrition|Communication: Rhet & Comm|Communication Science|Dental Hygiene|Emergency Medicine|Health Information Management|Health Services|Health and Physical Activity|Information Science|Legal Studies*|Media and Professional Communications|Nursing|Pharmacy|Public Service*|Rehabilitation Science|Social Work*'
)
else
  # handle validation errors
end

@college= College.find_or_create_by_name!('Harvard')
if @college.update_attributes(
  url: 'harvard',
  public: 'Private',
  years: '4-year',
  category: 'National University',
  calendar: 'Semester',
  location: 'Cambridge, MA',
  setting: 'Small City (50-250k)',
  degrees: 'Bachelors, Masters, Doctoral',
  rotc: 'Air Force, Army, Navy',
  considered: 'Academic GPA|Alumni Relation|Application Essay|Character/Personal Qualities|Extracurricular Activities|First Generation College Student|Geographical Residence|Interview|Racial/Ethnic Status|Recommendations|Rigor of secondary school record|Standardized Test Scores|Talent/Ability|Volunteer Work|Work Experience',
  student_faculty: '7 to 1')
else
  # handle validation errors
end

I left out all the other colleges. I have 200 schools so far, but Pitt is the only one with comprehensive data so far. The rest just have basic data, like Harvard. I also left out some of the data for Pitt for the sake of brevity, but let me know if you'd like me to include the rest.

Edit: I tried taking out the *'s in science_majors on down and then running rake db:seed, but that didn't change anything.

6 Answers 6

205

You're receiving an encoding error because your filesystem isn't configured to encode the date you've added (since presumably it includes new characters – possibly in your HTML entity encoded map URL – that didn't exist in your prior data seed).

The following will should resolve this error by setting the UTF-8 locale on your machine:

# from your command line
export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8
export LC_ALL=en_US.UTF-8
bundle

The benefit of setting a system locale is that all gems (going forward) will be bundled using UTF-8 encoding.

EDIT:

Alternatively, if you don't want to change your system encoding, you can set your encoding project-wide by specifying an encoding standard in your Gemfile:

if RUBY_VERSION =~ /1.9/ # assuming you're running Ruby ~1.9
  Encoding.default_external = Encoding::UTF_8
  Encoding.default_internal = Encoding::UTF_8
end
8
  • 1
    Glad it helped. If you take this approach, make sure that all environments that you've deployed your project to have their locales set to UTF-8. Otherwise, you'll need to add #encoding utf-8 to the top of your file to ensure that it's properly encoded, as the other poster astutely notes.
    – zeantsoi
    Commented Jun 10, 2013 at 20:10
  • i actually just tried to remove #encoding utf-8 and do what you said but it didn't work. ~/collegeanswers >> export LANG=en_US.UTF-8 ~/collegeanswers >> export LC_ALL="en_US.UTF-8" ~/collegeanswers >> bundle Using rake (10.0.4) . . Using i18n (0.6.1) Your bundle is complete! Use bundle show [gemname]` to see where a bundled gem is installed. ~/collegeanswers >> rake db:seed rake aborted!` do you know why? Commented Jun 10, 2013 at 20:15
  • What is your rake error? Your existing gems are probably not configured to work with your filesystem encoding yet. Try bbundle exec gem uninstall i18n, then bundle install.
    – zeantsoi
    Commented Jun 10, 2013 at 20:25
  • my rake error was invalid byte sequence in US-ASCII (ArgumentError) . I just tried what you said and got the same error. Commented Jun 10, 2013 at 20:33
  • Did you restart your webserver? Updated my answer to provide more options for accomplishing what you're trying to achieve.
    – zeantsoi
    Commented Jun 10, 2013 at 20:34
66

Add

#encoding: utf-8

at the top of the file

4
  • Why do I need to do that now, but before I didn't have to? Commented Jun 10, 2013 at 19:54
  • I just tried it and it worked btw, so thanks! But I'd still like to know why everything was working before, and now I need to add that for it to work. Commented Jun 10, 2013 at 19:56
  • 1
    Maybe you added some special utf-8 signs in your file.. :) Commented Jun 10, 2013 at 19:57
  • I just read through the updates to the other guy's answer and it is actually more comprehensive so I'm gonna mark that as the accepted answer, but I still really appreciate your help though! Commented Jun 10, 2013 at 20:04
41

You can also try export RUBYOPT="-KU -E utf-8:utf-8" as mentioned in this GH thread

1
  • Thanks! That's the only thing that helped with shotgun-0.9.1/bin/shotgun:94:in `[]': invalid byte sequence in US-ASCII (ArgumentError)
    – Slava V
    Commented Aug 30, 2016 at 17:45
23

If you are having this problem in a ruby docker container you might either set LANG=C.UTF-8:

docker run -it --rm -e LANG=C.UTF-8 ruby ...

or watch this issue.

1
  • or as seen in the link in this answer, if creating your own Docker image, add 'ENV LANG=C.UTF-8' to your Dockerfile
    – Alex
    Commented Sep 1, 2020 at 13:24
7

Like others suggested, I added #encoding: utf-8 to the top of my (in this case, seed.rb) file, but still couldn't get the rake task to work.

Interestingly enough, a friend recommended that I remove the additional, rails-generated comments from the top of the seed file so that only #encoding: utf-8 remained.

What would you know - it worked.

1

There is a invalid character in your seed file, in my case changing single quotes from ` to ' worked out.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.