Are there any gems able to parse XLS and XLSX files? I've found Spreadsheet and ParseExcel, but they both don't understand XLSX format.
10 Answers
I recently needed to parse some Excel files with Ruby. The abundance of libraries and options turned out to be confusing, so I wrote a blog post about it.
Here is a table of different Ruby libraries and what they support:
If you care about performance, here is how the xlsx
libraries compare:
I have sample code to read xlsx files with each supported library here
Here are some examples for reading xlsx
files with some different libraries:
rubyXL
require 'rubyXL'
workbook = RubyXL::Parser.parse './sample_excel_files/xlsx_500_rows.xlsx'
worksheets = workbook.worksheets
puts "Found #{worksheets.count} worksheets"
worksheets.each do |worksheet|
puts "Reading: #{worksheet.sheet_name}"
num_rows = 0
worksheet.each do |row|
row_cells = row.cells.map{ |cell| cell.value }
num_rows += 1
end
puts "Read #{num_rows} rows"
end
roo
require 'roo'
workbook = Roo::Spreadsheet.open './sample_excel_files/xlsx_500_rows.xlsx'
worksheets = workbook.sheets
puts "Found #{worksheets.count} worksheets"
worksheets.each do |worksheet|
puts "Reading: #{worksheet}"
num_rows = 0
workbook.sheet(worksheet).each_row_streaming do |row|
row_cells = row.map { |cell| cell.value }
num_rows += 1
end
puts "Read #{num_rows} rows"
end
creek
require 'creek'
workbook = Creek::Book.new './sample_excel_files/xlsx_500_rows.xlsx'
worksheets = workbook.sheets
puts "Found #{worksheets.count} worksheets"
worksheets.each do |worksheet|
puts "Reading: #{worksheet.name}"
num_rows = 0
worksheet.rows.each do |row|
row_cells = row.values
num_rows += 1
end
puts "Read #{num_rows} rows"
end
simple_xlsx_reader
require 'simple_xlsx_reader'
workbook = SimpleXlsxReader.open './sample_excel_files/xlsx_500000_rows.xlsx'
worksheets = workbook.sheets
puts "Found #{worksheets.count} worksheets"
worksheets.each do |worksheet|
puts "Reading: #{worksheet.name}"
num_rows = 0
worksheet.rows.each do |row|
row_cells = row
num_rows += 1
end
puts "Read #{num_rows} rows"
end
Here is an example of reading a legacy xls
file using the spreadsheet
library:
spreadsheet
require 'spreadsheet'
# Note: spreadsheet only supports .xls files (not .xlsx)
workbook = Spreadsheet.open './sample_excel_files/xls_500_rows.xls'
worksheets = workbook.worksheets
puts "Found #{worksheets.count} worksheets"
worksheets.each do |worksheet|
puts "Reading: #{worksheet.name}"
num_rows = 0
worksheet.rows.each do |row|
row_cells = row.to_a.map{ |v| v.methods.include?(:value) ? v.value : v }
num_rows += 1
end
puts "Read #{num_rows} rows"
end
-
1This was a great post and I upvoted, but unfortunately I found that neither roo nor spreadsheet worked with my .xls data.– guero64Jul 13, 2017 at 22:29
-
2Thnks @guero64 The xls functionality for roo is actually kept in a different project called roo-xls github.com/roo-rb/roo-xls. Did you try that library? Jul 14, 2017 at 13:08
-
4I found the problem. The source that was generating the files was saving them as .xls, but the content was HTML. Thanks for your input, though.– guero64Jul 14, 2017 at 15:03
-
Did you find any that did a reasonable job with defined names as ranges? E.g. with openpyxl: gist.github.com/empiricalthought/… Dec 21, 2017 at 21:27
-
Great work of comparision ! I am trying to use roo and I was wandering if you managed to extract comments from a cell, and how to achieve this with the provided function ? Apr 12, 2018 at 8:19
Just found roo, that might do the job - works for my requirements, reading a basic spreadsheet.
-
12roo certainly works, but it's frustratingly un-Ruby-like and (for me, anyway) very surprising: not being able to iterate over rows using each? not being able to iterate over sheets? a notion of a "default sheet" followed by access to cells through the workbook object? Jan 30, 2012 at 22:19
-
7It took me a while to find, but this now-official roo fork, which you have to explicitly pin, fixes my complaints about roo. It has #each, #to_a, reasonable sheet access, and doesn't pollute the global namespace with
Spreadsheet
by requiring ruby-spreadsheet.– WoahdaeJan 9, 2013 at 22:46 -
@woahdae Awesome! It'd be great to see an example with these new features. Is there any documentation available? I'm specifically interested in being able to iterate through each row of each worksheet of a workbook.– AnconiaJan 12, 2013 at 15:40
-
The README of that fork has an extra section about what is new in the fork. However, after implementing an xlsx upload requiring good typecasting, I found that roo typecasting had much to be desired. It choked when trying to parse "2" (formatted as a number) as a date. I wrote my own parser that I like much better, I'll upload it to github tonight and get back to you.– WoahdaeJan 14, 2013 at 21:59
-
@woahdae Good stuff. Looking forward to seeing your work. Please send the link when you can.– AnconiaJan 15, 2013 at 14:04
The roo gem works great for Excel (.xls and .xlsx) and it's being actively developed.
I agree the syntax is not great nor ruby-like. But that can be easily achieved with something like:
class Spreadsheet
def initialize(file_path)
@xls = Roo::Spreadsheet.open(file_path)
end
def each_sheet
@xls.sheets.each do |sheet|
@xls.default_sheet = sheet
yield sheet
end
end
def each_row
0.upto(@xls.last_row) do |index|
yield @xls.row(index)
end
end
def each_column
0.upto(@xls.last_column) do |index|
yield @xls.column(index)
end
end
end
-
2Careful with this naming convention - Spreadsheet is an existing constant referring to a module:
Spreadsheet.class # => Module
Renaming the class to something like "Roobook" solves this issue. However, great work!– AnconiaDec 30, 2012 at 18:10 -
2Latest roo (on the empact fork you point to) doesn't pollute the namespace, and comes with #each and such. Finally! yay empact.– WoahdaeJan 9, 2013 at 22:47
-
1Roo gem is terrible with large files. Opening a 5MB XLSx file can take 30-60 seconds which is just does not make any sense. Jan 9, 2014 at 14:53
-
1Roo takes a long time because it has to load everything into memory. It also seems to parse the spreadsheet into a usable data structure which can be slow. Sep 16, 2014 at 21:39
-
Please where in a rails project can such a file be kept @Bruno Buccolo. Jan 19, 2018 at 11:40
I'm using creek which uses nokogiri. It is fast. Used 8.3 seconds on a 21x11250 xlsx table on my Macbook Air. Got it to work on ruby 1.9.3+. The output format for each row is a hash of row and column name to cell content: {"A1"=>"a cell", "B1"=>"another cell"} The hash makes no guarantee that the keys will be in the original column order. https://github.com/pythonicrubyist/creek
dullard is another great one that uses nokogiri. It is super fast. Used 6.7 seconds on a 21x11250 xlsx table on my Macbook Air. Got it to work on ruby 2.0.0+. The output format for each row is an array: ["a cell", "another cell"] https://github.com/thirtyseven/dullard
simple_xlsx_reader which has been mentioned is great, a bit slow. Used 91 seconds on a 21x11250 xlsx table on my Macbook Air. Got it to work on ruby 1.9.3+. The output format for each row is an array: ["a cell", "another cell"] https://github.com/woahdae/simple_xlsx_reader
Another interesting one is oxcelix. It uses ox's SAX parser which supposedly faster than both nokogiri's DOM and SAX parser. It supposedly outputs a Matrix. I could not get it to work. Also, there were some dependency issues with rubyzip. Would not recommend it.
In conclusion, creek seems like a good choice. Other posts recommend simple_xlsx_parser as it has similar performance.
Removed dullard as recommended as it's outdated and people are getting errors/having problems with it.
-
2
-
2Thank you for sharing. I found that streaming 100K+ rows from an XLSX file fast and memory efficient using the Dullard gem.– scarver2Mar 24, 2015 at 5:50
-
1
dullard
was full of errors for me (with non latin data).creek
gave what i need– oklivApr 6, 2015 at 22:47 -
okliv, it would be awesome if you could specify which typesets did not work with dullard here. Also, shoot of a post to the dullard issue tracker on github! :)– frediyApr 8, 2015 at 0:37
-
1This answer mentions only reading 'xlsx' files, what about 'xls' files ? Aug 28, 2015 at 9:45
If you're looking for more modern libraries, take a look at Spreadsheet: http://spreadsheet.rubyforge.org/GUIDE_txt.html. I can't tell if it supports XLSX files, but considering that it is actively developed, I'm guessing it does (I'm not on Windows, or with Office, so I can't test).
At this point, it looks like roo is a good option again. It supports XLSX, allows (some) iteration by just using times
with cell access. I admit, it's not pretty though.
Also, RubyXL can now give you a sort of iteration using their extract_data
method, which gives you a 2d array of data, which can be easily iterated over.
Alternatively, if you're trying to work with XLSX files on Windows, you can use Ruby's Win32OLE library that allows you to interface with OLE objects, like the ones provided by Word and Excel. However, as @PanagiotisKanavos mentioned in the comments, this has a few major drawbacks:
- Excel must be installed
- A new Excel instance is started for each document
- Memory and other resource consumption is far more than what is necessary for simple XLSX document manipulation.
But if you choose to use it, you can choose not to display Excel, load your XLSX file, and access it through it. I'm not sure if it supports iteration, however, I don't think it would be too hard to build around the supplied methods, as it is the full Microsoft OLE API for Excel. Here's the documentation: http://support.microsoft.com/kb/222101 Here's the gem: http://www.ruby-doc.org/stdlib-1.9.3/libdoc/win32ole/rdoc/WIN32OLE.html
Again, the options don't look much better, but there isn't much else out there, I'm afraid. it's hard to parse a file format that is a black box. And those few who managed to break it didn't do it that visibly. Google Docs is closed source, and LibreOffice is thousands of lines of harry C++.
-
Very helpful info! I'm currently building an excel crawler and having fits with it (stackoverflow.com/questions/14044357/…). I've given up on roo as iteration is rather painful. However, I'm anxious to try
extract_data
with RubyXL.– AnconiaDec 26, 2012 at 20:44 -
2Using OLE for XLSX is a bad idea - XLSX is just compressed XML with a well known format.. It is definitely not a black box - the Open XML format is very well defined, the Open XML SDK provides all the information required to create the XML by hand and there are a lot of libraries that greatly simplify working with XLSX. Jan 5, 2016 at 8:50
-
@PanagiotisKanavos: Interesting point. While I certainly see why that would be better, is there any reason (again, out of curiosity) why using OLE is so bad? I haven't used or developed for Windows in a few years, so I might be missing something obvious.– LinuxiosJan 5, 2016 at 16:10
-
What you call OLE is the Automation interface for Excel - it requires Excel to be installed on the server and actually starts it up for each file request. It's slow - every call is am out-of-process call to Excel. It's also dangerous, as forgetting to close an instance means an Excel instance will stay in memory. This can quickly eat up server resources. In fact, XLSX was created so that any application could create a valid Excel file without requiring Excel on the server. The overhead is minimal - it's just XML processing Jan 5, 2016 at 16:22
-
@PanagiotisKanavos: Right. I forgot that OLE was more IPC than a library. Thanks! I'll add a note to the answer.– LinuxiosJan 5, 2016 at 16:23
I've been working heavily with both Spreadsheet and rubyXL these past couple weeks and I must say that both are great tools. However, one area that both suffer is the lack of examples on actually implementing anything useful. Currently I'm building a crawler and using rubyXL to parse xlsx files and Spreadsheet for anything xls. I hope the code below can serve as a helpful example and show just how effective these tools can be.
require 'find'
require 'rubyXL'
count = 0
Find.find('/Users/Anconia/crawler/') do |file| # begin iteration of each file of a specified directory
if file =~ /\b.xlsx$\b/ # check if file is xlsx format
workbook = RubyXL::Parser.parse(file).worksheets # creates an object containing all worksheets of an excel workbook
workbook.each do |worksheet| # begin iteration over each worksheet
data = worksheet.extract_data.to_s # extract data of a given worksheet - must be converted to a string in order to match a regex
if data =~ /regex/
puts file
count += 1
end
end
end
end
puts "#{count} files were found"
require 'find'
require 'spreadsheet'
Spreadsheet.client_encoding = 'UTF-8'
count = 0
Find.find('/Users/Anconia/crawler/') do |file| # begin iteration of each file of a specified directory
if file =~ /\b.xls$\b/ # check if a given file is xls format
workbook = Spreadsheet.open(file).worksheets # creates an object containing all worksheets of an excel workbook
workbook.each do |worksheet| # begin iteration over each worksheet
worksheet.each do |row| # begin iteration over each row of a worksheet
if row.to_s =~ /regex/ # rows must be converted to strings in order to match the regex
puts file
count += 1
end
end
end
end
end
puts "#{count} files were found"
The rubyXL gem parses XLSX files beautifully.
-
6rubyXL (like roo, above) also gets weird when you actually get around to accessing the data in a worksheet. Is there something fundamental about the data model for a spreadsheet that iteration over rows and columns cannot be simply provided? Jan 30, 2012 at 22:25
-
5
I couldn't find a satisfactory xlsx parser. RubyXL doesn't do date typecasting, Roo tried to typecast a number as a date, and both are a mess both in api and code.
So, I wrote simple_xlsx_reader. You'd have to use something else for xls, though, so maybe it's not the full answer you're looking for.
Most of the online examples including the author's website for the Spreadsheet gem demonstrate reading the entire contents of an Excel file into RAM. That's fine if your spreadsheet is small.
xls = Spreadsheet.open(file_path)
For anyone working with very large files, a better way is to stream-read the contents of the file. The Spreadsheet gem supports this--albeit not well documented at this time (circa 3/2015).
Spreadsheet.open(file_path).worksheets.first.rows do |row|
# do something with the array of CSV data
end
The RemoteTable library uses roo internally. It makes it easy to read spreadsheets of different formats (XLS, XLSX, CSV, etc. possibly remote, possibly stored inside a zip, gz, etc.):
require 'remote_table'
r = RemoteTable.new 'http://www.fueleconomy.gov/FEG/epadata/02data.zip', :filename => 'guide_jan28.xls'
r.each do |row|
puts row.inspect
end
Output:
{"Class"=>"TWO SEATERS", "Manufacturer"=>"ACURA", "carline name"=>"NSX", "displ"=>"3.0", "cyl"=>"6.0", "trans"=>"Auto(S4)", "drv"=>"R", "bidx"=>"60.0", "cty"=>"17.0", "hwy"=>"24.0", "cmb"=>"20.0", "ucty"=>"19.1342", "uhwy"=>"30.2", "ucmb"=>"22.9121", "fl"=>"P", "G"=>"", "T"=>"", "S"=>"", "2pv"=>"", "2lv"=>"", "4pv"=>"", "4lv"=>"", "hpv"=>"", "hlv"=>"", "fcost"=>"1238.0", "eng dscr"=>"DOHC-VTEC", "trans dscr"=>"2MODE", "vpc"=>"4.0", "cls"=>"1.0"}
{"Class"=>"TWO SEATERS", "Manufacturer"=>"ACURA", "carline name"=>"NSX", "displ"=>"3.2", "cyl"=>"6.0", "trans"=>"Manual(M6)", "drv"=>"R", "bidx"=>"65.0", "cty"=>"17.0", "hwy"=>"24.0", "cmb"=>"19.0", "ucty"=>"18.7", "uhwy"=>"30.4", "ucmb"=>"22.6171", "fl"=>"P", "G"=>"", "T"=>"", "S"=>"", "2pv"=>"", "2lv"=>"", "4pv"=>"", "4lv"=>"", "hpv"=>"", "hlv"=>"", "fcost"=>"1302.0", "eng dscr"=>"DOHC-VTEC", "trans dscr"=>"", "vpc"=>"4.0", "cls"=>"1.0"}
{"Class"=>"TWO SEATERS", "Manufacturer"=>"ASTON MARTIN", "carline name"=>"ASTON MARTIN VANQUISH", "displ"=>"5.9", "cyl"=>"12.0", "trans"=>"Auto(S6)", "drv"=>"R", "bidx"=>"1.0", "cty"=>"12.0", "hwy"=>"19.0", "cmb"=>"14.0", "ucty"=>"13.55", "uhwy"=>"24.7", "ucmb"=>"17.015", "fl"=>"P", "G"=>"G", "T"=>"", "S"=>"", "2pv"=>"", "2lv"=>"", "4pv"=>"", "4lv"=>"", "hpv"=>"", "hlv"=>"", "fcost"=>"1651.0", "eng dscr"=>"GUZZLER", "trans dscr"=>"CLKUP", "vpc"=>"4.0", "cls"=>"1.0"}