Questions tagged [fastercsv]

Popular Ruby library/gem for working with CSV format files.

fastercsv
Filter by
Sorted by
Tagged with
31 votes
4 answers
43k views

How to write columns header to a csv file with Ruby?

I am having trouble writing columns to a csv file with Ruby. Below is my snippet of code. calc = numerator/denominator.to_f data_out = "#{numerator}, #{denominator}, #{calc}" File.open('cdhu3_X....
2 votes
2 answers
1k views

Ruby parallel csv importing

I'm importing huge csv file, and i want to split it so, that importing will be faster (i didn't import directly to db, i have some calculation). code looks like this: def import_shatem require '...
24 votes
9 answers
40k views

CSV - Unquoted fields do not allow \r or \n (line 2)

Trying to parse a CSV file, but still getting the error message Unquoted fields do not allow \r or \n (line 2).. I found here at SO similar topic, where was a hint to do following: CSV.open('file....
0 votes
1 answer
128 views

Need to make search in PowerShell script faster

Who can help to make this search faster? With this code the search takes a few days. Search_Names.csv (about 10k names) Need_This_Long_Strings.csv (about 180k strings and it's 50MB) $TimeStamp = Get-...
3 votes
5 answers
7k views

Heroku file upload problem

I've been having a problem uploading a CSV file to Heroku and processing it. It works fine in my local environment. Just do be clear, I don't need to save the file on Heroku, just access it during the ...
3 votes
1 answer
582 views

How to prevent Rails from encoding entities in FasterCSV output

I'm using FasterCSV to produce CSV output of my reports in a Rails 3 application. Here's a code snippet: <%= FasterCSV.generate do |csv| @groups.each do |b| record = [ b.group, b....
4 votes
2 answers
5k views

Generating fields containing newline with Ruby CSV::Writer

I want to make CSV::Writer generate a line break within a quoted string: A,B,"Line Line",C So that the row would display in Excel as: A,B,Line,C Line Is it possible to prevent CSV:Writer from ...
1 vote
1 answer
884 views

Rails: FasterCSV - Unique Occurrences

I have my CSV file imported as such: records = FasterCSV.read(path, :headers => true, :header_converters => :symbol) How can I get the unique occurrences of my data? For instance, here some ...
2 votes
3 answers
4k views

merge CSV files on a common field with ruby/fastercsv

I have a 'master' file with a number of columns: 1 2 3 4 5. I have a few other files, with fewer rows than the master file, each with columns: 1 6. I'd like to merge these files matching on the column ...
2 votes
3 answers
4k views

Import CSV in batches of lines in Rails?

I'm using FasterCSV to import an uploaded file to a model, and it's working great for small files. However when I try to import a large dataset (21,000 lines) it takes ages and I get browser timeouts ...
10 votes
2 answers
3k views

Custom Ruby CSV converters for multiple fields

I am importing a CSV file that contains fields that need to be converted as follows: "True" (string) -> true (boolean) "False" (string) -> false (boolean) "%m/%d/%Y" (string format) -> Date object "%...
9 votes
4 answers
9k views

in rails, unable to override content-disposition filename

I am using Rails 3.1.0.rc8 and Chromium 15.0.874.102. I want to set the filename of a CSV download. I am following this SO solution, but find myself unable to modify the filename of the Content-...
3 votes
3 answers
9k views

How do you change headers in a CSV File with FasterCSV then save the new headers?

I'm having trouble understanding the :header_converters and :converters in FasterCSV. Basically, all I want to do is change column headers to their appropriate column names. something like: ...
0 votes
1 answer
135 views

FaterCSV Encoding Issue

I am using fastercsv gem for csv export with utf-8 encoding. When ever i export any arabic content in my csv using utf-8 , when i try to open in windows it comes out as some strange characters. where ...
0 votes
1 answer
885 views

special characters/chinese language text not rendering properly in csv

I am using charset=utf-8; in the csv options. The chinese characters and some special characters ( for eg :- '»', ) are still not rendered correctly in the csv, though the same text looks fine in the ...
0 votes
0 answers
79 views

Delay in writing values to CSV using FasterCSV gem

I'm reading values from one csv file using FasterCSV gem and doing some operations and dumping the same data to a different csv file by adding one additional column which specifies the status of the ...
-1 votes
1 answer
96 views

JSON Array to CSV via FasterCSV

I have a JSON array in the following format. [{"Name":"Ann","Gender":"Female","Age":"20"},{"Name":"John","Gender":"Male","Age":"22"}] I want to convert this into following CSV format without ...
0 votes
1 answer
128 views

undefined method `write_inheritable_attribute' for ContactsController:Class

I upgraded my Rails version from 3.2.13 to Rails 4.2.6. In the older version, they used map_fields plugin where in my contacts controller: class ContactsController < ApplicationController ...
1 vote
2 answers
1k views

Reading a CSV file into Ruby using Sequel and the SQLite3 gems

I need to read data from two CSV files and load them into SQLite3 in two different tables. I then want to query the two tables and create a new dataset that needs to be output into a new CSV/Excel. ...
1 vote
2 answers
1k views

Is there a Ruby CSV parser that generates column getter methods?

I'm trying to generalize a data retrieval mechanism with Ruby, and can't seem to find a way to retrieve a CSV file and access one of the row's columns by using a dot operator like so: Let's say I ...
17 votes
3 answers
8k views

How do I robustly parse malformed CSV?

I'm processing data from government sources (FEC, state voter databases, etc). It's inconsistently malformed, which breaks my CSV parser in all sorts of delightful ways. It's externally sourced and ...
3 votes
3 answers
3k views

Exporting to CSV from complex rails query

I was following this railscast video but I'm having serious difficulties to export my data to excel (or CSV for this matter). I'm using will_paginate in some data that I show on frontend like this: ...
3 votes
2 answers
1k views

Huge text or excel data Import FASTER into sql

I have a huge text file that has 1 million rows and each row there is only 28 length number as text. I want to import them into sql server that has table corresponding column. So that a million data ...
5 votes
3 answers
9k views

Rails FasterCSV "unquoted fields do not allow \r or \n"

I'm having an issue with FasterCSV and my rake db:seeds migration. I get the error: "rake aborted! Unquoted fields do not allow \r or \n (line 2)" on the following seeds.rb data: require 'csv' ...
78 votes
2 answers
30k views

What is Ruby 1.9 standard CSV library?

When I try the FasterCSV gem on my application I get this error: Please switch to Ruby 1.9's standard CSV library. It's FasterCSV plus support for Ruby 1.9's m17n encoding engine. By the way, I'm ...
4 votes
2 answers
2k views

Converting uneven rows to columns with FasterCSV

I have a CSV data file with rows that may have lots of columns 500+ and some with a lot less. I need to transpose it so that each row becomes a column in the output file. The problem is that the ...
4 votes
2 answers
2k views

Rails3 CSV putting &quot;&quot; instead of actual quotes

Similar to this question except I don't use html_safe anywhere in the whole project. I generate a CSV file in index.csv.erb like this: <%= response.content_type = 'application/octet-stream' CSV....
4 votes
2 answers
4k views

How to import CSV files in rails?

I am making an application in which it import a csv file names user.csv.But the problem i am facing is that it gives an error ArgumentError in CsvimportController#import wrong number of arguments (1 ...
0 votes
1 answer
3k views

Rails 3.2 or 4.x way to submit a form/file for preview

I'm trying to create a form, where I can upload an CSV file to import or to preview before import it. In my form I have: <%= form_for(@contact_import, :remote => true) do |f| %> <% if @...
0 votes
2 answers
312 views

How do I merge two CSV's with nearly identical sets using rules for which data is kept? (Using Ruby & FasterCSV)

I have two csv files, each with 13 columns. The first column of each row contains a unique string. Some are duplicated in each file, some only exist in one file. If the row exists in only one file I ...
21 votes
2 answers
12k views

fastercsv error with ruby 1.9.2

I have an existing rails application I'm running on ruby 1.9.2 and linux its rails version is rails 2.3.8 and it has a GEMFILE as well, in its vendor/gems directory it has 'fastercsv-1.5.4' gem and ...
2 votes
1 answer
702 views

FasterCSV importer to DataMapper model - No rows inserted

I have a model (called Test): property :id, Serial property :title, String, :length => 255, :required => true property :description, String, :length => 255, :...
3 votes
3 answers
1k views

Ruby CSV, using square brackets as row separators

I'm trying to use square brackets '[]' as a row separator in a CSV file. I must use this exact format for this project (output needs to match LEDES98 law invoicing format exactly). I'm trying to do ...
2 votes
1 answer
3k views

StreamReader not reading japanese characters from csv file [closed]

Reading csv file using stream reader with fields in double quotes of Japanese characters. Its not reading Japanese characters and taking it as unicode characters. I tried different encoding types but ...
3 votes
3 answers
3k views

FasterCSV: Check whether a file is invalid before accepting it - is there a simpler way?

I'm using FasterCSV on a Ruby on Rails application and currently it throws an Exception if the file is invalid. I've looked over the FasterCSV doc, and it seems that if I use FasterCSV::parse with a ...
1 vote
1 answer
2k views

How to write data and headers to csv dynamically?

I need to write headers and data into csv. The problem i'm facing is that my data may change order. I want to map my data exactly under the paricular header column. Can anyone tell me how to do this? ...
1 vote
1 answer
911 views

Importing a huge csv data file and using the header to access columns using Ruby

I need to import a huge csv data file (6880 columns) and I need to be able use the column headers to access it. What's the best way? Speed isn't important. Clarity is.
0 votes
1 answer
2k views

Line/Row separator/delimiter in exported CSV using Ruby CSV

Is it possible to change the default line separator from newline '\n' or '\r\n' to some other character, e.g '|' while importing. I know this seems foolish. But i have multiple records in csv file ...
0 votes
1 answer
2k views

How to add dropdown button into Csv file during exportation

I need to Add dropdown button for each row in csv export file i am using Fastercsv gem for csv exportation and my code is csv_string = FasterCSV.generate({:col_sep => "\t"}) do |csv| csv &...
0 votes
1 answer
851 views

how to pass unknown number of arguments to a function in R programming

I am parsing a csv with multiple columns. The number of columns is not fixed in the csv file. It varies from 5 to 10. I need to recreate a data.frame with these columns inside a function. I am ...
1 vote
1 answer
658 views

performance issues when importing large csv file into mdb using ADO

This is my code, but its taking about an hour to export all 17 million rows into mdb. I cannot use mysql or sql server for this purpose. I have to do it in access db and quickly coz this process runs ...
1 vote
1 answer
2k views

remove first row from csv file and create table according to it

Hi I am trying to be at point. I am trying to upload CSV into mysql atonce by using php in windows. so I am using the query LOAD DATA INFILE 'c:/Data/mysql/transaction_upload.csv' INTO TABLE ...
0 votes
1 answer
765 views

Large CSV manipulation

I have a very large CSV file (over 100 million records) that I would like to delete several columns from. I have tried the application CSVed (http://csved.sjfrancke.nl/#csvuni) but a file this size ...
0 votes
1 answer
343 views

set column width while generating CSV using fastercsv

I am using FasterCSV.generate method to generate csv. I want to set width to each column generated in csv. How can I do that?
1 vote
3 answers
4k views

Ruby scraper. How to export to CSV?

I wrote this ruby script to scrape product info from the manufacturer website. The scraping and storage of the product objects in an array works, but I can't figure out how to export the array data to ...
0 votes
1 answer
1k views

How to create multiple CSV file by using FasterCSV in Ruby

I am very new to ruby. I have a question that how can I export two files within one click. In the exsiting version, I can generate only one file at a time by calling 'def stream_csv', but when I try ...
0 votes
3 answers
2k views

Illegal quoting on line using FasterCSV in ruby 1.8.7

I am facing "Illegal quoting" error when parse the content from SQL dump and the dump file is in the format of TXT with tab (\t) separator. require 'rubygems' require 'faster_csv' begin ...
2 votes
1 answer
3k views

Ruby CSV.parse very picky when encountering quotes

I am finding the CSV parsing in Ruby 1.9.3 to be remarkably fragile. So much so that I am wondering if I am doing something wrong If I do the following in irb I get an error: 1.9.3-p125 :011 > ...
0 votes
0 answers
192 views

rails upload csv with FasterCSV to database IN UTF8

rails - 2.3.8 ruby - 1.8.7 I really searched old questions but found nothing. The deal is: Read from a .csv and update that info to my table in UTF8, but special chars like 'á','è','û' are changed ...
0 votes
0 answers
550 views

Why do I get "FasterCSV::MalformedCSVError: Illegal quoting error"?

I'm driving myself slight crazy with this one. I am trying to import a CSV file: uploaded_io = File.read("amazon_import.csv") FasterCSV.parse(uploaded_io, {:headers => true, :quote_char => '"',...