Questions tagged [sqldf]
SQLDF is an R package for running SQL statements on R data frames.
677
questions
-1
votes
0
answers
11
views
When using sqldf package for left join in python, is there a format to be followed for column names? [closed]
I'm using sqldf to perform a left join. Here's my code:
query="select a.*, b.* from df1 as a left join df2 as b on (a.name = b.name and a.Year\<=b.Year)"
df3=sqldf(query)
Even though ...
0
votes
1
answer
34
views
Extracting row value based on matching columns in separate dataframes using SQL or R
I have two dataframes as shown below:
Df1
date
a
b
2023-01-31
10
john
2022-02-30
15
elle
2020-03-04
20
david
Df2
date
a
2023-01-31
10
2022-02-30
30
2020-04-04
20
I want to extract the values ...
-1
votes
2
answers
501
views
While performing SQL query in python using pandas i am facing the error : TypeError: __init__() got multiple values for argument 'schema'
enter image description here
Trying to compile sql query in python. Which i used to do very frequently in python. But never faced this error in my past. Help me in fixing the same.
Query:
from ...
0
votes
2
answers
23
views
read.csv.sql inserts all rows despite the WHERE clause in its SQL clause
I use the following (pseudo) code to import huge CSV files in R when only a rather small portion of the data in the file has to be treated/imported and other rows should simply be ignored. I don't put ...
0
votes
1
answer
44
views
Does read.csv.sql puts the result dataframe in memory or in the database?
I'm using sqldf package to import CSV files into R and then produce statistics based on the data inserted into the created dataframes. We have a shared lab environment with a lot of users which means ...
0
votes
1
answer
25
views
SQL delete query not executing using sqldf in R
I have a table - "cla_retail" - in sql that has 20,000+ rows. I have read the tables in R using dbConnect and stored it in a dataframe called "cla_retail_df"
I am trying to execute ...
0
votes
1
answer
41
views
SQL in R - Transform a column into rows
I have the following query:
sqldf("
SELECT YEAR, SUM(SUB1) AS 'SUB1', SUM(SUB2) AS 'SUB2',
SUM(SUB3) AS 'SUB3', SUM(SUB4) AS 'SUB4', SUM(SUB5) AS 'SUB5'
FROM table1
GROUP ...
0
votes
1
answer
39
views
How can I fix the 'line x did not have y elements' error when trying to use read.csv.sql?
I am a relative beginner to R trying to load and explore a large (7GB) CSV file.
It's from the Open Food Facts database and the file is downloadable here: https://world.openfoodfacts.org/data (the raw ...
0
votes
1
answer
23
views
R- Iterating through each group and dynamically assigning values
I have the following dataset:
id
row_name
start_date
end_date
rows_overlap_period
person_1
1
2010-04-23
2010-06-22
2,3,4,5,6
person_1
2
2010-04-25
2010-06-24
3,4,5,6
person_1
3
2010-04-27
2010-06-...
0
votes
0
answers
34
views
How to solve the Error -> Error: near "then": syntax error
I used sqldf packges in R.
I tried:
project=sqldf('select distinct
ID
, sex
, case when age<30 then "1" --20대
when age<...
2
votes
1
answer
31
views
Does the R sqldf() function support a way to show the columns of a table (i.e. describe)
Does the R function sqldf() support a way to show table columns similar to what the sql command "desc my_table" would do. It does not seem to support "desc my_table" or "...
0
votes
1
answer
133
views
How to define column names while using read.csv.sql on a file without header?
I often use read.csv function to read large CSV files. The files are without header and therefore by using col.names parameter I define properly the name of the variables in the dataframe that would ...
1
vote
0
answers
20
views
Is it possible to use PostgreSQL driver in r programs on local dataframes via sqldf?
Of course, it is possible to connect from r to a PostgreSQL server and use sqldf to run queries on that server. But is it possible to use PostgreSQL's SQL syntax and functions on local dataframes ...
0
votes
0
answers
73
views
Is there a way to append multiple elements to an existing string using a FOR loop?
I am working on creating a data frame column summaries for multiple columns in R using SQL (sqldf). The reason I went with this method instead of doing it directly in SQL is because there was some ...
0
votes
1
answer
155
views
Trying to read 20GB of data, read.csv.sql Produces Errors
I have a 20GB dataset in csv format and I am trying to trim it down with a read.csv.sql command.
I am successfully able to load the first 10,000 observations with the following command:
testframe = ...
0
votes
0
answers
36
views
i am getting this error, misuse of aliased aggregate cost , when running this query in r studio
this is the code i am using inside r studio with sqldf package, please check the issue and tell how to solve this,
sqldf("select orders.id , firstname, sum(item_cost * quantity_ordered) as cost
...
0
votes
0
answers
48
views
Pandasql will not import: ImportError: cannot import name 'create_engine' from 'sqlalchemy' (unknown location)
Just a day ago I used pandasql to query
and now I'm getting an "import error" when importing (as seen in the picture).
Can someone please explain to me the reason for this, and also the ...
0
votes
1
answer
42
views
Putting the first table column (ID) last, without specifying the other table columns
Background
I am using R Studio to connect R to Microsoft SQL Sever Management Studio. I am reading tables into R as follows:
library(sqldf)
library(DBI)
library(odbc)
library(data.table)
TableX <- ...
0
votes
1
answer
65
views
how to subset from a data frame using sqldf
I want to subset only the cards that are :jack,queen,king using sqldf
color <- rep(c("black","red"),each=26)
suit <- rep(c("clubs","spades","diamonds&...
0
votes
2
answers
53
views
Converting time series transaction data into an occurrence count table of all products in R
I have 5 million+ rows of daily transactions data in a column format of (year, month, day of the month, hour of the day, product that is purchased):
I created this data frame as an example (please ...
1
vote
2
answers
21
views
Fill a data.frame column with values from other data.frame column with a condition R
I have this two data.frames:
df1 = data.frame(
"scientific_name"=c("Anchietea ballardii","Anemia patens","Cololobus argenteus"),
"threat_status&...
0
votes
1
answer
45
views
Pass variable to sqldf function in R? R session aborted
I am having a problem doing this. This code works fine, but I would like to pass t1.FIELD1 as a variable.
sqldf("SELECT DISTINCT t1.ID as ID, t2.ID as bdev
FROM V_LINK AS t1
INNER JOIN V_LINK AS ...
0
votes
3
answers
54
views
How to get sqldf to allow arguments from a r function
Consider the following minimal example:
DB1<-data.frame(x=c(1,2,3))
out1<- sqldf("SELECT x FROM DB1")
test <- function (db,column){
fn$sqldf("SELECT column FROM db")
...
0
votes
0
answers
25
views
Counting Similar Values in SQL With Similar Time Stamps - SQLDF
I am looking at ride share data that shows a unique identifier for each rider. I am trying to find if there are riders that may benefit from a membership depending on if they travel the same route on ...
1
vote
1
answer
63
views
How to troubleshoot no such column error sqldf in r
I'm getting an error (no such column: cs7.Start) while executing a Where clause using sqldf in r. I have looked at the other historical post and tested few things (single quote, underscore) but of no ...
0
votes
0
answers
28
views
sqldf JDBC ERROR: Function "date" not found
I am trying to merge columns with sqldf in Rstudio, however I receive an error says: "JDBC ERROR: Function "date" not found". The code is this:
library(sqldf)
sqldf("SELECT A....
0
votes
2
answers
35
views
referring to variables in sql in R
Any hints as to why this doesn't work. Is it because from is a keyword?
join_1 <- sqldf("select distinct ds1.from from ds1
except
select distinct ds2.name ...
0
votes
0
answers
46
views
Sqldf join using row id in r
For the data structure in the table below (peak_refined) I'd like to match the column ID with the unique row.names of the second table ('dat'). I'd prefer to use the sqldf package but stuck with ...
0
votes
0
answers
30
views
Adding index to sqldf() makes select slower/hang
I have an ip trace capture where I'm trying to match up the response datagrams to the originating:
This proved slightly difficult to do with a dplyr join as there are multiple candidates for every ...
0
votes
0
answers
66
views
Why do I get an error when running an sqldf query in R 3.3.3?
I am running sqldf using a WITH clause that works fine in R 3.6.3 but not in R 3.3.3.
Results using R 3.6.3:
sqldf("WITH cte_a AS (SELECT cyl, avg(mpg) FROM mtcars group by cyl) SELECT * from ...
0
votes
0
answers
117
views
Add lines to pandasql.sqldf string query based on a list for loop
I have an f-string SQL query that looks like this:
concatenated_df = ps.sqldf \
(f"""SELECT
a.*,
COALESCE(b.scores_estimator1,'NOT FOUND') as scores_estimator1
...
0
votes
0
answers
24
views
R Programming - Unable to insert data into SQL table from the data frame
I'm trying to insert data into SQL table from the data frame, but could not succeed as I'm getting invalid object error, CLIENT not found.
How to insert data to SQL table from dataframe?
'''...
0
votes
2
answers
89
views
How to left join two data frames conditionally - by rows that fall within a date range - and by two variables found in each data frame
I have two simulated data frames:
d, created below, which has all the rows of longitudinal data for two different people. Each row has a start and end date. Some rows have a visit date associated ...
0
votes
0
answers
41
views
Bring blood tests results between dates range
I have solved my problem with sqldf, unfortunatly I worked on a closed network and it isn't installed there. This code looks on each row in by_newborn and brings from a TSH_Results the maximal TSH ...
1
vote
1
answer
139
views
R sqldf Count, Group By, Having > 1
I have a table with 3 columns, and a few thousand records, sample is below:
df1 <- data.frame(
ID = c('V1', 'V1', 'V1', 'V3', 'V3', 'V3', 'V4', 'V5','V5','V5'),
Category = c('a', 'a', 'a', 'a', ...
-1
votes
1
answer
192
views
find timestamp difference between 2 columns with sqldf
According to this answer: https://stackoverflow.com/a/25863597/12304000
We can use something like this in mysql to calculate the time diff between two cols:
SELECT TIMESTAMPDIFF(<INTERVAL>,<...
1
vote
0
answers
162
views
Error with python using pandasql: "no such column:..." when the column actually exists
I'm Lucas and I'm a master student in management science in Belgium. I have a very poor background in coding and everything related to it, so my problem is probably very basic but still very confusing ...
0
votes
0
answers
5
views
value selection in a dataframe with sqldf
I am new to R and especially to using the Select data between two values with sqldf library. I would like to select rows in my table whose values in a specific column are between two values of my ...
3
votes
0
answers
319
views
Is there an easy way to pre-filter data into vroom?
In R, I have switched to using vroom due to its speed at reading in large delimited files, but I cannot find a simple way to pre-filter large datasets as I could do with say the sqldf package or ...
-3
votes
1
answer
59
views
How do you change format of dataset column to date format in R (easiest method)?
Trying to change the format of the column incident_timestamp so I can accurately query the data using sqldf correctly. Here is the dataset I have for reference.Column #21 is the column I am interested ...
0
votes
1
answer
135
views
How to run python sqldf module inside static function?
I have this code which works fine:
import pandas as pd
import sqldf
df1 = pd.DataFrame({'col1': [1, 2, 3], 'col2': ['Jhon1', 'Jhon2', 'Jhon3']})
df2 = pd.DataFrame({'col1': [1, 2, 3], 'col2': ['Doe1', ...
0
votes
3
answers
202
views
Error in tcltk::as.tclObj(X) : cannot handle object of mode 'list
I'm very new to R studio and Sqldf. I don't know what the issue is even if I wanted to thoroughly explain it. I am trying to use a SQL code to count up all distinct zip codes in the column zip_code. I ...
0
votes
0
answers
379
views
Import select rows from a csv file R
I am using the following code to import data into my program by SimulationID.
args <- commandArgs(TRUE)
j <- args[1]
library(sqldf)
select <- paste0("select * from ...
0
votes
1
answer
395
views
How to load a pandas dataframe that is stored as pickle or feather (df.pkl or df.ftr) into sqldf
I have two files - df.pkl and df.ftr.
And I am using run_qry = lambda q: sqldf(q, globals()) to run sql queries.
So for example:
I am creating a new DataFrame like new_dfr = run_qry("""...
0
votes
0
answers
394
views
sqldf is not working inside a user defined function
I am getting table from oracle database and converted that into pandas DataFrame (df).I access this dataframe into my summary function and create a new dataframe (crit_df) from it. Till here ...
0
votes
1
answer
306
views
Join tables using multiple columns
I have 2 tables in Python. I want them to join using the first 3 columns (i.e. Level1, Level2, and YearMonth). I used the following code but it would omit those rows where there was no value for ...
0
votes
1
answer
178
views
'Calloc' could not allocate memory When trying to Knit an RMD file into an HTML
I am trying to knit an RMD file into an HTML report using a pretty large dataset (2,600,000 rows with 18 columns) and I keep getting a memory error on the same line of code... Here is the full error:
...
0
votes
1
answer
336
views
Python: Sorting X axis value manually
I am trying to sort my x-axis on my graph as I believe my x-axis is not in an ascending order due to the data in my dataframe not being sorted properly. What I am seeing in my dataframe is"1,10,...
0
votes
1
answer
41
views
R language: Accidental Combining of 2 different variable
for(i in c(2001:2002)) {
ontime <- read.csv(paste0(i, ".csv"), header = TRUE)
if(i == 2001) {
dbWriteTable(conn, "ontime", ontime, append = TRUE)
} else {
dbWriteTable(...
0
votes
1
answer
186
views
Select Rows That Does Not Contain any Negative Or Missing Value
Assume a database table has a few hundred columns. In SQL statements, how would you select rows/records that do not contain any negative or missing value? Can you do it using the sqldf package for R ...