-1

Given page 4 of this PDF file, I was wondering if there is a way (using any R library) to import the 2 columns named SCALE SCORE and FREQ. into R as a .csv or other R-friendly formats?

1
  • 1
    @Reza There is nothing harsh in me basically quoting the official SO documentation: Please take the tour: "Focus on questions about an actual problem you have faced. Include details about what you have tried and exactly what you are trying to do. [...] show your work!". Cheers.
    – Henrik
    Jul 5, 2020 at 20:20

2 Answers 2

2

I have written a package that can help extract text from pdfs. It's written from scratch in C++ and is fairly fast (usually a bit faster than pdftools). At the moment you still need to wrangle the text into a table - as you would in pdftools. In your case, it would work like this:

library(dplyr)
library(PDFR)

df <- pdfpage("C:/users/Administrator/Documents/sales.pdf", 4)

df <- df[df$left > 440,] %>%
  group_by(top) %>%
  arrange(left, by_group = TRUE) %>%
  summarize(text = paste(text, collapse = ",")) %>%
  arrange(-top) %>%
  filter(seq(nrow(.)) > 4) %>%
  `[[`(2) %>%
  read.csv(text = ., header = FALSE, 
           col.names = c("freq", "cum_freq", "perc", "cum_perc"))

Which gives you:

#>     freq cum_freq perc cum_perc
#> 1    142      142 0.04     0.04
#> 2     15      157 0.00     0.04
#> 3     78      235 0.02     0.06
#> 4    269      504 0.07     0.13
#> 5    840     1344 0.21     0.34
#> 6   1690     3034 0.42     0.76
#> 7   3254     6288 0.81     1.57
#> 8   5413    11701 1.35     2.92
#> 9   7659    19360 1.91     4.83
#> 10  9696    29056 2.42     7.24
#> 11 11529    40585 2.87    10.12
#> 12 13145    53730 3.28    13.39
#> 13 13830    67560 3.45    16.84
#> 14 14844    82404 3.70    20.54
#> 15 15153    97557 3.78    24.32
#> 16 15120   112677 3.77    28.09
#> 17 15347   128024 3.83    31.92
#> 18 15525   143549 3.87    35.79
#> 19 15710   159259 3.92    39.70
#> 20 15596   174855 3.89    43.59
#> 21 15529   190384 3.87    47.46
#> 22 15451   205835 3.85    51.31
#> 23 15259   221094 3.80    55.12
#> 24 15028   236122 3.75    58.86
#> 25 15147   251269 3.78    62.64
#> 26 14683   265952 3.66    66.30
#> 27 14469   280421 3.61    69.91
#> 28 14229   294650 3.55    73.45
#> 29 13523   308173 3.37    76.82
#> 30 13246   321419 3.30    80.13
#> 31 12987   334406 3.24    83.36
#> 32 12264   346670 3.06    86.42
#> 33 11964   358634 2.98    89.40
#> 34 10841   369475 2.70    92.11
#> 35  9958   379433 2.48    94.59
#> 36  8529   387962 2.13    96.72
#> 37  6729   394691 1.68    98.39
#> 38  4437   399128 1.11    99.50
#> 39  2010   401138 0.50   100.00

Although this may seem a bit involved, it is great for pdfs like yours where the tables are the same on each page. If you ran the above code inside an lapply loop it could get multiple pages at a time far more quickly than cutting and pasting would.

To install you need devtools:

install.packages("devtools")
devtools::install_github("AllanCameron/PDFR")

Edit

If there are installation problems, here is the equivalent in pdftools:

install.packages(pdftools)

df <- pdftools::pdf_data("https://tea.texas.gov/sites/default/files/Scale%20Score%20Distribution%20Graph%201_Grade%203%20to%208%20English-r2_tagged.pdf")[[4]] 
df <- df[df$x > 440,] %>%
  group_by(y) %>%
  arrange(x, by_group = TRUE) %>%
  summarize(text = paste(text, collapse = ",")) %>%
  arrange(y) %>%
  `[[`(2) %>%
  `[`(3:41) %>%
  read.csv(text = ., header = FALSE, 
           col.names = c("freq", "cum_freq", "perc", "cum_perc"))
4
  • Thanks for sharing your package. I had been trying tabulizer, but bleh, rJava. Jul 5, 2020 at 20:45
  • @IanCampbell thanks - it's a work in progress. It's good at extracting text and grouping into words and paragraphs. I haven't figured out a consistent way of getting it to reconstruct tables though. Please consider forking it and having a look. Jul 5, 2020 at 20:49
  • Thanks, I have encountered many issues even trying to install your package.
    – rnorouzian
    Jul 5, 2020 at 20:55
  • @morouzian see my update for an alternative which is on CRAN and easier to install Jul 5, 2020 at 21:14
1

The quickest way to do this one time is to select the data columns (but not the column headings). On page 4 of your pdf that is from 1038 in the upper left to 100.00 in the bottom right. Copy that selection to the clipboard:

dta <- read.table(pipe("pbpaste"))  # For Mac Os
# For Windows use dta <- read.table("clipboard")
dta <- dta[, 1:2]  # Discard the last three columns
colnames(dta) <- c("ScaleScore", "Freq")
head(dta)
#   ScaleScore Freq
# 1       1038  142
# 2       1171   15
# 3       1250   78
# 4       1299  269
# 5       1335  840
# 6       1364 1690

If you want to do this repeatedly, you will have to install package pdftools.

3
  • Try pasting into a text file using the R script editor or Notepad to make sure you have the data in the clipboard. If not, try to copy again.
    – dcarlson
    Jul 5, 2020 at 20:30
  • Sure, but when I copy that content and paste onto a Notepad file, read.table("clipboard") gives me one column with everything meshed together?
    – rnorouzian
    Jul 5, 2020 at 20:37
  • You can wrap the single vector with x <- data.frame(matrix(unname(unlist(dta)), length(dta)/5, 5, byrow=TRUE)[, 1:2]). Then x will be your first two columns.
    – dcarlson
    Jul 5, 2020 at 21:06

Not the answer you're looking for? Browse other questions tagged or ask your own question.