0

Below code attempts to count the number of times "Apple" appears in an HTML file.

object Question extends App {

  def validWords(fileSentancesPart: List[String], wordList: List[String]): List[Option[String]] =
    fileSentancesPart.map(sentancePart => {
      if (isWordContained(wordList, sentancePart)) {
        Some(sentancePart)
      } else {
        None
      }

    })

    def isWordContained(wordList: List[String], sentancePart: String): Boolean = {

    for (word <- wordList) {
      if (sentancePart.contains(word)) {
        return true;
      }
    }

    false

  }

  lazy val lines = scala.io.Source.fromFile("c:\\data\\myfile.txt" , "latin1").getLines.toList.map(m => m.toUpperCase.split(" ")).flatten

  val vw = validWords(lines,   List("APPLE")) .flatten.size

  println("size is "+vw)


}

The count is 79 as per the Scala code. But when I open the file with a text editor it finds 81 words with "Apple" contained. The search is case insensitive. Can spot where the bug is ? (I'm assuming the bug is with my code and not the text editor!)

I've wrote a couple of tests but the code seems to behave as expected in these simple use cases :

import scala.collection.mutable.Stack;
import org.scalatest.FlatSpec;
import org.scalatest._;

class ConvertTes extends FlatSpec {

  "Valid words" should "be returned" in {

    val fileWords = List("this" , "is" , "apple" , "applehere")
    val validWords = List("apple")

    lazy val lines = scala.io.Source.fromFile("c:\\data\\myfile.txt" , "latin1").getLines.toList.map(m => m.toUpperCase.split(" ")).flatten

    val l : List[String] = validWords(fileWords, validWords).flatten

    l.foreach(println)

  }

    "Entire line " should "be returned for matched word" in {

    val fileWords = List("this" , "is" , "this apple is an" , "applehere")
    val validWords = List("apple")

    val l : List[String] = validWords(fileWords, validWords).flatten

    l.foreach(println)

  }


}

The HTML file being parsed (referred to as "c:\data\myfile.txt") in code above :

https://drive.google.com/file/d/0B1TIppVWd0LSVG9Edl9OYzh4Q1U/view?usp=sharing

Any suggestions on alternatives to code above welcome.

Think my issue is as per @Jack Leow comment. For code :

  val fileWords = List("this", "is", "this appleisapple an", "applehere")
  val validWords = List("apple")

  val l: List[String] = validWords(fileWords, validWords).flatten

  println("size : " + l.size)

size printed is 2, when it should be 3

5
  • Just going through the code real quick, if a single line in the file contains the word "APPLE" twice, how is that counted?
    – Jack Leow
    Feb 2, 2015 at 22:51
  • @JackLeow I think this is indeed my issue, please see update
    – blue-sky
    Feb 2, 2015 at 22:57
  • "(?i)apple".r findAllIn "this appleisApple an" size Feb 2, 2015 at 23:34
  • In case your are looking for an alternative method. Feb 2, 2015 at 23:35
  • s/sentance/sentence/g :) Feb 3, 2015 at 8:35

2 Answers 2

0

I think you should do the following:

def validWords(
  fileSentancesPart: List[String],
  wordList: List[String]): List[Option[String]] =

  fileSentancesPart /* add flatMap */ .flatMap(_.tails)
    .map(sentancePart => {
      if (isWordContained(wordList, sentancePart)) {
        Some(sentancePart)
      } else {
        None
      }
    })

def isWordContained(
  wordList: List[String],
  sentancePart: String): Boolean = {

  for (word <- wordList) {
    //if (sentancePart.contains(word)) {
    if (sentancePart.startsWith(word)) { // use startsWith
      return true;
    }
  }
  false
}
0

You could use regular expressions with a Source iterator:

val regex = "([Aa]pple)".r
val count = Source.fromFile("/test.txt").getLines.map(regex.findAllIn(_).length).sum

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.