2

I'm writing a lexer/scanner for the first time, and have run into a problem splitting the input-string. Example:

val result = "func add(Num x, Num y) = x+y;".split(???) 
result == Array("func", "add", "(", "Num", "x", ",", "Num", "y", ")", "=", "x", "+", "y", ";")

But the problem is that I can't simply split on whitespace characters, doing so wouldn't separate add from ( for example.

Any help with this?

2
  • Don't speak scala ;), so I can't tell you how to handle the empty entries, but splitting on \s|\b - adding word boundry - should do it. See regex101 sample
    – SamWhan
    Mar 29, 2016 at 13:45
  • 1
    @ClasG that did the trick, if you post it as an answer I'll accept it
    – Saveyy
    Mar 29, 2016 at 13:49

2 Answers 2

1

This will give you a bunch of empty items that your EE will have to handle, but adding word boundry - \b - should do it.

Check example at regex101.

I.e. ...split('\s|\b') (or /\s|\b/).

Regards

1
  • If I'm not wrong this will add spaces in the result which I think is not the expected Mar 29, 2016 at 15:27
0

Look into http://www.scala-lang.org/api/rc/index.html#scala.util.parsing.combinator.RegexParsers

Here is an unfinished example:

import scala.util.parsing.combinator.RegexParsers

trait Element

case class Function(name: String,
                    params:Map[String, String],
                    expression:Seq[String]) extends Element

case class Class(name: String,
                 params: Map[String,String],
                 body: Seq[String]) extends Element

object LanguageParser extends RegexParsers {

  val name: Parser[String] = ".*".r

  val `type`: Parser[String] = ???

  val parameters: Parser[Map[String,String]] = "(" ~> (`type` ~ name).* <~")" ^^ {
    case t => (t map {
      case a ~ b => a -> b
    }).toMap
  }

  val expression: Parser[Seq[String]] = ???

  val function: Parser[Function] =
    "func " ~> name ~ parameters ~ "="~ expression ^^ {
      case name ~  params ~ _ ~ expr => Function(name, params, expr)
    }
  
  val `class`: Parser[Class] =
    "class " ~> name ~ parameters ~ "{" ~ expression ~ "}" ^^ {
      case name ~  params ~ _ ~ expr ~_ => Class(name, params, expr)
    }

  val topLevelParsers: Parser[Seq[Element]] =
    function |
      `class` |
      value |
      ifelse

  def parse(s: String): Seq[Element] = parseAll(topLevelParsers, s.trim) getOrElse
    (throw newIllegalArgumentException("Could not parse the given string: " + s.trim))

  def parseAll(s: String):Seq[Element] =
    s split ";" flatMap parse
}

Cheers

2
  • For now the \b|\s is sufficient. but i fear i might have to rewrite the code i have, and learn about the regex parser to achieve a more reliable solution. Thanks for your reply !
    – Saveyy
    Mar 29, 2016 at 14:16
  • You'll save time learning it straight away. I didn't the first time because I was lazy, I had to rewrite everything and keep saying why I didn't know about this before.
    – Joan
    Mar 29, 2016 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.