LogTokenizer

Value Members

1. final def !=(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

2. final def ##(): Int

   

   Definition Classes

   AnyRef → Any

3. final def ==(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

4. final def asInstanceOf[T0]: T0

   

   Definition Classes

   Any

5. def clone(): AnyRef

   

   Attributes

   protected[java.lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... )

6. final def eq(arg0: AnyRef): Boolean

   

   Definition Classes

   AnyRef

7. def equals(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

8. def finalize(): Unit

   

   Attributes

   protected[java.lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( classOf[java.lang.Throwable] )

9. final def getClass(): Class[_]

   

   Definition Classes

   AnyRef → Any

10. def hashCode(): Int

    

    Definition Classes

    AnyRef → Any

11. final def isInstanceOf[T0]: Boolean

    

    Definition Classes

    Any

12. def leadingPunctuation: Regex

    

    A regular expression describing punctuation to strip from the beginning of tokens; matches will be stripped by replacing them with their first match group.

    A regular expression describing punctuation to strip from the beginning of tokens; matches will be stripped by replacing them with their first match group. Override this definition to customize tokenizer behavior. Defaults to

    "(\\s)[^\\sA-Za-z0-9-_/]+|()^[^\\sA-Za-z0-9-_/]+"
    

    .

    "(\\s)[^{\\sA-Za-z0-9-_/]+|()}[^\\sA-Za-z0-9-_/]+"

    Definition Classes

    LogTokenizing

13. final def ne(arg0: AnyRef): Boolean

    

    Definition Classes

    AnyRef

14. final def notify(): Unit

    

    Definition Classes

    AnyRef

15. final def notifyAll(): Unit

    

    Definition Classes

    AnyRef

16. def rejectedIntratokenPunctuation: Regex

    

    A regular expression describing punctuation to strip from within tokens; matches will be stripped by replacing them with the empty string.

    A regular expression describing punctuation to strip from within tokens; matches will be stripped by replacing them with the empty string. Override this definition to customize tokenizer behavior. Defaults to

    "[^A-Za-z0-9-_./:@]"
    

    if not overridden.

    "[^A-Za-z0-9-_./:@]"

    Definition Classes

    LogTokenizing

17. final def synchronized[T0](arg0: ⇒ T0): T0

    

    Definition Classes

    AnyRef

18. def toString(): String

    

    Definition Classes

    AnyRef → Any

19. def tokens(msg: String, post: (String) ⇒ String = identity[String], pred: (String) ⇒ Boolean = str => true): Seq[String]

    

    Splits a log message into a sequence of tokens, by

    Splits a log message into a sequence of tokens, by

    • collapsing runs of whitespace into single spaces,
    • stripping rejected intertoken punctuation,
    • stripping rejected intratoken punctuation,
    • splitting on whitespace,
    • rejecting candidate tokens not containing at least one letter, and
    • applying optional user-supplied transformation and filtering functions.

    returns

    a sequence of tokens

    Definition Classes

    LogTokenizing

    See also

    Using word2vec on log messages

20. def trailingPunctuation: Regex

    

    A regular expression describing punctuation to strip from the end of tokens; matches will be stripped by replacing them with their first match group.

    A regular expression describing punctuation to strip from the end of tokens; matches will be stripped by replacing them with their first match group. Override this definition to customize tokenizer behavior. Defaults to

    "[^\\sA-Za-z0-9-_/]+(\\s)|()[^\\sA-Za-z0-9-_/]+$"
    

    if not overridden.

    "[^{\\sA-Za-z0-9-_/]+(\\s)|()[}\\sA-Za-z0-9-_/]+$"

    Definition Classes

    LogTokenizing

21. final def wait(): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

22. final def wait(arg0: Long, arg1: Int): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

23. final def wait(arg0: Long): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )