Class Tokenizer

java.lang.Object
uk.ac.starlink.ttools.Tokenizer

public class Tokenizer extends Object
Utility class to help with tokenizing strings.
Since:
1 Sep 2005
Author:
Mark Taylor
  • Method Details

    • tokenizeLines

      public static String[] tokenizeLines(String text) throws uk.ac.starlink.task.UsageException
      Splits a string up into lines, separated by semicolons or newlines. Semicolons may appear inside quoted strings without terminating a line.
      Parameters:
      text - input string
      Returns:
      array of lines
      Throws:
      uk.ac.starlink.task.UsageException
    • tokenizeWords

      public static String[] tokenizeWords(String line) throws uk.ac.starlink.task.UsageException
      Chops up a line of text into tokens. Works roughly like the shell, as regards quotes, whitespace and comments.
      Parameters:
      line - line of text
      Returns:
      array of words corresponding to line
      Throws:
      uk.ac.starlink.task.UsageException