StringTokenizer class in Java provides a tokenizer String method for splitting a string into tokens based on a specified delimiter.
The tokenizer String can be any string that separates the tokens, such as a comma, semicolon, or whitespace. Using the tokenizer String method of StringTokenizer class, we can split a string into its constituent parts. By calling the nextToken() method, we can retrieve each token in turn, and by using the hasMoreTokens() method, we can check if there are any more tokens left. The length() method of the StringTokenizer class can be used to get the length of each token. StringTokenizer String is a useful tool for string manipulation and can be used to parse CSV files, URLs, or other text-based data.
The StringTokenizer class is part of the Java.util package and provides a simple way to split a string into tokens. The class has two constructors, one that takes a string to be tokenized and a delimiter character or string, and another that takes the same arguments as well as a Boolean flag indicating whether or not to include the delimiter as a token.
Once you have created a StringTokenizer object, you can use its various methods to iterate through the tokens and perform various operations on them.
Tokenization Method
The tokenization method is the process of splitting a string into smaller parts or tokens. This process is performed using a delimiter, which can be a character or a string of characters that separates each token. For example, consider the following string:
String input = "Hello World! How are you today?";
If we want to split this string into individual words, we can use the space character as the delimiter, like so:
StringTokenizer tokenizer = new StringTokenizer(input, " ");
while (tokenizer.hasMoreTokens()) {
String token = tokenizer.nextToken();
System.out.println(token);
}
Output
Hello
World!
How
are
you
today?
In this example, we create a new StringTokenizer object with the input string and a space character as the delimiter. We then loop through the tokens using the hasMoreTokens() method and retrieve each token using the nextToken() method. Finally, we print each token to the console.
Token Length
In addition to the basic functionality provided by the nextToken() method, the StringTokenizer class also provides methods for retrieving the length of each token and for retrieving a specific token by index. To get the length of the current token, you can use the token.length() method. For example:
public class StringTokenizerExample {
public static void main(String[] args) {
String input = "Hello World! How are you today?";
StringTokenizer tokenizer = new StringTokenizer(input, " ");
while (tokenizer.hasMoreTokens()) {
String token = tokenizer.nextToken();
System.out.println("Token: " + token + " Length: " + token.length());
}
}
}
Output
Token: Hello Length: 5
Token: World! Length: 6
Token: How Length: 3
Token: are Length: 3
Token: you Length: 3
Token: today? Length: 6
In this example, we retrieve each token as before but also use the length() method to get the length of each token, which we then print to the console.
Example
Let's take a look at a complete example that demonstrates how to use the StringTokenizer class in Java:
public class Example {
public static void main(String[] args) {
String input = "John,Doe,123 Main St.,Anytown,USA";
StringTokenizer tokenizer = new StringTokenizer(input, ",");
while (tokenizer.hasMoreTokens()) {
System.out.println(tokenizer.nextToken());
}
}
}
Output
John
Doe
123 Main St.
Anytown
USA
In this example, we have a string that represents a comma-separated list of values. We create a new StringTokenizer object with this string and a comma as the delimiter. We then loop through the tokens using the hasMoreTokens() method and retrieve each token using the nextToken() method. Finally, we print each token to the console.
Note that we did not use the length() method in this example, as we are only interested in the individual tokens themselves.
GO TO FULL VERSION