Match multiline text using regular expression

Question

I am trying to match a multi line text using java. When I use the Pattern class with the Pattern.MULTILINE modifier, I am able to match, but I am not able to do so with (?m).

The same pattern with (?m) and using String.matches does not seem to work.

I am sure I am missing something, but no idea what. Am not very good at regular expressions.

This is what I tried

String test = "User Comments: This is \t a\ta \n test \n\n message \n";

String pattern1 = "User Comments: (\\W)*(\\S)*";
Pattern p = Pattern.compile(pattern1, Pattern.MULTILINE);
System.out.println(p.matcher(test).find());  //true

String pattern2 = "(?m)User Comments: (\\W)*(\\S)*";
System.out.println(test.matches(pattern2));  //false - why?

Answer 1

First, you're using the modifiers under an incorrect assumption.

Pattern.MULTILINE or (?m) tells Java to accept the anchors ^ and $ to match at the start and end of each line (otherwise they only match at the start/end of the entire string).

Pattern.DOTALL or (?s) tells Java to allow the dot to match newline characters, too.

Second, in your case, the regex fails because you're using the matches() method which expects the regex to match the entire string - which of course doesn't work since there are some characters left after (\\W)*(\\S)* have matched.

So if you're simply looking for a string that starts with User Comments:, use the regex

^\s*User Comments:\s*(.*)

with the Pattern.DOTALL option:

Pattern regex = Pattern.compile("^\\s*User Comments:\\s+(.*)", Pattern.DOTALL);
Matcher regexMatcher = regex.matcher(subjectString);
if (regexMatcher.find()) {
    ResultString = regexMatcher.group(1);
} 

ResultString will then contain the text after User Comments:

Answer 2

This has nothing to do with the MULTILINE flag; what you're seeing is the difference between the find() and matches() methods. find() succeeds if a match can be found anywhere in the target string, while matches() expects the regex to match the entire string.

Pattern p = Pattern.compile("xyz");

Matcher m = p.matcher("123xyzabc");
System.out.println(m.find());    // true
System.out.println(m.matches()); // false

Matcher m = p.matcher("xyz");
System.out.println(m.matches()); // true

Furthermore, MULTILINE doesn't mean what you think it does. Many people seem to jump to the conclusion that you have to use that flag if your target string contains newlines--that is, if it contains multiple logical lines. I've seen several answers here on SO to that effect, but in fact, all that flag does is change the behavior of the anchors, ^ and $.

Normally ^ matches the very beginning of the target string, and $ matches the very end (or before a newline at the end, but we'll leave that aside for now). But if the string contains newlines, you can choose for ^ and $ to match at the start and end of any logical line, not just the start and end of the whole string, by setting the MULTILINE flag.

So forget about what MULTILINE means and just remember what it does: changes the behavior of the ^ and $ anchors. DOTALL mode was originally called "single-line" (and still is in some flavors, including Perl and .NET), and it has always caused similar confusion. We're fortunate that the Java devs went with the more descriptive name in that case, but there was no reasonable alternative for "multiline" mode.

In Perl, where all this madness started, they've admitted their mistake and gotten rid of both "multiline" and "single-line" modes in Perl 6 regexes. In another twenty years, maybe the rest of the world will have followed suit.

Answer 3

str.matches(regex) behaves like Pattern.matches(regex, str) which attempts to match the entire input sequence against the pattern and returns

true if, and only if, the entire input sequence matches this matcher's pattern

Whereas matcher.find() attempts to find the next subsequence of the input sequence that matches the pattern and returns

true if, and only if, a subsequence of the input sequence matches this matcher's pattern

Thus the problem is with the regex. Try the following.

String test = "User Comments: This is \t a\ta \ntest\n\n message \n";

String pattern1 = "User Comments: [\\s\\S]*^test$[\\s\\S]*";
Pattern p = Pattern.compile(pattern1, Pattern.MULTILINE);
System.out.println(p.matcher(test).find());  //true

String pattern2 = "(?m)User Comments: [\\s\\S]*^test$[\\s\\S]*";
System.out.println(test.matches(pattern2));  //true

Thus in short, the (\\W)*(\\S)* portion in your first regex matches an empty string as * means zero or more occurrences and the real matched string is User Comments: and not the whole string as you'd expect. The second one fails as it tries to match the whole string but it can't as \\W matches a non word character, ie [^a-zA-Z0-9_] and the first character is T, a word character.

Answer 4

The multiline flag tells regex to match the pattern to each line as opposed to the entire string for your purposes a wild card will suffice.