The answer given in the spoiler tag is not quite correct!
Test case
According to the spoiler, this shouldn’t match “abab”, but it does.
Corrected regex
This will match what the spoiler says: ^.?$|^((.)\2+?)\1+$
Full workup
Any Perl-compatible regex can be parsed into a syntax tree using the Common Lisp package CL-PPCRE. So if you already know Common Lisp, you don’t need to learn regex syntax too!
So let’s put the original regex into CL-PPCRE’s parser. (Note, we have to add a backslash to escape the backslash in the string.) The parser will turn the regex notation into a nice pretty S-expression.
At which point we can tell it’s tricky because there’s a capturing register using a non-greedy repetition. (That’s the \1 and the +? in the original.)
The top level is an alternation (the | in the original) and the first branch is pretty simple: it’s just zero or one of any character.
The second branch is the fun one. It’s looking for two or more repetitions of the captured group, which is itself two or more characters. So, for instance, “aaaa”, or “ababab”, or “abbabba”, but not “aaaaa” or “abba”.
So strings that this matches will be of non-prime length: zero, one, or a multiple of two numbers 2 or greater.
But it is not true that it matches only “any character repeated a non-prime number of times” because it also matches composite-length sequences formed by repeating a string of different characters, like “abcabc”.
If we actually want what the spoiler says — only non-prime repetitions of a single character — then we need to use a second capturing register inside the first. This gives us:
^.?$|^((.)\2+?)\1+$.
Where the \2 refers to the character captured by (.), meaning that it has to be the same character throughout the matched string.
Regex is good for a few very specific things, and sysadmins used to use it for goddamn everything. If all your server logs are in lightly-structured text files on a small number of servers, being able to improvise regex is damn useful for tracking down server problems. Just write a shell loop that spawns an ssh logging into each server and running grep over the log files, to look for that weird error.
These days, if you need to crunch production server logs you probably need to improvise in SQL and jq and protobufs or systemd assmonkery or something.
But if you actually need a parser, for goodness sake use a parser combinator toolkit, don’t roll your own, especially not with regex. Describing your input language in plain Haskell is much nicer than kludging it.
(This is the “totally serious software engineering advice” forum, right?)
I’ve worked mostly as a data scientist / analyst but regex was being user to identify various things in the SQL database (which was viewed locally via R table). I forget the exact is cases, mostly remembering how complex some of it got… Especially after certain people were using GPT to build them.
And GPT like to make up extra bits not necessary, but my coworkers didn’t exactly have the knowledge to read regex, which lead to nobody really checking it. Now it just gives me anxiety, haha.
I have to admit that using CL-PPCRE does not really help me understanding the regexp any better. But this may be because I deal with complex regexps for decades now, and I just read them.
The answer given in the spoiler tag is not quite correct!
Test case
According to the spoiler, this shouldn’t match “abab”, but it does.
Corrected regex
This will match what the spoiler says:
^.?$|^((.)\2+?)\1+$Full workup
Any Perl-compatible regex can be parsed into a syntax tree using the Common Lisp package CL-PPCRE. So if you already know Common Lisp, you don’t need to learn regex syntax too!
So let’s put the original regex into CL-PPCRE’s parser. (Note, we have to add a backslash to escape the backslash in the string.) The parser will turn the regex notation into a nice pretty S-expression.
> (cl-ppcre:parse-string "^.?$|^(..+?)\\1+$") (:ALTERNATION (:SEQUENCE :START-ANCHOR (:GREEDY-REPETITION 0 1 :EVERYTHING) :END-ANCHOR) (:SEQUENCE :START-ANCHOR (:REGISTER (:SEQUENCE :EVERYTHING (:NON-GREEDY-REPETITION 1 NIL :EVERYTHING))) (:GREEDY-REPETITION 1 NIL (:BACK-REFERENCE 1)) :END-ANCHOR))At which point we can tell it’s tricky because there’s a capturing register using a non-greedy repetition. (That’s the
\1and the+?in the original.)The top level is an alternation (the
|in the original) and the first branch is pretty simple: it’s just zero or one of any character.The second branch is the fun one. It’s looking for two or more repetitions of the captured group, which is itself two or more characters. So, for instance, “aaaa”, or “ababab”, or “abbabba”, but not “aaaaa” or “abba”.
So strings that this matches will be of non-prime length: zero, one, or a multiple of two numbers 2 or greater.
But it is not true that it matches only “any character repeated a non-prime number of times” because it also matches composite-length sequences formed by repeating a string of different characters, like “abcabc”.
If we actually want what the spoiler says — only non-prime repetitions of a single character — then we need to use a second capturing register inside the first. This gives us:
^.?$|^((.)\2+?)\1+$.Where the
\2refers to the character captured by(.), meaning that it has to be the same character throughout the matched string.Average Matt Parker code
Thanks, I now have insight into my own personal hell for when I die.
Regex is good for a few very specific things, and sysadmins used to use it for goddamn everything. If all your server logs are in lightly-structured text files on a small number of servers, being able to improvise regex is damn useful for tracking down server problems. Just write a shell loop that spawns an
sshlogging into each server and runninggrepover the log files, to look for that weird error.These days, if you need to crunch production server logs you probably need to improvise in SQL and
jqand protobufs or systemd assmonkery or something.But if you actually need a parser, for goodness sake use a parser combinator toolkit, don’t roll your own, especially not with regex. Describing your input language in plain Haskell is much nicer than kludging it.
(This is the “totally serious software engineering advice” forum, right?)
I’ve worked mostly as a data scientist / analyst but regex was being user to identify various things in the SQL database (which was viewed locally via R table). I forget the exact is cases, mostly remembering how complex some of it got… Especially after certain people were using GPT to build them.
And GPT like to make up extra bits not necessary, but my coworkers didn’t exactly have the knowledge to read regex, which lead to nobody really checking it. Now it just gives me anxiety, haha.
I upvoted this because I hate it.
Whatever you do, don’t get in a time machine back to 1998 and become a Unix sysadmin.
I was a sysadmin with some Linux usage in 1998, does that count?
I have to admit that using CL-PPCRE does not really help me understanding the regexp any better. But this may be because I deal with complex regexps for decades now, and I just read them.