we usually judge a regular expression by one question: does it match the right text?
that question is not enough.
a pattern can return the correct answer for every test you wrote and still become a denial-of-service vulnerability when somebody gives it the wrong string. the match fails eventually, but "eventually" is doing dangerous work there.
this is regex denial of service, or ReDoS.
when the engine starts guessing
many regex engines use backtracking. when there are several possible ways to match part of a string, the engine tries one path. if a later part fails, it goes back and tries another.
most of the time this is fast enough to be invisible. the trouble starts when nested or overlapping quantifiers create a huge number of possible paths.
consider a pattern shaped like this:
regex^(a+)+$
give it a string containing only a characters and it matches. add one different character at the end and the engine has to prove that all the possible groupings of those a characters fail.
textaaaaaaaaaaaaaaaaaaaaaaaa!
the input grew by one character. the work can grow far more than one step.
why this becomes security
if the pattern runs against attacker-controlled input, performance is no longer a private implementation detail.
an expensive regex inside a login route, search endpoint, firewall rule or validation function can hold a worker busy. repeat the request enough times and the service spends its time exploring useless regex paths instead of serving users.
the attacker does not need to bypass the validation. making the validation think for too long is already enough.
suspicious shapes
nested quantifiers deserve attention:
regex(.*)+ (a+)* ([a-zA-Z]+)*
so do alternatives that can match the same prefix:
regex(a|aa)+
these shapes are not automatically vulnerabilities. the engine, anchors, surrounding pattern and input limits all matter. but they are a good reason to stop and test the failure case, not only successful matches.
what i would do instead
first, make the pattern less ambiguous. replace broad wildcards with the actual character set. avoid nesting repetition when one repetition can express the rule. use atomic groups or possessive quantifiers when the engine supports them and backtracking is unnecessary.
second, limit the input before matching it. a username validator does not need to inspect a two-megabyte string.
third, benchmark hostile near-matches. the dangerous input is often a long valid-looking prefix followed by one character that forces failure.
and sometimes the correct fix is not another clever regex. if the input has nested structure or complicated grammar, use a parser. regex is excellent when the language is actually regular. beyond that point, cleverness becomes debt with punctuation.
a regex is code. it consumes time, handles untrusted data and can fail under load. it deserves the same suspicion as everything else sitting on the request path.