What a regular expression is
A regular expression, or regex, is a small pattern language for describing text. A program uses the pattern to search a string, test whether it fits a shape, pull out parts of it, or replace parts of it. Most languages and many editors include a regex engine.
The simplest pattern is plain text. The pattern cat matches the three letters c, a, t wherever they appear, including inside longer words such as "concatenate".
Literals and metacharacters
Most characters are literals: they match themselves. A small set are metacharacters with special meaning: . ^ $ * + ? { } [ ] \ | ( ). The dot matches any single character except a newline. The caret and dollar sign are anchors.
Escaping
To match a metacharacter as plain text, put a backslash in front of it. a\.c matches only the literal string "a.c", while a.c also matches "abc" and "a-c". Inside a character class most metacharacters lose their meaning, so [.] is another way to match a real dot. In Python, write patterns as raw strings such as r"\d+" so backslashes reach the engine unchanged.
Character classes and shorthands
Square brackets match one character from a set. [aeiou] matches one vowel, [0-9] matches one digit, and [^0-9] matches one character that is not a digit, because a caret at the start of a class negates it. Common sets have shorthand names:
\dmatches a digit, and\Dmatches anything else.\wmatches a "word" character: letters, digits, and the underscore.\Wis the opposite.\smatches whitespace such as a space, tab, or newline.\Sis the opposite.
Anchors
Anchors match a position, not a character. ^ matches the start of the string and $ matches the end, so ^\d+$ accepts a string made only of digits. \b matches a word boundary, the point between a word character and a non-word one. With multiline mode turned on, ^ and $ also match at the start and end of each line. In Python, re.findall(r'^\w+', 'first line\nsecond line') returns ['first'], and adding re.M returns ['first', 'second'].
Quantifiers
A quantifier says how many times the item before it may repeat:
*means zero or more times.+means one or more times.?means zero or one time, which makes the item optional.{n,m}means between n and m times;{n}means exactly n and{n,}means n or more.
Greedy versus lazy
By default, quantifiers are greedy: they take as much text as they can while still letting the whole pattern match. Adding a ? after a quantifier makes it lazy, so it takes as little as possible. The difference shows up clearly on HTML-like text:
>>> re.findall(r"<.+>", "<b>bold</b>")
['<b>bold</b>']
>>> re.findall(r"<.+?>", "<b>bold</b>")
['<b>', '</b>']
The greedy .+ runs to the last > in the string, so it swallows everything. The lazy .+? stops at the first > that lets the match finish, giving two separate tags.
Groups and alternation
Parentheses group part of a pattern so a quantifier can apply to the whole group, as in (ab)+. They also capture the text they matched so you can read it later. The pipe character means "or": cat|dog matches either word. Alternation has low precedence, so gr(a|e)y is needed to limit the choice to one letter. When a pattern has capturing groups, Python's findall returns the groups instead of the whole match; a non-capturing group, written (?:...), groups without capturing.
Patterns and their matches
Each row below was run through Python's re.findall, which returns every non-overlapping match from left to right.
| Pattern | Input | re.findall result |
|---|---|---|
cat | concatenate the cat category | ['cat', 'cat', 'cat'] |
\d+ | Order 66 shipped 3 items on day 12 | ['66', '3', '12'] |
colou?r | color colour colr | ['color', 'colour'] |
\d{2,3} | 1 22 333 4444 | ['22', '333', '444'] |
gr[ae]y | gray grey griy | ['gray', 'grey'] |
cat|dog | hotdog catalog | ['dog', 'cat'] |
a.c | abc a.c a-c ac | ['abc', 'a.c', 'a-c'] |
a\.c | abc a.c a-c ac | ['a.c'] |
\d+\.\d+ | pi is 3.14 and e is 2.718 | ['3.14', '2.718'] |
(\w+)-(\d+) | item-42 box-7 x-y | [('item', '42'), ('box', '7')] |
Notice that \d{2,3} on "4444" returns "444": the match is greedy, takes three digits, and the single leftover "4" is too short for another match.
Flavors differ
There is no single regex standard. Common families include PCRE-style engines, the engine built into JavaScript, Python's re module, and the POSIX basic and extended syntaxes used by tools such as grep and sed. The basics on this page behave alike in most of them, but details vary: which shorthands exist, how lookarounds and named groups are written, and whether + or ? need a backslash in a basic POSIX pattern. The character encoding of your input also matters when a pattern must handle non-ASCII text.
Common mistakes
- Unescaped dot. Writing
example.comalso matches "exampleXcom". Useexample\.comwhen you mean a real dot. - Missing anchors in validation.
\d+finds digits inside "abc123", so a check that only searches will accept that string. Use^\d+$, or a full-match function, to require the whole input to fit. - Greedy overreach.
.*between two delimiters often runs to the last delimiter instead of the next one. Use a lazy quantifier or a negated class such as[^>]*. - Catastrophic backtracking. Many engines try alternatives one by one and back up when a path fails. A pattern with nested repetition, such as
(a+)+$, gives the engine a huge number of ways to split a long run of "a" characters. When the input almost matches but fails at the end, the engine may try all of those splits, and the time can grow exponentially with input length. Avoid nesting quantifiers over the same characters, and make each part of the pattern match something distinct. - Parsing nested formats. Regex is a poor fit for nested structures such as HTML or JSON. Use a real parser for those.