Search and replace using regex in Visual Studio Code

Blog post — cleaning up Pandoc-converted Markdown

Archived blog post, originally published on 28 April 2020.

Sometimes I convert HTML pages extracted from websites to Markdown using Pandoc. Here is an example:

pandoc -s -r html https://codecast.wp.imt.fr/ -o codecast.md

The problem is that it often generates extra information following this pattern:

[interesting text that
I want
to keep]{uninteresting parameters extracted from HTML that I want to delete}

For example:

[Any CS educator has to explain sooner or later a portion of code or a
structured text to learners. The Codecast tool has been specially
designed by CS educators and developed for MOOCs to replace
non-interactive screencasts.]{style="font-weight: 400"}

I found an easy way to clean the Markdown using the Visual Studio Code find and replace feature with regex. Here is the regular expression I used:

^\[([\s\S\r]*?)\]\{.*\}

The ^ means beginning of a line. If you want to find this pattern in the middle of lines, just delete it.

The \[ \] means I want to search for something between square brackets.

The [\s\S\r] means a whitespace character \s (space, tab…) or a character that is not a whitespace \S (a, b, c…) or a newline character \r. Note that in the Visual Studio Code regex system, \s does not include the newline — even though I would have thought it was a whitespace character — which is why I specifically add \r.

The (x*?) — where x is the expression just above — asks to capture a group using ( and ), with the character x appearing zero or more times with *, and once or none with ?. Maybe you are lost at this point. Anyway: because we asked for a capture, we will be able to use $1 as a variable containing the captured text.

The \{.*\} means curly braces with something inside. Here I could also use [\s\S\r] instead of .*.

I then replace with $1, and that is it. Enjoy.