I spend a couple of hours investigating the XeLaTeX purgatory where character mapping live. I didn’t know what these were before, too, don’t worry. I’ll walk you through it.

Observation: My Quotation Marks Look Weird

The observable behavior is the following. Let’s say you start with Markdown with “dumb” quotes like this:

"Wie bitte?"

… and use a conversion tool like Pandoc or MultiMarkdown to turn it into LaTeX, and you pick German quotation marks (the non-guillemet ones). You’ll see why it’s important for this example to use the German quotation marks. They are different from the curly quotes you know from English texts.

What you get in the .tex file is going to be this:

,,Wie bitte?``

LaTeX (actually TeX) transforms these double-comma into a low double quote (U+201E DOUBLE LOW-9 QUOTATION MARK). Same for the double backticks, only high up (U+201C LEFT DOUBLE QUOTATION MARK – the “left” in the name is U.S. hegemony! 🙂). That’s the “mapping” process.

The intended result in a German text would be this:

„Wie bitte?“

Nicely formatted. Usually works that way just fine, except with a quotation mark at the end of the quoted text, because what you actually get in the PDF is:

 „Wie bitte¿‘

Explanation: There Are Mappings to Invert Quotation Marks

The upside-down quotation mark puzzled me. The curly single quote was an indicator that the sequence of ?` was not parsed as “quotation mark followed by curly double quotes”, but as “upside down quotation mark followed by single curly quote”.

To cut the research short:

There’s also a mapping for inverted exclamation marks and inverted question marks (aka “Spanish” exclamation and question marks resp.), it’s defined by the sequence of the right-side-up version followed by a backtick?

?`Qué pasa?

then becomes

¿Qué pasa?

Which works just fine in a text with English quotation marks.

To fix (aka disable) this in one place, you can add a separator manually: ?{}``, but I’m not going to automate that in the Markdown-to-LaTeX conversion step.

These character sequence mappings are defined in a file called tex-text.map. You can find it here on a MacTeX/TeXLive installation:

/Library/TeX/Root/texmf-dist/fonts/misc/xetex/fontmapping/base/tex-text.map

That’s a symlink to the current TeXLive version, and the actual path for the 2024 edition on my machine is this, where you’ll probably also find it on Linux machines:

/usr/local/texlive/2024/texmf-dist/fonts/misc/xetex/fontmapping/base/tex-text.map

It has these two lines that conflict with my curly quotation marks and that need to go:

...
U+0021 U+0060	<>	U+00A1	; !` -> inverted exclam
U+003F U+0060	<>	U+00BF	; ?` -> inverted question
...

Disable the Mapping, XeLaTeX edition

The most direct way to get rid of this in a XeLaTeX project is to

1. copy the tex-text.map into your project directory, e.g. as my-tex-text.map,
2. comment out or remove these lines,

3. then compile the .map file via

    $ teckit_compile my-tex-text.map
   

4. and finally use it in your document.

To use it, pass the optional argument Mapping=my-tex-text to your XeLaTeX font settings, which use the fontspec package most likely:

\documentclass{minimal}
\usepackage{fontspec}
\setmainfont{Arial}[Mapping=my-tex-text]  %% uses the mapping file
\begin{document}
,,Und warum?`` Oder ,,warum nicht!``
\end{document}

You can also install the mapping globally, so you don’t have to ship it with every project. But I couldn’t make it work from textmf-local (it complained that the mapping doesn’t correspond to a font, too).

It did work from texmf-dist and textmf-config, by placing the files next to the original tex-text.map. That’s brittle, though, because when TeXLive 2025+ comes around, you’d need to remember to copy the files over. If you don’t, and if you got used to the custom mapping, there’s a risk of not noticing that no mapping at all works anymore. Since it’s on the character level, it’s easy to miss. I don’t want to risk that.

But here it is, if you’re interested:

# I'm using the `texlive/2024/` path here to make
# the code harder to copy-paste. I really don't want
# anyone to use this.
$ cd /usr/local/texlive/2024/texmf-dist/fonts/misc/xetex/fontmapping/base
$ sudo cp tex-text.map my-tex-text.map

# Perform your changes:
$ sudo edit my-tex-text.map

# Compile the map and make it discoverable, registering
# it in the `ls-R` file of known TeX stuff:
$ sudo teckit_compile my-tex-text.map
$ sudo -H mktexlsr   # or texhash

Disable the Mapping, pdfLaTeX and LuaLaTeX Edition

Use microtype:

\usepackage{microtype}
\DisableLigatures[?,!]{}

That’s it.

You can imagine that I really regret having started with XeLaTeX. microtype’s ligature disabling isn’t compatible with the xetex engine, and that’s why it took so long to find out how mappings work, where fonts are installed, where anything is installed, really.

After figuring this stuff out I consider migrating to LuaLaTeX nevertheless. I didn’t research this properly: it’s going to be the long-term successor to pdfLaTeX and comes with Unicode support like XeLaTeX, which was the reason I picked the latter.

How did I Figure This Stuff Out Anyway?

With TeXLive on your computer, texdoc from the terminal is your friend. It produces the documentation PDFs for packages and opens them in your default PDF viewer.

• texdoc tds: “A Directory Structure for TEX Files” explains where fonts and .map files go.
• “Installing Personal Fonts”: https://www.tug.org/fonts/fontinstall-personal.html – Short list of things to keep in mind when installing fonts. Brought me to the “TDS” document.
• “TeX Live updmap and fmtutil, -sys vs. -user: Best practice and common cases”: https://www.tug.org/texlive/scripts-sys-user.html – tells you how to refresh the cache of known files, including .map files. Needed if you try to put font mapping files in texmf-local. You’ll also be using the updmap-sys command, then, to register mappings. (This is not required with the approach above.)
• “Installing TeX fonts: 4. Font map files: telling TeX about the new font”: https://www.tug.org/fonts/fontinstall.html – Step-by-step guide that helped to get texmf-local configuration started and double-check if I have missed a compilation or cache busting step. Useful to get to know the intended process more.

It took hours to understand what’s going on, how mappings work, and then fiddle with the whole system that I’m an absolute beginner with until it finally worked.

I hope I saved you some tears.