Sorting Out Overlapping File Types

May 13th, 2020

In my initial post about this problem, I talked about the observations and how I began to figure out where the permission problem came from. I turned out to be an attempt at changing the file extension from .txt to .md. When the user opens a .txt file in your app, macOS makes sure you only get access to that exact file path by default. You cannot just write willy-nilly anywhere else without the user’s permission. File extension changes are included in this protection.

So that’s good to know, but why does the app actually try to change file extensions? I guess I botched the document type setup a while back and never noticed until I added Sandboxing way later in the application’s lifetime.

Here’s a rundown of my own setup for failure.

You see, TableFlip opens Markdown files; Markdown files sometimes have the .md or .mmd or .markdown extension or similar, but often come as .txt as well. It is plain text, after all.

Now .txt file are by default reported to be of the Universal Type Identifier (UTI) "public.text" or "public.plain-text". That’s baked into macOS, it seems.

The .md extension and its friends isn’t a known default UTI, so any Markdown editor has to specify the UTI and a list of recognized extensions to help macOS and NSDocument figure things out.

Until today, TableFlip used to have only 2 document types registered: CSV and Markdown. Here’s a tl;dr representation of the stuff you put into the Info.plist in code form:

static let markdown = DocumentTypeConfiguration(
    documentType: .markdown,
    // List of all file type UTIs that are associated with this:
    fileType: DocumentTypeConfiguration.FileType(
        reading: [
            "net.multimarkdown.text",
            "net.daringfireball.markdown",
            "pro.writer.markdown",
            "public.plain-text",
            "public.text"
        ], writing: [
            "net.multimarkdown.text",
            "net.daringfireball.markdown"
        ]),
    // No matter what comes in: what should we treat this as?
    canonicalTypeName: "net.daringfireball.markdown",
    extensions: [
        "md", "mdown", "markdown", "mmd", "text", "txt"
    ])

I use this representation to implement the common NSDocument callbacks to determine writableTypes(for:) and recognize files by file type or extension. You can specify different “reading” and “writing” file types in NSDocument, so I wanted to reflect that here as well.

Way before TableFlip began to write CSV files, it could read CSV files but would only write out Markdown files. Similarly, with the setup above, TableFlip would read plain text files, e.g. .txt, but not write them. When a “read”-operation reported the file type public.plain-text, the app would know that this is supposed to be part of the markdown configuration and store this info locally in the NSDocument object. This DocumentTypeConfiguration instance would then be used when the app is saving changes, e.g. in the save operation callback writableTypes(for:). It would return the array ["net.multimarkdown.text", "net.daringfireball.markdown"] there. Note the absence of public.plain-text.

My interpretation:

The app reads a .txt as public.plain-text and sets the file type once; this info stays untouched while you edit the document in TableFlip.
But once you perform external changes in e.g. TextEdit, which treats the file as a document as well, and multiple Sandboxes are involved, and access now goes through a NSFileCoordinator behind the curtains, well then apparently TableFlip reassures itself about the writable file types.
Notably, the array of writable file types does not include the plain text UTI, so this is treated like a file type change internally, resulting in a file extension change as well.

Where does step (2), the “reassure what file type we’re dealing with”, happen?

In my implementation of the “file changed on disk” callback, NSDocument.presentedItemDidChange, I check if the contents really did change (and ignored events where only metadata would change, as e.g. Dropbox would regularly do). The API docs advise us to do exactly that:

Because this method notifies you of both attribute and content changes, you might want to check the modification date before needlessly rereading the contents of a file. To do that, you must store the date when your object last made changes to the file and compare that date with the item’s current modification date. Use the coordinate(readingItemAt:options:error:byAccessor:) method of a file coordinator to ensure exclusive access to the file when reading the current modification date.

If the user has pending changes, TableFlip asks what to do with them. If the document cache isn’t dirty, though, the new data should be read from disk; when there are no conflicts and the app can update its data to the stuff on disk, I call NSDocument.revert(toContentsOf:ofType:) for this:

guard let fileURL = self.fileURL else { return }
try self.revert(toContentsOf: fileURL, 
                ofType: self.documentType.canonicalTypeName)

Aaaaaaannd that’s where the canonicalTypeName is used. That’s the problem. For Markdown, as we’ve seen above, this produces "net.daringfireball.markdown". And for some reason the default extension (maybe because of precedence order in the file extensions array?), the system changes the file extension when the type name changes from .txt for plain text to .md for Markdown.

So that’s the source of the problem: I was getting a public.plain-text file for reading, stored it internally of type “Markdown”, then upon external (!) changes, reverted the contents and used the cached document type that didn’t know what the original type name really was, thus changing it to net.daringfireball.markdown.

The simplest fix to approach this is rely on NSDocument.fileType wherever possible, which reflects the UTI, and use the canonical type name as a fallback only:

guard let fileURL = self.fileURL else { return }
try self.revert(toContentsOf: fileURL, 
                ofType: self.fileType ?? self.documentType.canonicalTypeName)

With that quick fix to the file extension change problem, I’m left dealing with another issue: when the document type is public.plain-text, the NSSavePanel’s accessoryView file format picker doesn’t auto-select anything.

The file extension stays the same when saving, but now the document type cannot be inferred during Save-As operation by the default save panel for public.plain-text documents (don't be fooled by the file extension of the Save-As dialog; it works fine for Markdown document types).

I’m telling this story not in a chronological order. I arrived at this point waaaaay later, after I tried many different things, including adjustments to the recognizes document types and customizations to the save panel. So in the next installment, we’ll have a look at that.

Links to this article

Blog Series: TableFlip's NSDocument File Type Issues and Fixes

This is an overview of a short series of posts I wrote about TableFlip’s 2020 bug. People were getting a “You don’t have permissions” error, a Sandboxing error, when they worked with .txt files, but not with .md files, even though the app treated them the same. The issue boiled down to a misconfiguration on my side that went by unnoticed until I enabled Sandboxing. I was wrongly defining document types in the app’s Info.plist, and I was reporting supported file types in the NSDocument subclass in a manner that didn’t work out well.
Observations of the Curious Problem of NSDocument-Based App Changing the File Extension

Users have reported problems with TableFlip saving their files recently. One wrote about it in the Zettelkasten forums, if you want to see the problem in context. To reproduce the problem: when you open foo.txt in TableFlip and a text editor, then change the file in the editor rapidly, TableFlip would show a “You don’t have permissions” error once you tried to save changes from TableFlip later.