Today I work on The Archive. The focus is on an issue brought up by the community. Search results don’t update the sort order when you modify a note unless you refresh manually.

In fact, the issue is expected behavior. The Archive, being a note-taking app where you can filter notes on disk with a live search, is designed to not update the search results for an active search term. Att all. This should prevent the note from disappearing from the results if you remove the search term from its contents. If you search for “foo” and get 10 results, the note you currently edit should not disappear when you cut the search term, “foo”, from it. The Archive protects the search results; a mere live-reload would change the list to 9 results, removing the currently edited one, and that’d be pretty confusing.

The Archive does not just display search results like a Smart Search in Finder. The visible workspace contents are supposed to stay the same until you change them deliberately. Removing a note from the app is a deliverate change. Notice my choice of words here: “workspace contents” instead of “search results”. When you work on stuff in your note archive, imagine putting a bunch of paper notes on your desk. It’d be odd if someone else took away some the notes you wanted to continue to work with. In some sense, the workspace contents of The Archive are more like a manual selection of things you can look at than they are just search results. The means of getting to the selection of notes you focus on is via search, but that’s rather accidental.

The issue I linked to pertains updating the sort order of the workspace contents, not updating the contents. Technically, re-sorting the notes is a change to the workspace contents that don’t work at the moment. From a user’s perspective, sorting the workspace by modification date should make the last modified note in the workspace bubble up to the top. This expectation makes sense.

Implementing a “fix” for the current behavior means to change the protection of the workspace contents. Now, it’s close to a binary decision: does the user see all notes, unfiltered, or does the workspace contain a careful selection that should be protected? Allowing the workspace contents to re-sort based on file system changes is not tricky, but it might affect more use cases than I want.

Which brings me to the titular problem: sometimes, I’m afraid to change parts of my app because I cannot know all the (unintended) consequences. The app’s behavior is way too complex for this.

I deal with this problem in two ways:

1. I write short lists of intended behavior for intricate use cases to make sure I don’t mix things up in my head 2 months from now.
2. I write UI tests that remote-control the application and assert it behaves as expected.

UI tests are regression tests. They make sure I don’t break use cases, or complicate user interactions that would be tiresome to exercise manually hundreds of times.

But UI tests only make sure I don’t break behavior that is actually tested. I can still break things that I didn’t think about. I’m inclined to write tests to orchestrate external file changes, user-originated note editing, and the effects on the UI. Replicating this manually is too cumbersome. But I don’t want to write UI tests as thoroughly as I write unit tests. Testing every branch in your app using UI tests, or integration tests in general, is virtually impossible. There are too many cases. You have to focus on a selection of interaction patterns.

Or, as I quoted J.B. Rainsberger before, “Integration tests are a scam.”

Still, they are useful to make sure that I can test parts of the app that are not yet easily unit-testable, like asynchronous file change timing issues.

If I wrote my apps in a purely functional manner, say with Haskell, I probably wouldn’t have these problems, but also wouldn’t have the app in a working condition. It’s a pragmatic trade-off.

In the end, all I can do is my best to ensure I don’t break things in a bad way. But I will eventually break things to some degree. That’s an inevitable part of the development process. Like now, I’m even deliberately breaking the protection of workspace contents in my app, changing the perceived behavior. It’s hard to tell what’s broken and what’s a change sometimes, because the judgement depends on the interaction of a user with the software, not just the app as a static entity.