My highlights are instances where you have to think outside the box, like a good puzzle:
You cannot describe complex actions on random data directly (like cutting gaps from audio tracks); but you can offer inverse actions, like undo operations, and then essentially test that applying an action, undoing it, redoing it, then undoing it again doesn’t change the result.
Testing a search function, you can’t actually test the search (without implementing the search functionality inside the tests again) because you don’t know the test data. But you can test that filtering the search results works, because filtering should product subsets of the unfiltered search.
You can describe web apps as state machines with valid transitions and then go ballistics with trying all kinds of interactions. That’s the promise of Quickstrom: You don’t need to specify each state of the web page as an example, but you specify the properties of each valid state and then ensure that no combination of interactions and button presses produces an invalid state. (Relying on app introspection, this probably works much better on the web with headless browsers than in Xcode/on mobile.)
None of this sounds like a replacement for what unit tests would do. But they do sound like a much better high-level approach to regression testing!
If nothing else, I’m intrigued to learn more about this now.
I ran into this, too, recently and wondered how everyone’s dealing with this. Because this is so annoying, I expected more outcries on the interwebs every day.
Maybe nobody is writing tests?
Either way: These helpers look very nice, and I’ll try them for sure. Should be part of the XCTest libray in my opinion. I seriously wonder why there’s no async XCTest assertion functions for this already.
If you’re just running into this for main actor isolation: One workaround that got suggested to me is to annotate the XCTestCase subclass itself with @MainActor to circumvent having to await isolated calls everywhere. (That doesn’t help for non-main actor-isolated calls, obviously.)
You and I are probably pretty familiar with the XCTestExpectations helper expectation(description:) that we can .fulfill() in an async callback, and then wait in the test case for that to happen. When you have neither async callbacks or notifications handy from which you can create test expectations, then there’s a NSPredicate-based version, too. Like all NSPredicates, it works with key–value-coding, or with block-based checks. These run periodically to check if the predicate resolves to “true”.
Working on a greenfield project is nice, but it also brings up all the uncertainties of the start: which components, modules, objects, interfaces do you need? With Swift, you even have to decide if you want to design the system in an object-oriented or a functional way. As far as the core functionality is concerned, you do have this choice. (When it comes to UIKit/AppKit, you don’t.)
I was experimenting with PHP development over the weekend to whip up a properly unit-tested library to generate license codes, hosted on your own web server. I wanted to make use of my newfound emacs prowess and figure out how programming is supposed to work with the stuff I have and have not prepared so far.
Found a question about how to “best” write the tests for the Fizz Buzz kata. The author provides his approach and questions as a GitHub Gist. One interesting takeaway is his use of the name of his constants, e.g. anyNumberDivisibleBy3 = 3. It expresses very well what kind of placeholder this number is supposed to be.
On macOS, the field editor is a NSText instance that appears whenever you edit a NSTextField. This means the text fields themselves offer no editing features; they just tell the shared field editor to appear in their drawing area and show their content. When you write XCUITests, you may want to edit cells in a table or fill out a form with many text fields. Today I learned that you don’t get to the field editor in UI tests and send it the typeText message. You work with the text fields like the user does: as if they themselves accepted user input.
Today I work on The Archive. The focus is on an issue brought up by the community. Search results don’t update the sort order when you modify a note unless you refresh manually. In fact, the issue is expected behavior. The Archive, being a note-taking app where you can filter notes on disk with a live search, is designed to not update the search results for an active search term. Att all. This should prevent the note from disappearing from the results if you remove the search term from its contents. If you search for “foo” and get 10 results, the note you currently edit should not disappear when you cut the search term, “foo”, from it. The Archive protects the search results; a mere live-reload would change the list to 9 results, removing the currently edited one, and that’d be pretty confusing.
I was removing quite a few protocols and classes lately. Turns out, I like what’s left. I relied on classes for example because they can be subclassed as mocks in tests. Protocols provide a similar flexibility. But over the last 2 years, the behavior I was testing shrunk to simple value transformations.
When you write a Swift type, you should prefer to write its tests in Swift, too. In a mixed-language code base, you will run into the situation where your Objective-C tests need to reference Swift types to initialize and pass to the object under test. That’s a different story. But you cannot import the generated -Swift.h out-of-the-box:
Today I wrote my first asynchronous test in my latest project and suddenly, during the waitForExpectations run loop, a totally unrelated test’s stuff crashed. To make things worse, it was part of an RxSwift subscription. No idea who kept that alive and where the EXC_BAD_ACCESS really originated.
I dabble with RxSwift right now. Figuring out how to write tests for pure functions and reactive code, I tried to write an assertion for incoming events: Doesn’t compile because equating arrays with Optional<URL> in them won’t. In other words, [Recorded<Event<URL>>] (non-optional) would work.
The thing is, tests are just code. If you have a hard time constructing your own objects with some particular state, it might be a sign that your API is hard to use!
This is all there is to the magic of test-driven development, we could say. A test is client code. This means it doesn’t have the inside perspective of the object or module under test. It has an outside perspective. Through tests you see how your production code is used (by other parts of your code or by other people in the case of libraries, say).
You start with a test. This means: you start with an outside perspective, asking yourself questions like “What would be a good (public) interface for this?” – The opposite is ad hoc reasoning, something along the lines of “I have this NetworkManager there and maybe just call it and return the JSON. Okay. Oh, now JSON doesn’t work, I need to send a custom object along, so I’ll just parse the JSON and create an object. But if that fails, hmm, I cannot send nil, so I’ll force unwrap and let someone else deal with this.” Or something.
These improvised solutions can cause trouble because all they do is focus on the perceived requirements of the object you’re writing, ignoring the requirements of code that uses the resulting code.
Writing tests first kind of equals writing your app outside-in, starting with the calling code, then implementing the code that’s to be called. Only test cases are super focused and you verify a lot more behavior and setup code with a single run than you could ever do with manual testing.
Experienced programmers will be able to switch perspectives even without writing tests. Their experience helps them to make it less likely to produce waste, whereas “waste” is code you write and then discard because you find it doesn’t fit.
Daniel Jalkut of MarsEdit fame is slowly getting up to speed with Swift. I have a few Swift-only projects already (Move!, TableFlip and secret in-progress ones). On top of that, every feature I add to the Word Counter is a Swift module, too. The main code of the Word Counter is still in Objective-C, though.
Integrating new types to the project using Swift is awkward. There’re things you can only use in one direction. Most if not all Objective-C code can be made available for Swift and works great. The other way around, not so much. Also, Objective-C imports from the -Swift.h file won’t be available in Swift-based test targets. So you’ll have to isolate these even further, create wrappers, or whatever. Porting a class to Swift can work, but the side effects on testing make it harder.
I like Daniel’s advice to write your unit tests in Swift if you’re still not certain if you should make the switch to this new and volative language. It’s a good idea to get your feet wet.
Considering the problems I ran into with the Word Counter, though, I wouldn’t know how, honestly.
This is mostly a reminder for my future self: It doesn’t suffice to create a TXT file and add it to the target in the file inspector (⌘⌥1) to be able to read it as part of the bundle. You also have to drag it into the “Copy Bundle Resource” build phase of the target so it get, well, bundled into the product. (In my case, it was the test target.)
The .md file was part of the test target but loading it from the NSBundle failed until I added it to the proper Build Phase
Create an empty file or add an existing file to the target’s group in Xcode and make it part of the proper target,
drag it into “Copy Bundle Resources” build phase,
load it using NSBundle(forClass: TheTestCase.self).URLForResource("filename", withExtension: "txt").
Storing PLIST or JSON files for structured data works just as well. I happen to require plain text flavors most of the time and always wonder why the loading fails for a couple of minutes.
How do you test NSURLSession properly? After all, using it in tests directly will hit the servers; that’s not what you want to do 1000s of times per day. Even if you use a localhost based server replacement the tests will be slower and potentially error-prone as the server replacement may not be 100% accurate. Anyway – there’s a popular series of posts about that topic. There, you’ll learn how to mock NSURLSession in your tests. I think the advice is good to move on quickly, but it only shifts the problem slightly out of sight. There are better alternatives.
“Using a private function means having a hardwired link to an anonymous collaborator. Over time, this will slowly hurt more.” (@jbrains) I was thinking about this the other day when I wrote tests for a presenter. It receives a date, formats it, and makes the view update a label with the result. Simple. (The real thing actually does a bit more, but that doesn’t matter much.)
Xcode now offers two kinds of tests natively. You don’t have to rely on 3rd party JavaScript libraries anymore to automate the iOS simulator and assert view conditions. So with the advent of native UI Automation tests, can you rely on them to verify your app works? Here’s the two sides of the 1 criterion you’ll ever need to find out:
As I’m exploring the use of block based API, which means to assign closures or functions handles to properties or pass them around as parameters to other functions, I found a few benefits and drawbacks in comparison to protocol-based object interactions. Here’s a breakdown of criteria for blocks and delegates.
To debug my threading issues and help bring forth future problems, I have created a simmple object that slows the current queue down:
letIsRunningTests=NSClassFromString("XCTestCase")!=nilstructQueueJigglePoint{/// Randomly interfere with the thread.staticfuncjiggle(){guard!IsRunningTestselse{return}#if DEBUGusleep(2*1000000)// 2 seconds#endif}}
I got this idea from Brett Schuchert on pages 188–90 of Uncle Bob’s Clean Code. A Handbook of Agile Software Craftsmanship. There, interference with traditional threading should randomly sleep, yield, or fall through. Enqueued blocks are a lot less volatile, so I only came up with sleeping.
Randomizing the sleep interval is up next. But a fixed number of 2–10 seconds helps find UI-blocking code already.
Just throw in a QueueJigglePoint.jiggle() in NSManagedObjectContext.performBlock executions, when dispatching async to the background, or when reading files, for example.
I found a very useful distinction by Matthijs Hollemans to grasp what protocol extensions in Swift 2 can be: instead of interfaces, they are traits or even mixins. At the same time, I want to raise awareness for misuse: just because behavior is mixed-in doesn’t mean it’s additional behavior. The code may live in another file, but its functionality can still clutter your objects.
I just found a shortcut to use Dependency Injection less during tests. In “Unit Testing in Swift 2.0”, Jorge Ortiz adapts solid testing principles to Swift. If you don’t have a strong testing background, make sure to watch his talk for key insights into the How using Swift!
Singletons have their use. I use two Singletons regularly in my projects: a DomainPublisher and a ServiceLocator. The latter is a registry of service objects which is global so the service objects don’t have to be. For practical use, most Singletons are overcomplicated or overly restrictive. Here’s how I suggest you implement Singletons in your apps:
Refactoring Legacy Code is hard. There are a few safe refactorings you can do with caution. But most chirurgical cuts require you to put the code in a test harness first to guard against regression. With C and Swift, you can create free functions as part of your app. To verify your objects use that function, you need to find a way to insert a test double.
I’m writing unit tests for my Storyboard-based view controllers. Button interaction can be tested in view automation tests, but that’s slow, and it’s complicated, and it’s not even necessary for most cases. To write unit tests instead of UIAutomation tests for buttons, you test multiple things. Here are the tests.
You don’t have to learn anything new if you work with Core Data in Swift. It’s a pain to use NSManagedObject directly, so you better work with custom subclasses all the time. Unboxing optional NSNumbers is just too cumbersome. There are caveats you should be aware of, though: Swift’s static typing makes it a bit harder to write test doubles for managed objects.
Mocking and stubbing in tests is useful to verify behavior and replace collaborating objects with simpler alternatives. Since Swift is statically typed, you can’t use a mocking library anymore. But it’s trivially simple to write simple mock objects as subclasses in place:
I assume you do write tests. To test system boundaries, you have to find out whether methods are called. Usually, people reach out for mocks to verify behavior.
If you use a lot of mocks, though, the tests can reveal high coupling in your code: either you have to mock objects multiple levels deep, which is a sign of too much knowledge about how the collaborating objects work, or you have to mock lots of objects one level deep. The latter can be acceptable when you test a components which is some kind of coordinator. The former, though, is never to be accepted. Use the feedback of these hard-to-write tests to make your code simpler.
That’s what the Law of Demeter is for. While “law” is way too strict, its principle is sound: strive to call methods on the objects you have direct access to, but avoid calling methods on objects which you obtain through another method call in the first place.
For example, the following violates the principle:
funcviolateThePrinciple(aCollaborator:Collaborator){letaStranger=aCollaborator.obtainFriend()// this is okay so faraStranger.doSomething()// this is not, because it's 2nd level}
In your unit tests, you can add a Collaborator mock to verify the calls and then you’re good to go. No multiple levels of mock objects and stubs. You may call this “shallow reaching” (opposed to “deep reaching”), although no one else calls it that.
That’s the Law of Demeter in a nutshell: only talk to friends which you know through properties or parameters, but don’t talk to strangers which you get to know through the return values of your friends’ methods. You never know what to expect.
