If you don't like opinion pieces on the joys of particular programming styles/languages - don't read on ;-)
It has been months since I wrote any functional code in earnest. Most recently I have been busy coding in Objective-C, and I've a long history with Java, C++ and C, with the latter language being used for many years in conjunction with a very elegant home-brew object-like framework.
As something of a student of language technology: principles and practice, I usually follow all the news and buzz to be found spread around the infowebs, in such places as Lambda the Ultimate, various blogs and language mailing lists (such as Ruby-core).
Just today, I had reason to fire up the CAL tools again in order to knock up some prototypes. The overall experience was surprising to me, in terms of the effect that producing a program in a (lazy) functional language actually had on my psyche. Having given functional programming a rest for the several months, I was almost "accosted" by the flavours of functional programming afresh. Being one with a reasonable grasp of the concepts, I naturally did not have the learning curve that one would experience from scratch (i.e. this is not an ab initio experience), yet the hiatus was sufficient to being the differences between functional and procedural/OO programming into sharper relief, and evidently to tickle my brain anew in areas that I had forgotten were differentiated for the kind of cognition that you do when crafting a functional solution.
I was using the combination of CAL's textual and graphical development tools (the Eclipse plug-in and the Gem Cutter to be precise). I have erstwhile found that the combination of these two can be very potent, especially in the exploratory stages of development - when you are surveying the problem domain and starting to form hypotheses about the structure of a solution.
Once I had completed the tasks at hand, I sat back, and was aware of the 'buzz' that I was feeling. This is the buzz you get from general problem solving success, presumably a literal reaction to endorphins released (I assure you there were no other overt substances involved, unless you could my normal coffee intake that morning!). Thinking about why I was feeling so chipper, I surmised that it was for two reasons:
1. A strong sense of puzzle solving.
2. A sense of the elegance of the emergent solution.
On the first point, it occurred to me that the whole exercise of assembling a functional solution is, for the most part (that is, ignoring some of the operational issues), one of dealing directly with fitting shapes together. This is of course an analogy, but what is going on in my head feels like the assemblage of shapes - fitted together perfectly to create the required abstracts. Of course, this is nothing more than a whiff of mathematics in general - the perception of structures and order, and formal assemblies of such things. I think the whole perception of 'shape' is made even more tangible by the ingredient of having the visual development environment alongside the syntactic one - two kinds of symbols to aid the perception of abstracts. In CAL, the visual environment is also much more interactive with a lot of information-rich feedback (rollover type inferencing on any argument/output, popup lists of proposed functional compositions etc.). I suppose that this sense of "assemblage" is at least partly responsible for the strong sense of puzzle solving. One experiences similar sentiments having designed a nice OO pattern/architecture, but not in the same way.
To the second point, concerning elegance, this is something that is strongly related to the way symbols have such a direct relationship to the semantics of functional programs. Any part of a CAL program that actually compiles, makes a formal and precise (though not necessarily narrow) contract with the rest of the world as to the quantum of processing it contributes. Part of the elegance comes from the richness of the type system and the sets of types of values that can be described as forming this contract. Another part, however, comes from the fact that the function (as a package of processing) is a highly flexible unit. The contract that functions make in a lazy functional language concern the relationships between the input and output values, but these relationships can be composed into larger structures in any way that satisfies the strictures of the type system. Elegance, though is merely a perceived quality, what is more important is how the manifestation of elegance is related to practical effects on the economics of software development.
At a low level, this manifests as a beautiful ability to build abstracts through binding arguments in any order as larger logical structures are created. In other words, the way in which you abstract is fluid, and the little semantic package of the single function can be utilised in so many different ways, compared to the strict notion of a 'call' in procedural languages.
At a high level, this behaviour results in the functional packages being able to be combined (and importantly, reused) very powerful ways, with a direct bearing on the way the intended semantics can be conjured, but always under the watchful eye of the type system - that is able to act in both a confirming and informational way. The latter can feel like epiphany. Many times have I been focussed on a specific problem, and composing functions for that purpose, only to have the compiler's inferred type tell me that I've produced something much more general and powerful than I had thought (sometimes it even tells me that I've been stupid in overlooking that function already in the library that does exactly what I'm trying to do again!).
Today's fun with CAL had all these facets. The qualia of functional programming is quite different to that of OO programming and in many ways you are much more constrained than the latter. Good OO design is certainly critical to creating correct, efficient and maintainable software, and while there is therefore a real spectrum of 'better' and 'worse' designs/implementations for a given problem, much of the structure of an OO program itself is informal to the implementing language and lives in the minds of the developers who created it (and perhaps in their technical documentation). The reasons why certain encapsulations were chosen over others, and why certain code paths/sequences are established are undoubtedly built on reason, but they become major artifacts of the particular solution. In the functional world, things are both more sophisticated and more simple at the same time (naturally, 'complexity' has to go somewhere). Functions are not the atomically executed entities of the procedural world, and their algebraic composition is a very powerful method of abstraction, as described earlier. The type system is much more pervasive and complete, which is a double-edged sword: it forces you to think about the realities of a problem/solution much earlier (which feels like constraint), but it also enables the aforementioned much more meaningful 'conversation' with the compiler. The up-front requirement to conform to formal boundaries in expressing a solution costs a little more at the beginning, but pays back handsomely downstream - both in terms of the earlier/deeper accrued understanding of the problem domain, but also the much higher likelihood that better abstractions have been selected. As any first year Comp Sci undergraduate knows, the costs of correcting bad assumptions/design later in the software lifecycle are much higher than earlier. There are still choices about encapsulation in functional languages (which modules to create, how to deliver functionality in appropriately sized quanta to allow requisite reuse and generality of components), but the packets of semantic themselves, and the manner of their abstraction is far more fluid. The denotational quality of the syntax also has value for reasoning too, but that's another kettle o' fish.
At the end of the day, any developer will get a buzz out of using a tool that allows rapid manifestation of a good solution to the problem at hand (by some definition of "good"). The qualities of the functional approach however imbue a certain concision and confidence to the construction, and with the type system, really appear to have a pseudo-physical quality of "inserting the round peg into the round hole". So it is (I think) that when you stand back from the 'model' you have just assembled, there is a much more tangible quality to the functional construction - that it has been assembled from shapes, and that those shapes in turn had a robust definition. The whole model has been 'validated' by the type system (as an instance of a theorem prover), and you are now the proud owner of some new 'shape' with its tractable, testable, semantic, and its ability to be further glued into an even larger (and impressive?) construct, with some degrees of freedom about which vertices and edges 'stick out' to be perceived on the surface of the larger creation.
Whatever, dear reader, you may adjudge as the real reasons for my trippy experiences, I'm guessing that most developers who take the time to really understand what functional languages offer are likely to come away from the experience (and hopefully some real practice) appreciating some aspects of the foregoing. I'm not personally one of those who would use a functional language for every problem (at least the current batch, with the current libraries available, to say nothing of available developer skill sets), but I'm beyond persuaded as to the very real advantages they offer on certain problems, as a component of applications. Perhaps it is a growing appreciation of this sentiment that is driving the apparent uptake of what I'll loosely call 'functional concepts' within mainstream languages, or extensions thereof. Lambdas/blocks, closures etc. have appeared in Python, Ruby, Scala, C#/LINQ and (maybe) Java. It will be fascinating to see how these fare and evolve as embedded features of languages that are centred around long standing procedural concepts. Certainly these features allow, and even encourage, a more functional style for use where appropriate. However, the basic tenets of a language are a critical factor, and so far these languages are a far cry from the algebraically typed world, combinatorial construction and semantics of a modern lazy functional language.
Right. Back to the OO world now then...
[glow fades away]
Wednesday, February 27, 2008
Tuesday, February 26, 2008
Xcode 3.1 - come quickly!
As much as I've grown to really like much about Xcode and friends, unsurprisingly for a 'point zero' release, there are many annoying foibles, OK bugs, to be found lurking.
I'm quite sure that Apple's reasoning about the release of Leopard would not have extended to the quality of its developer tools - beyond fixing the known critical bugs of course, Leopard was not going to be held up by a lack of complete polish in software that only developers care about. This really shows in what must be the number of medium and lower severity issues that remain in the 3.0 tool suite. A lot of these problems are undeniably jarring, with potential costs in working around the problem (or disappointment in having to avoid a feature that would be convenient or improve productivity) - but they have clearly done a good job of removing the crashers.
Nevertheless, as has been mooted in other places, it is now four months since the release of Leopard and we have yet to have an incremental release of the Developer Tools. In all likelihood an update is imminent as a part of the upcoming iPhone development kit, so perhaps we don't have long to wait. I can only hope that the Xcode tool development group at Apple weren't so sequestered onto iPhone tooling that they haven't had the time to plough through some of the medium and low priority bugs. We'll see I guess.
While there are probably a half-dozen 'quirks' in the 3.0 tools that I have learnt to avoid, none of the tool limitations are as annoying as the issues that remain in Interface Builder (as I'm mentioned before). As I have been spending a deal of time recently knocking up UI for my application, these issues have really been getting under my skin.
The most heinous issue IMHO (!) is the lack of any real ability to re-parent view hierarchies in the IB3. Several times, I have made the mistake of building initial/prototype UI by constructing views in a top-down manner (i.e. split view, a tree control on the left, another split view on the right, then further views beneath that), only to get badly stung by the limitations in IB3 to rearrange things. Split views, for instance, are infuriating. You can easily enough create one, by first creating the left/right views, selecting them and then doing "Embed objects in... -> Split view" (I'll refrain from commenting much on how this creation methodology is odd when you are otherwise creating hierarchy top-down). The children you 'embed' might be some table controls, for example. So initially all is well, but then you realise that you didn't want the right hand table control at the top level of the right-hand content of the split view - perhaps you need another splitter, or simply to add some peer controls with the table. You would think you could do one of the following things:
- Morph the table view (I guess the top view of that cluster - its scroll view) into a new container (box, custom view, ...)
- Insert a new view 'between' the table view and the scroll view (essentially replace the right hand content of the scroller and let the existing view there become parented to this new view)
- Temporarily pop (or cut) the table view 'out' of the right pane of the scroll view, in order to drop another view in its place, before dragging the table view back onto this as the first child
Well, in the case of split views, none of the above is possible. Once a view is glued into a Split view (on either the left or right panel), it seems that it is impossible to remove. The only solution I have found is to delete the entire split view (with its descendants) and start over. After even a moderate amount of flying around Interface Builder's inspector to set up attributes/bindings etc., this is very frustrating - and often the process of setting up the views 'just right', with positioning or correct behaviours can represent a lot of iterations and remembering all the settings to recreate them again in another instance of the same views is quite tiresome. It seems that in IB3 the creation of split views is essentially atomic, and in order to ensure flexibility I have developed the habit of always putting a custom view in the left/right slots, irrespective of what I actually think I might need to build beneath these views.
Aside from this egregious case of IB3 lacking some rather critical functionality, there are other limitations with re-parenting. There are situations where you need to restructure a subtree of views, and want to preserve some of of the existing view hierarchy, but don't have the new parent to move it to yet. You can cut and paste views, perhaps parking the view hierarchy you want to preserve at the top level of the NIB while you rearrange the new environment for that part of the UI. However, more often than not, when you come to drop the UI back into the new parent, you will find that many of the settings have reset (attributes and bindings).
Despite the foregoing, the whole NeXTstep approach to UI (NIBs, frameworks) has a great deal to be admired. IB3 is also (in general) much improved over its forebear, but clearly as a "complete rewrite", it does suffer somewhat from version-1-ism. Fingers crossed, that will be a complaint that will be short-lived and we'll see marked improvements in the next point release.
I'm quite sure that Apple's reasoning about the release of Leopard would not have extended to the quality of its developer tools - beyond fixing the known critical bugs of course, Leopard was not going to be held up by a lack of complete polish in software that only developers care about. This really shows in what must be the number of medium and lower severity issues that remain in the 3.0 tool suite. A lot of these problems are undeniably jarring, with potential costs in working around the problem (or disappointment in having to avoid a feature that would be convenient or improve productivity) - but they have clearly done a good job of removing the crashers.
Nevertheless, as has been mooted in other places, it is now four months since the release of Leopard and we have yet to have an incremental release of the Developer Tools. In all likelihood an update is imminent as a part of the upcoming iPhone development kit, so perhaps we don't have long to wait. I can only hope that the Xcode tool development group at Apple weren't so sequestered onto iPhone tooling that they haven't had the time to plough through some of the medium and low priority bugs. We'll see I guess.
While there are probably a half-dozen 'quirks' in the 3.0 tools that I have learnt to avoid, none of the tool limitations are as annoying as the issues that remain in Interface Builder (as I'm mentioned before). As I have been spending a deal of time recently knocking up UI for my application, these issues have really been getting under my skin.
The most heinous issue IMHO (!) is the lack of any real ability to re-parent view hierarchies in the IB3. Several times, I have made the mistake of building initial/prototype UI by constructing views in a top-down manner (i.e. split view, a tree control on the left, another split view on the right, then further views beneath that), only to get badly stung by the limitations in IB3 to rearrange things. Split views, for instance, are infuriating. You can easily enough create one, by first creating the left/right views, selecting them and then doing "Embed objects in... -> Split view" (I'll refrain from commenting much on how this creation methodology is odd when you are otherwise creating hierarchy top-down). The children you 'embed' might be some table controls, for example. So initially all is well, but then you realise that you didn't want the right hand table control at the top level of the right-hand content of the split view - perhaps you need another splitter, or simply to add some peer controls with the table. You would think you could do one of the following things:
- Morph the table view (I guess the top view of that cluster - its scroll view) into a new container (box, custom view, ...)
- Insert a new view 'between' the table view and the scroll view (essentially replace the right hand content of the scroller and let the existing view there become parented to this new view)
- Temporarily pop (or cut) the table view 'out' of the right pane of the scroll view, in order to drop another view in its place, before dragging the table view back onto this as the first child
Well, in the case of split views, none of the above is possible. Once a view is glued into a Split view (on either the left or right panel), it seems that it is impossible to remove. The only solution I have found is to delete the entire split view (with its descendants) and start over. After even a moderate amount of flying around Interface Builder's inspector to set up attributes/bindings etc., this is very frustrating - and often the process of setting up the views 'just right', with positioning or correct behaviours can represent a lot of iterations and remembering all the settings to recreate them again in another instance of the same views is quite tiresome. It seems that in IB3 the creation of split views is essentially atomic, and in order to ensure flexibility I have developed the habit of always putting a custom view in the left/right slots, irrespective of what I actually think I might need to build beneath these views.
Aside from this egregious case of IB3 lacking some rather critical functionality, there are other limitations with re-parenting. There are situations where you need to restructure a subtree of views, and want to preserve some of of the existing view hierarchy, but don't have the new parent to move it to yet. You can cut and paste views, perhaps parking the view hierarchy you want to preserve at the top level of the NIB while you rearrange the new environment for that part of the UI. However, more often than not, when you come to drop the UI back into the new parent, you will find that many of the settings have reset (attributes and bindings).
Despite the foregoing, the whole NeXTstep approach to UI (NIBs, frameworks) has a great deal to be admired. IB3 is also (in general) much improved over its forebear, but clearly as a "complete rewrite", it does suffer somewhat from version-1-ism. Fingers crossed, that will be a complaint that will be short-lived and we'll see marked improvements in the next point release.
Friday, February 15, 2008
Using Core Data in NSArray controllers with a parameterised fetch predicate
Today I was wondering if it was possible to 'restart' the master->detail chain of a set of controls, by using a Core Data query (fetch predicate) in a controller, but parameterising the query with a value obtained from the selection in another, 'upstream' controller.
Normally, master->detail UIs are created by simply binding a selection from a master controller to the contents of a detail controller, and selecting some sub-component of the selection object(s) as the detail Model Key Path.
You can type any valid fetch predicate into the controller's properties panel, but in order to do what I want, you would need to be able to resolve a key from the 'upstream' object. That's the bit I don't think you can straightforwardly achieve (at the moment).
There are certainly a good many ways to actually achieve what I need - including binding the detail content to a custom controller key and having the get accessor for this key derive its results from the upstream selection content (which naturally it would need to observe in order to signal a change on its own key). The thing about how bindings work in general though is that they are so convenient and powerful, often requiring NO code in the application at all, so it's tempting to look if there's some way to contrive to get the required key to test in that filter predicate somehow, without resorting to 'external code'...
Normally, master->detail UIs are created by simply binding a selection from a master controller to the contents of a detail controller, and selecting some sub-component of the selection object(s) as the detail Model Key Path.
You can type any valid fetch predicate into the controller's properties panel, but in order to do what I want, you would need to be able to resolve a key from the 'upstream' object. That's the bit I don't think you can straightforwardly achieve (at the moment).
There are certainly a good many ways to actually achieve what I need - including binding the detail content to a custom controller key and having the get accessor for this key derive its results from the upstream selection content (which naturally it would need to observe in order to signal a change on its own key). The thing about how bindings work in general though is that they are so convenient and powerful, often requiring NO code in the application at all, so it's tempting to look if there's some way to contrive to get the required key to test in that filter predicate somehow, without resorting to 'external code'...
Thursday, February 14, 2008
NSValueTransformers update
Well, my experiment with NSValueTransformers for NSIndexPath morphing between two NSOutlineViews was interesting.
The plan was to have a subclass of NSValueTransformer that required to outlet connections to the two trees (to be more precise, to the NSTreeControllers), and then have instances live in the NIB file, all self contained, with the two NSTreeControllers binding to each other's selections (NSArray of NSIndexPaths) via the these transformers. The transformers' job is to reexpress the selection in terms of the local tree's outline, using the set of model (represented) objects as the underlying meaning of selection.
So, this idea is designed to work in circumstances (that I have) where certain constraints apply:
1. The trees refer to the same model objects (at least the selectable ones), though arranged differently
2. The trees are either single selection, or its OK to select ALL the occurances of nodes representing a given model object (even if this potentially widens the selection when returned to the source tree)
I created a small collection of NSValueTransformer subclasses: to handle converting to/from a represented object, to handle targeting the first matching object in the destination tree, and to handle selecting ALL matching nodes in the target tree.
When I came to try this out as intended, I met a hitch (naturally!). I had intended for the relevant instances of these transformers to live in the NIB, where they could be handily connected/bound to the local objects without requiring construction in the main body of code. As such they would be nicely encapsulated as UI behaviour. However, it turns out that there is no way to easily have two instances registering two different transformer names. NSValueTransformers work by registering a named instance of themselves with a central repository, and the registered name is then used to refer to them in binding parameters. I needed to register two instances: one with tree1 and tree2 as source and destination, and one with tree2 and tree1 as source and destination (i.e. reversed). Without creating a new Interface Builder palette object that contains a design-time property for the name an instance should register, you would have to have two *classes* to express the source and destination differences required. Even then, from a log message, it seems that the transformers need to be registered before the objects that use them are even awoken from the NIB, and that either means performing the registration in the -init method, or giving up on the idea of storing instances in the NIB altogether, and registering these objects in the main code (along with code to set the two trees into the transformer instances that are created).
So, all in all the circumstances were contriving to make this so it was barely working, with the need to register a name for the transformer instances really getting in the way of achieving the goals. This is somewhat annoying, as when you actually contrived to get the transformers initialised in time (separate classes), things worked rather well! It's clear that the real design intention for NSValueTransformers are more as 'global functions' than an application can initialise and register early, and then easily reference in binding parameters.
Anyway, I decided transformers were just a little too awkward for what I wanted, and set about thinking about an alternative.
I still wanted an object I could just drop into the NIB and connect up the relevant trees for synchronised behaviour. I decided to implement what I called a "TreeIndexPathsExchange". This is an object to which NSTreeControllers could directly bind their selection state, and from which they would receive new selection state (from changes in other controllers that were also bound). This is a outline of what I created:
- First, a set of 7 (arbitrary number) IBOutlets of type (NSTreeController *) to which tree controllers could be bound in a NIB. The idea is that up to 7 controllers could be linked to synchronise selection between them. Each outlet is labelled 'treeX' where X is 0 through 6.
- A ivar holding the set of currently selected model (represented) objects. These are common across all trees and this set represents what is truly selected in common.
- Next, an implementation of synthetic properties treeX (where X is an ordinal representing the 'port' that a particular tree is bound on). This was done by overriding -valueForKey: and -setValue:forKey: in the implementation and handling all keys that conform to the pattern "treeX" where the X is allowed to go up to the top port number (currently 6). The implementation of the getter for a treeX is to use take the set of currently selected model objects and search for these in tree connected on 'port' X. The nodes that are found to represent these objects are then converted to NSIndexPaths and the array of these is returned. The setter for a treeX property obtains all the represented objects in the tree that is providing the selection, and if these are different from the set we have cached, then we update the cache, and then notify ALL the other connected 'ports' that their value has changed. This is done by looping through from 0 to 6, checking if there's an NSTreeController attached to the outlet for the port, and if so, we do a -willChangeValueForKey: and -didChangeValueForKey: on the key "treeX".
This solution is great for something that can just live in a NIB, and is quite elegant. On testing, it worked first time with one exception - when going from a tree that only had on node representing an object, to a tree that had several nodes representing an object, I noticed that no selection was being made. The debugger seemed to demonstrate that the right things were happening, but the selection in the 'target' tree was resolutely being left with nothing selected. It occurred to me that both trees were configured for single selection, and that maybe the attempt to set multiple NSIndexPaths was causing no nodes _at all_ to be selected. I artificially restricted the number of index paths to 1 (the first matching node) an lo! everything was working again. I'm not sure why NSOutlineView rejects all selection when presented with multiple paths in its single selection mode (I would perhaps have expected it to select the first node in the list, or the one 'highest' in the tree). Maybe there's a good reason, but the current behaviour was cramping the style of this facility! What was needed was a way to determine whether a tree would accept multiple selections and, if not, pick a single NSIndexPath to send it. Unfortunately, the objects bound to the exchange are NSTreeControllers, and these have absolutely no knowledge of the selection mode set on any NSOutlineViews that might be bound to them. In order to allow the exchange to be able to make judgements such as this (whether to send a single, or multiple selections through), I added more IBOutlets, one for each treeX outlet, called viewX (X being 0-6). These are type (NSOutlineView *) and allow an appropriate outline control to be optionally connected for a port X, in complement to the required NSTreeController. In the case of finding a view connected, the exchange can query the outline view as it compiles a new selection to send to its NSTreeController, and can elect to send only one NSIndexPath if the control only supports single selection.
So, today ended with a nice behaviour object to put in my NIB(s) to synchronise outline views (in the scenarios I'm currently interested in). The result is, IMHO, very 'Cocoa-y' in that application code is decluttered of this UI behaviour detail, which is all nicely encapsulated in the relevant NIB. In any case, I got a buzz out of it :-)
The plan was to have a subclass of NSValueTransformer that required to outlet connections to the two trees (to be more precise, to the NSTreeControllers), and then have instances live in the NIB file, all self contained, with the two NSTreeControllers binding to each other's selections (NSArray of NSIndexPaths) via the these transformers. The transformers' job is to reexpress the selection in terms of the local tree's outline, using the set of model (represented) objects as the underlying meaning of selection.
So, this idea is designed to work in circumstances (that I have) where certain constraints apply:
1. The trees refer to the same model objects (at least the selectable ones), though arranged differently
2. The trees are either single selection, or its OK to select ALL the occurances of nodes representing a given model object (even if this potentially widens the selection when returned to the source tree)
I created a small collection of NSValueTransformer subclasses: to handle converting to/from a represented object, to handle targeting the first matching object in the destination tree, and to handle selecting ALL matching nodes in the target tree.
When I came to try this out as intended, I met a hitch (naturally!). I had intended for the relevant instances of these transformers to live in the NIB, where they could be handily connected/bound to the local objects without requiring construction in the main body of code. As such they would be nicely encapsulated as UI behaviour. However, it turns out that there is no way to easily have two instances registering two different transformer names. NSValueTransformers work by registering a named instance of themselves with a central repository, and the registered name is then used to refer to them in binding parameters. I needed to register two instances: one with tree1 and tree2 as source and destination, and one with tree2 and tree1 as source and destination (i.e. reversed). Without creating a new Interface Builder palette object that contains a design-time property for the name an instance should register, you would have to have two *classes* to express the source and destination differences required. Even then, from a log message, it seems that the transformers need to be registered before the objects that use them are even awoken from the NIB, and that either means performing the registration in the -init method, or giving up on the idea of storing instances in the NIB altogether, and registering these objects in the main code (along with code to set the two trees into the transformer instances that are created).
So, all in all the circumstances were contriving to make this so it was barely working, with the need to register a name for the transformer instances really getting in the way of achieving the goals. This is somewhat annoying, as when you actually contrived to get the transformers initialised in time (separate classes), things worked rather well! It's clear that the real design intention for NSValueTransformers are more as 'global functions' than an application can initialise and register early, and then easily reference in binding parameters.
Anyway, I decided transformers were just a little too awkward for what I wanted, and set about thinking about an alternative.
I still wanted an object I could just drop into the NIB and connect up the relevant trees for synchronised behaviour. I decided to implement what I called a "TreeIndexPathsExchange". This is an object to which NSTreeControllers could directly bind their selection state, and from which they would receive new selection state (from changes in other controllers that were also bound). This is a outline of what I created:
- First, a set of 7 (arbitrary number) IBOutlets of type (NSTreeController *) to which tree controllers could be bound in a NIB. The idea is that up to 7 controllers could be linked to synchronise selection between them. Each outlet is labelled 'treeX' where X is 0 through 6.
- A ivar holding the set of currently selected model (represented) objects. These are common across all trees and this set represents what is truly selected in common.
- Next, an implementation of synthetic properties treeX (where X is an ordinal representing the 'port' that a particular tree is bound on). This was done by overriding -valueForKey: and -setValue:forKey: in the implementation and handling all keys that conform to the pattern "treeX" where the X is allowed to go up to the top port number (currently 6). The implementation of the getter for a treeX is to use take the set of currently selected model objects and search for these in tree connected on 'port' X. The nodes that are found to represent these objects are then converted to NSIndexPaths and the array of these is returned. The setter for a treeX property obtains all the represented objects in the tree that is providing the selection, and if these are different from the set we have cached, then we update the cache, and then notify ALL the other connected 'ports' that their value has changed. This is done by looping through from 0 to 6, checking if there's an NSTreeController attached to the outlet for the port, and if so, we do a -willChangeValueForKey: and -didChangeValueForKey: on the key "treeX".
This solution is great for something that can just live in a NIB, and is quite elegant. On testing, it worked first time with one exception - when going from a tree that only had on node representing an object, to a tree that had several nodes representing an object, I noticed that no selection was being made. The debugger seemed to demonstrate that the right things were happening, but the selection in the 'target' tree was resolutely being left with nothing selected. It occurred to me that both trees were configured for single selection, and that maybe the attempt to set multiple NSIndexPaths was causing no nodes _at all_ to be selected. I artificially restricted the number of index paths to 1 (the first matching node) an lo! everything was working again. I'm not sure why NSOutlineView rejects all selection when presented with multiple paths in its single selection mode (I would perhaps have expected it to select the first node in the list, or the one 'highest' in the tree). Maybe there's a good reason, but the current behaviour was cramping the style of this facility! What was needed was a way to determine whether a tree would accept multiple selections and, if not, pick a single NSIndexPath to send it. Unfortunately, the objects bound to the exchange are NSTreeControllers, and these have absolutely no knowledge of the selection mode set on any NSOutlineViews that might be bound to them. In order to allow the exchange to be able to make judgements such as this (whether to send a single, or multiple selections through), I added more IBOutlets, one for each treeX outlet, called viewX (X being 0-6). These are type (NSOutlineView *) and allow an appropriate outline control to be optionally connected for a port X, in complement to the required NSTreeController. In the case of finding a view connected, the exchange can query the outline view as it compiles a new selection to send to its NSTreeController, and can elect to send only one NSIndexPath if the control only supports single selection.
So, today ended with a nice behaviour object to put in my NIB(s) to synchronise outline views (in the scenarios I'm currently interested in). The result is, IMHO, very 'Cocoa-y' in that application code is decluttered of this UI behaviour detail, which is all nicely encapsulated in the relevant NIB. In any case, I got a buzz out of it :-)
Tuesday, February 12, 2008
NSOutlineView mysteries
I gather from various mailing list histories that NSOutlineViews in Leopard are considerably improved over times of yore (Tiger)... and as a side note, it seems Leopard is a really great time to be learning Cocoa - not just because of Objective-C 2.0, but the considerable number of improvements in Cocoa. Many little holes have been filled and conveniences added, to say nothing of the big new features.
For me today it was time to get to grips with tree controls, i.e. NSOutlineView. Having relatively easily hooked up my Core Data model to one NSOutlineView, I proceeded to add yet another (in a tab view) to show the same model in a different way. The way that NSTreeController bindings work is really nice, in that you can have several 'protocols' (sort of 'key schemas') stamped on the model objects that allow descent through the model hierarchically in different ways.
Custom cells went well too, and I like the new SourceView style that you can turn on to get the similar appearance to iTunes, Finder etc. for those 'left pane' hierarchy navigators.
I then came to trying to hook up the autosave feature of NSOutlineView. In this case, autosave remembers a user's outline state (expanded/collapsed nodes) and can restore the users state next time they launch the application. This is one of those great framework features, providing wonderful user experience almost for free. Well, it would be if I could figure out the two methods I have to write to support it!
Once you provide a key to save the state under in the user's application preferences, you have two symmetrical methods to write. One of these is sent a tree 'item' and you are supposed to return an object 'suitable for archiving', and the other does the reverse - i.e. supposedly maps an archive object to a tree item.
Now, the documentation is a little vague, so I consulted Google for some mailing list hits. The one decent write-up I found suggested that NSKeyedArchiver and its reciprocal unarchiver could simply be applied to the item to convert in each direction. That seemed like a reasonable suggestion to me, so I tried it. Unfortunately, it doesn't work - at least not in Leopard.
Leopard has introduced a 'public' proxy object for tree nodes that NSTreeController creates around the model objects it finds and introduces into the NSOutlineView. Now, I understand that there was always some kind of 'opaque' proxy object before Leopard, but if the post is to be believed, then this object was able to archive and unarchive itself (i.e. it must have conformed to the NSCoding protocol). Nowadays, at least, this isn't the case, and the requisite error messages appear on the console.
So, if you cannot directly transform a tree item into something that can be inserted into the User Preferences, what should you be doing? In fact, the biggest question isn't so much what you could create as an archive object, but rather what exactly an unarchived object is supposed to be in relation to the 'new' outline that one will find when depersisting. Am I supposed to search for the exact tree item matching some key I wrote out? Is it sufficient to build a new NSTreeItem and have it match a different instance already in the tree - using associative semantics? The docs don't elucidate, and I have more experimenting to do tomorrow.
While thinking about this problem, it occurred to me that if I had to actually go find the matching NSTreeNode in the current outline content that matched some saved properties, then I'd need to be able to search or filter the existing nodes. However, I could find no API that would do this. Not on NSOutlineView, NSTreeController, NSTableView or NSObjectController (the latter being the two pertinent superclasses). This came as a bit of a shock, suggesting as it does that you are not supposed to ever need to do this, and perhaps reinforcing the notion that one shouldn't have to search the current tree to find matching instances of some persisted items in order to reset their expanded state.
At that time I decided to park the autosave problem and move on to another issue - that of synchonising the selection between my two outlines in the two tab views. The problem here is that, for any selection, the index paths describing the selected items will be different - because the outline schemes are different - even though the selectable items in the two trees are the same. Whatever eventing scheme is used to detect a change in selection (more about this in a moment), the selection from one tree needs to be transformed into the other outlines scheme. Because the relationship between them happens not to be a simple function, this effectively means being able to look up the selected object from one outline in the other tree. Oops, we're back here again.
This time I dove right in and implemented a fairly vanilla tree search routine and packaged it as an NSTreeController category. All the time suspicious that I was doing something unnecessary and for which there *must* be a more elegant solution, it is nonetheless true that Leopard's addition of NSTreeNode makes it easy enough to search through items that are already in the tree. I'm not exactly sure how NSTreeNode works in conjunction with the parsimonious fetching of model objects by NSTreeController, though it possibly faults the requesting of additional items from NSTreeController if you ask for child nodes it hasn't got yet. If this is how it works it could be a bad thing in some kinds of search, but in any case I achieved a working search (based on the represented model object of the tree item).
With this ability to find equivalent nodes in another tree, I was able to register Key Value Observers on the selected objects of each tree and carefully update the other tree (if a different node was selected - so as to avoid infinite loops). Job done.
Looking at the code though, I'm a little dissatisfied (!). Isn't binding supposed to allow you to connect objects together directly in an (almost) code free way. As I understand it, it is only the asymmetrical selection that is preventing this in this case. What if this could be overcome in the binding itself? Well, bindings support a concept called value transformers, and this sounds like just the medicine to convert between two selection index paths in the two trees. Moreover, my understanding is that KVB handles cyclicity automatically, so it is even more appealing to be synchonising two views in this way.
So, tomorrow I will begin with an experiment to create a subclass of NSValueTransformer. This will have an instance that lives in the NIB file itself, with outlets to connect to the two trees. It will register itself as an active transfomer on -awakeFromNib, and be referenced in selection bindings between the two outline views. If that works, it will be one of those sweet Cocoa moments - and I'll just have the autosave (de)persistence left to grok :-)
For me today it was time to get to grips with tree controls, i.e. NSOutlineView. Having relatively easily hooked up my Core Data model to one NSOutlineView, I proceeded to add yet another (in a tab view) to show the same model in a different way. The way that NSTreeController bindings work is really nice, in that you can have several 'protocols' (sort of 'key schemas') stamped on the model objects that allow descent through the model hierarchically in different ways.
Custom cells went well too, and I like the new SourceView style that you can turn on to get the similar appearance to iTunes, Finder etc. for those 'left pane' hierarchy navigators.
I then came to trying to hook up the autosave feature of NSOutlineView. In this case, autosave remembers a user's outline state (expanded/collapsed nodes) and can restore the users state next time they launch the application. This is one of those great framework features, providing wonderful user experience almost for free. Well, it would be if I could figure out the two methods I have to write to support it!
Once you provide a key to save the state under in the user's application preferences, you have two symmetrical methods to write. One of these is sent a tree 'item' and you are supposed to return an object 'suitable for archiving', and the other does the reverse - i.e. supposedly maps an archive object to a tree item.
Now, the documentation is a little vague, so I consulted Google for some mailing list hits. The one decent write-up I found suggested that NSKeyedArchiver and its reciprocal unarchiver could simply be applied to the item to convert in each direction. That seemed like a reasonable suggestion to me, so I tried it. Unfortunately, it doesn't work - at least not in Leopard.
Leopard has introduced a 'public' proxy object for tree nodes that NSTreeController creates around the model objects it finds and introduces into the NSOutlineView. Now, I understand that there was always some kind of 'opaque' proxy object before Leopard, but if the post is to be believed, then this object was able to archive and unarchive itself (i.e. it must have conformed to the NSCoding protocol). Nowadays, at least, this isn't the case, and the requisite error messages appear on the console.
So, if you cannot directly transform a tree item into something that can be inserted into the User Preferences, what should you be doing? In fact, the biggest question isn't so much what you could create as an archive object, but rather what exactly an unarchived object is supposed to be in relation to the 'new' outline that one will find when depersisting. Am I supposed to search for the exact tree item matching some key I wrote out? Is it sufficient to build a new NSTreeItem and have it match a different instance already in the tree - using associative semantics? The docs don't elucidate, and I have more experimenting to do tomorrow.
While thinking about this problem, it occurred to me that if I had to actually go find the matching NSTreeNode in the current outline content that matched some saved properties, then I'd need to be able to search or filter the existing nodes. However, I could find no API that would do this. Not on NSOutlineView, NSTreeController, NSTableView or NSObjectController (the latter being the two pertinent superclasses). This came as a bit of a shock, suggesting as it does that you are not supposed to ever need to do this, and perhaps reinforcing the notion that one shouldn't have to search the current tree to find matching instances of some persisted items in order to reset their expanded state.
At that time I decided to park the autosave problem and move on to another issue - that of synchonising the selection between my two outlines in the two tab views. The problem here is that, for any selection, the index paths describing the selected items will be different - because the outline schemes are different - even though the selectable items in the two trees are the same. Whatever eventing scheme is used to detect a change in selection (more about this in a moment), the selection from one tree needs to be transformed into the other outlines scheme. Because the relationship between them happens not to be a simple function, this effectively means being able to look up the selected object from one outline in the other tree. Oops, we're back here again.
This time I dove right in and implemented a fairly vanilla tree search routine and packaged it as an NSTreeController category. All the time suspicious that I was doing something unnecessary and for which there *must* be a more elegant solution, it is nonetheless true that Leopard's addition of NSTreeNode makes it easy enough to search through items that are already in the tree. I'm not exactly sure how NSTreeNode works in conjunction with the parsimonious fetching of model objects by NSTreeController, though it possibly faults the requesting of additional items from NSTreeController if you ask for child nodes it hasn't got yet. If this is how it works it could be a bad thing in some kinds of search, but in any case I achieved a working search (based on the represented model object of the tree item).
With this ability to find equivalent nodes in another tree, I was able to register Key Value Observers on the selected objects of each tree and carefully update the other tree (if a different node was selected - so as to avoid infinite loops). Job done.
Looking at the code though, I'm a little dissatisfied (!). Isn't binding supposed to allow you to connect objects together directly in an (almost) code free way. As I understand it, it is only the asymmetrical selection that is preventing this in this case. What if this could be overcome in the binding itself? Well, bindings support a concept called value transformers, and this sounds like just the medicine to convert between two selection index paths in the two trees. Moreover, my understanding is that KVB handles cyclicity automatically, so it is even more appealing to be synchonising two views in this way.
So, tomorrow I will begin with an experiment to create a subclass of NSValueTransformer. This will have an instance that lives in the NIB file itself, with outlets to connect to the two trees. It will register itself as an active transfomer on -awakeFromNib, and be referenced in selection bindings between the two outline views. If that works, it will be one of those sweet Cocoa moments - and I'll just have the autosave (de)persistence left to grok :-)
Ah... Bindings and Core Data
As kind of a balance to that last post, I have to say that the last few days exploits with bindings, with some of it intersecting with Core Data, have been a complete joy.
This is, of course, where the dynamic underpinnings of Cocoa/Obj-C really come into their own, and some of the elegant results are almost enough to induce weeping.
As a Cocoa n00b, I'm just beginning to appreciate the zen of Cocoa, but the recurring pattern is quite interesting:
1. Set out to achieve X, decide to investigate the 'right' Cocoa way of doing this
2. Quickly find promising area, excitement rises as I glimpse established patterns and 'depth' of the frameworks to solve the problem
3. Read guides, references - formulate specific vision of how the implementation will be. More excitement/anticipation...
4. Write-test-ARGGH!
5. MOMENTS OF FRUSTRATION (e.g. bindings wrong, nil being messaged somewhere, other unexpected details emerge)
6. *Enlightenment*
7. Joy. Usually bags more respect for NeXT and Apple (in terms of the general patterns of the solution). Occasionally moderated by remaining questions as to why a small feature or convenience is missing.
Like learning any significant library/framework, there's an uphill struggle, but the corpus of successes (and hence working samples that are really meaningful through personalised exercises) accrete rapidly as the general philosophies of the library start to take hold and judgement/first-guesses improve. From the number of guides I've worked through so far, I feel like I'm half way to a good broad-base of coverage of Cocoa - though there's detail everywhere that will remain to discovered on an as-needed basis.
The last time I got stuck into anything this large was probably the Java foundation libraries. Even then, I've been living with Java since 1.0, so have had a chance to accrete knowledge as the libraries have matured. The single biggest difference that stands out (and makes Cocoa much more challenging for the learner in many ways) is the dynamic nature of the language, with all the ensuing mistakes that are bound to happen as you are learning. Coupled with the somewhat simpler debugger (compared with something like the Eclipse tooling these days), and you are left using a fair bit of tracing code to test code paths, print values and learn about sequences of events in the complex library. Nevertheless, the aforementioned moments of 'respect' for the designers of the language and libraries provide ample payback for some of the frustrating moments.
This is, of course, where the dynamic underpinnings of Cocoa/Obj-C really come into their own, and some of the elegant results are almost enough to induce weeping.
As a Cocoa n00b, I'm just beginning to appreciate the zen of Cocoa, but the recurring pattern is quite interesting:
1. Set out to achieve X, decide to investigate the 'right' Cocoa way of doing this
2. Quickly find promising area, excitement rises as I glimpse established patterns and 'depth' of the frameworks to solve the problem
3. Read guides, references - formulate specific vision of how the implementation will be. More excitement/anticipation...
4. Write-test-ARGGH!
5. MOMENTS OF FRUSTRATION (e.g. bindings wrong, nil being messaged somewhere, other unexpected details emerge)
6. *Enlightenment*
7. Joy. Usually bags more respect for NeXT and Apple (in terms of the general patterns of the solution). Occasionally moderated by remaining questions as to why a small feature or convenience is missing.
Like learning any significant library/framework, there's an uphill struggle, but the corpus of successes (and hence working samples that are really meaningful through personalised exercises) accrete rapidly as the general philosophies of the library start to take hold and judgement/first-guesses improve. From the number of guides I've worked through so far, I feel like I'm half way to a good broad-base of coverage of Cocoa - though there's detail everywhere that will remain to discovered on an as-needed basis.
The last time I got stuck into anything this large was probably the Java foundation libraries. Even then, I've been living with Java since 1.0, so have had a chance to accrete knowledge as the libraries have matured. The single biggest difference that stands out (and makes Cocoa much more challenging for the learner in many ways) is the dynamic nature of the language, with all the ensuing mistakes that are bound to happen as you are learning. Coupled with the somewhat simpler debugger (compared with something like the Eclipse tooling these days), and you are left using a fair bit of tracing code to test code paths, print values and learn about sequences of events in the complex library. Nevertheless, the aforementioned moments of 'respect' for the designers of the language and libraries provide ample payback for some of the frustrating moments.
Interface Scrambler 3.0
... OK that's a tad unfair, but Interface Builder 3.0 is driving me crazy recently.
In general, the workflow and UI of V3.0 is significantly better than the prior incarnation (IMHO), but unfortunately 3.0 seems unfinished and rather buggy.
One of the biggest functional problems with IB3 IMO is the lack of a decent capability to reparent views - expect in simple cases. In the simple cases, you click and hold on a view in the spatial/layout designer. This "pops out" the selected view hierarchy, and you can then drag it to a new parent view. So far so good.
Any real interface however is likely to have containers like NSSplitViews and NSTabViews, and these seem to have special (arguably broken) editing rules.
For instance, I had created some prototype UI in a NIB a few days back, and had added an NSSplitView using the "Embed Objects In..." functionality, whereby you select two views, and IB3 adds the new container (split view) and putting each selected view as the child view each side of the split bar. So, creating these initially is no problem (at least in this bottom-up way).
I go on to make a considerable investment in time adding outlets, making connections, setting properties etc.
Then, when the prototype UI is working well enough, I realise I want to beef it up, to fill out the UI with other controls each side of the splitter. So, what I want to do is insert a new layer of views each side of the splitter, which will parent my existing UI and allow me to add adjacent views to that which already exists.
So, my first thought is "Embed Objects In..." again, selecting the current view snapped into one side of the splitter - and perhaps inserting a "Box" between the splitter and this view. But... no, when I select the view on one side of the splitter and go to the menu, it's all greyed out - no dice.
So, it occurs to me that my only option is to move the old split pane 'subview' out of the splitter, then add in the new 'box', then put the old view back onto this, as its new superview. OK, well, the way to reparent views, according to the IB3 manual, is to 'pop out' the child views as described earlier. However, it turns out that this doesn't work with split views. Apparently, they don't want to have an empty pane, so IB3 doesn't let you do this.
Suffice it to say, that I had to rebuild the split view from scratch, with a new custom view or box 'layer' in each side. Furthermore, copying and pasting view hierarchies appears to result in much of the set properties in the hierarchy being reset. The view hierarchy itself is preserved, but bindings and attributes are all lost - a considerable cost in time and frustration to recreate exactly.
I would dearly love to be able to reparent views in the NIB outline window rather than the layout window, but there's no support for this at all.
Other sundry issues I've experienced:
- Setting a new binding on a view deselects the view you're working with and effectively defocuses the inspector you're working in.
- If you set names on some views (Scroll views and Outline views I tried today) they are not saved (or if they are, then they're reset when the NIB is reloaded).
- Occasionally I have had other properties reset (often bindings)
- IB3 can get into a state where it refuses to 'see' a view in the NIB file overview (hierarchy view) and/or the inspector - whereas the view is clearly 'there' in the layout window.
I've wondered if some of this is down to the flavour of NIBs I've been using (Leopard only, but 2.0 format). Unfortunately, there hasn't been enough time in the day to try various configurations - and I posit that IB should be that broken with any format it supports anyway.
All in all, IB 3.0 seems to have escaped a little too early. I guess Apple can be forgiven for this in the sense that it was surely better for it not to have to hold up the release of Leopard. It's also effectively a "version 1", being a complete rewrite of Interface Builder - the first rewrite of this venerable tool in many years, by all accounts. Still, as I see it, it's currently the weakest part of the development tool suite, and I really hope that we'll see a rev. of the development tools really soon, with the attendant bug fixes (at least), if not some improved functionality for reparenting views and inserting view between existing parent-child views relationships.
Possibly the next release of the dev tools will be when Apple release the iPhone tooling - and naturally I hope the dev tools group has had at least some cycles to improve the core tools, even if they've had to be involved in supporting a lot of iPhone development features too. Otherwise I'll have to be even more patient to see some of the bugs I've raised on these issues get fixed - and unfortunately I can see a lot of IB3 usage coming up in the next little while on my project :-(
In general, the workflow and UI of V3.0 is significantly better than the prior incarnation (IMHO), but unfortunately 3.0 seems unfinished and rather buggy.
One of the biggest functional problems with IB3 IMO is the lack of a decent capability to reparent views - expect in simple cases. In the simple cases, you click and hold on a view in the spatial/layout designer. This "pops out" the selected view hierarchy, and you can then drag it to a new parent view. So far so good.
Any real interface however is likely to have containers like NSSplitViews and NSTabViews, and these seem to have special (arguably broken) editing rules.
For instance, I had created some prototype UI in a NIB a few days back, and had added an NSSplitView using the "Embed Objects In..." functionality, whereby you select two views, and IB3 adds the new container (split view) and putting each selected view as the child view each side of the split bar. So, creating these initially is no problem (at least in this bottom-up way).
I go on to make a considerable investment in time adding outlets, making connections, setting properties etc.
Then, when the prototype UI is working well enough, I realise I want to beef it up, to fill out the UI with other controls each side of the splitter. So, what I want to do is insert a new layer of views each side of the splitter, which will parent my existing UI and allow me to add adjacent views to that which already exists.
So, my first thought is "Embed Objects In..." again, selecting the current view snapped into one side of the splitter - and perhaps inserting a "Box" between the splitter and this view. But... no, when I select the view on one side of the splitter and go to the menu, it's all greyed out - no dice.
So, it occurs to me that my only option is to move the old split pane 'subview' out of the splitter, then add in the new 'box', then put the old view back onto this, as its new superview. OK, well, the way to reparent views, according to the IB3 manual, is to 'pop out' the child views as described earlier. However, it turns out that this doesn't work with split views. Apparently, they don't want to have an empty pane, so IB3 doesn't let you do this.
Suffice it to say, that I had to rebuild the split view from scratch, with a new custom view or box 'layer' in each side. Furthermore, copying and pasting view hierarchies appears to result in much of the set properties in the hierarchy being reset. The view hierarchy itself is preserved, but bindings and attributes are all lost - a considerable cost in time and frustration to recreate exactly.
I would dearly love to be able to reparent views in the NIB outline window rather than the layout window, but there's no support for this at all.
Other sundry issues I've experienced:
- Setting a new binding on a view deselects the view you're working with and effectively defocuses the inspector you're working in.
- If you set names on some views (Scroll views and Outline views I tried today) they are not saved (or if they are, then they're reset when the NIB is reloaded).
- Occasionally I have had other properties reset (often bindings)
- IB3 can get into a state where it refuses to 'see' a view in the NIB file overview (hierarchy view) and/or the inspector - whereas the view is clearly 'there' in the layout window.
I've wondered if some of this is down to the flavour of NIBs I've been using (Leopard only, but 2.0 format). Unfortunately, there hasn't been enough time in the day to try various configurations - and I posit that IB should be that broken with any format it supports anyway.
All in all, IB 3.0 seems to have escaped a little too early. I guess Apple can be forgiven for this in the sense that it was surely better for it not to have to hold up the release of Leopard. It's also effectively a "version 1", being a complete rewrite of Interface Builder - the first rewrite of this venerable tool in many years, by all accounts. Still, as I see it, it's currently the weakest part of the development tool suite, and I really hope that we'll see a rev. of the development tools really soon, with the attendant bug fixes (at least), if not some improved functionality for reparenting views and inserting view between existing parent-child views relationships.
Possibly the next release of the dev tools will be when Apple release the iPhone tooling - and naturally I hope the dev tools group has had at least some cycles to improve the core tools, even if they've had to be involved in supporting a lot of iPhone development features too. Otherwise I'll have to be even more patient to see some of the bugs I've raised on these issues get fixed - and unfortunately I can see a lot of IB3 usage coming up in the next little while on my project :-(
Friday, February 8, 2008
Core Data and uniqueness
Being such a newcomer to the many and varied wonders of Cocoa, I'm perennially accosted with that feeling that I *must* be missing something when I'm challenged by some apparent missing feature. So far, about half the time, my continued quest to learn how to achieve something will be rewarded with some new epiphany - a new pattern, the discovery of some previously arcane knowledge. The other half of the time I cave in and achieve what I'm trying to accomplish with some belt and braces - still wondering whether whether I'll kick myself at some future point for having missing the provision of some elegant solution.
So it is that I've recently been wondering about uniqueness in Core Data.
Now Apple are quite clear about the nature of Core Data (at least as it stands today). Despite the entity relationship models, the prepared parameterised fetch requests (queries) and the ability to use a SQL database for storage, Core Data is not a general purpose database. It is designed, of course, to provide an elegant way to persist your application's internal state in a way that is natural and requires the minimal amount of overhead (notwithstanding the need to conform to its design patterns).
I've been very impressed with Core Data (so far). I'm lucky enough to be beginning my Cocoa career with Leopard, and like a lot of the Mac OS X frameworks, Core Data in Leopard has clearly matured very nicely into a highly capable and general facility. At this time, my main data model spans a half dozen pages and uses a good many of the features available (inheritance, to-one/to-many relationships, delete rules and a little validation with more to come). For most of what my app does, modelling its data this way is clearly superior. However, I have been surprised by a couple of things that seem like omissions, but per the foregoing, leave me wondering whether I'm missing the 'right' way to approach the problem.
One such item is a need to store singleton instances of some entity. My application has global state that should be persisted, which is not necessarily per-user (though naturally I have some of those too). It would be nice to be able to create an entity to represent a unique object that will record this state - and declare to Core Data that there can be only one instance of this (at a given version of the model). Yet, I know of no way to achieve this. Of course, one can live without this formal uniqueness, and instead date-stamp an instance, and have the application 'housekeep' any excessive number of instances (perhaps clearing away all but the last written instance), but...
There's a rather more fundamental kind of uniqueness in data of course, that of unique keys - and again Core Data has no way that I know of to express that an attribute will contain a unique key. In Core Data, all the combinations (tuples) of matching data will be returned on a query, and one imagines that there can be no opimisations in how an underlying data base is queried when such fundamental metadata is missing. I got a little excited when I first saw the "Indexed" check box on what happened to be a String attribute in the model designer, but looking up what this did revealed nothing more than the vague "Indicates that an attribute may be useful for searching".
Even if Core Data itself has no formal way of indicating uniqueness or key values, you certainly need to be able to determine this from time to time in your application. For example, if your model records "Customer" in various places, you are likely to have the same actual Customer represented by multiple instances of the Customer entity (one perhaps attached to a 'recent calls' part of the model, and one attached to 'sales'). Now, because you cannot formally uniquify the details of a particular Customer (with a key like 'customer number'), if you were to query for all the Customers in your data, you will end up with an Array of Customers with 'clones'. So, how do you turn an array of objects into an, er... set of unique objects (by some definition of unique). Of course, normally a set collection does this quite handily, through the expedient of defining appropriate hash and isEqual methods on the objects to ascribe identity. Cocoa certainly has such a thing in NSSet/NSMutableSet. So recovering uniques, even if the data framework can't, should be a walk in the park, right?
Well, in one of those unsatisfying moments I alluded to earlier, you soon stumble over what looks like a major flaw in Core Data. Core Data manages objects that derive from NSManagedObject, and the documentation clearly states that -hash and -isEqual: are reserved for Core Data's use (i.e. you cannot override these methods as you can, and often do, override them as an NSObject subclass). Oops. Try as I might, I have not yet found any canonical way to reasonably filter out uniques from an array/set of NSManagedObjects (whether obtained from a to-many relationship, a fetch request or any other way). The most obvious solution is barred, given the reliance of NSSet on the out-of-bounds -hash and -isEqual:, which left me scrabbling to think of how you are _supposed_ to achieve these ends.
My reading and thinking led me to realise that without the emergence of some new arcane method, I was going to have to invent. A number of really ugly approaches came to mind, but mostly they seemed horribly expensive (lots of shuffling of objects, constructing wrappers, whatever). What I really wanted was a more flexible NSSet. That got me to find the somewhat undocumented (at least in the current Cocoa guides) NSHashTable and NSMapTable. These seemed to be offered as lower-level forms of NSSet and NSDictionary, and the ability to handle non-object keys and values was vaunted (though in actual fact, when you consult the supported configuration options on these Cocoa classes, objects are about all that is "guaranteed" to work!). It seems that the motivation for adding these collections to the Cocoa level was mostly to provide for weakly referenced keys and/or objects, in the presence of the new GC. However, clicking about in the class docs led me first to the NSPointerFunctionsOptions when initialising the collection, and then to a curious method -pointerFunctions. The latter returns an object of type NSPointerFunctions, and there right in front of me was the documentation for a couple of the properties of this object: hashFunction and isEqualFunction. Bingo! Perhaps I could concoct a set-like collection that used custom methods for identity - rather than fixed -hash/-isEquals: and therefore get around the limitations of NSManagedObject?
Experimenting with NSHashTable and pointerFunctions was frustrating - and I still haven't successfully managed to get this to work. The NSPointerFunctions returned from a freshly created NSHashTable have writeable properties for the functions I needed to provide, and the prototypes of these functions are documented enough, but try as I might, my provided functions were never called when adding objects. Are NSPointerFunctions only good for reading in this release (despite the writeable properties)? I have no idea.
However, the research into the pointer functions served as a segue into the murky world of the underlying Core Foundation implementation of CF(Mutable)Set. I've taken hardly any time to bother looking at the CF stuff - mostly because Cocoa itself is so complete, but also because it seems like a strange non-OO world where one doesn't go unless out of desperation...
As it happens, CFMutableSet is what Apple calls "toll-free bridged" to NSMutableSet, meaning that the same address can be used via either C-style pointers or Cocoa (id) pointers and with either the appropriate C functions or messages. This is very nice, but what was important was that CFSetCreateMutable (the C 'constructor' for the mutable set) takes a CFSetCallbacks structure, which is the analogue of the NSPointerFunctions. Creating appropriate 'alternate' -hash and -isEqual: functions was straightforward, and moments later... success! The CF version of the mutable set allows these alternate 'call backs' to be set up at construction time, and these are correctly called when objects were added (by sending the -addObject: message to the returned address cast to an 'id'!
Once I had had this mini-breakthrough, I was confident enough to create a 'new kind' of set collection back in Cocoa-land that constructed itself with CFSetCreateMutable, encapsulating the appropriate 'callbacks'. This collection expects to work with a category of objects that conform to an 'alternate identity' protocol, which requires the implementation of -altHash and -isAltEqual:. Furthermore, I now have a subclass of NSManagedObject that adopts this protocol and is the common super class for all my model entities, allowing me to create (albeit temporary) sets of unique instances derived from Core Data - according to the 'alternate identity functions' that they encode.
So I'm happy... but as I mentioned at the beginning of this piece, I still have that nagging doubt...
So it is that I've recently been wondering about uniqueness in Core Data.
Now Apple are quite clear about the nature of Core Data (at least as it stands today). Despite the entity relationship models, the prepared parameterised fetch requests (queries) and the ability to use a SQL database for storage, Core Data is not a general purpose database. It is designed, of course, to provide an elegant way to persist your application's internal state in a way that is natural and requires the minimal amount of overhead (notwithstanding the need to conform to its design patterns).
I've been very impressed with Core Data (so far). I'm lucky enough to be beginning my Cocoa career with Leopard, and like a lot of the Mac OS X frameworks, Core Data in Leopard has clearly matured very nicely into a highly capable and general facility. At this time, my main data model spans a half dozen pages and uses a good many of the features available (inheritance, to-one/to-many relationships, delete rules and a little validation with more to come). For most of what my app does, modelling its data this way is clearly superior. However, I have been surprised by a couple of things that seem like omissions, but per the foregoing, leave me wondering whether I'm missing the 'right' way to approach the problem.
One such item is a need to store singleton instances of some entity. My application has global state that should be persisted, which is not necessarily per-user (though naturally I have some of those too). It would be nice to be able to create an entity to represent a unique object that will record this state - and declare to Core Data that there can be only one instance of this (at a given version of the model). Yet, I know of no way to achieve this. Of course, one can live without this formal uniqueness, and instead date-stamp an instance, and have the application 'housekeep' any excessive number of instances (perhaps clearing away all but the last written instance), but...
There's a rather more fundamental kind of uniqueness in data of course, that of unique keys - and again Core Data has no way that I know of to express that an attribute will contain a unique key. In Core Data, all the combinations (tuples) of matching data will be returned on a query, and one imagines that there can be no opimisations in how an underlying data base is queried when such fundamental metadata is missing. I got a little excited when I first saw the "Indexed" check box on what happened to be a String attribute in the model designer, but looking up what this did revealed nothing more than the vague "Indicates that an attribute may be useful for searching".
Even if Core Data itself has no formal way of indicating uniqueness or key values, you certainly need to be able to determine this from time to time in your application. For example, if your model records "Customer" in various places, you are likely to have the same actual Customer represented by multiple instances of the Customer entity (one perhaps attached to a 'recent calls' part of the model, and one attached to 'sales'). Now, because you cannot formally uniquify the details of a particular Customer (with a key like 'customer number'), if you were to query for all the Customers in your data, you will end up with an Array of Customers with 'clones'. So, how do you turn an array of objects into an, er... set of unique objects (by some definition of unique). Of course, normally a set collection does this quite handily, through the expedient of defining appropriate hash and isEqual methods on the objects to ascribe identity. Cocoa certainly has such a thing in NSSet/NSMutableSet. So recovering uniques, even if the data framework can't, should be a walk in the park, right?
Well, in one of those unsatisfying moments I alluded to earlier, you soon stumble over what looks like a major flaw in Core Data. Core Data manages objects that derive from NSManagedObject, and the documentation clearly states that -hash and -isEqual: are reserved for Core Data's use (i.e. you cannot override these methods as you can, and often do, override them as an NSObject subclass). Oops. Try as I might, I have not yet found any canonical way to reasonably filter out uniques from an array/set of NSManagedObjects (whether obtained from a to-many relationship, a fetch request or any other way). The most obvious solution is barred, given the reliance of NSSet on the out-of-bounds -hash and -isEqual:, which left me scrabbling to think of how you are _supposed_ to achieve these ends.
My reading and thinking led me to realise that without the emergence of some new arcane method, I was going to have to invent. A number of really ugly approaches came to mind, but mostly they seemed horribly expensive (lots of shuffling of objects, constructing wrappers, whatever). What I really wanted was a more flexible NSSet. That got me to find the somewhat undocumented (at least in the current Cocoa guides) NSHashTable and NSMapTable. These seemed to be offered as lower-level forms of NSSet and NSDictionary, and the ability to handle non-object keys and values was vaunted (though in actual fact, when you consult the supported configuration options on these Cocoa classes, objects are about all that is "guaranteed" to work!). It seems that the motivation for adding these collections to the Cocoa level was mostly to provide for weakly referenced keys and/or objects, in the presence of the new GC. However, clicking about in the class docs led me first to the NSPointerFunctionsOptions when initialising the collection, and then to a curious method -pointerFunctions. The latter returns an object of type NSPointerFunctions, and there right in front of me was the documentation for a couple of the properties of this object: hashFunction and isEqualFunction. Bingo! Perhaps I could concoct a set-like collection that used custom methods for identity - rather than fixed -hash/-isEquals: and therefore get around the limitations of NSManagedObject?
Experimenting with NSHashTable and pointerFunctions was frustrating - and I still haven't successfully managed to get this to work. The NSPointerFunctions returned from a freshly created NSHashTable have writeable properties for the functions I needed to provide, and the prototypes of these functions are documented enough, but try as I might, my provided functions were never called when adding objects. Are NSPointerFunctions only good for reading in this release (despite the writeable properties)? I have no idea.
However, the research into the pointer functions served as a segue into the murky world of the underlying Core Foundation implementation of CF(Mutable)Set. I've taken hardly any time to bother looking at the CF stuff - mostly because Cocoa itself is so complete, but also because it seems like a strange non-OO world where one doesn't go unless out of desperation...
As it happens, CFMutableSet is what Apple calls "toll-free bridged" to NSMutableSet, meaning that the same address can be used via either C-style pointers or Cocoa (id) pointers and with either the appropriate C functions or messages. This is very nice, but what was important was that CFSetCreateMutable (the C 'constructor' for the mutable set) takes a CFSetCallbacks structure, which is the analogue of the NSPointerFunctions. Creating appropriate 'alternate' -hash and -isEqual: functions was straightforward, and moments later... success! The CF version of the mutable set allows these alternate 'call backs' to be set up at construction time, and these are correctly called when objects were added (by sending the -addObject: message to the returned address cast to an 'id'!
Once I had had this mini-breakthrough, I was confident enough to create a 'new kind' of set collection back in Cocoa-land that constructed itself with CFSetCreateMutable, encapsulating the appropriate 'callbacks'. This collection expects to work with a category of objects that conform to an 'alternate identity' protocol, which requires the implementation of -altHash and -isAltEqual:. Furthermore, I now have a subclass of NSManagedObject that adopts this protocol and is the common super class for all my model entities, allowing me to create (albeit temporary) sets of unique instances derived from Core Data - according to the 'alternate identity functions' that they encode.
So I'm happy... but as I mentioned at the beginning of this piece, I still have that nagging doubt...
Subscribe to:
Posts (Atom)
