Thanks, I looked at that earlier but it doesn't contain the solution I was hoping to find.
I ended up not using GroupingCollection. I think GroupingCollection is cool, but it isn't really customizable to the extent I need.
I just ended up transforming my data into a hierarchical form that could be used directly by AdvancedDataGrid. At the top level of my hierarchy, I have the detail txn groups, subtotal groups, and grand total groups. To sort the groups, I use custom SortFields. So, for example, if I want to sort by group name, I add a SortField for the groupName attribute. Both the Detail Group and the Subtotal Group share the same group name. Then I add a SortField for a summaryRow attribute. Only the subtotal group has that attribute. This ensures that the Detail Group is always followed by the associated Subtotal Group after sorting. I store my groups in an XMLListCollection and then set the sort attribute, then call refresh().
Not ideal, but it works. If my requirements were more complex, perhaps my approach wouldn't have worked as well. But as it stands it works well and has good performance.
I was hoping that ADG could handle this kind of sorting for me, but after looking at the source code, it seems like this is not possible without implementing my own custom GroupingCollection class and possibly other supporting classes.
One key thing to note is that clicking the column header to sort within groups works fine. However, if you then attempt to sort at the group level, ADG hangs while trying to refresh itself. The trick is to assign the dataProvider.sort attribute to a temporary variable, then set it to null (this sort variable contains the ADG column sorts that the user clicked). Apply your group level sort and then reapply the dataProvider.sort attribute.