lazy associations

[busitech]

We understand that FDS supports lazy associations, but the
documentation is unclear about whether it's necessary and in what
manner to partially initialize an array which is not empty but in
the "lazy" state, as opposed to the fully initialized state, and
the empty state.



We are using the Java adapter, so our application is
responsible for how data comes out of the DAO layer.



Any information would be appreciated - thank you.

[jvroom]

This is a good question. We will support the ability for your
assembler to return referenced objects which only have the id of
the referenced object populated for the final release but
unfortunately the code changes required to make this work did not
make it into beta2 or beta3. I think it will work in previous
releases but only when you have cache-items set to false and
auto-sync-enabled also set to false.



To give you more detail, in Beta2 and in Beta3 the system
expects that you return your items so that even lazy referenced
items have their complete state populated. Partly this is so that
we can cache those items and avoid calling your "get" method when
the client requests a lazily reference instance. You can turn that
off by adding <cache-items>false</cache-items> to your
destination's properties. The other reason we rely on that is so
that we can store the referenced object graph to detect changes
when auto-sync-enable is true. For example, client A may be caching
the entire graph but client B may only be caching a single leaf
item. We need to have that reference graph so we know to update
client A when client B changes the leaf object. Both of these will
be fixed in the final release so we just won't cache values of lazy
associations (unless you set a special flag) and we lazily build
out the referenced graph as needed.

[busitech]

Thank you very much for the information.



We deal with many tables with thousands, tens or even
hundreds of thousands of records in which case means returning all
of the results is not possible, even back to FDS in response to a
fill request - let alone to a client over the network. How does the
DAO layer need to respond to a fill request for a destination which
is paged? Can lazy associations still be paged?



When you mention caching, it reminds me of another aspect of
concern to us - stale data. Sometimes FDS is not the application
which updates the database. Instantly the cache can be rendered
obsolete. Has this problem been addressed in any way?



We are adding version columns to all tables to prevent
problems getting into the database. However, this kind of column
could be of interest to FDS if it is available. I have not thought
it through completely, but should a configurable interval of time
pass it might be useful for FDS to query the DB to get the latest
version numbers to see if it needs to refresh the cache and resend
to all subscribers a refresh.



I realize there is quite a bit here to contend with, but all
of these issues come up in about the first hour of development in
our environment....



Thank you.

[busitech]

The final aspects of the associated data set is filtering and
sortation (client defined but server performed).



I see support within the Flash AS component library for these
concepts but they are problems that really need to be solved on the
server side. Client side sorting and filtering only really work
with small datasets (not found often in the enterprise).



Do you know if FDS will conquer client-defined, server
performed sorting and filtering?

[jvroom]

If you are returning collections where the set of ids is too
large to be returned all at once, you should implement that
relationship using a fill method, not using an association
property. In this case, usually the id of the related item is at
least one of the fill parameters. For example, I have reimplemented
the crm demo in beta 2 so that it returns the list of employees for
a given company as a fill method which takes the company id as a
parameter. This allows you to either use fill parameters to
demarcate pages (i.e. you might have a page number parameter), or
to use flex's built in paging support.



Currently in FDMS, the fill method must return all of the
items to be returned in the fill so if the fill is too large to be
returned as a single result set even for the server, you need to
split it up using fill parameters to control the paging (in other
words, you have to do paging yourself and can't use fdms' builtin
paging support to do this automatically). Though we wanted to
support complete 'client controlled' paging all the way through to
the database, it is fairly challenging to keep a list sync'd
between two clients when you do not even know the complete contents
of the list, how big it is, the ways the client can access pages
etc. We might be able to support a variant of the fill method which
implements paging only for the auto-sync-enabled=false case (where
you are not sync'ing changes between clients) but it sounds like
you need clients to be kept in sync even on these large lists as
they change?



I think that in the general case of client controlled but
server performed filtering and sorting, you pass in the sorting and
filtering criteria (including ranges) via fill parameters. The
server performs the operation and returns that set of items so you
get an ArrayCollection on the client which contains just a page of
data at a time. If you want to implement a virtual array collection
on top of that which represents the entire data set that will then
make fill calls as needed to fetch pages you'd have to do that
yourself in action script code. Another strategy would be to make
the paging controls explicit in the UI (ie. have a next page button
which fetches the next page instead of scrolling through the list)
and that would make it easier.



In terms of caching, when you have auto-sync-enabled set to
true, FDMS must at least cache the graph of ids of all of the
managed items by all active managed clients. By default, it caches
the items too but you can turn this off with
<cache-items>false</cache-items>. If you need to keep
your clients up-to-date with changes performed by code that
accesses the DB directly or is just not going through flex, you can
use the 'server push api' to tell flex about these changes, or just
use this api to periodically refresh your clients. If your clients
do not need to be made aware of these changes, you can just let
them drift out of sync and use the conflict detection facilities to
deal with stale updates. If you are using timestamps for your
version numbers, you could probably implement a "get me a list of
everything that has changed" query in a scheduled task which then
uses the server push api to keep clients in sync.