I would like to put together a serverless distributed application, probably for devices. We’ve moved along pretty far in this field recently, but I’m not sure exactly where we stand.
Most forward in my mind is the problem of synchronizing data, when there is no gaurantee when or how often the nodes of the application will be connected. We want to avoid hosting a dedicated server specifically to act as a datastore for this information. This has a host of complications and startup costs. What else are our options? It seems like the use of XMPP as a data transport has come along, whereby you can use an XMPP service such as google talk to punch through NATs and ride around firewalls and deliver messages to the nodes in our application.
This approach is fine in terms of locating our nodes (as long as services like google talk are ok with application data being transported over their networks, this is rather unclear to me). But it doesn’t really handle the problem of missed messages, and XMPP, by default, does not make any kind of delivery gaurantees, I believe. So we may be able to layer on top of this to make the delivery reliable, but it seems things will get pretty complicated, and what about the case where we have a mobile device application where we will only very rarely connect and synchronize data, how likely is this client to be connected when other clients are connected so that they can “catch up”. Especially with devices like the iPhone, that have no ability to run background tasks to maintain a connection to a service. I belive some xmpp servers will store missed messages for a certain time, but I’m unsure how reliable this is.
Thinking about this though, I thought of the perfect solution, but I’m unsure as to whether it aligns with the terms of service of the services in question. What if our application posts structured data to a blog? Each node could even have its own blog, and synchronization can involve querying these blogs for posts that have not yet been seen. It seems like this would be straightforward with most of the blogging apis available for the major blog hosts out there. But this type of use would seem to fall in some sort of grey area, and may be against the TOS of these services. Anyone know?
Its an interesting generalization to think about though. When we use a blogging site, we are publishing human readable information to the internet, with the help of an online service. But could we not generalize this such that we allow users of an application to publish machine readable information to the internet, with the help of an online service? And if the structure of this information were to conform to an open specification, many applications would be able to make use of it, and add to it. Anyone know if there is an online service that is tailoring itself to this kind of interaction?
I guess another solution is some kind of cloud computing, but we still have an issue where if clients are so infrequently connected, it may be rather hard to keep the data store synchronized.