Automatic synchronization with iCloud

[Michael Kaufmann]

Hello,

First of all, congratulations! Tap Forms is the best personal database product I have ever seen. I have bought both the Mac and the iPhone version, and I’m very happy with it. :-)

However, I always have to push the sync button when I switch between my Mac and my iPhone.

My question is this: is there any possibility that in the future, there will be an “auto sync” option? For me, it would be much more convenient if Tap Forms synchronized all changes to iCloud automatically.

For example, you could leave the manual push / automatic pull as an option that could be chosen in the preferences, and provide an automatic push/pull option as well.

What do you think?

Kind regards,
Michael

[Brendan]

Hi Michael,

Thanks for the feature suggestion.

I would love to be able to offer this option. The only issue I have at the moment is that Tap Forms generates a sync file every time you tap on the upload button. In order for me to build auto-sync, I would need a way to keep the number of files generated down to a minimum. Every small change you make would generate a file that’s at a minimum about 25 KB. This is because Tap Forms does not just generate a file with only the change you’ve made. It creates a database file with the entire structure of the main database but containing only the data for the changes that you made. So imagine you change one character in a field name or value, Tap Forms generates a 25 KB sync file for that small change, sends it to your other devices and they auto-sync. Eventually Tap Forms is going to consume a lot of your iCloud storage. So putting the initiation of the sync process in the hands of the user reduces the amount of sync files that Tap Forms generates since it’s most likely that you’ll only initiate sync after you’ve made a bunch of changes.

I know there are other strategies that I could use such as delayed syncing after a period of time or after a certain number of changes, but that’s not an ideal situation.

Also, the more sync files there are in iCloud, the longer it takes new devices to auto-sync when they have iCloud sync initially enabled. You can always do a Reset iCloud Sync function at any time which will delete all the sync files to clear up space. But then you just need to re-initiate sync from your device to start the process all over again. There’s also timing of syncs to make sure Tap Forms doesn’t try syncing during long running processes such as importing thousands of records from a CSV file or importing a Bento template in the Mac version.

So there’s lots of things to think about how to make a full auto-sync system.

Thanks,

Brendan

[Leo]

Brendan,

how about autosync when opening or closing Tapforms? That way we probably allways stay in sync. (off course the button stays for immediate sync) :-)

[Michael Kaufmann]

Hi Brendan

I understand, there are technical challenges for auto-sync. However, seeing it from the perspective of “user experience design”: I’d love such a feature…

I wonder how Apple does it? When I enter an iCal event on my MacBook or in the iCloud App, one second later my iPhone jingles because the event has been automatically synced. I love that.

Consider my use case: I want to access a form I have created on my MacBook. Stupidly, I forgot to push the sync button – now I’m in my office in need of that information, but the MacBook is at home; and on my iPhone, the database is out of sync so I can’t access it.

Another super cool feature suggesteion (slightly off-topic): an (auto-synced) online app on iCloud.com for TapForms (similar to Calendar on iCloud.com) where I can access my database online (I don’t know if that is even possible for third party apps). The use case is this: Say I surf the web on a Windows pc and stumble upon a web link I want to add to my TapForms database. How do I get the link into TapForms without manually typing it? For now, I send myself an email with the link, and I open the email on Mac or iOS to copy and paste it into TapForms (tedious). If I had an online app on iCloud, I could add the link (or any information, for that matter) immediately online.

I hope my use cases help you by giving you information on the user’s perspective.

Thank you for TapForms, it’s really a great product.
Cheers,
Michael

[Brendan]

Hi Michael,

Understood. I would love to have full auto-sync too. I understand the limitations of the current implementation. I know it’s definitely an area ripe for improvement. It should “just work” without all the hassles.

iCal (aka Calendar) works the way it does because it’s not really iCloud sync you’re dealing with, even though it’s under the umbrella of iCloud. Your device is talking to a CalDAV server which handles specific information about calendars and appointments. Plus there’s very little information being synced and they have server processes which can handle conflict resolution, etc.

Tap Forms uses “Documents & Data” from the iCloud suite of services. There’s no logic on the server that I can implement that can handle things like baselining a bunch of syncs into a single file to reduce the number of sync files lying around and such.

There’s also no third party website integration available to offer an iCloud web based version of Tap Forms.

Thanks!

Brendan

[LeoC2]

Interesting reading Brendan. I just posted this same request before finding this old thread. iCal is not the only program that syncs automatically and seamlessly… what about Pages, Notes, or Numbers. Third party apps such as 2Do also are seamless. I don’t think any of these use CalDev servers.

Can’t you just have an optional setting in the iCloud sync preferences to perform a sync every user settable number of minutes? Say every minute, or every 5 minutes, or whatever.

Also automatic upload sync when quitting the app would also be a big help.

Leo

[Brendan]

Notes uses a different technology for syncing. It’s not iCloud Drive. Pages and Numbers use iCloud Drive, but they’re document based apps and not database type apps. Their documents actually live in iCloud, so when you save a document, it saves directly to iCloud Drive. That gets synced to your other devices. There’s no form of database transactions that are occurring in those apps. Because Tap Forms uses SQLite to store all its data, the only way to sync is to keep a local copy of the database on each device and then manage transmitting just the changes up to iCloud. Then on the other side the changes need to be merged with the local copy of the database taking into consideration changes that may have been made there, deletions, additions, etc.

I wish it were a simple matter of just copying a single file up to iCloud as is the case with many other apps, such as 2Do and others.

Tap Forms also does not use CoreData for the database access layer. It uses FMDB which accesses the SQLite database directly. CoreData offers a sync solution that provides transaction log base syncing built-in. I built Tap Forms way before Apple even provided CoreData on iOS, so it wasn’t an option for me to use that technology. And by the time CoreData came out, it was too late for me to use it as Tap Forms was already built upon direct SQLite commands.

So this is a really complex problem for me. I’ve been constantly thinking about ways to accomplish WiFi syncing and that’s also very complex. There are a lot of things that iCloud Drive gives me for free which I will now have to do myself with WiFi syncing. For example, when I store files in your Mobile Documents folder, I don’t worry about what files are on other devices or if files have been synced already or not. I just write files to disk. iCloud Drive determines what files need to be sent to the other devices for me. It also handles the actual transport of the files. When I do WiFi sync, I have to handle all of that myself. Finding out what files are different between all the devices, sending over only the files that are new or have changed. Deleting files that no longer exist on the source or target device, etc. Plus not to mention physically transmitting over all the sync files back and forth to process.

Thanks,

Brendan

[LeoC2]

Wow, I think my head just exploded! So many of us don’t know what goes on behind the scenes if iCloud, we just take it for granted it all works.

leo

[Author]

Posts