iCloud sync not correctly syncing photo field between macOS and iOS

[thelee]

I have about ~100 records, and I went through all of them on macOS and created a new “pic” photo field and added a photo to all of them.

Verifying that the most recent cloud sync happened recently (well after I did all that) on both devices, my iOS version of my database is inconsistent. *Some* records have photos attached to them, but as I go down alphabetically it becomes more sporadic until none of the later records have any photos.

If I go to one of these entries on my iOS and tap on the pic, it gives me a “missing photo” error and references the original file name of the image that was on my desktop before I deleted it after having imported into tap forms 5 on macOS.

With this clue, I tried restoring the images I deleted from my trash and resync-ing everything. Nothing happened. If, however, I go to a record on my desktop and delete and replace a photo with the exact same photo, and then manually force both iOS and macOS to sync (by enabling/disabling cloud sync) that specific photo then shows up on iOS.

It really seems that cloud sync should be syncing whatever internal copy of the image the macOS copy of the database has, not the external file copy. As it stands, now I have to go through and replace all images with the exact same copy and wait for a full sync to happen before deleting the images.

[thelee]

hm, even as I do this, some photos still refuse to sync over, and it takes multiple attempts at deleting/replacing a photo with itself beofre it actually shows up on iOS properly. What’s going on here?

[thelee]

In fact, despite repeated attempts I have literally two records that absolutely refuse to sync a photo to iOS. I have no idea what’s special about these records in particular.

If I examient he recrods on iOS and tap on the photo, it gives me the same missing photo message. Interestingly, if I rename the file on macOS and replace the photo, when iOS syncs it shows me a changed file name, so clearly some data is getting synced, just not the specific photo itself.

[thelee]

duplicating the remaining record and editing the copy, finally worked. i deleted the original record, and *that* synced across devices.

would love to know if this is something that would be solved by using something like the IBM cloudant sync solution.

[Brendan]

Hi Thelee,

When you were syncing photos, did your iOS device go to sleep at all during the time it was syncing? I’m wondering if maybe that prevented the actual photo from syncing. The photos sync separately from the record itself. The record itself has references to the photos, but if the photos haven’t synced yet, that’s how you’ll get the situation where the record thinks a photo should be there, but when you tap to view it, it says the photo is missing. It’s because the photo on disk hasn’t yet synced. You could try closing and re-opening the document, which will also trigger another iCloud sync.

Cloudant / Apache CouchDB / Nearby should be more reliable than iCloud because they are all native sync engines to the CouchbaseLite database engine Tap Forms uses. iCloud is an add-on and Apple does throttle the amount of data that can be synced. It should eventually sync, but sometimes the iCloud sync service gets an error from Apple telling it that it has to hold back for a bit and try again later.

Thanks,

Brendan

[thelee]

thanks for the response. no, the iphone didn’t sleep. if the problem comes up again, i’ll give one of the alternatives a try.

otherwise, great product! still love using it

[T.L. Ford]

Photo file type? Link to a file instead of the file itself? File permissions?

[thelee]

It is a photo file type. I don’t know how file permissions could matter, since it’s stored in whatever internal db tapforms uses (as verified by the fact taht i’ve long since moved/deleted the photos).

I ended up having more sync issues, and ended up creating a free IBM Cloudant account. So far it’s been impeccable and rapid. I’m guessing that iCloud remains the weakest consumer sync option out there.

[Sam Moffatt]

iCloud is not really designed for folk to sync a database against, it’s a much simpler system and shared to a much greater number of users. Like most shared/multitenant services you’re not getting anything dedicated and there are a lot of throttles in place to prevent apps from monopolising their resources. A challenge for something like Tap Forms is that when you’re adding a bunch of content you’re not just changing one thing, you’re changing a few different pieces and it’s easy to get multiple updates needing to be sync’d up to iCloud. iCloud starts to throttle that, which is reasonable for it to keep itself available, which slows sync down. Add in that there isn’t a background sync functionality so that Tap Forms can easily continue to slowly push it’s changes up (iOS rather restrictive on background processing), it makes it easy for the sync to take a while to complete.

Cloudant has similar limits but for the most part it’s a much more generous number because once you’re out of the low free tier, it’s a lot more expensive than Apple’s solution. It’s also a lot more transparent and because it’s using a similar underlying data model, it’s also able to compute deltas for changes much easier and bulk load them. A lot of that is a side effect of the CouchDB replication protocol supporting those concepts natively versus iCloud’s model where iCloud mediates a lot of the data transfer functionality limiting just how much you can build.

[Jong-soo Anh]

Hi all,

I have been scratching my head over the last few day about the exact same issue. I even have one particular record which only synchronized one single field out of about 8 in total. I tried de-activating the iCloud sync and deleting the remote data before to activate it again, it fixed the incomplete entries but created new problems in other ones which worked well previously. And as my database is growing larger, this option is also getting less practical over time. More entries get errors.
Well now with this clear explanation I understand what is going on and I’m guessing it’s probably not fixable by either the users or the app’s developer. As a matter of fact while I was looking for a solution I stumbled upon several users suggesting to use CouchDB instead of iCloud sync. I will give it a go, I just installed it on my NAS. There will be no priority issues, in there :-)

[JCK]

I too am having this issue. Everything syncs perfectly between macOS instances of TapForms via iCloud, but the iOS version just doesn’t see the image files for whatever reason.

It’s weird that iCloud is syncing fine between the Mac’s, but not between the macOS and iOS versions.

[Sam Moffatt]

The throttles for sync work in both directions, you are throttled on both pushing changes to iCloud as well as retrieving updates and records from iCloud.

Unfortunately there also isn’t much clarity around how Apple’s rate limiting works. Apple’s CloudKit page states a 40 requests per second up under the private database section. From what I’ve read this 40 requests per second is likely a global limit for the application (e.g. all Tap Forms users), it’s hard to tell though if that applies to just the public databases or if it also applies to the private databases. It also looks like there might also be a bandwidth cap as well which makes sense (40 requests per second of 2MB images is different to 40 requests per second of records that are a few kilobytes).

I’m not sure how much this applies to Tap Forms if it’s impacted by this or uses a different pathway but I would expect that rate limits apply and for stuff like images that are larger that they will hit bandwidth limits quicker. As a former SRE I’ve got to imagine that Apple has rate limits on everything though to ensure system stability.

[Kerry Maxwell]

As someone who has routinely had this issue with Tap Forms for years, I sincerely doubt it’s an iCloud throttling issue. I use iCloud for work as an image editor, and rely on it to sync images hundreds of megabytes in size all day, every day, across multiple machines, without a hiccup. The images that refuse to sync in the Tap Forms mobile app are typically under 1 meg.I also don’t see asking users to host their own database server as anything other than infuriating. This has been an issue for years that has caused untold hours of hair-pulling and aggravation, and has never been adequately addressed. It’s been an issue with Nearby sync as well as iCloud.

[Brendan]

Hi Kerry,

I’m working on new code for syncing over iCloud for a new version of Tap Forms. Probably won’t be available until next year, but I am working on it. So sorry for all the troubles. Sync is hard.

Thanks,

Brendan

[Kerry Maxwell]

Fantastic, I look forward to it! It’s still the best app for my purposes, no matter how much I swear at it!

[Francis]

I’ve given up synching over iCloud about a year ago. Instead, I use “nearby” Not as user friendly but more reliable. Also, as I need to share my database with my wife (different icloud account), it’s the only way I could find…

Looking forward the new Tap Forms!

[Author]

Posts