Syncing completely broken

[Brent McInnis]

I have tap forms on 4 devices – an iMac, an iPhone, and 2 iPads. The problem I was facing at first was that one of the iPads was not reporting the same data as the other 3 devices (in one form, it was reporting one fewer record than the other 3 devices, even after what otherwise appeared to be a successful sync).

In the course of trying to resolve this, I’ve deleted and reinstalled Tap Forms from the iPad which was missing the record, and also from the iPhone (I forget why I deleted it from the iPhone – I’ve been struggling with this for a couple of hours, and a fog has started to develop). So those two devices are simply showing the standard new installation sample forms and data. My intention was to upload to iCloud from one of the other two devices, which still hold my real data, and then sync that data down.

What I am having happen now is that
1) The devices can all “see” each other in the iCloud sync area, as they should
2) I can upload to iCloud from my iMac, but the other devices never recognize that the upload has occurred. That is, although they show that the iMac exists, they never show the sync timestamp which is visible on the iMac.
3) I can upload to iCloud from the iPad which has my real data – but it seems to upload in two “stages”, and the second stage hangs. In “stage 1”, it says something like “pushing records to cloud”, and then something like “deleting purged records”. A short while later, “stage 2” starts, in that I see the iPad reporting an “Upload %” in its Registered Devices area. But it gets as far as around 20% to 30% (I’ve tried it several times, and get a different number each time) and then just hangs there.

The new devices with the new installation, which don’t show any timestamp for the iMac, do at least show a timestamp for the iPad which performed the “stage 1” upload and hung on “stage 2” – so I guess they recognize that an upload has occurred – or at least been started or attempted – but they do not sync the data down.

I am at a loss as to what to do to get my data back on all 4 devices.

I saw that Brendan is on a well-deserved vacation this week. My situation is not urgent, but I would appreciate help after the vacation is over.

Thanks!

[Leo]

Hi Brent,

Reset syncing on all devices.
Then turn syncing on one by one after every cycle is completed. Remember, a cycle can take some time and it may seem your device hangs but it doesn’t. Be patience.

Leo

[Brent McInnis]

Leo,

Thank you for the advice.

I tried it, and found that, after resetting syncing on all devices, and then turning syncing on on just my iMac, the iMac was “seen” (in the Registered Devices area) on two of my other devices (an iPhone and one iPad, but not on the third one (another iPad). I decided to proceed with the experiment with just the iMac and the other two devices which could “see” it, and leave the “bad” iPad out of the mix for now. (This “bad iPad” is also the one on which I reported the “hang on upload” problem in my original post).

I then enabled syncing on the iPhone, and waited for it to be “seen” on the iMac and the “good” iPad. Then I enabled syncing on the “good” iPad, and waited for it to be seen on the iMac and iPhone. Soon, all 3 devices could “see” each other. This only took a couple of minutes.

I then performed an upload to iCloud from my iMac. This completed at 12:19 PM. This time appears as a “Last Synced” timestamp next to the iMac in its own “Registered Devices” area.

By 12:56 PM (over 35 minutes later!) the iMac “Last Synced” timestamp had still not appeared on the other two devices, nor had their data changed to match the data on the iMac (both of them had just new installation “sample data”.

I think 35 minutes should be more than enough time for the other 2 devices to sync the data down from the cloud, right? So it seems that I am experiencing a problem which cannot be resolved by the normal procedure.

By the way, my internet connection is 55 Mbps download, and I have the latest version of operating system and Tap Forms on all the devices.

Thanks again for taking the time to look at this, and please let me know if you have any other suggestions.

[Brent McInnis]

I have two new data points:

1) About 5 hours after I did the upload to iCloud from my iMac, I got into Tap Forms on both my iPhone and “good” iPad, and noticed that they both now reported the timestamp for the iMac upload. So they DID eventually see it – it just took a very long time for it to be recognized.

2) Both of them automatically started to download the data. This took about 15-20 minutes on each device. After they got to saying they had downloaded 100%, they started showing the “number of rows remaining” – I guess they were processing the downloaded data or something. The “number of rows remaining” started out around 23,000, then, over the course of 20 seconds or so, decreased to “1847 rows remaining” – and then hung there for about another 20 seconds or so – and then the program crashed. This happened on both an iPhone and an iPad.

Upon restarting Tap Forms on the iPhone and iPad, Tap Forms would immediately say “downloading from iMac” for 20-30 seconds – and then crash.

I guess this means my Tap Forms data is corrupted.

This leaves me with two questions:

1) Is the behavior of it taking hours for the other devices to recognize that an upload has occurred normal? In my prior Tap Forms experience, it hasn’t been – uploads have been recognized and reacted to in seconds or minutes, not hours.

2) What, if anything, can I do about the fact that my data has (I guess) been corrupted? (although the iMac doesn’t seem to be bothered by it).

Thanks,

[Brendan]

Hi David,

What kind of iPad and iPhone do you have? I’m just wondering if Tap Forms is running out of memory during one of the large updates. Possibly an older iPad and iPhone has less internal memory (not storage space) than the newer ones that may be causing that.

You could try a backup on your Mac, then move the backup over to your iOS devices and then restore those backups to them.

Also, when you see Tap Forms sitting there processing the changes from iCloud, are you tapping on the screen at all anywhere? I’m wondering if perhaps some interaction caused the crash after it’s doing it’s commits to the database. Or perhaps there was a deadlock caused.

I am working on Dropbox sync, so I’m hoping that these really big delays from iCloud will just go away by switching to Dropbox sync. Lots of customers successfully sync with iCloud, but lots also don’t, so Dropbox will provide an alternative. I’m still a ways away from finishing it though. Obviously this week is a bit of a bust for my productivity level with me being on vacation. But I’ll be back at it full time next week for sure.

Thanks!

Brendan

[Brent McInnis]

Brendan,

You are far too generous with your time. I want you to be able to relax a little and enjoy your vacation, so I’m not even going to answer your questions until around the end of the week – and even after that I expect you should take a few days to get caught up with things around your home.

I can get by without Tap Forms for a week or so – though I’ll be looking forward to accepting your help after you’ve returned to work.

Thanks,

Brent

[Brendan]

Hehe… thanks Brent. It’s no problem though. I do have a bit of downtime in the hotel at the end of the evening to answer some questions.

Took a helicopter ride around Oahu today. What a blast! Although my 13 year old daughter didn’t like it. Freaked her out a bit.

[Brent McInnis]

Brendan,

I hope you enjoyed your vacation. I suspect that, even if you’re back home, you’ll be going through some unpacking and catching up before you turn your full attention back to Tap Forms, but anyway, here is my reply to your reply #12917, for whenever you’re ready to process it.

My iPhone is a 5s (14 months old), one iPad is a “3rd generation” (almost three years old), and the other iPad is an iPad Air 2 (just 3 months old). The iMac is a new Retina 5K (about 4 months old). So the only device which is “pretty old” is one of the iPads – but that one currently appears to have the same Tap Forms data as my iMac. It’s the iPhone and new iPad which have new installations of Tap Forms, and which crash while trying to sync Tap Forms data from iCloud.

No, I’m not tapping on the screens while trying to download – but I might have tapped on a screen at some point in the past, while uploading or downloading. If I have corrupt data, that might have been the cause.

I’m afraid I don’t know how to “move a backup” from my Mac over to my iOS devices. Although I know how to sync apps, music, audiobooks, and podcasts between them, I don’t know how to transfer an app’s data between the two OSs. I’ll be glad to try it, but would you mind telling me more about how to go about doing this?

Thanks,

Brent

[Cynthia Segalla]

Hi,
I just wanted to chime in to say that sync isn’t working for me either. I bought Tap Forms for my iMac and iPad yesterday morning and it has been really frustrating trying to get them to sync. Sunday was basically an endless series of enabling/disabling sync, resetting sync, signing in/out of iCloud, and restarting. I combed through the forums for more tips, but haven’t found anything that works.

I’ve had marginal success with one form of 38 records, but nothing beyond that. Anyway, I’ll be following this thread in hopes that something will help.

[Brendan]

Hi Brent,

I would suggest running the Vacuum Database command on your database so that it compacts it to the smallest possible size. Then try a sync again. At the end of the sync process, Tap Forms updates the search index, so that could be a reason why it sits there for a while. But it shouldn’t crash after that, unless it was severely memory constrained during that operation. If there is some other reason for the crashing, sometimes uninstalling and re-installing can correct that problem.

Restoring a backup from the Mac to an iOS device requires the following steps:

1. Make backup on Mac using the Backup command.
2. Launch iTunes with your device plugged in.
3. Click on device, then click on Apps.
4. Scroll down to File Sharing area and find Tap Forms.
5. Click on Tap Forms to reveal the Tap Forms Documents view on the right.
6. Drag the backup file into the Tap Forms Documents view.
7. Launch Tap Forms on your iOS device.
8. Tap on Tools, then Backup & Restore.
9. Tap on the file you just dragged into iTunes.
10. Tap on the Restore Backup button.
11. There’s no step 11.

Thanks,

Brendan

[Brendan]

Hi Cynthia,

Did you watch the demo video in the online user manual about iCloud sync. It shows you the process. It’s basically a manual upload and automatic download process. So, upload from your Mac, then your iOS devices will automatically download the data.

The first thing to get going is to make sure that ALL devices are visible in the list of Registered Devices. This can be achieved by enabling iCloud Sync in Tap Forms on each device.

Once each device sees all the other devices, then you would press the upload to iCloud button and then just sit back and wait for the other devices to auto-sync those changes.

If things go awry, press the Reset iCloud Sync button, then re-enable iCloud sync on each device and try again. Pressing the upload to iCloud button on ONLY ONE device at a time. Each time waiting for the other devices to auto-sync.

Thanks,

Brendan

[Cynthia Segalla]

Thanks for the response. Yes, I watched the video and followed the instructions. Everything looks right in terms of the devices showing up in the list and the imac indicates a time for ‘last synced.’ The problem is that nothing shows up on the iPad. I tried the wait and see method, and even after 24 hours nothing has come up.

[Brent McInnis]

Hi Brendan,

I ran all 4 of the database commands on the Mac, then synced the Mac to the Cloud. The sync to my iPhone which followed moments after still crashed Tap Forms on the phone at 1847 records remaining. This was true even after removing and reinstalling Tap Forms on the iPhone.

I then tried backing up from the Mac and restoring on the iPhone using iTunes, as you described in your earlier message (good instructions – thanks!). At first, everything looks great on the iPhone. But soon after I make a change on the Mac and upload it to iCloud, it starts downloading to the iPhone as it should – and then Tap Forms on the iPhone hangs at 1847 records remaining, and then crashes about 15 seconds later.

I even redid this operation and then, instead of making a change on the Mac, I made a change on the phone and tried uploading it to iCloud. The upload hangs – and Tap Forms on the iPhone crashes – at 1847 records remaining!

It sure looks like my data has got something in it – at record 1847 – which is making Tap Forms gag – which is the last thing I want to hear.

Do you have any other ideas?

[Brendan]

Hi Brent,

I think perhaps I would need to try it out for myself with your backup file to try to determine why it’s crashing on 1847 records remaining all the time. Would you be willing to send me your backup file? Email it to me if it’s not too big or upload it to Dropbox and email me the link to my support@tapforms.com address.

Thanks,

Brendan

[Brendan]

Hi Cynthia,

Please try the Reset Network Settings function in the Settings app under General > Reset on your iPad. Then launch Tap Forms and check to see if it syncs the data.

Thanks,

Brendan

[Cynthia Segalla]

Thanks for the suggestion. Unfortunately, this did not work. I reset the network settings about an hour ago, but nothing has changed on the ipad. Any other ideas?

[Brent McInnis]

Brendan,

Thanks for offering to look at my data. It turns out that the zip of my backup file is 4 GB, so it is taking a while to upload. I will attempt to send it, or a link, to you soon.

Thanks,

Brent

[Brendan]

Hi Brent,

Instead of the entire backup, just send me a zip of the TapForms_db.sqlite file. That’s all I need.

Thanks,

Brendan

[Brendan]

Hi Cynthia,

Do you have all the latest versions of Tap Forms? 3.0.3 for Mac and 3.9.4 for iOS? Using iOS 8 and iCloud Drive enabled on all devices?

Did you press the upload from your Mac? You should see the iPad displaying the messages at the top of the status bar shortly after you press the upload to iCloud button on your Mac.

Thanks,

Brendan

[Cynthia Segalla]

Brendan,
Yes to all. I am using the most current versions of Tap Forms and the most current iOS. I pressed the upload button from the mac, but did not see any reaction at all on the iPad. There was no status bar message on the iPad. But when I look at the list of registered devices, the Last Synced date and time is accurate.

[Brent McInnis]

Hi Brendan,

OK, I’ll send you the TapForms_db.sqlite file from my Mac. Unfortunately, it’s nearly as big though – 3.8 GB – so I’m going to have to let it upload overnight.

Thanks again for your efforts on this. I’ll let you know when the file is ready.

Brent

[Brendan]

Hi Cynthia,

You mean the last synced date was correct for the Mac or for the iPad?

[Cynthia Segalla]

For the Mac. There is no Last Synced date for the ipad since I’ve been waiting for it to auto-sync.

[Brendan]

Hello Cynthia,

Ah ok. So then the sync just doesn’t happen on the iPad. I wasn’t sure if it was that the data didn’t come over or the sync process never kicked in.

And you definitely see your Mac and iPad listed at the same time on both devices? If the Mac’s last sync date as displayed on the iPad is updating shortly after you press the upload to iCloud button on your Mac, then the sync should be happening on your Mac too unless somehow the TFSyncDoc file (which contains the changes made since the last time you pressed the upload to iCloud button on your Mac) is not making it to iCloud from your Mac. But it seems odd that the TFSyncDevice file (which contains the information about the last sync time) is making it over, but not the TFSyncDoc file (that contains the actual data). That’s the only thing I can think of to explain a situation like this.

I know you’ve done it probably a hundred times already but I’d like you to follow this procedure exactly to the letter:

1. Reset iCloud Sync on your Mac AND iPad.
2. Wait about 2 minutes.
3. Re-enable iCloud sync on your Mac.
4. Wait for your Mac to show up on your iPad.
5. Re-enable iCloud sync on your iPad.
6. Wait for your iPad to show up on your Mac.
7. Press the upload to iCloud button on your Mac.
8. Wait for your iPad to auto-sync. You should see messages appearing at the top of the iPad’s status bar area indicating that it’s syncing. You may also see the % download indicator start to change when the TFSyncDoc file is downloading to your iPad. But it depends on how big the file is. If it’s small, you might not see the progress indicator change.

Your iPad should now have the same data as your Mac has. DON’T press the upload to iCloud button on your iPad before it auto-syncs the data you uploaded from the Mac version.

Let me know if that works.

If not, reboot all your devices and try again.

Thanks,

Brendan

[Brent McInnis]

Hi Brendan,

I’ve shared my TapForms_db.sqlite file with you from my Google drive (at least I think I have – let me know if you didn’t receive a message about it, and access to it.)

Thanks,

Brent

[Brendan]

Hi Brent,

Well I’m trying to download the file, but Google Drive keeps insisting that it can’t display a preview and when I click the Download button, it just opens a new webpage and wants me to use Google Docs for some reason.

I’m still trying to download it. Maybe Google Drive doesn’t work with Safari.

Thanks,

Brendan

[Brendan]

Ok, Chrome is letting me download the file.

[Brent McInnis]

Sorry about that, Brendan. Since I got your message a few minutes ago, I’ve been looking at my Microsoft OneDrive account, where I have enough free space, and which doesn’t seem to have any size restriction (that I’ve been able to find yet, at least). If you have any further trouble with Google Drive, let me know and I’ll pursue putting it on OneDrive.

[Brent McInnis]

For the benefit of those who may be “listening in”, Brendan and I have had a couple of private email exchanges so I could provide him with information to allow him to decrypt and edit my database, which he cleaned up and returned to me.

Brendan, you’ve worked wonders! I’ve installed the file you sent me on all 4 of my devices (using iTunes file sharing to get them from the Mac to the iOS devices), then successfully synced between them. I feel like an amputee who has had some miraculous new surgery in which a lost limb is replaced!

I suspect the original problem stemmed from, at one time in the past several weeks, interrupting a sync, or rebooting a device during a sync, or something. I don’t know what I did, but it was probably something like that. I’ll try to be more careful in the future.

Just one little thing to ask you about – something I would never have even noticed if I wasn’t watching everything so closely in this “reconstruction” process.

I notice that, on my 3 iOS devices, the size of the TapForms_db_sqlite file is identical in iTunes file sharing on my iPhone 5s and iPad Air 2 (41.2 MB), but is slightly larger (42.5 MB) on my older 3rd Generation iPad. All 3 devices have the same version of Tap Forms (3.9.4), and I wouldn’t have thought that older hardware would have made a difference in the size of the file. This isn’t causing me a problem with regard to usability – I just wonder if you view it as odd, and indicates that something is still a little bit not right.

Other than that, or even with that, since it isn’t really noticeable, I’m a happy camper! Thanks for the usual great support!!!

[Brendan]

Hi Brent,

I’m glad the file I sent you worked for you. I’ve seen this one other time by one of my beta testers. The table that stored the Link to Form information ballooned to over 18,000,000 rows! I’m not sure how that happened. It may have been due to a Bento import gone wrong as most of the records were referencing Bento records. In any case, the link records were invalid since they didn’t have all the foreign keys setup right so I was easily able to delete them all. So consequently we reduced the file from 3.8 GB to 34 MB.

As for the difference in file size between your different devices now, it could just be a difference between the block size allocation on the different iOS devices. Or it could be that the one has had some activity on it already? The only true way to know if the files are the same size is to download the files again and then examine the number of bytes in the Finder on the Get Info view for each of your files. The byte count should be roughly the same.

[Brent McInnis]

Brendan,

The iPad which has the slightly larger TapForms_db_sqlite file size didn’t have any activity that the other devices didn’t have, as I simply took the file you sent me, installed it using iTunes file sharing on all iOS devices, and then followed that by, in turn, modifying one record on each device, pushing that devices’s data up to iCloud, and then watching the modification automatically sync down to each of the other devices.’

But as long as there is some other reasonable explanation – such as the block size allocation difference you suggest – I’m perfectly willing to assume that’s what’s causing it and leave it at that. Syncing is now working perfectly from my perspective, and the file sizes on all devices are close, and that’s good enough for me!

Thanks again for the exceptional support you provide! I’m happy to be, as always, a very satisfied Tap Forms customer!

[Brendan]

Hi Brent,

That’s great to hear! That crash you were experiencing was definitely do to the additional 18 million bogus rows in the database file. Still wish I knew how that happened. But as I’ve said before, I’ve only seen it happen twice. Once with one of my beta testers and once from you! :-) Aren’t you lucky?

Thanks!

Brendan

[Cynthia Segalla]

Brendan,

I tried your directions, but it’s the same as before; nothing on the iPad. Thanks for trying. At this point, I’m not sure Tap Forms is for me. Aside from the sync issue, I noticed a lot of images are missing from my records. Having already invested so much time trying to get sync to work, I am not really interested in repeating the process with the images. In any case, I’m very grateful for your help. You obviously care a lot about the product and it’s a great app; it just doesn’t work for me. Thanks again.

[Author]

Posts