Cloudant / Bluemix changes

[scneophyte]

I’m exploring using Cloudant, after having issues with both iCloud and Nearby sync but it appears that things have recently changed (see attachment).

I’ve setup a free Bluemix account and then provided a Credit Card in order to add Storage to my account…now my account upgrade is “under review” and I am waiting for an email to be told that my upgrade has gone through at which time I assume that I can create a storage space.

My question is: will this new “Bluemix” paradigm work seamlessly with TF? or is the Cloudant sync no longer viable for new users?

Attachments:

You must be logged in to view attached files.

[scneophyte]

I found the “Cloudant NoSQL DB” under Services > Data & Analytics (see screenshot)

Attachments:

You must be logged in to view attached files.

[scneophyte]

I’ve gotten this far and now I’m stuck. The documentation states that it is accessible via an https api but and I’ve tried to enter that into the TF Cloudant settings but it doesn’t quite look right.

The online TF help for Cloudant doesn’t provide any direction for setting up Bluemix, but perhaps I’m missing something simple?

The screenshot shows the screen that I was able to get to that looks like the Cloudant dashboard.

Attachments:

You must be logged in to view attached files.

[scneophyte]

Okay, I was able to get it linked by creating a Service credential (see attachment). Click on “View credentials” and the service name and password are displayed.

The sign-in was successful but I received two immediate notifications that I had exceeded the limit on the number of writes.

Attachments:

You must be logged in to view attached files.

[scneophyte]

I had to upgrade to the Standard plan (see screenshot) which is a little scary since it estimated the monthly charge as $76…but I wasn’t completely clear on if that is applicable to my usage.

I was able to establish the sync…it wasn’t working reliably…Adding a new record to my Mac was syncing, editing that new record was syncing to the iPad but editing a previous record was not syncing in either direction.

I then closed the document on the iPad and reopened and received the message that “Synchronization has failed…Error sending to server.: the request entity is too large”

Closing / reopening the document on the iPad resulted in a perpetual “Sync sending” message

I’m about to try restarting both devices…

Attachments:

You must be logged in to view attached files.

[scneophyte]

I officially give up. After restarting, the Mac was stuck in Sync Sending for over ten minutes and never got past 1/390. I closed the iPad since I didn’t want them both syncing at the same time. Closing the document on the Mac and opening it on the iPad results in the synchronization error in the post above.

I can’t get Nearby, iCloud or Cloudant sync to work.

Would uninstalling / reinstalling the applications on both the iPad and Mac then trying the Nearby / iCloud be the next best step?

[Brendan]

I’m really not sure what’s going on with Cloudant & Bluemix. I’ve contacted them and I’m waiting to hear back from them. Unfortunately I’m getting the feeling that the people I’ve contacted might not know much of anything. They said they would send me an invitation for a voice call soon, but I never received the invitation and it’s been about a week now. I’m going to try and contact them again.

Tap Forms never had this issue before with Cloudant. The ‘request entity too large’ issue can still happen though if you’ve got records with a ton of data in them or a lot of attachments or large attachments.

I’m also working on some fixes to iCloud syncing which should make it work better and have fewer of those error messages popping up. The current implementation of iCloud sync receives a bunch of records and for each record it checks to see if there are attachments in it and then sends out requests to fetch those attachments. So if you have a lot of photos and/or file attachments in Tap Forms then that can cause a lot of network requests, thus increasing the chance for a failure message. In the next update Tap Forms will get all of the records and attachments for the current sync session first and then it will store them in the database, so there will be FAR fewer fetches from the server. Basically only a few, one for each batch of records being fetched, as opposed to possibly hundreds now.

I don’t know why your Nearby sync isn’t working though. I’ve been testing it too and it is working for me. I know that’s not much to say about anything to help you, but I did check it just yesterday to see if it’s still working and I was able to sync in both directions. Although I didn’t test it much longer than that to see if it’s still working.

[Brendan]

By the way, what server address did you use to sign-in to Cloudant with Tap Forms? As far as I know, the regular username.cloudant.com addresses aren’t working.

[scneophyte]

One of the main functions that TF performs for me is to keep records (photos) of paper receipts. I have hundreds of them in the database (550MB) so that may explain the iCloud / Cloudant issues. My thoughts for Nearby sync was to try with a different, smaller database and see if I can replicate the errors with it.

I’m so glad that I get to answer a question of yours!

1. Signup / Login to Bluemix / Cloudant. When signing up you have to create an organization, pick a region and create a “space”
2. Upgrade to the Standard Pricing Model (I got sync errors on the Lite plan)
3. Create the Cloudant NoSQL DB service
4. Select the Cloudant NoSQL DB service offering
5. You then are at the screen in my attachment. On the left, there is Manage, Service credentials, Plan and Connections. Select Service credentials
6. Click New credential
7. I accepted the default name and then OK
8. Use the dropdown arrow next to “View Credentials” and copy / paste the username and password. These are the “server name” and password you need to enter into TF. For example, my username is similar to this: “username”: “7891234-qll3-lk92-woiu4-da0d345tg7b5-bluemix”,
9. Now click Manage on the left-hand side then Launch

I believe at that point the screen should be familiar to Cloudant users. Once I entered the username and password from the service credentials into the TF database on my Mac it appeared in the Databases page from the Cloudant Dashboard.

Furthermore, I confirmed that you can login to cloudant using the username / password from the service credentials

I hope this helps!

For the moment, I increased my iCloud storage to 50GB so I can sync photos from my iPad to Mac that way and will attach them manually on the Mac and will keep playing around with Nearby sync.

Attachments:

You must be logged in to view attached files.

[Brendan]

Oh wow. That’s insane. But I can confirm that the missing step for me was having to manually create credentials. So far it’s working fine with the free tier with a basic database. Haven’t tried it with anything large just yet.

If you’re interested in beta testing my next version of Tap Forms which has some changes to iCloud syncing, you’re welcome to email me and I’ll set you up. I’m just working on an issue right now with one database where it’s not syncing all of the data. I’m not sure if it’s not uploading everything or not downloading everything. But with this particular database it always seems to be initially syncing the exact same number of records from one of the forms, which is about 27 records less than what it should be. I thought I had solved the bug yesterday, but I hadn’t. One thing for sure is the new way I’m syncing is quicker than what I was doing before an hopefully less prone to timeout errors.

[Adam Irvine]

Hi Brendan,

Updated today after having used Dropbox sync to share a database between the family members (different icloud users and not all at the same location) and cannot fathom the cloudant sync. Has it been impacted by the changes and if so, what is your plan to offer the advertised feature set? Dropbox was perfect for us, just a shame the development wasn’t possible to retain (or is it?)

Adam.

[Brendan]

Hi Adam,

I need to update the manual with information abut Bluemix, but if you follow the instructions a couple posts above by scneophyte you should be able to get a Cloudant account working for you to sync between different family members. I really with that Cloudant had stuck with their original sign-up and setup process because it was much much simpler than it is now. Dropbox has abandoned their Dropbox Sync API which Tap Forms was using to sync. So unless I want to write a whole new one by myself which they had an entire engineering team working on before the abandoned it, then Dropbox sync won’t be coming back to Tap Forms. But you never know. Perhaps an open-source project is being worked on which would make it possible again in the future. I should never say never.

[Adam Irvine]

Thanks Brendan for the quick reply.

I’m still unsure a bit on the potential of moving to a standard service and if that means I’ll start getting charges – the lite service just shows up as no orgs available and no space available.
I must admit I’m lost completely with the mention of charging units at lookups/second, writes/second and query/second but the fees involved look like they may amass quickly – what am I missing?

[Brendan]

Hi Adam,

I’m not sure myself just yet. I just created a lite account tonight to test it out, but I got an error almost right away trying to sync a document with a lot of photos. I’m not sure if that’s a limitation of the lite account or what. I’m going to try it with a simpler document and then choose the standard account to see how that works out.

[Eddzo]

Just tried to setup icloudant

I’m getting on the ipad: Error sending to server: Service Unavailable
I’m getting on the macbook: Error sending to server: the request entity is too large…

I deleted all the photos on the test record and still the same thing.
Any ideas?

Amazon servers better?
Godaddy?
????
:)

Attachments:

You must be logged in to view attached files.

[Adam Irvine]

How have you got on Brendan with your account testing from a few weeks back? Really keen to get the cloudant sync working as advertised

[Adam Irvine]

Any news at all? The syncing between different people’s phones (on different icloud accounts) some distance apart is a major thing for how we use this.

[Brendan]

The “Request Entity is too large” error happens when you have a file attachment or photo in your record that’s too big for Cloudant to handle. You’ll have to have smaller files or fewer attachments or fewer images.

[John Richardson]

Quote:

‘Abandoned’ that’s not quite…they in fact did the complete opposite and upgraded the API completely, and also introduced a fully functional open-source SDK!! ;-)

I think it’s worth you really looking again at the information available from Dropbox on this as (re)integration with this service would greatly boost your apps functionality etc.

Here’s a couple of key pointers:

– Dropbox Blog: Deprecation of Sync API V1 (June 2016)
https://blogs.dropbox.com/developers/2016/06/api-v1-deprecated/

Next Steps:“…you will need to update all of your existing apps to use API v2 as soon as possible. We will be shutting down API v1 in June 2017, so you have a year to transition your app to API v2. Check out our API v1 to v2 migration guide for more details on how to migrate your app.”

Detailed Migration Guide:https://www.dropbox.com/developers/reference/migration-guide

Within this documentation you’ll find a list of comparative functions and the changes (if any) from V1 -> V2 along with useful notes on what other than minor syntax has changed between the versions.

For example:

Downloading a file from Dropbox – [V1] /1/files (GET) becomes [V2] /2/files/download

Uploading a file to Dropbox – [V1] /1/files_put becomes [V2] /2/files/upload

Dependant upon the amount of commands you invoke from the API there may not necessarily be a substantial amount of work to re-provision this for V2; using a ‘find & replace’ type matrix could refactor the code for the majority of API calls where there hasn’t been any underlying parameter and/or output changes for example.

Hope this helps.

John.

[Brendan]

Hi John,

By abandon, I mean they abandoned their objective-c Dropbox Sync framework. Tap Forms never used the REST API that you are referring to. At least not directly. I wish it was as simple as renaming a few REST API calls to their version 2 counterparts, but since I never used the REST API directly, it wasn’t possible for me to follow those guidelines.

This was the documentation for the Dropbox Sync API that I was using:

https://www.dropbox.com/developers-v1/sync/docs/osx

I think it’s worth you really looking again at the information available from Dropbox on this as (re)integration with this service would greatly boost your apps functionality etc.

It would be nice to have Dropbox syncing again, but unless you want to volunteer to build one that fits in with my schema, it’s probably not going to happen anytime soon. Unless there’s an open-source sync API that integrates with the Dropbox V2 REST API. But even if there was, I’m not sure I’d be comfortable integrating with it. Who says they won’t deprecate their V2 REST API down the road? Not only did they deprecate their Dropbox Sync framework, they also deprecated their Dropbox Datastore API, which was like a database in the cloud, similar to what Apple’s CloudKit framework provides. That’s what I’m using to provide iCloud sync.

It’s good to discuss these things though. That’s how ideas are created.

Brendan

[Adam Irvine]

Hi Brandon,

I get to the screen on the attachment and can’t seem to progress further without providing credit card details which I’m nervous about doing as I don’t understand the bluemix metrics. Where do you go from there and are there other costs?

Adam.

Attachments:

You must be logged in to view attached files.

[Brendan]

Hi Adam,

Did you follow the guide here?

https://www.tapforms.com/help-mac/5/en/topic/ibm-cloudant-sync

Perhaps you’re missing the “space”.

You may want to contact Cloudant about how to configure it if you’re having troubles with that part. Perhaps they changed things since I updated the manual to discuss the steps for creating a Bluemix account.

Thanks,

Brendan

[Author]

Posts