Sync via Synology NAS

[Pyromixer]

Syncing via Synology DiskStation would be nice. Unfortunately there is no CouchDB available for Synology DiskStations.

I found hundreds of apps for Synology. Unfortunately no CouchDB. Here is a selection of Synology Apps. Maybe with one of them the TapForms synchronization would be possible?

Here are the original Synology apps:

https://www.synology.com/en-global/dsm/packages

and here are a list of 3rd party apps for the synology boxes:

http://www.synorepo.tk/index.php?fulllist=true

[Brendan]

Ya, I also have a Synology NAS and it would have been great to have CouchDB available for it.

[Robert Marko]

I’ve actually been researching this. I’ve made a Docker image of CouchDB, and will test the container to see if I can get a workable sync. If I can’t find a package for my Synology DSM I usually go straight to Docker Hub. I believe you can download Docker as an official package for Synology DSM. Give it a shot!

I’ve also made a trial NextCloud container for my Bookmarks since XMarks sync is shutting down soon and the NC container is definitely persistent, even when I turn it on and off within Docker(!).

I’m trialing WebDav sync with other iOS apps unrelated to TapForms as well for my NextCloud instance… It’s been going great so far. I’m consolidating my personal stuff and moving away from proprietary cloud systems, gradually.

Good luck!

[Brendan]

I’d be interested to hear your experience if you manage to get a CouchDB instance running on a Synology NAS.

[David Verdonck]

I got CouchDB running in a Docker container on my Synology NAS for the purpose of syncing Tap Forms data across all my devices.

It’s very simple actually, you can just install the official CouchDB Docker image from the Synology Docker UI, make sure you mount some folders on your NAS for the data and configuration volumes on your Docker container and you’re good to go.

I also got it running via SSL on port 443 through a reverse proxy defined in the Synology Application portal, and secured with the Synology built-in Let’s Encrypt certificate system.

Works like a charm!

[Brendan]

Hi David,

Oh that’s great!

I don’t think it works on my ds411slim though. Docker doesn’t show up as an installable package.

[David Verdonck]

Yes, you’re right, that model is not supported it seems:

https://www.synology.com/en-global/dsm/packages/Docker

[Carmen S.]

Hallo David

We also have a Synology therefore we are also very interested in this solution. Thanks for posting it :-)

However, I have some questions… Can you help me?

It’s very simple actually, you can just install the official CouchDB Docker image from the Synology Docker UI, make sure you mount some folders on your NAS for the data and configuration volumes on your Docker container and you’re good to go.

We have done this.

I also got it running via SSL on port 443 through a reverse proxy defined in the Synology Application portal, and secured with the Synology built-in Let’s Encrypt certificate system.

Can you specify this?
We installed Dockers and CouchDB and now we don’t know what to do.

Thank you!!

Carmen

[Brendan]

Hi Carmen,

I don’t have experience with the Docker thing, but there may be useful information for you in the CouchDB chapter of the online user manual here:

https://www.tapforms.com/help-mac/5/en/topic/couchdb-sync

[ELLIOTTCABLE]

Since I came across this early-on when preparing to set all this up, and it looks like a few people have had trouble with this, I thought I’d apply some of my knowledge and leave a resource for future Googlers.

First off, some links from which I pieced all of this together:

• Brendan’s own Tap Forms documentation: Apache CouchDB Sync
• couchdb – Docker hub: ‘How to use this image’
• Using volumes | Docker Documentation
• Configuration – Apache CouchDB Documentation
• Security: The ‘Admin Party’ – Apache CouchDB Documentation

~~ Prerequisites ~~

First off, I’m assuming you already have Synology’s dynamic DNS aka ‘DDNS’ (tutorial), a custom domain-name (tutorial), and a Let’s Encrypt SSL certificate (tutorial) setup. If you haven’t done those, do them! They’ll make your life easier not just for Tap Forms, but for … basically everything. Very worth your time (and a couple bucks if you don’t already have a domain.)

~~ Installing Docker and CouchDB ~~

First off, visit the ‘Package Center’ to install Docker and Synology’s Docker ‘app’:

Once it’s installed, open it from the Main Menu, in the top left. The overview you’ll be presented with is pretty useless, as you haven’t downloaded any “images” (templates from which you can create apps, basically), nor started any “containers” (said apps) yet.

There’s two images we’re going to want to install and configure: the CouchDB server itself, and the official GUI CouchDB management tool known as ‘Fauxton.’ Go ahead and select the ‘Registry’ tab on the left in the Docker app:

You’re going to download those two images here. For each of these, use the search field to find the image, right-click it, and then click “Download this image.” When asked to choose a version, accept the default of “latest.”

Repeat this for both these image names:

• couchdb
• 3apaxicom/fauxton

When they’re both downloaded, click ‘Images’ in the sidebar, and verify that both of the above are listed.

~~ Starting CouchDB ~~

Next, we’re going to create an actual container out of the CouchDB image — an instance of the app, configured and running on your Synology.

In that ‘Images’ section of the Docker app, select the CouchDB image, and then click ‘Launch’ to bring up Synology’s container-wizard:

Give the container a unique name, and feel free to limit the available resources; then click ‘Advanced Settings:’

Configure the following:

1. Enable ‘auto-restart,’ to ensure Tap Forms can always sync.
2. Create a folder on your NAS for the CouchDB server’s data and configuration. I used <volume>/docker/tapforms/ as the root, for instance; and you’ll need several subdirectories, as described below.

   Within that root, you’re then going to add add Docker volume mappings to each of the CouchDB directories:

   • <volume>/docker/tapforms/etc, mounted to the image’s /opt/couchdb/etc/local.d
   • <volume>/docker/tapforms/data, mounted to the image’s /opt/couchdb/data
   • <volume>/docker/tapforms/logs (plural), mounted to the image’s /opt/couchdb/log (singular)

   For each of the above (under the ‘Volumes’ tab of ‘advanced settings’ while creating the container) you’ll have to click ‘Add Folder’, and select the first path — on your NAS volume — from a tree:

   

   Then, you’ll have to click the ‘Mount path’ field, and type the second path — the one inside the container — into the field:

   

3. Map the Docker container’s CouchDB port to the NAS’s local, internal CouchDB port.

   Switch to the ‘Port Settings’ tab. Click ‘Auto’ next to the entry for 5984, and change that to also be 5984:

   

Finally, click ‘Apply’ to close the advanced settings, and then click ‘Next.’ You should see a summary of the settings (the super-important volume-mount paths won’t be listed, so watch out.) Click ‘Apply’ again:

Synology will now boot the container in the background; so let’s go setup the other one while we wait.

~~ Starting Fauxton ~~

Repeat the above, but for the 3apaxicom/fauxton container. Give it a unique name, and limit the resources appropriately. Tell it to create a DSM desktop shortcut, at the port 8006 — that’s not a typo, we’ll get it all hooked up together later:

Change the Synology-local port from ‘Auto’ to 8005; (again, not a typo):

Now, for a weird bit: you need to create a ‘link’ between the containers. This is a secure, secret port, basically. Click the ‘Links’ tab of the advanced settings, and click the ‘+’ to add one. Select your CouchDB container; and then type db as the alias:

Finally, change the ‘Execution Command’ under ‘Environment’ to exactly this command, quote-marks and everything: sh -c 'fauxton -c http://$DB_PORT_5984_TCP_ADDR:$DB_PORT_5984_TCP_PORT'

Apply, Next, double-check, Apply. Boom! Both containers should be running. Click over to the ‘Containers’ tab in the Docker-app’s sidebar to double-check:

~~ Configuring port-forwarding, SSL, and reverse-proxying ~~

Phew! We’re halfway there. Let’s take a break from Docker, and double-check some Synology DSM settings that are really important. First, Synology’s built-in reverse proxy — this will ‘hide’ all our shmancy CouchDB passwords and stuff behind secure HTTPS / TLS encryption.

Twice, we’re gonna: open ‘Control Panel,’ ‘Application Portal’ (yeah, I don’t get the name either), ‘Reverse Proxy.’ Click ‘Create.’

You’re gonna route incoming requests from (confusingly called ‘Source’) HTTPS, on your domain-name, port 6984, to (called ‘Destination’) HTTP on localhost, port 5984:

Hit ‘OK,’ then repeat for the web-GUI we installed — this time, route incoming requests from HTTPS, on your domain-name, port 8006, to HTTP on localhost, port 8005. You can also enable HSTS and HTTP/2 on this one.

Then, you’ll need to configure your router’s port-forwarding. Luckily, Synology is absurdly badass, and can do most of that for most of us — as long as your router has something called UPNP. Synology has rather thorough documentation on this, but at least for me, it was about this simple: open ‘Control Panel,’ ‘External Access,’ ‘Router Configuration.’ Click the ‘Set up router’ button at the top.

Then, ‘Create’ some rules, and select ‘Built-in application.’ Do not select the Docker entries — select, instead, the Reverse Proxy entries you just created for ports 8006 and 6984; then hit ‘Apply.’

(You may have to do this manually, on your router, instead. I’m afraid I can’t help you there!)

~~ Configuring CouchDB ~~

Let’s verify that everything’s routed properly. Open https://yourdomain.blah:8006 in a new tab (you should be able to just click the icon on the DSM homescreen, too, if you set that up above.)

If Fauxton loads, then head on over to the ‘Verify’ tab on the left, and click ‘Verify installation.’ Everything should succeed (except maybe the replication section? Not sure what’s up with that, but it consistently fails for me.)

Next, we need to lock down the installation. Right now, your CouchDB is running in “Admin Party” mode — which is a cute way of saying ‘horribly insecure.’ Literally everybody on the Internet currently has admin access to your database, oh no!

Let’s fix that. Click ‘Setup’ in the sidebar, then ‘Configure a Single Node.’

You’ll be presented with a form via which you can add what is effectively a root password to your CouchDB server. (Note that this is *not* your personal non-admin user-account! We’ll create that later.) Choose a secure root password, write it down somewhere (may I heartily suggest 1Password? ?), and create your administration account. Finally, enter :: (exactly two colons, no spacing) as the ‘bind address’, and leave the port alone.

Once you’ve submitted that, reload the page — Fauxton won’t be able to successfully make any more changes to your CouchDB installation until you’re logged-in under that root account. Enter the information you just saved, and hit ‘Log In.’

Switch over to the ‘Config’ sidebar entry, and let’s tweak some configuration values:

1. Set [chttpd] require_valid_user to true
2. Set [log] file to /opt/couchdb/log/couch.log
3. Set [log] writer to file

Now — before you close the page! — you need to make sure Fauxton can still authenticate with CouchDB. This isn’t very clear, and took me ages of searching, but it turns out you need to *add* a config-option. Click ‘Add Option’ at the top of the page, and enter the following three values, precisely: httpd, WWW-Authenticate, and Basic realm="administrator" — spaces, quotes, and all. Then click ‘Create.’

The very last step of configuration there, is to create a non-admin user for Tap Forms to authenticate as. (This is optional, but a good idea in general.)

Click over to the ‘Databases’ tab in the sidebar, and select the _users database. Hit ‘Create Document.’

Delete everything in the text-editor that appears, and replace it with the snippet from this Stack Overflow answer, changing your desired user-name (in two places!) and password as appropriate:

Click ‘Create Document.’ (Note that CouchDB will hash up that password you enter and store it safely; you’ll never be able to view it again. Store it in 1Password, too!)

Now that you’ve done all that, reboot the CouchDB container so that the new configuration will take effect. Return to DSM, open the Docker app’s ‘Containers’ tab, right-click on your CouchDB container, and select the ‘Restart’ action:

If you’re Terminal-savvy, you can give your server a real quick test: run curl https://ds.ell.io:6984/, replacing the domain with the one you’ve set up for your DiskStation or your Synology.me DDNS domain. If you’ve got everything working, you will get an error — that’s what we want! Something along the lines of {"error":"unauthorized","reason":"Authentication required."}.

~~ Configuring Tap Forms ~~

You’re in the home stretch! I know it’s been a slog, but we’re, finally, almost done.

The very last step is to configure Tap Forms itself on your devices. Brendan covers it pretty well, so I’ll just add some addenda and errata to his writeup here:

• Ignore ‘Installing CouchDB’ and ‘Setup,’ we just did all that …
• … except for the notes on admin accounts. Basically, you should sign into the root account we created earlier inside a Tap Forms app at least once, so it can create your db-deadb33f... databases. Then you can tap sign out, and sign back in using the second, non-administrator account you created.
• Pretty much everything about SSL can be ignored: Synology’s reverse-proxy handled this for us.
• It was unclear to me, and maybe also to you … so note that all of the above is specific to a single ‘database’ in Tap Forms. There’s no way to configure Tap Forms as a whole to sync all of its databases via CouchDB, and you’ll have to duplicate the setup for every new database you create.

Hope this was helpful to someone! Maybe I shouldn’t have spent like six hours writing it up, yikes. I’ll be ‘following’ this thread via WordPress’s settings, so I’ll get an e-mail if you have any questions — though I’m not sure how much help I can offer, as I’m definitely not an expert on any of the above!

[ELLIOTTCABLE]

I think my posts to this thread are getting spam-filtered. @Brendan, can you restore my original post? It has, uh, a … lot of links and images, so no wonder. ¯\_(ツ)_/¯

[Brendan]

I’ll look into it.

[Brendan]

I restored two posts that were marked as spam, but they were both the same, so I deleted one of them.

That’s really great information!

Thanks very much for sharing Elliott!

[daffy]

Hi, this is in reply to the detailed help provided by @elliottcable above.

I am wanting to use CouchDB on my synology drive because I’m having trouble with the iCloud sync.

My concern is that I am not quite techie enough to do it. I did manage to set up the NAS drive, but it took a lot of reading and testing (and patience!).

How user friendly is the system you have described? Would I need specialist knowledge to enable me to set it up?

I have a domain name I can use and I understand SSl enough to be able to set it up (I think), but I’m unsure about the DDNS instructions you linked to.
https://www.synology.com/en-us/knowledgebase/DSM/help/DSM/AdminCenter/connection_ddns
I thought it would be something I needed to install on my NAS, but perhaps I misunderstood. I find the Synology ‘help’ files often miss out crucial info that would enable someone with slightly less tech knowledge to be able to implement their instructions.

I’d very much appreciate your thoughts on how achievable this might be for me. If it’s way above my knowledge and I can’t learn enough by reading up, then perhaps it’s too much for me to embark on.

Cheers,
Nicky

[ELLIOTTCABLE]

Hey there!

1. The process is definitely long, but, and this is key, not necessarily particularly arcane, as these things go. That is, each of the steps or components of the process I detailed above, is a well-known / well-documented Thing That You Can Google; those ‘TTYCGs’ are mostly an issue of figuring out the right words to search for, and I hope you can glean some of those from my instructions if you run into any problems!
2. DDNS is a good example: if you google “dyndns” or “dynamic DNS”, you can find a wealth of information. It won’t be instant or trivial, but you will probably be able to relate most of what you find back to either the Synology docs, the Synology interface buttons ‘n widgets ‘n doodles, or at worst, the tutorial I link above (note that I linked an external tutorial for each of those topics; I, too, thought the Synology docs weren’t particularly beginner-friendly.) In the specific case of DDNS, don’t worry too much — basically select the Synology option, register a free account, and setup a daffy.synology.me or something domain right there in the Synology settings once you’re registered.
3. Finally, a general thought, if you’re largely not technical: simultaneously the best, and worst, thing about tech? No matter how complicated it is, you can always get there. It just might take a while. The thing to note is that it took me most of an afternoon and evening to figure out (well, and document … but still.) all of the above, and I’m very experienced with a lot of the various moving components. I strongly feel that you could figure it out, too, whatever your background — but do be prepared to spend a lot of time Googling things, banging your head against poor documentation, or writing up questions on the Internet! If “many hours invested” doesn’t sound like a good trade-off for this, that’s when to go ‘hm, is this really worth my time.’

tl;dr you can do it! it might take a while tho. Good luck!

[daffy]

Wow, thank you so much for such a thoughtful and detailed reply. Much appreciated.
I do feel optimistic about giving it a go. Setting up the NAS drive itself took several head bangs against the wall, lots of swearing and waking away (sometimes for several days!) before coming back and trying again, but I did it and it felt great.

I think the key for me will be choosing the right time to look at it. I have a lot on at the minute and if I let this distract me I will probably get more frustrated with it because I know I should be spending my time elsewhere.

Thanks again for the info. I will certainly be trying it out. :)

[t fullerton]

I also have a synology so im super excited to try and navigate these steps by steps provided above.

TIA

[Wonnie Bad]

Thank you ELLIOTTCABLE. I am working on a QNAP NAS, and am starting to see the light here and there, peeking through.

For other people with a QNAP, there is a CouchDB package available, not directly from QNAP but from a third party, qnapclub.eu. It requires that you sign up, which I did, installed the package and it worked right away.

[Jong-soo Anh]

Thanks a million for this detailed tutorial! I am far from being an expert or even an advanced user myself, and I definitely couldn’t have done it without this precious walkthrough.
A thing has changed since it was written: CouchDB will no longer accept to run on “admin party” mode, so I added an admin and password in the variables environment. I have no clue about the security of such thing, but well… It works.

From the log:

ERROR: CouchDB 3.0+ will no longer run in "Admin Party"
       mode. You *MUST* specify an admin user and
       password, either via your own .ini file mapped
       into the container at /opt/couchdb/etc/local.ini
       or inside /opt/couchdb/etc/local.d, or with
       "-e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password"
       to set it via "docker run".

Something funny, when I start with a fresh install of the iOS Tap Forms app, there is apparently no way to directly retrieve the existing CouchDB database, I had to first go through the iCloud sync in order to have the same DB in all devices. Otherwise Tap Forms just creates a new one and won’t sync. Maybe it’s me though… Anyway, it works now, synchronization is done flawlessly.

Another thing: I first created an autosigned SSL certificate. It worked fine with Tap Forms on my Mac being on the local network, but when I tried to connect from the iOS app it wouldn’t work, stating that the domain might be attempting to steal the original domain’s identity or something. When I switched to a certificate generated by Let’s Encrypt, then there was no problem anymore (at first I thought you needed your very own domain name in order to get such certificate, that’s why I didn’t go with this option).

Thanks again.
Great app, by the way.

[Sam Moffatt]

Generally my first step when creating a new document is to sync it to iCloud to get the placeholder document on all devices and then disable iCloud sync. That gives me an empty placeholder document on every device I own which makes it easy to open, it does the empty document sync initially which is quick, I then go to disable iCloud sync to swap over to CouchDB. It seems to work well for me and makes it easy to open the document and get going. One obviously needs access to the internet and your CouchDB server to get fully set up but I generally don’t see that as problematic.

For existing documents that you didn’t have on iCloud already, if you send the document from the original document you setup your sync from to the other devices. You should be able to use Airdrop to transfer between devices relatively easily and you can use iTunes to transfer as well. Check out the “send document to another device” page in the sync section.

SSL is a bit of a moving target as Apple keeps increasing their minimum security requirements for apps, great to hear the Let’s Encrypt certificate worked for you though.

[Brendan]

Hi Jong-Soo,

Excellent! I’m glad that you got sync working with CouchDB on your NAS!

Ya, as Sam mentioned, I just always use the Send Document function to send a copy of the document from device to device. You can also just use the Finder’s AirDrop function too, to send to your iOS devices. That works too.

Thanks!

Brendan

[Cornelius Fischer]

Hi all – I just followed the installation guide from @ELLIOTTCABL Thanks for this superb guide!

As I struggled a bit with the Fauxton, I like to add some addons here for the installation.
For me it seems that the Fauxton Container is not necessary anymore. I followed the guide but could not connect with the Fauxton (https://mynas.xy:8005). It just did not load anything.

But when I opened https://local-ip:5984/_util#login, which is basically the local port for the CouchDB container, I ended up on a Fauxton UI, while the fauxton docker container was offline. ?‍♂️
So it seems, that the latest CouchDB docker package already contains a fauxton UI for administration.

After I found out this, I could finally configure my CouchDB following the steps from @ELLIOTTCABL (the part about fauxton) and establish a connection from TapForms. ?✅

So as summary, you can use this superb guidline but just jump over the installation (docker container, port/reverse proxy etc. stuff) for the Fauxton docker package. It seems, that this is not necessary anymore.

[Author]

Posts