Duplicate records

[Tim Flick]

Why is it that duplicate records are created when I create the same entry on different computers?

Thanks

[Sam Moffatt]

Tap Forms is based on a multi-master sync approach to data replication. This means that you can create and update records on various devices without it needing to check in with a central server. When you create a new record on each computer, that creates a new unique record on each of the computers. Internally records have a globally unique identifier that identifies them regardless of the record value. When the sync system kicks in, it’ll replicate that record to the peer devices and you end up with what appears to be a duplicate record because you have the record from each computer. Similarly if you edit a record on two different computers, one of the write conflicts will be lost.

The architecture that solves this is a hosted centralised database which enforces consistency but means that if you’re offline or unable to connect that you can’t make any changes to the system. This ensures strong consistency but obviously at the cost of availability.

[Brendan]

Well said Sam.

The analogy I like to use is with identical twins. They both look identical, but they’re not the same person. So you get one of each.

[Tim Flick]

So if I understand, I have to be careful not to enter the same information on two different computers because this will result in duplicate value because each record will have a different record identifier

[Sam Moffatt]

Correct, new records on different devices will show up as duplicates. I’ve been meaning to do a script that could merge records together reasonably intelligently because I sometimes do that with shipment records where I create a new record in two different places without realising it and then need to merge them down. Maybe at the end of the year I’ll knock that one out.

[Brendan]

I suppose at some point I could add a “Merge Selected Records” function to the Records menu in some future distant version of Tap Forms.

[Sam Moffatt]

That’d be cool, it’s a reasonable chunk of work to do the UI for it to make it really pop as an experience. A simple field like “text” could be use left, use right, use left then right or use right then left. Then do you allow editing of the value during the merge process or only after? A field like “number” really only lets you do left, right or a directly entered value at merge time. That works for most of the other scalar types but falls apart when we deal with composite fields like the date/time field which internally has a lot of options behind it as well.

There are a lot of smaller meta-interface problems as well for composite field types. A table field could be left, right or merged for example but if someone wants to review that, what does it look like? Similar for a photos field, how do you handle file name collisions? Do you hash the field values to get unique values that are then cache.

Like I’d like the feature but I feel to do it justice would take a lot of effort. For the record, my dinky little script isn’t going to do the feature justice but should help me out :D

[Author]

Posts