Slow import of very large file

Viewing 3 reply threads
  • Author
    Posts
  • July 14, 2019 at 12:18 PM #35902

    Philip Tate
    Participant

    I’m not surprised, or complaining, about the slow import of 400,000 records, but I do wonder how long it will take. It’s been a couple of hours and it’s still using 100% cpu and the app is non-responsive.
    Running on 2017 iMac Retina 4K with 16GB RAM. OS X 10.14.5 and TapForms 5.3.7.
    I don’t have the option of importing a subset of the csv file. It has at least a dozen columns.
    I’ve also quit apps to keep RAM usage down.

    July 14, 2019 at 2:24 PM #35903

    Philip Tate
    Participant

    Update. It finished the import after four hours or so.

    July 15, 2019 at 12:40 PM #35905

    Brendan
    Keymaster

    Oh wow, ya, that’s a heck of a lot of records for Tap Forms to handle.

    The most I’ve ever really imported was about 125,000 and even with that many it took about 10 seconds or so just to fetch the records.

    If you can split that up into smaller forms with fewer records that would be better. Or at least create a bunch of Saved Searches to filter the records so you don’t have to click the main form. Tap Forms will load in every record into memory when you click on the main form you imported. That will take time.

    Tap Forms isn’t really optimized to handle that many records at once.

    July 15, 2019 at 1:42 PM #35909

    Philip Tate
    Participant

    Thanks. I finally used BBEdit to cut out the records I didn’t need and re-imported so TapForms is much quicker now.

Viewing 3 reply threads

You must be logged in to reply to this topic.