Tap Forms for RAG

[JScottA]

I see a lot of potential for using Tap Forms for RAG (Retrieval Augmented Generation) for AI tools. Is there any chance this is being explored? Is there anything that the users can do to help with this if it is being explored? Are you waiting to see what Apple introduces at WWDC in the way of AI developer tools and frameworks for GenAI?

[Brendan]

Hi James,

I’m not sure what Retrieval Augmented Generation is. But yes, I’m waiting to see what Apple provides developers at WWDC. Hopefully something that doesn’t cost us to integrate into our apps.

I did write a simple ChatGPT script in Tap Forms to call out to their service though. It worked pretty well. Type in your question or statement into a field, then it goes out and gets the response and populates the Script field with the results. Great to keep int he database and be able to do queries on it.

But I can see other uses for AI such as to create forms for you or analyze your data.

Thanks,

Brendan

[Chris Ju]

I suppose many LLMs should be possible (e.g. Meta LLaMa) because many of them can be called with http/url commands using JavaScript …

[JScottA]

Good morning, Brendan. RAG is a technique for a Generative AI (GenAI or just AI these days), to access a specific external data source for specific knowledge. For example, a chatbot on a website for customer service or sales support would be best if it had a live, up-to-date, source of information for policies, procedures, products, and services. One or more databases would be linked to the chatbot so that it has what it needs to give good information and not hallucinate or just be ignorant of possible answers.

So, to keep it simple, imagine a Tap Form database of internal contact information for a company. It would have your standard phone numbers, email addresses, physical addresses, etc. and maybe a bit on the actual responsibilities and authority of each person. If you wanted to find the right person in your company to deal with an issue, you could explain the issue to your internal chatbot and it would be able to find the right person or people for you.

And this goes beyond keyword searches. GenAI is able to infer intent and additional meaning and context from a conversation. Hopefully, Apple will provide you with the Core[whatever they call it] framework so you can integrate GenAI directly into your app for directly helping users with your app.

But the first thing I thought of as a useful tool would be access to a database for structured RAG that is also local and live. If you ever want to discuss the possibilities and potential methods of executing and of this let me know.

As a side note, I’m really hoping that Apple will integrate GenAI into Xcode for enable faster, better, app development. There are so many possibilities with this in Xcode.

BTW…you may want to consider beefing up your AppleScript support and adding Shortcuts triggers/actions.

A use case for this (and it would replace my current use of AirTable and Zapier Tables) is helping a Mac magazine with the press release handling. This is where they have a bunch of RSS feeds from manufacturers that cover the company news (press releases) that each sends out. The workflow pulls the press release (article) information from the RSS feed and places all of the information into a database (AirTable for this one). Then the workflow goes through a variety of tasks that use Generative AI to perform. Finally, a magazine editor reviews the processed articles for publishing and makes the decisions on things like titles, pull quotes, summaries, adds opinions, images, etc. All of this and the status of the article is stored in the database. Finally, the workflow pulls the final article information from the database and publishes it to the various platforms (magazine website, X.com, LinkedIn, etc.).

Sorry for the long post, but I see a huge opportunity here for a local Mac database tool. Filemaker has blown it for smaller developers with local use cases because of their cost. I think that aggressive AI integration and positioning as a local Mac RAG resource could be a nice first mover for you.

Have a great day.

[JScottA]

I think you are thinking of calling an LLM from within Tap Forms. That is another use case. I’m suggesting that Brendan look into making sure his tool is set up for the various GenAI tools are able to access Tap Forms database easily. This could be accomplished by providing an API into Tap Forms, AppleScript, or Shortcuts. Each method has pros/cons, but all should probably be pursued.

However, I think that Brendan, like everyone else in the Apple ecosystem, is waiting to see what Apple does with AI. Hopefully, the annual release schedule with long times before actual release will not cause Apple a fall similar to Nokia/Blackberry. And annual release schedule in a world where the state of the technology and tools/products is advancing daily seems like a very bad idea.

[Brendan]

Hi James,

That’s an interesting use case for AI. We’ll see what Apple has up its sleeves for integrating AI into their OS. Right now the script I wrote simply establishes a REST connection to OpenAI and submits a message and gets a response. I also wrote another script I can use for translating forms to different languages. It’s still a work-in-progress though.

I agree better AppleScript and Shortcuts support is needed. You can also call into Tap Forms with a URL that can execute a script. You can even pass Tap Forms parameters that way.

See the section on this page called “Calling a Script from a URL”:

https://www.tapforms.com/help-mac/5.3/en/topic/javascript-api

Thanks,

Brendan

[JScottA]

Brenan, what are your thoughts now? With App Intents and Apple Intelligence, I think you will have your plate full of things that you can do and add to Tap Forms and you may have a larger audience that will have a reason to store more things in a database since that information can be made available to Apple Intelligence for Siri to use in other apps easily.

For example, a user could ask Siri to pull sales for the past month made to people in the state of Maine over the past month and put them in a table in a new email to Bob, the CEO. Siri will do all of that for them without any further user interaction.

And, if I’m seeing it correctly, using App Intents you might be able to chat with Siri to get a list of common attributes to track in a home inventory database and then ask Siri to create that table in Tap Forms for you–with no further interaction from the user. Then you could ask Siri to use a Numbers spreadsheet you’ve been tracking your home inventory inside of and import that to the new Tap Forms database…all with voice.

What are your thoughts?

[Brendan]

Well, the one about asking Siri to create a table for you in Tap Forms is something I’ve already been thinking about for the new template library I’m building. With a prompt of something like “Can’t find a form you’re looking for? Describe it here and we’ll make it for you:”. Or something like that.

I’ll have to play around with it for a while to get an idea of what sorts of things I can do with it from a development perspective.

Lots of fun things to play with for sure from WWDC today.

[JScottA]

Excellent! I’ve been using a workaround for that. I ask ChatGPT for a suggestion of items to track in a database for a particular need. It gives me a list of fields to track. Then I ask it to give me a CSV file with those fields and some sample data in say 10 records. I download the file and import the CSV into a new Tap Form form and I have a database that just needs me to change up the fields types and other details.

But don’t forget the RAG. Apple Intelligence is personal intelligence. Making your database fully accessible to Siri will enable the GenAI to know a lot more personal information about the user’s context. Apple Notes and loose documents laying around the file system will provide a lot of unstructured context. But Tap Forms could provide a pace for more structured context.

[JScottA]

Siri will have access to data if the application opts in.

 

[JScottA]

FYI…if you want to see another use case for TapForms, take a look at AirTable, Zapier’s Tables, and even DataJar on iOS. These are all places where data is being stored/retrieved for use in automation. Depending on how implemented, if TapForms can expose data and provide useful Actions in Shortcuts, it could be a very nice addition to the Apple Automation scene.

[Brendan]

the troubles I always had with Shortcuts was because Tap Forms is a dynamic database engine, it’s impossible to present Shortcuts with a specific list of fields to ask the user to enter data for. For example, if an app was solely designed for keeping track of your movies, you would have fields for movie title, actor, rating, genre, and so on. Shortcuts needs to have a list of fields to prompt the user for at compile time. But with Tap Forms you can define your own fields, so it’s impossible to know what they are at compile time. Tricky.

[JScottA]

I think that is a tech problem that has been solved in most places, else tools like Make.com couldn’t access resources like AirTable. Even Zapier accesses another large dynamic database for a CRM (Daylite). However, it might be a problem for Shortcuts because Apple may not provide the methods for you to specify which database/fields an Action wants to access.

For an intermediary step, what can you access/control using AppleScript? I see that you do have some access to Tap Forms through AppleScript, but not enough to really use it for automation or RAG.

Here is a real world use case: Content Management for News Journal

Say I am an editor/journalist for a tech magazine. One of my less interesting but expected jobs is to post news about various company press releases. Typically, this is a time consumer without really being “journalistic”. So I want to use Shortcuts daily to poll all of the vendors I track (e.g. Apple, Belkin, Microsoft, Logitech, OpenAI, etc. there could be hundreds here). It will pull the news stories posted in the past 24 hours and store all of the information for each press release into Tap Forms (normally I use AirTable to do this currently). It then passes certain information from Tap Forms to ChatGPT to process (Title suggestions, keyword suggestions, summary suggestions per platform I publish to, suggested images for the my posts, etc). It presents the articles to me review along with the suggestions for me to edit/approve for publishing. The Shortcut then sends the edits and status of the article (e.g. published, postponed, not published, etc.) to Tap Forms.

This approach keeps everything on my local machine, justifies my buying and keeping up-to-date Tap Forms because I can eliminate one or more cloud subscriptions, and I have greater control, privacy, and security for my data. And this is just one use case. But we need Tap Forms to powerfully support AppleScript/Shortcuts.

[Brendan]

Sounds like a pretty comprehensive workflow.

Hopefully at some point Tap Forms can do some of that for you with shortcuts. We’ll see. It’s a lot of work to integrate shortcuts into an app like Tap Forms (for the reasons I mentioned above).

[Author]

Posts