In this release, Jade’s output has been improved to limit scrolling when responses are shown, making it easier to read through the specific information and code samples. In addition, Jade now does a better job of parsing the results it gets, which prevents situations where responses could sometimes get cut off.

Lastly, Jade has had some fixes to improve reliability and has been updated to use Anthropic Claude Sonnet 4.6 for more detailed and accurate responses.

These improvements may be small, but they allow Jade to continue to be a helpful Xojo coding assistant.

Paul learned to program in BASIC at age 13 and has programmed in more languages than he remembers, with Xojo being an obvious favorite. When not working on Xojo, you can find him talking about retrocomputing at Goto 10 and on Mastodon @lefebvre@hachyderm.io.

• Facebook
• X
• LinkedIn
• GitHub
• YouTube

I wanted something better. Not a generic chatbot that happens to know some Xojo, but a tool that could actually see what I was working on. That’s why I built Zotto (with Xojo!).

What Is Zotto?

Zotto is a cross-platform macOS and Windows app that acts as an AI pair programmer specifically for Xojo developers. It connects directly to the Xojo IDE, reads the project you’re working on, and gives the AI real context about your code such as classes, method signatures, properties. Essentially the whole project’s structure.

The Problem It Solves

The core issue with using general-purpose AI tools for Xojo development is context. When you ask Claude or ChatGPT a question about your project, the AI has no idea what your codebase looks like. You become the middleman, manually feeding it snippets and hoping it has enough information to give useful advice.

Zotto removes some of that friction. When connected to the IDE, the AI can explore your project on its own using built-in tools:

• ProjectOverview gives it the high-level folder structure
• ListItems lets it drill into specific folders
• GetSignatures shows method and property signatures without dumping all the code
• ReadCode pulls up full method implementations when needed
• SearchCode does grep-like pattern searches across the codebase
• SearchDocs lets it search and retrieve Xojo documentation directly

These tools are designed to be token-efficient. Instead of dumping your entire project into the context window, the AI requests only what it needs, when it needs it. A ProjectOverview call might use 300-500 tokens. A GetSignatures call on a class gives the AI the API surface without the implementation details. This layered approach means even large projects work well within context limits.

Read-Only by Design

One decision I made early on was that Zotto would be read-only. It can read your code, analyse it, suggest changes, and generate snippets — but it cannot modify your project files, create new items, or execute IDE scripts.

This is intentional. I think AI-generated code should be reviewed by a developer before it goes into a project. Zotto gives you copy-pasteable suggestions and lets you decide what to implement. You stay in control.

Bring Your Own Model

Zotto isn’t locked to a single AI provider. It supports Claude, OpenAI, Gemini, Ollama, LM Studio, and any OpenAI-compatible API (which covers services like Grok and OpenRouter). If you’re privacy-conscious or working on sensitive code, you can run it entirely offline with a local model through Ollama or LM Studio. Your code never has to leave your machine.

This also means you’re not paying for Zotto’s AI usage — you use your own API keys or local models, so you’re in full control of costs.

How It Works in Practice

A typical workflow looks like this:

1. Open your project in the Xojo IDE
2. Launch Zotto — it detects the IDE automatically
3. Select which open project to work with
4. Start chatting

From there, you might ask the AI to review a class, suggest how to refactor a method, help you understand unfamiliar code in a project you’ve inherited, or brainstorm an approach to a new feature. Because the AI can explore the project itself, the conversation feels much more natural than the copy-paste dance with a generic chatbot.

For example, you could ask: “What does the ProcessOrder method in OrderManager do?” Instead of you having to find and paste the code, Zotto’s AI will call ReadCode to look at the method itself, then maybe GetSignatures on the parent class to understand the broader context, and give you a thorough explanation.

Extending with MCP

For developers who want to go further, Zotto supports the Model Context Protocol (MCP). This lets you add custom tool servers — for file system access, Git operations, database queries, or anything else you might want the AI to have access to during a conversation. It’s the same protocol that Claude Desktop and other AI tools support, so existing MCP servers work out of the box.

Built for the Community

Zotto exists because I needed it for my own Xojo development. I was spending too much time being the translator between AI tools and my codebase. Once I had something working, it seemed like other Xojo developers would find it useful too.

There’s a free version you can try that includes full IDE integration and all the built-in tools with a single conversation. A full license is a one-time purchase of £69 — no subscription. You can learn more and download it at zotto.app.

If you have questions or feedback, I’d love to hear from you on the forum.

Garry Pettet is a Consultant Radiologist, Xojo developer and the creator of Zotto. You can find him on the Xojo forum or at zotto.app.

• Facebook
• X
• LinkedIn
• GitHub
• YouTube

Getting Started

Jade uses Anthropic Claude as its engine, so before using Jade, you first need to sign up for an Anthropic API key to access Claude. You can do this from the Anthropic Claude Console.

From there you can create your account. After you’ve done that, you will want to get an API key to use with Xojo. Click on the “API keys” tab to see existing keys and create new ones. Click the “Create Key” button to generate a new key. In the dialog, give it a name to indicate its purpose.

When you click Add, the actual key will be shown. Copy it to the clipboard.

Note that you cannot see the key again after closing that dialog, but you can always create a new key at any time.

Now go to Xojo Settings and in the General tab, paste it into the “Anthropic API Key” field.

Go back to the Claude Console and click on the Billing tab to buy credits. Interacting with Jade requires credits. The amount of credits used depends on the complexity of your interactions. Until you get used to Jade and credit usage, you’ll probably want to start with a small amount, like $5, and disable auto-reload.

Using Jade

To use Jade, select Help->Ask Jade or click the Ask Jade button in the toolbar. You’ll get some short introductory text. Note: Using Jade requires an internet connection.

You interact with Jade similarly to a chat app by typing prompts in the conversation area (bottom section) and pressing Send. Jade will think for a moment and then display its response.

If Jade’s response includes Xojo source code you can click one of the buttons above the code block to copy the code to the clipboard or to directly insert it into the active code editor.

When interacting with Jade, keep in mind that although it remembers the context from the current conversation, it has no knowledge of prior conversations. Click the “Start Over” button to clear things and start a new conversation. Starting a new conversation is useful when switching to a different topic as it prevents Jade from relying on information you previously discussed that is no longer relevant.

The Export button saves the current Jade conversation to a JSON file which is a great way to save information you want to retain and refer to later.

Other Tips

• Jade currently uses Claude Sonnet 4, but this may change as newer models become available.
• The only thing Jade knows about your project is what type it is (Desktop, Web, iOS, etc.). If you want Jade to offer suggestions based on your code, you will need to copy/paste the relevant code into the Jade conversation area.
• Your Jade conversation is restored when the IDE restarts. If you get errors, ensure that you have valid internet access, that the API key is valid and you have a positive credit amount.

Wrap

We hope you find Jade helpful with your Xojo development. Remember, Jade in Xojo 2025r3 is the first version of an AI assistant for Xojo and will improve over time. In subsequent releases we expect to add more capabilities and project integration. Read more about Jade in Xojo’s Documentation.

Paul learned to program in BASIC at age 13 and has programmed in more languages than he remembers, with Xojo being an obvious favorite. When not working on Xojo, you can find him talking about retrocomputing at Goto 10 and on Mastodon @lefebvre@hachyderm.io.

• Facebook
• X
• LinkedIn
• GitHub
• YouTube

The reason to create this kind of project, as the maintainer of the Xojo Web framework, is to find pain points and bugs in larger projects, as the issue reports I normally work with are very specific sample projects demonstrating a bug. The next Xojo release, 2024r3, will come with some bug fixes I found while building Cosmic Trader.

Final Result

Let’s begin at the end. Give the game a try and then come back to this blog post. It’s a very simple space trading game, don’t expect too much from it, but I hope you enjoy it.

It can be played from the following URL:
https://cosmictrader.rcruz.es

The Challenge

Create a simple game using Xojo Web, without writing a line of JavaScript, or even using the Web SDK.

In addition, while all this could be just a couple of WebListBox and WebButton controls, I really wanted this game to look like an actual game. As you can see in the screenshots, unless you check the code, nobody will notice (or care!) it’s a Xojo Web application.

As you can imagine, Xojo Web was not created with the idea of building web based games. In order to make it easier for the developer, everything is processed server side: When you press a button, the browser will send a notification to the server, your code written in Xojo will be executed and, finally, the response will be sent back to the browser.

That’s the known knowns. I knew the latency would be a problem, but I’ve found more.

What I’ve Enjoyed

Implementing the first prototype was really easy using the tools Xojo Web offers. For example, in the prototype, instead of the custom popover I ended up building for buying and selling, I just used some contextual menus:

You can go very far and prototype your idea very quickly this way. This helped me to deploy and share the game with a few friends and gather some initial feedback.

Once you have the game mechanics, you can iterate as much as you want until the game is ready. The welcome screen with the game logo wasn’t added until very late. Just so you can compare, this is how the current version looks:

Much better!

I’ve really enjoyed being able to use Containers for building custom controls and reusing them. Also, being able to just drop some images into the IDE and assign them to image viewers is very pleasant.

From the IDE preview, it’s deadly easy to identify where my ship entity is, click on it, then edit the code if needed:

Navigating through the project is a joy compared to jumping through piles of code in plain text files.

I’ve also found having Control Sets useful, allowing me to implement the Pressed event just once for all the planets, for example.

Roadblocks and How I Handled Them

That said, I found some difficulties. All of them were due the nature of web projects. When you travel from one planet to another, you will hear an engine sound. It would be much (much!) easier to know when I should stop the sound in a Desktop or a Mobile project.

Having to deal with latency means I had to use a timer and guess more or less how many milliseconds is reasonable before stopping the sound. Using the same amount of time as the animation would cause the sound to continue playing after the ship had arrived at its destination, again due the latency.

Where I spent most of the time was optimizing the images. Specifically, delivering the images. When you play the game locally, everything works pretty fast. I found this flickering issue once I deployed the game for the first time using the custom popovers:

This is something that you just don’t need to think about with the typical database application you normally build with Xojo Web. But if you build a lot of images on the fly, it will mean the browser will have to download them again and again.

In order to optimize it, instead of using a WebImageViewer.Image property, you could store WebImage instances. Using WebImageViewer.URL pointing to the WebImage.URL property of those instances will make the browser first check if it has downloaded that URL before. If so, bingo! It will reuse the image without having to re-download it again.

If you go back to the game, cache is the reason behind the “New Game” button, from the welcome screen, and the “Sell Everything” button having the same width. It wasn’t a coincidence, that means a browser cache hit.

Last but not least, when I shared the URL with some friends, they told me it didn’t look great on mobile. So I had to change the stats a bit so they fit in a mobile screen. While in this case it wasn’t a big deal, at Xojo we know making responsive web applications must be easier, and we will improve the workflow in future releases.

It would be much easier to use the Web SDK for some of those controls. If I was creating a game seriously, I’d 100% personally prefer creating some custom controls to allow some code to run on the browser side, using the Web SDK with JavaScript or TypeScript, to avoid dealing with latency or round-trips.

Using AI for Creativity

I personally don’t find any joy using AI for coding. I always end up frustrated by how it hallucinates and keep proposing, very confidently, things that just don’t exist. I understand some might like it but, in the end, I just like coding.

But for a second brain and helping me with (my lack of) creativity… oh boy, that’s another story. The game idea came from a conversation with Claude AI. Initially, it came up with tons of features and ideas that would require me (and a team of another 100 people) to work on it for several years. I kept asking it to simplify it until I felt it was approachable:

After implementing the game mechanics, I felt the game didn’t make any sense. I mean, why 30 turns exactly? Why 5 planets? Why can’t I go somewhere else to trade? Why do I have to trade at all?… While it is still fiction, the AI wrote a reasonable background story that added some depth to the game. That’s what you will be reading when playing Cosmic Trader.

No spoilers, but there are still a few features and game mechanics, recommended by the AI, that I left for future releases.

While working on this game, I even had a little ant invasion at home. The AI came up with a multiplayer ant colony simulator game idea that I will probably never implement, but who knows!

For the logo, I had some spare credits to use with DreamStudio. I had to retouch it using Affinity Photo, but that’s it:

All in all, and while a team of real humans will always be much better, I reckon AI helped me a lot with this experiment.

Credits and Acknowledgements

I could focus on building the game thanks to using the Kenney game assets. The background music author is Dmitrii Kolesnikov, make sure to check his profile.

Thanks to Claude for being my second brain for this project. And, of course, thanks to Xojo. It wouldn’t be as fun without it.

Conclusion

I enjoyed building this project a lot. I learned things, and I fixed some bugs I found. Did you know you couldn’t add a WebStyle CSS transition without adding other styles? Me neither, but it will be fixed in Xojo 2024r3.

Happy coding!

Ricardo has always been curious about how things work. Growing up surrounded by computers he became interested in web technologies in the dial-up connections era. Xojo has been his secret weapon and language of preference since 2018. When he’s not online, chances are he will be scuba diving … or crocheting amigurumis. Find Ricardo on Twitter @piradoiv.

LX Aer is an AI-powered educational platform developed by student-turned-entrepreneur Leo Medvinsky. In 2020 as the pandemic forced schools to begin using remote learning tools, Leo, a student at the time, saw the gaps and the flaws in the tools he was offered and did what so many Xojo developers do, he decided to build something better. A quick view of LX Aer demonstrates it’s the solution someone who actually uses such a system would create. LX Aer is a modern take on the LMS with a user interface that centers the student and teacher. Currently, LX Aer uses Xojo Web for the backend and the project has 30,000 lines of Xojo code and growing.

10 Questions with Leo Medvinsky of LX Aer

Using Xojo Web for just the backend (web services) is somewhat surprising, tell us about that?

The LX Aer web app is designed to be the digital hub for students, teachers, and soon, administrators. The original web app was written in between 2020 and 2021 and used Xojo Web for some web service work but was mainly used the conventional way for Xojo Web pages. To increase speed and reliability as well as to improve performance, the decision was made that our next product (a standalone attendance tracking system) would be written using Xojo Web as a backend, with a Xojo iOS app and a standard HTML/CSS/JS web app to accompany it. We found that Xojo Web was a great way for us to have business logic on the backend, with great performance and amazing ease of management with Tim Parnell’s Lifeboat. When, in 2022, we were unifying this attendance system into the LX Aer suite, it was decided that the LX Aer suite would be combined into the attendance technology stack, rather than the other way around. This was a large time investment as it involved rewriting and rethinking significant portions of the existing product offering but has brought our software to a much stronger position.

What is your experience with Xojo Web performance for larger scale apps like LX Aer?

Up until recently, when we began self-hosting most things, we had two instances of the LX Aer Backend running on a Debian server in AWS with just 2 GB of RAM and performance was very good. We are actively developing solutions to allow for much greater scalability and distribution of load across not just instances (with Lifeboat), but across several different servers. Our database server runs CubeSQL, and Marco and I have been discussing ways to implement database sharding into CubeSQL to allow for greater scalability. While we are a startup without too many users right now, LX Aer needs to be able to support the thousands of users each school brings, along with all of their data requirements.

Can you tell us why Xojo was chosen as the backend and what other options were ruled out?

My first programming language was Visual Basic .NET. The original software I had was a simple quizzing application designed to help the user study for tests, but it could only run on Windows. This was a significant limitation. I (mostly) loved the syntax of VB compared to other languages I’ve worked with (Python, C++, Swift…), so I decided to try Xojo when I came across it in 2020. It was an amazing feeling to be able to write with the syntax I already knew to and make an iOS app, albeit a limited one, the same day I downloaded Xojo. Having experimented with Xojo, I started working on major projects in it very soon, and when a web development need came up, it was my go-to. Over the years, I have considered many alternatives, and many in the developer community (not Xojo’s) have discouraged me from relying on a proprietary language for such critical software, but I have never had any interest in switching. Every step of the way, Xojo has scaled with our needs.

Mac, Windows or Linux?

I flip-flop constantly between my dev machine, a Mac, and my business machine, a Surface. I tried many Linux distros on several occasions and have never been much of a fan of it as a consumer operating system.

What do you wish more people would ask/talk to you about when it comes to LX Aer?

Security. LX Aer’s tagline is “Security, Reliability, Innovation.” Notice how “security” comes first. While the Aer Intelligence platform offers incredible AI tools built into LX Aer to improve educational outcomes, and this is one of the largest attention grabbers, I would argue that security is our most interesting innovation. Having graduated from high school just over a year ago, I recall when I found my student data leaked online after a massive breach of PupilPath, a learning management system that my school used at the time. I also recall the school I transferred to for my senior year and the software they used, Genesis, which allows a user to view their own password in plaintext after logging on. This, among other things, motivated me to always put security first. In 2023, we launched LX SecurShield, which provides encryption for student data WITHOUT storing keys on the server, while still allowing a user to seamlessly log on from any device, without any of the hassle typically associated with client-side encryption. LX SecurShield is truly unique and is something I wish more people took an interest in.

How would you explain your most recent project to a 5-year-old?

LX Aer is creating an AI chatbot that will help teachers teach and students learn!

What’s next on your “Learn Next” list?

Tough question… maybe French?

What programming moments made you think “Wow, I love my job so much”?

I’ve always been technically minded, but I have seen it as a means to an end, rather than the end itself. I never wrote code for the sake of writing code. I always had a long-term project in mind. LX Aer has been a project of mine for over 4.5 years, and has changed immensely from March 2020, when it first began. Consequently, the programming moments that made me think how much I love my job were when I was able to finally push an innovation over the edge into a working state. When I was developing LX SecurShield, it required significant portions of LX Aer to be rewritten to support the new encryption. After developing a proof of concept, I worked almost nonstop for close to a week to develop it into a usable and stable state, with several all-nighters along the way. At the end of it all, I thought, “wow, I love my job so much, even with all the craziness associated with it!”

What is something that has surprised you about coding in the last 10 5 years?

10 years ago, I was 8 and had no experience writing code (though I was still the go-to tech support for my friends/family). This is a tough question, but if I had to pick it would be the difference in writing code for someone else vs. writing code for your own company. Writing code for someone else is 1,000x more difficult than writing code when you’re your own boss. As for something in the industry itself, I am always surprised by the tendency of web developers to jump around from platform to platform… Angular, React, etc. It just fascinates me.

What is a piece of software more people should know about?

To those in the Xojo community writing a web service, I cannot recommend Tim Parnell’s Lifeboat and Marco Bambini’s CubeSQL enough. To anyone working with cryptography, Björn Eiriksson’s Einhugur plugins are just incredible! 

To people in general, I think it would be great if more people knew about Proton. They’re a Swiss company that develops mail, cloud storage, calendar, VPN, and password storage software, all of which uses client-side encryption and is easy-to-use. They were my inspiration for LX SecurShield. 

What’s something you worked on recently that you are excited about?

It would be somewhat cliché to say that I am excited about AI (Aer Intelligence), but I am. Especially in the EdTech industry, I see so much potential for it to improve educational outcomes and make lives easier for overworked teachers.  With our software having won the Pitchfest award at the prestigious Future of Education Technology Conference this year, I believe, largely because of our AI capabilities, I think it isn’t just the LX Aer team that’s excited about AI in education.

Thank you to Leo Medvinsky for answering our questions. If you want to learn more about LX Aer visit https://www.lxaer.com/. To all those in the Xojo community, as a token of gratitude, Leo is offering a free license until December 31^st, 2024. Email sales@lxaer.com and say that you’re part of the Xojo community! “Although I have not been very active in the Xojo community in terms of engagement,” Leo says, “without the exceptionally great resource that is the Xojo Programming Forum, as well as the software and plugins made by those in the community, such as Tim Parnell, Marco Bambini, Christian Schmitz, Björn Eiriksson, and Sam Rowlands, none of this would have been possible.” See the full LX Aer Xojo community credits at https://www.lxaer.com/credits.

As an example of how you might use ChatGPT in Xojo, I looked for a way to turn statements into questions. So I ask ChatGPT:

Turn “numerical limits of double in xojo” into a sentence.

ChatGPT’s response is:

What are the numerical limits of the double data type in XOJO?

Getting Started

Before you can run the example to test out the ChatGPTConnection class or use it in your own projects, you’ll need to do three things:

1. Create an account at OpenAI.com. This is free.
2. Add a payment method to your account (to use ChatGPT via an API, you need pay – more on that in a bit).
3. Get an API key. You’ll need to have added a payment method before you can get an API key.

One you have done this, open the ChatGPT Example project, open the ChatGPTConnection class and in the Inspector assign your API key to the APIKey constant. Now you can run the example. The example presents a window where you converse with ChatGPT.

Each window is a separate context. Try choosing File > New Conversation. Referring to questions you asked in another window won’t work because in this example each has its own context.

Using the ChatGPT Classes in Your Project

To use the ChatGPTConnection class in your project, copy both the ChatGPTConnection and the ChatGPTException classes to your project.

To use the ChatGPTConnection class, either drag it onto a window, webpage or mobile screen or create an instance in code. Most of the work is done by a single method: ReplyToPrompt. The syntax for ReplyToPrompt is:

ReplyToPrompt(prompt As String, Optional MaintainContext As Boolean = False) As String

In its simplest form, just pass your prompt and the class will return ChatGPT’s response. For example:

Var gpt As New ChatGPTConnection
Var response As String = gpt.ReplyToPrompt("Where were the 1980 Olympic Games held?")
MessageBox(response)

When you want ChatGPT to consider past requests in the same session, pass True for the MaintainContext parameter. This tells the ChatGPTConnection class to keep a history of all your requests and the responses from ChatGPT which it then sends along with your new request each time you call the ReplyToPrompt method. The amount of text you can send is not unlimited so only set MaintainContext to True when you really do need it to consider previous requests and responses. ChatGPTConnection defaults to GPT 3.5 Turbo which has a maximum limit of 16,385 tokens per context. A token is a number of characters. The conversion is approximately 4 characters per token. Think of a token as a typical word. When you set the MaintainContext property to True, the ChatGPTConnection class monitors your usage of tokens. When it gets to 90% of the allowed maximum for the model you are using, it will start trimming the oldest requests and responses from your history (stored in the ContextHistory property) to avoid you going over the limit.

ChatGPTConnection Class Properties

You don’t need to set any of the properties in the ChatGPTConnection class as they all have useful default values. However, if you want to use a model different from the default (gpt-3.5-turbo), you can set the Model property before calling the ReplyToPrompt method. You can find a list of models here. Models have different limits as to how many tokens you can use so when you change the Model property, make sure to also change the MaximumTokensPerContext property to the appropriate value for the model. The Temperature property indicates how creative ChatGPT will be. Another way of saying this is how much it will hallucinate. Its value range is from 0 (don’t hallucinate at all) to 2 (hallucinate all you want). The default is 0.5. Finally, the TimeOut property is how long ReplyToPrompt will wait for a response. The default is 30 seconds.

If anything goes wrong, the ChatGPTConnection class will raise a ChatGPTException so make sure to put your calls to the ReplyToPrompt method in a Try Catch statement:

Try
   Var gpt As New ChatGPTConnection
   Var response As String = gpt.ReplyToPrompt("Where were the 1980 Olympic Games held?")    MessageBox(response)
Catch e As ChatGPTException
  MessageBox("Error: " + e.Message)
End Try

Pricing for ChatGPT is by millions of tokens. Chat GPT 3.5 is relatively inexpensive so if it gives you the results you want, use it. As of this writing, every 1 million words/tokens you send it will only cost 50 cents in US dollars. Every 1 million words/tokens you get back in response will cost $1.50 USD. If you need higher quality responses, consider Chat GPT 4.

Geoff Perlman is the Founder and CEO of Xojo. When he’s not leading the Xojo team he can be found playing drums in Austin, Texas and spending time with his family.

I’m going to show you how to connect to IBM’s Watson services with REST APIs and how to use them with Xojo projects to identify images. This is just one example, of course, of the many ways to utilize Watson and AI in your Xojo apps.

IBM Watson

IBM’s Watson services have been online for several years and are continuously updated. They are paid services but you can create a Lite user to test and evaluate them. The documentation and variety of services are huge and includes (among other things) image identification services.

Identify images with Watson

This service mainly offers two possibilities: identifying the faces in an image (returning the position, the probable gender of the subject and the probable age range) and the classification (i.e. the recognition of possible tags that can be associated with the image).

Watson’s Natural Language Classifier, is a service that “applies cognitive computing techniques to return best matching predefined classes for short text inputs, such as a sentence or phrase”. It is really large (and the definitions are localized in many languages); moreover you can create your own classifiers specific to projects. The Lite user can create only one classifier at a time that is replaceable but not updatable. Another limit is the number and total size of the images that you can use to address your classifier.

Watson’s documentation is complete and easy to use. After creating the service in your account, you can start using them with the terminal or through the web page relative to the selected service.

How to use Watson with Xojo

The services use a REST API so you’ll use Xojo.Net.HTTPSocket as the base class to create the object that will consume these services. To learn more about using Xojo with a REST API, you should read up on that before continuing: Cats Up: Using the HTTPSocket with the Cat REST API, PDF File Generation? There is an API for that and Web Services Video Playlist.

In this example, the base class is called “WatsonAPI”. This class will deal with the communication with the API (sending and initial interpretation of the answer) and will have some common features (such as zipping several images so they can be sent all at once). Moreover, since the interaction with the service is asynchronous, the class will have to manage the serialization of the different requests and take care to return the result to the correct call.

Next, you’ll define a delegate who will have as an argument the answer (positive, negative or error) that will have as its signature a Xojo.Core.Dictionary:

WatsonReplyFunction(reply as Xojo.Core.Dictionary)

Now you’ll define two private properties: id and callback. You’ll use id to distinguish the various call, and callback will be the function to call when there’s a the result.

Private Property id as Text
Private Property callback as WatsonReplyFunction

The constructor (protected because the subclasses will call it) can be of this kind:

Protected Sub Constructor( cb as WatsonAPI.WatsonReplyFunction)
  // Register the callback
  callback=cb
  Super.Constructor
  register Me
End Sub

Register is a private shared function that assigns the identifier to the object and saves the pair id, object in a shared dictionary. There’s also a deRegister function that will delete the object whose identifier is passed.

As noted before, you’ll use a dictionary to represent the API result because it’s the format used for positive or negative API replies. Moreover, we can use the same format for network error replies. The dictionary will contain at least 3 values: success as boolean for a positive or negative reply, status as integer for the reply http status and result as text or dictionary to represent the current reply. Since it’s a dictionary, we can easly add more information as needed. Subclasses or consumer classes can transform this data structure in a more specific way (class, record or whatever).

Now you will implement the major events:

Sub Error(err as RuntimeException) Handles Error
  //All replies will have the same structure
  //In the event of an error, the returned dictionary must be structured as correct one
  Dim d As New Xojo.Core.Dictionary
  d.Value("success")=False
  d.Value("status")=0
  d.Value("result")=err.Reason
  //Delete the object from the dictionary
  deRegister(Me)
  //return the reply
  callback.Invoke(d)
End Sub


Sub PageReceived(URL as Text, HTTPStatus as Integer, Content as Xojo.Core.MemoryBlock>) Handles PageReceived
  #Pragma Unused url

  //evaluate the reply
  Dim d As Xojo.Core.Dictionary
  Dim t As Text
  Try
    t=Xojo.Core.TextEncoding.UTF8.ConvertDataToText(Content)
  Catch
    t=""
  End Try
  If Not t.Empty Then
    Try
      d=Xojo.Data.ParseJSON(t)
    Catch
      d=Nil
    End Try
  End If
  Dim reply As New Xojo.Core.Dictionary
  reply.Value("success")=HTTPStatus=200 And d<>Nil
  reply.Value("status")=HTTPStatus
  If d=Nil Then
    reply.Value("result")=t
  Else
    reply.Value("result")=d
  End If
  //Delete the object from the dictionary
  deRegister(Me)
  //return the reply
  callback.Invoke(reply)
End Sub

Now you can create a subclass to classify the images: WatsonVisualRecognition as subclass of WastonAPI.

For the image classification you can use one or more of the classifiers and/or the default one or even those in beta (currently Food and Explicit). So let’s define the constants related to these classifiers:

Public Const IBMDefault as Text =default
Public Const IBMExplicit as Text = explicit
Public Const IBMFood As Text = food

… and the ones related to the services:

//The current version of the service
Private Const version As Text = 2016-05-20

//The address actually depends on the user settings
Private Const kBaseUrl As Text = https://gateway.watsonplatform.net/visual-recognition/api/v3/

//The key to use the service
Private Const keyVision As Text = •••••••

You can now define a public shared method for analyzing an image on the web:

Public Shared Sub classifyImage(cb As WastonAPI.WatsonReplyFunction, imageUrl As Text, threshold As Single=0.5, paramArray classifiers As Text)
  //The method requires a method to be called to return the results,
  // the address of the image to be analyzed
  // the minimum value to be considered for recognition
  // a list of classifiers to use

  //Let's create the instance linking it to the callback
  Dim w As New WatsonVisualRecognition(cb)

  //threshold is the minimum acceptable value for classification
  //  must be between 0 and 1
  If threshold<0.0 Then threshold=0.0
  If threshold>1.0 Then threshold=1.0

  //For classifiers I can use both the ones provided and mine
  //none means the default one
  Dim useIBM As Boolean
  Dim usePersonal As Boolean
  Dim usedClassifiers() As Text
  For i As Integer=0 To classifiers.Ubound
    Select Case classifiers(i)
    Case IBMDefault
      useIBM=True
    Case IBMExplicit, IBMFood
      useIBM=True
      //These classifiers are in English only 
      w.RequestHeader("Accept-Language")="en"
    Else
      usePersonal=True
    End Select
    If usedClassifiers.IndexOf(classifiers(i))=-11 Then usedClassifiers.Append  classifiers(i)
  Next
  Dim classifierIds As Text=Text.Join(usedClassifiers, ",")

  //Set the kind of the classifiers used
  Dim usedOwners() As Text
  If useIBM Then usedOwners.Append "IBM"
  If usePersonal Then usedOwners.Append "me"
  Dim owners As Text=Text.Join(usedOwners, ",")

  //Create the URL to be called
  Dim url As Text=kBaseUrl+"classify"

  //I create the list of arguments
  Dim args() As Text
  args.Append "api_key="+keyVision
  args.Append "version="+version
  args.Append "url="+imageUrl
  If Not owners.Empty Then args.Append "owners="+owners
  If Not classifierIds.Empty Then args.Append "classifier_ids="+classifierIds
  If threshold>0 Then args.Append "threshold="+threshold.ToText
  Dim parameters As Text=Text.>Join(args, "&")

  w.Send "GET", url+If(parameters.Empty, "", "?"+parameters)
End Sub

Finally, you can request the classification of an image. For example, put a button in a Window and in the Action event put the following code:

WatsonVisualRecognition.classifyImage(WeakAddressOf analyzeResponse, "https://watson-developer-cloud.github.io/doc-tutorial-downloads/visual-recognition/fruitbowl.jpg", .3)

Where analyzeResponse is the method that will read the results translating the Dictionary into something useful like actions for a database, a textual list or a simple text message.

Starting from this simple method, it is possible to create all the others, adding a bit of utility functions to the WatsonAPI base class.

As an example, it is possible to obtain the basic information about the people in this image and display it as an overlay:

Or update a database of images with classifications to then find the images of a specific type:

Create your own classifiers

Starting from this object it is easy to create an application to generate, update, and verify a classifier that is specific to this solution. While you can do it with the provided web interface, you can create a Xojo app that can return the feedback in a much easier and manageable way; and you can add some methods to automate the process of automatically discarding bad images and adding new ones in order to refine your classifier better.

Conclusions

Watson’s API services allow you to add a bit of artificial intelligence to your Xojo projects. The simplicity of the classes required to do this are clear proof of Xojo’s versatility.

It’s important to keep in mind that this service is not instantaneous. This is due in large part to network traffic; generally for sending the data and receiving the answer.

A really interesting option if you develop for MacOS would be to download your classifier in CoreML format and use it offline with MBS Core ML plugin.

Antonio Rinaldi is a professional engineer, Xojo developer for almost twenty years, Xojo evangelist for Italy since 2014, consultant and teacher. He develops extensions for Xojo iOS that you can find in the Xojo Store, and manages XojoItaliaBlog.com. Musician, composer, lover of good food, traveler and constantly curious, he is always looking for new ideas to expand his knowledge.