Primarily, we focus on the issues having the greatest impact on the Xojo community. However, that focus is not absolute. The engineering team at Xojo is using Xojo all-day, every day. They will often catch bugs in the early stages of the development of a new feature or as a side effect of fixing some other bug, long before that release gets to pre-release testers. The number of combinations of interactions between various parts of the Xojo frameworks is effectively infinite and no system can test everything. That’s why pre-release testing by actual users is so important. When you look at the bugs that are reported, 95.6% have only one user reporting them. Though we provide, in fact require, users to search Feedback before submitting a case, users often describe the same bug in varying ways resulting in duplicates. Given both the infinite number of interactions and some number of duplicate reports, the fact that overwhelming majority of reports have just one user is understandable. That we focus on the reports that have multiple users is also understandable. And of course there are exceptions, if it’s obvious that a new bug is going to effect a lot of users, we will get to work on it ASAP. In addition, if an engineer is working in a particular part of the IDE or a particular framework, since they have take the time to refamiliarize themselves with that code, they will often look for bugs that have been reported against it. 

Development tools are special because the type of people who use them are more likely to report bugs and are more likely to want to search the list of open cases, get updates about bugs and more. While it’s important to report bugs, and we can’t stress enough how much we appreciate the time and care you take in doing so, even if the bug is fixed, you still have to wait for the release in which it’s fixed. Even our engineers have to wait for bugs fixes just like you do. Still, when you run into a bug, please search Feedback first to see if it’s been reported, perhaps think of a few ways it could have been reported and do a few searches. If you find a match, add yourself to the case. If you don’t, please file a case. If you haven’t found a workaround, ask on the forum or contact us directly and we see what we can do to help.

There is a lot of automated testing that goes on before we put out a pre-release to Testers. And we will continue to add more because automated testing is a terrific and efficient way to test anything that can be automatically tested. We are always on the lookout for ways to improve efficiency in our processes.

Challenges

The foremost issue that we had to fix was that the original protocol was designed so that it required you to close the IPCSocket to get your command to run. That meant there was no longer a connection on which the IDE could send any kind of response. It also meant that dealing with issues required making multiple connections.

The second was that there were many things that raised dialogs when something went wrong, which meant that an unattended IDE could get “stuck” waiting for user input which would never come. We’d heard that developers would have their systems take a screenshot if an hour had passed with no activity so they could figure out what had gone wrong, but that can be a huge waste of time if you’re waiting on a build.

Solutions

The new protocol (which was beta in 2016r4 and officially released in 2017r1) addresses both of these problems. The new protocol uses JSON for the command format and the IDE can send back all kinds of information including (but not limited to):

* Script syntax errors
* Script runtime errors
* IDE version conflicts
* Missing assets
* Compile errors

All of these things are returned to your calling script so you can take appropriate action. Because this is truly a two-way protocol now, you can also send data back to your script, like if you wanted to return the path of the built app back to your build system.

We’ve also been working to make anything that would normally show a dialog to the user gets sent back to the app communicating with the Xojo IDE. As far was we know, all but one was caught, and a fix for it has been submitted for 2017r2.

If you’d like to see the new IDECommunicator protocol in action, look in the Examples folder next to the IDE: Examples > Advanced > IDE Scripting > IDECommunicator > v2 > IDECommunicator-Tester.xojo_binary_project

For documentation and more information refer to the doc page here: IDE Communicator

Just a note for those of you who have already been using the IDECommunicator project (or something similar) for controlling your builds, when updating to the new protocol make sure you don’t include a QuitIDE command in the script you initially send to the IDE. Doing so will cause the IDE to quit before you get a response which would present as a false positive. I suggest waiting until you do get a response and then issue a separate QuitIDE command at the end.