You’ve probably wondered how we decide what features and bugs fixes will be added in any given Xojo release. What we care about most is what will have the greatest benefit to users and what it will cost to provide that benefit. In business terms this is called a cost-benefit analysis. Something that costs little but provides a lot of benefit is far better than something that will cost a lot but brings little benefit. That’s obvious enough. Of course most cases are in the middle rather than at the extremes.
The Impact Surface is the benefit in our cost-benefit analysis. The more users affected by a bug or feature and the more enabling or disabling the case may be, the greater the impact surface. The greater the impact surface, the more likely the case will be addressed sooner rather than later. Even in the instance where the impact surface is large, some issues can be resolved quickly while others may take a long time. A typo in a dialog box can be fixed in seconds. Supporting an entirely new platform takes longer.
Requests from Users – Feedback is the app we provide to users to make feature requests and report bugs. In Feedback users can specify which 5 cases are most important to them, be they bugs or feature requests. Doing so assigns points to the case with the user’s #1 case getting the most points and #5 getting the least. The User Favorites list in Feedback shows the top 100 cases with the greatest number of points. While this is only one criterion, it’s an extremely important one because it helps us determine the impact surface of any issue.
We sometimes get contacted by users asking why a case that was reported and then verified (or reviewed in the case of a feature request) by our team long ago has not yet been fixed or implemented. This goes back to the impact surface. Cases with a large impact surface will typically be handled before those with a small one. Many (but not all) of the open bug reports in Feedback only impact a small number of users (and many just a single user) so we focus our attention first on those that impact the most. The age of a case is not a criterion for resolving an issue. The impact surface (with the exceptions noted below), is what matters.
It’s important to note that development tools will always have far more open bug reports than most other software. Why is this? Because development tools generally have such a large number of APIs that the total possible interactions can’t be calculated. In a small, very focused app, the total number of interactions are few by comparison making them easier to thoroughly test. Every large development tool, even those from teams that are orders of magnitude larger than Xojo’s team, have thousands and thousands of open bug reports. Reviewing Microsoft’s bug base for cases regarding Visual Studio is a good comparison. In terms of ratios (open vs. closed cases), we are similar but with a team and budget several orders of magnitude smaller. That there are thousands of open cases doesn’t mean Visual Studio and Xojo are not powerful and useful. Most of those cases are almost certainly at the fringes and thus do not affect many people.
Our Ideas – As you can imagine, we think about Xojo all the time and we use it all the time as well. Much of the code we write is in Xojo. While the Xojo IDE is written in Xojo, many of our internal systems are written in Xojo as well. Sometimes we come up with ideas that users have never thought to ask for. For example, at the time we added the ability to create web applications, no user had ever asked for that ability. And yet today a large portion of Xojo users build web applications.
Show-stopping Bugs – Occasionally a user contacts us about a bug that is preventing them from deploying their app to their users. It may not be impacting many other users, but it’s having a major impact on them. We will typically first see if there’s an immediate workaround and then determine if a fix makes sense. I say that because sometimes the workaround is easy and the fix is incredibly difficult. All of these factors have to be considered.
Sometimes a bug is reported by just a single user and yet we can tell the bug will impact a large number of users so we don’t wait to address it.
Bugs that Bug Us – As we develop much of Xojo in Xojo, we run into bugs ourselves and fortunately are in the unique position to fix them at that point. This is one of the benefits to writing much of Xojo in Xojo.
Xojo Pro Plus User Issues – Pro Plus users pay more to be able to get their issues resolved sooner. It’s an extra level of support – for example, perhaps there’s a bug that is normally not one we would prioritize because it doesn’t impact many users or you need help tracking down an issue in your code. Xojo Pro Plus users will be given priority.
Sponsored Development – Occasionally we get requests for features that while certainly worthy of addition to Xojo, don’t have enough impact for us to prioritize them at the moment. The user requesting said feature can sponsor the development of that feature which pays for the opportunity cost to us of doing it now as we will not be working on something that the far more users are requesting. Two examples that immediately leap to mind are support for Security-Enhanced Linux and most recently, the ability to store and load compiled XojoScript.
Shouldn’t Xojo Be Bug-Free? In the ideal world, yes. In reality, the cost of attempting to get the bug count down to or even close to zero goes up exponentially with the size of the project. For example, the software that ran the space shuttle was 420,000 lines of code and supposedly had just one bug in any particular version. As one of the programmer’s said, “If the software isn’t perfect, some of the people we go to meetings with might die.” That level of reliability doesn’t come cheap. NASA projected that the software would cost $20 million but ended up spending $200 million. Given that the source code to Xojo is far larger than that of the space shuttle, the cost of making it bug-free would also be higher.
Looking at the top 100 cases in Feedback is also illuminating. 80% of them are feature requests. Only 8 of the top 50 are bugs, the first of which doesn’t even make the top 10. As a result, we spend a lot of time on new features but that doesn’t mean bug fixes aren’t a priority. In fact, we fix hundreds of bugs a year.
Bugs Fixes – We fix hundreds of bugs every year. The bug fixes counted in each Xojo release are all bugs that existed in the previous released version. For example when Xojo 2020r1 says it includes 161 bug fixes, that number doesn’t include bugs that were introduced in 2020r1.
The Roadmap – We plan out about 18 months in advance for the bigger new features that will appear in Xojo in future releases. We outline these on our Roadmap. It’s important to recognize that our engineering team is quite diverse and as a result we are generally working on many different features at any given time, some that will ship soon and others that may be many months down the road. And of course in every release we fix lots of bugs.
In Conclusion – The Xojo team is a very dedicated and hardworking group of people and we care about the issues you care about. The whole point of Feedback is to guide us in finding a way to prioritize and balance everyones’ needs and desires. Through our 20+ years, we’ve tried multiple ways of managing this process. What this boils down to is we very much appreciate all the assistance of the community and our testers who help us make Xojo the growing success it is, and we’ll continue to use Feedback and the guidelines above to evaluate your bug reports and feature requests. We want you to know if you have reported a bug or need a workaround for an issue, Xojo wants to help you find a solution so reach out to us via Feedback, or email our support team, or on the Forums.