Xojo in 2017: A Review [Part 2: The Framework]

This is the second part of a multi-part review series on the Xojo programming language, environment, framework, add-ons, community, and more.

The first part can be found here: http://www.dev.1701software.com/blog/xojo-in-2017-part1

-----

The truth is you cannot easily make a distinction between Xojo the language from its framework or “standard library” in typical programming language speak. This is because of how Xojo is produced as a commercial product. In most programming languages (especially recent open source ones) the syntax and rules that govern it is distinct from the built-in functionality that the language possesses. Beyond those capabilities is a packaging system or an import mechanism that allows you to draw on additional libraries to complete your desired objective.

In some languages like Swift the standard library is small in scope as most fundamental functionality is already found in the Apple/Next foundation libraries. The cross-platform objectives of Swift are being enabled by duplicating the said Objective-C/Darwin frameworks to other platforms such as Linux. This allows Swift code to more easily be ported from one platform to another. Google Go (or Golang) is the opposite and provides a very extensive standard library capable of powering web servers, performing compression and cryptography for example. A great deal of network code built in Go is ready to be compiled on any supported platform due to the standard library being so robust.

Xojo is like Go except without the benefits of an open source standard library. If you want to build a web server you have no choice but to use the built in TCPSocket and ServerSocket implementation provided in Xojo’s standard library (or framework as they call it). Xojo has no packaging system or central repository of plugins or third party libraries. While the Xojo standard library is quite large it varies based on the platform you intend to use it on. This can make coding difficult because you have an expectation of a method or class being available or working identically on all support platforms but this is often not the case.

More damaging to the process of writing code in Xojo is that the standard library is not the only nuanced aspect of the platform. Core data types that should be iron clad and completely reliable on every supported target can vary wildly in their implementation and support. Feedback item #46943 as of this date is ranked 880th and “Needs Review” despite ample evidence that the new ‘Text’ data type simply does not function on Linux as advertised. For those of you new to Xojo and not aware of what Feedback is we will discuss the pros and cons of this tool in a later review part. You should know that the Xojo community is likely unaware of this bug because if they were it certainly would not rank 879 other bugs to be more important. Without reliable data types the rest of the platform is also unreliable because you cannot predict outcomes of standard type operations.

[ For those of you unable or unwilling to use Xojo Feedback to review case #46943 it demonstrates in a reproducible fashion that the ‘Text’ data type is magnitudes slower on Linux. Specifically using the ‘Text.Characters’ iterator is extremely slow compared to any other target. Fortunately there are other ways to iterate through the contents of a ‘Text’ variable but they certainly are not pretty and basic unit tests should have identified this issue or shown a disparity. ]

If you are not using the new ‘Text’ type on the platform affected then you are unlikely to use caution when using it. You only find yourself in this situation after spending weeks developing your application and then deploying it to the platform to discover it is not running as expected. Oh no!

Now you will tell me “Phillip it’s okay nobody uses the ‘Text’ type yet except for iOS” and you might be right. Except that I do for all my projects because I want to use the new framework so I can write cross-platform code. If I cannot share business logic between my desktop app and my iPad app then the tool has failed at its primary objective. Suddenly we must maintain different business logic code bases because iPad can only accept the ‘Text’ type and Linux cannot use the ‘Text’ type reliably.

Even worse these nuances can vary from one Xojo version to the next. Often with customer consulting projects I am forced to use one release of Xojo for one project and a newer release for another. In a larger project of mine with multiple executables (multiple Xojo projects unfortunately as mentioned in part 1 of my series) must use different Xojo versions for the same ultimate deliverable! Absurd.

There’s two reasons for this: One is old history now and cannot be changed and all we can hope for is Xojo takes my thoughts to heart and reconsiders their strategy for the second reason.

The first reason takes us back to iOS on Xojo. iOS is such a different beast from the rest of their targets requiring IDE enhancements and substantial reworking of many of their classes. Specifically, on iOS few of the core operating system libraries are synchronous in nature so many of the Xojo classes needed to be redesigned to accept this new paradigm. They could have taken the easy route and bolted on overloaded methods and new events to existing classes but Xojo engineers decided that asynchronous operation was the future. Rightfully too as we see languages like Javascript with Node.js, Google Go, and Erlang take off in popularity. However instead of just designing new classes that would either be iOS only or slowly reach all the targets as the “new framework”, they re-designed the entire language.

While the syntax looks familiar every class was touched, name spaced, and even core data types altered. Friendly favorites such as the ‘Variant’ object ceded to the ‘Auto’ type which means “I could be anything including a runtime error that I wasn’t what you wanted when you wanted it.” The new ‘MemoryBlock’ type has countless forum posts asking, “How do I use this thing” and “how do I get Text in and out of a MemoryBlock”. If that weren’t enough it turns out the new data types don’t even work (as addressed above) and aren’t being fully tested in every Xojo release.

Finally, what really will upset you about the new framework is how awfully long it has taken Xojo to implement the classes necessary to utilize it on all platforms. With iOS receiving extremely slow updates to its limited standard library there is little incentive for Xojo to push the new framework when its widely avoided by the community for being cumbersome to use, broken, and/or not available when needed.

This leads me to my second conclusion about why I am often forced to use multiple Xojo versions for any given project. The reality is the IDE only gets better with each release. It gets faster, adds new features, themes, documentation, etc. I want to use the new IDE when it ships and I want to continue supporting the company because I believe the IDE represents how programming should be presented. I absolutely hate having to compile piles of C++ code and switching compiler flags and getting my build environment together. On macOS that could mean screwing around with Homebrew to acquire the Linux-esque packages needed to compile a library or executable. On Linux that means gathering the bits to the compiler toolchain which vary from distribution to distribution. On Windows you have to struggle with Visual Studio project versions and now with Windows 8 & 10 multiple SDK versions that are changing rapidly.

I do love working with Xojo daily. I want to fire up my IDE, write some code, hit compile and have executables that work. Xojo is the closest thing to the perfect environment I have found for all the types of devices in my life and in those of my customers.

Truthfully, I absolutely hate that core data types can be broken in obvious and unbelievable ways. I hate regressions and how Release .1 and .2 are often re-fixing things that were fixed in a previous release. Most of all I hate how the standard library can fail on you entirely in unpredictable ways from one release to the next. You often find yourself in a situation where you are debating the pros and cons of two releases and which is the path of least resistance. Which version will cause the least amount of pain hoping and praying that the next Xojo release fixes it for you. If you file a feedback there is a good chance it might. It will probably break something else though.

What I am trying to say is the quality control at Xojo is not good. I am also saying that if you are patient and understand the complexity of what they are trying to achieve it can be great. However, what is a lengthy rant if not for at least one suggestion:

Open source the standard library.

Xojo does not need to be 100% proprietary. For one it’s absurd in 2017 to be a programming language who safe guards all of your inner workings. You have plenty of intellectual property in your IDE and compiler toolchains. I would subscribe to your product even if it was an IDE for more common languages because I enjoy the experience. Second most of your classes are just rehashes of operating system provided classes that you have written a wrapper around. If not an operating system library then some other open source library like Crypto++. Xojo writes wrappers for other libraries for a large portion of their standard library yet hypocritically denies their users the ability to write wrappers around their framework in the EULA. Huh?

I am advocating and would continue purchasing and supporting Xojo if the standard library was open source. The community could help fix bugs and extend the platform with classes. Xojo could package the “officially” supported version with the IDE but the source and instructions on how to build for all platforms would be available on say Github. If a new release of Xojo breaks the HTMLViewer for example then we could take the version from the last release and compile it for the new IDE.

In fact, we could even mix and match which versions of the library classes we need for a given project. How about exclude classes we would never need to minimize the size of the framework and our executables? The community could do so much to extend the framework beyond its original scope and breathe new life into every release of Xojo instead of playing two steps forward, one step back.

I will cover the Plugin SDK in another part of this series and some of the most popular plugin vendors in the Xojo community. In short: the Xojo standard library suffers extensively from lowest common denominator and an additional investment in third party plugins is essentially required. Xojo wants to move away from the Plugin SDK and to their new LLVM pre-compiled plugins that they have mentioned but never fully scoped out for the rest of us. While I find that intriguing and useful it should not be a binary choice. I want to see the entire standard library turned into one giant Plugin SDK.

-----

For those of you that will tell me I am being overly critical I apologize but I do not accept your premise. I am writing this from the perspective of someone who uses Xojo daily. It is not my job to advertise for Xojo or help them generate more revenue. They market their product as a tool that does cross platform development with a large and useful standard library. I and most consumers using the product will take them for their word and assume the product works as advertised.

My objective in this series is to communicate the pros and cons as a professional Xojo developer making a living with this tool. I want to communicate to the business owner, future client, and forthcoming developer the challenges that they will face. I also hope to build consensus around logical and attainable solutions that would make the product better for everyone. If I did not love using Xojo then I would not take the time to write all of this down so I hope you find my criticisms useful. I also do not want to spend cycles regurgitating content you can find on Xojo’s developer center or marketing materials. I am reviewing the product as it stands in 2017 and where I see it headed in the future. I am not developing a brochure to market it to you or suggest you should not use it.

The truth is that Xojo necessitates this level of scrutiny by staying 100% proprietary. They so far have not wanted users contributing to the core framework or optimizing the product for their own use cases. They want you to use Xojo exactly as they have designed it to be used and unfortunately the right answer probably lies somewhere in the middle. 

Microsoft DeepCoder: Will machines write their own code?

There is a lot of optimism and fear recently about how artificial intelligence will augment the human experience. Recent efforts for machines to write their own software could accelerate innovation on multiple fronts where software development is absent or poorly embraced. 

I am personally not afraid of a machine writing code based on descriptions of a desired program. Frankly the actual typing of the code and conforming to various syntaxes is the least enjoyable part of the job. 

What I enjoy most in software development is identifying how a program might:

•    Solve a problem
•    Positively help the users
•    Negatively impact users or existing processes
•    Function in terms of aesthetics and user experience
•    Be priced and sold in the market
•    Be maintained in the future 

I enjoy the product development and architecture of software projects more than the actual coding itself. After a decade and a half of writing code I have mastery over the computer and can compel it to do whatever I want. The more challenging problem is making sure your vision for the program and the task to be completed is delightful and productive for the human user.

In the future, there will be a great deal more machines operating under rules designed to protect and enhance our lives. I believe that no artificial intelligence will every truly empathize with the human experience. As such while the job of software developer may disappear a new one will appear: software illustrator. 

With broad strokes and very specific details we will describe to the machine the task that needs to be completed. We will illustrate through conversation, charts, graphics, pictures, and video the workflow of any automated task. Whether it be driving you to the store or doing your laundry the machine will understand a set of rules and aspire for your desired outcomes. 

My bet is our craft will be as complex as writing code is today. Software illustration will come with many different architectures, styles, syntaxes, and licenses. Much like lawyers are hired to describe and interpret the law we software illustrators will describe and interpret the world around us and how we want the machine to change it.

Xojo in 2017: A Review [Part 1: The Language]

This is the first part of a multi-part review series on the Xojo programming language, environment, framework, add-ons, community, and more. 

-----

Xojo is without a doubt a derivative of the BASIC family of programming languages. The CEO of Xojo Geoff Perlman has said that if a user associated Xojo with the original BASIC then that would be a “wildly inaccurate association”. Xojo is very cautious to embrace their BASIC beginnings and always asserts that the languages only share a common heritage. That may be true from a certain point of view as Xojo is most comparable to Visual Basic. Visual Basic despite its popularity was always seen as the ugly step child of the Visual Studio family and today is seemingly neglected by Microsoft. 

When REALbasic was ready to launch a new aesthetic for their IDE it was time to disassociate further from BASIC. The company and language were renamed to Xojo and now share only a common syntax. The best way to describe Xojo to someone unfamiliar is a “modern event-oriented cross platform alternative to Visual Basic 6”. For this I think there is little dispute so I leave it to you to decide if Xojo is a BASIC or not.

Like any BASIC language, you have your usual suspects of keywords, data types, modules, etc. You “Dim” to create a variable, you “Redim” to reset an array. You can easily create event handlers in the IDE or use “AddHandler” to alter the event flow. Container controls give you the ability to create complex UI controls that you can reuse. A good portion of the core language constructs work on all platforms in the same way. There are some platform differences especially when you go between target categories like Desktop to Web or Web to Mobile. You will most likely discover these differences the hard way as Xojo makes it super easy to compile for multiple platforms but does not always point out obvious mistakes. You also have advanced functions like delegates, cooperative threads, and declares to optimize your code paths and tap into the available underlying operating system functions. 

In fact, one of the best strengths of Xojo is due to its cross-platform nature you find yourself running on all different types of devices. Xojo while not always providing all the standard functionality for each device lets you tap into the host operating system via declares. Unfortunately, when a declare is not enough or the functions you require are not provided by the operating system there is little you can do to expand Xojo’s core functionality. 

One option is to develop plugins for Xojo using C++ (and possibly others undocumented or not supported by Xojo). Plugin development is not immediately intuitive I think in part because Xojo would prefer you to write native Xojo code then augment their standard library. What documentation there is on developing plugins is difficult to utilize and you also should develop equivalent functionality for all potential targets (Windows, macOS, and Linux) meaning three code bases for one plugin. Plugins cannot be written in Xojo now although they have promised the ability in the future. When that occurs, they have indicated that both plugin mechanisms will co-exist for some time.

I will write more about the framework and standard library of Xojo as well as the plugins and add-ons available in a later part of this series. For now, I want to touch on some things that they need to add to Xojo sooner than later. 

Generics

While a contentious item for many software developers in various languages there is immense benefit in adding them to Xojo. One example of how generics could make programming in Xojo easier than ever is when using Dictionaries. Unlike an Array that has a built-in type such as:

    Dim stringArray() As String

Dictionaries have no equivalent:

    Dim myDictionary As New Dictionary

The  compiler can enforce that all elements placed into stringArray are String objects (or a derivative). This happens at compile time offsetting potential errors if you were to attempt to add something else to it later. You also can catch errors at the compilation stage when reading out of the Array. 

The Dictionary however does not have this protection and can hold any type of object in its collection. When accessing a Dictionary there is a potential run time exception when I assume every item in the Dictionary is a String when there is a possibility it is not. I also must cast each item as a String when I read it back out. 

Look at the following:

    Dim myDictionary<String> As New Dictionary

    or

    Dim myDictionary As New Dictionary<String> 

Either of these methods would provide an immediate benefit to the developer as the compiler could catch obvious exceptions to this. On the backside when I go to read items out:

    Dim stringValue = myDictionary<String>.Value(“stringValue”)

Right now I have to do:

    Dim stringValue = myDictionary.Value(“stringValue”).StringValue

That raises a nasty exception at run time if the object is not a String. This could have been prevented by the compiler when developing if we sprinkled some sugar in there called generics. I would be perfectly comfortable with generics making a very limited entry into the language and standard library and expanding on it further where it made sense. They should never be required but when used provide more compile time safety.

Threads

Xojo uses a cooperative threading model and thus your application can only utilize one CPU core at a time. To potentially get better performance from long running or mostly independent functions you are recommended by Xojo to build helper apps. Essentially window-less or console-based applications that are executed by Xojo using the Shell class.

I recognize the cognitive challenges of pre-emptive threading and how that runs afoul of the easy to use Xojo language as it stands today. However, if you are to force us to use background applications to perform critical tasks then the path to doing so must be made considerably easier. 

Right now, you must either pass the necessary arguments to an entirely separate process via command line arguments or IPC sockets. To develop said additional process you need to develop a second Xojo project in parallel with your first one and synchronize the necessary shared code. As they are two unique projects with two unique builds you must construct a folder hierarchy to manage how the Windows version of your command line helper ends up in the Resources folder of your Windows GUI app. Your Windows GUI app now should verify each time it starts that it has access to your Windows command line helper. You must perform these steps for each platform target and for each console helper application and it quickly gets out of control. Working in a multi-programmer environment can exacerbate these problems even further.

Fully utilizing your CPU in Xojo while maintaining a responsive (not locked up or wheel spinning) application is extremely difficult. Far more difficult than it should be and it is a real stain on Xojo when competing with newer tools with complex concurrency and parallelism. Clearly not an apples to apples comparison but Golang is cross platform language that builds a single binary with all libraries bundled in and can use all of your CPU cores with ease. I don’t expect Xojo to completely rewrite itself to accommodate these ideas as they require a different level of thinking and would break existing code. 

However, Xojo could allow for sub-projects or background functions that it automatically prepares and builds the necessary console helper apps as needed. I should never have to think about how the primary application talks to the console application. Like XojoScript it could share in a very limited ability to exchange parameters to offset the fact that all parameters would be sent via command line arguments. 

For example, if I have a very long running function that produces a very complex report and I want to run it in the background. I should be able to say, “Add New Background Function” and I name it “BuildReport(parameter1, parameter2, …)”. It can only accept basic data types such as String, Integer, etc. just like a command line argument would. I develop my function with the parameter names like any other function in the Xojo IDE. I then call my function from anywhere else in my program.

Behind the scenes the Xojo IDE created a console helper application, parsed command line arguments for every time I call the “BuildReport(parameter1, parameter2, …)” function and handles it accordingly. It can then return a simple datatype result that can be parsed back by Xojo using their Shell class and then ultimately raises a Done event with the result. It should automatically prepare my background function console application for each platform I build for so I do not have to think about paths, command line arguments, Shell, etc. It has all the tools to do this already and if it did would mark a new chapter in accelerating your applications.

Custom Event Handlers

Today building custom event handlers for an object that has not been sub-classed is a real waste of my time. The consensus is generally that you should create a sub-class any time functionality changes and thus you can use the hander IDE helpers to handle class events. 

For me the ability to share code between projects in Xojo is very limited. One limitation is external modules cannot have classes embedded in them. This means you cannot maintain a strict hierarchy of objects in modules and share that code between projects very easily. Using external items in the IDE allows you to update all your projects with one set of modules. The alternative is copying your latest module over to your accessory projects each time which is prone to error.

This means I do not like to create tons of sub-classes because it is more items I must copy or make external and then import as external. I find myself often just not creating sub-classes and using an instance of the more generic base class. I then must use “AddHandler” to catch the events of that class and redirect it to a given method. This is terribly painful when you do it often because each time you must add methods with the same signature as the event handler and then use “AddHandler” to bind it. Over and over it gets tedious fast. 

Today:

    Dim myObject As New Object
    AddHandler myObject.myEvent, addressOf handler_myObject_myEvent

    Function handler_myObject_myEvent (sender As Object)
        MsgBox("The event was raised.")
    End function

This is very tedious with any complex objects like HTTPSocket. I don't want a subclass for every single time I want to use a socket.

I would like:

    Dim myObject As New Object
    myObject.myEvent = Function(
        MsgBox("The event was raised.")
    )
    
Final Thoughts

I hope some of these language ideas gave you food for thought and the Xojo team some ideas on how to compete in this multi-core world. 

The next part of this series will take an in-depth review of the various Xojo frameworks available to you when developing on its many platform targets. We will look at iOS, the old framework, the new framework, awesome features, caveats and where it should all be headed. Stay tuned.

-----

This series is going to be my assessments and opinions on Xojo the language and framework after several years of using it on projects of all kinds. I personally enjoy using Xojo but it is not without its shortcomings. There are projects where it is not a good fit or where developer experience is going to be required to get the desired result.

Choosing a platform for your software is an important decision but not a deal breaker. Every platform and development environment has its strengths and weaknesses and Xojo is no different. Success starts with a good design, a strong business model and hiring those who are passionate about the tools and business to produce the best possible product.

I’ll demonstrate in this series different areas where Xojo can improve but also areas where it excels. I look forward to your comments and feedback while diving deep into Xojo.