Xojo Retina/HiDPI: The journey of a thousand pixels…

“Retina” is the name for high resolution screens on Mac and iOS devices while “HiDPI” is the Windows equivalent. For simplicity, I’ll use HiDPI (which really is the universal technical term) for the rest of this blog post. Now that we have HiDPI support in Xojo, if you app doesn’t use any pictures, you can simply open your project, click on Shared under Build Settings and turn on the “Supports Retina/HiDPI” option. That’s all you need to do to have a HiDPI version of your app!

Having said that, if you are creating or using pictures in your project, there may be a few adjustments you’ll need to make to your code. A little over a year ago the process of making sure we had all of the necessary graphics together to build a Retina/HiDPI IDE was added to my to-do list. While 95% of the icons created for the Xojo IDE in 2013 already existed, most of the graphics that made up the IDE itself did not, and the IDE itself needed a bit of an overhaul to get it ready for the big change, both in graphics and in code…

Graphics

Since the Xojo framework didn’t have any HiDPI support yet, everything done last year was done brute force, that is, 2x images were loaded, scaled down by 50% and drawn where they belonged, just to make sure things would mostly line up correctly. While doing this did reveal the amount of work that we were in for, it also hid some issues that we ran into when the HiDPI-enabled frameworks started coming available this spring.

To give you an idea of the size of the task, there are a little over 1000 distinct icons and images used in the IDE today, of which ~370 we did not have a HiDPI version. Now while I like using programs like Photoshop® or Fireworks® for doing work like this, I wanted to transition our graphics to a vector drawing tool so that if/when a higher resolution screen is available, we’ll be able to go back to the originals and start from resolution-independent source-material. That said, we settled on using Bohemian Coding’s Sketch for the majority of the new graphics work. Total time to generate the missing images was about two man-months.

Code

Once the HiDPI-enabled frameworks started coming together, every place where a picture was being drawn needed to be audited (see why in the next section). This work took us from mid-December all the way through mid-March. I won’t tell you that it took two of us three full months of 60 hour weeks to accomplish, but it sure felt like it.

A couple of legacy coding issues came to light which would have to be resolved for everything to work right and I wanted to go over them just in case your projects don’t render correctly right off the bat. Let’s look at what’s changed.

Points vs. Pixels

This is by far the most common issue you’ll probably run into. Before Xojo had HiDPI support, a single element of color on a Picture and a single element of color on a Graphics object were exactly the same thing, pixel for pixel.. If your code looked like this:

Dim p As New Picture(100, 100)
Dim g as Graphics = p.Graphics
Dim widthPic as Integer = p.Width
Dim widthGraphics as Integer = g.Width

both widthPic and widthGraphics would have values of 100.

Now with HiDPI turned on, Picture coordinates and Graphics coordinates can be different from one another if the scale factor of the screen is greater than one. If you were to change the code above to:

Dim p As TrueWindow.BitmapForCaching(100, 100)
Dim g as Graphics = p.Graphics
Dim widthPic as Integer = p.Width
Dim widthGraphics as Integer = g.Width

The value of widthPic will differ depending on whether your application is running on a HiDPI screen or not:

…and this is where the trouble begins because the number of physical pixels has actually doubled (in both directions, by the way) while the width and height of the Graphics object still reflect the original dimensions. The difference between these values is the scale factor of the screen you are drawing to.

Now, if you consider that Picture.Width and Graphics.Width have been interchangeable until now, you can see that this can cause a bit of a problem. When drawing to the Graphics object, you need to be working in Graphics pixels, not Picture pixels:

Dim p As TrueWindow.BitmapForCaching(100, 100)
Dim g as Graphics = p.Graphics
Dim widthPic as Integer = p.Width
Dim widthGraphics as Integer = g.Width
g.ForeColor = &cFF0000
g.DrawOval(0, 0, g.Width, g.Height) // Correct
g.DrawOval(0, 0, p.Width, p.Height) // This will draw too large

To wrap up this already long story, make sure all of your drawing routines are using coordinates relative to the Graphics object.

Immutable Pictures

Another hurdle you’ll undoubtedly encounter is the fact that multiresolution images are not inherently editable. This allows the framework to be more intelligent about loading and unloading images at runtime. This affects Image objects created in the IDE as well as ones created in code using the new Picture Constructor API. The most obvious effect is that you can’t draw directly to these images any more and the Graphics and RGBSurface properties will be Nil. If you need the ability to draw onto a picture via its Graphics property, draw the image onto a Picture created in code and work from that – don’t forget to copy the scale properties!

Nested Drawing

If your application creates pictures, and draws other pictures and clips into it using several levels of nesting, make sure you start from the top level when refactoring your code. You’ll find any compounded scaling errors much earlier this way, but you’ll also run a much smaller risk of creating issues that will only manifest when you go back and fix the top level drawing routines.

Drawing Without Context

For most of you, your drawing code will be directly in the Paint event of a Canvas and the Graphics object passed to the event already has all of the information that you need to succeed. In addition, you can always get a mutable picture that is all set up for the current screen by calling the new Window.BitmapForCaching method. This method is available from any RectControl, Window or ContainerControl subclass using TrueWindow.BitmapForCaching.

If (for some reason) you don’t have access to a window from your code, you can create a HiDPI image from a Graphics context by using the new ScaleX and ScaleY properties to set things up. For consistency, we created an extends method in a module like this:

Function BitmapForCaching(Extends g as Graphics, Width as Integer,  Height as Integer) As Picture
  Dim p as New Picture(Width * g.ScaleX, Height * g.ScaleY)
  // Set the appropriate resolution
  p.HorizontalResolution = 72 * g.ScaleX
  p.VerticalResolution = 72 * g.ScaleY

  // Set the scale factor so drawing to it will be correct
  p.Graphics.ScaleX = g.ScaleX
  p.Graphics.ScaleY = g.ScaleY

  // Very important to remember the mask!
  p.Mask.Graphics.ScaleX = g.ScaleX
  p.Mask.Graphics.ScaleY = g.ScaleY

  // Return the new picture
  Return p
End Function

If neither a Window nor a Graphics context is available, you can always create a multi-representation picture and leave it up to the framework to choose the right one when the image is being drawn to a Window, like this:

Dim pa() as Picture
Dim p1 as new Picture(100, 100)
p1.Graphics.DrawOval(0, 0, 100, 100)
pa.append p1

Dim p2 as New Picture(200, 200)
p2.Graphics.DrawOval(0, 0, 200, 200)
pa.append p2

Dim RetinaPicture as New Picture(100, 100, pa)

You’ll have to draw everything twice, so I suggest caching these images so you don’t need to do this more than once, especially if your drawing code is complicated. You can use a similar technique to load images from disk:

Dim pa() as Picture
Dim f as Folderitem = GetFolderItem("Test.png")
Dim pic as Picture = Picture.Open(f)
pa.append pic

f = GetFolderItem("Test@2x.png")
pic = Picture.Open(f)
pa.append pic

Dim RetinaPicture as New Picture(100, 100, pa)

Just remember that the images passed into this form of the Picture.Constructor must all have exactly the same aspect ratio or the framework will throw an InvalidArgumentException!

Conclusion

We’re looking forward to seeing all of your apps become HiDPI-aware and hearing about your experiences. I hope that our experience and suggestions make your transition just a little bit easier!

Read more: Advanced Retina/HiDPI: BitmapForCatching and ScaleFactorChanged