Sunday 19 July 2009

Adapting Swing

In a post of mine a while ago I mentioned an idea I'd had about making the separation of the controller and view classes in swing a bit less blurry while also simplifying the swing component classes. After that post I kind of forgot about it until today when it just popped into my head again so I thought I'd try to expand on some of my ideas.

What we have now

Swing currently follows a modified model-view-controller pattern to separate it's data from it's presentation. The reason it's modified is that the controller and view are basically strongly coupled, I mean strongly coupled in the sense that the particular instance of the view doesn't matter but generally the controllable (no pun intended) aspects of the view are defined by the controller.

This is good because the user only needs to use methods on the controller to change the look of the component, but is bad because it limits the flexibility of the view to only those properties that are already defined (and even increases the complexity of the view by making it support properties that may not make sense). One of the big things that this means is that a general developer will never use the view classes at all, basically reducing the view class to an implementation detail for the controller which just seems like a bit of a waste to me.

In order to get around the problem of not being able to add extra properties to the view swing uses the clientProperty approach which allows you to store a name-value pair along with a component instance that can be monitored by the view. The problem with this is that you don't know what is supported, it's not type safe and it's very very dirty.

To get around the required complexity level imposed on the view swing basically says that properties on the controller may affect the view but there are no guarantees. So again the controller is kind of not type safe, in the sense that calling a method may not do what you expect.

If you don't know what I'm talking about here's a couple of examples:

Say you have a component called ImagePanel, it's sole purpose is to show an image, ImagePanel extends JComponent and as such exposes methods like setFont. Now what exactly does setFont mean in the context of a component that shows no text? Beats me!

OK from the other side of it say you make or use a look and feel that is based on painters (like Nimbus) the only way to expose these painters to the user is via clientProperties which generally aren't particularly well documented, aren't checked by the compiler and won't show up in your IDE's auto complete lists which means they aren't very discoverable.

Where can we go from here

Well for a start why not move the view related properties to the view, that way you don't need to worry about supporting all those non-needed properties and you can even define properties specific to a particular view implementation if done right.

Personally to accomplish this I would use an adapter pattern to allow the view to say what it supports in a type safe way. For example here's how you might change the font size of a label:

JLabel label = new JLabel("Adaptive");
TextView view = label.getView(TextView.class);
if (view != null) {
view.setFontSize(12.0);
}

Now that may look a little complicated for something that is fairly simple at the moment. When this approach comes into it's own is when your look and feel decides to provide more functionality. Have a look at these use cases:

// Set the background painter if the look and feel  supports it
// otherwise try to set the background colour
JPanel panel = new JPanel();
BackgroundPainterView bgpView =
panel.getView(BackgroundPainterView.class);
if (bgpView != null) {
bgpView.setPainter(new MyPainter());
} else {
BackgroundColorView bgcView =
panel.getView(BackgroundColorView.class);
if (bgcView != null) {
bgcView.setColor(Color.RED);
}
}

// Use specific features of a look and feel
JButton but = new JButton("Large button");
NimbusScaledView nsView = but.getView(NimbusScaledView.class);
if (nsView != null) {
// set the scale for the component
nsView.setScale(NimbusScaledView.LARGE);
}

JTabbedPane tabs = createTabPaneAndAddSomeTabs();
SubstanceTabView stView = tabs.getView(SubstanceTabView.class);
if (stView != null) {
stView.setTabPreviewsActive(true);
}

As you can imagine there are a lot of examples that I could list, from menu bar searching to animation settings and because the separate APIs are grouped into related interfaces more details can be exposed, like with the text view example, to help the user accomplish what they want.

What can I say

It’s worth mentioning that I haven’t actually tried implementing any of this functionality yet so I can’t say for certain if it will work (or feel right) at all. having said that I think it would be an interesting experiment to see if it makes life easier for both the user and developer of the component.

Tuesday 26 May 2009

Where does the time go

Okay, so a friend of mine started a new blog recently which got me thinking "I hadn't blogged for a while".

I checked back here and was surprised to find that it was all the way back in February when I last posted something, which isn't good.

The usual excuse of 'I've been very busy' applies and mostly on things that aren't directly UI related but I have been working on one of the most interesting projects I've ever worked on; and I get paid to do it :)

Anyway, I hope that I will be posting thoughts and snippets more often than I have been, but then that's what I thought I'd be doing when I started this blog :/

Tuesday 3 February 2009

Swing 2 - a feature list

There is a very interesting discussion going on over on Jonathan Giles blog to do with features would be nice for Swing 2. He seems intent on keeping the features at a reasonable level, which I think is commendable; for example language changes like properties are left off the table.

I thought that I'd follow up with a few of my own thoughts:
  1. Painters would be awesome, being able to plug in your own painting code for defined parts of a component without doing dirty LnF hacks would save lots of my time. They do however have their problems, namely sizing. The painting logic of a component is intrisicly linked to the size of that component, the space needed for the component to paint its self should also be part of the painter. Adding an API like the fitInto API in the sclabaleicon project to painters would allow this to happen.
  2. While I'm at it adding this API to component its self would also remove the two confising minimum and maximum size APIs
  3. Now I don't know if this one is possible or not but some way to make UIDefaults type safe would be great, maybe the decorator pattern or something like that. I have no idea if that would work or not but it could facilitate making more of a seperaton between the controller and the ui, I mean text-icon gap should really be in the ui, come to thing of it so should font, foreground/background colour, text alignment, cell spacing, and most of the other properties found in the swing components. Imagine being able to do something like component.getUI(TextUI.class) to get text related ui properties like antialiasing or decoration or font, or component.getUI(NimbusUI.class) to get type-safe properties for the numbus look and feel.
Anyway, that's just a few ideas that would be nice to have, let's hope that sun get back to Jonathan soon about what they think.

Friday 28 November 2008

That shouldn't be too hard (part 2)

After the last post, I've been working on making use of the new dominant colour API to create a flashy Icon decoration demo that looks a bit like the Windows 7 task bar.

This is what I've come up with:

With a corresponding demo (requires Java 1.6 to run).

Be warned, you will spend a large amount of time just playing around with it and may be heard muttering words like 'ooo, shiny'.

I'm sure that I will keep tweaking the demo; adding animation and fading for enter/exit events and will work on improving the performance and style when I get some more free time but I think that's about it for now.

All I need to do now is think up an excuse to use it ;)

Welcome

Hello, World!

I've started this blog to put out there some of my thoughts, discoveries and bits of code that I've thought could be useful to other people.

Most of what interests me is UI related but I may talk of other things once in a while.

Anyway, let's get to it :)

[update]
For some reason this post doesn't appear first, oh well I'm sure you get the gist

That shouldn't be too hard

The other day I was reading about the new Windows 7 task bar, specifically the new Colour Hot-track that is available on mouse over. For those that don't know what this is here is a description straight from the horses mouth:

Color Hot-track

Color hot-track is a small touch that typifies the new taskbar’s personality. When a person moves her mouse over a running program on the taskbar, she will be pleasantly surprised to find that a light source tracks her mouse and the color of the light is actually based on the icon itself. We calculate the most dominant RGB of the icon and dynamically paint the button with this color. Color hot-track provides a delight factor, it offers feedback that a program is running and it showcases a program’s icon. We’ve always believed that programs light up the Windows platform and now, we’re returning the favor.

Windows 7 Color Hot-track

Fig 9. Color Hot-track: moving the mouse across a running window reveals a dynamically colored light effect

Now I may be wrong but I thought this looked pretty cool and at first glance thought that it wouldn't be too hard to do.

The first step would be to work out how to find the dominant colour of an icon. There are many approaches that I went through to accomplish this, a lot of them were very complex and not particularly good until finally it occurred to me that GIF compression already does a lot of the work for you; it takes a full colour image and reduces the colour set down to the 256 colours that best represent the content. Tracing through the GIF ImageIO writer code I discovered the com.sun.imageio.plugins.common.PaletteBuilder class that does the colour reduction following an "octree quantization method". With a small amount of modification this class does everything needed to reduce the image colours down to a reasonable number, as a bonus it also counts the pixels that contribute to each colour as it goes.

So now I have this reduced set of colours, say 6 colours that make up the image, all I need to do is return the colour with the most pixels that contributed to it, right? While this may be technically correct there are also a few other factors that our eyes seem to take into account, like the vividness of the colour or how bright it is.

Take the above image as an example, if you were to ask anyone what the dominant colour of the image is they would probably say blue, when in fact there is more white there than blue. To cater for this case I needed to take the actual colour into account and weigh the pixel count against factors like saturation and brightness to give a more accurate representation of what a human would say.

So from an initial idea that it would be fairly easy to find the dominant colour of an image I ended up with the following process:
  1. Reduce the image down to 6 colours using octree quantization
  2. Go through each of the 6 colours finding the colour that gets the highest score when weighed using the following algorithm:
weight = numPixelsOfColour +
(numPixelsOfColour * saturation * 1.1) +
(numPixelsOfColour * brightness * 0.8)

Where saturation and brightness are taken from the HSB representation of the colour.

I'll leave you with a simple demo so that you can see what the dominant colour of your own images are. You can get the source from the subversion repository of the scalableicons project.

In the next post I'll try to implement the glowing effect using this dominant colour to try and imitate the hot-track effect.