Form and Function

US Trade Deficit Visualized

2008-12-22T13:29:00.000-08:00

see post here: http://www.twgonzalez.com/blog/?p=78

360 Flex Data Visualization and Dashboard Preso

2008-10-13T18:33:00.000-07:00

Adobe just released my Data Visualization and Dashboard Presentation from 360 Flex in San Jose. You can see it here on my new blog:

http://www.twgonzalez.com/blog/?p=59

My Blog Has Moved

2008-09-19T19:03:00.000-07:00

Well all, the time has come to move over to a blog tool that provides a little more customization, aka: Word Press. I am sure these are not the words Google/Blogger want to hear, but alas it is so. I have been very happy with my Blogger experience, but I need more control over my templates and the domain in general to provide a more unique blogging experience for myself and the few of you out there who actually read this stuff.

So without further ado: www.twgonzalez.com/blog/

I will see you there.

- Tom

p.s. I have moved all the posts and comments from this blog over to the new one.

iTunes Genius, Tricky like a Fox

2008-09-19T09:58:00.000-07:00

Well, thus far the iTunes "Genius" has quickly allowed me to fill some gaps in my music collection by finding some obscure tracks that I did not know existed without me having to lift a finger. It was a win-win for me and Apple, I got to listen to some new cool music, and Apple got to take even more of my $$$.

But alas, the Genius ain't so smart. On three separate occasions I have now purchased tracks that the Genius said I was missing, only to realize I actually owned the song. It is a little hard to keep track of the thousands of songs I own in my head, and I have mistakenly trusted the Genius when it says I am missing those tracks.

The biggest benefit I see to Genius is that it is a time-saver and reduces the friction for me to find music that interests me and at the same time reduces the friction between my bank account and Apples ;). But, if I have to go validate each Genius recommendation by doing a manual search (on the same song title) each time, it completely undermines the benefit and REALLY makes it for an annoying experience.

Now I am stuck in the unenviable position of having to go figure out where I purchased music I already own and then struggle with the abysmal "refund" UX in iTunes to get my money back. So far, I have purchased about $30 of music via iTunes Genius, of which about $10 was music I already owned. The cost for me to get my $10 back will easily be $100 of my time. So not only did Apple abscond with my $10, they make the friction to get it back very high.

Way to go Apple, excellent customer experience.

360 Flex Feedback - Thanks Community

2008-09-19T07:20:00.000-07:00

I just got feedback from my 360 Flex session, and I must say it was very pleased to see how many people found the session valuable. It is always very hard when preparing to present technical information to guess what people already know and what they want to learn.

I spent quite a bit of time polishing code and generating some new stuff that I could share with the community, and the reality is that if this were a billable engagement (the work required to prepare for my preso) it would have been a good size project. So I was very gratified to see such positive feedback from the community.

I have not done a lot of public speaking, and as such it is something I am trying to learn how to enjoy more and become more effective at. One of the real challenges I find is how to be engaging and dynamic when discussing detailed technical information. I tend to be more about "here are the facts" and less about "here I am to entertain you." I think there is a happy medium somewhere in the middle that a speaker finds over time, this is something I am striving for.

I look forward to MAX, and even more so the next 360 Flex to have an opportunity to create even more engaging and valuable presentations that I can share with the community.

A Few Processing Sketches

2008-09-11T17:38:00.000-07:00

Here are a couple of sketches I was playing with today that were derivatives of the Fireworks subsystem (okay lets be honest, the subsystem is less than 100 lines of code)

Which Data Visualization works Best?

2008-09-10T08:26:00.001-07:00

1) Heat Index Grid (click for full view)

2) Micro Histogram (click for interactive version)

3) Punch Card Grid

Recently I was asked by Universal Mind if I would be interested in consulting for them on the User Experience and Data Visualization aspects of their ground-breaking geo-spatial product SpatialKey. I felt it was a great compliment and privilege to have such a well recognized and respected company in the RIA world ask for my input, and I was excited by the prospect of working with such people as Mike Connor (VP Business Development), Tom Link (CTO) and Doug McCune (Flex Rockstar) and the many other talented people at UM.

Whiile many of you in the Flex community probably have already heard about SpatialKey, for those of you who have not, definitely go check out their technology preview. Basically, SpatialKey represents some really innovative work on showing high volume data sets as they relate to geo-coded data with visual interpolation techniques that far eclipse the standard pin-based metaphor found on most geo-spatial visualization tools.

In that vein one of the tactical areas I was asked to look at is the effectiveness of certain types of visualizations for specific types of analysis. One of those areas was trying to plot crime incident data data by hour and day of week for a given 7 day period. From this discussion we came up with three alternative ways to visualize this data... one existed currrently within the system, one I created based on the visualization problem as i saw it, and one was found by surfing the web. One of the challenges in doing Data Viz work, is the more work you do in the field the more unique visualizations you create the stronger your preference for certain visual patterns becomes, thus creating a bias in what seems most effective for you.

While I believe I have scientific reasons to explain why I think one of the visualizations above is more effective than others, it comes down to what casual users find most intuitive. To that end, I would be greatly appreciative for any readers to vote on the visualization they feel is most compelling to solve the analysis problem of spotting trends and subtle differences in numeric counts (arrests) as plotted hour-by-hour for a seven day period. While we are interviewing real users in the SpatialKey target market, getting feedback from the Flex community is just as valuable. Just use the comments section to vote for 1 (heat index grid), 2 (micro histogram), or 3 (punch card). (Pics 1 & 2 show the same data, Pic 3 is a different data set, which I realize make the comparison a little harder, but hopefully still valid.)

Processing - A Fireworks Show

2008-08-29T19:37:00.000-07:00

This is my first attempt at creating something in Processing which I started to work with a few days ago. Originally my goal was to create a fireworks show that resembled what I saw this 4th of July on the Mall in Washington DC, with my wife, two boys, and my sisters family. I had fond memories of that trip and was hoping to re-create it in some artwork. I am not sure this qualifies as artwork, but when my three year old came into my office, saw this running and instantly exclaimed "Fireworks Daddy! Auntie Kim!" I knew I had reached a modicum of success.

With Processing you can very rapidly iterate and experiment with sketches, this version is about my 5th iteration, after experimenting with some Handel (Classical) I settled on Radiohead (my favorite band.) Each iteration had its own unique appeal, and some things happened quite by accident. For instance the circles that form and shoot off were a result of tweaking one number by .01. I continue to be impressed with the power of processing, and I will probably see if something similar can be done in Flash.

I am also using this work to serve as partial inspiration for what I am doing within Degrafa and with the Flash Player.

You will need the java runtime installed and the appropriate security permissions to view this, also please be patient when it launches as it is loading up a 4mb song. Click here to view.

Chart Slicer Component Released

2008-08-27T15:31:00.000-07:00

Today I am releasing a chart slicer component that behaves like the one seen on Google Finance.

I had several requests for the source code for my Google Finance sample, but regretfully the code is embarrassingly sloppy, and I am not sure when I will be getting back to cleaning it up. One of the components within that example is this chart slicer control, which I have now created a simple example for, with source code included.

This control can target any Flex cartesian chart, although I have only tested it with a few, so I am sure people will find edge cases (or perhaps not so edge cases) that break the component. It is easy enough to change the display in the control from an area chart to any of the MicroCharts included with a little customization.

This component has a liberal use of Degrafa to make all the magic work, from custom skins on the sliders, to the microcharts used to represent the data, to the tick marks within the slider track the reference the positioning of the items in the target chart. You will also notice some extra logic that has the sliders snap into alignment with the actual item renderers of the target chart. With larger data sets this is less noticeable, but with small ones it is very effective.

Many thanks to Doug McCune for his HSlider control, and the efforts put forth by Brandon Meutzner.

You can see the example and download the code here. As usual all code is released under an MIT license.

Processing is Amazing

2008-08-23T12:59:00.000-07:00

I recently discovered Processing, and I am still in awe of what can be created with it. For those of you not familiar with Processing, it is a c syntax (pretty much java) language created at MIT's multi-media center designed to be used by artists and non-programmers to create visual/animated/physical works of art. The primer written by two of Processing's creators walks a person with no prior programming experience through learning to program.

As someone who has been programming since I was 10 years old and close to 30 years now, I found their approach so simple, powerful, and very approachable. Probably even more valuable to me was the plethora of examples that come with the simple IDE. Full particle and physics systems encompassed in less than 100 lines of code! OpenGL support, texture, uvt mapping, typography, and the list goes on.

I see some of what has been accomplished with Processing as the inspiration of what I think will be possible with Degrafa, and I look forward to seeing how far I can push the boundaries of what is possible in Flex/Flash by adopting similar constructs within Degrafa. The interesting twist Degrafa will bring is that we may be able to step away from procedural abstracts and work directly with declarative ones. Degrafa repeaters is one example of how this might manifest.

The animation at the top of this entry was created by Glenn Marshall, who just started working with Processing, but obviously has an incredible amount of artistic talent. What I find just mind blowing is that the animation he created is all done procedurally using sin/cos, looping, recursion and math. There was no use of After Effects, 3DS, or any other tool that translates analog inputs (hand drawing) into something digital, this is purely abstract programming - which just adds a sublime level of beauty to the finished work.

Note: Watch the video in its entirety it gets better, and better, and better ....

360 Flex Data Visualization Code Examples

2008-08-16T11:08:00.000-07:00

This is a pretty short post that I wanted to get out to the community. Below are the links and source code (view source) for the presentation I gave today at 360 Flex in San Jose.

GoogleFinance Example: (view source coming soon, still needs some refactoring and clean up.)

http://www.brightpointinc.com/flexdemos/googlefinance.html

DataSet Example:

http://www.brightpointinc.com/flexdemos/360flex/DataSetExample.html

Interactivity Example: http://www.brightpointinc.com/flexdemos/360flex/InteractivityExample.html

MicroChart Example:

http://www.brightpointinc.com/flexdemos/360flex/MicroChartsSample.html

Gauge Component - New Features - v04

2008-07-28T20:04:00.001-07:00

First a thanks to everyone who has commented and used the previous versions of the gauge component seen here.

A few weeks ago a publicly traded company (that has asked to remain anonymous) hired me to make some improvements to the gauge to meet some of their specific needs. Some of the improvements are things that other people had asked for, so I offered to give them a discounted rate if we could keep the improvements under the original open source license. It was more important to me that I be able to share the enhancements with the community than the additional revenue I would have generated to lock up the intellectual property.

So here is a list of the latest enhancements.

1. Bounce Effect can now be set on/off by a boolean useBounceEffect. (code changes in control.)

2. Tick Mark Alignment Fixed - Tick marks for the gauge skin now use a best fit algorithm to align exactly with the min/max angles (before they sometimes would not cover the full range. (code changes in skin)

3. A X-Offset style has been provided for the value label placement (code changes in skin and control)

4. Alert Levels: A popular request was the addition of alert levels, something that I had in the original gauge I built. The gauge now supports contiguous alert levels by passing it an array of alertValues, alertColors, and alertAlphas. The control expects that the alertColors and alertAlphas will have one element less than the alertValues, since the alertValues contains the additional min or max boundary element, and the colors/alphas apply to the range within the boundary described by the alertValue array. (code changes in skin and control)

5. Log AND Linear scale: This was by far the most complex enhancement. The gauge now supports movement through a log scale as well as the original linear scale, and since my math skills seem to be stuck somewhere between 8th grade trig and 9th grade algebra it took me a while to figure out the algorithm to support not only programatic value assignments, but also when a user clicks along the and sets the value directly - yes now I remember that, Math.pow(10,value) is the inverse of Math.log(value)/Math.LN10 :) This log scale also gets applied with the alert ranges, and the skin leverages the now public caclulateAngleFromValue() function exposed by the control itself - which takes into account the min/max value range, the min/max angle range, AND the linear or log scale. (code changes in control)

I have tried my best to run it through a bunch of tests and ferret out any bugs, but if anyone finds something I missed please post and comment, and I will try my best to fix or point you in the right direction. Once again this code is posted under the every friendly MIT license.

Check out the sample here, with full source code.

Note: This gauge uses the latest version of Degrafa Beta 3, please reference the included .swc - as older versions of degrafa will cause conflicts.

Apple Numbers - More than a spreadsheet

2008-07-23T17:36:00.000-07:00

For any designers or UX people out there, I don't know about you, but I have yet to find a wire-framing tool/software that beats a whiteboard or sketch-pad, but I think I have found something that comes pretty close.

Making wireframes is an essential part of the design process (at least for me) that allows me to quickly brainstorm layouts and different configurations for an overall application or dashboard UI. Most illustration tools that I am familar with (like Illustrator or Flash Pro) are just too cumbersome and have way too many settings/details that are completely irrelevant to the wireframing process. Tools like Visio (which I just can't stand) and SmartDraw come a little closer, but they have all sorts of awkward UI issues that at best, slow down the creative process and in some cases completely hinder it due to the way they expect you to create content.

Well, after many years, I think I have found a very good tool to make my wire frames in, and ironically the software was never explicitly designed to make drawings. What I am referring to is Apple's Numbers - the supposed Excel replacement. Well, as a replacement for Excel it doesn't even come close - for doing any type of semi-sophisticated numeric modeling it just doesn't cut it.

But, as a very easy to use, quick and easy, creative wireframe tool it is superb. It has all the basic shapes I could ask for, sans all the detailed property controls that make things tedious in most other tools. The inspector panel allows me to create sophisticated gradients, fills, strokes and best of all, it has auto-align/snapping capabilities built right into its WYSIWYG interface. Most of my wire-frames are some form of nested rectangles, and this tool just lends itself to the task beautifully. Because of the way they designed numbers, it is a layout tool first, and and a spreadsheet second. So you don't even need to have a worksheet at all, you can just have a blank canvas and start dropping images, shapes, text, etc. Add the fact that you have an automatic export to PDF and it makes it ideal for delivering to clients and end-users.

For me it has been very liberating, and one of the first times I felt a software tool actually accelerated my productivity for doing this type of work, versus just being a tedious step I needed to do to translate my whiteboard/paper sketches.

Anyone running on OS X and doing any type of wire-framing work should give it a try, I would be curious what other designers think.

I will be speaking at MAX 2008

2008-07-11T10:22:00.000-07:00

Well, it looks like it is official. I will be speaking at MAX 2008 this year on Data Visualization with Flex. I am still working out the final details of the presentation. When I spoke on the same topic in 2006, I did a detailed walkthrough of building a dashboard. This year, I think I will change it up a bit and more do some highlights of what is possible with little code tidbits and how-to's. I think this might appeal to a wider audience (and potentially shorter attention spans since my session is going to be at 3:30 on Wednesday November 19th, the last day.)

I would love to get some feedback from the community as to what level to target the session. Should I stick with beginner level material (i.e. here are all the available components and what they do, or go to a more intermediate advanced level?) Any comments very welcomed, as I still have a couple of weeks to flush out the preso.

Flash Player Bug - Stroking a Corner

2008-06-20T17:52:00.000-07:00

It appears there is a known bug in the Flash Player that creates rendering artifacts and uneven results when trying to apply strokes to a corner radius. Let me outline the situation by having you look at the attached image with three separate rectangles and the associated code for each.

So for the left rectangle we used the code below, and one would expect a nice pretty red rectangle with a smooth black stroke and 6 pixel corner radius, but what we get is a rectangle with 4 uneven corner and these weird rendering artifacts which extend beyond the boundary of the shape.

//Left Rectangle

graphics.lineStyle(1,0,1,false);

graphics.beginFill(0xFF0000,1);

graphics.drawRoundRectComplex(10,10,100,30,6,6,6,6);

For the middle rectangle we set the "pixelHinting" property of the lineStyle to true, and this seems to remove the funky artifacts, but still leaves the different radius on the corners.

//Middle Rectangle

graphics.lineStyle(1,0,1,true);

graphics.drawRoundRectComplex(150,10,100,30,6,6,6,6);

graphics.endFill();

For the right most rectangle it appears we have it drawn correctly, but we were only able to do this by NOT using any line style at all, and instead having to create two rectangles with one nested inside the other.

//Right Rectangle Outer

graphics.lineStyle(0,0,0,false);

graphics.beginFill(0,1);

graphics.drawRoundRectComplex(300,10,100,30,6,6,6,6);

graphics.endFill();

//Right Rectangle Inner

graphics.beginFill(0xFFFFFF,1);

graphics.drawRoundRectComplex(301,11,98,28,5,5,5,5);

graphics.endFill();

So it is obvious to me that the Player is capable of rending this correctly, but it does NOT. I am not sure if the cause is in some type of sub-pixel interpolation algorithm, or it lies within the "curveTo" code for line strokes where the player might be using some type of cubicBezier algorithm and the control points are too close together to get an accurate curve.

It appears that the Flex team is aware of this issue and thus renders all of its "cornerRadius" borders by applying the same technique I am using above by doing two separate compexRectangle fills.

So apparently Adobe is well aware of this issue, and from what I understand, through unnamed sources, this isn't even on the radar for the next major release (Flash Player 10.) I think is a real shame since I believe it really undermines the quality of what can be produced with Flash when you want a high level of detail and your are forced to take a bunch of unnecessary hacks to work around it.

What is wrong with the Business Intelligence Industry?

2008-06-17T15:15:00.001-07:00

I just returned from a great summit meeting for BI hosted by Dashboard Insight, where I sat on a panel with other industry experts. It was a small group of about a dozen of us, and it was a great two day meeting. It was fantastic to have an opportunity to talk with other people who work with the same issues I face every day while working on BI stuff.

I was a bit concerned attending the event that I would not see eye-to-eye with some of these other experts, since I hold a relatively contrarian view when it comes to designing dashboard solutions. I believe that ALL business intelligence systems need to be designed from the top (user) down, versus the bottom (data) up. Pretty much all big BI vendors and industry experts have been preaching the bottom-up mantra for the past 20 years.

On the flight back from the summit I wrote an article that (hopefully) more clearly articulates my position and the logic behind it. You can read it here. In a nutshell I basically state that Big BI and current industry experts have a strong financial interest in not rocking the status quo, and are in fact hindering the efficacy of the very solutions they purport to be advancing.

By any objective measure, BI is long way away from reaching its potential, especially in the eyes of business users and organizations who have implemented seven figure BI solutions. My thesis is that because BI has its history rooted in the collection, transformation, aggregation and dissemination of data that the lions share of focus on any BI project revolves around the data and the not the end user. Reports and dashboards end up being after-thoughts at the end of big BI initiatives.

The BI industry has created its wealth through the selling of these solutions and expertise and insist they are needed to provide an effective BI solution. In my experience this is absolutely NOT the case. You can create a very effective dashboard solution with a very minimal BI infrastructure, although in some cases having a robust BI back end is definitely going to make things easier. As I mention in my article the BI leaders have painted themselves into a corner because even if they recognize the back end is not the place to start, they would end up undermining their primary revenue streams (not to mention customer confidence) if they openly acknowledged this state of affairs.

What I suppose will begin happening, is that more and more start up companies will start to address this schism and we will see a slow movement to more user-centric BI solutions that focus on business and user requirements with online tools that allow integration of distributed and heterogenous data. At the same time I still see a very valid place for traditional BI, as there are still thousands of companies with vast stores of transactional data that needs to be collected, processed, and aggregated into common structures that can be used to help feed the user-centric BI solutions.

New Degrafa Repeaters

2008-06-16T17:22:00.001-07:00

Sample Code Here

One of the first areas of Degrafa I started working on was a revamp of the current Repeater objects. Jason had some cool ideas on what we could do to make repeaters more concise, flexible, and powerful. After a couple of IM chats I had a reasonably good idea of what we wanted to accomplish and I rolled up my sleeves to see what I could do. You can see the most current rev in our Dev branch at Google Code, although I suspect we will end up cleaning up some of the name spacing and move the new GeometryRepeater class somewhere else.

One of the challenges the current Degrafa Beta 2 suffers from is that we have a myriad of specific geometry repeater classes like Line, Circle, Rectangle, etc... Not only does this clutter up our core class libraries, but it is also much less flexible than it can be.

The new architecture we are developing takes a different approach, instead of creating gemoetry specific repeater classes we wanted to create an abstract repeater engine. This is accomplished through our GeometryRepeater class which inherits from Geometry and implements the IRepeaterModifier interface via its modifiers array. Okay, so what does this all mean?

1. You define the geometry you want repeated with any number of Degrafa geometry objects, which can be separate objects or complex compositions. You are no longer limited to the current repeater classes, but can easily repeat ANY Degrafa geometry you create.

2. You define HOW the geometry repeats via one or more IRepeaterModifiers. Initially we have created a PropertyModifier that allows you to target one or more geometry objects within your GeometryRepeater and an associated property you want to modify. You can have as many RepeaterModifiers as you want to create composited repetitions. Over time we may add additional core modifiers, or you can create your own by implementing the IRepeaterModifier interface.

Below you can see how one might implement a Circle repeater that repeats along a horizontal Axis:

</modifiers>

</GeometryRepeater>

For a more complex example you can check out this demo that has view source enabled.

When you look at the demo you will see a simple circle geometry that is being repeated 30 times with multiple property modifiers being applied to it. If you look at the code there are some interesting things to note here about the PropertyModifier class.

1. We support targeting common properties across multiple targets via just one repeater like are doing here with the myCircle and myRect in the targets properties. We also support not only direct object properties, but also descendent properties via a dot notation (look at the property="fill.alpha") setting for example:

2. We support built in static operators as well as offset functions.

Not only can you provide static offset amounts (as in the example above) with fixed operators like add, subtract, or none you can also provide offsets as an array that will behave like a modulo function against the iteration number as in the radius modifier below. Additionally you can also provide offsets as dynamic functions as in the fill.color modifier that uses the custom colorOffset function found in the sample source files.

There are many more advanced features, but I will wait until we have pinned down the exact functionality for our Beta 3 release before delving into it further. Please check out the sample, and use view source to get an idea of what is possible.

Officially Working on Degrafa Team

2008-06-09T20:52:00.000-07:00

Over the past couple of months I have become more deeply involved in the open source Degrafa project that was started by Juan Sanchez and Jason Hawryluk over 1 year ago. Today the Degrafa team has grown quite substantially and I represent the 8th official team member.

There were three primary motivations for me joining the Degrafa team.

Reason 1. Degrafa just plain ROCKS!

What I love most about Degrafa is that it represents an extremely expressive, yet compact markup syntax to produce amazing graphical assets. Basically it allows me to do some really cool things with very little code.

Degrafa leverages the low level flash drawing API's to implement sophisticated vector and raster rendering via MXML markup. What does this mean to your average Flex developer? It means that you no longer have to resort to working in Flash or other vector based tools to create complex vector assets, and you no longer have to struggle with using procedural API's to manually draw your graphics. Degrafa piggy-backs off of the same metaphors that most XML based markup syntax does, in that the nested XML structures you use to define your page layout create a visual corollary to the actual physical layout of the code. With XML markup you can easily see the primary relationships between objects due to their nesting structures.

When you combine XML Markup with Flash/Flex's powerful binding mechanism you get a very expressive, compact, and intuitive way to code graphical assets. One example of this effect in action can be seen in some of the GeometryRepeater code I have been working on for Degrafa (to be covered in a later blog post.) I was able to take a couple very simple base classes and in less than 15 lines of MXML and 10 lines of AS create a completely dynamic single series Column/Area/Point/Line chart. That is 25 lines of total code, that is what I call powerful!

Reason 2. I wanted to give back.

When I started working with Flex in late 2005, I had never been directly exposed to the open source community and always regarded the open source movement with a detached fascination and certain level of disbelief that so many developers would volunteer their time for an apparently non-commercial effort. Over the past couple of years, my eyes have been opened to the many advantages to open source on commercial, personal, and social level. In learning Flex and watching the growth of the community I have repeatedly reaped the benefits of others efforts in many areas ranging from free (and excellent) components, blogs, and sharing of ideas.

I think an individual can only "take" for so long before starting to feel an indebtedness and desire to give back to the community. Probably most of all is the great level of personal satisfaction I receive when I contribute something to the community that others find great value in. I probably receive more satisfaction from releasing a component I spent time working on and seeing many people positively respond to it in my blog than I do from landing a large consulting contract.

Reason 3. Team Culture.

Both Jason and Juan (the two founders of degrafa) are so genuinely enthusiastic about the project and immanently accessible. When I dove into Degrafa for the first time a couple of months back for work on my Gauge component I started trading e-mails with Juan, which quickly led to multiple chat sessions per day. Juan was always available to answer questions and point me in the right direction, and then he connected me with Jason, who was equally accessible and helpful. Once I started working with Jason and he showed me the guts of Degrafa I realized "wow, these are some truly bright people, creating a really amazing framework." I was hooked.

Moving forward my role at Degrafa will probably have many facets, but I will most likely be focusing as a direct contributor to the core engine as well as helping to provide some organizational support for Degrafa as an entity and open source project.

3G iPhone appears to be thicker not thinner.

2008-06-09T16:25:00.000-07:00

Is it me, or Apple? The recently (a few hours ago) announced 3G iPhone was stated to be thinner version of the current model. But a close look at the tech specs on the Apple store site indicate that it is actually .48 inches (12.3mm) thick while the previous version was only .46 (11.6mm) thick.

The difference in the new iPhone design is that the edges appear to be more sculpted like a Mac Book Air, thus the phone probably "feels" thinner, when in reality it is thicker.

Flex is MORE than just a pretty front-end

2008-05-08T18:57:00.000-07:00

One of the most powerful aspects of using Flex is not necessarily the language or IDE itself, but the design patterns that the combination of Flex and the Flash Player allow you as an software architect to implement. For developers who are used to building large scale web applications, the traditional approach is to use some version of an n-tier architecture where your platform consists of a data persistence layer (fault tolerant DB's), web application layer such as .Net, Java, ColdFusion, etc (to maintain session state for stateless HTTP and enforce business logic), and your presentation layer (HTML, Javascript, Flash, etc..)

This is a topic that I have been thinking about quite a bit over the last couple of years since I started using flex, and the more I work with Flex the more I stand behind the conviction that packages like Flex/Flash offer us a new paradigm for web application development. I realize this is a long post, but my hope is that others out there in reading this post will start to think about web application development in a new light and realize there is potentially a more powerful (and enjoyable) way to design web applications.

The traditional n-tier approach, while tried and true (and an architecture that I have implemented in more solutions than I care to remember) has several shortcomings that are primarily related to the limitations of web technologies at the time the n-tier approach came into vogue almost 10 years ago. Below I list four fundamental shortcomings with this approach, although I am sure there are several more.

Shortcoming 1 - Multiple Languages/Environments:

Taking this layered approach to building applications, requires the developer to code in several languages across several tiers and IDE's. Even in the most simplistic web application, the developer is going to have to be writing and debugging some type of SQL via one IDE, then working on some application tier code with something like Java or .NET, and finally writing their presentation layer with some HTML/Javascript or Ajax Library. All said, the developer is going to have to be pretty efficient at knowing at least 3-4 completely different languages as well as their associated tooling, or even worse, have to distribute that work across multiple team members. Then, when it comes time to debug the application, even in the most integrated of environments, the developer will be debugging through several different debuggers, trying to track data structures and logic as they pass through each layer, all of which takes time and not a particularly efficient way to troubleshoot an application.

Shortcoming 2 - Brittleness:

I don't care how "loosely" coupled or abstracted you have designed each layer, by its nature this approach tends to create brittle designs where each layer is written in its own language and has unique implementation details. In my experience, no amount of planning for future functionality and abstraction fully covers you in your design, and more times than not, an innocuous feature request like "Gee, it would be nice if we can capture the users surname on our registration form" has the developer modifying the presentation layer, business/app layer, and data layer to accommodate a relatively simple request that ends up rippling through every layer of abstraction in unavoidable ways.

Shortcoming 3 - Maintaining Client State on the Server:

HTTP by its very nature is a stateless protocol, each time that browser loads a new page, all state is lost (cookies aside), sorry folks, but that is the way it was designed. For that reason, web application architects find themselves jumping through all sorts of hoops to maintain state on behalf of the client within their centralized web servers. This problem becomes further exacerbated when we are designing a scalable web application with many web application servers. You now have to deal with load balancing IP requests across multiple web servers and dynamic client IP pools. This makes the architecture required to build out a web farm a much more complex issue for your application design, testing, scaling and debugging.

Shortcoming 4 - Browser/Desktop Compatibility:

This is so obvious I don't need to hammer it further, but needless to say writing web applications (or even desktop ones) that behave nicely on a wide variety of clients and environments requires writing LOTS of code that deal with compatibility issues and adds zero additional functional value to your application.

So how does a technology like Flex address these issues? To me, it wasn't immediately apparent, as I when I started using Flex I just thought of it as an easier way to build a more attractive and interactive presentation layer, but the more I used it and the more I saw how powerful the Flash Player, was I started to realize that there was a real opportunity to do things differently. It was at about this time (a couple of years ago) that I was starting the initial architecture for our on-demand BI dashboard offering, which was the flagship product of the startup I had co-founded.

When evaluating architectures and design patterns I wanted an approach that was going to maximize developer productivity while giving us a consistent and maintainable code base. I looked at frameworks like Cairngorm (which largely borrow from established Java design patterns) and ultimately with the help of my VP of Engineering ended up designing a custom framework and architecture over the course of about 6 weeks. This framework allowed us to successfully go from a written spec to commercial product with two live Fortune 500 clients in just under 6 months with a team of three developers including myself who was only working on the product part time. All in all, I was very proud of our effort and what we were able to accomplish in such a short amount of time, and outside of having a very dedicated team who worked extremely hard, I attribute that success to being able to leverage a completely new paradigm for web application development.

So what did we do that was all that different, what was our secret? Well, obviously Flex was a big part of it, but more so it was what Flex allowed us to do from an architectural perspecitve. Fundamentally we still had all the basic pieces of an n-tier architecture in place with a DB on the back end, a web service bridging the client to the DB, and our client code within Flex. But what was different was WHERE we put the code for those layers. When you looked at the composition of our code which was spread across SQL in the form of creation scripts and stored procs, .NET for our C# web service and Flex for our client, 98% of our code was all written in Flex. We had less than 3000 lines of code within SQL, less than 1000 lines of code in .NET and the rest was all Flex!

How did we do this? Why did we do this? Well, we essentially condensed all of our abstractions layers into one platform and language - Flex/ActionScript. We still had a "presentation, business, data" layer but it all existed within our Flex classes via a MVC framework approach. Our model classes would serialize themselves into XML which would then call our web service which primarily served as a secure pass through to our DB, and then the DB would digest the XML via associated stored procedures and update or retrieve data in the appropriate tables. This meant when we had changes to the spec and had to add that a new data field we simply updated our Flex UI and underlying model class and the appropriate stored proc on the DB. The separation of layers worked beautifully, all the UI code and underlying model code was in one environment and IDE, while all the code to maintain referential integrity of the persistence layer sat in the DB where it belonged. The primary motivation for me to adopt this approach was to focus as much of our development energy on directly enhancing functionality versus working on plumbing and repetitive coding tasks that did not directly add to the feature set.

This approach afforded us huge advantages in our development. The single biggest advantage was the speed of development, once our data tier and pass through web service were stabilized (with less than 4k lines of code total, that did not take too long) we could debug all of our code in one environment. The whole architecture of the platform sat in one place, Flex, and while our .Net web service exposed some utility functions for us (like SMTP access, and some server side file manipulation) we rarely had to leave the Flex debugger for troubleshooting. The second big advantage that we had was how we could scale our application on the web tier. Since all of our complex client state existed within Flash (and thus on the client) the only load our web servers received was for the stateless pass through requests to/from the DB and for some utility work. Thus, we did not have to maintain any session state on our servers and scaling out the web farm was simply a matter of adding more web servers behind a load balancer. The back end DB still required clustering for fault tolerance and scalability, but we had a relatively simple DB schema that made this easy enough to achieve.

All in all, had we taken a more traditional approach to designing our application I can confidently say it would have taken us at least 50% longer to build with half of the interactivity and robustness that our application released with on version 1.

This approach worked for one primary reason, and that is the ubiquity and the power of the Flash Player in combination with the abstraction of ActionScript 3. Yes, everyone knows that Flash has something like a 98% penetration rate, and that a .swf file is going to act pretty much the same regardless of the browser or OS it plays within. But, when Adobe released Flash Player 9, it left the leagues of a neat way to show cool animated effects, and offered a true platform to deploy robust applications against. The performance of the Flash Virtual machine is pretty amazing when you consider how compact it is. For instance I can manually filter row by row through thousands of database records using powerful regular expression match patterns within mili-seconds ON the client. Flash is not without its warts, but all-in-all a very compact, and powerful platform to write code to. I encourage other web application architects to look closely at the Flex/Flash combination for their next project and consider how they might benefit from this approach.

Steve Jobs would cringe - Apple Wireless Keyboard Add-On

2008-04-25T12:01:00.000-07:00

When I purchased my Mac Pro, I had an option of which keyboard I wanted, I ordered the wireless keyboard, knowing how small it was, and thinking it would be next to useless for coding, but perfect for a family Mac-Mini. Well, I actually kind of like the keyboard, the key-throw is good, I can type fast and it is really light/elegant. BUT, it is pretty diminutive, and so light that when I use it in my regular seated position (feet elevated on desk, very reclined in my chair) with the keyboard on my lap, it kept slipping down. I found myself unconsciously holding it in place with my left thumb so it wouldn't slide down. Typically with large keyboards, the pad of my palm would rest against the bottom keeping it in place.

So I devised a super high-tech holder custom designed for the Apple Wireless Keyboard. It is very lightweight fits perfectly and made from bio-degradable earth friendly materials. See the photos below. It works perfectly, the keyboard is now secure in my lap and very comfortable. Perhaps when this one wears out I will create one out of a nice piece of lucite :)

Update:

My v1 keyboard holder ended up becoming a little "floppy" after 2 weeks of use, so I created a new one out of PVC/Sheet. Other than a slight mishap with the router in my haste to replace the more organic version, it worked out fine. A little hook-and-loop tape allows me to mount and remove the keyboard with ease. The size is perfect for typing in my lap.

Should Apple purchase Adobe?

2008-04-25T11:22:00.000-07:00

I am sure I am not the only one who has posed this question, I know there was lots of conjecture about this a couple of years ago. But I think the argument is even stronger today. It appears that Apple is really trying to take advantage of the fiasco that is Windows Vista and is pushing new PC buyers to make the jump to Apple (hey it worked on me) But, Apple is going to have a tougher going with the developer crowd and the corporate culture. If Apple ever hopes to jump the chasm and become an integral part of the enterprise they are going to have offer compelling reasons for developers to jump ship as well and write software for Apple. Right now the only compelling reason for developers to make the switch and write software for Apple is because they become so enthralled with the Apple experience that they want to be part of it (once again, I am guilty as charged) or they are specifically targeting OS X or the iPhone for their offering.

But, as motivated as I am to develop the next great iPhone/OS X app, each and every time I put forth an earnest effort to do so I am struck by how antiquated the development tools, languages, and platforms feel to me. For a corporate developer who just needs to get the job done and could care less about how sexy something looks/feels they will have even less impetus to make the jump. One thing that Microsoft did extremely well, although not so much any more, was to practically spoon feed developers with their technologies. It was SOOO easy to pick up a MS technology, not necessarily because they were the best technologies, but because MS invested heavily in providing training materials, live events, practically giving away their software via MSDN, etc. Now, I see the open source movements doing a better job of this with the robust communities and communication infrastructures that have developed as a result of the internet.

So here I sit at a cross-roads, do I invest the time to develop an application that is specific to Apple either via Objective-C/XCode or via proprietary AJAX for Safari extended WebKit? Well, either choice seem like several steps backwards from where I am right now. Currently I develop in Adobe Flex, and while the tooling can stand room for improvement, it is still far ahead of XCode. But where Flex really shines is the combination of the language and the design patterns it easily supports in combination with the power and ubiquity of the Flash player. Having to go back and write AJAX code and deal with browser compatibility issues and the lack of robust profiling/tracing tools is just painful. Once you know how much better it can be, having to go back and use tooling/languages that you were using 8 years ago just hurts. The XCode/Objective-C road does not appear any better. Damn Flex and Adobe, if I never went down this path I wouldn't know any better, and I would be happily struggling with AJAX, HTML5, and excited that I now have a Canvas HTML element that has a graphics context I could draw to.

Okay, back to my original theme for this post, why should Apple purchase Adobe? If for nothing else, Flash and Flex. If Apple controlled Flash they could easily put it on the iPhone and still force their lock-in for flash-enabled iPhone apps having to be distributed via their AppStore. With ActionScript, Flash, and AIR in their back pocket, what they have is universal development platform that would open the doors to a much wider developer audience. Because a developer knows when they are targeting Flash/AIR they are not only targeting Apple but pretty much any OS/Client. In the words of Bob Warfield, the friction is greatly reduced. It would also put Apple on practically EVERY desktop because Flash sits on every desktop. Similar to the logic I used in purchasing a Mac Pro, where I knew worse case I could run Windows natively (but have yet to do so) I would be able to develop for the iPhone/Apple knowing that my application would also work anywhere elese. Even better, instead of having to learn the ins-and-outs of a new language based on an old development paradigm I would either leverage the knowledge I already have (if I knew Flex) or be investing my time in learning something new, but more powerful and more efficient.

But, wait, there is more to this argument outside of my myopic developer/engineer perspective. Apple is really pushing to deliver software to creative folks, albeit more at the hobby level, but nonetheless what company has the best and most established creative software? Adobe!!! If Apple wants to position themselves as the company that creates the hardware, software and distribution channels for creative content (music, video, multi-media, etc.) why not also control and offer the tools to create such content... it would go a long way to reducing the friction in some of these marketplaces, where the creative authors can seamlessly distribute their works via the Apple channel. This would allow them to target all strata of their verticals that they want to sell into, and these offerings create a self-reinforcing viral effect. This said, I am not sure how comfortable I am with a company like Apple holding such a dominant position in the market based on their historically closed nature.

And this is where I come to why I think it might not work, and that is due to cultural issues between the two companies. Adobe has been making GREAT strides to become a much more transparent and open company, they are investing heavily in contributing open source code and really seem to be figuring out how to be a good corporate citizen while still turning a profit. Apple, not so much. They seem to still cling the old-school closed source mindset, where everything is shrouded in secrecy and tightly controlled. This does have its advantages, but long term I don't think it will benefit them. So what happens when a company like Apple acquires a company like Adobe with the differences in their respective cultures? I don't think it would be a very pretty picture and could have the potential to destroy Adobe and the value they bring to the table. But from a purely mercenary/capitalistic view I think it would still be in Apple's best interest to do so.

From looking at the Yahoo finance stats today, here are some metrics for both companies. It would seem with Apple's cash on hand they have enough resources to make something like this happen (they have almost as much cash on hand as Microsoft) if they were motivated to do so.

Apple:

Market Cap: $148B

Annual Sales (2007): $24B

Total Cash: $18B

Adobe:

Market Cap: $19B

Annual Sales (2007): $3.1B

Total Cash: $1.57B

I will be speaking at Flex 360

2008-04-24T07:18:00.000-07:00

For those Flex developers out there who have not heard of the Flex 360 events, it is a GREAT opportunity to learn more about Flex, share information with other Flex devs and just generally have a great time. I attended the event in Atlanta, and Tom & John put on a great show. The sessions are really informative and pretty informal, which creates a great atmosphere for sharing and collaboration. At this event I will be speaking about Data Visualization and Dashboards with Flex 3. I did a similar talk at Adobe MAX back in 2006, but it was on Flex 2, and geared to a less advanced audience. So for this preso, I am going to have to crank it up a notch and come up with some pretty slick examples that push the boundaries of Flex, and show people how I have been building and teaching dashboard development to other Flex devs over the past couple of years.

Mac Pro 8-core and Leopard Week 1:

2008-04-23T08:11:00.000-07:00

It has been almost a week now after my transition off a wintel machine onto my new heavily loaded Mac Pro.  Like so many others before me, and the millions who will follow after me, it has been an epiphany.

I have been a hard-core windows guy for the past 15 years, just devouring everything Microsoft.  I am/was deeply intimate with almost all of Microsoft's software offerings from their (anti)productivity suites in their Office Tools to their back end server stacks like Exchange, SQL Server, Biz-Talk, Commerce Server, IIS, GreatPlains/Dynamics, Sharepoint and more.  I also exclusively developed in MS frameworks like VB (from years ago), C# etc..  all designing and developing business applications.   I was one of three Principals at a medium sized local MS Gold Partner consulting shop in charge of delivering multi-million dollar custom MS applications leveraging technologies in the above mentioned stacks.   A few years ago I slowly started to drift away from MS, the first material separation occurred when I adopted Adobe Flex as the primary development technology for my new startup and was just shocked with the refreshing change of the "open" community nature that Adobe adopted in its beta.

Now a couple of years later, with Apple making it almost impossible to ignore their presence with their adoption on Intel chipsets and the ability to run Windows side-by-side with OS X I made the leap.  What is amazing, is that leap is a more like a short step, the effort involved to make the switch is getting smaller and smaller.   I can't say enough good things about the quality of the Apple hardware itself, the 8-core is a dream, and even more importantly OS X feels like what a computer is supposed to feel like in 2008.   Spotlight (OS X's universal search tool) is just so easy... you have a thought like "where is file XXX" and you just type "XXX" into spotlight and there it is.   No more waiting several minutes while Windows search agonizingly moves through your non-indexed files at a snails pace (although  I am not sure how fair that is since I was not using an 8-core processor with 14GB of RAM on my wintel box.)   Lots and lots of little things are just so much more seamless and intuitive with OS X.

This is not to say it is all roses, there are little things here and there that I miss from Windows, but overall the experience is such that you feel like you can forgive OS X for its shortcomings, while you want to blame Windows for theirs.  To Microsoft's credit they still have a few amazing products that I have mentioned before, like Visual Studio and SQL Server 2005.  Microsoft also has the very unenviable challenge of trying to support an OS that is 20+ years old and has successfully maintained backward compatibility on a ever shifting hardware platform that has to support billions of permutations of hardware configurations.   Apple on the other hand got a fresh start with OS X and they completely control the hardware platform, which is a much easier proposition.  It will be interesting to see how Apple is able to successfully avoid the bloat that will come with time as their adoption rates increase and the need to support backward compatibility becomes more prevalent.

So my message to any windows users considering the switch to Apple is to go for it, you will not regret it in the slightest.  The only issues I have found thus far is that GoToMeeting requires Windows to host meetings, and QuickBooks online has tied themselves to ActiveX technology via Internet Explorer.  For both of these I simply run VMWare fusion, although I will probably migrate to Adobe Connect for meetings, and I am hopeful Inuit will release a Flex client for QuickBooks sometime in the near future with their QuickBase announcement.

The Cult of Apple

2008-04-08T08:07:00.000-07:00

Help, I think I have become addicted .... to Apple.

This weekend, while out with my wife (sans the kids), I made my SECOND trip in 2 days to the local Apple Retail Store. I left the store about $500 lighter after picking up a bright pink Nano for my wife and an Apple TV (at $229 how can you resist??) At this point I can just feel myself getting inexorably sucked into the vortex of quality, attention to detail, and desire that Apple has come to embody in most of its products. It started a couple of years ago with my MacBook purchase that was simply used to make it easier for me to test our website within safari. This was followed up a few months ago with an iPhone purchase, under the auspices of keeping up to speed on current UI and usability trends in the industry. Now my addiction to apple … I mean research … has me buying iPods like candy and buying devices I didn’t know I had a need for. Next up is replacing my aging P4 with a brand new Mac Pro 8-core with some serious memory (12GB should do.) After buying the iPhone I was truly hooked… there is something about the products Apple produces that creates this incredible emotional desire for “more of that” (at least for tech people like me.) I commented to my wife that the desire I have for Apple products is quite similar to what I witnessed in her when we walked into the Louis Vuitton store and my wife was ogling a pair of $1,200 shoes. It is really an admirable and amazing feat that Apple has pulled off, by focusing on quality and attention to detail at every level from product design, implementation, to even the jewel like packaging materials, they have created a desire for their products that I have not experienced with any other technology outside of the lust people feel for cars like BMW, etc. Apple doesn’t get it right all the time though. On OS X I still can’t understand their fascination with spawning windows like rabbits in heat, and on a much more trivial note the shopping bag that the Apple Genius put my purchase into required an instruction book to figure out how to carry. It was this weird contraption with two separate ropes going through a total of four riveted holes in the bag… I wasn’t sure if it wanted to be a knapsack, hammock, or some weird S&M device… Walking through the mall I constantly found myself trying to figure out how to hold the damn thing properly… I think they got a little too clever for themselves on that one.

Okay, back to my addiction…. Yesterday I plugged in the Apple TV to our widescreen HDTV in our bedroom, within 10 minutes I had our modest iTunes collection of about 3k songs and our photo library of about 2.5k pictures streaming from it. So last night my wife and I sat down to “rent” a movie, I was a little disheartened to see that once we started watching it we only had 24 hours to finish it… I wrongly assumed it was “ours” for 30 days. That issue aside, we never even got to the movie… We became so fascinated with watching the slideshow of our photos and listening to music. It was an amazing experience for me… we had about 7 years of photos from before our wedding, various trips to Tahiti, Hawaii, coast of California, the birth of our two sons, etc… It was just amazing to watch and recall. It gave me a whole new appreciation for taking photos, as I was completely engrossed and entertained. It gave my wife and I an opportunity to reminiscence and connect at a level that so far eclipsed just watching a movie together. After a couple of hours I ended up knocking off and going to sleep. This morning I found out my wife stayed up past midnight watching our slide show.. Now I am not sure how entertaining this experience would be after seeing these photos for the umpteenth time, but I can tell you it was a real eye opener for me in terms of its intrinsic entertainment value. When we finally get around to watching the movie we rented and delving into some of the other features of the Apple TV I will report back.

- Tom

Gauge Component v.02

2008-04-04T18:05:00.000-07:00

After the response I received on the first release of the Gauge component I thought it was time for an overhaul of the code since this original code was written when I was learning Flex back on the alpha bits of Flex 2.0. Since that time Flex has moved on, and I have learned quite a bit.

This version of the component is a complete re-write of the code and is vastly simpler and easier to read/extend. First a thanks to Peter Ent, as his stab at a gauge was infinitely better thought out than mine and much more compact, the component code is very close to his, with a couple of modifications. The other bigger, and what I consider very significant change for this component is how it leverages Degrafa… which if you are not familiar with and you are doing any component development you are doing yourself a disservice by not taking a close look at it. I used this little mini project as an excuse to really dig in and find out what Juan, Jason and the rest of the Degrafa team had been working on since they seemed to be popping up everywhere.

When I dug into Degrafa I was REALLY impressed at several levels. Not only are the examples just plain sweet visually, but the framework itself is very elegant, compact, and infinitely extendable. In this version of the gauge I built two skins completely in Degrafa and MXML markup, you can toggle the skins and they will still respect the color of the styles you choose. BTW, the skins can be set via CSS, and you can have separate skins for each part of the gauge (bezel, face, pointer, center, indicators.)

The beauty of this approach with Degrafa is three fold. First I am using MXML to create complex geometric relationships which is infinitely easier to do in markup versus procedurally because the markup itself creates visual relationships through the inherent nesting and grouping structure of XML. Second, Degrafa does a great job of leveraging Flex’s binding mechanism, which in my opinion is one of the most powerful things within Flex. Finally the workflow is vastly improved, I can do everything from flex builder and not jump between Flash for building vector assets and exporting to symbol .swfs etc…

On some later post I will probably dive into the framework itself, since I ended up having to extend it in a couple of areas and it only took me a few minutes to do so. For example the “tick” marks you see on the gauge I ended up copying the VerticalLineRepeater which extends the base Repeater class, and within 20 minutes after modifying a few methods and adding a couple trig functions, I had this arc of lines that I could declare in MXML… I remember the first time I did that in the original gauge component it took me hours to fine tune procedurally via the API.

At Juan’s urging I wanted to get this up so it could be shared with the community, as such the code might stand a little clean up here and there, but overall it should be pretty easy to follow. I am also going to be writing an article for InsideRIA that goes into all the gory details of building out this component with Degrafa, which was actually quite simple and very enjoyable.

----------------------------------------------
Caveat... The tweening of the pointer is a little flakey, as I threw it in at the last second, and I need to do a better job of handling the rotate effect and updateDisplayList collisions.

Update: The newest version of this component can be found here.

iPhone Developer Program Take 3

2008-04-01T15:21:00.001-07:00

Well it looks like i have been accepted into the iPhone SDK beta developer program.. I must say that thus far the experience has been far more like a Microsoft Beta than anything else, where you are made to feel privileged to have been accepted after jumping through a corporate bureaucracy.

I am not sure how widespread the program is now, potentially they are allowing everyone in. What was odd though, is that last week I got an email requesting faxed copies of my articles of incorporation for BrightPoint Consulting (the company I registered under) which I faxed over. Just a few moments ago (a week later) I receive an email telling me that upon signing the license agreement and forking over my $99 I would be enrolled. I promptly did both, and now I am an "official" iPhone developer. I find it odd that it required such a manual process... I would assume this is pretty manually intensive if they are having humans read and correlate faxes to each developer account.

Last week I downloaded the latest bits from the second release of the iPhone SDK beta. I see that Interface Builder is now included, which is supposed to be the "magic sauce" for creating UI' s in Cocoa applications. But as I had thought, it still requires quite a bit of work to set up an application. The good news is that it appears that there is finally some documentation on the JavasScript DOM api for touch/gesture events. I planned on exploring that a bit further when I could carve out some time.

I will post again when I receive the SDK if there are additional resources provided outside of what everyone else see, and if my sharing does not violate some license EULA.

Gauge Component v.01

2008-03-21T16:11:00.001-07:00

Well, the time has come for me to give back a little to the community from which I have learned and benefited so much. People like Ely, Doug, and too many others to list have been a source of inspiration and a repository for useful components that I have used across many projects.

This gauge component actually stems from some of my first work with the beta bits of Flex 2.0 over two years ago. For business and IP reasons I could not share this code, but now as conditions have changed I can offer this code to the community under an MIT license. It is far from a polished component, but the foundation is there. Follow this link to BrightPoint Inc. to go see the gauge in action and right click for view source.

Some of the things on the to do list will be the following, based on interest from the community and feedback.

1. ASDocs (duh...)
2. Exposing the .swc skin more easily so all the gauge parts can be swapped via MXML or at runtime.
3. Cleaning up some of the min/max label stuff (try clicking on the radio buttons at 5 and 8 o'clock.)
4. Providing the user a way to input values via scroll-wheel our mouse.

At some point I might create a commercial version of this component, but everyone is free to use this code in whichever way they see fit.

Any suggestions/comments for improvement are welcome.

Update: The newest version of this component can be found here.

Removing items from an Array Collection

2008-03-18T17:45:00.000-07:00

Today I was troubleshooting a bug where items weren't not being removed from a Flex DataGrid the way I expected them to. What I discovered is that in this particular case my laziness in abusing the ArrayCollection was causing an unobvious (to me) bug.

Here is the simplified code that looks for a match to remove items from an ArrayCollection (yes I could use a filter, but that doesn't TRULY remove items from the ArrayCollection.source, and I can never remember the 5 lines of code I need to create one correctly.) So this was my first attempt at removing items from an ArrayCollection.

  
 ac.addItem("foo1");
 ac.addItem("foo2");
 ac.addItem("foo3");

 for each (var foo:String in ac) {
     if (foo.charAt(0)=="f")
         ac.removeItemAt(ac.getItemIndex(foo));
 }

Now, what would you expect that trace statement to output? Personally I expected to see a big "0", but alas the ArrayCollection is not truly keeping proper indexes as iterates through its for each. Realizing that this was probably an index issue with the iterator I wrote the following which works correctly as it backs down from the last element, thus not distrupting the index as it moves backwards.

for (var i:int=ac.length-1; i>=0; i--)
{
  if (ac.source[i].charAt(0)=="f")
  ac.removeItemAt(i);
}

The above code does what I was intending, correctly. I guess a lesson learned on thinking that Flex did EVERYTHING for me... guess I can't be a completely lazy programmer.

Why I won't be writing native iPhone apps anytime soon.

2008-03-18T08:42:00.001-07:00

Here is why I won't be writing native iPhone Applications: (at least not yet)

After spending about 40+ hours with the new iPhone SDK, getting up to speed on Objective-C (which is kinda cool) and modifying a few sample apps with XCode I have come to realize that while Apple makes amazing consumer products and interfaces, they fall completely short of the mark when it comes to developer tooling interfaces. Perhaps it is because I do NOT come from the land of Unix (well okay I used Unix in University, but that was over 20 years ago) and I am not a huge proponent of the command line and I prefer nice comfortable IDE's like Visual Studio or Eclipse that I don't see the magic here.

Just watch the video above to witness what kind of gyrations, both in code and UI, you need to go through to wire up a button to make a beeping noise. Windows popping up all over, connecting objects from one window to another with visual lines... opaque NIB files, etc. I am sure compared to command line C++ development this might seem like a godsend, but for someone who has spent the last 2 years developing almost exclusively with Adobe Flex, this seems like a step back into the stone ages. While I am not a huge fan of Microsoft, Visual Studio really sets the bar for a highly productive and functional IDE, Eclipse is getting better and almost there. When it comes to programming languages to create expressive and immersive UI's there is no beating flex.

I am so eager to develop immersive multi-touch applications on such an amazing device as the iPhone that I have seriously considered investing the 90 days or so I think it would take to develop a modicum of proficiency on new tooling (XCode) and language (Objective-C). But when I look at how pleasurable the experience of developing with these tools would be it looks like it would be trying to go surfing on a huge 100lb redwood board versus the 6lb epoxy short boards I prefer. While I would still be catching waves, things would feel so ponderous and slow. I thought that perhaps I am just missing it, and I needed to shift my mental paradigm to a different one so I would "get-it". After watching the video above, I am not sure that I am missing anything.

Obviously an ideal dev platform for me would be having the iPhone support Flash with Multi-touch so I could develop in flex, but I don't see that happening anytime soon. Especially considering the restrictions in the SDK. The one nuggets I did find in doing my research was that one of the SDK videos talked about the Javascript extensions they were building and the new CSS standards they were supporting. Some of the glimpses I got were:

Gesture support via Javascript (gestureStart, gestureEnd...)
CSS Animations
Transitions
Full SVG support and drawing (although not sure if that is restricted to markup or can be dynamic.)
Javascript access to SQLite !!! (this is huge in my mind as it could really allow online/offline apps)

The challenge I have though is that I can't find ANY documentation to support those javascript API's. I am thinking that perhaps creating web-apps with these javascript API's will afford me the ability to use more productive tooling and get the same great features. I just need to figure out how/if the SDK will allow us to support web apps running in an offline state, and if it will be possible to sell web-apps via the AppStore.

iPhone SDK first impressions

2008-03-13T18:06:00.000-07:00

Well, I am pretty impressed with the level of effort that went into this SDK. I am on day four of just looking around, and I have yet to write a single line of code, which is very uncharacteristic of me.

The SDK is pretty polished, with custom graphics, and a deep set of well put together videos (available via Apple Developer Connection) free on iTunes. The videos were a great way to get a lay of the land.

Other than starting to get my mind around Objective C after spending the last two plus years immersed in Adobe Flex feels like a bit of a step backwards. I am not super fond of the C/C++ structures and coding paradigms, as they feel like I am having to do a bunch of unnecessary work at lower levels of abstraction that is needed. Having to deal with memory allocations and stuff like that is a bit of PIA, but perhaps it is just because I have been working in higher level languages for so long.

One interesting bit I ran across in the videos is that they have extended the javascript API for Web Kit to support the handling of multi-touch getsure events like gesturestart, gestureend, and gesturechange. But I have been unable to find any other documented API references to this, and in loading up Dashcode (their web IDE for the iPhone) I could not get any more info.

Learning the iPhone SDK

2008-03-09T10:41:00.000-07:00

I thought it might be interesting to chronicle my learning of the iPhone SDK and building native apps for the iPhone. I suspect that many engineers like myself will be coming from different development platforms and might find some value in learning from someone else's trials and tribulations of learning how do program for an iPhone.

Never having programmed for the MAC before this will be a completely new experience. At this point I don't even own an iPhone, but I see it opening up new markets, distribution channels, and most of all a new way for humans to interact with computer (smart) devices. For these reasons I wanted to try my hand at creating a native iPhone app.

Step 1 was downloading the SDK, which was a 2 day exercise in itself, due to glitches at the dev center site at Apple.

After downloading the SDK, I did something I rarely do, and this is start reading the docs. Usually I just dive in, fire up an IDE and start trying to bang out an application. In this case since, I have never programmed in objective C nor worked with the Xcode IDE I thought I would start at a higher level.

More to come if my interest does not wane and it appears that these type of posts provide value to others.

Why I don't think Microsoft will ever "Get IT"

2008-02-15T18:36:00.001-08:00

First a caveat, I come from a strong Microsoft background. Since 1995 when the web really started to take off I have used MS technologies almost exclusively. Prior to that, my experience in Pascal, C, Fortran, etc was all really on pre-PC systems. Circa 2004, I found myself as a Principal of a well known San Diego based MS gold partner consulting firm. I was responsible for delivering million dollar plus projects on the MS stack, and was a frequent speaker at MS events. At the time I thought MS was the best thing since sliced bread. They had phenomenal developer support, and went to great lengths to ramp people on their technology.

Well a lot has changed in the last few years, least of which is my impression of MS technologies. I always had a hard time understanding the prejudice I would see against MS technology, especially in the Fortune 1000 enterprise space, where even mentioning the name would get you a sour look from entrenched IT people. I pretty much chalked it up to unsubstantiated emotional prejudice on the part of non MS people who liked to look down their nose at MS technologies and the people who used them.

Now that I have departed from the MS stack and have been working almost exclusively with Adobe Flex and its related technologies for the past 2.5 years, I have been exposed to the other side of the coin. I still do/manage some .Net development for our web services, file I/O and DB related stuff, but it only takes less than 5% of my engineering time. The Flex community has a very open-source feel to it (I guess now it really is becoming open-source.) When I first started working with Flex I was astonished to see that Macromedia (now Adobe) gave me as a developer direct access to the source code. What joy! I could see how things worked, tweak it, and learn from it. Once I got involved with the Flex 2.0 beta, things got even better. The team had a high level of transparency and I could actually talk with their engineers.

What a stark contrast to my experience with MS. Trying to talk to the real engineers in a MS beta was an impossibility. The closest I ever got was a couple of product managers that were interested in what I was doing because I somehow managed to get a solution put together that integrated Active Directory, BizTalk, MS Great Plains, Commerce Server, E-Connect, and several other major MS server technologies. Apparently, my team was the first in North America, or even the world that actually got all the technologies to play nice with each other (interestingly enough it would have been impossible if I had not leveraged VMWare.) The attitude at MS on their betas, at least in my experience, was that it was a HUGE privilege to be invited, like getting to sit at the altar of some god.

All in all, MS runs a very opaque shop. In light of how much traction open source is getting, and what a refreshingly cooperative and community feel it has to it, the MS approach leaves one with a bitter taste once they have experience life on the other side.

What prompted me to write this post, is that just today I was working with one of our engineers, helping him to create a .Net setup project. This was something I had done a few years ago, and I remembered it as being pretty trivial. The challenge I was facing today, is that our engineer - who is a very bright guy, was struggling to put together the setup package. The MS Docs were next to useless, and when I saw what the MSDN has devolved to I was shocked at how hard it was to find good information on how to do the most basic of things.

What became clear to me today, and something I have been coming to understand over the past couple of years is this. Microsoft will never "get-it", it is a cultural thing and it starts at the top and ripples down to permeate everything at MS. What I mean by "get-it" is that MS just does NOT understand basics of human computer ( or anything for that matter) interaction. Their applications all seem to have been designed with a bottom up approach, where usability is bolted on versus built upon, their documentation is obtuse and hard to follow in any practical way, their UI's suffer from some significant usability issues, etc... Overall, I just see MS falling behind quickly, at least when it comes to the relevance (versus the dominance) of their technology.

I realize that is some pretty harsh criticism, and I am not saying I think MS is evil or puts out terrible software, I just think they are having a hard time embracing the innovation that is occurring around them, and they are out of touch with the evolving paradigms of developing usable software and products. The result is sub-optimal software and tooling.

I would be remiss if I did not mention a few MS products that I hold in high esteem. First, Excel, it is an amazing tool and incredibly powerful for what it is, it meets so many use cases and has a broad and varied user base. Second, SQL Server 2005 - amazing set of tools that do pretty much anything you want with a DB, their ETL and OLAP has some really nice tooling. Finally, Visual Studio, a GREAT IDE, that has yet to be matched by any other product I know of. Eclipse has quite a ways to go before it matches the functionality and utility of Visual Studio. I only wish I could build flex/air apps in Visual Studio!

Commercial Components for Flex

2008-01-23T09:46:00.000-08:00

I just recently became aware of ILOG Elixir, which is a set of advanced data visualization components for Flex (Gauges/Dials, Maps, Treemaps Gantt, etc) that will be distributed and sold directly by Adobe for approximately $800. Last night I downloaded the beta to check them out, as we have already built several of the less complex components internally.

While I was impressed with the work that I know must have gone into building these components, I was less impressed with the end result on a couple of fronts. First, most of the components I looked at did not appear to be very extensible, at least on the surface. For instance their Gauge component came in one color, BLACK...., with no obvious property to alter the primary color. Admittedly I did not do a deep dive in the API, and I could be missing something here. The second area of disappointment was the lack of a polished UI and set of skins. The components seemed to be highly functional, but lacked the polish we are all coming to expect of commercial level UI that can be seen in most premier RIA and commercial OS like Leopard and Vista. Putting that polish on the components does not seem like it would be a lot of work normally, but when you take into account the lake of available properties in the API it could be a bit challenging.

What I found really interesting is that just this morning I discovered an open source effort that builds one of the more complex components in Elixir. It appears Josh Tynjala has released his own TreeMap component. On the surface this component looks every bit as good as the one offered by ILOG AND it is open source.

This begs the question: "Will we see more third party components developed commercially or via the community?" Coming from more of a closed source mindset (Microsoft) where there was a proliferation of commercial third party developer tools it is hard for me to evaluate. My experiences over the past couple years working with Flex, the team at Adobe, and the growing community tells me that more and more developer tools and components will be created via the community. Part of the reasoning here is that it is a cultural issue, the second part is that AS3 and Flex just make it sooo easy to do.

There are other complexities to this question, as it brings into question the viability of third party developer markets for Flex, which I think will be very strategic for Adobe to successfully penetrate the larger developer market. Microsoft has been very good at this, but they did so with a completely different set of cultural values (closed source proprietary.) I will be following this closely as the it could have a significant impact on the direction we take our business.

Flex Community really taking off

2008-01-16T18:18:00.000-08:00

I fell in love with flex when I first started using it back in October of 2005 with the Flex 2.0 alpha. I loved it so much I started a business that used it is as its primary technology and based a big part of the business strategy on the assumption everyone would love it as much as I did and the developer community would grow quickly. Well I was partly right, everyone DID love it, especially the end-users in our target market (BI dashboard consumers.) What I didn't anticipate was that the DEMAND for flex talent would be so high that trying to find a flex developer was next to impossible.

The wrinkle for me as a business owner was that I built a platform that was designed to host Flex dashboards, but it would require flex developers to build client specific content for the platform in the form of flex dashboard applications. For a client to gain any value from our platform they would need content tailored to their unique needs (think of a typical intranet portal.) For our marquee (read Fortune 500) clients we would offer professional services to build their dashboards, but our whole goal was to sell recurring revenue via a SaaS business model, not via professional services. We soon ran into roadblocks in the sales process when we would tell clients that in order to extend, or add dashboards they simply had to just "hire a flex developer.." Well that was pretty much where the sales conversation ended, because we were asking them to find a resource, we ourselves couldn't even find (I ended up hiring good Java, and C++ developers and trained them on Flex myself.) The demand for Flex had far outstripped the supply. I was hearing rates of $250/hr + for Flex engineering work!

What I didn't anticipate, or said more accurately "think-through", is that there would be a lag between the demand generated for Flex and RIA, and the resources to fulfill the demand. Well things are getting slightly better. I see evidence (anecdotal) that the Flex developer community is really starting grow, with community sponsored events like Flex 360 we see a huge groundswell starting to manifest.

I also am starting to see the community filling needs in regards to the flex code base/framework faster than Adobe can with projects like Model Glue, Degrafa, and FlexMDI. Projects like this are really exciting to see, and have huge implications in regards to the efficacy of open source and the implications it has with business strategy when it comes to running a software company.
It seems now that the supply is starting to slowly catch up with the demand, and I realize one of the major accelerators that are filling the supply side of the equation is the open source community. This realization is partly responsible for me taking a serious look at how we can contribute to the effort because it appears to be a true win-win.

What would be needed in a Flex Dashboard Framework?

2008-01-13T14:36:00.001-08:00

Flex/AS3 does so many things well, and in my opinion is one of the most productive languages to create complex MVC type UI applications. That being said, some work in Flex I can find to be very tedious. While the class structures and OO all seem very elegant at the documentation/class level, the pragmatic use of some of the structures can create some odious and repetitive coding.

Take the charting for example. I can't tell you how many times I had to hunt around to figure out how to "adjust" the color/thickness/alpha for the horizontal and vertical axis, while not terribly complicated you either need to go down down about three levels of property chains in MXML or AS to get to the correct properties. When doing this for half a dozen charts in one application, it can get time consuming and verbose.

The other challenge we run into, is having to munge source data into a format appropriate for the charts to consume. Internally we have built some more hardwired models that allow us to point to a data structure and have it "transformed" into something the charts can read. I think more ideally we could extend the ICollectionView to include some transforms so that not only can we see sorted/filtered data as bindable properties, but also transformed (aka OLAP pivot/slice/query) data that multiple charts could consume.

Finally, the last main challenge we run into, which is not unique to dashboards, is architecting a proper event driven MVC. Dashboards tend to bring the limits of a non MVC approach to a head rather quickly (specifically highly interactive dashboards with drill up/down/across capabilities.) What first starts off as an easy "hard-coded" approach shortly winds up with piles of spaghetti strewn across several MXML components. Most MVC frameworks like Cairngorm are way to heavy though for simple dashboards. Frameworks (and I am probably using the term incorrectly here) like Model Glue seem more appropriate, but I suspect we could develop something even lighter and more tightly geared for building Flex based dashboards.

So as I see it the three primary areas to build a framework/convention/micro-architecture around would solve these three challenges.

1. Easy way to set styles/properties of objects without having to traverse deep object trees.
2. Easy way to load source data and transform it for charting.
3. A ultra-lightweight MVC framework geared for Dashboard UI challenges.

- Tom

Flash Player Update - Not so good

2008-01-12T08:36:00.000-08:00

I am not sure how many other flex developers have experienced this, but the latest rev's of the Flash player break some existing Flex generated content. I don't mean slightly annoying UI issues, the player downright stops the application from functioning.

The two issues I have found when running Flex content that has been generated by the latest version of Flex Builder 2 (2.0.1) inside a flash player version higher than 9.0.4xx (mostly the player targeted for AIR and Flex 3) are the following:

1. var myObj:SharedObject=new SharedObject();
2. myObject.function = function dummyFunc():void { //do something }

In the first case I suspect that statement was never valid in the first place, but we had code that was compiled in Flex 2 that had these statements and the compiler never complained. Now if we try to run these .swf's within the latest player the application stops working after burping an error telling us we can not instantiate a SharedObject (i believe it is a static class)

The second case is less obvious, but basically the new player will not allow anonymous NAMED functions. So the statement must read like the below.

myObject.function = function ():void { //do something }

I remember reading another blog post that pointed me to this.

Anyway, I am surprised Adobe is doing auto-updates with these types of changes and not making the developer community aware of the potential pitfalls.... The more Flex becomes prevalent on the web in public web sites the more things like this will become an issue.

- Tom

Open Source Dashboard Framwork for Flex

2008-01-10T19:13:00.000-08:00

After spending the last two years building Flex based dashboard technologies primarily for our VFX Dashboard Platform we are ready to start contributing something to the community. In the past couple years building flex based dashboards for our clients we have streamlined the process to go from requirements to final solution. We have iterated over design patterns and frameworks, each time surprised at our ignorance from the iteration before as we found yet another nugget in Flex we were not leveraging.

At this point I am interested in creating an open source dashboard framework for Flex that accelerates the development time to build a rich, complex dashboard bridged across many data sources and highly interconnected and interactive. My goal is to create a framework powerful and compact enough to allow a new Flex developer to build a compelling solution purely in MXML. I am not sure if that goal is attainable, but I think it is one worthy shooting for.

Flex 3 has introduced many new features which we can take advantage of, both in the charting classes and the new OLAP classes. One of the biggest struggles is the repetitive, but client specific code we have to write that takes source data and mungesit into an appropriate format that we can chart from. I am interested in spending some time with the OLAP classes to see if we have some easier ways of transforming source data into usable charting data.

Initially I will probably just release some sample code, and depnending on the audience it gets over time potentially creating an open source project. Right now I am in a pure Proof of Concept state on putting together an appropriate set of samples that would sit outside, but work tightly with some of our more proprietary VFX platform code.

- Tom

The Purist and their Ivory Towers

2006-02-16T15:08:00.000-08:00

There is a school of thought in the world of data visualization that I like to refer to as the "purist" approach. This outlook and view has been adopted by a few prominent individuals who specialize in data visualization, most of who are simply re-hashing or misinterpreting the work of Edward Tufte. The reason why I felt the need to comment on this phenomena, is that these practitioners speak with such an air of authority and arrogance that a neophyte reading their material would come away with a very skewed perspective on how to apply data visualization best practices in a pragmatic manner, specifically when it comes to designing effective business performance dashboards..

To help understand how to recognize this school of thought I have listed some of the more salient characteristics that can be seen over and over again when reading their materials.

The purist spends 80% of their time pointing out what is wrong in others work through diligent critiques while only showing brief examples of what they propose to be more optimized solutions.
The purist has a strong disdain for any aesthetic “decoration” or consideration within a dashboard design.
The purist favors “efficient’ but more obscure visualization metaphors that leave the typical information consumer confused.
The purist usually ignores the interactivity of the medium and usually reserve their critiques to static 2d analysis, as if the computer monitor where a piece of paper versus a device that the user can interact with (via the keyboard, mouse, etc..)
The purist is usually not a technologist, and as such offers advice that is more suited to float in the rarified air of academia than being grounded in the pragmatism of one who actually uses technology on a daily basis to accomplish business objectives.

Let’s take a closer look at some of these characteristics and how they truly undermine the purported goal of improving the practice of data visualization

Critique before Praise: I will give them this: the purist, if nothing else, is passionate. Unfortunately this passion seems to take the form of an almost evangelistic criticism of most software vendors and their dashboard visualization tools. In my experience when writing articles that are aimed at teaching best practices in data visualization, one of the easiest things to do is to point out bad examples of other peoples work. Unfortunately this approach does little to help readers quickly master the basics of good design; instead they are left with a catalogue of what not to do and no information on what should be done. The irony of this approach is that contrary to the visual “efficiency” the purist is trying to advocate, their literary approach is extremely inefficient as the big take away the reader has is all of the things that should NOT be done.

Aesthetics have no place in Data Visualization: While it is true that “sex sells”, and most dashboard vendors spend a lot of time drawing attention to their hyper realistic gauges and dials, it is not true that these “decorations” are ineffective. When properly used, aesthetics can play a huge rule in the effectiveness of a dashboard. Studies have shown that aesthetics play an important role in the perception, and ease of use. Dashboards have a unique set of requirements from a business and data visualization perspective, some of these requirements are quite a bit different from the requirements of doing data exploration or data analysis across large volumes of data. If a dashboard is being built to help monitor certain measures or operations that support a key business strategy or objective, I would think doing everything in your power to encourage end users to use and look at these measures would be a positive thing. The aesthetic appeal of the dashboard and the resulting emotional reaction to the visualization plays a large role in how easy users find the tool to use and how receptive they are in wanting to look deeper at what is being displayed. Using interactive and eye-catching features like realistic gauges, specular highlighting, and other visual cues helps to engage the end user and can also be used to emphasize the more important information within a dashboard. In some of my presentations and writing I talk about creating a visual hierarchy to your dashboard designs. This hierarchy is designed to reinforce the level of importance of each KPI or measure so that it reflects the importance of the business decisions that are being made from the particular metrics. Using these more engaging visualizations and contrasting with more subtle and flatter visualizations can be an ideal way to reinforce this visual hierarchy.

Efficient but Obscure: One thing I am struck with immediately when looking at some of the more esoteric visualization approaches advocated by some of the purists is that the meaning of the data being showed is not immediately obvious. While many of these visualizations are extremely ingenious and very efficient in terms how much information that can be imparted in a small amount of space, the problem is that the user ends up struggling to understand what is being shown. The user ends up spending a significant amount of attention reading through explanations, digesting the visual relationships, and learning the new graphical metaphor. Once the visual representation is understood and assimilated through repeated use we can began to let our natural pattern recognition processes within our brains help us to quickly digest the data. But this misses the point, that the goal is not to teach end users some new and novel way to visualize information, but rely upon the current mechanisms they are used to in terms of visualizing data that allow for that “instant recognition” when seeing data. When a user is forced to invest intellectual thought into digesting the meaning of the spatial and visual representation of the data – they are no longer leveraging their powerful and native pattern recognition capabilities. Not that these novel approaches to visualization don’t have their place in data analysis, they just don’t belong on a business dashboard. The trick to doing great dashboard visualization work is to capitalize upon the visualizations and metaphors that users are accustomed to (and thus recognize naturally) to impart important business information that allows the user to make more efficient and timely decisions.

Escaping Flatland:

"We envision information in order to reason about, communicate, document, and preserve that knowledge -- activities nearly always carried out on two-dimensional paper and computer screen. Escaping this flatland and enriching the density of data displays are the essential tasks of information design."
- Edward R. Tufte, Envisioning Information.

Tufte serves as the canonical figure most often quoted or referenced by the purists, and the sad irony is that the purists seem to miss some of his most fundamental points. This is no more obvious than in the critiques that show a screen shot of a particular vendor’s dashboard tool that is then dissected as if it were a static object. Very little (if any) analysis is given to the more dynamic and interactive features of these dashboards. When an designer capitalizes upon the more dynamic nature of the vendors toolset and is able to effectively leverage the technology to selectively drill, filter, magnify, and highlight relevant data, these screens come to life and provide a very rich and dynamic interface for the user to immerse themselves more cognitively with the information at hand.

The Ivory Tower: In the real world we have users that don’t have the time to understand new and novel ways of visualizing data, customers who are impressed by the “sex and sizzle” and make purchase decisions accordingly, software tools that cant do everything perfectly, data that doesn’t meet clean and pre-defined notions. Real world solution providers have experience creating businesses and successful solutions while dealing with all of these variables and more. The purists on the other hand climb up into their Ivory Towers (usually a self-promoting column they have secured in one of the plethora of industry trade publications) and pontificate upon the finer points of data visualization expecting business users to change the way they think to meet their strict views on how to best present information. If the purist truly wants to change the way business people perceive data, they would develop a product that does that, market it, sell it, and implement it. Or at the very minimum, provide pragmatic and real world examples of how to do that with common software found in today’s market place. But then again that might actually force the purist to climb down from their tower and get their hands dirty in the work of solving real world problems with real world technology.

A Picture is worth 3 billion words

2005-11-14T21:38:00.000-08:00

One of the phenomenal aspects of the human brain is its ability to behave like a massively parallel set of processors, being able to simultaneously digest multiple streams of information at speeds that would stymie some of the most advanced super computers.

While traditional x86 computer processors can only process one instruction set at a time, albeit at amazing speeds, they can not process information in parallel. While a human might be able to make 1 or 2 mathematical calculations per second, and a computer is capable of billions, a human can still instantly recognize a smile, while a computer can not. All of the computers power comes from its ability to do calculations in serial, one after another, at amazing speeds; at its core all a computer is doing is performing simple Boolean equations between bits that either have a 1 or 0 value. It is just doing billions of them in fractions of a second.

But, when you apply the computers incredible serial processing capabilities to very complex mathematical problems (like using multivariate analysis to discern the meaning of a facial expression) you hit a brick wall very quickly. Using a brute force linear approach to solve these problems will bring some of the most powerful computers to its knees. So if you ask a computer to search gigabytes of text data for the word “flower” it will do an amazingly quick job of finding it (relatively easy mathematical calculations using powerful sort algorithms), but if you scan a picture of your front yard in the spring time and ask the computer to identify a blooming flower, it will have a much more difficult time.

Humans have an amazing ability to process certain types of information in a more instantaneous way, this is most evident in the way we process visual information. A very real example is the way we can analyze numbers represented as visual patterns exponentially quicker than we can analyze a set of numbers in their native form. If you were to look at a spreadsheet filled with hundreds of weekly sales numbers across product lines, versus looking at a simple bar chart with the same data, which would you prefer to determine which products sold more than others?

Lets look at this example more closely. When we look at the bar chart we can “instantly” see each bar in relationship to its neighbor and “instantly” see which ones are higher than others. This analysis and understanding happen in an instant, it isn’t even a conscious thought. It is the same as you identifying what color something is, you simply know it. Conversely, if we were looking at the numbers in a spreadsheet we need to do many more mental gymnastics. First we need to read the number which is a visual pattern in itself, and a combination of visual symbols usually consisting of one or more digits like: “123,456.” We then have to consciously make a mental comparison of each number against the others. This is a conscious and thus linear process, in which we are much slower than a computer.

So if you look at a visual representation of data you are able to process information almost instantly, but if you need to look at a raw data itself, you will be forced to consciously digest it in a linear fashion.

One of the fundamental areas we focus on in our information visualization work at BrightPoint Consulting, is to figure out ways to leverage humans naturally given pattern processing engines. To the degree we can present relevant information in visual patterns that users can take advantage of, we succeed in creating a more effective and efficient interface between the user and the data they interact with.

In the face of the exponential growth of data and information that 21st century humans need to deal with on a daily basis, technologies that allow people to more effectively process that information is not only a great opportunity for new business, but it is something that helps society as a whole. We are relieved of the tedium of repetitive information processing tasks and freed up to focus on higher level functional tasks that result from the understanding of the underlying patterns.

There Has to be a Better Way.

2005-10-14T17:55:00.000-07:00

Problem Domain:
During the course of the day I probably work with and open 75-100 files, all organized into the dozens of client folders that I have. Each client might have mulitple projects etc, so repeatedly getting to files requires this endless drilling through the directory structures. I find this task repetitive, tedious, and annoying. Each time I search for a file I have to consciously read each directory label since they each look the same visually, once a directory unfolds I then have to repeat the process. I estimate that I spend probably about 4-6 seconds to find a file on average. I probably repeat this task 100 times a day. The cost to me is anywhere between 400-600 seconds a day... multipy that out over the year (assuming 250 work days) and you are talking conservatively over 25 hours of my year spent searching for and opening files !!!! Now I am not sure what your time is worth to you, but I certainly have better things to do than spend what could amount to a 3 day vacation opening and closing files.

A Potential Solution:
During the course of the day, there are some files I use over and over again, whether it is particular to a client or project I am working on, or a certain technology or package. Most of these are located in the same proximity to each other on the file system as I organize my files by related subject areas.

What if the File Explorer (Windows) remembered which files I openend most recently, and how often I opened them? It could remember this over the course of minutes, hours, days, weeks. Now, if the File Explorer knew what I had openend most recently, and how many times I opened it, it could use some deductive logic and mark these directories and files for me visually. What if the directory or file was a different color/size/shape based on how recently I had been there, or how many times I opened it? This would allow me to use my naturally given parallel processor (visual cortex) to instantly see these patterns and guide me to the most likely navigation path without me to have to constantly read directory and file names. Even if I just shaved 2 seconds off my time, that would amount to 100 hours in my pocket :)

I have done googles across the web to try and locate such a tool, because to me it would seem that someone smarter and more adept would probably have already built one. But alas, I could find none. This is a product waiting to happen!

Macromedia/Adobe versus Microsoft

2005-10-10T19:24:00.000-07:00

My software development experience dates back over 20 years, but in the past 8 years I have worked almost exclusively on the Microsoft platform. One thing that I thought Microsoft did that was absolutely ingenious (and viral) was to practically spoon feed its development community. With their MSDN subscriptions and robust API documentation you could practically teach yourself to do anything with their technology. They also made it a primary focus to improve their development environments to make things as easy as possible. This is not to say their technologies are not without their problems and flaws, only that compared to any other platform I have tried, MS really got it right about catering to the development community.

Lately I have been exploring new technologies to use for data visualization applications and have been looking into the wave of technology around Rich Internert Applications. After setting up OpenLazlso, Microsoft Atlas, and Macromedia Flex in a lab, It started to dawn on me that Microsoft has a good chance of loosing the desktop completely.

It appears that Microsoft is betting the technological farm on Vista. All of the new and interesting desktop development efforts all will require Vista to support them, specfically Avalon or the Win FX platform. The challenge I have as business person and software architecture with that approach is that it would require me to develop applications around a platform that does not exist, and would require a user or company to upgrade their whole OS to take advantage of those new and importan features. In the best case scenario, if Vista ships on time, I could realistically expect only 50% of my target market to actually have it installed in about 3 years from now. That is quite a gamble for developing cutting edge client apps today.

Then comes along Flash. As a programmer at heart, I always considered flash and action script to be a bit of a toy versus a real development platform, but you can't argue with some of the incredible things people are doing with flash. After a couple of aborted attempts to trying to program within timelines and layers I pretty much set it aside as I found the programming paradigm to cumbersome and akward to be productive in. But, I kept on getting drawn back to the fact that I was seeing amazing things being done. Specifically, amazing visual things, that could really open doors from a data visualization perspective.

Along comes Zorn, Macromedia's merging of the open source Eclipse IDE with Flex, their flash based development platform. Now were talking.... When I found out the Macromedia hired away one of the guys who started and ran the Microsoft .NET product team, I really started to get interested. When I started poking around and realized that Flash has a 98% install base, across ALL types of devices (desktops, mobiles, even cameras) it really got my attention. Then when Macormedia announced last week they were dropping the price point of Flex from the enterprise level down to a price that pretty much any developer could afford AND they were going to release alpha bits to engage the developer community I realized they were going for the jugular at Microsoft. If Macromedia succeeds in winning over the development community they have a very strong chance of obviating the desktop, or at least the Windows one.

The new flash player 8.5 that is planned on being released by Q2 2006, has a complete virtual machine encoded into it, allowing run-time debugging, exception handling, and support for an ECMA standard OO language. While a majority of the people who run flash are running off a windows desktop, it does not require one, and since the flash player footprint is so small the deployment concerns usually faced when you pair up your application with a development platform are pretty much mitigated. Try leveraging MS .Net with a desktop or server app, and you basically force your user into a 30MB download of the .net framework, if they don't already have it installed, or if they don't have the correct version.

All in all, I will be very interested to see how much traction Macromedia can get with the developer community around Flex and Flash, because if they succeed and are able to leverage their amazing market penetration with their Flash player, it could really change the way we approach client application development, and the way users view their desktop.

Welcome

2005-10-10T19:12:00.000-07:00

My name is Tom Gonzalez and I am the managing director of a software development firm that specializes in a very small niche of the business intelligence community, dashboards and data visualization. In the course of delivering projects for clients and creating unique software products, I spend a major portion of my time working with new technologies and trying to predict what turns the industry will take and how different technologies will fit together.

I have been skeptically sitting on the sidelines for the last several months, pondering the benefits of blogging. After reading the article in Business Week a few months back I realized that this medium is here to stay. What has prompted me to start my own blog, is that over the past several months I have gotten some very valuable information and insight from reading other blogs.

I have a background in computer science and visual arts and one of my primary interests is to create technologies that make life easier for people and improve the aesthetics of their environment in a way that brings them more pleasure and satisfaction with their surroundings.