BrowserUk has asked for the wisdom of the Perl Monks concerning the following question:
I have a set of 4D data (HSV + frequency) that I wish to visualise, but I'm stuck for how to display it and am looking for ideas.
HSVs (pixel colors) can be mapped to points within a cone or cylinder, but how to represent the frequency at that point?
What I seeking to be able to see is: a) how the colors cluster (or not); b) to pick out the color with the highest frequency within each cluster.
I swear I've seen an image analysis dialog that did this exact thing at some time in the past, but I cannot find anything that does it.
I'm not looking for code, I'm happy to write that myself once I come up with a method of doing so.
Thoughts, pointers, references?
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re: [OT] Displaying 4D data in a 2D image.
by choroba (Cardinal) on Dec 09, 2011 at 14:02 UTC | |
| [reply] |
|
Re: [OT] Displaying 4D data in a 2D image.
by zentara (Cardinal) on Dec 09, 2011 at 14:25 UTC | |
The PDL::Graphics engine may also have some way to display that type of data as colored flux density plots. I'm not really a human, but I play one on earth. Old Perl Programmer Haiku ................... flash japh | [reply] |
|
Re: [OT] Displaying 4D data ... (Quick SVG Sketch)
by luis.roca (Deacon) on Dec 09, 2011 at 15:58 UTC | |
BrowserUK, Note: SVG output using Illustrator so please forgive the bloated markup :) Read more... (68 kB)
If this isn't quite it, let me know, I have some time this afternoon for a quick edit or two. Hope this helps.
"...the adversities born of well-placed thoughts should be considered mercies rather than misfortunes." — Don Quixote
| [reply] [d/l] |
by BrowserUk (Patriarch) on Dec 10, 2011 at 03:13 UTC | |
Luis, many thankyous for your time. That is certainly a pretty picture :) I'm not yet sure that I understand what it displays, but I am very interested in how you produced it? What is Illustrator? It took me a while to work out that I could view it directly in my browser. (Aside: Would be nice if we could embed such things in our PM posts -- though I can feel the resistance to that suggestion building even before I hit 'create' :) What might make my understanding of the representation easier is if you could also post the basic data points that the image represents? | [reply] |
by luis.roca (Deacon) on Dec 10, 2011 at 05:06 UTC | |
Sorry about not clarifying how to view the image. I used Adobe Illustrator CS5 but you could get away with Inkscape or maybe even using Cairo then exporting an SVG file. I just read your longer reply and would like to take a day to think about it if that's alright. Let's ignore the sketch I made for now. :-)
"...the adversities born of well-placed thoughts should be considered mercies rather than misfortunes." — Don Quixote
| [reply] |
by luis.roca (Deacon) on Jan 04, 2012 at 02:53 UTC | |
I suppose better late than never. You've probably solved this and moved on but I increasingly feel a good candidate to represent a large volume of data like this would be a heatmap. I'm pretty certain that through color value and saturation you could abstractly accommodate data volume and frequency. In addition to the heatmap examples (starting with the fifth one down) on the following page there are number of well executed 2 dimensional graphs.
"...the adversities born of well-placed thoughts should be considered mercies rather than misfortunes." — Don Quixote
| [reply] |
by BrowserUk (Patriarch) on Jan 04, 2012 at 06:38 UTC | |
|
Re: [OT] Displaying 4D data in a 2D image.
by jdporter (Paladin) on Dec 09, 2011 at 14:34 UTC | |
You could make a 3D version of a bubble chart. Using a good graphics library like OpenGL, your bubbles could really look like bubbles, with transparency and reflection and whatnot. That would be cool. | [reply] |
|
Re: [OT] Displaying 4D data in a 2D image. (Reply to all respondants. (Thank you!) :)
by BrowserUk (Patriarch) on Dec 10, 2011 at 03:03 UTC | |
Okay. I guess I should have supplied some data and some explanation of what I've tried to date. (I didn't want to influence responses or cloud the underlying question with specifics of how and with what tools to do the actual drawing.) Here is an illustrative, though rather small set of sample data. The keys are HSV (all scaled 0 .. 1 rather than H being 0 .. 360 ). The values are the frequencies of pixels with that HSV value within the image being analysed:
And here is a 2D/3D representation of that dataset. The H,S,V -> X,Y,Z in a 256x256x256 'cube'. The color of the plotted points is the pixel color. As you can see, the HSV space discriminates the points into two very clear clusters when viewed this way. However, if viewed from different angles, it is possible to discern 5 or even eight clusterings with a few outliers. If you look at the frequency data, it is possible to also find two very obvious peaks; and 4 or 5 or 7 more, depending where you apply the cutoff. The challenge is to try and a) find that cut-off; b) partition the dataset around those peaks. I've made no attempt in this image to plot the frequency. The problem with trying to represent the frequency is the range of the values -- from 1 up to 8.7 million. It could be much bigger for larger images. In the past I've been lucky enough to have access to high-end proprietary software than would allow you to rotate a 3D plots on screen with the mouse or keyboard in real-time, but I don't know of any free tool that allows this. Nor is there any graphical toolkit (for Perl or anything else), that I'm aware of that would allow me to 'knock up' such an application quickly and easily. The point here is that this visualisation is not the underlying problem I exploring. Just a step along the way to trying to get to grips with understanding a dataset. Whilst the particular sample image appears -- yet to be confirmed -- to have its pixels clustered both by frequency and HSV into a small number of distinct groups; it may be that when I apply this process to other relevant images, no such clustering occurs. So the goal is not to develop an all singing and dancing 4D data visualisation tool. It is to find a way to visualise a few example sets of data, in a few different ways, to see if there is anything there worth exploring. Update: Here is another view. With the points plotted as circles using log2( freq ), and the image rotated so that V->X, S->Y, H->Z. It is interesting because it highlights the presence of more than two groups when viewed from this angle. It is also disappointing because whilst it is easy enough to pick out the bigger sploges, could anyone pick out the 8.7 million splog versus the 1.5 million? Or even the 600,000? With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply] [d/l] |
by Anonymous Monk on Dec 10, 2011 at 04:53 UTC | |
Though this pure Tk.yellowstone.earthquake.gnuplot2.pl doesn't do rotation Gnuplot allows you to rotate it, see Earthquakes in three dimensions I'm not sure, but I think I also saw a Prima demo do 3d-ish rotation | [reply] |
by BrowserUk (Patriarch) on Dec 10, 2011 at 09:44 UTC | |
Gnuplot is the tool I need. Just a shame it is so badly documented. (And that the Win32 port has limitations :( ) The rotating 3D earthquakes demo is ... um .. not well programmed. There is no need to reload the data for each rotation. That's what makes it so flickery. Here's my sample data rotating in X & Y using refresh:
Just shame I can't get it to use variable sized circles instead of crosses? With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply] [d/l] |
by Anonymous Monk on Dec 10, 2011 at 10:14 UTC | |
by BrowserUk (Patriarch) on Dec 10, 2011 at 10:21 UTC | |
by RichardK (Parson) on Dec 10, 2011 at 14:43 UTC | |
I've made simple 3d plots of your data mapping H,V,F and H,S,F and they show interesting clustering. I think reducing the number of points will help, I created a grid 100 x 100 and summed all the frequencies of points within each cell. | [reply] |
|
Re: [OT] Displaying 4D data in a 2D image.
by Taulmarill (Deacon) on Dec 09, 2011 at 13:58 UTC | |
Consider, that you not only can differentiate data points by there position, but you can also alter size, color and maybe even shape of single points based upon certain values. | [reply] |
|
Re: [OT] Displaying 4D data in a 2D image.
by RichardK (Parson) on Dec 09, 2011 at 14:04 UTC | |
I've never seen anything like that, but the only thing I can come up with is to draw a series of 2D frequency contour maps each for a given hue and stack them in some way. or scroll through them like rolodex cards, if you want to get fancy ;) | [reply] |
|
Re: [OT] Displaying 4D data in a 2D image.
by Khen1950fx (Canon) on Dec 09, 2011 at 23:05 UTC | |
Cool question! I'd use voxels, or more specifically---doxels, to get the job done. Graphics::VTK has some support for voxels. It requires The Visualization ToolKit. And for some quick examples, there's the Voxlap Engine and download here. See also: Wikipedia: Voxel.Visualization Lab: Fundamentals of Voxelization. | [reply] |
|
Re: [OT] Displaying 4D data in a 2D image.
by cavac (Prior) on Dec 10, 2011 at 01:32 UTC | |
I'm not sure i completly follow. But i saw something like that realized as some kind of LED based audio visualizer: YouTube Video
Don't use '#ff0000': use Acme::AutoColor; my $redcolor = RED(); All colors subject to change without notice. | [reply] |
|
Re: [OT] Displaying 4D data in a 2D image.
by TJPride (Pilgrim) on Dec 09, 2011 at 19:00 UTC | |
http://www.dmoz.org/Science/Math/Software/Graphing/ | [reply] |
|
Re: [OT] Displaying 4D data in a 2D image.
by JavaFan (Canon) on Dec 09, 2011 at 22:20 UTC | |
how to represent the frequency at that pointNetscape gave a solution for that in the 1990s: <BLINK>. There's got to be at least some useragent that has a CSS property that allows to set the frequency of the blinking. A more serious note, one way to visualize 6D data in 2D is to use coloured disks: 3 dimensions for its colour, 2 dimensions for the position of its center, and using the radius gives you a sixth. | [reply] [d/l] |
by choroba (Cardinal) on Dec 10, 2011 at 00:35 UTC | |
| [reply] |