Actual FormLis Powered Plots

I’ve been busy integrating charts into FormLis. Heres how they look together. Isn’t that lovely?

You generate charts from view data. In this case I used a food intake tracker that is helping people make sure they’re getting their recommended daily amounts. Here is the
relevant data — I’ve added the highlighting, column numbers and outlines — this is what the chart is seeing.
And here is the magic tying them together:
PLOT bar chart ;
ON “/*/” leaf-totals ;
X p# 0 case “Protein” ; then
     1 case “Grains” ; then
     2 case “Vegetables” ; then
     3 case “Fruits” ; then
     4 case “Milk” ; then
     5 case “Fats” ; then ;
Y 5 p# + column ;

Plots describe how views are transformed into (X & Y) pairs. ON “/*/” leaf-totals ;  slices the data up into multiple plots. In this case 1 plot per  top level group (e.g. Tom, Peggy, etc). Its the pink boxes in the view above.  I’m using the leaf-totals as data because they contain my daily sum (I’ve highlighted them in white). Leaf-totals means the same as the sub-sub-sub-sub-totals. I could have chosen to use the rows or the overall totals.

I want to restrict the plot to just todays data, which is the first row in each group, so I say LIMIT 1 ;.

Each of these rows will produce several (X &Y) pairs. For example Tom’s row will produce (“Protein” & 12), (“Grains” & 6), (“Vegatables” & 2.5) etc. So I say POINTS-PER-ROW 6 ;

Lastly, I have to tell FormLis how to compute the X & Y values. That’s what the X and Y functions above do. X says “take the point number (p#), if its zero -> Protein, if its 1 -> Grains, etc”. Y says take the point number, add 5, and use that column’s data.

Chart rendering is now (pretty much) in the bag.

I've got charts pretty much licked. The following image shows line charts, a bar chart, and a cumulative line chart.  I've still got to make scatter plot charts and cumulative bar charts.  Then I need to pick some better default colors — I dropped the sophisticated color AI I mentioned earlier because it had a tendency to pick a lovely shade of puke green and some complementary colors.    Now I just use preset colors that can be overridden.

The colors I'm using (blue, red, green) are okay, but I'll try to create a default set that is both lovely, sufficiently contrasted, and as usable by the visual impaired as can be without giving up too much prettyness. 

Also I dropped the object orientation.  As my design took shape I found the OO approach wasn't pulling its weight and was making me design an API that was good for subclasses rather than an API that was good for making charts.

Progress on bar, line and scatter plots.

With pie charts well in hand, I still have to do bar, line and scatter plots.  All three need a grid to draw over, so that’s what I’ve been working on.   The system is able to determine a nice range for the data, then appropriately size the labels.



Surprisingly, these charts are the object oriented; the only thing in my entire system that is.  Without getting controversial, I find objects usually makes problems more complicated and they make code more difficult to change.   In this case, my objects describe the plots and have routines to draw portions of it (i.e. draw titles, draw axis, draw data).  At a high level the plots are more same then different,  so it works out pretty well.

One Day, FormLis will build lovely charts. Pie charts, bar charts, line charts. Here is a sample.

FormLis makes data collection and reporting easy. One day (months away?)  I’ll add charting.  I’ve already got most of the chart description language planned out; so I took the time to make some lisp-assisted pie chart shapes:




This took work.  First I researched a bunch of pi charts to determine what I thought looked best: simple, round, and solid color. Then I needed super accurate math to compute the wedges, if they didn’t line up perfectly, little slivers of white would ruin the asthetic.   I ended up using fixed point math (and defining PI as exactly 4096), and I had to write my own trig functions.


That was the easy part.  The hard part is color.  My plan is to let users specify colors (blue, red, or via RGB triplets) if they want.  Regardless of what they do FormLis will have to select complimentary colors.


However, its not obvious how you code that.  My plan would bore you, but the trick is to operate in a color space where choosing complementary colors is easy.  In my case I select colors from the LAB color space and then map them to RGB.  In LAB, I can control brightness and saturation while picking colors, while in RGB all I could do would be to hold red and blue intensity while selecting green intensity.  The conversion is very math heavy though, I cargo-culted it from Bruce Lindbloom, who has the best resource on the mathematics I’ve ever seen.

Throwing Students out of Work

When I was a student at the University of Calgary, I earned money over summers doing IT work at a well-known facility management company. They loved safety so I built a safety data tracking application. For their travelers, I built a travel expense tracking application. When they got the maintenance contracts for a pair of hospitals, I wrote the work tracker that kept their on-site workers up-to-speed and so on. I worked over three summers and was payed $30 an hour, which was good value for them and me. 

When I finished university I decided to do a startup and took my dad’s advice to commercialize some of my old work. But, I realized that data trackers to offices are like artwork to homes — everyone needs a little to tie things together. So I stopped drawing and built a camera. The result (FormLis) makes it easy to do the sort of things I did — for example, I rebuilt all three summers of work in about an hour.

Before, companies hired ‘painters’ (consultants) and payed tens of thousands for just the right portrait. Or they could try to find a ‘print’ (pre-built database application) that was close enough. Now they have a third option in FormLis; it’s like the point and shoot digital camera of database applications.

Just like the digital camera stole the best gigs (e.g. portraiture) from artists, so too will FormLis steal the work of young programmers. I am literally throwing students out of work. I just hope, as a society, we can keep them off the streets. Nobody likes drunk unshaven nerds.


Image Credit

Walkerby. by Flickr User bugBBQ

Parsing CSV files with Lisp.

In a previous blog post I talked about the parsing method I use here. It’s a hand made state machine, which (if you do it right) can be remarkably short. This parser parses CSV files. I haven’t widely tested it, but it should be compatible with excel & gmail exported csv files. CSV isn’t standard but generally:

  • Use Commas as the deliminator.
  • Any field that has a comma or a newline is treated as complex.
  • Complex values are wrapped in quotes.
  • Quotes in complex values are represented as double quotes.

This parser shoud support any line ending (cr+lf, lf or cr) and handles regular and complex values. I’ve heard of CSV files using C style escapes (like \n) but I’ve never seen one, and this this parser doesn’t support that.

Read the rest of this entry »

How can you use a database without SQL?

I’ve talked a bit about databases, and it’s an open secret that I’m doing some unusual stuff with mine. Here is an example of how I approached a problem that is familiar to anyone who has worked with databases.

Groups: in typical DB parlance you say there’s a many to many relationship between groups and users.

The big idea is to answer these questions:

  1. What groups is user X enrolled in?
  2. What users & groups are enrolled in group Y?

Read the rest of this entry »


Get every new post delivered to your Inbox.

Join 34 other followers