cue circus music

Executive Summary

Bigtop is a language for describing database backed web applications and a code generator to make web apps from those descriptions.

Bigtop allows for generation of...

It is quite different from code generators provided by web app frameworks. What distinguishes it is...

Background

I'm about to tell you a story. It's the story of invention. As I am the inventor I can't tell whether the invention is the best thing since sliced bread or a candidate for the most outrageous category on the American Inventor TV show. You'll have to decide whether I've made a better bread slicer or a Perfect Pet Petter. (For those who didn't see it: The Perfect Pet Petter has a motion sensor, so it knows when a dog is ready for petting, and a mechanical hand to do the petting. Very scary.)

It was Fred Brooks who told us that revealing our table structure would make the workings of our apps obvious. Some may think that the wide adoption of object-oriented programming and the advent of web delivered apps obviates Mr. Brooks assertion. I don't.

I had been out of the data driven app development business for quite a while, when I took my current job a little more than a year ago. After coming up to speed on mod_perl and Postgres, I discovered that our apps are strikingly similar to the academic records apps I worked on during my first job, as a registrar's office programmer for a small college, twenty years ago.

While the apps look nicer and generally run faster, their basic ideas have not changed. They are still firmly centered around their data models.

As I worked on the tedious bits of a new app last year I found myself violating the first principle of coding I learned in college: don't duplicate code. (Since I used to teach college Math and Computer Science, I don't use the term Don't Repeat Yourself or DRY. Just try to imagine a college teacher who never repeats himself.) Many others had already discovered and named the bit of code I was writing over and over. It was CRUD (Create, Retrieve, Update, and Delete).

There are many web frameworks. My guess is most of them started to avoid hand coding CRUD. They must have all started on the premise of duplication reduction.

Since we were using a framework, yet were forced to repeat ourselves, we began to think that a major revision of the framework was in order. This led to (Gantry), which kept the best bits of our shop's original framework while eliminating yet more duplication.

At the risk of shameless self promotion -- since I wrote the relevant parts -- I'll assert that Gantry has a pair of nice CRUD schemes. The first, called AutoCRUD, does what you want for the 60-80% of your tables which are dull. The second, called CRUD, takes care of the others, with a bit more help from you.

Having test driven the CRUD schemes in Gantry, I wanted more. I didn't want to have to write an SQL schema file describing the table structure, then go back and write object relational mapping code (for Class::DBI or something similar), and still have to return a third time, to the same data model, to describe its appearance for an on-screen form.

I wanted a single place to say everything that needed saying about the data. That place is now a bigtop file. In it you can fully describe your data model and its controllers. A quick generation step builds the app. Further, the code it builds is always separated into two pieces. One piece is for you to revise and expand, to do the things that make your app different from others. The other piece is generated and safely regenerated to allow for changes to the data model.

Bigtop already supports Gantry with a choice of three database engines and three object relational mappers. But it allows for backend plugins to generate whatever is needed for your app. Backends can easily add keywords to the grammar and are free to generate whatever they like. So it is possible for Bigtop to support other frameworks. Since I'm rather committed to Gantry, it probably won't support others unless someone else writes the backends.

Example

Here I will show the smallest app I can think of (that does anything useful) with bigtop. It manages one table which stores names and phone numbers. I built this app with the tentmaker, a browser delivered bigtop editor. (See the following section for more info on tentmaker.) I did beautify it slightly for readability.

If you want to understand better what the bigtop description means, see Bigtop::Docs::Tutorial. I won't show details here.
config { engine CGI; template_engine TT; Init Std { } CGI Gantry { with_server 1; } Control Gantry { } SQL Postgres { } Model GantryCDBI { } SiteLook GantryDefault { gantry_wrapper `/home/pcrow/srcgantry/root/sample_wrapper.tt`; } } app Contact { config { dbconn `dbi:Pg:dbname=contact` => no_accessor; dbuser some_user => no_accessor; dbpass `$ecret` => no_accessor; template_wrapper `genwrapper.tt` => no_accessor; root `/home/pcrow/Contact/html:/home/pcrow/gan/root` => no_accessor; } authors `Phil Crow` => `philcrow2000@yahoo.com`; sequence contact_seq {} table contact { sequence contact_seq; field id { is int4, primary_key, auto; } field name { is varchar; label Name; html_form_type text; } field phone { is varchar; label Phone; html_form_type text; } } controller Contacts is AutoCRUD { rel_location contacts; controls_table contact; text_description Contact; method do_main is main_listing { cols name, phone; header_options Add; row_options Edit, Delete; title `Contact Management`; } method form is AutoCRUD_form { all_fields_but id; } } }

Building and Running

With the bigtop file in hand (I'll call it contact.bigtop), you can be running your app in a few steps:
bigtop --create contact.bigtop all cd Contact createdb contact -U postgres psql contact -U regular_user < docs/schema.postgres ./app.server [ port_defaults_to_8080 ]
(Supply database passwords as needed.)

You are ready to use the app through a browser.

You may need to update the data model, even after you've added code to Contact/Contacts.pm. This is a completely safe operation (but source code control is always a good idea).

Simply edit the bigtop file, in the docs subdirectory of the working directory, (with your favorite editor or tentmaker). Save the result and rebuild:
bigtop docs/contact.bigtop all
Note that you may want to turn off generation of things which shouldn't change. Do this by adding no_gen statements (or checking no_gen boxes in tentmaker):
Init Std { no_gen 1; }
Turning off Init generation is particularly common, since failure to do so invites bigtop to overwrite README, Changes, etc.

You can turn off generation for any backend in this way. Init Std also allows you to control regeneration of individual files, so README can be safe from overwriting while the MANIFEST is kept up to date. Further, you can include no_gen statements at the table and controller levels too (as well as for the whole app, a feature for the truly paranoid).

Making Tents

Even as the bigtop language has grown, the basic structure remains simple. But that simplicity of structure is often lost in the complexity of keyword proliferation. So, tentmaker comes to the rescue.

The structure of bigtop can be described briefly:
There are two top level blocks: config and app. In the config block there are statements and backend blocks. In the app block there are sequence, table, config, and controller blocks (plus a set of literal statements). All blocks allow statements; in addition, table blocks have field blocks, while controller blocks have methods. Finally, controller blocks also have literal statements.
But this structure still leaves a little to be desired. As note above, the problem is the volume of useful keywords. While each has its place and use, remembering them is hard.

After much thought and several abortive attempts at helping you write bigtop files, I began tentmaker. It allows you to edit your bigtop files with a browser (as long as your browser is DOM compliant, think Firefox not IE).

If you've install bigtop and the tentmaker templates that come with it, you can start tentmaker easily enough:

tentmaker [--port=8081]
Then point your browser to the port you chose. tentmaker looks like a complex wizard, but is a reusable editor of bigtop files. It can do everything except remember comments and preserve all whitespace. (Well there is also one statement it can't handle, but you don't need it).

Learn More

To start using Bigtop see Bigtop::Docs::TentTut or Bigtop::Docs::Tutorial.

In reply to Introducing the Bigtop Web App Language by philcrow

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.