Tests are useful for verifying that the system works as specified, but are also useful for future development. Having to write tests generally motivates developers to create a clean API for their modules. And a comprehensive test suite gives more confidence, that when one does make changes, the original functionality doesn't break.

If the test suite is written in the context of a test-driven development process (i.e., tests first, then code), even better.

Update: corrected a grammar mistake.

I like this idea quite a bit, but one interesting question is who should write them? You could ask the software company to write tests for you as well if you're busy, but I think it would be a very interesting if you wrote them yourself. That way you have a very simple, quantifiable way of determining that they are fulfilling the terms of their contract. On the other hand, if you're busy enough to contract out programming work, you may too busy to write tests. Writing your own tests involves might involve you in the development process more than you (and possibly the software company) want.

Now my question: How can I assure that the system written is well-done? I am assuming it should be OO, with clean documentation and sensible classes, a documented and thought-out database structure as well as abstraction where possible. These are things I can observe. What else could/should I check for?

You can do a lot by explicitly defining the object structure and APIs that you want. If you don't want to be this involved in the design, you could just look at the code as it is returned to you and give the development team feedback. AFAIK there is no automated way of determining code "quality," so I think you'll have to just describe your desire for good, modular code and see how well the company can do with it. I'm assuming that you've already picked your software company of choice, if not you should definitely attempt to shop around and get code samples from a variety of places.

Thanks for the suggestion. Could you give me a pointer towards resources regarding unit testing in Perl?

I am currently thinking of having one company write the code and a freelancer write the tests. This might lead to some conflict (pointing fingers), but it should provide me with quite good results.

Why assume that it should be OO? The world is not object oriented.

In your place, I'd do a short code review as a "quality check", to try to get a feel for how easy the code is to understand. I'd look for clean interfaces, short functions, and good identifiers. I'd ask myself: "If I had to fix a show-stopping bug, how long would it take for me to get familiar with this piece of code?"

-sam

• Make sure there aren't a lot of anonymous functions that are hard to reuse.
• Look for clearly defined function libraries with small functions that take arguments related to the calculations the function will make and returns a value that's related to the functions name.
• If it's a web-based project, make sure the HTML and perl code isn't mingled much, if at all.
• Make sure any part of code that is potentially confusing to someone who isn't necessarily a perl guru is documented.
• A decent gauge of a good perl programmer is one who uses a lot of CPAN modules for core functionality.
• The program, if large, should be split into appropriately named modules or classes (which should both be stored in simple reusable .pm files).
• Anything the programmer does to make the code more readable and more resuable is A Good Thing and is a testament to the coder.
• Of course, if there is a huge time crunch, some of the things mentioned above won't be done or won't be done correctly. However, a good programmer *always* shows effort to make sure code is elegant and reusable.

Make sure there aren't a lot of anonymous functions that are hard to reuse.

I use a lot of anonymous functions, and that doesn't make it hard to reuse them. On the contrary, I do it to easier reuse code. I use it as an local abstraction technique, and we agree on that abstraction is good, while I at the same time don't want to pollute the global namespace for a function that would only be confusing there since it only has value where it is. In the future, if that piece of code has value outside the lexical scope it's defined in, it's easy to move it out. Sometimes these functions are closures, sometimes they're not.

Using anonymous functions is a good sign for me, if done right. I see it as the programmer appreciates abstractions and tight scoping.

ihb

Read argumentation in its context!

Make sure there aren't a lot of anonymous functions that are hard to reuse.

I use a lot of anonymous functions, and that doesn't make it hard to reuse them. On the contrary, I do it to easier reuse code. I use it as an local abstraction technique, and we agree on that abstraction is good, while I at the same time don't want to pollute the global namespace for a function that would only be confusing there since it only has value where it is. In the future, if that piece of code has value outside the lexical scope it's defined in, it's easy to move it out. Sometimes these functions are closures, sometimes they're not.

Edit: I'm not arguing with ihb, I'm adding to the above argument.

See also Is it possible to create a sub exclusive to a sub?.

Further, function generators often allow a level of abstraction that "ordinary" programming techniques don't: the ability to build code from data, not just operate on data. Look at Specializing Functions with Currying for a simple example of how higher-order functions can promote reuse and remove redundant code.

I agree with ihb that anonymous functions (well, let's say "lots of coderefs") are a good sign, not a bad one. I'm sure there are exceptions, but functional abstraction isn't the kind of thing I can easily imagine a mediocre programmer doing.

--
F o x t r o t U n i f o r m
Found a typo in this node? /msg me
% man 3 strfry

http://search.cpan.org/src/SOFTDIA/Docs-US_DOD-STD2167A-0.06

I found out in my experience, that if you are going to use lots of CPAN modules, you'd better encapsulate their functionality in your classes, at least for the most critical.

In this way, when some CPAN class changes the way it works over time, you are not going to be hurt for that, and you can always adapt (and improve) your code base.

Maybe it depends on the CPAN modules used, but I was just remarking to someone last week how, with over 300 installed CPAN packages, I can only remember finding one backward compatibility issue in 5 years. We do wrap some, but only if we're extending functionality. Still not a bad idea though.

I would agree that the world is not all OO. We have mostly OO code but there are a fair number of procedural modules. I think the point a lot of people miss is that, regardless of the coding paradigm used, the most important thing is packaging -- things should always be relegated to packages, properly factored out, in a directory structure that allows for additions and changes over time. A friend of mine refers to the base directory layout and packaging scheme one sets up as the pegboard of the programmer's workshop: a place to hang everything and everything in its place.

It seems that a lot of programmers are not very comfortable working by building packages, unit testing those, and building small scripts that use them. For whatever reason, I always find that the average programmer gravitates towards long scripts. I think a lot of people just can't see the pieces as they present themselves. For that reason I'd try to hire at least one senior person who has done a lot of package building. Typically people who have had to build APIs for others to use are a good choice, whether it be in the open source world (like CPAN) or for a company that sells software. The experience of having to define interfaces is not as common as it seems sometimes.

I'm not knocking your Perl ability. It's just that anyone who has used Perl for a while is likely to have experimented with different ways of doing things. Some of those ways will seem great for a while, and will only gradually reveal their weaknesses. Without having gone through a good deal of that process yourself, you just aren't equipped to differentiate between "solid and robust" and "fatally flawed".

My suggestion would be to find a very good, experienced Perl programmer to quickly eyeball the code a few times during the development, and make his -- or her -- feedback a gating factor in the contract. (Perhaps indirectly -- you should have no difficulty in understanding his objections once he explains them to you.)

Perl is a bit unusual in this respect. There are many ways of doing things, which makes it hard sometimes to differentiate genius from impending doom. At the same time, it expands the space of possibilities in such a way that it is actually much easier for an experienced practitioner to quickly tell how good code is. I find that doing Perl code reviews is much, much faster and easier than doing C, C++, or Java code reviews -- you can sniff out the overall quality and style in about 30 seconds, and robust Perl is so idiomatic that finding the troublesome bits is far easier. And although there are many valid ways of doing things, one code base should use one consistently. Fortunately, the different techniques tend to look different enough that it is easy to judge consistency.

Is there a fully automated test suite? The lack of one doesn't necessarily mean the system is crap, but it's not a good sign.

Try and implement a new feature (or ask another programmer to do so). How hard was it? Can it be done? Can it be tested?

Will the software engineering company stand behind their code? Will they give you a warranty (unlikely) it (or offer a reasonably priced support contract).

Instead ask for code samples from the small company up front. And as YOUR code arrives, look at it. If you have time, perhaps you can write some of the source code comments or test cases, based only on the code you receive.

A second set of eyes is often the best path to clarity. The original author of the code is often too intimate with it to understand what the subsequent maintainers might find confusing.

I've maintained a lot of code. Far too often what the original author documented was already obvious to me based on the code. Usually I needed higher level hints -- like what the overall purpose of a code section is for. Or a description of the reason for a methodology or algorithm.

Surely documentation that is "not worth the bits it is printed on" doesn't count as good documentation!

Far too often what the original author documented was already obvious to me based on the code.

Sounds like the difference between comments and documentation. Documentation tells you *what* the software does - the API, perhaps a high-level description of the algorithm. Documentation exists to help people who want to use your code (including other programmers) as well as maintainers. Comments are about *how* it does the job, and help explain much lower level stuff, eg "treat \ character as special", "these constants copied from MySQL headers", and are only for maintainers. Even so, they shouldn't say things like "add one to $i".