Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??

My Test::BrewBuild test deployment system is now reasonably stable, and has the ability to dispatch test runs to remote test servers. This means with a single command, you can run your unit tests across any number of perl instances on any Operating Systems automatically, and get the results delivered back to you.

To do this, we use the bbdispatch to send brewbuild commands to previously configured remote testers. To start a bbtester, log on to the system and run bbtester start. That'll start the tester and put it in the background.

Now's probably a good time to state that Git is required to be installed on all systems used for network testing, and one should peruse the basic system setup doc.

In these examples, I have three testers set up. tester1 is the DNS name of a Ubuntu Linux system running perlbrew, tester2 is a Windows 2008 Server running berrybrew, and the localhost is a FreeBSD server, again running perlbrew.

There are three flags for bbdispatch:

  • -c: A quoted string containing the brewbuild commands to run. Eg: "brewbuild -i 5.20.3 -R". If omitted, we will default to simply "brewbuild"
  • -r: A string containing a repository you want to test. Eg: https://github.com/stevieb9/mock-sub. If you are currently in a repository directory, we'll automatically fetch and use it and you can omit this flag
  • -t: These are your testers. This flag can be specified multiple times.

Note that you can alternatively use a config file to store the dispatcher information.

Here's the most basic example. We're already in a repository working directory so we can omit -r, we're only working on a tester on localhost, and we'll just use the default brewbuild test run:

# start a tester on the localhost me@host:~/repos/mock-sub$ bbtester start Started the Test::BrewBuild test server at PID 20584 on IP address 0.0 +.0.0 and TCP port 7800... # dispatch a test run of my [metamod://Mock::Sub] # module (from within the repo dir) me@host:~/repos/mock-sub$ bbdispatch -t localhost localhost - x86_64-linux 5.18.4 :: PASS 5.20.3 :: PASS 5.22.1 :: PASS

Things get more useful when you have multiple testers across multiple different Operating Systems.

This example does a basic run using the same repo as above, but this time I'm explicitly setting it. I'm also dispatching to three tester systems:

$ bbdispatch -t localhost -t tester1 -t tester2 -r https://github.com/ +stevieb9/mock-sub tester1 - MSWin32-x64-multi-thread 5.22.1_64 :: PASS 5.10.1_32 :: PASS 5.20.3_32 :: PASS tester2 - x86_64-linux 5.18.4 :: PASS 5.20.3 :: PASS 5.22.1 :: PASS localhost - amd64-freebsd 5.10.1 :: PASS 5.14.4 :: PASS 5.22.1 :: PASS

Below, we use the -c flag to tell brewbuild that we want to perform test on the current module (I'm back in the repo dir again so I omit -r), and then have brewbuild run unit tests of all the module's reverse dependencies (-R) to ensure our proposed updated module doesn't break down-river (ie. modules that require/use your module) modules.

$ bbdispatch -t localhost -t tester1 -t tester2 -c "brewbuild -R" tester1 - MSWin32-x64-multi-thread reverse dependencies: File::Edit::Portable, Devel::Examine::Subs, Deve +l::Trace::Subs File::Edit::Portable 5.10.1_32 :: PASS 5.20.3_32 :: PASS 5.22.1_64 :: PASS Devel::Examine::Subs 5.20.3_32 :: PASS 5.22.1_64 :: PASS 5.10.1_32 :: FAIL Devel::Trace::Subs 5.20.3_32 :: PASS 5.22.1_64 :: PASS 5.10.1_32 :: FAIL localhost - amd64-freebsd reverse dependencies: File::Edit::Portable, Devel::Examine::Subs, Deve +l::Trace::Subs File::Edit::Portable 5.10.1 :: PASS 5.22.1 :: PASS 5.14.4 :: FAIL Devel::Examine::Subs 5.10.1 :: PASS 5.14.4 :: PASS 5.22.1 :: PASS Devel::Trace::Subs 5.10.1 :: PASS 5.14.4 :: PASS 5.22.1 :: FAIL tester2 - x86_64-linux reverse dependencies: File::Edit::Portable, Devel::Examine::Subs, Deve +l::Trace::Subs File::Edit::Portable 5.18.4 :: PASS 5.20.3 :: PASS 5.22.1 :: PASS Devel::Examine::Subs 5.18.4 :: PASS 5.20.3 :: PASS 5.22.1 :: PASS Devel::Trace::Subs 5.18.4 :: PASS 5.20.3 :: PASS 5.22.1 :: FAIL

Notice that some results are FAIL. In this case, we create a bblog directory in the directory you're working in, and generate log files for each individual fail. This allows you to see what broke and where, without having to go to each individual system. You can then update/fix code, then run another dispatch. Here's an example of the files that were generated by the above run:

$ ls bblog tester1_Devel-Examine-Subs-5.10.1_32-FAIL.bblog tester1_Devel-Trace-Subs-5.10.1_32-FAIL.bblog tester2_Devel-Trace-Subs-5.22.1-FAIL.bblog localhost_Devel-Trace-Subs-5.22.1-FAIL.bblog localhost_File-Edit-Portable-5.14.4-FAIL.bblog

The log files contain all errors that the tester would have produced to STDOUT and STDERR, with the cpanm build logs appended.

On a normal dispatch run (without running revdep), the log file would have appeared as tester1_5.10.1_32-FAIL.bblog. Running brewbuild in standalone mode (no dispatching): 5.10.1_32-FAIL.bblog. You can optionally save even the PASS logs if you choose by using the -S, --save flag to brewbuild.

FURTHER READING:

Notes:

  • we don't actually shell out and run the brewbuild commands directly with system() or backticks or the like. We dissect the command string, turn the arguments into their respective parameters, and use the API to do the actual testing

In reply to Automate and dispatch unit test runs across perl/berrybrew on remote Windows and Unix systems by stevieb

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others sharing their wisdom with the Monastery: (4)
As of 2024-03-28 17:49 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found